2024-03-04T09:08:59.865 INFO:root:teuthology version: 0.0.1.dev260+gb1dac55 2024-03-04T09:08:59.865 DEBUG:teuthology.run:Teuthology command: teuthology --description fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/legacy wsync/no} objectstore-ec/bluestore-ec-root omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{balancer/random export-check n/3 replication/default} standby-replay tasks/{0-subvolume/{with-quota} 1-check-counter 2-scrub/no 3-snaps/yes 4-flush/no 5-workunit/suites/fsx}} --name leonidus-2024-03-04_07:05:03-fs-wip-lusov-qdb-distro-default-smithi --verbose --owner scheduled_leonidus@teuthology --archive /home/teuthworker/archive/leonidus-2024-03-04_07:05:03-fs-wip-lusov-qdb-distro-default-smithi/7580854 -- /home/teuthworker/archive/leonidus-2024-03-04_07:05:03-fs-wip-lusov-qdb-distro-default-smithi/7580854/orig.config.yaml 2024-03-04T09:08:59.931 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-04T09:09:00.038 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/leonidus-2024-03-04_07:05:03-fs-wip-lusov-qdb-distro-default-smithi/7580854 branch: wip-lusov-qdb description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/legacy wsync/no} objectstore-ec/bluestore-ec-root omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{balancer/random export-check n/3 replication/default} standby-replay tasks/{0-subvolume/{with-quota} 1-check-counter 2-scrub/no 3-snaps/yes 4-flush/no 5-workunit/suites/fsx}} email: null first_in_suite: false job_id: '7580854' kernel: client: sha1: distro kdb: true sha1: distro ktype: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: leonidus-2024-03-04_07:05:03-fs-wip-lusov-qdb-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-lusov-qdb ceph: cephfs: ec_profile: - m=2 - k=2 - crush-failure-domain=osd max_mds: 3 session_timeout: 300 standby_replay: true conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true mds_export_ephemeral_random_max: 0.1 osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: cephsqlite lock renewal timeout: 900000 debug cephsqlite: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon op complaint time: 120 osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 osd shutdown pgref assert: true osd_max_omap_entries_per_request: 10 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - POOL_APP_NOT_ENABLED - overall HEALTH_ - Replacing daemon - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding sha1: 1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd subvols: create: 2 subvol_options: --size 25000000000 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs ceph-fuse: client.0: mount_subvol_num: 0 check-counter: counters: mds: - mds.exported - mds.imported - mds.root_rsnaps - mds_server.req_mksnap_latency.avgcount - mds_server.req_rmsnap_latency.avgcount - mds.dir_split install: ceph: flavor: default sha1: 1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd kclient: client.0: mount_subvol_num: 1 mntopts: - ms_mode=legacy - nowsync syntax: v1 selinux: whitelist: - scontext=system_u:system_r:logrotate_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-lusov-qdb sha1: 1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd owner: scheduled_leonidus@teuthology priority: 80 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.x - mds.a - mds.d - mds.g - mds.j - osd.0 - osd.3 - osd.6 - osd.9 - client.0 - - mon.b - mgr.y - mds.b - mds.e - mds.h - mds.k - osd.1 - osd.4 - osd.7 - osd.10 - - mon.c - mgr.z - mds.c - mds.f - mds.i - mds.l - osd.2 - osd.5 - osd.8 - osd.11 seed: 5072 sha1: 1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd sleep_before_teardown: 0 subset: 246/10000 suite: fs suite_branch: wip-lusov-qdb suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: 1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi052.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBOCRgDLI5Y6Tg75a9Z4mh6kmPhCY2toW14tkw0N+6pdt1jMiCxQ3wYuBySlkL7a6tKxax+E5Lyxs8WB4Gl9EURo= smithi106.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBG3MEDf+K+51GsJZ9QOubP1vezcv/tyHsNeu6pkX185J0QgzlBEzHel7I0N7mXLjh9J+od5kRLVWYqp74ymUnI4= smithi191.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJ43/Q3nxuJJMLGm2oXtfk4ywk4Ph1vbJR1Us+A/c6v8/ELZZTikTmw5nOq9D80sVlEGHT4evblo8dkyMQUT++c= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - cephadm: roleless: false - cephadm.shell: mon.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: mon.a: - ceph fs dump - ceph osd dump - fs.ready: timeout: 300 - kclient: null - exec: mon.a: - ceph fs set cephfs balance_automate false - ceph fs subvolumegroup pin cephfs qa random 0.10 - check-counter: {} - exec: mon.a: - ceph mgr module enable snap_schedule - ceph config set mgr mgr/snap_schedule/allow_m_granularity true - ceph config set mgr mgr/snap_schedule/dump_on_update true - ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m - ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - date +%s > START_TIME - full_sequential_finally: - exec: mon.a: - date +%s > END_TIME - START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - workunit: clients: all: - suites/fsx.sh teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: b1dac5519c57c269ea98a22fb7729016a1d0e4d2 timestamp: 2024-03-04_07:05:03 tube: smithi user: leonidus verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-03-04T09:09:00.038 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd/qa; will attempt to use it 2024-03-04T09:09:00.039 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd/qa/tasks 2024-03-04T09:09:00.039 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-03-04T09:09:00.040 INFO:teuthology.task.internal:Checking packages... 2024-03-04T09:09:00.066 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd' 2024-03-04T09:09:00.066 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-03-04T09:09:00.066 INFO:teuthology.packaging:ref: None 2024-03-04T09:09:00.067 INFO:teuthology.packaging:tag: None 2024-03-04T09:09:00.067 INFO:teuthology.packaging:branch: wip-lusov-qdb 2024-03-04T09:09:00.067 INFO:teuthology.packaging:sha1: 1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd 2024-03-04T09:09:00.067 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-lusov-qdb 2024-03-04T09:09:00.220 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-1739.g1bc250c7 2024-03-04T09:09:00.222 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-03-04T09:09:00.228 INFO:teuthology.task.internal:no buildpackages task found 2024-03-04T09:09:00.228 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-03-04T09:09:00.235 INFO:teuthology.task.internal:Saving configuration 2024-03-04T09:09:00.256 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-03-04T09:09:00.261 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-03-04T09:09:00.291 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi052.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/leonidus-2024-03-04_07:05:03-fs-wip-lusov-qdb-distro-default-smithi/7580854', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-04 09:04:04.459582', 'locked_by': 'scheduled_leonidus@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBOCRgDLI5Y6Tg75a9Z4mh6kmPhCY2toW14tkw0N+6pdt1jMiCxQ3wYuBySlkL7a6tKxax+E5Lyxs8WB4Gl9EURo='} 2024-03-04T09:09:00.319 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi106.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/leonidus-2024-03-04_07:05:03-fs-wip-lusov-qdb-distro-default-smithi/7580854', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-04 09:04:04.461234', 'locked_by': 'scheduled_leonidus@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBG3MEDf+K+51GsJZ9QOubP1vezcv/tyHsNeu6pkX185J0QgzlBEzHel7I0N7mXLjh9J+od5kRLVWYqp74ymUnI4='} 2024-03-04T09:09:00.347 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi191.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/leonidus-2024-03-04_07:05:03-fs-wip-lusov-qdb-distro-default-smithi/7580854', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-04 09:04:04.458321', 'locked_by': 'scheduled_leonidus@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJ43/Q3nxuJJMLGm2oXtfk4ywk4Ph1vbJR1Us+A/c6v8/ELZZTikTmw5nOq9D80sVlEGHT4evblo8dkyMQUT++c='} 2024-03-04T09:09:00.347 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-03-04T09:09:00.353 INFO:teuthology.task.internal:roles: ubuntu@smithi052.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2024-03-04T09:09:00.354 INFO:teuthology.task.internal:roles: ubuntu@smithi106.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2024-03-04T09:09:00.354 INFO:teuthology.task.internal:roles: ubuntu@smithi191.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2024-03-04T09:09:00.354 INFO:teuthology.run_tasks:Running task console_log... 2024-03-04T09:09:00.515 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7febb84dc0d0>, signals=[15]) 2024-03-04T09:09:00.516 INFO:teuthology.run_tasks:Running task internal.connect... 2024-03-04T09:09:00.522 INFO:teuthology.task.internal:Opening connections... 2024-03-04T09:09:00.522 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi052.front.sepia.ceph.com 2024-03-04T09:09:00.524 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi052.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-04T09:09:00.644 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi106.front.sepia.ceph.com 2024-03-04T09:09:00.646 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi106.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-04T09:09:00.747 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi191.front.sepia.ceph.com 2024-03-04T09:09:00.748 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi191.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-04T09:09:00.846 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-03-04T09:09:00.852 DEBUG:teuthology.orchestra.run.smithi052:> uname -m 2024-03-04T09:09:00.871 INFO:teuthology.orchestra.run.smithi052.stdout:x86_64 2024-03-04T09:09:00.871 DEBUG:teuthology.orchestra.run.smithi052:> cat /etc/os-release 2024-03-04T09:09:00.930 INFO:teuthology.orchestra.run.smithi052.stdout:NAME="CentOS Stream" 2024-03-04T09:09:00.930 INFO:teuthology.orchestra.run.smithi052.stdout:VERSION="9" 2024-03-04T09:09:00.930 INFO:teuthology.orchestra.run.smithi052.stdout:ID="centos" 2024-03-04T09:09:00.931 INFO:teuthology.orchestra.run.smithi052.stdout:ID_LIKE="rhel fedora" 2024-03-04T09:09:00.931 INFO:teuthology.orchestra.run.smithi052.stdout:VERSION_ID="9" 2024-03-04T09:09:00.931 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM_ID="platform:el9" 2024-03-04T09:09:00.931 INFO:teuthology.orchestra.run.smithi052.stdout:PRETTY_NAME="CentOS Stream 9" 2024-03-04T09:09:00.931 INFO:teuthology.orchestra.run.smithi052.stdout:ANSI_COLOR="0;31" 2024-03-04T09:09:00.931 INFO:teuthology.orchestra.run.smithi052.stdout:LOGO="fedora-logo-icon" 2024-03-04T09:09:00.931 INFO:teuthology.orchestra.run.smithi052.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-03-04T09:09:00.931 INFO:teuthology.orchestra.run.smithi052.stdout:HOME_URL="https://centos.org/" 2024-03-04T09:09:00.932 INFO:teuthology.orchestra.run.smithi052.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-03-04T09:09:00.932 INFO:teuthology.orchestra.run.smithi052.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-03-04T09:09:00.932 INFO:teuthology.orchestra.run.smithi052.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-03-04T09:09:00.933 INFO:teuthology.lock.ops:Updating smithi052.front.sepia.ceph.com on lock server 2024-03-04T09:09:01.228 DEBUG:teuthology.orchestra.run.smithi106:> uname -m 2024-03-04T09:09:01.246 INFO:teuthology.orchestra.run.smithi106.stdout:x86_64 2024-03-04T09:09:01.246 DEBUG:teuthology.orchestra.run.smithi106:> cat /etc/os-release 2024-03-04T09:09:01.303 INFO:teuthology.orchestra.run.smithi106.stdout:NAME="CentOS Stream" 2024-03-04T09:09:01.304 INFO:teuthology.orchestra.run.smithi106.stdout:VERSION="9" 2024-03-04T09:09:01.304 INFO:teuthology.orchestra.run.smithi106.stdout:ID="centos" 2024-03-04T09:09:01.304 INFO:teuthology.orchestra.run.smithi106.stdout:ID_LIKE="rhel fedora" 2024-03-04T09:09:01.304 INFO:teuthology.orchestra.run.smithi106.stdout:VERSION_ID="9" 2024-03-04T09:09:01.304 INFO:teuthology.orchestra.run.smithi106.stdout:PLATFORM_ID="platform:el9" 2024-03-04T09:09:01.304 INFO:teuthology.orchestra.run.smithi106.stdout:PRETTY_NAME="CentOS Stream 9" 2024-03-04T09:09:01.304 INFO:teuthology.orchestra.run.smithi106.stdout:ANSI_COLOR="0;31" 2024-03-04T09:09:01.304 INFO:teuthology.orchestra.run.smithi106.stdout:LOGO="fedora-logo-icon" 2024-03-04T09:09:01.304 INFO:teuthology.orchestra.run.smithi106.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-03-04T09:09:01.304 INFO:teuthology.orchestra.run.smithi106.stdout:HOME_URL="https://centos.org/" 2024-03-04T09:09:01.304 INFO:teuthology.orchestra.run.smithi106.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-03-04T09:09:01.304 INFO:teuthology.orchestra.run.smithi106.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-03-04T09:09:01.304 INFO:teuthology.orchestra.run.smithi106.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-03-04T09:09:01.305 INFO:teuthology.lock.ops:Updating smithi106.front.sepia.ceph.com on lock server 2024-03-04T09:09:01.344 DEBUG:teuthology.orchestra.run.smithi191:> uname -m 2024-03-04T09:09:01.362 INFO:teuthology.orchestra.run.smithi191.stdout:x86_64 2024-03-04T09:09:01.362 DEBUG:teuthology.orchestra.run.smithi191:> cat /etc/os-release 2024-03-04T09:09:01.419 INFO:teuthology.orchestra.run.smithi191.stdout:NAME="CentOS Stream" 2024-03-04T09:09:01.419 INFO:teuthology.orchestra.run.smithi191.stdout:VERSION="9" 2024-03-04T09:09:01.419 INFO:teuthology.orchestra.run.smithi191.stdout:ID="centos" 2024-03-04T09:09:01.419 INFO:teuthology.orchestra.run.smithi191.stdout:ID_LIKE="rhel fedora" 2024-03-04T09:09:01.419 INFO:teuthology.orchestra.run.smithi191.stdout:VERSION_ID="9" 2024-03-04T09:09:01.419 INFO:teuthology.orchestra.run.smithi191.stdout:PLATFORM_ID="platform:el9" 2024-03-04T09:09:01.419 INFO:teuthology.orchestra.run.smithi191.stdout:PRETTY_NAME="CentOS Stream 9" 2024-03-04T09:09:01.420 INFO:teuthology.orchestra.run.smithi191.stdout:ANSI_COLOR="0;31" 2024-03-04T09:09:01.420 INFO:teuthology.orchestra.run.smithi191.stdout:LOGO="fedora-logo-icon" 2024-03-04T09:09:01.420 INFO:teuthology.orchestra.run.smithi191.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-03-04T09:09:01.420 INFO:teuthology.orchestra.run.smithi191.stdout:HOME_URL="https://centos.org/" 2024-03-04T09:09:01.420 INFO:teuthology.orchestra.run.smithi191.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-03-04T09:09:01.420 INFO:teuthology.orchestra.run.smithi191.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-03-04T09:09:01.420 INFO:teuthology.orchestra.run.smithi191.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-03-04T09:09:01.420 INFO:teuthology.lock.ops:Updating smithi191.front.sepia.ceph.com on lock server 2024-03-04T09:09:01.460 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-03-04T09:09:01.472 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-03-04T09:09:01.706 INFO:teuthology.task.internal:Checking for old test directory... 2024-03-04T09:09:01.707 DEBUG:teuthology.orchestra.run.smithi052:> test '!' -e /home/ubuntu/cephtest 2024-03-04T09:09:01.709 DEBUG:teuthology.orchestra.run.smithi106:> test '!' -e /home/ubuntu/cephtest 2024-03-04T09:09:01.712 DEBUG:teuthology.orchestra.run.smithi191:> test '!' -e /home/ubuntu/cephtest 2024-03-04T09:09:01.728 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-03-04T09:09:01.736 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-03-04T09:09:01.736 DEBUG:teuthology.orchestra.run.smithi052:> test -z $(ls -A /var/lib/ceph) 2024-03-04T09:09:01.766 DEBUG:teuthology.orchestra.run.smithi106:> test -z $(ls -A /var/lib/ceph) 2024-03-04T09:09:01.769 DEBUG:teuthology.orchestra.run.smithi191:> test -z $(ls -A /var/lib/ceph) 2024-03-04T09:09:01.792 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-03-04T09:09:01.903 INFO:teuthology.run_tasks:Running task kernel... 2024-03-04T09:09:02.250 INFO:teuthology.task.kernel:normalize config orig: {'client': {'sha1': 'distro'}, 'kdb': True, 'sha1': 'distro'} 2024-03-04T09:09:02.250 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'sha1': 'distro'}} 2024-03-04T09:09:02.250 INFO:teuthology.task.kernel:config {'client.0': {'sha1': 'distro'}}, timeout 300 2024-03-04T09:09:02.251 DEBUG:teuthology.orchestra.run.smithi052:> test -f /run/.containerenv -o -f /.dockerenv 2024-03-04T09:09:02.268 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-04T09:09:02.268 DEBUG:teuthology.orchestra.run.smithi052:> uname -r 2024-03-04T09:09:02.327 INFO:teuthology.orchestra.run.smithi052.stdout:5.14.0-361.el9.x86_64 2024-03-04T09:09:02.327 INFO:teuthology.task.kernel:Running kernel on smithi052: 5.14.0-361.el9.x86_64 2024-03-04T09:09:02.327 DEBUG:teuthology.orchestra.run.smithi052:> sudo yum install -y kernel 2024-03-04T09:09:04.468 INFO:teuthology.orchestra.run.smithi052.stdout:CentOS-9 - AppStream 16 MB/s | 6.7 MB 00:00 2024-03-04T09:09:05.952 INFO:teuthology.orchestra.run.smithi052.stdout:CentOS-9 - BaseOS 6.2 MB/s | 2.0 MB 00:00 2024-03-04T09:09:07.951 INFO:teuthology.orchestra.run.smithi052.stdout:CentOS Stream 9 - BaseOS 4.9 MB/s | 8.0 MB 00:01 2024-03-04T09:09:26.867 INFO:teuthology.orchestra.run.smithi052.stdout:CentOS Stream 9 - AppStream 1.1 MB/s | 19 MB 00:17 2024-03-04T09:09:31.361 INFO:teuthology.orchestra.run.smithi052.stdout:CentOS Stream 9 - CRB 7.2 MB/s | 6.1 MB 00:00 2024-03-04T09:09:33.668 INFO:teuthology.orchestra.run.smithi052.stdout:CentOS Stream 9 - Extras packages 13 kB/s | 15 kB 00:01 2024-03-04T09:09:34.886 INFO:teuthology.orchestra.run.smithi052.stdout:Extra Packages for Enterprise Linux 19 MB/s | 20 MB 00:01 2024-03-04T09:09:40.012 INFO:teuthology.orchestra.run.smithi052.stdout:lab-extras 29 kB/s | 1.7 kB 00:00 2024-03-04T09:09:42.318 INFO:teuthology.orchestra.run.smithi052.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-03-04T09:09:42.319 INFO:teuthology.orchestra.run.smithi052.stdout:Package kernel-5.14.0-361.el9.x86_64 is already installed. 2024-03-04T09:09:42.417 INFO:teuthology.orchestra.run.smithi052.stdout:Dependencies resolved. 2024-03-04T09:09:42.425 INFO:teuthology.orchestra.run.smithi052.stdout:================================================================================ 2024-03-04T09:09:42.425 INFO:teuthology.orchestra.run.smithi052.stdout: Package Architecture Version Repository Size 2024-03-04T09:09:42.425 INFO:teuthology.orchestra.run.smithi052.stdout:================================================================================ 2024-03-04T09:09:42.425 INFO:teuthology.orchestra.run.smithi052.stdout:Installing: 2024-03-04T09:09:42.425 INFO:teuthology.orchestra.run.smithi052.stdout: kernel x86_64 5.14.0-427.el9 baseos 6.0 M 2024-03-04T09:09:42.425 INFO:teuthology.orchestra.run.smithi052.stdout:Installing dependencies: 2024-03-04T09:09:42.425 INFO:teuthology.orchestra.run.smithi052.stdout: kernel-core x86_64 5.14.0-427.el9 baseos 21 M 2024-03-04T09:09:42.425 INFO:teuthology.orchestra.run.smithi052.stdout: kernel-modules x86_64 5.14.0-427.el9 baseos 39 M 2024-03-04T09:09:42.425 INFO:teuthology.orchestra.run.smithi052.stdout: kernel-modules-core x86_64 5.14.0-427.el9 baseos 33 M 2024-03-04T09:09:42.426 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-03-04T09:09:42.426 INFO:teuthology.orchestra.run.smithi052.stdout:Transaction Summary 2024-03-04T09:09:42.426 INFO:teuthology.orchestra.run.smithi052.stdout:================================================================================ 2024-03-04T09:09:42.426 INFO:teuthology.orchestra.run.smithi052.stdout:Install 4 Packages 2024-03-04T09:09:42.426 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-03-04T09:09:42.426 INFO:teuthology.orchestra.run.smithi052.stdout:Total download size: 100 M 2024-03-04T09:09:42.426 INFO:teuthology.orchestra.run.smithi052.stdout:Installed size: 123 M 2024-03-04T09:09:42.427 INFO:teuthology.orchestra.run.smithi052.stdout:Downloading Packages: 2024-03-04T09:09:42.980 INFO:teuthology.orchestra.run.smithi052.stdout:(1/4): kernel-5.14.0-427.el9.x86_64.rpm 12 MB/s | 6.0 MB 00:00 2024-03-04T09:09:43.347 INFO:teuthology.orchestra.run.smithi052.stdout:(2/4): kernel-core-5.14.0-427.el9.x86_64.rpm 24 MB/s | 21 MB 00:00 2024-03-04T09:09:44.055 INFO:teuthology.orchestra.run.smithi052.stdout:(3/4): kernel-modules-5.14.0-427.el9.x86_64.rpm 25 MB/s | 39 MB 00:01 2024-03-04T09:09:44.405 INFO:teuthology.orchestra.run.smithi052.stdout:(4/4): kernel-modules-core-5.14.0-427.el9.x86_6 23 MB/s | 33 MB 00:01 2024-03-04T09:09:44.406 INFO:teuthology.orchestra.run.smithi052.stdout:-------------------------------------------------------------------------------- 2024-03-04T09:09:44.406 INFO:teuthology.orchestra.run.smithi052.stdout:Total 50 MB/s | 100 MB 00:01 2024-03-04T09:09:45.330 INFO:teuthology.orchestra.run.smithi052.stdout:Running transaction check 2024-03-04T09:09:45.760 INFO:teuthology.orchestra.run.smithi052.stdout:Transaction check succeeded. 2024-03-04T09:09:45.760 INFO:teuthology.orchestra.run.smithi052.stdout:Running transaction test 2024-03-04T09:09:47.140 INFO:teuthology.orchestra.run.smithi052.stdout:Transaction test succeeded. 2024-03-04T09:09:47.141 INFO:teuthology.orchestra.run.smithi052.stdout:Running transaction 2024-03-04T09:09:50.015 INFO:teuthology.orchestra.run.smithi052.stdout: Preparing : 1/1 2024-03-04T09:09:50.687 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : kernel-modules-core-5.14.0-427.el9.x86_64 1/4 2024-03-04T09:09:50.831 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : kernel-core-5.14.0-427.el9.x86_64 2/4 2024-03-04T09:09:53.733 INFO:teuthology.orchestra.run.smithi052.stdout: Running scriptlet: kernel-core-5.14.0-427.el9.x86_64 2/4 2024-03-04T09:09:53.910 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : kernel-modules-5.14.0-427.el9.x86_64 3/4 2024-03-04T09:09:59.010 INFO:teuthology.orchestra.run.smithi052.stdout: Running scriptlet: kernel-modules-5.14.0-427.el9.x86_64 3/4 2024-03-04T09:09:59.130 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : kernel-5.14.0-427.el9.x86_64 4/4 2024-03-04T09:10:04.251 INFO:teuthology.orchestra.run.smithi052.stdout: Running scriptlet: kernel-modules-core-5.14.0-427.el9.x86_64 4/4 2024-03-04T09:10:34.647 INFO:teuthology.orchestra.run.smithi052.stdout: Running scriptlet: kernel-core-5.14.0-427.el9.x86_64 4/4 2024-03-04T09:10:34.647 INFO:teuthology.orchestra.run.smithi052.stdout:dracut: Disabling early microcode, because kernel does not support it. CONFIG_MICROCODE_[AMD|INTEL]!=y 2024-03-04T09:10:34.648 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-03-04T09:10:34.734 INFO:teuthology.orchestra.run.smithi052.stdout: Running scriptlet: kernel-modules-5.14.0-427.el9.x86_64 4/4 2024-03-04T09:10:35.173 INFO:teuthology.orchestra.run.smithi052.stdout: Running scriptlet: kernel-5.14.0-427.el9.x86_64 4/4 2024-03-04T09:10:35.173 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : kernel-5.14.0-427.el9.x86_64 1/4 2024-03-04T09:10:35.173 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : kernel-core-5.14.0-427.el9.x86_64 2/4 2024-03-04T09:10:35.173 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : kernel-modules-5.14.0-427.el9.x86_64 3/4 2024-03-04T09:10:35.699 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : kernel-modules-core-5.14.0-427.el9.x86_64 4/4 2024-03-04T09:10:35.699 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-03-04T09:10:35.699 INFO:teuthology.orchestra.run.smithi052.stdout:Installed: 2024-03-04T09:10:35.699 INFO:teuthology.orchestra.run.smithi052.stdout: kernel-5.14.0-427.el9.x86_64 2024-03-04T09:10:35.699 INFO:teuthology.orchestra.run.smithi052.stdout: kernel-core-5.14.0-427.el9.x86_64 2024-03-04T09:10:35.699 INFO:teuthology.orchestra.run.smithi052.stdout: kernel-modules-5.14.0-427.el9.x86_64 2024-03-04T09:10:35.700 INFO:teuthology.orchestra.run.smithi052.stdout: kernel-modules-core-5.14.0-427.el9.x86_64 2024-03-04T09:10:35.700 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-03-04T09:10:35.700 INFO:teuthology.orchestra.run.smithi052.stdout:Complete! 2024-03-04T09:10:36.204 DEBUG:teuthology.orchestra.run.smithi052:> rpm -q kernel | sort -rV | head -n 1 2024-03-04T09:10:36.355 INFO:teuthology.orchestra.run.smithi052.stdout:kernel-5.14.0-427.el9.x86_64 2024-03-04T09:10:36.355 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-427.el9.x86_64 2024-03-04T09:10:36.355 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-361.el9.x86_64 Expected: 5.14.0-427.el9.x86_64 2024-03-04T09:10:36.355 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-03-04T09:10:36.355 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-03-04T09:10:36.356 INFO:teuthology.task.kernel:Installing distro kernel on client.0... 2024-03-04T09:10:36.356 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi052.front.sepia.ceph.com, path=None, version=distro) 2024-03-04T09:10:36.356 DEBUG:teuthology.orchestra.run.smithi052:> rpm -q kernel | sort -rV | head -n 1 2024-03-04T09:10:36.482 INFO:teuthology.orchestra.run.smithi052.stdout:kernel-5.14.0-427.el9.x86_64 2024-03-04T09:10:36.482 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-427.el9.x86_64 2024-03-04T09:10:36.483 DEBUG:teuthology.orchestra.run.smithi052:> sudo rpm -qi grub2-tools 2024-03-04T09:10:36.529 INFO:teuthology.orchestra.run.smithi052.stdout:Name : grub2-tools 2024-03-04T09:10:36.529 INFO:teuthology.orchestra.run.smithi052.stdout:Epoch : 1 2024-03-04T09:10:36.529 INFO:teuthology.orchestra.run.smithi052.stdout:Version : 2.06 2024-03-04T09:10:36.530 INFO:teuthology.orchestra.run.smithi052.stdout:Release : 68.el9 2024-03-04T09:10:36.530 INFO:teuthology.orchestra.run.smithi052.stdout:Architecture: x86_64 2024-03-04T09:10:36.530 INFO:teuthology.orchestra.run.smithi052.stdout:Install Date: Fri 01 Sep 2023 09:14:30 PM UTC 2024-03-04T09:10:36.530 INFO:teuthology.orchestra.run.smithi052.stdout:Group : Unspecified 2024-03-04T09:10:36.530 INFO:teuthology.orchestra.run.smithi052.stdout:Size : 8264254 2024-03-04T09:10:36.530 INFO:teuthology.orchestra.run.smithi052.stdout:License : GPLv3+ 2024-03-04T09:10:36.530 INFO:teuthology.orchestra.run.smithi052.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-03-04T09:10:36.530 INFO:teuthology.orchestra.run.smithi052.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-03-04T09:10:36.530 INFO:teuthology.orchestra.run.smithi052.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-03-04T09:10:36.531 INFO:teuthology.orchestra.run.smithi052.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-03-04T09:10:36.531 INFO:teuthology.orchestra.run.smithi052.stdout:Packager : builder@centos.org 2024-03-04T09:10:36.531 INFO:teuthology.orchestra.run.smithi052.stdout:Vendor : CentOS 2024-03-04T09:10:36.531 INFO:teuthology.orchestra.run.smithi052.stdout:URL : http://www.gnu.org/software/grub/ 2024-03-04T09:10:36.531 INFO:teuthology.orchestra.run.smithi052.stdout:Summary : Support tools for GRUB. 2024-03-04T09:10:36.531 INFO:teuthology.orchestra.run.smithi052.stdout:Description : 2024-03-04T09:10:36.531 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-03-04T09:10:36.531 INFO:teuthology.orchestra.run.smithi052.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-03-04T09:10:36.531 INFO:teuthology.orchestra.run.smithi052.stdout:customizable bootloader with modular architecture. It supports a rich 2024-03-04T09:10:36.531 INFO:teuthology.orchestra.run.smithi052.stdout:variety of kernel formats, file systems, computer architectures and 2024-03-04T09:10:36.531 INFO:teuthology.orchestra.run.smithi052.stdout:hardware devices. 2024-03-04T09:10:36.531 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-03-04T09:10:36.531 INFO:teuthology.orchestra.run.smithi052.stdout:This subpackage provides tools for support of all platforms. 2024-03-04T09:10:36.533 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-03-04T09:10:36.533 INFO:teuthology.task.kernel:Updating grub on smithi052 to boot 5.14.0-427.el9.x86_64 2024-03-04T09:10:36.533 DEBUG:teuthology.orchestra.run.smithi052:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-03-04T09:10:37.131 INFO:teuthology.orchestra.run.smithi052.stderr:Generating grub configuration file ... 2024-03-04T09:10:38.599 INFO:teuthology.orchestra.run.smithi052.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-03-04T09:10:38.660 INFO:teuthology.orchestra.run.smithi052.stderr:done 2024-03-04T09:10:38.662 DEBUG:teuthology.orchestra.run.smithi052:> mktemp 2024-03-04T09:10:38.677 INFO:teuthology.orchestra.run.smithi052.stdout:/tmp/tmp.paUonFNwtt 2024-03-04T09:10:38.677 DEBUG:teuthology.orchestra.run.smithi052:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.paUonFNwtt 2024-03-04T09:10:38.763 DEBUG:teuthology.orchestra.run.smithi052:> sudo chmod 0666 /tmp/tmp.paUonFNwtt 2024-03-04T09:10:38.997 DEBUG:teuthology.orchestra.remote:smithi052:/tmp/tmp.paUonFNwtt is 6KB 2024-03-04T09:10:39.046 DEBUG:teuthology.orchestra.run.smithi052:> rm -fr /tmp/tmp.paUonFNwtt 2024-03-04T09:10:39.060 DEBUG:teuthology.orchestra.run.smithi052:> sudo /bin/ls /boot/loader/entries || true 2024-03-04T09:10:39.129 INFO:teuthology.orchestra.run.smithi052.stdout:15949e3c88704300882482eddfd7baae-0-rescue.conf 2024-03-04T09:10:39.130 INFO:teuthology.orchestra.run.smithi052.stdout:15949e3c88704300882482eddfd7baae-5.14.0-239.el9.x86_64.conf 2024-03-04T09:10:39.130 INFO:teuthology.orchestra.run.smithi052.stdout:15949e3c88704300882482eddfd7baae-5.14.0-361.el9.x86_64.conf 2024-03-04T09:10:39.130 INFO:teuthology.orchestra.run.smithi052.stdout:15949e3c88704300882482eddfd7baae-5.14.0-427.el9.x86_64.conf 2024-03-04T09:10:39.131 DEBUG:teuthology.orchestra.run.smithi052:> sudo grub2-set-default 15949e3c88704300882482eddfd7baae-5.14.0-427.el9.x86_64 2024-03-04T09:10:39.284 DEBUG:teuthology.orchestra.run.smithi052:> sudo shutdown -r now 2024-03-04T09:11:09.288 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2024-03-04T09:11:09.289 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi052.front.sepia.ceph.com' 2024-03-04T09:11:09.290 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi052.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-04T09:11:27.804 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.52 2024-03-04T09:11:36.805 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi052.front.sepia.ceph.com' 2024-03-04T09:11:36.806 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi052.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-04T09:12:36.817 DEBUG:teuthology.orchestra.remote:timed out 2024-03-04T09:12:48.819 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi052.front.sepia.ceph.com' 2024-03-04T09:12:48.820 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi052.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-04T09:12:49.225 DEBUG:teuthology.orchestra.run.smithi052:> true 2024-03-04T09:12:49.624 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi052.front.sepia.ceph.com' 2024-03-04T09:12:49.624 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "5.14.0-427.el9.x86_64"... 2024-03-04T09:12:49.624 DEBUG:teuthology.orchestra.run.smithi052:> uname -r 2024-03-04T09:12:49.640 INFO:teuthology.orchestra.run.smithi052.stdout:5.14.0-427.el9.x86_64 2024-03-04T09:12:49.640 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-427.el9.x86_64 vs 5.14.0-427.el9.x86_64 2024-03-04T09:12:49.640 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-03-04T09:12:49.640 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-03-04T09:12:50.641 DEBUG:teuthology.parallel:result is None 2024-03-04T09:12:50.641 INFO:teuthology.run_tasks:Running task internal.base... 2024-03-04T09:12:50.649 INFO:teuthology.task.internal:Creating test directory... 2024-03-04T09:12:50.649 DEBUG:teuthology.orchestra.run.smithi052:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-03-04T09:12:50.652 DEBUG:teuthology.orchestra.run.smithi106:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-03-04T09:12:50.654 DEBUG:teuthology.orchestra.run.smithi191:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-03-04T09:12:50.672 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-03-04T09:12:50.679 INFO:teuthology.run_tasks:Running task internal.archive... 2024-03-04T09:12:50.687 INFO:teuthology.task.internal:Creating archive directory... 2024-03-04T09:12:50.687 DEBUG:teuthology.orchestra.run.smithi052:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-03-04T09:12:50.708 DEBUG:teuthology.orchestra.run.smithi106:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-03-04T09:12:50.712 DEBUG:teuthology.orchestra.run.smithi191:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-03-04T09:12:50.748 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-03-04T09:12:50.755 INFO:teuthology.task.internal:Enabling coredump saving... 2024-03-04T09:12:50.755 DEBUG:teuthology.orchestra.run.smithi106:> test -f /run/.containerenv -o -f /.dockerenv 2024-03-04T09:12:50.795 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-04T09:12:50.796 DEBUG:teuthology.orchestra.run.smithi191:> test -f /run/.containerenv -o -f /.dockerenv 2024-03-04T09:12:50.811 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-04T09:12:50.812 DEBUG:teuthology.orchestra.run.smithi052:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-03-04T09:12:50.814 DEBUG:teuthology.orchestra.run.smithi106:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-03-04T09:12:50.841 DEBUG:teuthology.orchestra.run.smithi191:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-03-04T09:12:50.902 INFO:teuthology.orchestra.run.smithi052.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-04T09:12:50.903 INFO:teuthology.orchestra.run.smithi106.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-04T09:12:50.917 INFO:teuthology.orchestra.run.smithi191.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-04T09:12:50.921 INFO:teuthology.orchestra.run.smithi106.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-04T09:12:50.940 INFO:teuthology.orchestra.run.smithi191.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-04T09:12:50.943 INFO:teuthology.orchestra.run.smithi052.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-04T09:12:50.945 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-03-04T09:12:50.953 INFO:teuthology.task.internal:Configuring sudo... 2024-03-04T09:12:50.953 DEBUG:teuthology.orchestra.run.smithi052:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-03-04T09:12:50.955 DEBUG:teuthology.orchestra.run.smithi106:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-03-04T09:12:50.957 DEBUG:teuthology.orchestra.run.smithi191:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-03-04T09:12:50.986 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-03-04T09:12:50.996 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-03-04T09:12:50.996 DEBUG:teuthology.orchestra.run.smithi052:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-03-04T09:12:51.024 DEBUG:teuthology.orchestra.run.smithi106:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-03-04T09:12:51.027 DEBUG:teuthology.orchestra.run.smithi191:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-03-04T09:12:51.044 DEBUG:teuthology.orchestra.run.smithi052:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-04T09:12:51.122 DEBUG:teuthology.orchestra.run.smithi052:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-04T09:12:51.177 DEBUG:teuthology.orchestra.run.smithi052:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-04T09:12:51.269 DEBUG:teuthology.orchestra.run.smithi052:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-04T09:12:51.301 DEBUG:teuthology.orchestra.run.smithi052:> set -ex 2024-03-04T09:12:51.301 DEBUG:teuthology.orchestra.run.smithi052:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-03-04T09:12:51.368 DEBUG:teuthology.orchestra.run.smithi106:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-04T09:12:51.420 DEBUG:teuthology.orchestra.run.smithi106:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-04T09:12:51.465 DEBUG:teuthology.orchestra.run.smithi106:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-04T09:12:51.555 DEBUG:teuthology.orchestra.run.smithi106:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-04T09:12:51.591 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2024-03-04T09:12:51.591 DEBUG:teuthology.orchestra.run.smithi106:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-03-04T09:12:51.659 DEBUG:teuthology.orchestra.run.smithi191:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-04T09:12:51.711 DEBUG:teuthology.orchestra.run.smithi191:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-04T09:12:51.760 DEBUG:teuthology.orchestra.run.smithi191:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-04T09:12:51.852 DEBUG:teuthology.orchestra.run.smithi191:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-04T09:12:51.890 DEBUG:teuthology.orchestra.run.smithi191:> set -ex 2024-03-04T09:12:51.890 DEBUG:teuthology.orchestra.run.smithi191:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-03-04T09:12:51.959 DEBUG:teuthology.orchestra.run.smithi052:> sudo service rsyslog restart 2024-03-04T09:12:51.961 DEBUG:teuthology.orchestra.run.smithi106:> sudo service rsyslog restart 2024-03-04T09:12:51.964 DEBUG:teuthology.orchestra.run.smithi191:> sudo service rsyslog restart 2024-03-04T09:12:51.998 INFO:teuthology.orchestra.run.smithi106.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-04T09:12:52.010 INFO:teuthology.orchestra.run.smithi052.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-04T09:12:52.048 INFO:teuthology.orchestra.run.smithi191.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-04T09:12:52.410 INFO:teuthology.run_tasks:Running task internal.timer... 2024-03-04T09:12:52.418 INFO:teuthology.task.internal:Starting timer... 2024-03-04T09:12:52.418 INFO:teuthology.run_tasks:Running task pcp... 2024-03-04T09:12:52.428 INFO:teuthology.run_tasks:Running task selinux... 2024-03-04T09:12:52.437 DEBUG:teuthology.task:Applying overrides for task selinux: {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0']} 2024-03-04T09:12:52.437 DEBUG:teuthology.orchestra.run.smithi052:> sudo service auditd rotate 2024-03-04T09:12:52.495 INFO:teuthology.orchestra.run.smithi052.stdout:Rotating logs: 2024-03-04T09:12:52.497 DEBUG:teuthology.orchestra.run.smithi106:> sudo service auditd rotate 2024-03-04T09:12:52.554 INFO:teuthology.orchestra.run.smithi106.stdout:Rotating logs: 2024-03-04T09:12:52.556 DEBUG:teuthology.orchestra.run.smithi191:> sudo service auditd rotate 2024-03-04T09:12:52.639 INFO:teuthology.orchestra.run.smithi191.stdout:Rotating logs: 2024-03-04T09:12:52.641 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-03-04T09:12:52.641 DEBUG:teuthology.orchestra.run.smithi052:> /usr/sbin/getenforce 2024-03-04T09:12:52.677 INFO:teuthology.orchestra.run.smithi052.stdout:Permissive 2024-03-04T09:12:52.678 DEBUG:teuthology.orchestra.run.smithi106:> /usr/sbin/getenforce 2024-03-04T09:12:52.703 INFO:teuthology.orchestra.run.smithi106.stdout:Permissive 2024-03-04T09:12:52.703 DEBUG:teuthology.orchestra.run.smithi191:> /usr/sbin/getenforce 2024-03-04T09:12:52.723 INFO:teuthology.orchestra.run.smithi191.stdout:Permissive 2024-03-04T09:12:52.723 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi052.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi106.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi191.front.sepia.ceph.com': 'permissive'} 2024-03-04T09:12:52.723 DEBUG:teuthology.orchestra.run.smithi052:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-03-04T09:12:52.752 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-04T09:12:52.752 DEBUG:teuthology.orchestra.run.smithi106:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-03-04T09:12:52.781 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-04T09:12:52.781 DEBUG:teuthology.orchestra.run.smithi191:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-03-04T09:12:52.808 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-04T09:12:52.809 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-03-04T09:12:52.809 DEBUG:teuthology.orchestra.run.smithi052:> sudo /usr/sbin/setenforce permissive 2024-03-04T09:12:52.838 DEBUG:teuthology.orchestra.run.smithi106:> sudo /usr/sbin/setenforce permissive 2024-03-04T09:12:52.872 DEBUG:teuthology.orchestra.run.smithi191:> sudo /usr/sbin/setenforce permissive 2024-03-04T09:12:52.923 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-03-04T09:12:52.933 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2024-03-04T09:12:52.939 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2024-03-04T09:12:53.111 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-03-04T09:12:53.131 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-03-04T09:12:53.132 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi052.front.sepia.ceph.com,smithi106.front.sepia.ceph.com,smithi191.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-03-04T09:17:34.682 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi052.front.sepia.ceph.com'), Remote(name='ubuntu@smithi106.front.sepia.ceph.com'), Remote(name='ubuntu@smithi191.front.sepia.ceph.com')] 2024-03-04T09:17:34.684 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi052.front.sepia.ceph.com' 2024-03-04T09:17:34.685 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi052.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-04T09:17:34.787 DEBUG:teuthology.orchestra.run.smithi052:> true 2024-03-04T09:17:34.861 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi052.front.sepia.ceph.com' 2024-03-04T09:17:34.861 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi106.front.sepia.ceph.com' 2024-03-04T09:17:34.862 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi106.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-04T09:17:34.959 DEBUG:teuthology.orchestra.run.smithi106:> true 2024-03-04T09:17:35.028 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi106.front.sepia.ceph.com' 2024-03-04T09:17:35.028 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi191.front.sepia.ceph.com' 2024-03-04T09:17:35.029 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi191.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-04T09:17:35.126 DEBUG:teuthology.orchestra.run.smithi191:> true 2024-03-04T09:17:35.195 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi191.front.sepia.ceph.com' 2024-03-04T09:17:35.196 INFO:teuthology.run_tasks:Running task clock... 2024-03-04T09:17:35.208 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-03-04T09:17:35.208 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-03-04T09:17:35.208 DEBUG:teuthology.orchestra.run.smithi052:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-04T09:17:35.211 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-03-04T09:17:35.211 DEBUG:teuthology.orchestra.run.smithi106:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-04T09:17:35.214 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-03-04T09:17:35.214 DEBUG:teuthology.orchestra.run.smithi191:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-04T09:17:35.243 INFO:teuthology.orchestra.run.smithi052.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-03-04T09:17:35.245 INFO:teuthology.orchestra.run.smithi106.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-03-04T09:17:35.261 INFO:teuthology.orchestra.run.smithi052.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-03-04T09:17:35.263 INFO:teuthology.orchestra.run.smithi106.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-03-04T09:17:35.266 INFO:teuthology.orchestra.run.smithi191.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-03-04T09:17:35.283 INFO:teuthology.orchestra.run.smithi191.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-03-04T09:17:35.292 INFO:teuthology.orchestra.run.smithi052.stderr:sudo: ntpd: command not found 2024-03-04T09:17:35.293 INFO:teuthology.orchestra.run.smithi106.stderr:sudo: ntpd: command not found 2024-03-04T09:17:35.306 INFO:teuthology.orchestra.run.smithi052.stdout:506 Cannot talk to daemon 2024-03-04T09:17:35.307 INFO:teuthology.orchestra.run.smithi106.stdout:506 Cannot talk to daemon 2024-03-04T09:17:35.313 INFO:teuthology.orchestra.run.smithi191.stderr:sudo: ntpd: command not found 2024-03-04T09:17:35.323 INFO:teuthology.orchestra.run.smithi052.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-03-04T09:17:35.325 INFO:teuthology.orchestra.run.smithi106.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-03-04T09:17:35.327 INFO:teuthology.orchestra.run.smithi191.stdout:506 Cannot talk to daemon 2024-03-04T09:17:35.339 INFO:teuthology.orchestra.run.smithi052.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-03-04T09:17:35.341 INFO:teuthology.orchestra.run.smithi106.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-03-04T09:17:35.343 INFO:teuthology.orchestra.run.smithi191.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-03-04T09:17:35.359 INFO:teuthology.orchestra.run.smithi191.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-03-04T09:17:35.384 INFO:teuthology.orchestra.run.smithi106.stderr:bash: line 1: ntpq: command not found 2024-03-04T09:17:35.388 INFO:teuthology.orchestra.run.smithi106.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-04T09:17:35.389 INFO:teuthology.orchestra.run.smithi106.stdout:=============================================================================== 2024-03-04T09:17:35.389 INFO:teuthology.orchestra.run.smithi106.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-04T09:17:35.389 INFO:teuthology.orchestra.run.smithi106.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-04T09:17:35.389 INFO:teuthology.orchestra.run.smithi106.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-04T09:17:35.389 INFO:teuthology.orchestra.run.smithi106.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-04T09:17:35.390 INFO:teuthology.orchestra.run.smithi052.stderr:bash: line 1: ntpq: command not found 2024-03-04T09:17:35.394 INFO:teuthology.orchestra.run.smithi052.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-04T09:17:35.394 INFO:teuthology.orchestra.run.smithi052.stdout:=============================================================================== 2024-03-04T09:17:35.394 INFO:teuthology.orchestra.run.smithi052.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-04T09:17:35.394 INFO:teuthology.orchestra.run.smithi052.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-04T09:17:35.394 INFO:teuthology.orchestra.run.smithi052.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-04T09:17:35.394 INFO:teuthology.orchestra.run.smithi052.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-04T09:17:35.403 INFO:teuthology.orchestra.run.smithi191.stderr:bash: line 1: ntpq: command not found 2024-03-04T09:17:35.408 INFO:teuthology.orchestra.run.smithi191.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-04T09:17:35.408 INFO:teuthology.orchestra.run.smithi191.stdout:=============================================================================== 2024-03-04T09:17:35.408 INFO:teuthology.orchestra.run.smithi191.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-04T09:17:35.408 INFO:teuthology.orchestra.run.smithi191.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-04T09:17:35.408 INFO:teuthology.orchestra.run.smithi191.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-04T09:17:35.409 INFO:teuthology.orchestra.run.smithi191.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-04T09:17:35.409 INFO:teuthology.run_tasks:Running task install... 2024-03-04T09:17:35.417 DEBUG:teuthology.task.install:project ceph 2024-03-04T09:17:35.417 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd'}} 2024-03-04T09:17:35.417 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd'} 2024-03-04T09:17:35.417 INFO:teuthology.task.install:Using flavor: default 2024-03-04T09:17:35.425 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-03-04T09:17:35.426 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-03-04T09:17:35.426 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd', 'tag': None, 'wait_for_package': False} 2024-03-04T09:17:35.426 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd 2024-03-04T09:17:35.429 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd', 'tag': None, 'wait_for_package': False} 2024-03-04T09:17:35.429 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd 2024-03-04T09:17:35.430 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd', 'tag': None, 'wait_for_package': False} 2024-03-04T09:17:35.430 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd 2024-03-04T09:17:35.618 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/wip-lusov-qdb/1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd/centos/9/flavors/default/ 2024-03-04T09:17:35.619 INFO:teuthology.task.install.rpm:Package version is 19.0.0-1739.g1bc250c7 2024-03-04T09:17:35.655 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/wip-lusov-qdb/1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd/centos/9/flavors/default/ 2024-03-04T09:17:35.656 INFO:teuthology.task.install.rpm:Package version is 19.0.0-1739.g1bc250c7 2024-03-04T09:17:35.667 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/wip-lusov-qdb/1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd/centos/9/flavors/default/ 2024-03-04T09:17:35.667 INFO:teuthology.task.install.rpm:Package version is 19.0.0-1739.g1bc250c7 2024-03-04T09:17:35.785 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/wip-lusov-qdb/1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-lusov-qdb/1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-lusov-qdb/1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-03-04T09:17:35.785 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2024-03-04T09:17:35.785 DEBUG:teuthology.orchestra.run.smithi106:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-03-04T09:17:35.808 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/wip-lusov-qdb/1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-lusov-qdb/1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-lusov-qdb/1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-03-04T09:17:35.808 DEBUG:teuthology.orchestra.run.smithi191:> set -ex 2024-03-04T09:17:35.808 DEBUG:teuthology.orchestra.run.smithi191:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-03-04T09:17:35.813 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-03-04T09:17:35.813 DEBUG:teuthology.orchestra.run.smithi106:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-03-04T09:17:35.837 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/wip-lusov-qdb/1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-lusov-qdb/1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-lusov-qdb/1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-03-04T09:17:35.837 DEBUG:teuthology.orchestra.run.smithi052:> set -ex 2024-03-04T09:17:35.837 DEBUG:teuthology.orchestra.run.smithi052:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-03-04T09:17:35.838 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-03-04T09:17:35.838 DEBUG:teuthology.orchestra.run.smithi191:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-03-04T09:17:35.866 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-03-04T09:17:35.866 DEBUG:teuthology.orchestra.run.smithi052:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-03-04T09:17:35.881 DEBUG:teuthology.orchestra.run.smithi106:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-03-04T09:17:35.906 DEBUG:teuthology.orchestra.run.smithi191:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-03-04T09:17:35.933 DEBUG:teuthology.orchestra.run.smithi052:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-03-04T09:17:35.960 DEBUG:teuthology.orchestra.run.smithi106:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-03-04T09:17:35.987 DEBUG:teuthology.orchestra.run.smithi191:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-03-04T09:17:36.013 DEBUG:teuthology.orchestra.run.smithi052:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-03-04T09:17:36.025 INFO:teuthology.orchestra.run.smithi106.stdout:check_obsoletes = 1 2024-03-04T09:17:36.028 DEBUG:teuthology.orchestra.run.smithi106:> sudo yum clean all 2024-03-04T09:17:36.054 INFO:teuthology.orchestra.run.smithi191.stdout:check_obsoletes = 1 2024-03-04T09:17:36.055 DEBUG:teuthology.orchestra.run.smithi191:> sudo yum clean all 2024-03-04T09:17:36.080 INFO:teuthology.orchestra.run.smithi052.stdout:check_obsoletes = 1 2024-03-04T09:17:36.081 DEBUG:teuthology.orchestra.run.smithi052:> sudo yum clean all 2024-03-04T09:17:36.264 INFO:teuthology.orchestra.run.smithi106.stdout:74 files removed 2024-03-04T09:17:36.288 INFO:teuthology.orchestra.run.smithi191.stdout:74 files removed 2024-03-04T09:17:36.290 DEBUG:teuthology.orchestra.run.smithi106:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-03-04T09:17:36.314 DEBUG:teuthology.orchestra.run.smithi191:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-03-04T09:17:36.329 INFO:teuthology.orchestra.run.smithi052.stdout:74 files removed 2024-03-04T09:17:36.356 DEBUG:teuthology.orchestra.run.smithi052:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-03-04T09:17:36.954 INFO:teuthology.orchestra.run.smithi191.stdout:ceph packages for x86_64 238 kB/s | 83 kB 00:00 2024-03-04T09:17:36.982 INFO:teuthology.orchestra.run.smithi106.stdout:ceph packages for x86_64 206 kB/s | 83 kB 00:00 2024-03-04T09:17:36.997 INFO:teuthology.orchestra.run.smithi052.stdout:ceph packages for x86_64 261 kB/s | 83 kB 00:00 2024-03-04T09:17:37.313 INFO:teuthology.orchestra.run.smithi191.stdout:ceph noarch packages 31 kB/s | 10 kB 00:00 2024-03-04T09:17:37.341 INFO:teuthology.orchestra.run.smithi106.stdout:ceph noarch packages 32 kB/s | 10 kB 00:00 2024-03-04T09:17:37.380 INFO:teuthology.orchestra.run.smithi052.stdout:ceph noarch packages 30 kB/s | 10 kB 00:00 2024-03-04T09:17:37.680 INFO:teuthology.orchestra.run.smithi191.stdout:ceph source packages 5.6 kB/s | 1.9 kB 00:00 2024-03-04T09:17:37.683 INFO:teuthology.orchestra.run.smithi106.stdout:ceph source packages 6.1 kB/s | 1.9 kB 00:00 2024-03-04T09:17:37.740 INFO:teuthology.orchestra.run.smithi052.stdout:ceph source packages 5.9 kB/s | 1.9 kB 00:00 2024-03-04T09:17:38.075 INFO:teuthology.orchestra.run.smithi191.stdout:CentOS-9 - AppStream 18 MB/s | 6.7 MB 00:00 2024-03-04T09:17:38.153 INFO:teuthology.orchestra.run.smithi106.stdout:CentOS-9 - AppStream 15 MB/s | 6.7 MB 00:00 2024-03-04T09:17:38.200 INFO:teuthology.orchestra.run.smithi052.stdout:CentOS-9 - AppStream 16 MB/s | 6.7 MB 00:00 2024-03-04T09:17:39.443 INFO:teuthology.orchestra.run.smithi191.stdout:CentOS-9 - BaseOS 7.9 MB/s | 2.0 MB 00:00 2024-03-04T09:17:39.520 INFO:teuthology.orchestra.run.smithi106.stdout:CentOS-9 - BaseOS 7.6 MB/s | 2.0 MB 00:00 2024-03-04T09:17:39.793 INFO:teuthology.orchestra.run.smithi052.stdout:CentOS-9 - BaseOS 4.8 MB/s | 2.0 MB 00:00 2024-03-04T09:17:40.802 INFO:teuthology.orchestra.run.smithi106.stdout:CentOS Stream 9 - BaseOS 8.5 MB/s | 8.0 MB 00:00 2024-03-04T09:17:40.985 INFO:teuthology.orchestra.run.smithi052.stdout:CentOS Stream 9 - BaseOS 9.5 MB/s | 8.0 MB 00:00 2024-03-04T09:17:44.178 INFO:teuthology.orchestra.run.smithi106.stdout:CentOS Stream 9 - AppStream 8.3 MB/s | 19 MB 00:02 2024-03-04T09:17:47.176 INFO:teuthology.orchestra.run.smithi052.stdout:CentOS Stream 9 - AppStream 3.8 MB/s | 19 MB 00:05 2024-03-04T09:17:48.058 INFO:teuthology.orchestra.run.smithi191.stdout:CentOS Stream 9 - BaseOS 993 kB/s | 8.0 MB 00:08 2024-03-04T09:17:49.262 INFO:teuthology.orchestra.run.smithi106.stdout:CentOS Stream 9 - CRB 4.1 MB/s | 6.1 MB 00:01 2024-03-04T09:17:50.846 INFO:teuthology.orchestra.run.smithi106.stdout:CentOS Stream 9 - Extras packages 34 kB/s | 15 kB 00:00 2024-03-04T09:17:52.392 INFO:teuthology.orchestra.run.smithi191.stdout:CentOS Stream 9 - AppStream 5.9 MB/s | 19 MB 00:03 2024-03-04T09:17:56.525 INFO:teuthology.orchestra.run.smithi106.stdout:Extra Packages for Enterprise Linux 3.7 MB/s | 20 MB 00:05 2024-03-04T09:17:56.768 INFO:teuthology.orchestra.run.smithi191.stdout:CentOS Stream 9 - CRB 7.0 MB/s | 6.1 MB 00:00 2024-03-04T09:17:58.552 INFO:teuthology.orchestra.run.smithi191.stdout:CentOS Stream 9 - Extras packages 23 kB/s | 15 kB 00:00 2024-03-04T09:17:59.028 INFO:teuthology.orchestra.run.smithi052.stdout:CentOS Stream 9 - CRB 769 kB/s | 6.1 MB 00:08 2024-03-04T09:18:00.271 INFO:teuthology.orchestra.run.smithi191.stdout:Extra Packages for Enterprise Linux 13 MB/s | 20 MB 00:01 2024-03-04T09:18:00.643 INFO:teuthology.orchestra.run.smithi052.stdout:CentOS Stream 9 - Extras packages 36 kB/s | 15 kB 00:00 2024-03-04T09:18:01.497 INFO:teuthology.orchestra.run.smithi106.stdout:lab-extras 14 kB/s | 1.7 kB 00:00 2024-03-04T09:18:02.606 INFO:teuthology.orchestra.run.smithi052.stdout:Extra Packages for Enterprise Linux 11 MB/s | 20 MB 00:01 2024-03-04T09:18:04.011 INFO:teuthology.orchestra.run.smithi106.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-03-04T09:18:04.012 INFO:teuthology.orchestra.run.smithi106.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-03-04T09:18:04.028 INFO:teuthology.orchestra.run.smithi106.stdout:Package NetworkManager-1:1.44.0-2.el9.x86_64 is already installed. 2024-03-04T09:18:04.029 INFO:teuthology.orchestra.run.smithi106.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-03-04T09:18:04.030 INFO:teuthology.orchestra.run.smithi106.stdout:Package util-linux-2.37.4-15.el9.x86_64 is already installed. 2024-03-04T09:18:04.031 INFO:teuthology.orchestra.run.smithi106.stdout:Package libacl-devel-2.3.1-3.el9.x86_64 is already installed. 2024-03-04T09:18:04.033 INFO:teuthology.orchestra.run.smithi106.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-03-04T09:18:04.036 INFO:teuthology.orchestra.run.smithi106.stdout:Package xfsdump-3.1.12-3.el9.x86_64 is already installed. 2024-03-04T09:18:04.038 INFO:teuthology.orchestra.run.smithi106.stdout:Package xfsprogs-5.19.0-4.el9.x86_64 is already installed. 2024-03-04T09:18:04.119 INFO:teuthology.orchestra.run.smithi106.stdout:Dependencies resolved. 2024-03-04T09:18:04.139 INFO:teuthology.orchestra.run.smithi106.stdout:============================================================================================ 2024-03-04T09:18:04.140 INFO:teuthology.orchestra.run.smithi106.stdout: Package Arch Version Repository Size 2024-03-04T09:18:04.140 INFO:teuthology.orchestra.run.smithi106.stdout:============================================================================================ 2024-03-04T09:18:04.140 INFO:teuthology.orchestra.run.smithi106.stdout:Installing: 2024-03-04T09:18:04.140 INFO:teuthology.orchestra.run.smithi106.stdout: bison x86_64 3.7.4-5.el9 CentOS-AppStream 920 k 2024-03-04T09:18:04.140 INFO:teuthology.orchestra.run.smithi106.stdout: ceph x86_64 2:19.0.0-1739.g1bc250c7.el9 ceph 6.6 k 2024-03-04T09:18:04.140 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-base x86_64 2:19.0.0-1739.g1bc250c7.el9 ceph 5.3 M 2024-03-04T09:18:04.140 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-fuse x86_64 2:19.0.0-1739.g1bc250c7.el9 ceph 888 k 2024-03-04T09:18:04.140 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-1739.g1bc250c7.el9 ceph 145 k 2024-03-04T09:18:04.141 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-mgr x86_64 2:19.0.0-1739.g1bc250c7.el9 ceph 1.5 M 2024-03-04T09:18:04.141 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-mgr-cephadm noarch 2:19.0.0-1739.g1bc250c7.el9 ceph-noarch 138 k 2024-03-04T09:18:04.141 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-mgr-dashboard noarch 2:19.0.0-1739.g1bc250c7.el9 ceph-noarch 1.8 M 2024-03-04T09:18:04.141 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-1739.g1bc250c7.el9 ceph-noarch 7.4 M 2024-03-04T09:18:04.141 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-mgr-rook noarch 2:19.0.0-1739.g1bc250c7.el9 ceph-noarch 49 k 2024-03-04T09:18:04.141 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-radosgw x86_64 2:19.0.0-1739.g1bc250c7.el9 ceph 9.8 M 2024-03-04T09:18:04.141 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-test x86_64 2:19.0.0-1739.g1bc250c7.el9 ceph 47 M 2024-03-04T09:18:04.141 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-volume noarch 2:19.0.0-1739.g1bc250c7.el9 ceph-noarch 264 k 2024-03-04T09:18:04.141 INFO:teuthology.orchestra.run.smithi106.stdout: cephadm noarch 2:19.0.0-1739.g1bc250c7.el9 ceph-noarch 748 k 2024-03-04T09:18:04.141 INFO:teuthology.orchestra.run.smithi106.stdout: cephfs-mirror x86_64 2:19.0.0-1739.g1bc250c7.el9 ceph 220 k 2024-03-04T09:18:04.141 INFO:teuthology.orchestra.run.smithi106.stdout: cephfs-shell x86_64 2:19.0.0-1739.g1bc250c7.el9 ceph 27 k 2024-03-04T09:18:04.141 INFO:teuthology.orchestra.run.smithi106.stdout: cephfs-top noarch 2:19.0.0-1739.g1bc250c7.el9 ceph-noarch 18 k 2024-03-04T09:18:04.142 INFO:teuthology.orchestra.run.smithi106.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 CentOS-AppStream 20 k 2024-03-04T09:18:04.142 INFO:teuthology.orchestra.run.smithi106.stdout: flex x86_64 2.6.4-9.el9 CentOS-AppStream 307 k 2024-03-04T09:18:04.142 INFO:teuthology.orchestra.run.smithi106.stdout: libaio-devel x86_64 0.3.111-13.el9 CentOS-AppStream 10 k 2024-03-04T09:18:04.142 INFO:teuthology.orchestra.run.smithi106.stdout: libcephfs-devel x86_64 2:19.0.0-1739.g1bc250c7.el9 ceph 31 k 2024-03-04T09:18:04.142 INFO:teuthology.orchestra.run.smithi106.stdout: libcephfs2 x86_64 2:19.0.0-1739.g1bc250c7.el9 ceph 747 k 2024-03-04T09:18:04.142 INFO:teuthology.orchestra.run.smithi106.stdout: librados-devel x86_64 2:19.0.0-1739.g1bc250c7.el9 ceph 127 k 2024-03-04T09:18:04.142 INFO:teuthology.orchestra.run.smithi106.stdout: libtool x86_64 2.4.6-45.el9 CentOS-AppStream 578 k 2024-03-04T09:18:04.142 INFO:teuthology.orchestra.run.smithi106.stdout: libuuid-devel x86_64 2.37.4-18.el9 CentOS-AppStream 24 k 2024-03-04T09:18:04.142 INFO:teuthology.orchestra.run.smithi106.stdout: openssl-devel x86_64 1:3.0.7-27.el9 CentOS-AppStream 3.0 M 2024-03-04T09:18:04.142 INFO:teuthology.orchestra.run.smithi106.stdout: postgresql x86_64 13.11-1.el9 CentOS-AppStream 1.5 M 2024-03-04T09:18:04.142 INFO:teuthology.orchestra.run.smithi106.stdout: postgresql-contrib x86_64 13.11-1.el9 CentOS-AppStream 811 k 2024-03-04T09:18:04.142 INFO:teuthology.orchestra.run.smithi106.stdout: postgresql-server x86_64 13.11-1.el9 CentOS-AppStream 5.7 M 2024-03-04T09:18:04.143 INFO:teuthology.orchestra.run.smithi106.stdout: python3-cephfs x86_64 2:19.0.0-1739.g1bc250c7.el9 ceph 162 k 2024-03-04T09:18:04.143 INFO:teuthology.orchestra.run.smithi106.stdout: python3-rados x86_64 2:19.0.0-1739.g1bc250c7.el9 ceph 321 k 2024-03-04T09:18:04.143 INFO:teuthology.orchestra.run.smithi106.stdout: python3-rbd x86_64 2:19.0.0-1739.g1bc250c7.el9 ceph 298 k 2024-03-04T09:18:04.143 INFO:teuthology.orchestra.run.smithi106.stdout: python3-rgw x86_64 2:19.0.0-1739.g1bc250c7.el9 ceph 99 k 2024-03-04T09:18:04.143 INFO:teuthology.orchestra.run.smithi106.stdout: rbd-fuse x86_64 2:19.0.0-1739.g1bc250c7.el9 ceph 85 k 2024-03-04T09:18:04.143 INFO:teuthology.orchestra.run.smithi106.stdout: rbd-mirror x86_64 2:19.0.0-1739.g1bc250c7.el9 ceph 3.1 M 2024-03-04T09:18:04.143 INFO:teuthology.orchestra.run.smithi106.stdout: rbd-nbd x86_64 2:19.0.0-1739.g1bc250c7.el9 ceph 171 k 2024-03-04T09:18:04.143 INFO:teuthology.orchestra.run.smithi106.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 CentOS-AppStream 87 k 2024-03-04T09:18:04.143 INFO:teuthology.orchestra.run.smithi106.stdout:Upgrading: 2024-03-04T09:18:04.143 INFO:teuthology.orchestra.run.smithi106.stdout: NetworkManager x86_64 1:1.46.0-1.el9 CentOS-BaseOS 2.3 M 2024-03-04T09:18:04.143 INFO:teuthology.orchestra.run.smithi106.stdout: NetworkManager-libnm x86_64 1:1.46.0-1.el9 CentOS-BaseOS 1.8 M 2024-03-04T09:18:04.143 INFO:teuthology.orchestra.run.smithi106.stdout: NetworkManager-team x86_64 1:1.46.0-1.el9 CentOS-BaseOS 41 k 2024-03-04T09:18:04.143 INFO:teuthology.orchestra.run.smithi106.stdout: NetworkManager-tui x86_64 1:1.46.0-1.el9 CentOS-BaseOS 247 k 2024-03-04T09:18:04.144 INFO:teuthology.orchestra.run.smithi106.stdout: acl x86_64 2.3.1-4.el9 CentOS-BaseOS 69 k 2024-03-04T09:18:04.144 INFO:teuthology.orchestra.run.smithi106.stdout: elfutils-debuginfod-client x86_64 0.190-2.el9 CentOS-BaseOS 35 k 2024-03-04T09:18:04.144 INFO:teuthology.orchestra.run.smithi106.stdout: elfutils-libelf x86_64 0.190-2.el9 CentOS-BaseOS 191 k 2024-03-04T09:18:04.144 INFO:teuthology.orchestra.run.smithi106.stdout: elfutils-libs x86_64 0.190-2.el9 CentOS-BaseOS 254 k 2024-03-04T09:18:04.144 INFO:teuthology.orchestra.run.smithi106.stdout: iproute x86_64 6.7.0-2.el9 CentOS-BaseOS 802 k 2024-03-04T09:18:04.144 INFO:teuthology.orchestra.run.smithi106.stdout: iproute-tc x86_64 6.7.0-2.el9 CentOS-BaseOS 433 k 2024-03-04T09:18:04.144 INFO:teuthology.orchestra.run.smithi106.stdout: libacl x86_64 2.3.1-4.el9 CentOS-BaseOS 22 k 2024-03-04T09:18:04.144 INFO:teuthology.orchestra.run.smithi106.stdout: libacl-devel x86_64 2.3.1-4.el9 CentOS-AppStream 68 k 2024-03-04T09:18:04.144 INFO:teuthology.orchestra.run.smithi106.stdout: libblkid x86_64 2.37.4-18.el9 CentOS-BaseOS 107 k 2024-03-04T09:18:04.144 INFO:teuthology.orchestra.run.smithi106.stdout: libfdisk x86_64 2.37.4-18.el9 CentOS-BaseOS 154 k 2024-03-04T09:18:04.144 INFO:teuthology.orchestra.run.smithi106.stdout: libmount x86_64 2.37.4-18.el9 CentOS-BaseOS 135 k 2024-03-04T09:18:04.144 INFO:teuthology.orchestra.run.smithi106.stdout: librados2 x86_64 2:19.0.0-1739.g1bc250c7.el9 ceph 3.4 M 2024-03-04T09:18:04.145 INFO:teuthology.orchestra.run.smithi106.stdout: librbd1 x86_64 2:19.0.0-1739.g1bc250c7.el9 ceph 3.1 M 2024-03-04T09:18:04.145 INFO:teuthology.orchestra.run.smithi106.stdout: libsmartcols x86_64 2.37.4-18.el9 CentOS-BaseOS 63 k 2024-03-04T09:18:04.145 INFO:teuthology.orchestra.run.smithi106.stdout: libuuid x86_64 2.37.4-18.el9 CentOS-BaseOS 28 k 2024-03-04T09:18:04.145 INFO:teuthology.orchestra.run.smithi106.stdout: openssl x86_64 1:3.0.7-27.el9 CentOS-BaseOS 1.2 M 2024-03-04T09:18:04.145 INFO:teuthology.orchestra.run.smithi106.stdout: openssl-libs x86_64 1:3.0.7-27.el9 CentOS-BaseOS 2.2 M 2024-03-04T09:18:04.145 INFO:teuthology.orchestra.run.smithi106.stdout: util-linux x86_64 2.37.4-18.el9 CentOS-BaseOS 2.2 M 2024-03-04T09:18:04.145 INFO:teuthology.orchestra.run.smithi106.stdout: util-linux-core x86_64 2.37.4-18.el9 CentOS-BaseOS 438 k 2024-03-04T09:18:04.145 INFO:teuthology.orchestra.run.smithi106.stdout: util-linux-user x86_64 2.37.4-18.el9 CentOS-BaseOS 30 k 2024-03-04T09:18:04.145 INFO:teuthology.orchestra.run.smithi106.stdout: xfsdump x86_64 3.1.12-4.el9 CentOS-BaseOS 321 k 2024-03-04T09:18:04.145 INFO:teuthology.orchestra.run.smithi106.stdout: xfsprogs x86_64 6.3.0-1.el9 CentOS-BaseOS 1.1 M 2024-03-04T09:18:04.145 INFO:teuthology.orchestra.run.smithi106.stdout:Installing dependencies: 2024-03-04T09:18:04.145 INFO:teuthology.orchestra.run.smithi106.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-03-04T09:18:04.145 INFO:teuthology.orchestra.run.smithi106.stdout: automake noarch 1.16.2-8.el9 CentOS-AppStream 662 k 2024-03-04T09:18:04.146 INFO:teuthology.orchestra.run.smithi106.stdout: boost-program-options x86_64 1.75.0-8.el9 CentOS-AppStream 106 k 2024-03-04T09:18:04.146 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-common x86_64 2:19.0.0-1739.g1bc250c7.el9 ceph 20 M 2024-03-04T09:18:04.146 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-grafana-dashboards noarch 2:19.0.0-1739.g1bc250c7.el9 ceph-noarch 31 k 2024-03-04T09:18:04.146 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-mds x86_64 2:19.0.0-1739.g1bc250c7.el9 ceph 2.3 M 2024-03-04T09:18:04.146 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-mgr-modules-core noarch 2:19.0.0-1739.g1bc250c7.el9 ceph-noarch 248 k 2024-03-04T09:18:04.146 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-mon x86_64 2:19.0.0-1739.g1bc250c7.el9 ceph 4.7 M 2024-03-04T09:18:04.146 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-osd x86_64 2:19.0.0-1739.g1bc250c7.el9 ceph 17 M 2024-03-04T09:18:04.146 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-prometheus-alerts noarch 2:19.0.0-1739.g1bc250c7.el9 ceph-noarch 15 k 2024-03-04T09:18:04.146 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-selinux x86_64 2:19.0.0-1739.g1bc250c7.el9 ceph 25 k 2024-03-04T09:18:04.146 INFO:teuthology.orchestra.run.smithi106.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2024-03-04T09:18:04.146 INFO:teuthology.orchestra.run.smithi106.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2024-03-04T09:18:04.147 INFO:teuthology.orchestra.run.smithi106.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2024-03-04T09:18:04.147 INFO:teuthology.orchestra.run.smithi106.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2024-03-04T09:18:04.147 INFO:teuthology.orchestra.run.smithi106.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-03-04T09:18:04.147 INFO:teuthology.orchestra.run.smithi106.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-03-04T09:18:04.147 INFO:teuthology.orchestra.run.smithi106.stdout: jq x86_64 1.6-16.el9 CentOS-BaseOS 186 k 2024-03-04T09:18:04.147 INFO:teuthology.orchestra.run.smithi106.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-03-04T09:18:04.147 INFO:teuthology.orchestra.run.smithi106.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-03-04T09:18:04.147 INFO:teuthology.orchestra.run.smithi106.stdout: libcephsqlite x86_64 2:19.0.0-1739.g1bc250c7.el9 ceph 163 k 2024-03-04T09:18:04.147 INFO:teuthology.orchestra.run.smithi106.stdout: libgfortran x86_64 11.4.1-3.el9 CentOS-BaseOS 802 k 2024-03-04T09:18:04.147 INFO:teuthology.orchestra.run.smithi106.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-03-04T09:18:04.147 INFO:teuthology.orchestra.run.smithi106.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2024-03-04T09:18:04.147 INFO:teuthology.orchestra.run.smithi106.stdout: libquadmath x86_64 11.4.1-3.el9 CentOS-BaseOS 194 k 2024-03-04T09:18:04.147 INFO:teuthology.orchestra.run.smithi106.stdout: librabbitmq x86_64 0.11.0-7.el9 CentOS-AppStream 44 k 2024-03-04T09:18:04.148 INFO:teuthology.orchestra.run.smithi106.stdout: libradosstriper1 x86_64 2:19.0.0-1739.g1bc250c7.el9 ceph 501 k 2024-03-04T09:18:04.148 INFO:teuthology.orchestra.run.smithi106.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2024-03-04T09:18:04.148 INFO:teuthology.orchestra.run.smithi106.stdout: librgw2 x86_64 2:19.0.0-1739.g1bc250c7.el9 ceph 5.1 M 2024-03-04T09:18:04.148 INFO:teuthology.orchestra.run.smithi106.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-03-04T09:18:04.148 INFO:teuthology.orchestra.run.smithi106.stdout: libzstd-devel x86_64 1.5.1-2.el9 CentOS-AppStream 46 k 2024-03-04T09:18:04.149 INFO:teuthology.orchestra.run.smithi106.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2024-03-04T09:18:04.149 INFO:teuthology.orchestra.run.smithi106.stdout: lua x86_64 5.4.4-4.el9 CentOS-AppStream 187 k 2024-03-04T09:18:04.149 INFO:teuthology.orchestra.run.smithi106.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-03-04T09:18:04.149 INFO:teuthology.orchestra.run.smithi106.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-03-04T09:18:04.149 INFO:teuthology.orchestra.run.smithi106.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-BaseOS 217 k 2024-03-04T09:18:04.149 INFO:teuthology.orchestra.run.smithi106.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2024-03-04T09:18:04.149 INFO:teuthology.orchestra.run.smithi106.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2024-03-04T09:18:04.149 INFO:teuthology.orchestra.run.smithi106.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-03-04T09:18:04.149 INFO:teuthology.orchestra.run.smithi106.stdout: postgresql-private-libs x86_64 13.11-1.el9 CentOS-AppStream 134 k 2024-03-04T09:18:04.149 INFO:teuthology.orchestra.run.smithi106.stdout: protobuf x86_64 3.14.0-13.el9 CentOS-AppStream 1.0 M 2024-03-04T09:18:04.149 INFO:teuthology.orchestra.run.smithi106.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-03-04T09:18:04.149 INFO:teuthology.orchestra.run.smithi106.stdout: python3-asyncssh noarch 2.13.2-4.el9 epel 548 k 2024-03-04T09:18:04.150 INFO:teuthology.orchestra.run.smithi106.stdout: python3-attrs noarch 20.3.0-7.el9 CentOS-AppStream 78 k 2024-03-04T09:18:04.150 INFO:teuthology.orchestra.run.smithi106.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2024-03-04T09:18:04.150 INFO:teuthology.orchestra.run.smithi106.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-03-04T09:18:04.150 INFO:teuthology.orchestra.run.smithi106.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-03-04T09:18:04.150 INFO:teuthology.orchestra.run.smithi106.stdout: python3-ceph-argparse x86_64 2:19.0.0-1739.g1bc250c7.el9 ceph 45 k 2024-03-04T09:18:04.150 INFO:teuthology.orchestra.run.smithi106.stdout: python3-ceph-common x86_64 2:19.0.0-1739.g1bc250c7.el9 ceph 131 k 2024-03-04T09:18:04.150 INFO:teuthology.orchestra.run.smithi106.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-03-04T09:18:04.150 INFO:teuthology.orchestra.run.smithi106.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2024-03-04T09:18:04.150 INFO:teuthology.orchestra.run.smithi106.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-03-04T09:18:04.150 INFO:teuthology.orchestra.run.smithi106.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-03-04T09:18:04.151 INFO:teuthology.orchestra.run.smithi106.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-03-04T09:18:04.151 INFO:teuthology.orchestra.run.smithi106.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-03-04T09:18:04.151 INFO:teuthology.orchestra.run.smithi106.stdout: python3-cryptography x86_64 36.0.1-4.el9 CentOS-BaseOS 1.1 M 2024-03-04T09:18:04.151 INFO:teuthology.orchestra.run.smithi106.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2024-03-04T09:18:04.151 INFO:teuthology.orchestra.run.smithi106.stdout: python3-google-auth noarch 1:2.28.0-1.el9 epel 203 k 2024-03-04T09:18:04.151 INFO:teuthology.orchestra.run.smithi106.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-03-04T09:18:04.151 INFO:teuthology.orchestra.run.smithi106.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-03-04T09:18:04.151 INFO:teuthology.orchestra.run.smithi106.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-03-04T09:18:04.151 INFO:teuthology.orchestra.run.smithi106.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-03-04T09:18:04.152 INFO:teuthology.orchestra.run.smithi106.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-03-04T09:18:04.152 INFO:teuthology.orchestra.run.smithi106.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-03-04T09:18:04.152 INFO:teuthology.orchestra.run.smithi106.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-03-04T09:18:04.152 INFO:teuthology.orchestra.run.smithi106.stdout: python3-jinja2 noarch 2.11.3-5.el9 CentOS-AppStream 227 k 2024-03-04T09:18:04.152 INFO:teuthology.orchestra.run.smithi106.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2024-03-04T09:18:04.152 INFO:teuthology.orchestra.run.smithi106.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2024-03-04T09:18:04.152 INFO:teuthology.orchestra.run.smithi106.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-03-04T09:18:04.152 INFO:teuthology.orchestra.run.smithi106.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-03-04T09:18:04.152 INFO:teuthology.orchestra.run.smithi106.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2024-03-04T09:18:04.153 INFO:teuthology.orchestra.run.smithi106.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2024-03-04T09:18:04.153 INFO:teuthology.orchestra.run.smithi106.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-03-04T09:18:04.153 INFO:teuthology.orchestra.run.smithi106.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-03-04T09:18:04.153 INFO:teuthology.orchestra.run.smithi106.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2024-03-04T09:18:04.153 INFO:teuthology.orchestra.run.smithi106.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2024-03-04T09:18:04.153 INFO:teuthology.orchestra.run.smithi106.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2024-03-04T09:18:04.153 INFO:teuthology.orchestra.run.smithi106.stdout: python3-packaging noarch 20.9-5.el9 CentOS-AppStream 69 k 2024-03-04T09:18:04.153 INFO:teuthology.orchestra.run.smithi106.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-03-04T09:18:04.153 INFO:teuthology.orchestra.run.smithi106.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2024-03-04T09:18:04.153 INFO:teuthology.orchestra.run.smithi106.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-03-04T09:18:04.153 INFO:teuthology.orchestra.run.smithi106.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2024-03-04T09:18:04.153 INFO:teuthology.orchestra.run.smithi106.stdout: python3-protobuf noarch 3.14.0-13.el9 CentOS-AppStream 238 k 2024-03-04T09:18:04.153 INFO:teuthology.orchestra.run.smithi106.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-03-04T09:18:04.154 INFO:teuthology.orchestra.run.smithi106.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2024-03-04T09:18:04.154 INFO:teuthology.orchestra.run.smithi106.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2024-03-04T09:18:04.154 INFO:teuthology.orchestra.run.smithi106.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2024-03-04T09:18:04.154 INFO:teuthology.orchestra.run.smithi106.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-03-04T09:18:04.154 INFO:teuthology.orchestra.run.smithi106.stdout: python3-pytz noarch 2021.1-5.el9 CentOS-AppStream 47 k 2024-03-04T09:18:04.154 INFO:teuthology.orchestra.run.smithi106.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-03-04T09:18:04.154 INFO:teuthology.orchestra.run.smithi106.stdout: python3-requests noarch 2.25.1-8.el9 CentOS-BaseOS 113 k 2024-03-04T09:18:04.155 INFO:teuthology.orchestra.run.smithi106.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2024-03-04T09:18:04.155 INFO:teuthology.orchestra.run.smithi106.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-03-04T09:18:04.155 INFO:teuthology.orchestra.run.smithi106.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-03-04T09:18:04.155 INFO:teuthology.orchestra.run.smithi106.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2024-03-04T09:18:04.155 INFO:teuthology.orchestra.run.smithi106.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-03-04T09:18:04.155 INFO:teuthology.orchestra.run.smithi106.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2024-03-04T09:18:04.155 INFO:teuthology.orchestra.run.smithi106.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-03-04T09:18:04.155 INFO:teuthology.orchestra.run.smithi106.stdout: python3-urllib3 noarch 1.26.5-5.el9 CentOS-BaseOS 187 k 2024-03-04T09:18:04.155 INFO:teuthology.orchestra.run.smithi106.stdout: python3-wcwidth noarch 0.2.5-8.el9 CentOS-AppStream 41 k 2024-03-04T09:18:04.155 INFO:teuthology.orchestra.run.smithi106.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-03-04T09:18:04.155 INFO:teuthology.orchestra.run.smithi106.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-03-04T09:18:04.156 INFO:teuthology.orchestra.run.smithi106.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-03-04T09:18:04.156 INFO:teuthology.orchestra.run.smithi106.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-03-04T09:18:04.156 INFO:teuthology.orchestra.run.smithi106.stdout: qatlib x86_64 23.11.0-1.el9 CentOS-AppStream 215 k 2024-03-04T09:18:04.156 INFO:teuthology.orchestra.run.smithi106.stdout: qatzip-libs x86_64 1.1.2-2.el9 CentOS-AppStream 43 k 2024-03-04T09:18:04.156 INFO:teuthology.orchestra.run.smithi106.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-03-04T09:18:04.156 INFO:teuthology.orchestra.run.smithi106.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2024-03-04T09:18:04.156 INFO:teuthology.orchestra.run.smithi106.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-03-04T09:18:04.156 INFO:teuthology.orchestra.run.smithi106.stdout: uuid x86_64 1.6.2-55.el9 CentOS-AppStream 56 k 2024-03-04T09:18:04.156 INFO:teuthology.orchestra.run.smithi106.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2024-03-04T09:18:04.156 INFO:teuthology.orchestra.run.smithi106.stdout: zlib-devel x86_64 1.2.11-40.el9 CentOS-AppStream 44 k 2024-03-04T09:18:04.156 INFO:teuthology.orchestra.run.smithi106.stdout:Installing weak dependencies: 2024-03-04T09:18:04.157 INFO:teuthology.orchestra.run.smithi106.stdout: qatlib-service x86_64 23.11.0-1.el9 CentOS-AppStream 34 k 2024-03-04T09:18:04.157 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-03-04T09:18:04.157 INFO:teuthology.orchestra.run.smithi106.stdout:Transaction Summary 2024-03-04T09:18:04.157 INFO:teuthology.orchestra.run.smithi106.stdout:============================================================================================ 2024-03-04T09:18:04.157 INFO:teuthology.orchestra.run.smithi106.stdout:Install 150 Packages 2024-03-04T09:18:04.157 INFO:teuthology.orchestra.run.smithi106.stdout:Upgrade 26 Packages 2024-03-04T09:18:04.157 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-03-04T09:18:04.158 INFO:teuthology.orchestra.run.smithi106.stdout:Total download size: 224 M 2024-03-04T09:18:04.158 INFO:teuthology.orchestra.run.smithi106.stdout:Downloading Packages: 2024-03-04T09:18:04.848 INFO:teuthology.orchestra.run.smithi106.stdout:(1/176): ceph-19.0.0-1739.g1bc250c7.el9.x86_64. 37 kB/s | 6.6 kB 00:00 2024-03-04T09:18:05.074 INFO:teuthology.orchestra.run.smithi106.stdout:(2/176): ceph-fuse-19.0.0-1739.g1bc250c7.el9.x8 3.9 MB/s | 888 kB 00:00 2024-03-04T09:18:05.113 INFO:teuthology.orchestra.run.smithi191.stdout:lab-extras 29 kB/s | 1.7 kB 00:00 2024-03-04T09:18:05.124 INFO:teuthology.orchestra.run.smithi106.stdout:(3/176): ceph-immutable-object-cache-19.0.0-173 2.8 MB/s | 145 kB 00:00 2024-03-04T09:18:05.458 INFO:teuthology.orchestra.run.smithi106.stdout:(4/176): ceph-mds-19.0.0-1739.g1bc250c7.el9.x86 6.9 MB/s | 2.3 MB 00:00 2024-03-04T09:18:05.575 INFO:teuthology.orchestra.run.smithi106.stdout:(5/176): ceph-base-19.0.0-1739.g1bc250c7.el9.x8 5.9 MB/s | 5.3 MB 00:00 2024-03-04T09:18:05.683 INFO:teuthology.orchestra.run.smithi106.stdout:(6/176): ceph-mgr-19.0.0-1739.g1bc250c7.el9.x86 6.6 MB/s | 1.5 MB 00:00 2024-03-04T09:18:06.150 INFO:teuthology.orchestra.run.smithi106.stdout:(7/176): ceph-mon-19.0.0-1739.g1bc250c7.el9.x86 8.2 MB/s | 4.7 MB 00:00 2024-03-04T09:18:06.867 INFO:teuthology.orchestra.run.smithi106.stdout:(8/176): ceph-common-19.0.0-1739.g1bc250c7.el9. 9.3 MB/s | 20 MB 00:02 2024-03-04T09:18:06.926 INFO:teuthology.orchestra.run.smithi106.stdout:(9/176): ceph-selinux-19.0.0-1739.g1bc250c7.el9 433 kB/s | 25 kB 00:00 2024-03-04T09:18:07.351 INFO:teuthology.orchestra.run.smithi106.stdout:(10/176): ceph-radosgw-19.0.0-1739.g1bc250c7.el 8.1 MB/s | 9.8 MB 00:01 2024-03-04T09:18:07.434 INFO:teuthology.orchestra.run.smithi106.stdout:(11/176): cephfs-mirror-19.0.0-1739.g1bc250c7.e 2.6 MB/s | 220 kB 00:00 2024-03-04T09:18:07.493 INFO:teuthology.orchestra.run.smithi106.stdout:(12/176): cephfs-shell-19.0.0-1739.g1bc250c7.el 464 kB/s | 27 kB 00:00 2024-03-04T09:18:07.551 INFO:teuthology.orchestra.run.smithi106.stdout:(13/176): libcephfs-devel-19.0.0-1739.g1bc250c7 537 kB/s | 31 kB 00:00 2024-03-04T09:18:07.662 INFO:teuthology.orchestra.run.smithi191.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-03-04T09:18:07.663 INFO:teuthology.orchestra.run.smithi191.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-03-04T09:18:07.677 INFO:teuthology.orchestra.run.smithi191.stdout:Package NetworkManager-1:1.44.0-2.el9.x86_64 is already installed. 2024-03-04T09:18:07.678 INFO:teuthology.orchestra.run.smithi052.stdout:lab-extras 22 kB/s | 1.7 kB 00:00 2024-03-04T09:18:07.679 INFO:teuthology.orchestra.run.smithi191.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-03-04T09:18:07.680 INFO:teuthology.orchestra.run.smithi191.stdout:Package util-linux-2.37.4-15.el9.x86_64 is already installed. 2024-03-04T09:18:07.681 INFO:teuthology.orchestra.run.smithi191.stdout:Package libacl-devel-2.3.1-3.el9.x86_64 is already installed. 2024-03-04T09:18:07.683 INFO:teuthology.orchestra.run.smithi191.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-03-04T09:18:07.686 INFO:teuthology.orchestra.run.smithi191.stdout:Package xfsdump-3.1.12-3.el9.x86_64 is already installed. 2024-03-04T09:18:07.687 INFO:teuthology.orchestra.run.smithi191.stdout:Package xfsprogs-5.19.0-4.el9.x86_64 is already installed. 2024-03-04T09:18:07.727 INFO:teuthology.orchestra.run.smithi106.stdout:(14/176): ceph-osd-19.0.0-1739.g1bc250c7.el9.x8 8.2 MB/s | 17 MB 00:02 2024-03-04T09:18:07.760 INFO:teuthology.orchestra.run.smithi106.stdout:(15/176): libcephfs2-19.0.0-1739.g1bc250c7.el9. 3.5 MB/s | 747 kB 00:00 2024-03-04T09:18:07.768 INFO:teuthology.orchestra.run.smithi191.stdout:Dependencies resolved. 2024-03-04T09:18:07.783 INFO:teuthology.orchestra.run.smithi191.stdout:============================================================================================ 2024-03-04T09:18:07.784 INFO:teuthology.orchestra.run.smithi191.stdout: Package Arch Version Repository Size 2024-03-04T09:18:07.784 INFO:teuthology.orchestra.run.smithi191.stdout:============================================================================================ 2024-03-04T09:18:07.784 INFO:teuthology.orchestra.run.smithi191.stdout:Installing: 2024-03-04T09:18:07.784 INFO:teuthology.orchestra.run.smithi191.stdout: bison x86_64 3.7.4-5.el9 CentOS-AppStream 920 k 2024-03-04T09:18:07.784 INFO:teuthology.orchestra.run.smithi191.stdout: ceph x86_64 2:19.0.0-1739.g1bc250c7.el9 ceph 6.6 k 2024-03-04T09:18:07.784 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-base x86_64 2:19.0.0-1739.g1bc250c7.el9 ceph 5.3 M 2024-03-04T09:18:07.784 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-fuse x86_64 2:19.0.0-1739.g1bc250c7.el9 ceph 888 k 2024-03-04T09:18:07.784 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-1739.g1bc250c7.el9 ceph 145 k 2024-03-04T09:18:07.784 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-mgr x86_64 2:19.0.0-1739.g1bc250c7.el9 ceph 1.5 M 2024-03-04T09:18:07.785 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-mgr-cephadm noarch 2:19.0.0-1739.g1bc250c7.el9 ceph-noarch 138 k 2024-03-04T09:18:07.785 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-mgr-dashboard noarch 2:19.0.0-1739.g1bc250c7.el9 ceph-noarch 1.8 M 2024-03-04T09:18:07.785 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-1739.g1bc250c7.el9 ceph-noarch 7.4 M 2024-03-04T09:18:07.785 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-mgr-rook noarch 2:19.0.0-1739.g1bc250c7.el9 ceph-noarch 49 k 2024-03-04T09:18:07.785 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-radosgw x86_64 2:19.0.0-1739.g1bc250c7.el9 ceph 9.8 M 2024-03-04T09:18:07.785 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-test x86_64 2:19.0.0-1739.g1bc250c7.el9 ceph 47 M 2024-03-04T09:18:07.785 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-volume noarch 2:19.0.0-1739.g1bc250c7.el9 ceph-noarch 264 k 2024-03-04T09:18:07.785 INFO:teuthology.orchestra.run.smithi191.stdout: cephadm noarch 2:19.0.0-1739.g1bc250c7.el9 ceph-noarch 748 k 2024-03-04T09:18:07.785 INFO:teuthology.orchestra.run.smithi191.stdout: cephfs-mirror x86_64 2:19.0.0-1739.g1bc250c7.el9 ceph 220 k 2024-03-04T09:18:07.786 INFO:teuthology.orchestra.run.smithi191.stdout: cephfs-shell x86_64 2:19.0.0-1739.g1bc250c7.el9 ceph 27 k 2024-03-04T09:18:07.786 INFO:teuthology.orchestra.run.smithi191.stdout: cephfs-top noarch 2:19.0.0-1739.g1bc250c7.el9 ceph-noarch 18 k 2024-03-04T09:18:07.786 INFO:teuthology.orchestra.run.smithi191.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 CentOS-AppStream 20 k 2024-03-04T09:18:07.786 INFO:teuthology.orchestra.run.smithi191.stdout: flex x86_64 2.6.4-9.el9 CentOS-AppStream 307 k 2024-03-04T09:18:07.786 INFO:teuthology.orchestra.run.smithi191.stdout: libaio-devel x86_64 0.3.111-13.el9 CentOS-AppStream 10 k 2024-03-04T09:18:07.786 INFO:teuthology.orchestra.run.smithi191.stdout: libcephfs-devel x86_64 2:19.0.0-1739.g1bc250c7.el9 ceph 31 k 2024-03-04T09:18:07.786 INFO:teuthology.orchestra.run.smithi191.stdout: libcephfs2 x86_64 2:19.0.0-1739.g1bc250c7.el9 ceph 747 k 2024-03-04T09:18:07.786 INFO:teuthology.orchestra.run.smithi191.stdout: librados-devel x86_64 2:19.0.0-1739.g1bc250c7.el9 ceph 127 k 2024-03-04T09:18:07.786 INFO:teuthology.orchestra.run.smithi191.stdout: libtool x86_64 2.4.6-45.el9 CentOS-AppStream 578 k 2024-03-04T09:18:07.787 INFO:teuthology.orchestra.run.smithi191.stdout: libuuid-devel x86_64 2.37.4-18.el9 CentOS-AppStream 24 k 2024-03-04T09:18:07.787 INFO:teuthology.orchestra.run.smithi191.stdout: openssl-devel x86_64 1:3.0.7-27.el9 CentOS-AppStream 3.0 M 2024-03-04T09:18:07.787 INFO:teuthology.orchestra.run.smithi191.stdout: postgresql x86_64 13.11-1.el9 CentOS-AppStream 1.5 M 2024-03-04T09:18:07.787 INFO:teuthology.orchestra.run.smithi191.stdout: postgresql-contrib x86_64 13.11-1.el9 CentOS-AppStream 811 k 2024-03-04T09:18:07.787 INFO:teuthology.orchestra.run.smithi191.stdout: postgresql-server x86_64 13.11-1.el9 CentOS-AppStream 5.7 M 2024-03-04T09:18:07.787 INFO:teuthology.orchestra.run.smithi191.stdout: python3-cephfs x86_64 2:19.0.0-1739.g1bc250c7.el9 ceph 162 k 2024-03-04T09:18:07.787 INFO:teuthology.orchestra.run.smithi191.stdout: python3-rados x86_64 2:19.0.0-1739.g1bc250c7.el9 ceph 321 k 2024-03-04T09:18:07.787 INFO:teuthology.orchestra.run.smithi191.stdout: python3-rbd x86_64 2:19.0.0-1739.g1bc250c7.el9 ceph 298 k 2024-03-04T09:18:07.787 INFO:teuthology.orchestra.run.smithi191.stdout: python3-rgw x86_64 2:19.0.0-1739.g1bc250c7.el9 ceph 99 k 2024-03-04T09:18:07.788 INFO:teuthology.orchestra.run.smithi191.stdout: rbd-fuse x86_64 2:19.0.0-1739.g1bc250c7.el9 ceph 85 k 2024-03-04T09:18:07.788 INFO:teuthology.orchestra.run.smithi191.stdout: rbd-mirror x86_64 2:19.0.0-1739.g1bc250c7.el9 ceph 3.1 M 2024-03-04T09:18:07.788 INFO:teuthology.orchestra.run.smithi191.stdout: rbd-nbd x86_64 2:19.0.0-1739.g1bc250c7.el9 ceph 171 k 2024-03-04T09:18:07.788 INFO:teuthology.orchestra.run.smithi191.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 CentOS-AppStream 87 k 2024-03-04T09:18:07.788 INFO:teuthology.orchestra.run.smithi191.stdout:Upgrading: 2024-03-04T09:18:07.788 INFO:teuthology.orchestra.run.smithi191.stdout: NetworkManager x86_64 1:1.46.0-1.el9 CentOS-BaseOS 2.3 M 2024-03-04T09:18:07.788 INFO:teuthology.orchestra.run.smithi191.stdout: NetworkManager-libnm x86_64 1:1.46.0-1.el9 CentOS-BaseOS 1.8 M 2024-03-04T09:18:07.788 INFO:teuthology.orchestra.run.smithi191.stdout: NetworkManager-team x86_64 1:1.46.0-1.el9 CentOS-BaseOS 41 k 2024-03-04T09:18:07.789 INFO:teuthology.orchestra.run.smithi191.stdout: NetworkManager-tui x86_64 1:1.46.0-1.el9 CentOS-BaseOS 247 k 2024-03-04T09:18:07.789 INFO:teuthology.orchestra.run.smithi191.stdout: acl x86_64 2.3.1-4.el9 CentOS-BaseOS 69 k 2024-03-04T09:18:07.789 INFO:teuthology.orchestra.run.smithi191.stdout: elfutils-debuginfod-client x86_64 0.190-2.el9 CentOS-BaseOS 35 k 2024-03-04T09:18:07.789 INFO:teuthology.orchestra.run.smithi191.stdout: elfutils-libelf x86_64 0.190-2.el9 CentOS-BaseOS 191 k 2024-03-04T09:18:07.789 INFO:teuthology.orchestra.run.smithi191.stdout: elfutils-libs x86_64 0.190-2.el9 CentOS-BaseOS 254 k 2024-03-04T09:18:07.789 INFO:teuthology.orchestra.run.smithi191.stdout: iproute x86_64 6.7.0-2.el9 CentOS-BaseOS 802 k 2024-03-04T09:18:07.789 INFO:teuthology.orchestra.run.smithi191.stdout: iproute-tc x86_64 6.7.0-2.el9 CentOS-BaseOS 433 k 2024-03-04T09:18:07.789 INFO:teuthology.orchestra.run.smithi191.stdout: libacl x86_64 2.3.1-4.el9 CentOS-BaseOS 22 k 2024-03-04T09:18:07.789 INFO:teuthology.orchestra.run.smithi191.stdout: libacl-devel x86_64 2.3.1-4.el9 CentOS-AppStream 68 k 2024-03-04T09:18:07.789 INFO:teuthology.orchestra.run.smithi191.stdout: libblkid x86_64 2.37.4-18.el9 CentOS-BaseOS 107 k 2024-03-04T09:18:07.790 INFO:teuthology.orchestra.run.smithi191.stdout: libfdisk x86_64 2.37.4-18.el9 CentOS-BaseOS 154 k 2024-03-04T09:18:07.790 INFO:teuthology.orchestra.run.smithi191.stdout: libmount x86_64 2.37.4-18.el9 CentOS-BaseOS 135 k 2024-03-04T09:18:07.790 INFO:teuthology.orchestra.run.smithi191.stdout: librados2 x86_64 2:19.0.0-1739.g1bc250c7.el9 ceph 3.4 M 2024-03-04T09:18:07.790 INFO:teuthology.orchestra.run.smithi191.stdout: librbd1 x86_64 2:19.0.0-1739.g1bc250c7.el9 ceph 3.1 M 2024-03-04T09:18:07.790 INFO:teuthology.orchestra.run.smithi191.stdout: libsmartcols x86_64 2.37.4-18.el9 CentOS-BaseOS 63 k 2024-03-04T09:18:07.790 INFO:teuthology.orchestra.run.smithi191.stdout: libuuid x86_64 2.37.4-18.el9 CentOS-BaseOS 28 k 2024-03-04T09:18:07.790 INFO:teuthology.orchestra.run.smithi191.stdout: openssl x86_64 1:3.0.7-27.el9 CentOS-BaseOS 1.2 M 2024-03-04T09:18:07.790 INFO:teuthology.orchestra.run.smithi191.stdout: openssl-libs x86_64 1:3.0.7-27.el9 CentOS-BaseOS 2.2 M 2024-03-04T09:18:07.790 INFO:teuthology.orchestra.run.smithi191.stdout: util-linux x86_64 2.37.4-18.el9 CentOS-BaseOS 2.2 M 2024-03-04T09:18:07.790 INFO:teuthology.orchestra.run.smithi191.stdout: util-linux-core x86_64 2.37.4-18.el9 CentOS-BaseOS 438 k 2024-03-04T09:18:07.790 INFO:teuthology.orchestra.run.smithi191.stdout: util-linux-user x86_64 2.37.4-18.el9 CentOS-BaseOS 30 k 2024-03-04T09:18:07.790 INFO:teuthology.orchestra.run.smithi191.stdout: xfsdump x86_64 3.1.12-4.el9 CentOS-BaseOS 321 k 2024-03-04T09:18:07.791 INFO:teuthology.orchestra.run.smithi191.stdout: xfsprogs x86_64 6.3.0-1.el9 CentOS-BaseOS 1.1 M 2024-03-04T09:18:07.791 INFO:teuthology.orchestra.run.smithi191.stdout:Installing dependencies: 2024-03-04T09:18:07.791 INFO:teuthology.orchestra.run.smithi191.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-03-04T09:18:07.791 INFO:teuthology.orchestra.run.smithi191.stdout: automake noarch 1.16.2-8.el9 CentOS-AppStream 662 k 2024-03-04T09:18:07.791 INFO:teuthology.orchestra.run.smithi191.stdout: boost-program-options x86_64 1.75.0-8.el9 CentOS-AppStream 106 k 2024-03-04T09:18:07.791 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-common x86_64 2:19.0.0-1739.g1bc250c7.el9 ceph 20 M 2024-03-04T09:18:07.791 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-grafana-dashboards noarch 2:19.0.0-1739.g1bc250c7.el9 ceph-noarch 31 k 2024-03-04T09:18:07.791 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-mds x86_64 2:19.0.0-1739.g1bc250c7.el9 ceph 2.3 M 2024-03-04T09:18:07.791 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-mgr-modules-core noarch 2:19.0.0-1739.g1bc250c7.el9 ceph-noarch 248 k 2024-03-04T09:18:07.791 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-mon x86_64 2:19.0.0-1739.g1bc250c7.el9 ceph 4.7 M 2024-03-04T09:18:07.792 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-osd x86_64 2:19.0.0-1739.g1bc250c7.el9 ceph 17 M 2024-03-04T09:18:07.792 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-prometheus-alerts noarch 2:19.0.0-1739.g1bc250c7.el9 ceph-noarch 15 k 2024-03-04T09:18:07.792 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-selinux x86_64 2:19.0.0-1739.g1bc250c7.el9 ceph 25 k 2024-03-04T09:18:07.792 INFO:teuthology.orchestra.run.smithi191.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2024-03-04T09:18:07.792 INFO:teuthology.orchestra.run.smithi191.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2024-03-04T09:18:07.792 INFO:teuthology.orchestra.run.smithi191.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2024-03-04T09:18:07.792 INFO:teuthology.orchestra.run.smithi191.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2024-03-04T09:18:07.792 INFO:teuthology.orchestra.run.smithi191.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-03-04T09:18:07.792 INFO:teuthology.orchestra.run.smithi191.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-03-04T09:18:07.792 INFO:teuthology.orchestra.run.smithi191.stdout: jq x86_64 1.6-16.el9 CentOS-BaseOS 186 k 2024-03-04T09:18:07.792 INFO:teuthology.orchestra.run.smithi191.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-03-04T09:18:07.792 INFO:teuthology.orchestra.run.smithi191.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-03-04T09:18:07.793 INFO:teuthology.orchestra.run.smithi191.stdout: libcephsqlite x86_64 2:19.0.0-1739.g1bc250c7.el9 ceph 163 k 2024-03-04T09:18:07.793 INFO:teuthology.orchestra.run.smithi191.stdout: libgfortran x86_64 11.4.1-3.el9 CentOS-BaseOS 802 k 2024-03-04T09:18:07.793 INFO:teuthology.orchestra.run.smithi191.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-03-04T09:18:07.793 INFO:teuthology.orchestra.run.smithi191.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2024-03-04T09:18:07.793 INFO:teuthology.orchestra.run.smithi191.stdout: libquadmath x86_64 11.4.1-3.el9 CentOS-BaseOS 194 k 2024-03-04T09:18:07.793 INFO:teuthology.orchestra.run.smithi191.stdout: librabbitmq x86_64 0.11.0-7.el9 CentOS-AppStream 44 k 2024-03-04T09:18:07.793 INFO:teuthology.orchestra.run.smithi191.stdout: libradosstriper1 x86_64 2:19.0.0-1739.g1bc250c7.el9 ceph 501 k 2024-03-04T09:18:07.793 INFO:teuthology.orchestra.run.smithi191.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2024-03-04T09:18:07.793 INFO:teuthology.orchestra.run.smithi191.stdout: librgw2 x86_64 2:19.0.0-1739.g1bc250c7.el9 ceph 5.1 M 2024-03-04T09:18:07.793 INFO:teuthology.orchestra.run.smithi191.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-03-04T09:18:07.793 INFO:teuthology.orchestra.run.smithi191.stdout: libzstd-devel x86_64 1.5.1-2.el9 CentOS-AppStream 46 k 2024-03-04T09:18:07.793 INFO:teuthology.orchestra.run.smithi191.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2024-03-04T09:18:07.793 INFO:teuthology.orchestra.run.smithi191.stdout: lua x86_64 5.4.4-4.el9 CentOS-AppStream 187 k 2024-03-04T09:18:07.793 INFO:teuthology.orchestra.run.smithi191.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-03-04T09:18:07.794 INFO:teuthology.orchestra.run.smithi191.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-03-04T09:18:07.794 INFO:teuthology.orchestra.run.smithi191.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-BaseOS 217 k 2024-03-04T09:18:07.794 INFO:teuthology.orchestra.run.smithi191.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2024-03-04T09:18:07.794 INFO:teuthology.orchestra.run.smithi191.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2024-03-04T09:18:07.794 INFO:teuthology.orchestra.run.smithi191.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-03-04T09:18:07.794 INFO:teuthology.orchestra.run.smithi191.stdout: postgresql-private-libs x86_64 13.11-1.el9 CentOS-AppStream 134 k 2024-03-04T09:18:07.794 INFO:teuthology.orchestra.run.smithi191.stdout: protobuf x86_64 3.14.0-13.el9 CentOS-AppStream 1.0 M 2024-03-04T09:18:07.794 INFO:teuthology.orchestra.run.smithi191.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-03-04T09:18:07.794 INFO:teuthology.orchestra.run.smithi191.stdout: python3-asyncssh noarch 2.13.2-4.el9 epel 548 k 2024-03-04T09:18:07.794 INFO:teuthology.orchestra.run.smithi191.stdout: python3-attrs noarch 20.3.0-7.el9 CentOS-AppStream 78 k 2024-03-04T09:18:07.794 INFO:teuthology.orchestra.run.smithi191.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2024-03-04T09:18:07.794 INFO:teuthology.orchestra.run.smithi191.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-03-04T09:18:07.794 INFO:teuthology.orchestra.run.smithi191.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-03-04T09:18:07.795 INFO:teuthology.orchestra.run.smithi191.stdout: python3-ceph-argparse x86_64 2:19.0.0-1739.g1bc250c7.el9 ceph 45 k 2024-03-04T09:18:07.795 INFO:teuthology.orchestra.run.smithi191.stdout: python3-ceph-common x86_64 2:19.0.0-1739.g1bc250c7.el9 ceph 131 k 2024-03-04T09:18:07.795 INFO:teuthology.orchestra.run.smithi191.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-03-04T09:18:07.795 INFO:teuthology.orchestra.run.smithi191.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2024-03-04T09:18:07.795 INFO:teuthology.orchestra.run.smithi191.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-03-04T09:18:07.795 INFO:teuthology.orchestra.run.smithi191.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-03-04T09:18:07.795 INFO:teuthology.orchestra.run.smithi191.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-03-04T09:18:07.795 INFO:teuthology.orchestra.run.smithi191.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-03-04T09:18:07.795 INFO:teuthology.orchestra.run.smithi191.stdout: python3-cryptography x86_64 36.0.1-4.el9 CentOS-BaseOS 1.1 M 2024-03-04T09:18:07.795 INFO:teuthology.orchestra.run.smithi191.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2024-03-04T09:18:07.795 INFO:teuthology.orchestra.run.smithi191.stdout: python3-google-auth noarch 1:2.28.0-1.el9 epel 203 k 2024-03-04T09:18:07.795 INFO:teuthology.orchestra.run.smithi191.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-03-04T09:18:07.795 INFO:teuthology.orchestra.run.smithi191.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-03-04T09:18:07.795 INFO:teuthology.orchestra.run.smithi191.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-03-04T09:18:07.796 INFO:teuthology.orchestra.run.smithi191.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-03-04T09:18:07.796 INFO:teuthology.orchestra.run.smithi191.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-03-04T09:18:07.796 INFO:teuthology.orchestra.run.smithi191.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-03-04T09:18:07.796 INFO:teuthology.orchestra.run.smithi191.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-03-04T09:18:07.796 INFO:teuthology.orchestra.run.smithi191.stdout: python3-jinja2 noarch 2.11.3-5.el9 CentOS-AppStream 227 k 2024-03-04T09:18:07.796 INFO:teuthology.orchestra.run.smithi191.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2024-03-04T09:18:07.796 INFO:teuthology.orchestra.run.smithi191.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2024-03-04T09:18:07.796 INFO:teuthology.orchestra.run.smithi191.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-03-04T09:18:07.796 INFO:teuthology.orchestra.run.smithi191.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-03-04T09:18:07.796 INFO:teuthology.orchestra.run.smithi191.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2024-03-04T09:18:07.796 INFO:teuthology.orchestra.run.smithi191.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2024-03-04T09:18:07.796 INFO:teuthology.orchestra.run.smithi191.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-03-04T09:18:07.796 INFO:teuthology.orchestra.run.smithi191.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-03-04T09:18:07.797 INFO:teuthology.orchestra.run.smithi191.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2024-03-04T09:18:07.797 INFO:teuthology.orchestra.run.smithi191.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2024-03-04T09:18:07.797 INFO:teuthology.orchestra.run.smithi191.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2024-03-04T09:18:07.797 INFO:teuthology.orchestra.run.smithi191.stdout: python3-packaging noarch 20.9-5.el9 CentOS-AppStream 69 k 2024-03-04T09:18:07.797 INFO:teuthology.orchestra.run.smithi191.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-03-04T09:18:07.797 INFO:teuthology.orchestra.run.smithi191.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2024-03-04T09:18:07.797 INFO:teuthology.orchestra.run.smithi191.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-03-04T09:18:07.797 INFO:teuthology.orchestra.run.smithi191.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2024-03-04T09:18:07.797 INFO:teuthology.orchestra.run.smithi191.stdout: python3-protobuf noarch 3.14.0-13.el9 CentOS-AppStream 238 k 2024-03-04T09:18:07.797 INFO:teuthology.orchestra.run.smithi191.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-03-04T09:18:07.797 INFO:teuthology.orchestra.run.smithi191.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2024-03-04T09:18:07.797 INFO:teuthology.orchestra.run.smithi191.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2024-03-04T09:18:07.797 INFO:teuthology.orchestra.run.smithi191.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2024-03-04T09:18:07.797 INFO:teuthology.orchestra.run.smithi191.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-03-04T09:18:07.798 INFO:teuthology.orchestra.run.smithi191.stdout: python3-pytz noarch 2021.1-5.el9 CentOS-AppStream 47 k 2024-03-04T09:18:07.798 INFO:teuthology.orchestra.run.smithi191.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-03-04T09:18:07.798 INFO:teuthology.orchestra.run.smithi191.stdout: python3-requests noarch 2.25.1-8.el9 CentOS-BaseOS 113 k 2024-03-04T09:18:07.798 INFO:teuthology.orchestra.run.smithi191.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2024-03-04T09:18:07.798 INFO:teuthology.orchestra.run.smithi191.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-03-04T09:18:07.798 INFO:teuthology.orchestra.run.smithi191.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-03-04T09:18:07.798 INFO:teuthology.orchestra.run.smithi191.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2024-03-04T09:18:07.798 INFO:teuthology.orchestra.run.smithi191.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-03-04T09:18:07.798 INFO:teuthology.orchestra.run.smithi191.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2024-03-04T09:18:07.798 INFO:teuthology.orchestra.run.smithi191.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-03-04T09:18:07.798 INFO:teuthology.orchestra.run.smithi191.stdout: python3-urllib3 noarch 1.26.5-5.el9 CentOS-BaseOS 187 k 2024-03-04T09:18:07.798 INFO:teuthology.orchestra.run.smithi191.stdout: python3-wcwidth noarch 0.2.5-8.el9 CentOS-AppStream 41 k 2024-03-04T09:18:07.798 INFO:teuthology.orchestra.run.smithi191.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-03-04T09:18:07.799 INFO:teuthology.orchestra.run.smithi191.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-03-04T09:18:07.799 INFO:teuthology.orchestra.run.smithi191.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-03-04T09:18:07.799 INFO:teuthology.orchestra.run.smithi191.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-03-04T09:18:07.799 INFO:teuthology.orchestra.run.smithi191.stdout: qatlib x86_64 23.11.0-1.el9 CentOS-AppStream 215 k 2024-03-04T09:18:07.799 INFO:teuthology.orchestra.run.smithi191.stdout: qatzip-libs x86_64 1.1.2-2.el9 CentOS-AppStream 43 k 2024-03-04T09:18:07.799 INFO:teuthology.orchestra.run.smithi191.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-03-04T09:18:07.799 INFO:teuthology.orchestra.run.smithi191.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2024-03-04T09:18:07.799 INFO:teuthology.orchestra.run.smithi191.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-03-04T09:18:07.799 INFO:teuthology.orchestra.run.smithi191.stdout: uuid x86_64 1.6.2-55.el9 CentOS-AppStream 56 k 2024-03-04T09:18:07.799 INFO:teuthology.orchestra.run.smithi191.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2024-03-04T09:18:07.800 INFO:teuthology.orchestra.run.smithi191.stdout: zlib-devel x86_64 1.2.11-40.el9 CentOS-AppStream 44 k 2024-03-04T09:18:07.800 INFO:teuthology.orchestra.run.smithi191.stdout:Installing weak dependencies: 2024-03-04T09:18:07.800 INFO:teuthology.orchestra.run.smithi191.stdout: qatlib-service x86_64 23.11.0-1.el9 CentOS-AppStream 34 k 2024-03-04T09:18:07.800 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-03-04T09:18:07.800 INFO:teuthology.orchestra.run.smithi191.stdout:Transaction Summary 2024-03-04T09:18:07.800 INFO:teuthology.orchestra.run.smithi191.stdout:============================================================================================ 2024-03-04T09:18:07.800 INFO:teuthology.orchestra.run.smithi191.stdout:Install 150 Packages 2024-03-04T09:18:07.800 INFO:teuthology.orchestra.run.smithi191.stdout:Upgrade 26 Packages 2024-03-04T09:18:07.800 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-03-04T09:18:07.801 INFO:teuthology.orchestra.run.smithi106.stdout:(16/176): libcephsqlite-19.0.0-1739.g1bc250c7.e 2.7 MB/s | 163 kB 00:00 2024-03-04T09:18:07.801 INFO:teuthology.orchestra.run.smithi191.stdout:Total download size: 224 M 2024-03-04T09:18:07.801 INFO:teuthology.orchestra.run.smithi191.stdout:Downloading Packages: 2024-03-04T09:18:07.819 INFO:teuthology.orchestra.run.smithi106.stdout:(17/176): librados-devel-19.0.0-1739.g1bc250c7. 2.1 MB/s | 127 kB 00:00 2024-03-04T09:18:07.861 INFO:teuthology.orchestra.run.smithi106.stdout:(18/176): libradosstriper1-19.0.0-1739.g1bc250c 6.5 MB/s | 501 kB 00:00 2024-03-04T09:18:07.928 INFO:teuthology.orchestra.run.smithi106.stdout:(19/176): python3-ceph-argparse-19.0.0-1739.g1b 678 kB/s | 45 kB 00:00 2024-03-04T09:18:08.012 INFO:teuthology.orchestra.run.smithi106.stdout:(20/176): python3-ceph-common-19.0.0-1739.g1bc2 1.5 MB/s | 131 kB 00:00 2024-03-04T09:18:08.087 INFO:teuthology.orchestra.run.smithi106.stdout:(21/176): python3-cephfs-19.0.0-1739.g1bc250c7. 2.1 MB/s | 162 kB 00:00 2024-03-04T09:18:08.171 INFO:teuthology.orchestra.run.smithi106.stdout:(22/176): python3-rados-19.0.0-1739.g1bc250c7.e 3.8 MB/s | 321 kB 00:00 2024-03-04T09:18:08.254 INFO:teuthology.orchestra.run.smithi106.stdout:(23/176): python3-rbd-19.0.0-1739.g1bc250c7.el9 3.5 MB/s | 298 kB 00:00 2024-03-04T09:18:08.371 INFO:teuthology.orchestra.run.smithi106.stdout:(24/176): librgw2-19.0.0-1739.g1bc250c7.el9.x86 9.3 MB/s | 5.1 MB 00:00 2024-03-04T09:18:08.396 INFO:teuthology.orchestra.run.smithi106.stdout:(25/176): python3-rgw-19.0.0-1739.g1bc250c7.el9 699 kB/s | 99 kB 00:00 2024-03-04T09:18:08.430 INFO:teuthology.orchestra.run.smithi106.stdout:(26/176): rbd-fuse-19.0.0-1739.g1bc250c7.el9.x8 1.4 MB/s | 85 kB 00:00 2024-03-04T09:18:08.497 INFO:teuthology.orchestra.run.smithi106.stdout:(27/176): rbd-nbd-19.0.0-1739.g1bc250c7.el9.x86 2.5 MB/s | 171 kB 00:00 2024-03-04T09:18:08.524 INFO:teuthology.orchestra.run.smithi191.stdout:(1/176): ceph-19.0.0-1739.g1bc250c7.el9.x86_64. 25 kB/s | 6.6 kB 00:00 2024-03-04T09:18:08.572 INFO:teuthology.orchestra.run.smithi106.stdout:(28/176): ceph-grafana-dashboards-19.0.0-1739.g 419 kB/s | 31 kB 00:00 2024-03-04T09:18:08.656 INFO:teuthology.orchestra.run.smithi106.stdout:(29/176): ceph-mgr-cephadm-19.0.0-1739.g1bc250c 1.6 MB/s | 138 kB 00:00 2024-03-04T09:18:08.807 INFO:teuthology.orchestra.run.smithi106.stdout:(30/176): rbd-mirror-19.0.0-1739.g1bc250c7.el9. 7.5 MB/s | 3.1 MB 00:00 2024-03-04T09:18:08.957 INFO:teuthology.orchestra.run.smithi106.stdout:(31/176): ceph-mgr-dashboard-19.0.0-1739.g1bc25 5.9 MB/s | 1.8 MB 00:00 2024-03-04T09:18:09.057 INFO:teuthology.orchestra.run.smithi106.stdout:(32/176): ceph-mgr-modules-core-19.0.0-1739.g1b 2.4 MB/s | 248 kB 00:00 2024-03-04T09:18:09.140 INFO:teuthology.orchestra.run.smithi106.stdout:(33/176): ceph-mgr-rook-19.0.0-1739.g1bc250c7.e 593 kB/s | 49 kB 00:00 2024-03-04T09:18:09.233 INFO:teuthology.orchestra.run.smithi106.stdout:(34/176): ceph-prometheus-alerts-19.0.0-1739.g1 166 kB/s | 15 kB 00:00 2024-03-04T09:18:09.325 INFO:teuthology.orchestra.run.smithi106.stdout:(35/176): ceph-volume-19.0.0-1739.g1bc250c7.el9 2.8 MB/s | 264 kB 00:00 2024-03-04T09:18:09.566 INFO:teuthology.orchestra.run.smithi106.stdout:(36/176): cephadm-19.0.0-1739.g1bc250c7.el9.noa 3.0 MB/s | 748 kB 00:00 2024-03-04T09:18:09.642 INFO:teuthology.orchestra.run.smithi106.stdout:(37/176): cephfs-top-19.0.0-1739.g1bc250c7.el9. 243 kB/s | 18 kB 00:00 2024-03-04T09:18:09.665 INFO:teuthology.orchestra.run.smithi191.stdout:(2/176): ceph-fuse-19.0.0-1739.g1bc250c7.el9.x8 778 kB/s | 888 kB 00:01 2024-03-04T09:18:09.766 INFO:teuthology.orchestra.run.smithi191.stdout:(3/176): ceph-immutable-object-cache-19.0.0-173 1.4 MB/s | 145 kB 00:00 2024-03-04T09:18:09.767 INFO:teuthology.orchestra.run.smithi106.stdout:(38/176): ceph-mgr-diskprediction-local-19.0.0- 7.7 MB/s | 7.4 MB 00:00 2024-03-04T09:18:09.859 INFO:teuthology.orchestra.run.smithi106.stdout:(39/176): bison-3.7.4-5.el9.x86_64.rpm 10 MB/s | 920 kB 00:00 2024-03-04T09:18:09.884 INFO:teuthology.orchestra.run.smithi106.stdout:(40/176): automake-1.16.2-8.el9.noarch.rpm 2.7 MB/s | 662 kB 00:00 2024-03-04T09:18:09.910 INFO:teuthology.orchestra.run.smithi106.stdout:(41/176): boost-program-options-1.75.0-8.el9.x8 2.1 MB/s | 106 kB 00:00 2024-03-04T09:18:09.926 INFO:teuthology.orchestra.run.smithi106.stdout:(42/176): elfutils-libelf-devel-0.190-2.el9.x86 487 kB/s | 20 kB 00:00 2024-03-04T09:18:09.952 INFO:teuthology.orchestra.run.smithi106.stdout:(43/176): flex-2.6.4-9.el9.x86_64.rpm 7.2 MB/s | 307 kB 00:00 2024-03-04T09:18:09.994 INFO:teuthology.orchestra.run.smithi106.stdout:(44/176): flexiblas-3.0.4-8.el9.x86_64.rpm 475 kB/s | 32 kB 00:00 2024-03-04T09:18:10.301 INFO:teuthology.orchestra.run.smithi052.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-03-04T09:18:10.302 INFO:teuthology.orchestra.run.smithi052.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-03-04T09:18:10.317 INFO:teuthology.orchestra.run.smithi052.stdout:Package NetworkManager-1:1.44.0-2.el9.x86_64 is already installed. 2024-03-04T09:18:10.318 INFO:teuthology.orchestra.run.smithi052.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-03-04T09:18:10.319 INFO:teuthology.orchestra.run.smithi052.stdout:Package util-linux-2.37.4-15.el9.x86_64 is already installed. 2024-03-04T09:18:10.320 INFO:teuthology.orchestra.run.smithi052.stdout:Package libacl-devel-2.3.1-3.el9.x86_64 is already installed. 2024-03-04T09:18:10.322 INFO:teuthology.orchestra.run.smithi052.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-03-04T09:18:10.326 INFO:teuthology.orchestra.run.smithi052.stdout:Package xfsdump-3.1.12-3.el9.x86_64 is already installed. 2024-03-04T09:18:10.327 INFO:teuthology.orchestra.run.smithi052.stdout:Package xfsprogs-5.19.0-4.el9.x86_64 is already installed. 2024-03-04T09:18:10.415 INFO:teuthology.orchestra.run.smithi052.stdout:Dependencies resolved. 2024-03-04T09:18:10.430 INFO:teuthology.orchestra.run.smithi052.stdout:============================================================================================ 2024-03-04T09:18:10.430 INFO:teuthology.orchestra.run.smithi052.stdout: Package Arch Version Repository Size 2024-03-04T09:18:10.430 INFO:teuthology.orchestra.run.smithi052.stdout:============================================================================================ 2024-03-04T09:18:10.430 INFO:teuthology.orchestra.run.smithi052.stdout:Installing: 2024-03-04T09:18:10.430 INFO:teuthology.orchestra.run.smithi052.stdout: bison x86_64 3.7.4-5.el9 CentOS-AppStream 920 k 2024-03-04T09:18:10.431 INFO:teuthology.orchestra.run.smithi052.stdout: ceph x86_64 2:19.0.0-1739.g1bc250c7.el9 ceph 6.6 k 2024-03-04T09:18:10.431 INFO:teuthology.orchestra.run.smithi052.stdout: ceph-base x86_64 2:19.0.0-1739.g1bc250c7.el9 ceph 5.3 M 2024-03-04T09:18:10.431 INFO:teuthology.orchestra.run.smithi052.stdout: ceph-fuse x86_64 2:19.0.0-1739.g1bc250c7.el9 ceph 888 k 2024-03-04T09:18:10.431 INFO:teuthology.orchestra.run.smithi052.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-1739.g1bc250c7.el9 ceph 145 k 2024-03-04T09:18:10.431 INFO:teuthology.orchestra.run.smithi052.stdout: ceph-mgr x86_64 2:19.0.0-1739.g1bc250c7.el9 ceph 1.5 M 2024-03-04T09:18:10.431 INFO:teuthology.orchestra.run.smithi052.stdout: ceph-mgr-cephadm noarch 2:19.0.0-1739.g1bc250c7.el9 ceph-noarch 138 k 2024-03-04T09:18:10.431 INFO:teuthology.orchestra.run.smithi052.stdout: ceph-mgr-dashboard noarch 2:19.0.0-1739.g1bc250c7.el9 ceph-noarch 1.8 M 2024-03-04T09:18:10.431 INFO:teuthology.orchestra.run.smithi052.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-1739.g1bc250c7.el9 ceph-noarch 7.4 M 2024-03-04T09:18:10.431 INFO:teuthology.orchestra.run.smithi052.stdout: ceph-mgr-rook noarch 2:19.0.0-1739.g1bc250c7.el9 ceph-noarch 49 k 2024-03-04T09:18:10.431 INFO:teuthology.orchestra.run.smithi052.stdout: ceph-radosgw x86_64 2:19.0.0-1739.g1bc250c7.el9 ceph 9.8 M 2024-03-04T09:18:10.431 INFO:teuthology.orchestra.run.smithi052.stdout: ceph-test x86_64 2:19.0.0-1739.g1bc250c7.el9 ceph 47 M 2024-03-04T09:18:10.431 INFO:teuthology.orchestra.run.smithi052.stdout: ceph-volume noarch 2:19.0.0-1739.g1bc250c7.el9 ceph-noarch 264 k 2024-03-04T09:18:10.431 INFO:teuthology.orchestra.run.smithi052.stdout: cephadm noarch 2:19.0.0-1739.g1bc250c7.el9 ceph-noarch 748 k 2024-03-04T09:18:10.432 INFO:teuthology.orchestra.run.smithi052.stdout: cephfs-mirror x86_64 2:19.0.0-1739.g1bc250c7.el9 ceph 220 k 2024-03-04T09:18:10.432 INFO:teuthology.orchestra.run.smithi052.stdout: cephfs-shell x86_64 2:19.0.0-1739.g1bc250c7.el9 ceph 27 k 2024-03-04T09:18:10.432 INFO:teuthology.orchestra.run.smithi052.stdout: cephfs-top noarch 2:19.0.0-1739.g1bc250c7.el9 ceph-noarch 18 k 2024-03-04T09:18:10.432 INFO:teuthology.orchestra.run.smithi052.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 CentOS-AppStream 20 k 2024-03-04T09:18:10.432 INFO:teuthology.orchestra.run.smithi052.stdout: flex x86_64 2.6.4-9.el9 CentOS-AppStream 307 k 2024-03-04T09:18:10.432 INFO:teuthology.orchestra.run.smithi052.stdout: libaio-devel x86_64 0.3.111-13.el9 CentOS-AppStream 10 k 2024-03-04T09:18:10.432 INFO:teuthology.orchestra.run.smithi052.stdout: libcephfs-devel x86_64 2:19.0.0-1739.g1bc250c7.el9 ceph 31 k 2024-03-04T09:18:10.432 INFO:teuthology.orchestra.run.smithi052.stdout: libcephfs2 x86_64 2:19.0.0-1739.g1bc250c7.el9 ceph 747 k 2024-03-04T09:18:10.432 INFO:teuthology.orchestra.run.smithi052.stdout: librados-devel x86_64 2:19.0.0-1739.g1bc250c7.el9 ceph 127 k 2024-03-04T09:18:10.432 INFO:teuthology.orchestra.run.smithi052.stdout: libtool x86_64 2.4.6-45.el9 CentOS-AppStream 578 k 2024-03-04T09:18:10.432 INFO:teuthology.orchestra.run.smithi052.stdout: libuuid-devel x86_64 2.37.4-18.el9 CentOS-AppStream 24 k 2024-03-04T09:18:10.432 INFO:teuthology.orchestra.run.smithi052.stdout: openssl-devel x86_64 1:3.0.7-27.el9 CentOS-AppStream 3.0 M 2024-03-04T09:18:10.432 INFO:teuthology.orchestra.run.smithi052.stdout: postgresql x86_64 13.11-1.el9 CentOS-AppStream 1.5 M 2024-03-04T09:18:10.433 INFO:teuthology.orchestra.run.smithi052.stdout: postgresql-contrib x86_64 13.11-1.el9 CentOS-AppStream 811 k 2024-03-04T09:18:10.433 INFO:teuthology.orchestra.run.smithi052.stdout: postgresql-server x86_64 13.11-1.el9 CentOS-AppStream 5.7 M 2024-03-04T09:18:10.433 INFO:teuthology.orchestra.run.smithi052.stdout: python3-cephfs x86_64 2:19.0.0-1739.g1bc250c7.el9 ceph 162 k 2024-03-04T09:18:10.433 INFO:teuthology.orchestra.run.smithi052.stdout: python3-rados x86_64 2:19.0.0-1739.g1bc250c7.el9 ceph 321 k 2024-03-04T09:18:10.433 INFO:teuthology.orchestra.run.smithi052.stdout: python3-rbd x86_64 2:19.0.0-1739.g1bc250c7.el9 ceph 298 k 2024-03-04T09:18:10.433 INFO:teuthology.orchestra.run.smithi052.stdout: python3-rgw x86_64 2:19.0.0-1739.g1bc250c7.el9 ceph 99 k 2024-03-04T09:18:10.433 INFO:teuthology.orchestra.run.smithi052.stdout: rbd-fuse x86_64 2:19.0.0-1739.g1bc250c7.el9 ceph 85 k 2024-03-04T09:18:10.433 INFO:teuthology.orchestra.run.smithi052.stdout: rbd-mirror x86_64 2:19.0.0-1739.g1bc250c7.el9 ceph 3.1 M 2024-03-04T09:18:10.433 INFO:teuthology.orchestra.run.smithi052.stdout: rbd-nbd x86_64 2:19.0.0-1739.g1bc250c7.el9 ceph 171 k 2024-03-04T09:18:10.433 INFO:teuthology.orchestra.run.smithi052.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 CentOS-AppStream 87 k 2024-03-04T09:18:10.433 INFO:teuthology.orchestra.run.smithi052.stdout:Upgrading: 2024-03-04T09:18:10.433 INFO:teuthology.orchestra.run.smithi052.stdout: NetworkManager x86_64 1:1.46.0-1.el9 CentOS-BaseOS 2.3 M 2024-03-04T09:18:10.433 INFO:teuthology.orchestra.run.smithi052.stdout: NetworkManager-libnm x86_64 1:1.46.0-1.el9 CentOS-BaseOS 1.8 M 2024-03-04T09:18:10.434 INFO:teuthology.orchestra.run.smithi052.stdout: NetworkManager-team x86_64 1:1.46.0-1.el9 CentOS-BaseOS 41 k 2024-03-04T09:18:10.434 INFO:teuthology.orchestra.run.smithi052.stdout: NetworkManager-tui x86_64 1:1.46.0-1.el9 CentOS-BaseOS 247 k 2024-03-04T09:18:10.434 INFO:teuthology.orchestra.run.smithi052.stdout: acl x86_64 2.3.1-4.el9 CentOS-BaseOS 69 k 2024-03-04T09:18:10.434 INFO:teuthology.orchestra.run.smithi052.stdout: elfutils-debuginfod-client x86_64 0.190-2.el9 CentOS-BaseOS 35 k 2024-03-04T09:18:10.434 INFO:teuthology.orchestra.run.smithi052.stdout: elfutils-libelf x86_64 0.190-2.el9 CentOS-BaseOS 191 k 2024-03-04T09:18:10.434 INFO:teuthology.orchestra.run.smithi052.stdout: elfutils-libs x86_64 0.190-2.el9 CentOS-BaseOS 254 k 2024-03-04T09:18:10.434 INFO:teuthology.orchestra.run.smithi052.stdout: iproute x86_64 6.7.0-2.el9 CentOS-BaseOS 802 k 2024-03-04T09:18:10.434 INFO:teuthology.orchestra.run.smithi052.stdout: iproute-tc x86_64 6.7.0-2.el9 CentOS-BaseOS 433 k 2024-03-04T09:18:10.434 INFO:teuthology.orchestra.run.smithi052.stdout: libacl x86_64 2.3.1-4.el9 CentOS-BaseOS 22 k 2024-03-04T09:18:10.434 INFO:teuthology.orchestra.run.smithi052.stdout: libacl-devel x86_64 2.3.1-4.el9 CentOS-AppStream 68 k 2024-03-04T09:18:10.434 INFO:teuthology.orchestra.run.smithi052.stdout: libblkid x86_64 2.37.4-18.el9 CentOS-BaseOS 107 k 2024-03-04T09:18:10.435 INFO:teuthology.orchestra.run.smithi052.stdout: libfdisk x86_64 2.37.4-18.el9 CentOS-BaseOS 154 k 2024-03-04T09:18:10.435 INFO:teuthology.orchestra.run.smithi052.stdout: libmount x86_64 2.37.4-18.el9 CentOS-BaseOS 135 k 2024-03-04T09:18:10.435 INFO:teuthology.orchestra.run.smithi052.stdout: librados2 x86_64 2:19.0.0-1739.g1bc250c7.el9 ceph 3.4 M 2024-03-04T09:18:10.435 INFO:teuthology.orchestra.run.smithi052.stdout: librbd1 x86_64 2:19.0.0-1739.g1bc250c7.el9 ceph 3.1 M 2024-03-04T09:18:10.435 INFO:teuthology.orchestra.run.smithi052.stdout: libsmartcols x86_64 2.37.4-18.el9 CentOS-BaseOS 63 k 2024-03-04T09:18:10.435 INFO:teuthology.orchestra.run.smithi052.stdout: libuuid x86_64 2.37.4-18.el9 CentOS-BaseOS 28 k 2024-03-04T09:18:10.435 INFO:teuthology.orchestra.run.smithi052.stdout: openssl x86_64 1:3.0.7-27.el9 CentOS-BaseOS 1.2 M 2024-03-04T09:18:10.435 INFO:teuthology.orchestra.run.smithi052.stdout: openssl-libs x86_64 1:3.0.7-27.el9 CentOS-BaseOS 2.2 M 2024-03-04T09:18:10.436 INFO:teuthology.orchestra.run.smithi052.stdout: util-linux x86_64 2.37.4-18.el9 CentOS-BaseOS 2.2 M 2024-03-04T09:18:10.436 INFO:teuthology.orchestra.run.smithi052.stdout: util-linux-core x86_64 2.37.4-18.el9 CentOS-BaseOS 438 k 2024-03-04T09:18:10.436 INFO:teuthology.orchestra.run.smithi052.stdout: util-linux-user x86_64 2.37.4-18.el9 CentOS-BaseOS 30 k 2024-03-04T09:18:10.436 INFO:teuthology.orchestra.run.smithi052.stdout: xfsdump x86_64 3.1.12-4.el9 CentOS-BaseOS 321 k 2024-03-04T09:18:10.436 INFO:teuthology.orchestra.run.smithi052.stdout: xfsprogs x86_64 6.3.0-1.el9 CentOS-BaseOS 1.1 M 2024-03-04T09:18:10.436 INFO:teuthology.orchestra.run.smithi052.stdout:Installing dependencies: 2024-03-04T09:18:10.436 INFO:teuthology.orchestra.run.smithi052.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-03-04T09:18:10.436 INFO:teuthology.orchestra.run.smithi052.stdout: automake noarch 1.16.2-8.el9 CentOS-AppStream 662 k 2024-03-04T09:18:10.436 INFO:teuthology.orchestra.run.smithi052.stdout: boost-program-options x86_64 1.75.0-8.el9 CentOS-AppStream 106 k 2024-03-04T09:18:10.436 INFO:teuthology.orchestra.run.smithi052.stdout: ceph-common x86_64 2:19.0.0-1739.g1bc250c7.el9 ceph 20 M 2024-03-04T09:18:10.436 INFO:teuthology.orchestra.run.smithi052.stdout: ceph-grafana-dashboards noarch 2:19.0.0-1739.g1bc250c7.el9 ceph-noarch 31 k 2024-03-04T09:18:10.437 INFO:teuthology.orchestra.run.smithi052.stdout: ceph-mds x86_64 2:19.0.0-1739.g1bc250c7.el9 ceph 2.3 M 2024-03-04T09:18:10.437 INFO:teuthology.orchestra.run.smithi052.stdout: ceph-mgr-modules-core noarch 2:19.0.0-1739.g1bc250c7.el9 ceph-noarch 248 k 2024-03-04T09:18:10.437 INFO:teuthology.orchestra.run.smithi052.stdout: ceph-mon x86_64 2:19.0.0-1739.g1bc250c7.el9 ceph 4.7 M 2024-03-04T09:18:10.437 INFO:teuthology.orchestra.run.smithi052.stdout: ceph-osd x86_64 2:19.0.0-1739.g1bc250c7.el9 ceph 17 M 2024-03-04T09:18:10.437 INFO:teuthology.orchestra.run.smithi052.stdout: ceph-prometheus-alerts noarch 2:19.0.0-1739.g1bc250c7.el9 ceph-noarch 15 k 2024-03-04T09:18:10.437 INFO:teuthology.orchestra.run.smithi052.stdout: ceph-selinux x86_64 2:19.0.0-1739.g1bc250c7.el9 ceph 25 k 2024-03-04T09:18:10.437 INFO:teuthology.orchestra.run.smithi052.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2024-03-04T09:18:10.437 INFO:teuthology.orchestra.run.smithi052.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2024-03-04T09:18:10.437 INFO:teuthology.orchestra.run.smithi052.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2024-03-04T09:18:10.437 INFO:teuthology.orchestra.run.smithi052.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2024-03-04T09:18:10.437 INFO:teuthology.orchestra.run.smithi052.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-03-04T09:18:10.437 INFO:teuthology.orchestra.run.smithi052.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-03-04T09:18:10.437 INFO:teuthology.orchestra.run.smithi052.stdout: jq x86_64 1.6-16.el9 CentOS-BaseOS 186 k 2024-03-04T09:18:10.437 INFO:teuthology.orchestra.run.smithi052.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-03-04T09:18:10.438 INFO:teuthology.orchestra.run.smithi052.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-03-04T09:18:10.438 INFO:teuthology.orchestra.run.smithi052.stdout: libcephsqlite x86_64 2:19.0.0-1739.g1bc250c7.el9 ceph 163 k 2024-03-04T09:18:10.438 INFO:teuthology.orchestra.run.smithi052.stdout: libgfortran x86_64 11.4.1-3.el9 CentOS-BaseOS 802 k 2024-03-04T09:18:10.438 INFO:teuthology.orchestra.run.smithi052.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-03-04T09:18:10.438 INFO:teuthology.orchestra.run.smithi052.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2024-03-04T09:18:10.438 INFO:teuthology.orchestra.run.smithi052.stdout: libquadmath x86_64 11.4.1-3.el9 CentOS-BaseOS 194 k 2024-03-04T09:18:10.438 INFO:teuthology.orchestra.run.smithi052.stdout: librabbitmq x86_64 0.11.0-7.el9 CentOS-AppStream 44 k 2024-03-04T09:18:10.438 INFO:teuthology.orchestra.run.smithi052.stdout: libradosstriper1 x86_64 2:19.0.0-1739.g1bc250c7.el9 ceph 501 k 2024-03-04T09:18:10.438 INFO:teuthology.orchestra.run.smithi052.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2024-03-04T09:18:10.438 INFO:teuthology.orchestra.run.smithi052.stdout: librgw2 x86_64 2:19.0.0-1739.g1bc250c7.el9 ceph 5.1 M 2024-03-04T09:18:10.438 INFO:teuthology.orchestra.run.smithi052.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-03-04T09:18:10.438 INFO:teuthology.orchestra.run.smithi052.stdout: libzstd-devel x86_64 1.5.1-2.el9 CentOS-AppStream 46 k 2024-03-04T09:18:10.438 INFO:teuthology.orchestra.run.smithi052.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2024-03-04T09:18:10.438 INFO:teuthology.orchestra.run.smithi052.stdout: lua x86_64 5.4.4-4.el9 CentOS-AppStream 187 k 2024-03-04T09:18:10.439 INFO:teuthology.orchestra.run.smithi052.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-03-04T09:18:10.439 INFO:teuthology.orchestra.run.smithi052.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-03-04T09:18:10.439 INFO:teuthology.orchestra.run.smithi052.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-BaseOS 217 k 2024-03-04T09:18:10.439 INFO:teuthology.orchestra.run.smithi052.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2024-03-04T09:18:10.439 INFO:teuthology.orchestra.run.smithi052.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2024-03-04T09:18:10.439 INFO:teuthology.orchestra.run.smithi052.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-03-04T09:18:10.439 INFO:teuthology.orchestra.run.smithi052.stdout: postgresql-private-libs x86_64 13.11-1.el9 CentOS-AppStream 134 k 2024-03-04T09:18:10.439 INFO:teuthology.orchestra.run.smithi052.stdout: protobuf x86_64 3.14.0-13.el9 CentOS-AppStream 1.0 M 2024-03-04T09:18:10.439 INFO:teuthology.orchestra.run.smithi052.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-03-04T09:18:10.439 INFO:teuthology.orchestra.run.smithi052.stdout: python3-asyncssh noarch 2.13.2-4.el9 epel 548 k 2024-03-04T09:18:10.439 INFO:teuthology.orchestra.run.smithi052.stdout: python3-attrs noarch 20.3.0-7.el9 CentOS-AppStream 78 k 2024-03-04T09:18:10.440 INFO:teuthology.orchestra.run.smithi052.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2024-03-04T09:18:10.440 INFO:teuthology.orchestra.run.smithi052.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-03-04T09:18:10.440 INFO:teuthology.orchestra.run.smithi052.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-03-04T09:18:10.440 INFO:teuthology.orchestra.run.smithi052.stdout: python3-ceph-argparse x86_64 2:19.0.0-1739.g1bc250c7.el9 ceph 45 k 2024-03-04T09:18:10.440 INFO:teuthology.orchestra.run.smithi052.stdout: python3-ceph-common x86_64 2:19.0.0-1739.g1bc250c7.el9 ceph 131 k 2024-03-04T09:18:10.440 INFO:teuthology.orchestra.run.smithi052.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-03-04T09:18:10.440 INFO:teuthology.orchestra.run.smithi052.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2024-03-04T09:18:10.440 INFO:teuthology.orchestra.run.smithi052.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-03-04T09:18:10.440 INFO:teuthology.orchestra.run.smithi052.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-03-04T09:18:10.440 INFO:teuthology.orchestra.run.smithi052.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-03-04T09:18:10.440 INFO:teuthology.orchestra.run.smithi052.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-03-04T09:18:10.440 INFO:teuthology.orchestra.run.smithi052.stdout: python3-cryptography x86_64 36.0.1-4.el9 CentOS-BaseOS 1.1 M 2024-03-04T09:18:10.440 INFO:teuthology.orchestra.run.smithi052.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2024-03-04T09:18:10.441 INFO:teuthology.orchestra.run.smithi052.stdout: python3-google-auth noarch 1:2.28.0-1.el9 epel 203 k 2024-03-04T09:18:10.441 INFO:teuthology.orchestra.run.smithi052.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-03-04T09:18:10.441 INFO:teuthology.orchestra.run.smithi052.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-03-04T09:18:10.441 INFO:teuthology.orchestra.run.smithi052.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-03-04T09:18:10.441 INFO:teuthology.orchestra.run.smithi052.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-03-04T09:18:10.441 INFO:teuthology.orchestra.run.smithi052.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-03-04T09:18:10.441 INFO:teuthology.orchestra.run.smithi052.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-03-04T09:18:10.441 INFO:teuthology.orchestra.run.smithi052.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-03-04T09:18:10.441 INFO:teuthology.orchestra.run.smithi052.stdout: python3-jinja2 noarch 2.11.3-5.el9 CentOS-AppStream 227 k 2024-03-04T09:18:10.441 INFO:teuthology.orchestra.run.smithi052.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2024-03-04T09:18:10.441 INFO:teuthology.orchestra.run.smithi052.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2024-03-04T09:18:10.441 INFO:teuthology.orchestra.run.smithi052.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-03-04T09:18:10.441 INFO:teuthology.orchestra.run.smithi052.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-03-04T09:18:10.441 INFO:teuthology.orchestra.run.smithi052.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2024-03-04T09:18:10.442 INFO:teuthology.orchestra.run.smithi052.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2024-03-04T09:18:10.442 INFO:teuthology.orchestra.run.smithi052.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-03-04T09:18:10.442 INFO:teuthology.orchestra.run.smithi052.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-03-04T09:18:10.442 INFO:teuthology.orchestra.run.smithi052.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2024-03-04T09:18:10.442 INFO:teuthology.orchestra.run.smithi052.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2024-03-04T09:18:10.442 INFO:teuthology.orchestra.run.smithi052.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2024-03-04T09:18:10.442 INFO:teuthology.orchestra.run.smithi052.stdout: python3-packaging noarch 20.9-5.el9 CentOS-AppStream 69 k 2024-03-04T09:18:10.442 INFO:teuthology.orchestra.run.smithi052.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-03-04T09:18:10.442 INFO:teuthology.orchestra.run.smithi052.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2024-03-04T09:18:10.442 INFO:teuthology.orchestra.run.smithi052.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-03-04T09:18:10.442 INFO:teuthology.orchestra.run.smithi052.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2024-03-04T09:18:10.442 INFO:teuthology.orchestra.run.smithi052.stdout: python3-protobuf noarch 3.14.0-13.el9 CentOS-AppStream 238 k 2024-03-04T09:18:10.442 INFO:teuthology.orchestra.run.smithi052.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-03-04T09:18:10.442 INFO:teuthology.orchestra.run.smithi052.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2024-03-04T09:18:10.443 INFO:teuthology.orchestra.run.smithi052.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2024-03-04T09:18:10.443 INFO:teuthology.orchestra.run.smithi052.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2024-03-04T09:18:10.443 INFO:teuthology.orchestra.run.smithi052.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-03-04T09:18:10.443 INFO:teuthology.orchestra.run.smithi052.stdout: python3-pytz noarch 2021.1-5.el9 CentOS-AppStream 47 k 2024-03-04T09:18:10.443 INFO:teuthology.orchestra.run.smithi052.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-03-04T09:18:10.443 INFO:teuthology.orchestra.run.smithi052.stdout: python3-requests noarch 2.25.1-8.el9 CentOS-BaseOS 113 k 2024-03-04T09:18:10.443 INFO:teuthology.orchestra.run.smithi052.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2024-03-04T09:18:10.443 INFO:teuthology.orchestra.run.smithi052.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-03-04T09:18:10.443 INFO:teuthology.orchestra.run.smithi052.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-03-04T09:18:10.443 INFO:teuthology.orchestra.run.smithi052.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2024-03-04T09:18:10.443 INFO:teuthology.orchestra.run.smithi052.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-03-04T09:18:10.443 INFO:teuthology.orchestra.run.smithi052.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2024-03-04T09:18:10.443 INFO:teuthology.orchestra.run.smithi052.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-03-04T09:18:10.443 INFO:teuthology.orchestra.run.smithi052.stdout: python3-urllib3 noarch 1.26.5-5.el9 CentOS-BaseOS 187 k 2024-03-04T09:18:10.443 INFO:teuthology.orchestra.run.smithi052.stdout: python3-wcwidth noarch 0.2.5-8.el9 CentOS-AppStream 41 k 2024-03-04T09:18:10.444 INFO:teuthology.orchestra.run.smithi052.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-03-04T09:18:10.444 INFO:teuthology.orchestra.run.smithi052.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-03-04T09:18:10.444 INFO:teuthology.orchestra.run.smithi052.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-03-04T09:18:10.444 INFO:teuthology.orchestra.run.smithi052.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-03-04T09:18:10.444 INFO:teuthology.orchestra.run.smithi052.stdout: qatlib x86_64 23.11.0-1.el9 CentOS-AppStream 215 k 2024-03-04T09:18:10.444 INFO:teuthology.orchestra.run.smithi052.stdout: qatzip-libs x86_64 1.1.2-2.el9 CentOS-AppStream 43 k 2024-03-04T09:18:10.444 INFO:teuthology.orchestra.run.smithi052.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-03-04T09:18:10.445 INFO:teuthology.orchestra.run.smithi052.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2024-03-04T09:18:10.445 INFO:teuthology.orchestra.run.smithi052.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-03-04T09:18:10.445 INFO:teuthology.orchestra.run.smithi052.stdout: uuid x86_64 1.6.2-55.el9 CentOS-AppStream 56 k 2024-03-04T09:18:10.445 INFO:teuthology.orchestra.run.smithi052.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2024-03-04T09:18:10.445 INFO:teuthology.orchestra.run.smithi052.stdout: zlib-devel x86_64 1.2.11-40.el9 CentOS-AppStream 44 k 2024-03-04T09:18:10.445 INFO:teuthology.orchestra.run.smithi052.stdout:Installing weak dependencies: 2024-03-04T09:18:10.445 INFO:teuthology.orchestra.run.smithi052.stdout: qatlib-service x86_64 23.11.0-1.el9 CentOS-AppStream 34 k 2024-03-04T09:18:10.445 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-03-04T09:18:10.445 INFO:teuthology.orchestra.run.smithi052.stdout:Transaction Summary 2024-03-04T09:18:10.445 INFO:teuthology.orchestra.run.smithi052.stdout:============================================================================================ 2024-03-04T09:18:10.445 INFO:teuthology.orchestra.run.smithi052.stdout:Install 150 Packages 2024-03-04T09:18:10.445 INFO:teuthology.orchestra.run.smithi052.stdout:Upgrade 26 Packages 2024-03-04T09:18:10.445 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-03-04T09:18:10.446 INFO:teuthology.orchestra.run.smithi191.stdout:(4/176): ceph-mds-19.0.0-1739.g1bc250c7.el9.x86 3.5 MB/s | 2.3 MB 00:00 2024-03-04T09:18:10.446 INFO:teuthology.orchestra.run.smithi052.stdout:Total download size: 224 M 2024-03-04T09:18:10.447 INFO:teuthology.orchestra.run.smithi052.stdout:Downloading Packages: 2024-03-04T09:18:10.469 INFO:teuthology.orchestra.run.smithi106.stdout:(45/176): ceph-test-19.0.0-1739.g1bc250c7.el9.x 13 MB/s | 47 MB 00:03 2024-03-04T09:18:10.528 INFO:teuthology.orchestra.run.smithi106.stdout:(46/176): flexiblas-netlib-3.0.4-8.el9.x86_64.r 5.2 MB/s | 3.0 MB 00:00 2024-03-04T09:18:10.545 INFO:teuthology.orchestra.run.smithi106.stdout:(47/176): flexiblas-openblas-openmp-3.0.4-8.el9 31 kB/s | 17 kB 00:00 2024-03-04T09:18:10.561 INFO:teuthology.orchestra.run.smithi106.stdout:(48/176): libaio-devel-0.3.111-13.el9.x86_64.rp 112 kB/s | 10 kB 00:00 2024-03-04T09:18:10.587 INFO:teuthology.orchestra.run.smithi106.stdout:(49/176): librabbitmq-0.11.0-7.el9.x86_64.rpm 1.0 MB/s | 44 kB 00:00 2024-03-04T09:18:10.620 INFO:teuthology.orchestra.run.smithi106.stdout:(50/176): libpmemobj-1.12.1-1.el9.x86_64.rpm 2.1 MB/s | 159 kB 00:00 2024-03-04T09:18:10.645 INFO:teuthology.orchestra.run.smithi106.stdout:(51/176): librdkafka-1.6.1-102.el9.x86_64.rpm 7.7 MB/s | 661 kB 00:00 2024-03-04T09:18:10.679 INFO:teuthology.orchestra.run.smithi106.stdout:(52/176): libtool-2.4.6-45.el9.x86_64.rpm 6.2 MB/s | 578 kB 00:00 2024-03-04T09:18:10.696 INFO:teuthology.orchestra.run.smithi106.stdout:(53/176): libuuid-devel-2.37.4-18.el9.x86_64.rp 495 kB/s | 24 kB 00:00 2024-03-04T09:18:10.716 INFO:teuthology.orchestra.run.smithi106.stdout:(54/176): libzstd-devel-1.5.1-2.el9.x86_64.rpm 703 kB/s | 46 kB 00:00 2024-03-04T09:18:10.733 INFO:teuthology.orchestra.run.smithi191.stdout:(5/176): ceph-base-19.0.0-1739.g1bc250c7.el9.x8 2.2 MB/s | 5.3 MB 00:02 2024-03-04T09:18:10.747 INFO:teuthology.orchestra.run.smithi106.stdout:(55/176): lttng-ust-2.12.0-6.el9.x86_64.rpm 4.1 MB/s | 282 kB 00:00 2024-03-04T09:18:10.763 INFO:teuthology.orchestra.run.smithi106.stdout:(56/176): openblas-0.3.21-2.el9.x86_64.rpm 693 kB/s | 34 kB 00:00 2024-03-04T09:18:10.775 INFO:teuthology.orchestra.run.smithi191.stdout:(6/176): ceph-mgr-19.0.0-1739.g1bc250c7.el9.x86 4.4 MB/s | 1.5 MB 00:00 2024-03-04T09:18:10.788 INFO:teuthology.orchestra.run.smithi106.stdout:(57/176): lua-5.4.4-4.el9.x86_64.rpm 2.4 MB/s | 187 kB 00:00 2024-03-04T09:18:10.872 INFO:teuthology.orchestra.run.smithi106.stdout:(58/176): openblas-openmp-0.3.21-2.el9.x86_64.r 38 MB/s | 4.7 MB 00:00 2024-03-04T09:18:10.923 INFO:teuthology.orchestra.run.smithi106.stdout:(59/176): openssl-devel-3.0.7-27.el9.x86_64.rpm 19 MB/s | 3.0 MB 00:00 2024-03-04T09:18:10.965 INFO:teuthology.orchestra.run.smithi106.stdout:(60/176): postgresql-13.11-1.el9.x86_64.rpm 8.8 MB/s | 1.5 MB 00:00 2024-03-04T09:18:10.989 INFO:teuthology.orchestra.run.smithi106.stdout:(61/176): postgresql-contrib-13.11-1.el9.x86_64 6.9 MB/s | 811 kB 00:00 2024-03-04T09:18:11.015 INFO:teuthology.orchestra.run.smithi106.stdout:(62/176): postgresql-private-libs-13.11-1.el9.x 1.5 MB/s | 134 kB 00:00 2024-03-04T09:18:11.057 INFO:teuthology.orchestra.run.smithi106.stdout:(63/176): protobuf-3.14.0-13.el9.x86_64.rpm 15 MB/s | 1.0 MB 00:00 2024-03-04T09:18:11.157 INFO:teuthology.orchestra.run.smithi106.stdout:(64/176): postgresql-server-13.11-1.el9.x86_64. 34 MB/s | 5.7 MB 00:00 2024-03-04T09:18:11.182 INFO:teuthology.orchestra.run.smithi106.stdout:(65/176): python3-attrs-20.3.0-7.el9.noarch.rpm 464 kB/s | 78 kB 00:00 2024-03-04T09:18:11.275 INFO:teuthology.orchestra.run.smithi106.stdout:(66/176): python3-babel-2.9.1-2.el9.noarch.rpm 27 MB/s | 5.8 MB 00:00 2024-03-04T09:18:11.292 INFO:teuthology.orchestra.run.smithi191.stdout:(7/176): ceph-mon-19.0.0-1739.g1bc250c7.el9.x86 8.4 MB/s | 4.7 MB 00:00 2024-03-04T09:18:11.299 INFO:teuthology.orchestra.run.smithi106.stdout:(67/176): python3-jinja2-2.11.3-5.el9.noarch.rp 1.6 MB/s | 227 kB 00:00 2024-03-04T09:18:11.328 INFO:teuthology.orchestra.run.smithi106.stdout:(68/176): python3-jsonpatch-1.21-16.el9.noarch. 175 kB/s | 24 kB 00:00 2024-03-04T09:18:11.350 INFO:teuthology.orchestra.run.smithi106.stdout:(69/176): python3-jsonpointer-2.0-4.el9.noarch. 242 kB/s | 18 kB 00:00 2024-03-04T09:18:11.375 INFO:teuthology.orchestra.run.smithi106.stdout:(70/176): python3-markupsafe-1.1.1-12.el9.x86_6 633 kB/s | 32 kB 00:00 2024-03-04T09:18:11.401 INFO:teuthology.orchestra.run.smithi106.stdout:(71/176): python3-mako-1.1.4-6.el9.noarch.rpm 1.9 MB/s | 144 kB 00:00 2024-03-04T09:18:11.456 INFO:teuthology.orchestra.run.smithi052.stdout:(1/176): ceph-19.0.0-1739.g1bc250c7.el9.x86_64. 24 kB/s | 6.6 kB 00:00 2024-03-04T09:18:11.484 INFO:teuthology.orchestra.run.smithi106.stdout:(72/176): python3-numpy-1.20.1-5.el9.x86_64.rpm 36 MB/s | 4.8 MB 00:00 2024-03-04T09:18:11.509 INFO:teuthology.orchestra.run.smithi106.stdout:(73/176): python3-numpy-f2py-1.20.1-5.el9.x86_6 1.6 MB/s | 217 kB 00:00 2024-03-04T09:18:11.535 INFO:teuthology.orchestra.run.smithi106.stdout:(74/176): python3-oauthlib-3.1.1-5.el9.noarch.r 1.3 MB/s | 174 kB 00:00 2024-03-04T09:18:11.560 INFO:teuthology.orchestra.run.smithi106.stdout:(75/176): python3-packaging-20.9-5.el9.noarch.r 920 kB/s | 69 kB 00:00 2024-03-04T09:18:11.577 INFO:teuthology.orchestra.run.smithi106.stdout:(76/176): python3-prettytable-0.7.2-27.el9.noar 990 kB/s | 41 kB 00:00 2024-03-04T09:18:11.602 INFO:teuthology.orchestra.run.smithi106.stdout:(77/176): python3-protobuf-3.14.0-13.el9.noarch 3.5 MB/s | 238 kB 00:00 2024-03-04T09:18:11.627 INFO:teuthology.orchestra.run.smithi106.stdout:(78/176): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.9 MB/s | 132 kB 00:00 2024-03-04T09:18:11.644 INFO:teuthology.orchestra.run.smithi106.stdout:(79/176): python3-pytz-2021.1-5.el9.noarch.rpm 1.1 MB/s | 47 kB 00:00 2024-03-04T09:18:11.669 INFO:teuthology.orchestra.run.smithi106.stdout:(80/176): python3-pyasn1-modules-0.4.8-6.el9.no 3.1 MB/s | 211 kB 00:00 2024-03-04T09:18:11.686 INFO:teuthology.orchestra.run.smithi106.stdout:(81/176): python3-requests-oauthlib-1.3.0-12.el 730 kB/s | 43 kB 00:00 2024-03-04T09:18:12.113 INFO:teuthology.orchestra.run.smithi106.stdout:(82/176): python3-scipy-1.6.2-8.el9.x86_64.rpm 33 MB/s | 16 MB 00:00 2024-03-04T09:18:12.204 INFO:teuthology.orchestra.run.smithi106.stdout:(83/176): python3-toml-0.10.2-6.el9.noarch.rpm 75 kB/s | 39 kB 00:00 2024-03-04T09:18:12.221 INFO:teuthology.orchestra.run.smithi106.stdout:(84/176): python3-wcwidth-0.2.5-8.el9.noarch.rp 77 kB/s | 41 kB 00:00 2024-03-04T09:18:12.246 INFO:teuthology.orchestra.run.smithi106.stdout:(85/176): qatlib-23.11.0-1.el9.x86_64.rpm 1.6 MB/s | 215 kB 00:00 2024-03-04T09:18:12.262 INFO:teuthology.orchestra.run.smithi106.stdout:(86/176): qatzip-libs-1.1.2-2.el9.x86_64.rpm 1.0 MB/s | 43 kB 00:00 2024-03-04T09:18:12.280 INFO:teuthology.orchestra.run.smithi106.stdout:(87/176): qatlib-service-23.11.0-1.el9.x86_64.r 579 kB/s | 34 kB 00:00 2024-03-04T09:18:12.284 INFO:teuthology.orchestra.run.smithi191.stdout:(8/176): ceph-radosgw-19.0.0-1739.g1bc250c7.el9 9.9 MB/s | 9.8 MB 00:00 2024-03-04T09:18:12.305 INFO:teuthology.orchestra.run.smithi106.stdout:(88/176): socat-1.7.4.1-5.el9.x86_64.rpm 5.0 MB/s | 300 kB 00:00 2024-03-04T09:18:12.305 INFO:teuthology.orchestra.run.smithi052.stdout:(2/176): ceph-fuse-19.0.0-1739.g1bc250c7.el9.x8 1.0 MB/s | 888 kB 00:00 2024-03-04T09:18:12.322 INFO:teuthology.orchestra.run.smithi106.stdout:(89/176): uuid-1.6.2-55.el9.x86_64.rpm 967 kB/s | 56 kB 00:00 2024-03-04T09:18:12.343 INFO:teuthology.orchestra.run.smithi106.stdout:(90/176): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.6 MB/s | 63 kB 00:00 2024-03-04T09:18:12.368 INFO:teuthology.orchestra.run.smithi106.stdout:(91/176): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 1.3 MB/s | 87 kB 00:00 2024-03-04T09:18:12.394 INFO:teuthology.orchestra.run.smithi106.stdout:(92/176): zlib-devel-1.2.11-40.el9.x86_64.rpm 617 kB/s | 44 kB 00:00 2024-03-04T09:18:12.397 INFO:teuthology.orchestra.run.smithi052.stdout:(3/176): ceph-immutable-object-cache-19.0.0-173 1.6 MB/s | 145 kB 00:00 2024-03-04T09:18:12.419 INFO:teuthology.orchestra.run.smithi106.stdout:(93/176): fuse-2.9.9-15.el9.x86_64.rpm 1.0 MB/s | 79 kB 00:00 2024-03-04T09:18:12.445 INFO:teuthology.orchestra.run.smithi106.stdout:(94/176): libgfortran-11.4.1-3.el9.x86_64.rpm 15 MB/s | 802 kB 00:00 2024-03-04T09:18:12.470 INFO:teuthology.orchestra.run.smithi106.stdout:(95/176): jq-1.6-16.el9.x86_64.rpm 2.4 MB/s | 186 kB 00:00 2024-03-04T09:18:12.495 INFO:teuthology.orchestra.run.smithi106.stdout:(96/176): libquadmath-11.4.1-3.el9.x86_64.rpm 2.5 MB/s | 194 kB 00:00 2024-03-04T09:18:12.520 INFO:teuthology.orchestra.run.smithi106.stdout:(97/176): oniguruma-6.9.6-1.el9.5.x86_64.rpm 2.8 MB/s | 217 kB 00:00 2024-03-04T09:18:12.526 INFO:teuthology.orchestra.run.smithi191.stdout:(9/176): ceph-osd-19.0.0-1739.g1bc250c7.el9.x86 9.6 MB/s | 17 MB 00:01 2024-03-04T09:18:12.546 INFO:teuthology.orchestra.run.smithi106.stdout:(98/176): python3-cffi-1.14.5-5.el9.x86_64.rpm 4.8 MB/s | 241 kB 00:00 2024-03-04T09:18:12.551 INFO:teuthology.orchestra.run.smithi191.stdout:(10/176): ceph-selinux-19.0.0-1739.g1bc250c7.el 94 kB/s | 25 kB 00:00 2024-03-04T09:18:12.588 INFO:teuthology.orchestra.run.smithi106.stdout:(99/176): python3-cryptography-36.0.1-4.el9.x86 13 MB/s | 1.1 MB 00:00 2024-03-04T09:18:12.613 INFO:teuthology.orchestra.run.smithi106.stdout:(100/176): python3-ply-3.11-14.el9.noarch.rpm 1.1 MB/s | 103 kB 00:00 2024-03-04T09:18:12.638 INFO:teuthology.orchestra.run.smithi106.stdout:(101/176): python3-pycparser-2.20-6.el9.noarch. 1.3 MB/s | 124 kB 00:00 2024-03-04T09:18:12.660 INFO:teuthology.orchestra.run.smithi191.stdout:(11/176): cephfs-mirror-19.0.0-1739.g1bc250c7.e 2.0 MB/s | 220 kB 00:00 2024-03-04T09:18:12.663 INFO:teuthology.orchestra.run.smithi106.stdout:(102/176): python3-requests-2.25.1-8.el9.noarch 2.2 MB/s | 113 kB 00:00 2024-03-04T09:18:12.688 INFO:teuthology.orchestra.run.smithi106.stdout:(103/176): python3-urllib3-1.26.5-5.el9.noarch. 2.4 MB/s | 187 kB 00:00 2024-03-04T09:18:12.761 INFO:teuthology.orchestra.run.smithi191.stdout:(12/176): cephfs-shell-19.0.0-1739.g1bc250c7.el 270 kB/s | 27 kB 00:00 2024-03-04T09:18:12.764 INFO:teuthology.orchestra.run.smithi052.stdout:(4/176): ceph-base-19.0.0-1739.g1bc250c7.el9.x8 3.4 MB/s | 5.3 MB 00:01 2024-03-04T09:18:12.844 INFO:teuthology.orchestra.run.smithi191.stdout:(13/176): libcephfs-devel-19.0.0-1739.g1bc250c7 375 kB/s | 31 kB 00:00 2024-03-04T09:18:12.922 INFO:teuthology.orchestra.run.smithi106.stdout:(104/176): python3-devel-3.9.17-2.el9.x86_64.rp 866 kB/s | 245 kB 00:00 2024-03-04T09:18:13.139 INFO:teuthology.orchestra.run.smithi052.stdout:(5/176): ceph-mgr-19.0.0-1739.g1bc250c7.el9.x86 4.0 MB/s | 1.5 MB 00:00 2024-03-04T09:18:13.397 INFO:teuthology.orchestra.run.smithi106.stdout:(105/176): lua-devel-5.4.4-4.el9.x86_64.rpm 31 kB/s | 22 kB 00:00 2024-03-04T09:18:13.423 INFO:teuthology.orchestra.run.smithi106.stdout:(106/176): abseil-cpp-20211102.0-3.el9.x86_64.r 1.1 MB/s | 548 kB 00:00 2024-03-04T09:18:13.448 INFO:teuthology.orchestra.run.smithi052.stdout:(6/176): ceph-mds-19.0.0-1739.g1bc250c7.el9.x86 2.2 MB/s | 2.3 MB 00:01 2024-03-04T09:18:13.498 INFO:teuthology.orchestra.run.smithi106.stdout:(107/176): grpc-data-1.46.7-10.el9.noarch.rpm 260 kB/s | 19 kB 00:00 2024-03-04T09:18:13.807 INFO:teuthology.orchestra.run.smithi106.stdout:(108/176): libarrow-9.0.0-10.el9.x86_64.rpm 14 MB/s | 4.4 MB 00:00 2024-03-04T09:18:13.832 INFO:teuthology.orchestra.run.smithi106.stdout:(109/176): gperftools-libs-2.9.1-2.el9.x86_64.r 712 kB/s | 309 kB 00:00 2024-03-04T09:18:13.882 INFO:teuthology.orchestra.run.smithi106.stdout:(110/176): libarrow-doc-9.0.0-10.el9.noarch.rpm 346 kB/s | 26 kB 00:00 2024-03-04T09:18:13.907 INFO:teuthology.orchestra.run.smithi106.stdout:(111/176): liboath-2.6.7-2.el9.x86_64.rpm 645 kB/s | 49 kB 00:00 2024-03-04T09:18:13.966 INFO:teuthology.orchestra.run.smithi106.stdout:(112/176): libunwind-1.6.2-1.el9.x86_64.rpm 806 kB/s | 67 kB 00:00 2024-03-04T09:18:13.999 INFO:teuthology.orchestra.run.smithi106.stdout:(113/176): luarocks-3.9.2-1.el9.noarch.rpm 1.6 MB/s | 151 kB 00:00 2024-03-04T09:18:14.028 INFO:teuthology.orchestra.run.smithi191.stdout:(14/176): libcephfs2-19.0.0-1739.g1bc250c7.el9. 631 kB/s | 747 kB 00:01 2024-03-04T09:18:14.058 INFO:teuthology.orchestra.run.smithi106.stdout:(114/176): parquet-libs-9.0.0-10.el9.x86_64.rpm 8.9 MB/s | 838 kB 00:00 2024-03-04T09:18:14.133 INFO:teuthology.orchestra.run.smithi106.stdout:(115/176): python3-bcrypt-3.2.2-1.el9.x86_64.rp 579 kB/s | 43 kB 00:00 2024-03-04T09:18:14.145 INFO:teuthology.orchestra.run.smithi191.stdout:(15/176): libcephsqlite-19.0.0-1739.g1bc250c7.e 1.4 MB/s | 163 kB 00:00 2024-03-04T09:18:14.175 INFO:teuthology.orchestra.run.smithi106.stdout:(116/176): python3-asyncssh-2.13.2-4.el9.noarch 3.0 MB/s | 548 kB 00:00 2024-03-04T09:18:14.217 INFO:teuthology.orchestra.run.smithi106.stdout:(117/176): python3-cachetools-4.2.4-1.el9.noarc 385 kB/s | 32 kB 00:00 2024-03-04T09:18:14.240 INFO:teuthology.orchestra.run.smithi052.stdout:(7/176): ceph-mon-19.0.0-1739.g1bc250c7.el9.x86 4.3 MB/s | 4.7 MB 00:01 2024-03-04T09:18:14.253 INFO:teuthology.orchestra.run.smithi191.stdout:(16/176): librados-devel-19.0.0-1739.g1bc250c7. 1.1 MB/s | 127 kB 00:00 2024-03-04T09:18:14.259 INFO:teuthology.orchestra.run.smithi106.stdout:(118/176): python3-certifi-2021.10.8-2.el9.noar 179 kB/s | 15 kB 00:00 2024-03-04T09:18:14.301 INFO:teuthology.orchestra.run.smithi106.stdout:(119/176): python3-cheroot-8.6.0-4.el9.noarch.r 2.0 MB/s | 172 kB 00:00 2024-03-04T09:18:14.368 INFO:teuthology.orchestra.run.smithi106.stdout:(120/176): python3-cherrypy-18.6.1-2.el9.noarch 3.2 MB/s | 358 kB 00:00 2024-03-04T09:18:14.413 INFO:teuthology.orchestra.run.smithi106.stdout:(121/176): protobuf-compiler-3.14.0-13.el9.x86_ 502 kB/s | 863 kB 00:01 2024-03-04T09:18:14.435 INFO:teuthology.orchestra.run.smithi106.stdout:(122/176): python3-cmd2-2.4.2-2.el9.noarch.rpm 2.2 MB/s | 303 kB 00:00 2024-03-04T09:18:14.460 INFO:teuthology.orchestra.run.smithi106.stdout:(123/176): python3-colorama-0.4.4-12.el9.noarch 384 kB/s | 35 kB 00:00 2024-03-04T09:18:14.537 INFO:teuthology.orchestra.run.smithi191.stdout:(17/176): ceph-common-19.0.0-1739.g1bc250c7.el9 3.3 MB/s | 20 MB 00:06 2024-03-04T09:18:14.561 INFO:teuthology.orchestra.run.smithi106.stdout:(124/176): python3-grpcio-tools-1.46.7-10.el9.x 1.4 MB/s | 144 kB 00:00 2024-03-04T09:18:14.611 INFO:teuthology.orchestra.run.smithi106.stdout:(125/176): python3-grpcio-1.46.7-10.el9.x86_64. 12 MB/s | 2.0 MB 00:00 2024-03-04T09:18:14.645 INFO:teuthology.orchestra.run.smithi106.stdout:(126/176): python3-jaraco-8.2.1-3.el9.noarch.rp 128 kB/s | 11 kB 00:00 2024-03-04T09:18:14.687 INFO:teuthology.orchestra.run.smithi106.stdout:(127/176): python3-jaraco-classes-3.2.1-5.el9.n 235 kB/s | 18 kB 00:00 2024-03-04T09:18:14.729 INFO:teuthology.orchestra.run.smithi106.stdout:(128/176): python3-jaraco-collections-3.0.0-8.e 277 kB/s | 23 kB 00:00 2024-03-04T09:18:14.771 INFO:teuthology.orchestra.run.smithi106.stdout:(129/176): python3-jaraco-functools-3.5.0-2.el9 232 kB/s | 19 kB 00:00 2024-03-04T09:18:14.796 INFO:teuthology.orchestra.run.smithi106.stdout:(130/176): python3-google-auth-2.28.0-1.el9.noa 526 kB/s | 203 kB 00:00 2024-03-04T09:18:14.821 INFO:teuthology.orchestra.run.smithi106.stdout:(131/176): python3-jaraco-text-3.2.0-6.el9.noar 213 kB/s | 20 kB 00:00 2024-03-04T09:18:14.863 INFO:teuthology.orchestra.run.smithi106.stdout:(132/176): python3-kubernetes-26.1.0-2.el9.noar 11 MB/s | 1.0 MB 00:00 2024-03-04T09:18:14.879 INFO:teuthology.orchestra.run.smithi191.stdout:(18/176): libradosstriper1-19.0.0-1739.g1bc250c 802 kB/s | 501 kB 00:00 2024-03-04T09:18:14.888 INFO:teuthology.orchestra.run.smithi106.stdout:(133/176): python3-logutils-0.3.5-21.el9.noarch 503 kB/s | 46 kB 00:00 2024-03-04T09:18:14.914 INFO:teuthology.orchestra.run.smithi106.stdout:(134/176): python3-more-itertools-8.12.0-2.el9. 855 kB/s | 79 kB 00:00 2024-03-04T09:18:14.947 INFO:teuthology.orchestra.run.smithi106.stdout:(135/176): python3-natsort-7.1.1-5.el9.noarch.r 688 kB/s | 58 kB 00:00 2024-03-04T09:18:14.963 INFO:teuthology.orchestra.run.smithi191.stdout:(19/176): python3-ceph-argparse-19.0.0-1739.g1b 542 kB/s | 45 kB 00:00 2024-03-04T09:18:14.998 INFO:teuthology.orchestra.run.smithi106.stdout:(136/176): python3-portend-3.1.0-2.el9.noarch.r 196 kB/s | 16 kB 00:00 2024-03-04T09:18:15.031 INFO:teuthology.orchestra.run.smithi106.stdout:(137/176): python3-pecan-1.4.2-3.el9.noarch.rpm 1.9 MB/s | 272 kB 00:00 2024-03-04T09:18:15.048 INFO:teuthology.orchestra.run.smithi106.stdout:(138/176): python3-pyOpenSSL-21.0.0-1.el9.noarc 896 kB/s | 90 kB 00:00 2024-03-04T09:18:15.063 INFO:teuthology.orchestra.run.smithi191.stdout:(20/176): python3-ceph-common-19.0.0-1739.g1bc2 1.3 MB/s | 131 kB 00:00 2024-03-04T09:18:15.073 INFO:teuthology.orchestra.run.smithi106.stdout:(139/176): python3-pyperclip-1.8.2-2.el9.noarch 374 kB/s | 28 kB 00:00 2024-03-04T09:18:15.124 INFO:teuthology.orchestra.run.smithi106.stdout:(140/176): python3-repoze-lru-0.7-16.el9.noarch 408 kB/s | 31 kB 00:00 2024-03-04T09:18:15.150 INFO:teuthology.orchestra.run.smithi106.stdout:(141/176): python3-routes-2.5.1-5.el9.noarch.rp 1.8 MB/s | 188 kB 00:00 2024-03-04T09:18:15.166 INFO:teuthology.orchestra.run.smithi106.stdout:(142/176): python3-rsa-4.9-2.el9.noarch.rpm 642 kB/s | 59 kB 00:00 2024-03-04T09:18:15.181 INFO:teuthology.orchestra.run.smithi191.stdout:(21/176): python3-cephfs-19.0.0-1739.g1bc250c7. 1.4 MB/s | 162 kB 00:00 2024-03-04T09:18:15.208 INFO:teuthology.orchestra.run.smithi106.stdout:(143/176): python3-tempora-5.0.0-2.el9.noarch.r 427 kB/s | 36 kB 00:00 2024-03-04T09:18:15.233 INFO:teuthology.orchestra.run.smithi106.stdout:(144/176): python3-typing-extensions-4.4.0-2.el 608 kB/s | 51 kB 00:00 2024-03-04T09:18:15.258 INFO:teuthology.orchestra.run.smithi106.stdout:(145/176): python3-webob-1.8.7-6.el9.noarch.rpm 2.4 MB/s | 230 kB 00:00 2024-03-04T09:18:15.302 INFO:teuthology.orchestra.run.smithi106.stdout:(146/176): python3-websocket-client-1.2.3-2.el9 960 kB/s | 90 kB 00:00 2024-03-04T09:18:15.306 INFO:teuthology.orchestra.run.smithi191.stdout:(22/176): python3-rados-19.0.0-1739.g1bc250c7.e 2.5 MB/s | 321 kB 00:00 2024-03-04T09:18:15.335 INFO:teuthology.orchestra.run.smithi106.stdout:(147/176): python3-zc-lockfile-2.0-10.el9.noarc 261 kB/s | 20 kB 00:00 2024-03-04T09:18:15.360 INFO:teuthology.orchestra.run.smithi106.stdout:(148/176): python3-werkzeug-2.0.3-3.el9.1.noarc 3.3 MB/s | 427 kB 00:00 2024-03-04T09:18:15.386 INFO:teuthology.orchestra.run.smithi106.stdout:(149/176): re2-20211101-3.el9.x86_64.rpm 2.2 MB/s | 192 kB 00:00 2024-03-04T09:18:15.423 INFO:teuthology.orchestra.run.smithi191.stdout:(23/176): python3-rbd-19.0.0-1739.g1bc250c7.el9 2.5 MB/s | 298 kB 00:00 2024-03-04T09:18:15.523 INFO:teuthology.orchestra.run.smithi191.stdout:(24/176): python3-rgw-19.0.0-1739.g1bc250c7.el9 990 kB/s | 99 kB 00:00 2024-03-04T09:18:15.536 INFO:teuthology.orchestra.run.smithi106.stdout:(150/176): thrift-0.15.0-2.el9.x86_64.rpm 7.9 MB/s | 1.6 MB 00:00 2024-03-04T09:18:15.570 INFO:teuthology.orchestra.run.smithi106.stdout:(151/176): libacl-devel-2.3.1-4.el9.x86_64.rpm 2.0 MB/s | 68 kB 00:00 2024-03-04T09:18:15.632 INFO:teuthology.orchestra.run.smithi191.stdout:(25/176): rbd-fuse-19.0.0-1739.g1bc250c7.el9.x8 788 kB/s | 85 kB 00:00 2024-03-04T09:18:15.653 INFO:teuthology.orchestra.run.smithi106.stdout:(152/176): NetworkManager-1.46.0-1.el9.x86_64.r 27 MB/s | 2.3 MB 00:00 2024-03-04T09:18:15.720 INFO:teuthology.orchestra.run.smithi106.stdout:(153/176): NetworkManager-libnm-1.46.0-1.el9.x8 27 MB/s | 1.8 MB 00:00 2024-03-04T09:18:15.745 INFO:teuthology.orchestra.run.smithi106.stdout:(154/176): NetworkManager-team-1.46.0-1.el9.x86 1.6 MB/s | 41 kB 00:00 2024-03-04T09:18:15.789 INFO:teuthology.orchestra.run.smithi106.stdout:(155/176): NetworkManager-tui-1.46.0-1.el9.x86_ 5.5 MB/s | 247 kB 00:00 2024-03-04T09:18:15.823 INFO:teuthology.orchestra.run.smithi106.stdout:(156/176): acl-2.3.1-4.el9.x86_64.rpm 2.0 MB/s | 69 kB 00:00 2024-03-04T09:18:15.857 INFO:teuthology.orchestra.run.smithi106.stdout:(157/176): elfutils-debuginfod-client-0.190-2.e 1.0 MB/s | 35 kB 00:00 2024-03-04T09:18:15.898 INFO:teuthology.orchestra.run.smithi106.stdout:(158/176): elfutils-libelf-0.190-2.el9.x86_64.r 4.5 MB/s | 191 kB 00:00 2024-03-04T09:18:15.940 INFO:teuthology.orchestra.run.smithi106.stdout:(159/176): elfutils-libs-0.190-2.el9.x86_64.rpm 6.0 MB/s | 254 kB 00:00 2024-03-04T09:18:15.999 INFO:teuthology.orchestra.run.smithi106.stdout:(160/176): iproute-6.7.0-2.el9.x86_64.rpm 13 MB/s | 802 kB 00:00 2024-03-04T09:18:16.041 INFO:teuthology.orchestra.run.smithi106.stdout:(161/176): iproute-tc-6.7.0-2.el9.x86_64.rpm 10 MB/s | 433 kB 00:00 2024-03-04T09:18:16.074 INFO:teuthology.orchestra.run.smithi106.stdout:(162/176): libacl-2.3.1-4.el9.x86_64.rpm 660 kB/s | 22 kB 00:00 2024-03-04T09:18:16.108 INFO:teuthology.orchestra.run.smithi106.stdout:(163/176): libblkid-2.37.4-18.el9.x86_64.rpm 3.2 MB/s | 107 kB 00:00 2024-03-04T09:18:16.142 INFO:teuthology.orchestra.run.smithi106.stdout:(164/176): libfdisk-2.37.4-18.el9.x86_64.rpm 4.5 MB/s | 154 kB 00:00 2024-03-04T09:18:16.175 INFO:teuthology.orchestra.run.smithi106.stdout:(165/176): libmount-2.37.4-18.el9.x86_64.rpm 4.0 MB/s | 135 kB 00:00 2024-03-04T09:18:16.209 INFO:teuthology.orchestra.run.smithi106.stdout:(166/176): libsmartcols-2.37.4-18.el9.x86_64.rp 1.9 MB/s | 63 kB 00:00 2024-03-04T09:18:16.243 INFO:teuthology.orchestra.run.smithi106.stdout:(167/176): libuuid-2.37.4-18.el9.x86_64.rpm 827 kB/s | 28 kB 00:00 2024-03-04T09:18:16.249 INFO:teuthology.orchestra.run.smithi191.stdout:(26/176): rbd-mirror-19.0.0-1739.g1bc250c7.el9. 5.0 MB/s | 3.1 MB 00:00 2024-03-04T09:18:16.301 INFO:teuthology.orchestra.run.smithi106.stdout:(168/176): openssl-3.0.7-27.el9.x86_64.rpm 20 MB/s | 1.2 MB 00:00 2024-03-04T09:18:16.376 INFO:teuthology.orchestra.run.smithi106.stdout:(169/176): openssl-libs-3.0.7-27.el9.x86_64.rpm 29 MB/s | 2.2 MB 00:00 2024-03-04T09:18:16.452 INFO:teuthology.orchestra.run.smithi106.stdout:(170/176): util-linux-2.37.4-18.el9.x86_64.rpm 29 MB/s | 2.2 MB 00:00 2024-03-04T09:18:16.493 INFO:teuthology.orchestra.run.smithi106.stdout:(171/176): util-linux-core-2.37.4-18.el9.x86_64 10 MB/s | 438 kB 00:00 2024-03-04T09:18:16.527 INFO:teuthology.orchestra.run.smithi106.stdout:(172/176): util-linux-user-2.37.4-18.el9.x86_64 905 kB/s | 30 kB 00:00 2024-03-04T09:18:16.569 INFO:teuthology.orchestra.run.smithi106.stdout:(173/176): xfsdump-3.1.12-4.el9.x86_64.rpm 7.5 MB/s | 321 kB 00:00 2024-03-04T09:18:16.628 INFO:teuthology.orchestra.run.smithi106.stdout:(174/176): xfsprogs-6.3.0-1.el9.x86_64.rpm 18 MB/s | 1.1 MB 00:00 2024-03-04T09:18:16.683 INFO:teuthology.orchestra.run.smithi191.stdout:(27/176): ceph-test-19.0.0-1739.g1bc250c7.el9.x 11 MB/s | 47 MB 00:04 2024-03-04T09:18:16.708 INFO:teuthology.orchestra.run.smithi191.stdout:(28/176): rbd-nbd-19.0.0-1739.g1bc250c7.el9.x86 372 kB/s | 171 kB 00:00 2024-03-04T09:18:16.756 INFO:teuthology.orchestra.run.smithi052.stdout:(8/176): ceph-common-19.0.0-1739.g1bc250c7.el9. 3.7 MB/s | 20 MB 00:05 2024-03-04T09:18:16.792 INFO:teuthology.orchestra.run.smithi191.stdout:(29/176): librgw2-19.0.0-1739.g1bc250c7.el9.x86 2.3 MB/s | 5.1 MB 00:02 2024-03-04T09:18:16.826 INFO:teuthology.orchestra.run.smithi191.stdout:(30/176): ceph-grafana-dashboards-19.0.0-1739.g 220 kB/s | 31 kB 00:00 2024-03-04T09:18:16.828 INFO:teuthology.orchestra.run.smithi106.stdout:(175/176): librbd1-19.0.0-1739.g1bc250c7.el9.x8 2.2 MB/s | 3.1 MB 00:01 2024-03-04T09:18:16.851 INFO:teuthology.orchestra.run.smithi191.stdout:(31/176): ceph-mgr-cephadm-19.0.0-1739.g1bc250c 967 kB/s | 138 kB 00:00 2024-03-04T09:18:16.857 INFO:teuthology.orchestra.run.smithi052.stdout:(9/176): ceph-radosgw-19.0.0-1739.g1bc250c7.el9 3.7 MB/s | 9.8 MB 00:02 2024-03-04T09:18:16.882 INFO:teuthology.orchestra.run.smithi052.stdout:(10/176): ceph-selinux-19.0.0-1739.g1bc250c7.el 201 kB/s | 25 kB 00:00 2024-03-04T09:18:16.943 INFO:teuthology.orchestra.run.smithi191.stdout:(32/176): ceph-mgr-modules-core-19.0.0-1739.g1b 2.6 MB/s | 248 kB 00:00 2024-03-04T09:18:16.974 INFO:teuthology.orchestra.run.smithi052.stdout:(11/176): cephfs-mirror-19.0.0-1739.g1bc250c7.e 2.4 MB/s | 220 kB 00:00 2024-03-04T09:18:16.994 INFO:teuthology.orchestra.run.smithi106.stdout:(176/176): librados2-19.0.0-1739.g1bc250c7.el9. 2.1 MB/s | 3.4 MB 00:01 2024-03-04T09:18:17.000 INFO:teuthology.orchestra.run.smithi106.stdout:-------------------------------------------------------------------------------- 2024-03-04T09:18:17.000 INFO:teuthology.orchestra.run.smithi106.stdout:Total 17 MB/s | 224 MB 00:12 2024-03-04T09:18:17.027 INFO:teuthology.orchestra.run.smithi191.stdout:(33/176): ceph-mgr-rook-19.0.0-1739.g1bc250c7.e 592 kB/s | 49 kB 00:00 2024-03-04T09:18:17.047 INFO:teuthology.orchestra.run.smithi106.stdout:Running transaction check 2024-03-04T09:18:17.049 INFO:teuthology.orchestra.run.smithi052.stdout:(12/176): cephfs-shell-19.0.0-1739.g1bc250c7.el 360 kB/s | 27 kB 00:00 2024-03-04T09:18:17.094 INFO:teuthology.orchestra.run.smithi191.stdout:(34/176): ceph-prometheus-alerts-19.0.0-1739.g1 230 kB/s | 15 kB 00:00 2024-03-04T09:18:17.125 INFO:teuthology.orchestra.run.smithi052.stdout:(13/176): libcephfs-devel-19.0.0-1739.g1bc250c7 417 kB/s | 31 kB 00:00 2024-03-04T09:18:17.178 INFO:teuthology.orchestra.run.smithi191.stdout:(35/176): ceph-volume-19.0.0-1739.g1bc250c7.el9 3.1 MB/s | 264 kB 00:00 2024-03-04T09:18:17.278 INFO:teuthology.orchestra.run.smithi191.stdout:(36/176): ceph-mgr-dashboard-19.0.0-1739.g1bc25 3.7 MB/s | 1.8 MB 00:00 2024-03-04T09:18:17.322 INFO:teuthology.orchestra.run.smithi106.stdout:Transaction check succeeded. 2024-03-04T09:18:17.322 INFO:teuthology.orchestra.run.smithi106.stdout:Running transaction test 2024-03-04T09:18:17.342 INFO:teuthology.orchestra.run.smithi052.stdout:(14/176): libcephfs2-19.0.0-1739.g1bc250c7.el9. 3.4 MB/s | 747 kB 00:00 2024-03-04T09:18:17.345 INFO:teuthology.orchestra.run.smithi191.stdout:(37/176): cephadm-19.0.0-1739.g1bc250c7.el9.noa 4.4 MB/s | 748 kB 00:00 2024-03-04T09:18:17.370 INFO:teuthology.orchestra.run.smithi191.stdout:(38/176): cephfs-top-19.0.0-1739.g1bc250c7.el9. 198 kB/s | 18 kB 00:00 2024-03-04T09:18:17.417 INFO:teuthology.orchestra.run.smithi052.stdout:(15/176): libcephsqlite-19.0.0-1739.g1bc250c7.e 2.1 MB/s | 163 kB 00:00 2024-03-04T09:18:17.484 INFO:teuthology.orchestra.run.smithi052.stdout:(16/176): librados-devel-19.0.0-1739.g1bc250c7. 1.9 MB/s | 127 kB 00:00 2024-03-04T09:18:17.587 INFO:teuthology.orchestra.run.smithi191.stdout:(39/176): ceph-mgr-diskprediction-local-19.0.0- 9.7 MB/s | 7.4 MB 00:00 2024-03-04T09:18:17.613 INFO:teuthology.orchestra.run.smithi191.stdout:(40/176): automake-1.16.2-8.el9.noarch.rpm 2.4 MB/s | 662 kB 00:00 2024-03-04T09:18:17.646 INFO:teuthology.orchestra.run.smithi191.stdout:(41/176): bison-3.7.4-5.el9.x86_64.rpm 3.3 MB/s | 920 kB 00:00 2024-03-04T09:18:17.651 INFO:teuthology.orchestra.run.smithi052.stdout:(17/176): ceph-osd-19.0.0-1739.g1bc250c7.el9.x8 4.0 MB/s | 17 MB 00:04 2024-03-04T09:18:17.672 INFO:teuthology.orchestra.run.smithi191.stdout:(42/176): boost-program-options-1.75.0-8.el9.x8 1.3 MB/s | 106 kB 00:00 2024-03-04T09:18:17.684 INFO:teuthology.orchestra.run.smithi052.stdout:(18/176): libradosstriper1-19.0.0-1739.g1bc250c 2.4 MB/s | 501 kB 00:00 2024-03-04T09:18:17.688 INFO:teuthology.orchestra.run.smithi191.stdout:(43/176): elfutils-libelf-devel-0.190-2.el9.x86 489 kB/s | 20 kB 00:00 2024-03-04T09:18:17.714 INFO:teuthology.orchestra.run.smithi191.stdout:(44/176): flex-2.6.4-9.el9.x86_64.rpm 4.5 MB/s | 307 kB 00:00 2024-03-04T09:18:17.731 INFO:teuthology.orchestra.run.smithi191.stdout:(45/176): flexiblas-3.0.4-8.el9.x86_64.rpm 541 kB/s | 32 kB 00:00 2024-03-04T09:18:17.735 INFO:teuthology.orchestra.run.smithi052.stdout:(19/176): python3-ceph-argparse-19.0.0-1739.g1b 905 kB/s | 45 kB 00:00 2024-03-04T09:18:17.765 INFO:teuthology.orchestra.run.smithi191.stdout:(46/176): flexiblas-openblas-openmp-3.0.4-8.el9 341 kB/s | 17 kB 00:00 2024-03-04T09:18:17.810 INFO:teuthology.orchestra.run.smithi052.stdout:(20/176): python3-ceph-common-19.0.0-1739.g1bc2 1.7 MB/s | 131 kB 00:00 2024-03-04T09:18:17.815 INFO:teuthology.orchestra.run.smithi191.stdout:(47/176): flexiblas-netlib-3.0.4-8.el9.x86_64.r 30 MB/s | 3.0 MB 00:00 2024-03-04T09:18:17.832 INFO:teuthology.orchestra.run.smithi191.stdout:(48/176): libaio-devel-0.3.111-13.el9.x86_64.rp 100 kB/s | 10 kB 00:00 2024-03-04T09:18:17.857 INFO:teuthology.orchestra.run.smithi191.stdout:(49/176): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.7 MB/s | 159 kB 00:00 2024-03-04T09:18:17.875 INFO:teuthology.orchestra.run.smithi191.stdout:(50/176): librabbitmq-0.11.0-7.el9.x86_64.rpm 1.0 MB/s | 44 kB 00:00 2024-03-04T09:18:17.877 INFO:teuthology.orchestra.run.smithi052.stdout:(21/176): python3-cephfs-19.0.0-1739.g1bc250c7. 2.4 MB/s | 162 kB 00:00 2024-03-04T09:18:17.908 INFO:teuthology.orchestra.run.smithi191.stdout:(51/176): librdkafka-1.6.1-102.el9.x86_64.rpm 8.6 MB/s | 661 kB 00:00 2024-03-04T09:18:17.942 INFO:teuthology.orchestra.run.smithi191.stdout:(52/176): libtool-2.4.6-45.el9.x86_64.rpm 6.8 MB/s | 578 kB 00:00 2024-03-04T09:18:17.952 INFO:teuthology.orchestra.run.smithi052.stdout:(22/176): python3-rados-19.0.0-1739.g1bc250c7.e 4.2 MB/s | 321 kB 00:00 2024-03-04T09:18:17.959 INFO:teuthology.orchestra.run.smithi191.stdout:(53/176): libuuid-devel-2.37.4-18.el9.x86_64.rp 492 kB/s | 24 kB 00:00 2024-03-04T09:18:17.976 INFO:teuthology.orchestra.run.smithi191.stdout:(54/176): libzstd-devel-1.5.1-2.el9.x86_64.rpm 700 kB/s | 46 kB 00:00 2024-03-04T09:18:18.001 INFO:teuthology.orchestra.run.smithi191.stdout:(55/176): lttng-ust-2.12.0-6.el9.x86_64.rpm 4.7 MB/s | 282 kB 00:00 2024-03-04T09:18:18.027 INFO:teuthology.orchestra.run.smithi191.stdout:(56/176): lua-5.4.4-4.el9.x86_64.rpm 3.7 MB/s | 187 kB 00:00 2024-03-04T09:18:18.027 INFO:teuthology.orchestra.run.smithi052.stdout:(23/176): python3-rbd-19.0.0-1739.g1bc250c7.el9 3.9 MB/s | 298 kB 00:00 2024-03-04T09:18:18.052 INFO:teuthology.orchestra.run.smithi191.stdout:(57/176): openblas-0.3.21-2.el9.x86_64.rpm 455 kB/s | 34 kB 00:00 2024-03-04T09:18:18.095 INFO:teuthology.orchestra.run.smithi052.stdout:(24/176): python3-rgw-19.0.0-1739.g1bc250c7.el9 1.5 MB/s | 99 kB 00:00 2024-03-04T09:18:18.161 INFO:teuthology.orchestra.run.smithi191.stdout:(58/176): openblas-openmp-0.3.21-2.el9.x86_64.r 30 MB/s | 4.7 MB 00:00 2024-03-04T09:18:18.161 INFO:teuthology.orchestra.run.smithi052.stdout:(25/176): rbd-fuse-19.0.0-1739.g1bc250c7.el9.x8 1.3 MB/s | 85 kB 00:00 2024-03-04T09:18:18.220 INFO:teuthology.orchestra.run.smithi191.stdout:(59/176): openssl-devel-3.0.7-27.el9.x86_64.rpm 18 MB/s | 3.0 MB 00:00 2024-03-04T09:18:18.261 INFO:teuthology.orchestra.run.smithi191.stdout:(60/176): postgresql-13.11-1.el9.x86_64.rpm 7.4 MB/s | 1.5 MB 00:00 2024-03-04T09:18:18.287 INFO:teuthology.orchestra.run.smithi191.stdout:(61/176): postgresql-contrib-13.11-1.el9.x86_64 6.4 MB/s | 811 kB 00:00 2024-03-04T09:18:18.312 INFO:teuthology.orchestra.run.smithi191.stdout:(62/176): postgresql-private-libs-13.11-1.el9.x 1.5 MB/s | 134 kB 00:00 2024-03-04T09:18:18.462 INFO:teuthology.orchestra.run.smithi191.stdout:(63/176): protobuf-3.14.0-13.el9.x86_64.rpm 5.8 MB/s | 1.0 MB 00:00 2024-03-04T09:18:18.546 INFO:teuthology.orchestra.run.smithi191.stdout:(64/176): postgresql-server-13.11-1.el9.x86_64. 22 MB/s | 5.7 MB 00:00 2024-03-04T09:18:18.571 INFO:teuthology.orchestra.run.smithi191.stdout:(65/176): python3-attrs-20.3.0-7.el9.noarch.rpm 300 kB/s | 78 kB 00:00 2024-03-04T09:18:18.620 INFO:teuthology.orchestra.run.smithi052.stdout:(26/176): librgw2-19.0.0-1739.g1bc250c7.el9.x86 5.3 MB/s | 5.1 MB 00:00 2024-03-04T09:18:18.687 INFO:teuthology.orchestra.run.smithi052.stdout:(27/176): rbd-nbd-19.0.0-1739.g1bc250c7.el9.x86 2.5 MB/s | 171 kB 00:00 2024-03-04T09:18:18.722 INFO:teuthology.orchestra.run.smithi191.stdout:(66/176): python3-babel-2.9.1-2.el9.noarch.rpm 23 MB/s | 5.8 MB 00:00 2024-03-04T09:18:18.747 INFO:teuthology.orchestra.run.smithi191.stdout:(67/176): python3-jinja2-2.11.3-5.el9.noarch.rp 1.3 MB/s | 227 kB 00:00 2024-03-04T09:18:18.754 INFO:teuthology.orchestra.run.smithi052.stdout:(28/176): rbd-mirror-19.0.0-1739.g1bc250c7.el9. 5.2 MB/s | 3.1 MB 00:00 2024-03-04T09:18:18.764 INFO:teuthology.orchestra.run.smithi191.stdout:(68/176): python3-jsonpatch-1.21-16.el9.noarch. 128 kB/s | 24 kB 00:00 2024-03-04T09:18:18.779 INFO:teuthology.orchestra.run.smithi052.stdout:(29/176): ceph-grafana-dashboards-19.0.0-1739.g 341 kB/s | 31 kB 00:00 2024-03-04T09:18:18.790 INFO:teuthology.orchestra.run.smithi191.stdout:(69/176): python3-jsonpointer-2.0-4.el9.noarch. 269 kB/s | 18 kB 00:00 2024-03-04T09:18:18.821 INFO:teuthology.orchestra.run.smithi052.stdout:(30/176): ceph-mgr-cephadm-19.0.0-1739.g1bc250c 2.0 MB/s | 138 kB 00:00 2024-03-04T09:18:18.824 INFO:teuthology.orchestra.run.smithi191.stdout:(70/176): python3-mako-1.1.4-6.el9.noarch.rpm 2.4 MB/s | 144 kB 00:00 2024-03-04T09:18:18.841 INFO:teuthology.orchestra.run.smithi191.stdout:(71/176): python3-markupsafe-1.1.1-12.el9.x86_6 413 kB/s | 32 kB 00:00 2024-03-04T09:18:18.983 INFO:teuthology.orchestra.run.smithi191.stdout:(72/176): python3-numpy-1.20.1-5.el9.x86_64.rpm 25 MB/s | 4.8 MB 00:00 2024-03-04T09:18:18.991 INFO:teuthology.orchestra.run.smithi106.stdout:Transaction test succeeded. 2024-03-04T09:18:18.991 INFO:teuthology.orchestra.run.smithi106.stdout:Running transaction 2024-03-04T09:18:19.008 INFO:teuthology.orchestra.run.smithi191.stdout:(73/176): python3-numpy-f2py-1.20.1-5.el9.x86_6 1.3 MB/s | 217 kB 00:00 2024-03-04T09:18:19.034 INFO:teuthology.orchestra.run.smithi191.stdout:(74/176): python3-oauthlib-3.1.1-5.el9.noarch.r 920 kB/s | 174 kB 00:00 2024-03-04T09:18:19.059 INFO:teuthology.orchestra.run.smithi191.stdout:(75/176): python3-packaging-20.9-5.el9.noarch.r 920 kB/s | 69 kB 00:00 2024-03-04T09:18:19.075 INFO:teuthology.orchestra.run.smithi191.stdout:(76/176): python3-prettytable-0.7.2-27.el9.noar 995 kB/s | 41 kB 00:00 2024-03-04T09:18:19.101 INFO:teuthology.orchestra.run.smithi191.stdout:(77/176): python3-protobuf-3.14.0-13.el9.noarch 3.5 MB/s | 238 kB 00:00 2024-03-04T09:18:19.126 INFO:teuthology.orchestra.run.smithi191.stdout:(78/176): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.9 MB/s | 132 kB 00:00 2024-03-04T09:18:19.138 INFO:teuthology.orchestra.run.smithi052.stdout:(31/176): ceph-mgr-dashboard-19.0.0-1739.g1bc25 5.0 MB/s | 1.8 MB 00:00 2024-03-04T09:18:19.151 INFO:teuthology.orchestra.run.smithi191.stdout:(79/176): python3-pyasn1-modules-0.4.8-6.el9.no 4.1 MB/s | 211 kB 00:00 2024-03-04T09:18:19.168 INFO:teuthology.orchestra.run.smithi191.stdout:(80/176): python3-pytz-2021.1-5.el9.noarch.rpm 711 kB/s | 47 kB 00:00 2024-03-04T09:18:19.185 INFO:teuthology.orchestra.run.smithi191.stdout:(81/176): python3-requests-oauthlib-1.3.0-12.el 736 kB/s | 43 kB 00:00 2024-03-04T09:18:19.221 INFO:teuthology.orchestra.run.smithi052.stdout:(32/176): ceph-mgr-modules-core-19.0.0-1739.g1b 2.9 MB/s | 248 kB 00:00 2024-03-04T09:18:19.244 INFO:teuthology.orchestra.run.smithi191.stdout:(82/176): python3-toml-0.10.2-6.el9.noarch.rpm 515 kB/s | 39 kB 00:00 2024-03-04T09:18:19.288 INFO:teuthology.orchestra.run.smithi052.stdout:(33/176): ceph-mgr-rook-19.0.0-1739.g1bc250c7.e 741 kB/s | 49 kB 00:00 2024-03-04T09:18:19.356 INFO:teuthology.orchestra.run.smithi052.stdout:(34/176): ceph-prometheus-alerts-19.0.0-1739.g1 230 kB/s | 15 kB 00:00 2024-03-04T09:18:19.369 INFO:teuthology.orchestra.run.smithi191.stdout:(83/176): python3-wcwidth-0.2.5-8.el9.noarch.rp 224 kB/s | 41 kB 00:00 2024-03-04T09:18:19.431 INFO:teuthology.orchestra.run.smithi052.stdout:(35/176): ceph-volume-19.0.0-1739.g1bc250c7.el9 3.4 MB/s | 264 kB 00:00 2024-03-04T09:18:19.531 INFO:teuthology.orchestra.run.smithi191.stdout:(84/176): python3-scipy-1.6.2-8.el9.x86_64.rpm 44 MB/s | 16 MB 00:00 2024-03-04T09:18:19.553 INFO:teuthology.orchestra.run.smithi191.stdout:(85/176): qatlib-23.11.0-1.el9.x86_64.rpm 703 kB/s | 215 kB 00:00 2024-03-04T09:18:19.570 INFO:teuthology.orchestra.run.smithi191.stdout:(86/176): qatlib-service-23.11.0-1.el9.x86_64.r 170 kB/s | 34 kB 00:00 2024-03-04T09:18:19.587 INFO:teuthology.orchestra.run.smithi191.stdout:(87/176): qatzip-libs-1.1.2-2.el9.x86_64.rpm 750 kB/s | 43 kB 00:00 2024-03-04T09:18:19.615 INFO:teuthology.orchestra.run.smithi052.stdout:(36/176): cephadm-19.0.0-1739.g1bc250c7.el9.noa 4.0 MB/s | 748 kB 00:00 2024-03-04T09:18:19.621 INFO:teuthology.orchestra.run.smithi191.stdout:(88/176): socat-1.7.4.1-5.el9.x86_64.rpm 5.9 MB/s | 300 kB 00:00 2024-03-04T09:18:19.637 INFO:teuthology.orchestra.run.smithi191.stdout:(89/176): uuid-1.6.2-55.el9.x86_64.rpm 847 kB/s | 56 kB 00:00 2024-03-04T09:18:19.662 INFO:teuthology.orchestra.run.smithi191.stdout:(90/176): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 1.1 MB/s | 87 kB 00:00 2024-03-04T09:18:19.679 INFO:teuthology.orchestra.run.smithi191.stdout:(91/176): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.5 MB/s | 63 kB 00:00 2024-03-04T09:18:19.698 INFO:teuthology.orchestra.run.smithi052.stdout:(37/176): cephfs-top-19.0.0-1739.g1bc250c7.el9. 219 kB/s | 18 kB 00:00 2024-03-04T09:18:19.704 INFO:teuthology.orchestra.run.smithi191.stdout:(92/176): zlib-devel-1.2.11-40.el9.x86_64.rpm 659 kB/s | 44 kB 00:00 2024-03-04T09:18:19.732 INFO:teuthology.orchestra.run.smithi191.stdout:(93/176): fuse-2.9.9-15.el9.x86_64.rpm 1.2 MB/s | 79 kB 00:00 2024-03-04T09:18:19.755 INFO:teuthology.orchestra.run.smithi191.stdout:(94/176): jq-1.6-16.el9.x86_64.rpm 3.6 MB/s | 186 kB 00:00 2024-03-04T09:18:19.788 INFO:teuthology.orchestra.run.smithi191.stdout:(95/176): libgfortran-11.4.1-3.el9.x86_64.rpm 9.4 MB/s | 802 kB 00:00 2024-03-04T09:18:19.813 INFO:teuthology.orchestra.run.smithi191.stdout:(96/176): libquadmath-11.4.1-3.el9.x86_64.rpm 2.3 MB/s | 194 kB 00:00 2024-03-04T09:18:19.839 INFO:teuthology.orchestra.run.smithi191.stdout:(97/176): oniguruma-6.9.6-1.el9.5.x86_64.rpm 4.3 MB/s | 217 kB 00:00 2024-03-04T09:18:19.857 INFO:teuthology.orchestra.run.smithi052.stdout:(38/176): automake-1.16.2-8.el9.noarch.rpm 4.1 MB/s | 662 kB 00:00 2024-03-04T09:18:19.864 INFO:teuthology.orchestra.run.smithi191.stdout:(98/176): python3-cffi-1.14.5-5.el9.x86_64.rpm 3.2 MB/s | 241 kB 00:00 2024-03-04T09:18:19.906 INFO:teuthology.orchestra.run.smithi191.stdout:(99/176): python3-cryptography-36.0.1-4.el9.x86 13 MB/s | 1.1 MB 00:00 2024-03-04T09:18:19.923 INFO:teuthology.orchestra.run.smithi052.stdout:(39/176): bison-3.7.4-5.el9.x86_64.rpm 14 MB/s | 920 kB 00:00 2024-03-04T09:18:19.931 INFO:teuthology.orchestra.run.smithi191.stdout:(100/176): python3-ply-3.11-14.el9.noarch.rpm 1.5 MB/s | 103 kB 00:00 2024-03-04T09:18:19.956 INFO:teuthology.orchestra.run.smithi191.stdout:(101/176): python3-pycparser-2.20-6.el9.noarch. 1.3 MB/s | 124 kB 00:00 2024-03-04T09:18:19.973 INFO:teuthology.orchestra.run.smithi191.stdout:(102/176): python3-requests-2.25.1-8.el9.noarch 1.7 MB/s | 113 kB 00:00 2024-03-04T09:18:19.999 INFO:teuthology.orchestra.run.smithi191.stdout:(103/176): python3-urllib3-1.26.5-5.el9.noarch. 4.4 MB/s | 187 kB 00:00 2024-03-04T09:18:20.007 INFO:teuthology.orchestra.run.smithi052.stdout:(40/176): ceph-mgr-diskprediction-local-19.0.0- 6.2 MB/s | 7.4 MB 00:01 2024-03-04T09:18:20.033 INFO:teuthology.orchestra.run.smithi052.stdout:(41/176): boost-program-options-1.75.0-8.el9.x8 972 kB/s | 106 kB 00:00 2024-03-04T09:18:20.059 INFO:teuthology.orchestra.run.smithi052.stdout:(42/176): elfutils-libelf-devel-0.190-2.el9.x86 408 kB/s | 20 kB 00:00 2024-03-04T09:18:20.084 INFO:teuthology.orchestra.run.smithi052.stdout:(43/176): flex-2.6.4-9.el9.x86_64.rpm 6.0 MB/s | 307 kB 00:00 2024-03-04T09:18:20.091 INFO:teuthology.orchestra.run.smithi191.stdout:(104/176): lua-devel-5.4.4-4.el9.x86_64.rpm 191 kB/s | 22 kB 00:00 2024-03-04T09:18:20.110 INFO:teuthology.orchestra.run.smithi052.stdout:(44/176): flexiblas-3.0.4-8.el9.x86_64.rpm 639 kB/s | 32 kB 00:00 2024-03-04T09:18:20.116 INFO:teuthology.orchestra.run.smithi191.stdout:(105/176): python3-devel-3.9.17-2.el9.x86_64.rp 1.5 MB/s | 245 kB 00:00 2024-03-04T09:18:20.184 INFO:teuthology.orchestra.run.smithi052.stdout:(45/176): flexiblas-netlib-3.0.4-8.el9.x86_64.r 30 MB/s | 3.0 MB 00:00 2024-03-04T09:18:20.209 INFO:teuthology.orchestra.run.smithi052.stdout:(46/176): flexiblas-openblas-openmp-3.0.4-8.el9 176 kB/s | 17 kB 00:00 2024-03-04T09:18:20.235 INFO:teuthology.orchestra.run.smithi052.stdout:(47/176): libaio-devel-0.3.111-13.el9.x86_64.rp 201 kB/s | 10 kB 00:00 2024-03-04T09:18:20.260 INFO:teuthology.orchestra.run.smithi052.stdout:(48/176): libpmemobj-1.12.1-1.el9.x86_64.rpm 3.1 MB/s | 159 kB 00:00 2024-03-04T09:18:20.285 INFO:teuthology.orchestra.run.smithi052.stdout:(49/176): librabbitmq-0.11.0-7.el9.x86_64.rpm 887 kB/s | 44 kB 00:00 2024-03-04T09:18:20.310 INFO:teuthology.orchestra.run.smithi052.stdout:(50/176): librdkafka-1.6.1-102.el9.x86_64.rpm 13 MB/s | 661 kB 00:00 2024-03-04T09:18:20.347 INFO:teuthology.orchestra.run.smithi052.stdout:(51/176): libtool-2.4.6-45.el9.x86_64.rpm 9.9 MB/s | 578 kB 00:00 2024-03-04T09:18:20.369 INFO:teuthology.orchestra.run.smithi052.stdout:(52/176): libuuid-devel-2.37.4-18.el9.x86_64.rp 424 kB/s | 24 kB 00:00 2024-03-04T09:18:20.391 INFO:teuthology.orchestra.run.smithi191.stdout:(106/176): protobuf-compiler-3.14.0-13.el9.x86_ 2.2 MB/s | 863 kB 00:00 2024-03-04T09:18:20.394 INFO:teuthology.orchestra.run.smithi052.stdout:(53/176): libzstd-devel-1.5.1-2.el9.x86_64.rpm 926 kB/s | 46 kB 00:00 2024-03-04T09:18:20.420 INFO:teuthology.orchestra.run.smithi052.stdout:(54/176): lttng-ust-2.12.0-6.el9.x86_64.rpm 5.5 MB/s | 282 kB 00:00 2024-03-04T09:18:20.445 INFO:teuthology.orchestra.run.smithi052.stdout:(55/176): lua-5.4.4-4.el9.x86_64.rpm 3.7 MB/s | 187 kB 00:00 2024-03-04T09:18:20.470 INFO:teuthology.orchestra.run.smithi052.stdout:(56/176): openblas-0.3.21-2.el9.x86_64.rpm 687 kB/s | 34 kB 00:00 2024-03-04T09:18:20.566 INFO:teuthology.orchestra.run.smithi191.stdout:(107/176): gperftools-libs-2.9.1-2.el9.x86_64.r 686 kB/s | 309 kB 00:00 2024-03-04T09:18:20.570 INFO:teuthology.orchestra.run.smithi052.stdout:(57/176): openblas-openmp-0.3.21-2.el9.x86_64.r 38 MB/s | 4.7 MB 00:00 2024-03-04T09:18:20.616 INFO:teuthology.orchestra.run.smithi191.stdout:(108/176): abseil-cpp-20211102.0-3.el9.x86_64.r 1.0 MB/s | 548 kB 00:00 2024-03-04T09:18:20.621 INFO:teuthology.orchestra.run.smithi052.stdout:(58/176): openssl-devel-3.0.7-27.el9.x86_64.rpm 20 MB/s | 3.0 MB 00:00 2024-03-04T09:18:20.642 INFO:teuthology.orchestra.run.smithi191.stdout:(109/176): grpc-data-1.46.7-10.el9.noarch.rpm 78 kB/s | 19 kB 00:00 2024-03-04T09:18:20.663 INFO:teuthology.orchestra.run.smithi052.stdout:(59/176): postgresql-13.11-1.el9.x86_64.rpm 17 MB/s | 1.5 MB 00:00 2024-03-04T09:18:20.684 INFO:teuthology.orchestra.run.smithi191.stdout:(110/176): libarrow-doc-9.0.0-10.el9.noarch.rpm 387 kB/s | 26 kB 00:00 2024-03-04T09:18:20.696 INFO:teuthology.orchestra.run.smithi052.stdout:(60/176): postgresql-contrib-13.11-1.el9.x86_64 11 MB/s | 811 kB 00:00 2024-03-04T09:18:20.722 INFO:teuthology.orchestra.run.smithi052.stdout:(61/176): postgresql-private-libs-13.11-1.el9.x 2.3 MB/s | 134 kB 00:00 2024-03-04T09:18:20.759 INFO:teuthology.orchestra.run.smithi191.stdout:(111/176): liboath-2.6.7-2.el9.x86_64.rpm 415 kB/s | 49 kB 00:00 2024-03-04T09:18:20.776 INFO:teuthology.orchestra.run.smithi191.stdout:(112/176): libunwind-1.6.2-1.el9.x86_64.rpm 733 kB/s | 67 kB 00:00 2024-03-04T09:18:20.843 INFO:teuthology.orchestra.run.smithi191.stdout:(113/176): luarocks-3.9.2-1.el9.noarch.rpm 1.8 MB/s | 151 kB 00:00 2024-03-04T09:18:20.855 INFO:teuthology.orchestra.run.smithi052.stdout:(62/176): postgresql-server-13.11-1.el9.x86_64. 36 MB/s | 5.7 MB 00:00 2024-03-04T09:18:20.889 INFO:teuthology.orchestra.run.smithi052.stdout:(63/176): protobuf-3.14.0-13.el9.x86_64.rpm 6.2 MB/s | 1.0 MB 00:00 2024-03-04T09:18:20.914 INFO:teuthology.orchestra.run.smithi052.stdout:(64/176): python3-attrs-20.3.0-7.el9.noarch.rpm 1.3 MB/s | 78 kB 00:00 2024-03-04T09:18:20.918 INFO:teuthology.orchestra.run.smithi191.stdout:(114/176): libarrow-9.0.0-10.el9.x86_64.rpm 13 MB/s | 4.4 MB 00:00 2024-03-04T09:18:20.960 INFO:teuthology.orchestra.run.smithi191.stdout:(115/176): python3-asyncssh-2.13.2-4.el9.noarch 4.6 MB/s | 548 kB 00:00 2024-03-04T09:18:20.994 INFO:teuthology.orchestra.run.smithi191.stdout:(116/176): python3-bcrypt-3.2.2-1.el9.x86_64.rp 577 kB/s | 43 kB 00:00 2024-03-04T09:18:21.027 INFO:teuthology.orchestra.run.smithi191.stdout:(117/176): python3-cachetools-4.2.4-1.el9.noarc 480 kB/s | 32 kB 00:00 2024-03-04T09:18:21.048 INFO:teuthology.orchestra.run.smithi052.stdout:(65/176): python3-babel-2.9.1-2.el9.noarch.rpm 36 MB/s | 5.8 MB 00:00 2024-03-04T09:18:21.052 INFO:teuthology.orchestra.run.smithi191.stdout:(118/176): parquet-libs-9.0.0-10.el9.x86_64.rpm 3.0 MB/s | 838 kB 00:00 2024-03-04T09:18:21.073 INFO:teuthology.orchestra.run.smithi052.stdout:(66/176): python3-jinja2-2.11.3-5.el9.noarch.rp 1.4 MB/s | 227 kB 00:00 2024-03-04T09:18:21.078 INFO:teuthology.orchestra.run.smithi191.stdout:(119/176): python3-certifi-2021.10.8-2.el9.noar 179 kB/s | 15 kB 00:00 2024-03-04T09:18:21.098 INFO:teuthology.orchestra.run.smithi052.stdout:(67/176): python3-jsonpatch-1.21-16.el9.noarch. 492 kB/s | 24 kB 00:00 2024-03-04T09:18:21.103 INFO:teuthology.orchestra.run.smithi191.stdout:(120/176): python3-cheroot-8.6.0-4.el9.noarch.r 2.2 MB/s | 172 kB 00:00 2024-03-04T09:18:21.124 INFO:teuthology.orchestra.run.smithi052.stdout:(68/176): python3-jsonpointer-2.0-4.el9.noarch. 359 kB/s | 18 kB 00:00 2024-03-04T09:18:21.128 INFO:teuthology.orchestra.run.smithi191.stdout:(121/176): python3-cherrypy-18.6.1-2.el9.noarch 4.7 MB/s | 358 kB 00:00 2024-03-04T09:18:21.154 INFO:teuthology.orchestra.run.smithi191.stdout:(122/176): python3-cmd2-2.4.2-2.el9.noarch.rpm 3.9 MB/s | 303 kB 00:00 2024-03-04T09:18:21.157 INFO:teuthology.orchestra.run.smithi052.stdout:(69/176): python3-mako-1.1.4-6.el9.noarch.rpm 2.4 MB/s | 144 kB 00:00 2024-03-04T09:18:21.179 INFO:teuthology.orchestra.run.smithi191.stdout:(123/176): python3-colorama-0.4.4-12.el9.noarch 468 kB/s | 35 kB 00:00 2024-03-04T09:18:21.183 INFO:teuthology.orchestra.run.smithi052.stdout:(70/176): python3-markupsafe-1.1.1-12.el9.x86_6 540 kB/s | 32 kB 00:00 2024-03-04T09:18:21.204 INFO:teuthology.orchestra.run.smithi191.stdout:(124/176): python3-google-auth-2.28.0-1.el9.noa 2.6 MB/s | 203 kB 00:00 2024-03-04T09:18:21.263 INFO:teuthology.orchestra.run.smithi191.stdout:(125/176): python3-grpcio-tools-1.46.7-10.el9.x 1.7 MB/s | 144 kB 00:00 2024-03-04T09:18:21.288 INFO:teuthology.orchestra.run.smithi191.stdout:(126/176): python3-jaraco-8.2.1-3.el9.noarch.rp 128 kB/s | 11 kB 00:00 2024-03-04T09:18:21.291 INFO:teuthology.orchestra.run.smithi052.stdout:(71/176): python3-numpy-1.20.1-5.el9.x86_64.rpm 36 MB/s | 4.8 MB 00:00 2024-03-04T09:18:21.317 INFO:teuthology.orchestra.run.smithi052.stdout:(72/176): python3-numpy-f2py-1.20.1-5.el9.x86_6 1.6 MB/s | 217 kB 00:00 2024-03-04T09:18:21.339 INFO:teuthology.orchestra.run.smithi191.stdout:(127/176): python3-grpcio-1.46.7-10.el9.x86_64. 11 MB/s | 2.0 MB 00:00 2024-03-04T09:18:21.342 INFO:teuthology.orchestra.run.smithi052.stdout:(73/176): python3-oauthlib-3.1.1-5.el9.noarch.r 3.4 MB/s | 174 kB 00:00 2024-03-04T09:18:21.367 INFO:teuthology.orchestra.run.smithi191.stdout:(128/176): python3-jaraco-classes-3.2.1-5.el9.n 175 kB/s | 18 kB 00:00 2024-03-04T09:18:21.368 INFO:teuthology.orchestra.run.smithi052.stdout:(74/176): python3-packaging-20.9-5.el9.noarch.r 1.3 MB/s | 69 kB 00:00 2024-03-04T09:18:21.389 INFO:teuthology.orchestra.run.smithi191.stdout:(129/176): python3-jaraco-collections-3.0.0-8.e 229 kB/s | 23 kB 00:00 2024-03-04T09:18:21.393 INFO:teuthology.orchestra.run.smithi052.stdout:(75/176): python3-prettytable-0.7.2-27.el9.noar 821 kB/s | 41 kB 00:00 2024-03-04T09:18:21.415 INFO:teuthology.orchestra.run.smithi191.stdout:(130/176): python3-jaraco-functools-3.5.0-2.el9 258 kB/s | 19 kB 00:00 2024-03-04T09:18:21.418 INFO:teuthology.orchestra.run.smithi052.stdout:(76/176): python3-protobuf-3.14.0-13.el9.noarch 4.6 MB/s | 238 kB 00:00 2024-03-04T09:18:21.443 INFO:teuthology.orchestra.run.smithi052.stdout:(77/176): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.6 MB/s | 132 kB 00:00 2024-03-04T09:18:21.465 INFO:teuthology.orchestra.run.smithi191.stdout:(131/176): python3-jaraco-text-3.2.0-6.el9.noar 262 kB/s | 20 kB 00:00 2024-03-04T09:18:21.469 INFO:teuthology.orchestra.run.smithi052.stdout:(78/176): python3-pyasn1-modules-0.4.8-6.el9.no 4.1 MB/s | 211 kB 00:00 2024-03-04T09:18:21.494 INFO:teuthology.orchestra.run.smithi052.stdout:(79/176): python3-pytz-2021.1-5.el9.noarch.rpm 949 kB/s | 47 kB 00:00 2024-03-04T09:18:21.507 INFO:teuthology.orchestra.run.smithi191.stdout:(132/176): python3-logutils-0.3.5-21.el9.noarch 504 kB/s | 46 kB 00:00 2024-03-04T09:18:21.519 INFO:teuthology.orchestra.run.smithi052.stdout:(80/176): python3-requests-oauthlib-1.3.0-12.el 856 kB/s | 43 kB 00:00 2024-03-04T09:18:21.540 INFO:teuthology.orchestra.run.smithi191.stdout:(133/176): python3-kubernetes-26.1.0-2.el9.noar 6.8 MB/s | 1.0 MB 00:00 2024-03-04T09:18:21.565 INFO:teuthology.orchestra.run.smithi191.stdout:(134/176): python3-more-itertools-8.12.0-2.el9. 786 kB/s | 79 kB 00:00 2024-03-04T09:18:21.591 INFO:teuthology.orchestra.run.smithi191.stdout:(135/176): python3-natsort-7.1.1-5.el9.noarch.r 691 kB/s | 58 kB 00:00 2024-03-04T09:18:21.616 INFO:teuthology.orchestra.run.smithi191.stdout:(136/176): python3-pecan-1.4.2-3.el9.noarch.rpm 3.5 MB/s | 272 kB 00:00 2024-03-04T09:18:21.650 INFO:teuthology.orchestra.run.smithi191.stdout:(137/176): python3-portend-3.1.0-2.el9.noarch.r 195 kB/s | 16 kB 00:00 2024-03-04T09:18:21.661 INFO:teuthology.orchestra.run.smithi052.stdout:(81/176): python3-toml-0.10.2-6.el9.noarch.rpm 278 kB/s | 39 kB 00:00 2024-03-04T09:18:21.695 INFO:teuthology.orchestra.run.smithi191.stdout:(138/176): python3-pyOpenSSL-21.0.0-1.el9.noarc 871 kB/s | 90 kB 00:00 2024-03-04T09:18:21.717 INFO:teuthology.orchestra.run.smithi191.stdout:(139/176): python3-pyperclip-1.8.2-2.el9.noarch 281 kB/s | 28 kB 00:00 2024-03-04T09:18:21.742 INFO:teuthology.orchestra.run.smithi191.stdout:(140/176): python3-repoze-lru-0.7-16.el9.noarch 334 kB/s | 31 kB 00:00 2024-03-04T09:18:21.768 INFO:teuthology.orchestra.run.smithi191.stdout:(141/176): python3-routes-2.5.1-5.el9.noarch.rp 2.5 MB/s | 188 kB 00:00 2024-03-04T09:18:21.785 INFO:teuthology.orchestra.run.smithi191.stdout:(142/176): python3-rsa-4.9-2.el9.noarch.rpm 878 kB/s | 59 kB 00:00 2024-03-04T09:18:21.810 INFO:teuthology.orchestra.run.smithi191.stdout:(143/176): python3-tempora-5.0.0-2.el9.noarch.r 533 kB/s | 36 kB 00:00 2024-03-04T09:18:21.827 INFO:teuthology.orchestra.run.smithi191.stdout:(144/176): python3-typing-extensions-4.4.0-2.el 869 kB/s | 51 kB 00:00 2024-03-04T09:18:21.860 INFO:teuthology.orchestra.run.smithi191.stdout:(145/176): python3-webob-1.8.7-6.el9.noarch.rpm 3.0 MB/s | 230 kB 00:00 2024-03-04T09:18:21.870 INFO:teuthology.orchestra.run.smithi052.stdout:(82/176): python3-scipy-1.6.2-8.el9.x86_64.rpm 42 MB/s | 16 MB 00:00 2024-03-04T09:18:21.886 INFO:teuthology.orchestra.run.smithi191.stdout:(146/176): python3-websocket-client-1.2.3-2.el9 1.2 MB/s | 90 kB 00:00 2024-03-04T09:18:21.895 INFO:teuthology.orchestra.run.smithi052.stdout:(83/176): python3-wcwidth-0.2.5-8.el9.noarch.rp 177 kB/s | 41 kB 00:00 2024-03-04T09:18:21.911 INFO:teuthology.orchestra.run.smithi191.stdout:(147/176): python3-werkzeug-2.0.3-3.el9.1.noarc 5.0 MB/s | 427 kB 00:00 2024-03-04T09:18:21.920 INFO:teuthology.orchestra.run.smithi052.stdout:(84/176): qatlib-23.11.0-1.el9.x86_64.rpm 4.2 MB/s | 215 kB 00:00 2024-03-04T09:18:21.928 INFO:teuthology.orchestra.run.smithi191.stdout:(148/176): python3-zc-lockfile-2.0-10.el9.noarc 298 kB/s | 20 kB 00:00 2024-03-04T09:18:21.945 INFO:teuthology.orchestra.run.smithi052.stdout:(85/176): qatlib-service-23.11.0-1.el9.x86_64.r 678 kB/s | 34 kB 00:00 2024-03-04T09:18:21.962 INFO:teuthology.orchestra.run.smithi191.stdout:(149/176): re2-20211101-3.el9.x86_64.rpm 2.5 MB/s | 192 kB 00:00 2024-03-04T09:18:21.971 INFO:teuthology.orchestra.run.smithi052.stdout:(86/176): qatzip-libs-1.1.2-2.el9.x86_64.rpm 866 kB/s | 43 kB 00:00 2024-03-04T09:18:21.996 INFO:teuthology.orchestra.run.smithi052.stdout:(87/176): socat-1.7.4.1-5.el9.x86_64.rpm 5.8 MB/s | 300 kB 00:00 2024-03-04T09:18:22.012 INFO:teuthology.orchestra.run.smithi191.stdout:(150/176): thrift-0.15.0-2.el9.x86_64.rpm 16 MB/s | 1.6 MB 00:00 2024-03-04T09:18:22.021 INFO:teuthology.orchestra.run.smithi052.stdout:(88/176): uuid-1.6.2-55.el9.x86_64.rpm 1.1 MB/s | 56 kB 00:00 2024-03-04T09:18:22.045 INFO:teuthology.orchestra.run.smithi191.stdout:(151/176): libacl-devel-2.3.1-4.el9.x86_64.rpm 2.0 MB/s | 68 kB 00:00 2024-03-04T09:18:22.047 INFO:teuthology.orchestra.run.smithi052.stdout:(89/176): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 1.7 MB/s | 87 kB 00:00 2024-03-04T09:18:22.072 INFO:teuthology.orchestra.run.smithi052.stdout:(90/176): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.2 MB/s | 63 kB 00:00 2024-03-04T09:18:22.097 INFO:teuthology.orchestra.run.smithi052.stdout:(91/176): zlib-devel-1.2.11-40.el9.x86_64.rpm 879 kB/s | 44 kB 00:00 2024-03-04T09:18:22.123 INFO:teuthology.orchestra.run.smithi052.stdout:(92/176): fuse-2.9.9-15.el9.x86_64.rpm 1.5 MB/s | 79 kB 00:00 2024-03-04T09:18:22.129 INFO:teuthology.orchestra.run.smithi191.stdout:(152/176): NetworkManager-1.46.0-1.el9.x86_64.r 27 MB/s | 2.3 MB 00:00 2024-03-04T09:18:22.148 INFO:teuthology.orchestra.run.smithi052.stdout:(93/176): jq-1.6-16.el9.x86_64.rpm 3.6 MB/s | 186 kB 00:00 2024-03-04T09:18:22.182 INFO:teuthology.orchestra.run.smithi052.stdout:(94/176): libgfortran-11.4.1-3.el9.x86_64.rpm 13 MB/s | 802 kB 00:00 2024-03-04T09:18:22.205 INFO:teuthology.orchestra.run.smithi191.stdout:(153/176): NetworkManager-libnm-1.46.0-1.el9.x8 24 MB/s | 1.8 MB 00:00 2024-03-04T09:18:22.207 INFO:teuthology.orchestra.run.smithi052.stdout:(95/176): libquadmath-11.4.1-3.el9.x86_64.rpm 3.3 MB/s | 194 kB 00:00 2024-03-04T09:18:22.232 INFO:teuthology.orchestra.run.smithi052.stdout:(96/176): oniguruma-6.9.6-1.el9.5.x86_64.rpm 4.3 MB/s | 217 kB 00:00 2024-03-04T09:18:22.238 INFO:teuthology.orchestra.run.smithi191.stdout:(154/176): NetworkManager-team-1.46.0-1.el9.x86 1.2 MB/s | 41 kB 00:00 2024-03-04T09:18:22.258 INFO:teuthology.orchestra.run.smithi052.stdout:(97/176): python3-cffi-1.14.5-5.el9.x86_64.rpm 4.7 MB/s | 241 kB 00:00 2024-03-04T09:18:22.300 INFO:teuthology.orchestra.run.smithi052.stdout:(98/176): python3-cryptography-36.0.1-4.el9.x86 17 MB/s | 1.1 MB 00:00 2024-03-04T09:18:22.322 INFO:teuthology.orchestra.run.smithi191.stdout:(155/176): NetworkManager-tui-1.46.0-1.el9.x86_ 2.9 MB/s | 247 kB 00:00 2024-03-04T09:18:22.355 INFO:teuthology.orchestra.run.smithi191.stdout:(156/176): acl-2.3.1-4.el9.x86_64.rpm 2.0 MB/s | 69 kB 00:00 2024-03-04T09:18:22.389 INFO:teuthology.orchestra.run.smithi191.stdout:(157/176): elfutils-debuginfod-client-0.190-2.e 1.0 MB/s | 35 kB 00:00 2024-03-04T09:18:22.422 INFO:teuthology.orchestra.run.smithi191.stdout:(158/176): elfutils-libelf-0.190-2.el9.x86_64.r 5.6 MB/s | 191 kB 00:00 2024-03-04T09:18:22.450 INFO:teuthology.orchestra.run.smithi052.stdout:(99/176): python3-ply-3.11-14.el9.noarch.rpm 543 kB/s | 103 kB 00:00 2024-03-04T09:18:22.464 INFO:teuthology.orchestra.run.smithi191.stdout:(159/176): elfutils-libs-0.190-2.el9.x86_64.rpm 6.0 MB/s | 254 kB 00:00 2024-03-04T09:18:22.475 INFO:teuthology.orchestra.run.smithi052.stdout:(100/176): python3-pycparser-2.20-6.el9.noarch. 708 kB/s | 124 kB 00:00 2024-03-04T09:18:22.501 INFO:teuthology.orchestra.run.smithi052.stdout:(101/176): python3-requests-2.25.1-8.el9.noarch 2.2 MB/s | 113 kB 00:00 2024-03-04T09:18:22.514 INFO:teuthology.orchestra.run.smithi191.stdout:(160/176): iproute-6.7.0-2.el9.x86_64.rpm 16 MB/s | 802 kB 00:00 2024-03-04T09:18:22.526 INFO:teuthology.orchestra.run.smithi052.stdout:(102/176): python3-urllib3-1.26.5-5.el9.noarch. 3.7 MB/s | 187 kB 00:00 2024-03-04T09:18:22.556 INFO:teuthology.orchestra.run.smithi191.stdout:(161/176): iproute-tc-6.7.0-2.el9.x86_64.rpm 10 MB/s | 433 kB 00:00 2024-03-04T09:18:22.590 INFO:teuthology.orchestra.run.smithi191.stdout:(162/176): libacl-2.3.1-4.el9.x86_64.rpm 663 kB/s | 22 kB 00:00 2024-03-04T09:18:22.623 INFO:teuthology.orchestra.run.smithi191.stdout:(163/176): libblkid-2.37.4-18.el9.x86_64.rpm 3.1 MB/s | 107 kB 00:00 2024-03-04T09:18:22.657 INFO:teuthology.orchestra.run.smithi191.stdout:(164/176): libfdisk-2.37.4-18.el9.x86_64.rpm 4.5 MB/s | 154 kB 00:00 2024-03-04T09:18:22.676 INFO:teuthology.orchestra.run.smithi052.stdout:(103/176): python3-devel-3.9.17-2.el9.x86_64.rp 1.4 MB/s | 245 kB 00:00 2024-03-04T09:18:22.691 INFO:teuthology.orchestra.run.smithi191.stdout:(165/176): libmount-2.37.4-18.el9.x86_64.rpm 4.0 MB/s | 135 kB 00:00 2024-03-04T09:18:22.726 INFO:teuthology.orchestra.run.smithi052.stdout:(104/176): lua-devel-5.4.4-4.el9.x86_64.rpm 111 kB/s | 22 kB 00:00 2024-03-04T09:18:22.741 INFO:teuthology.orchestra.run.smithi191.stdout:(166/176): librados2-19.0.0-1739.g1bc250c7.el9. 4.2 MB/s | 3.4 MB 00:00 2024-03-04T09:18:22.767 INFO:teuthology.orchestra.run.smithi191.stdout:(167/176): libsmartcols-2.37.4-18.el9.x86_64.rp 834 kB/s | 63 kB 00:00 2024-03-04T09:18:22.801 INFO:teuthology.orchestra.run.smithi191.stdout:(168/176): libuuid-2.37.4-18.el9.x86_64.rpm 471 kB/s | 28 kB 00:00 2024-03-04T09:18:22.834 INFO:teuthology.orchestra.run.smithi191.stdout:(169/176): openssl-3.0.7-27.el9.x86_64.rpm 17 MB/s | 1.2 MB 00:00 2024-03-04T09:18:22.884 INFO:teuthology.orchestra.run.smithi191.stdout:(170/176): openssl-libs-3.0.7-27.el9.x86_64.rpm 26 MB/s | 2.2 MB 00:00 2024-03-04T09:18:22.893 INFO:teuthology.orchestra.run.smithi052.stdout:(105/176): protobuf-compiler-3.14.0-13.el9.x86_ 3.9 MB/s | 863 kB 00:00 2024-03-04T09:18:22.942 INFO:teuthology.orchestra.run.smithi191.stdout:(171/176): util-linux-2.37.4-18.el9.x86_64.rpm 21 MB/s | 2.2 MB 00:00 2024-03-04T09:18:22.968 INFO:teuthology.orchestra.run.smithi191.stdout:(172/176): util-linux-core-2.37.4-18.el9.x86_64 5.3 MB/s | 438 kB 00:00 2024-03-04T09:18:22.993 INFO:teuthology.orchestra.run.smithi191.stdout:(173/176): util-linux-user-2.37.4-18.el9.x86_64 606 kB/s | 30 kB 00:00 2024-03-04T09:18:23.019 INFO:teuthology.orchestra.run.smithi191.stdout:(174/176): xfsdump-3.1.12-4.el9.x86_64.rpm 6.3 MB/s | 321 kB 00:00 2024-03-04T09:18:23.052 INFO:teuthology.orchestra.run.smithi191.stdout:(175/176): xfsprogs-6.3.0-1.el9.x86_64.rpm 18 MB/s | 1.1 MB 00:00 2024-03-04T09:18:23.136 INFO:teuthology.orchestra.run.smithi191.stdout:(176/176): librbd1-19.0.0-1739.g1bc250c7.el9.x8 2.7 MB/s | 3.1 MB 00:01 2024-03-04T09:18:23.143 INFO:teuthology.orchestra.run.smithi191.stdout:-------------------------------------------------------------------------------- 2024-03-04T09:18:23.143 INFO:teuthology.orchestra.run.smithi191.stdout:Total 15 MB/s | 224 MB 00:15 2024-03-04T09:18:23.187 INFO:teuthology.orchestra.run.smithi191.stdout:Running transaction check 2024-03-04T09:18:23.235 INFO:teuthology.orchestra.run.smithi052.stdout:(106/176): abseil-cpp-20211102.0-3.el9.x86_64.r 1.1 MB/s | 548 kB 00:00 2024-03-04T09:18:23.310 INFO:teuthology.orchestra.run.smithi052.stdout:(107/176): grpc-data-1.46.7-10.el9.noarch.rpm 260 kB/s | 19 kB 00:00 2024-03-04T09:18:23.336 INFO:teuthology.orchestra.run.smithi052.stdout:(108/176): gperftools-libs-2.9.1-2.el9.x86_64.r 700 kB/s | 309 kB 00:00 2024-03-04T09:18:23.411 INFO:teuthology.orchestra.run.smithi052.stdout:(109/176): libarrow-doc-9.0.0-10.el9.noarch.rpm 347 kB/s | 26 kB 00:00 2024-03-04T09:18:23.463 INFO:teuthology.orchestra.run.smithi191.stdout:Transaction check succeeded. 2024-03-04T09:18:23.463 INFO:teuthology.orchestra.run.smithi191.stdout:Running transaction test 2024-03-04T09:18:23.495 INFO:teuthology.orchestra.run.smithi052.stdout:(110/176): liboath-2.6.7-2.el9.x86_64.rpm 583 kB/s | 49 kB 00:00 2024-03-04T09:18:23.612 INFO:teuthology.orchestra.run.smithi052.stdout:(111/176): libarrow-9.0.0-10.el9.x86_64.rpm 15 MB/s | 4.4 MB 00:00 2024-03-04T09:18:23.629 INFO:teuthology.orchestra.run.smithi052.stdout:(112/176): libunwind-1.6.2-1.el9.x86_64.rpm 504 kB/s | 67 kB 00:00 2024-03-04T09:18:23.687 INFO:teuthology.orchestra.run.smithi052.stdout:(113/176): luarocks-3.9.2-1.el9.noarch.rpm 2.0 MB/s | 151 kB 00:00 2024-03-04T09:18:23.771 INFO:teuthology.orchestra.run.smithi052.stdout:(114/176): python3-asyncssh-2.13.2-4.el9.noarch 6.4 MB/s | 548 kB 00:00 2024-03-04T09:18:23.813 INFO:teuthology.orchestra.run.smithi052.stdout:(115/176): parquet-libs-9.0.0-10.el9.x86_64.rpm 4.5 MB/s | 838 kB 00:00 2024-03-04T09:18:23.900 INFO:teuthology.orchestra.run.smithi052.stdout:(116/176): python3-bcrypt-3.2.2-1.el9.x86_64.rp 338 kB/s | 43 kB 00:00 2024-03-04T09:18:23.955 INFO:teuthology.orchestra.run.smithi052.stdout:(117/176): python3-cachetools-4.2.4-1.el9.noarc 227 kB/s | 32 kB 00:00 2024-03-04T09:18:23.980 INFO:teuthology.orchestra.run.smithi052.stdout:(118/176): python3-certifi-2021.10.8-2.el9.noar 186 kB/s | 15 kB 00:00 2024-03-04T09:18:24.031 INFO:teuthology.orchestra.run.smithi052.stdout:(119/176): python3-cheroot-8.6.0-4.el9.noarch.r 2.2 MB/s | 172 kB 00:00 2024-03-04T09:18:24.064 INFO:teuthology.orchestra.run.smithi052.stdout:(120/176): python3-cherrypy-18.6.1-2.el9.noarch 4.2 MB/s | 358 kB 00:00 2024-03-04T09:18:24.107 INFO:teuthology.orchestra.run.smithi052.stdout:(121/176): python3-cmd2-2.4.2-2.el9.noarch.rpm 3.9 MB/s | 303 kB 00:00 2024-03-04T09:18:24.148 INFO:teuthology.orchestra.run.smithi052.stdout:(122/176): python3-colorama-0.4.4-12.el9.noarch 421 kB/s | 35 kB 00:00 2024-03-04T09:18:24.182 INFO:teuthology.orchestra.run.smithi052.stdout:(123/176): python3-google-auth-2.28.0-1.el9.noa 2.6 MB/s | 203 kB 00:00 2024-03-04T09:18:24.257 INFO:teuthology.orchestra.run.smithi052.stdout:(124/176): python3-grpcio-tools-1.46.7-10.el9.x 1.9 MB/s | 144 kB 00:00 2024-03-04T09:18:24.316 INFO:teuthology.orchestra.run.smithi052.stdout:(125/176): python3-grpcio-1.46.7-10.el9.x86_64. 12 MB/s | 2.0 MB 00:00 2024-03-04T09:18:24.341 INFO:teuthology.orchestra.run.smithi052.stdout:(126/176): python3-jaraco-8.2.1-3.el9.noarch.rp 127 kB/s | 11 kB 00:00 2024-03-04T09:18:24.383 INFO:teuthology.orchestra.run.smithi052.stdout:(127/176): python3-jaraco-classes-3.2.1-5.el9.n 265 kB/s | 18 kB 00:00 2024-03-04T09:18:24.409 INFO:teuthology.orchestra.run.smithi052.stdout:(128/176): python3-jaraco-collections-3.0.0-8.e 346 kB/s | 23 kB 00:00 2024-03-04T09:18:24.451 INFO:teuthology.orchestra.run.smithi052.stdout:(129/176): python3-jaraco-functools-3.5.0-2.el9 290 kB/s | 19 kB 00:00 2024-03-04T09:18:24.476 INFO:teuthology.orchestra.run.smithi052.stdout:(130/176): python3-jaraco-text-3.2.0-6.el9.noar 294 kB/s | 20 kB 00:00 2024-03-04T09:18:24.534 INFO:teuthology.orchestra.run.smithi052.stdout:(131/176): python3-kubernetes-26.1.0-2.el9.noar 12 MB/s | 1.0 MB 00:00 2024-03-04T09:18:24.552 INFO:teuthology.orchestra.run.smithi052.stdout:(132/176): python3-logutils-0.3.5-21.el9.noarch 616 kB/s | 46 kB 00:00 2024-03-04T09:18:24.610 INFO:teuthology.orchestra.run.smithi052.stdout:(133/176): python3-more-itertools-8.12.0-2.el9. 1.0 MB/s | 79 kB 00:00 2024-03-04T09:18:24.635 INFO:teuthology.orchestra.run.smithi052.stdout:(134/176): python3-natsort-7.1.1-5.el9.noarch.r 691 kB/s | 58 kB 00:00 2024-03-04T09:18:24.686 INFO:teuthology.orchestra.run.smithi052.stdout:(135/176): python3-pecan-1.4.2-3.el9.noarch.rpm 3.5 MB/s | 272 kB 00:00 2024-03-04T09:18:24.711 INFO:teuthology.orchestra.run.smithi052.stdout:(136/176): python3-portend-3.1.0-2.el9.noarch.r 218 kB/s | 16 kB 00:00 2024-03-04T09:18:25.122 INFO:teuthology.orchestra.run.smithi191.stdout:Transaction test succeeded. 2024-03-04T09:18:25.122 INFO:teuthology.orchestra.run.smithi191.stdout:Running transaction 2024-03-04T09:18:25.128 INFO:teuthology.orchestra.run.smithi052.stdout:(137/176): ceph-test-19.0.0-1739.g1bc250c7.el9. 5.7 MB/s | 47 MB 00:08 2024-03-04T09:18:25.153 INFO:teuthology.orchestra.run.smithi052.stdout:(138/176): python3-pyperclip-1.8.2-2.el9.noarch 64 kB/s | 28 kB 00:00 2024-03-04T09:18:25.178 INFO:teuthology.orchestra.run.smithi052.stdout:(139/176): python3-pyOpenSSL-21.0.0-1.el9.noarc 183 kB/s | 90 kB 00:00 2024-03-04T09:18:25.229 INFO:teuthology.orchestra.run.smithi052.stdout:(140/176): python3-routes-2.5.1-5.el9.noarch.rp 2.4 MB/s | 188 kB 00:00 2024-03-04T09:18:25.279 INFO:teuthology.orchestra.run.smithi052.stdout:(141/176): python3-rsa-4.9-2.el9.noarch.rpm 819 kB/s | 59 kB 00:00 2024-03-04T09:18:25.304 INFO:teuthology.orchestra.run.smithi052.stdout:(142/176): python3-tempora-5.0.0-2.el9.noarch.r 474 kB/s | 36 kB 00:00 2024-03-04T09:18:25.447 INFO:teuthology.orchestra.run.smithi052.stdout:(143/176): python3-typing-extensions-4.4.0-2.el 306 kB/s | 51 kB 00:00 2024-03-04T09:18:25.471 INFO:teuthology.orchestra.run.smithi052.stdout:(144/176): python3-repoze-lru-0.7-16.el9.noarch 90 kB/s | 31 kB 00:00 2024-03-04T09:18:25.497 INFO:teuthology.orchestra.run.smithi052.stdout:(145/176): python3-webob-1.8.7-6.el9.noarch.rpm 1.2 MB/s | 230 kB 00:00 2024-03-04T09:18:25.522 INFO:teuthology.orchestra.run.smithi052.stdout:(146/176): python3-websocket-client-1.2.3-2.el9 1.2 MB/s | 90 kB 00:00 2024-03-04T09:18:25.573 INFO:teuthology.orchestra.run.smithi052.stdout:(147/176): python3-zc-lockfile-2.0-10.el9.noarc 265 kB/s | 20 kB 00:00 2024-03-04T09:18:25.598 INFO:teuthology.orchestra.run.smithi052.stdout:(148/176): re2-20211101-3.el9.x86_64.rpm 2.5 MB/s | 192 kB 00:00 2024-03-04T09:18:25.681 INFO:teuthology.orchestra.run.smithi052.stdout:(149/176): thrift-0.15.0-2.el9.x86_64.rpm 15 MB/s | 1.6 MB 00:00 2024-03-04T09:18:25.765 INFO:teuthology.orchestra.run.smithi052.stdout:(150/176): python3-werkzeug-2.0.3-3.el9.1.noarc 1.4 MB/s | 427 kB 00:00 2024-03-04T09:18:25.798 INFO:teuthology.orchestra.run.smithi052.stdout:(151/176): libacl-devel-2.3.1-4.el9.x86_64.rpm 2.0 MB/s | 68 kB 00:00 2024-03-04T09:18:25.890 INFO:teuthology.orchestra.run.smithi052.stdout:(152/176): NetworkManager-1.46.0-1.el9.x86_64.r 25 MB/s | 2.3 MB 00:00 2024-03-04T09:18:25.937 INFO:teuthology.orchestra.run.smithi106.stdout: Preparing : 1/1 2024-03-04T09:18:25.974 INFO:teuthology.orchestra.run.smithi052.stdout:(153/176): NetworkManager-libnm-1.46.0-1.el9.x8 22 MB/s | 1.8 MB 00:00 2024-03-04T09:18:26.008 INFO:teuthology.orchestra.run.smithi052.stdout:(154/176): NetworkManager-team-1.46.0-1.el9.x86 1.2 MB/s | 41 kB 00:00 2024-03-04T09:18:26.040 INFO:teuthology.orchestra.run.smithi106.stdout: Upgrading : openssl-libs-1:3.0.7-27.el9.x86_64 1/202 2024-03-04T09:18:26.041 INFO:teuthology.orchestra.run.smithi052.stdout:(155/176): NetworkManager-tui-1.46.0-1.el9.x86_ 7.2 MB/s | 247 kB 00:00 2024-03-04T09:18:26.075 INFO:teuthology.orchestra.run.smithi106.stdout: Upgrading : libuuid-2.37.4-18.el9.x86_64 2/202 2024-03-04T09:18:26.083 INFO:teuthology.orchestra.run.smithi052.stdout:(156/176): acl-2.3.1-4.el9.x86_64.rpm 1.6 MB/s | 69 kB 00:00 2024-03-04T09:18:26.117 INFO:teuthology.orchestra.run.smithi052.stdout:(157/176): elfutils-debuginfod-client-0.190-2.e 1.0 MB/s | 35 kB 00:00 2024-03-04T09:18:26.150 INFO:teuthology.orchestra.run.smithi052.stdout:(158/176): elfutils-libelf-0.190-2.el9.x86_64.r 5.6 MB/s | 191 kB 00:00 2024-03-04T09:18:26.187 INFO:teuthology.orchestra.run.smithi106.stdout: Upgrading : libblkid-2.37.4-18.el9.x86_64 3/202 2024-03-04T09:18:26.201 INFO:teuthology.orchestra.run.smithi052.stdout:(159/176): librbd1-19.0.0-1739.g1bc250c7.el9.x8 6.0 MB/s | 3.1 MB 00:00 2024-03-04T09:18:26.212 INFO:teuthology.orchestra.run.smithi106.stdout: Running scriptlet: libblkid-2.37.4-18.el9.x86_64 3/202 2024-03-04T09:18:26.226 INFO:teuthology.orchestra.run.smithi052.stdout:(160/176): elfutils-libs-0.190-2.el9.x86_64.rpm 3.3 MB/s | 254 kB 00:00 2024-03-04T09:18:26.251 INFO:teuthology.orchestra.run.smithi052.stdout:(161/176): iproute-6.7.0-2.el9.x86_64.rpm 16 MB/s | 802 kB 00:00 2024-03-04T09:18:26.259 INFO:teuthology.orchestra.run.smithi106.stdout: Upgrading : elfutils-libelf-0.190-2.el9.x86_64 4/202 2024-03-04T09:18:26.285 INFO:teuthology.orchestra.run.smithi052.stdout:(162/176): iproute-tc-6.7.0-2.el9.x86_64.rpm 7.4 MB/s | 433 kB 00:00 2024-03-04T09:18:26.298 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 5/202 2024-03-04T09:18:26.310 INFO:teuthology.orchestra.run.smithi052.stdout:(163/176): libacl-2.3.1-4.el9.x86_64.rpm 380 kB/s | 22 kB 00:00 2024-03-04T09:18:26.335 INFO:teuthology.orchestra.run.smithi052.stdout:(164/176): libblkid-2.37.4-18.el9.x86_64.rpm 2.1 MB/s | 107 kB 00:00 2024-03-04T09:18:26.351 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 6/202 2024-03-04T09:18:26.361 INFO:teuthology.orchestra.run.smithi052.stdout:(165/176): libfdisk-2.37.4-18.el9.x86_64.rpm 3.0 MB/s | 154 kB 00:00 2024-03-04T09:18:26.385 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : thrift-0.15.0-2.el9.x86_64 7/202 2024-03-04T09:18:26.386 INFO:teuthology.orchestra.run.smithi052.stdout:(166/176): libmount-2.37.4-18.el9.x86_64.rpm 2.6 MB/s | 135 kB 00:00 2024-03-04T09:18:26.403 INFO:teuthology.orchestra.run.smithi052.stdout:(167/176): libsmartcols-2.37.4-18.el9.x86_64.rp 1.5 MB/s | 63 kB 00:00 2024-03-04T09:18:26.410 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 8/202 2024-03-04T09:18:26.420 INFO:teuthology.orchestra.run.smithi052.stdout:(168/176): libuuid-2.37.4-18.el9.x86_64.rpm 831 kB/s | 28 kB 00:00 2024-03-04T09:18:26.452 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 9/202 2024-03-04T09:18:26.462 INFO:teuthology.orchestra.run.smithi052.stdout:(169/176): openssl-3.0.7-27.el9.x86_64.rpm 20 MB/s | 1.2 MB 00:00 2024-03-04T09:18:26.512 INFO:teuthology.orchestra.run.smithi052.stdout:(170/176): openssl-libs-3.0.7-27.el9.x86_64.rpm 24 MB/s | 2.2 MB 00:00 2024-03-04T09:18:26.562 INFO:teuthology.orchestra.run.smithi052.stdout:(171/176): util-linux-2.37.4-18.el9.x86_64.rpm 23 MB/s | 2.2 MB 00:00 2024-03-04T09:18:26.588 INFO:teuthology.orchestra.run.smithi052.stdout:(172/176): util-linux-core-2.37.4-18.el9.x86_64 5.9 MB/s | 438 kB 00:00 2024-03-04T09:18:26.613 INFO:teuthology.orchestra.run.smithi052.stdout:(173/176): util-linux-user-2.37.4-18.el9.x86_64 608 kB/s | 30 kB 00:00 2024-03-04T09:18:26.672 INFO:teuthology.orchestra.run.smithi052.stdout:(174/176): librados2-19.0.0-1739.g1bc250c7.el9. 3.2 MB/s | 3.4 MB 00:01 2024-03-04T09:18:26.697 INFO:teuthology.orchestra.run.smithi052.stdout:(175/176): xfsdump-3.1.12-4.el9.x86_64.rpm 2.9 MB/s | 321 kB 00:00 2024-03-04T09:18:26.747 INFO:teuthology.orchestra.run.smithi052.stdout:(176/176): xfsprogs-6.3.0-1.el9.x86_64.rpm 7.9 MB/s | 1.1 MB 00:00 2024-03-04T09:18:26.754 INFO:teuthology.orchestra.run.smithi052.stdout:-------------------------------------------------------------------------------- 2024-03-04T09:18:26.754 INFO:teuthology.orchestra.run.smithi052.stdout:Total 14 MB/s | 224 MB 00:16 2024-03-04T09:18:26.772 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 10/202 2024-03-04T09:18:26.801 INFO:teuthology.orchestra.run.smithi052.stdout:Running transaction check 2024-03-04T09:18:26.828 INFO:teuthology.orchestra.run.smithi106.stdout: Upgrading : librados2-2:19.0.0-1739.g1bc250c7.el9.x86_64 11/202 2024-03-04T09:18:26.950 INFO:teuthology.orchestra.run.smithi106.stdout: Running scriptlet: librados2-2:19.0.0-1739.g1bc250c7.el9.x86_64 11/202 2024-03-04T09:18:26.978 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : libcephfs2-2:19.0.0-1739.g1bc250c7.el9.x86_64 12/202 2024-03-04T09:18:27.062 INFO:teuthology.orchestra.run.smithi106.stdout: Running scriptlet: libcephfs2-2:19.0.0-1739.g1bc250c7.el9.x86_64 12/202 2024-03-04T09:18:27.091 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-rados-2:19.0.0-1739.g1bc250c7.el9.x86_64 13/202 2024-03-04T09:18:27.141 INFO:teuthology.orchestra.run.smithi052.stdout:Transaction check succeeded. 2024-03-04T09:18:27.141 INFO:teuthology.orchestra.run.smithi052.stdout:Running transaction test 2024-03-04T09:18:27.240 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : postgresql-private-libs-13.11-1.el9.x86_64 14/202 2024-03-04T09:18:27.288 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : postgresql-13.11-1.el9.x86_64 15/202 2024-03-04T09:18:27.316 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : libcephsqlite-2:19.0.0-1739.g1bc250c7.el9.x86_64 16/202 2024-03-04T09:18:27.420 INFO:teuthology.orchestra.run.smithi106.stdout: Running scriptlet: libcephsqlite-2:19.0.0-1739.g1bc250c7.el9.x86_64 16/202 2024-03-04T09:18:27.442 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : libradosstriper1-2:19.0.0-1739.g1bc250c7.el9.x86 17/202 2024-03-04T09:18:27.506 INFO:teuthology.orchestra.run.smithi106.stdout: Running scriptlet: libradosstriper1-2:19.0.0-1739.g1bc250c7.el9.x86 17/202 2024-03-04T09:18:27.545 INFO:teuthology.orchestra.run.smithi106.stdout: Upgrading : libmount-2.37.4-18.el9.x86_64 18/202 2024-03-04T09:18:27.568 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : qatlib-service-23.11.0-1.el9.x86_64 19/202 2024-03-04T09:18:27.584 INFO:teuthology.orchestra.run.smithi106.stdout: Running scriptlet: qatlib-service-23.11.0-1.el9.x86_64 19/202 2024-03-04T09:18:27.776 INFO:teuthology.orchestra.run.smithi106.stdout: Running scriptlet: qatlib-23.11.0-1.el9.x86_64 20/202 2024-03-04T09:18:27.807 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : qatlib-23.11.0-1.el9.x86_64 20/202 2024-03-04T09:18:27.847 INFO:teuthology.orchestra.run.smithi106.stdout: Upgrading : libsmartcols-2.37.4-18.el9.x86_64 21/202 2024-03-04T09:18:27.970 INFO:teuthology.orchestra.run.smithi106.stdout: Upgrading : libacl-2.3.1-4.el9.x86_64 22/202 2024-03-04T09:18:28.066 INFO:teuthology.orchestra.run.smithi106.stdout: Upgrading : NetworkManager-libnm-1:1.46.0-1.el9.x86_64 23/202 2024-03-04T09:18:28.118 INFO:teuthology.orchestra.run.smithi106.stdout: Running scriptlet: NetworkManager-libnm-1:1.46.0-1.el9.x86_64 23/202 2024-03-04T09:18:28.191 INFO:teuthology.orchestra.run.smithi106.stdout: Running scriptlet: NetworkManager-1:1.46.0-1.el9.x86_64 24/202 2024-03-04T09:18:28.244 INFO:teuthology.orchestra.run.smithi106.stdout: Upgrading : NetworkManager-1:1.46.0-1.el9.x86_64 24/202 2024-03-04T09:18:28.471 INFO:teuthology.orchestra.run.smithi106.stdout: Running scriptlet: NetworkManager-1:1.46.0-1.el9.x86_64 24/202 2024-03-04T09:18:28.588 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : re2-1:20211101-3.el9.x86_64 25/202 2024-03-04T09:18:28.677 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 26/202 2024-03-04T09:18:28.708 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 27/202 2024-03-04T09:18:28.776 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : liboath-2.6.7-2.el9.x86_64 28/202 2024-03-04T09:18:28.830 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 29/202 2024-03-04T09:18:28.861 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 30/202 2024-03-04T09:18:28.896 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 31/202 2024-03-04T09:18:28.936 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 32/202 2024-03-04T09:18:29.000 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 33/202 2024-03-04T09:18:29.035 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 34/202 2024-03-04T09:18:29.079 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 35/202 2024-03-04T09:18:29.111 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 36/202 2024-03-04T09:18:29.139 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : lua-5.4.4-4.el9.x86_64 37/202 2024-03-04T09:18:29.187 INFO:teuthology.orchestra.run.smithi052.stdout:Transaction test succeeded. 2024-03-04T09:18:29.187 INFO:teuthology.orchestra.run.smithi052.stdout:Running transaction 2024-03-04T09:18:29.209 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 38/202 2024-03-04T09:18:29.253 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-ceph-common-2:19.0.0-1739.g1bc250c7.el9. 39/202 2024-03-04T09:18:29.287 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-ceph-argparse-2:19.0.0-1739.g1bc250c7.el 40/202 2024-03-04T09:18:29.316 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-cephfs-2:19.0.0-1739.g1bc250c7.el9.x86_6 41/202 2024-03-04T09:18:29.384 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 42/202 2024-03-04T09:18:29.431 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : luarocks-3.9.2-1.el9.noarch 43/202 2024-03-04T09:18:29.504 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 44/202 2024-03-04T09:18:29.637 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 45/202 2024-03-04T09:18:29.700 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 46/202 2024-03-04T09:18:30.214 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-rsa-4.9-2.el9.noarch 47/202 2024-03-04T09:18:30.294 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 48/202 2024-03-04T09:18:30.372 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 49/202 2024-03-04T09:18:30.423 INFO:teuthology.orchestra.run.smithi106.stdout: Upgrading : util-linux-core-2.37.4-18.el9.x86_64 50/202 2024-03-04T09:18:30.431 INFO:teuthology.orchestra.run.smithi106.stdout: Running scriptlet: util-linux-core-2.37.4-18.el9.x86_64 50/202 2024-03-04T09:18:30.469 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : qatzip-libs-1.1.2-2.el9.x86_64 51/202 2024-03-04T09:18:30.507 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : librados-devel-2:19.0.0-1739.g1bc250c7.el9.x86_6 52/202 2024-03-04T09:18:30.539 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 53/202 2024-03-04T09:18:30.627 INFO:teuthology.orchestra.run.smithi106.stdout: Upgrading : elfutils-libs-0.190-2.el9.x86_64 54/202 2024-03-04T09:18:30.689 INFO:teuthology.orchestra.run.smithi106.stdout: Upgrading : iproute-6.7.0-2.el9.x86_64 55/202 2024-03-04T09:18:30.930 INFO:teuthology.orchestra.run.smithi106.stdout: Upgrading : libfdisk-2.37.4-18.el9.x86_64 56/202 2024-03-04T09:18:31.093 INFO:teuthology.orchestra.run.smithi106.stdout: Upgrading : util-linux-2.37.4-18.el9.x86_64 57/202 2024-03-04T09:18:31.158 INFO:teuthology.orchestra.run.smithi106.stdout: Upgrading : xfsprogs-6.3.0-1.el9.x86_64 58/202 2024-03-04T09:18:31.203 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 59/202 2024-03-04T09:18:31.339 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 60/202 2024-03-04T09:18:31.380 INFO:teuthology.orchestra.run.smithi106.stdout: Upgrading : openssl-1:3.0.7-27.el9.x86_64 61/202 2024-03-04T09:18:31.518 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 62/202 2024-03-04T09:18:31.726 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 63/202 2024-03-04T09:18:31.756 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 64/202 2024-03-04T09:18:31.793 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 65/202 2024-03-04T09:18:31.839 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 66/202 2024-03-04T09:18:31.875 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 67/202 2024-03-04T09:18:31.918 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 68/202 2024-03-04T09:18:31.960 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 69/202 2024-03-04T09:18:32.093 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 70/202 2024-03-04T09:18:32.140 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 71/202 2024-03-04T09:18:32.171 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 72/202 2024-03-04T09:18:32.194 INFO:teuthology.orchestra.run.smithi191.stdout: Preparing : 1/1 2024-03-04T09:18:32.212 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 73/202 2024-03-04T09:18:32.306 INFO:teuthology.orchestra.run.smithi191.stdout: Upgrading : openssl-libs-1:3.0.7-27.el9.x86_64 1/202 2024-03-04T09:18:32.306 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 74/202 2024-03-04T09:18:32.348 INFO:teuthology.orchestra.run.smithi191.stdout: Upgrading : libuuid-2.37.4-18.el9.x86_64 2/202 2024-03-04T09:18:32.354 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-google-auth-1:2.28.0-1.el9.noarch 75/202 2024-03-04T09:18:32.379 INFO:teuthology.orchestra.run.smithi191.stdout: Upgrading : libblkid-2.37.4-18.el9.x86_64 3/202 2024-03-04T09:18:32.394 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 76/202 2024-03-04T09:18:32.402 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: libblkid-2.37.4-18.el9.x86_64 3/202 2024-03-04T09:18:32.420 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 77/202 2024-03-04T09:18:32.452 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 78/202 2024-03-04T09:18:32.457 INFO:teuthology.orchestra.run.smithi191.stdout: Upgrading : elfutils-libelf-0.190-2.el9.x86_64 4/202 2024-03-04T09:18:32.496 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 5/202 2024-03-04T09:18:32.523 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 79/202 2024-03-04T09:18:32.557 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 6/202 2024-03-04T09:18:32.591 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : thrift-0.15.0-2.el9.x86_64 7/202 2024-03-04T09:18:32.617 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 8/202 2024-03-04T09:18:32.644 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 80/202 2024-03-04T09:18:32.659 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 9/202 2024-03-04T09:18:32.744 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 81/202 2024-03-04T09:18:32.785 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 82/202 2024-03-04T09:18:32.833 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-ply-3.11-14.el9.noarch 83/202 2024-03-04T09:18:32.888 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 84/202 2024-03-04T09:18:32.980 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 10/202 2024-03-04T09:18:33.044 INFO:teuthology.orchestra.run.smithi191.stdout: Upgrading : librados2-2:19.0.0-1739.g1bc250c7.el9.x86_64 11/202 2024-03-04T09:18:33.080 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 85/202 2024-03-04T09:18:33.130 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 86/202 2024-03-04T09:18:33.165 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: librados2-2:19.0.0-1739.g1bc250c7.el9.x86_64 11/202 2024-03-04T09:18:33.194 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : libcephfs2-2:19.0.0-1739.g1bc250c7.el9.x86_64 12/202 2024-03-04T09:18:33.258 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 87/202 2024-03-04T09:18:33.285 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: libcephfs2-2:19.0.0-1739.g1bc250c7.el9.x86_64 12/202 2024-03-04T09:18:33.303 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-asyncssh-2.13.2-4.el9.noarch 88/202 2024-03-04T09:18:33.315 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-rados-2:19.0.0-1739.g1bc250c7.el9.x86_64 13/202 2024-03-04T09:18:33.336 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 89/202 2024-03-04T09:18:33.368 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 90/202 2024-03-04T09:18:33.403 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : jq-1.6-16.el9.x86_64 91/202 2024-03-04T09:18:33.434 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : fuse-2.9.9-15.el9.x86_64 92/202 2024-03-04T09:18:33.464 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : postgresql-private-libs-13.11-1.el9.x86_64 14/202 2024-03-04T09:18:33.477 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : zlib-devel-1.2.11-40.el9.x86_64 93/202 2024-03-04T09:18:33.513 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 94/202 2024-03-04T09:18:33.528 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : postgresql-13.11-1.el9.x86_64 15/202 2024-03-04T09:18:33.552 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : uuid-1.6.2-55.el9.x86_64 95/202 2024-03-04T09:18:33.556 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : libcephsqlite-2:19.0.0-1739.g1bc250c7.el9.x86_64 16/202 2024-03-04T09:18:33.584 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 96/202 2024-03-04T09:18:33.614 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 97/202 2024-03-04T09:18:33.660 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: libcephsqlite-2:19.0.0-1739.g1bc250c7.el9.x86_64 16/202 2024-03-04T09:18:33.683 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : libradosstriper1-2:19.0.0-1739.g1bc250c7.el9.x86 17/202 2024-03-04T09:18:33.690 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 98/202 2024-03-04T09:18:33.733 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 99/202 2024-03-04T09:18:33.746 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: libradosstriper1-2:19.0.0-1739.g1bc250c7.el9.x86 17/202 2024-03-04T09:18:33.766 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 100/202 2024-03-04T09:18:33.786 INFO:teuthology.orchestra.run.smithi191.stdout: Upgrading : libmount-2.37.4-18.el9.x86_64 18/202 2024-03-04T09:18:33.807 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 101/202 2024-03-04T09:18:33.809 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : qatlib-service-23.11.0-1.el9.x86_64 19/202 2024-03-04T09:18:33.824 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: qatlib-service-23.11.0-1.el9.x86_64 19/202 2024-03-04T09:18:33.842 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 102/202 2024-03-04T09:18:33.931 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 103/202 2024-03-04T09:18:34.017 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: qatlib-23.11.0-1.el9.x86_64 20/202 2024-03-04T09:18:34.020 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 104/202 2024-03-04T09:18:34.048 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : qatlib-23.11.0-1.el9.x86_64 20/202 2024-03-04T09:18:34.080 INFO:teuthology.orchestra.run.smithi191.stdout: Upgrading : libsmartcols-2.37.4-18.el9.x86_64 21/202 2024-03-04T09:18:34.080 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 105/202 2024-03-04T09:18:34.113 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 106/202 2024-03-04T09:18:34.159 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 107/202 2024-03-04T09:18:34.184 INFO:teuthology.orchestra.run.smithi191.stdout: Upgrading : libacl-2.3.1-4.el9.x86_64 22/202 2024-03-04T09:18:34.255 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-packaging-20.9-5.el9.noarch 108/202 2024-03-04T09:18:34.273 INFO:teuthology.orchestra.run.smithi191.stdout: Upgrading : NetworkManager-libnm-1:1.46.0-1.el9.x86_64 23/202 2024-03-04T09:18:34.318 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 109/202 2024-03-04T09:18:34.359 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: NetworkManager-libnm-1:1.46.0-1.el9.x86_64 23/202 2024-03-04T09:18:34.421 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: NetworkManager-1:1.46.0-1.el9.x86_64 24/202 2024-03-04T09:18:34.494 INFO:teuthology.orchestra.run.smithi191.stdout: Upgrading : NetworkManager-1:1.46.0-1.el9.x86_64 24/202 2024-03-04T09:18:34.867 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: NetworkManager-1:1.46.0-1.el9.x86_64 24/202 2024-03-04T09:18:35.034 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 110/202 2024-03-04T09:18:35.086 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 111/202 2024-03-04T09:18:35.108 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : re2-1:20211101-3.el9.x86_64 25/202 2024-03-04T09:18:35.141 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 112/202 2024-03-04T09:18:35.192 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 26/202 2024-03-04T09:18:35.193 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 113/202 2024-03-04T09:18:35.232 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 27/202 2024-03-04T09:18:35.272 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 114/202 2024-03-04T09:18:35.299 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : liboath-2.6.7-2.el9.x86_64 28/202 2024-03-04T09:18:35.312 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 115/202 2024-03-04T09:18:35.345 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 29/202 2024-03-04T09:18:35.385 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 30/202 2024-03-04T09:18:35.420 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 31/202 2024-03-04T09:18:35.469 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 32/202 2024-03-04T09:18:35.488 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : openblas-0.3.21-2.el9.x86_64 116/202 2024-03-04T09:18:35.519 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 117/202 2024-03-04T09:18:35.532 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 33/202 2024-03-04T09:18:35.567 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 34/202 2024-03-04T09:18:35.604 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 118/202 2024-03-04T09:18:35.611 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 35/202 2024-03-04T09:18:35.644 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 36/202 2024-03-04T09:18:35.672 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : lua-5.4.4-4.el9.x86_64 37/202 2024-03-04T09:18:35.740 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 38/202 2024-03-04T09:18:35.777 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-ceph-common-2:19.0.0-1739.g1bc250c7.el9. 39/202 2024-03-04T09:18:35.820 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-ceph-argparse-2:19.0.0-1739.g1bc250c7.el 40/202 2024-03-04T09:18:35.849 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-cephfs-2:19.0.0-1739.g1bc250c7.el9.x86_6 41/202 2024-03-04T09:18:35.917 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 42/202 2024-03-04T09:18:35.972 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : luarocks-3.9.2-1.el9.noarch 43/202 2024-03-04T09:18:36.044 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 44/202 2024-03-04T09:18:36.179 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 45/202 2024-03-04T09:18:36.232 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 119/202 2024-03-04T09:18:36.233 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 46/202 2024-03-04T09:18:36.341 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 120/202 2024-03-04T09:18:36.377 INFO:teuthology.orchestra.run.smithi052.stdout: Preparing : 1/1 2024-03-04T09:18:36.469 INFO:teuthology.orchestra.run.smithi052.stdout: Upgrading : openssl-libs-1:3.0.7-27.el9.x86_64 1/202 2024-03-04T09:18:36.495 INFO:teuthology.orchestra.run.smithi052.stdout: Upgrading : libuuid-2.37.4-18.el9.x86_64 2/202 2024-03-04T09:18:36.566 INFO:teuthology.orchestra.run.smithi052.stdout: Upgrading : libblkid-2.37.4-18.el9.x86_64 3/202 2024-03-04T09:18:36.742 INFO:teuthology.orchestra.run.smithi052.stdout: Running scriptlet: libblkid-2.37.4-18.el9.x86_64 3/202 2024-03-04T09:18:36.750 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-rsa-4.9-2.el9.noarch 47/202 2024-03-04T09:18:36.787 INFO:teuthology.orchestra.run.smithi052.stdout: Upgrading : elfutils-libelf-0.190-2.el9.x86_64 4/202 2024-03-04T09:18:36.827 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 48/202 2024-03-04T09:18:36.834 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 5/202 2024-03-04T09:18:36.889 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 6/202 2024-03-04T09:18:36.897 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 49/202 2024-03-04T09:18:36.929 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : thrift-0.15.0-2.el9.x86_64 7/202 2024-03-04T09:18:36.948 INFO:teuthology.orchestra.run.smithi191.stdout: Upgrading : util-linux-core-2.37.4-18.el9.x86_64 50/202 2024-03-04T09:18:36.953 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 8/202 2024-03-04T09:18:36.956 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: util-linux-core-2.37.4-18.el9.x86_64 50/202 2024-03-04T09:18:36.995 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : qatzip-libs-1.1.2-2.el9.x86_64 51/202 2024-03-04T09:18:37.001 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 9/202 2024-03-04T09:18:37.032 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : librados-devel-2:19.0.0-1739.g1bc250c7.el9.x86_6 52/202 2024-03-04T09:18:37.064 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 53/202 2024-03-04T09:18:37.154 INFO:teuthology.orchestra.run.smithi191.stdout: Upgrading : elfutils-libs-0.190-2.el9.x86_64 54/202 2024-03-04T09:18:37.215 INFO:teuthology.orchestra.run.smithi191.stdout: Upgrading : iproute-6.7.0-2.el9.x86_64 55/202 2024-03-04T09:18:37.342 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 10/202 2024-03-04T09:18:37.396 INFO:teuthology.orchestra.run.smithi052.stdout: Upgrading : librados2-2:19.0.0-1739.g1bc250c7.el9.x86_64 11/202 2024-03-04T09:18:37.429 INFO:teuthology.orchestra.run.smithi191.stdout: Upgrading : libfdisk-2.37.4-18.el9.x86_64 56/202 2024-03-04T09:18:37.521 INFO:teuthology.orchestra.run.smithi052.stdout: Running scriptlet: librados2-2:19.0.0-1739.g1bc250c7.el9.x86_64 11/202 2024-03-04T09:18:37.544 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 121/202 2024-03-04T09:18:37.573 INFO:teuthology.orchestra.run.smithi191.stdout: Upgrading : util-linux-2.37.4-18.el9.x86_64 57/202 2024-03-04T09:18:37.614 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 122/202 2024-03-04T09:18:37.640 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 123/202 2024-03-04T09:18:37.672 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : libcephfs2-2:19.0.0-1739.g1bc250c7.el9.x86_64 12/202 2024-03-04T09:18:37.753 INFO:teuthology.orchestra.run.smithi191.stdout: Upgrading : xfsprogs-6.3.0-1.el9.x86_64 58/202 2024-03-04T09:18:37.764 INFO:teuthology.orchestra.run.smithi052.stdout: Running scriptlet: libcephfs2-2:19.0.0-1739.g1bc250c7.el9.x86_64 12/202 2024-03-04T09:18:37.797 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-rados-2:19.0.0-1739.g1bc250c7.el9.x86_64 13/202 2024-03-04T09:18:37.951 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 124/202 2024-03-04T09:18:37.958 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : postgresql-private-libs-13.11-1.el9.x86_64 14/202 2024-03-04T09:18:37.987 INFO:teuthology.orchestra.run.smithi106.stdout: Upgrading : librbd1-2:19.0.0-1739.g1bc250c7.el9.x86_64 125/202 2024-03-04T09:18:37.989 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 59/202 2024-03-04T09:18:38.024 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : postgresql-13.11-1.el9.x86_64 15/202 2024-03-04T09:18:38.051 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : libcephsqlite-2:19.0.0-1739.g1bc250c7.el9.x86_64 16/202 2024-03-04T09:18:38.077 INFO:teuthology.orchestra.run.smithi106.stdout: Running scriptlet: librbd1-2:19.0.0-1739.g1bc250c7.el9.x86_64 125/202 2024-03-04T09:18:38.116 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-rbd-2:19.0.0-1739.g1bc250c7.el9.x86_64 126/202 2024-03-04T09:18:38.148 INFO:teuthology.orchestra.run.smithi052.stdout: Running scriptlet: libcephsqlite-2:19.0.0-1739.g1bc250c7.el9.x86_64 16/202 2024-03-04T09:18:38.152 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 127/202 2024-03-04T09:18:38.157 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 60/202 2024-03-04T09:18:38.177 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : libradosstriper1-2:19.0.0-1739.g1bc250c7.el9.x86 17/202 2024-03-04T09:18:38.214 INFO:teuthology.orchestra.run.smithi191.stdout: Upgrading : openssl-1:3.0.7-27.el9.x86_64 61/202 2024-03-04T09:18:38.225 INFO:teuthology.orchestra.run.smithi052.stdout: Running scriptlet: libradosstriper1-2:19.0.0-1739.g1bc250c7.el9.x86 17/202 2024-03-04T09:18:38.264 INFO:teuthology.orchestra.run.smithi052.stdout: Upgrading : libmount-2.37.4-18.el9.x86_64 18/202 2024-03-04T09:18:38.269 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 62/202 2024-03-04T09:18:38.287 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : qatlib-service-23.11.0-1.el9.x86_64 19/202 2024-03-04T09:18:38.302 INFO:teuthology.orchestra.run.smithi052.stdout: Running scriptlet: qatlib-service-23.11.0-1.el9.x86_64 19/202 2024-03-04T09:18:38.342 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 63/202 2024-03-04T09:18:38.382 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 64/202 2024-03-04T09:18:38.419 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 65/202 2024-03-04T09:18:38.462 INFO:teuthology.orchestra.run.smithi052.stdout: Running scriptlet: qatlib-23.11.0-1.el9.x86_64 20/202 2024-03-04T09:18:38.464 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 66/202 2024-03-04T09:18:38.493 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : qatlib-23.11.0-1.el9.x86_64 20/202 2024-03-04T09:18:38.501 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 67/202 2024-03-04T09:18:38.525 INFO:teuthology.orchestra.run.smithi052.stdout: Upgrading : libsmartcols-2.37.4-18.el9.x86_64 21/202 2024-03-04T09:18:38.544 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 68/202 2024-03-04T09:18:38.586 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 69/202 2024-03-04T09:18:38.628 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 128/202 2024-03-04T09:18:38.701 INFO:teuthology.orchestra.run.smithi052.stdout: Upgrading : libacl-2.3.1-4.el9.x86_64 22/202 2024-03-04T09:18:38.728 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 70/202 2024-03-04T09:18:38.784 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 71/202 2024-03-04T09:18:38.814 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 72/202 2024-03-04T09:18:38.837 INFO:teuthology.orchestra.run.smithi052.stdout: Upgrading : NetworkManager-libnm-1:1.46.0-1.el9.x86_64 23/202 2024-03-04T09:18:38.855 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 73/202 2024-03-04T09:18:38.878 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : librgw2-2:19.0.0-1739.g1bc250c7.el9.x86_64 129/202 2024-03-04T09:18:38.889 INFO:teuthology.orchestra.run.smithi052.stdout: Running scriptlet: NetworkManager-libnm-1:1.46.0-1.el9.x86_64 23/202 2024-03-04T09:18:38.948 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 74/202 2024-03-04T09:18:38.989 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-google-auth-1:2.28.0-1.el9.noarch 75/202 2024-03-04T09:18:39.002 INFO:teuthology.orchestra.run.smithi106.stdout: Running scriptlet: librgw2-2:19.0.0-1739.g1bc250c7.el9.x86_64 129/202 2024-03-04T09:18:39.004 INFO:teuthology.orchestra.run.smithi052.stdout: Running scriptlet: NetworkManager-1:1.46.0-1.el9.x86_64 24/202 2024-03-04T09:18:39.025 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-rgw-2:19.0.0-1739.g1bc250c7.el9.x86_64 130/202 2024-03-04T09:18:39.029 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 76/202 2024-03-04T09:18:39.049 INFO:teuthology.orchestra.run.smithi052.stdout: Upgrading : NetworkManager-1:1.46.0-1.el9.x86_64 24/202 2024-03-04T09:18:39.063 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 77/202 2024-03-04T09:18:39.088 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 78/202 2024-03-04T09:18:39.160 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 79/202 2024-03-04T09:18:39.245 INFO:teuthology.orchestra.run.smithi052.stdout: Running scriptlet: NetworkManager-1:1.46.0-1.el9.x86_64 24/202 2024-03-04T09:18:39.287 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 80/202 2024-03-04T09:18:39.356 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : re2-1:20211101-3.el9.x86_64 25/202 2024-03-04T09:18:39.386 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 81/202 2024-03-04T09:18:39.437 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 82/202 2024-03-04T09:18:39.485 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-ply-3.11-14.el9.noarch 83/202 2024-03-04T09:18:39.541 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 84/202 2024-03-04T09:18:39.562 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 26/202 2024-03-04T09:18:39.696 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 27/202 2024-03-04T09:18:39.723 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 85/202 2024-03-04T09:18:39.775 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : liboath-2.6.7-2.el9.x86_64 28/202 2024-03-04T09:18:39.775 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 86/202 2024-03-04T09:18:39.828 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 29/202 2024-03-04T09:18:39.857 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 30/202 2024-03-04T09:18:39.893 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 31/202 2024-03-04T09:18:39.902 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 87/202 2024-03-04T09:18:39.933 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 32/202 2024-03-04T09:18:39.947 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-asyncssh-2.13.2-4.el9.noarch 88/202 2024-03-04T09:18:39.980 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 89/202 2024-03-04T09:18:39.999 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 33/202 2024-03-04T09:18:40.012 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 90/202 2024-03-04T09:18:40.038 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 34/202 2024-03-04T09:18:40.047 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : jq-1.6-16.el9.x86_64 91/202 2024-03-04T09:18:40.084 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 35/202 2024-03-04T09:18:40.087 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : fuse-2.9.9-15.el9.x86_64 92/202 2024-03-04T09:18:40.115 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : zlib-devel-1.2.11-40.el9.x86_64 93/202 2024-03-04T09:18:40.117 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 36/202 2024-03-04T09:18:40.144 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : lua-5.4.4-4.el9.x86_64 37/202 2024-03-04T09:18:40.149 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 94/202 2024-03-04T09:18:40.207 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : uuid-1.6.2-55.el9.x86_64 95/202 2024-03-04T09:18:40.215 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 38/202 2024-03-04T09:18:40.245 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 96/202 2024-03-04T09:18:40.257 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-ceph-common-2:19.0.0-1739.g1bc250c7.el9. 39/202 2024-03-04T09:18:40.275 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 97/202 2024-03-04T09:18:40.301 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-ceph-argparse-2:19.0.0-1739.g1bc250c7.el 40/202 2024-03-04T09:18:40.329 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-cephfs-2:19.0.0-1739.g1bc250c7.el9.x86_6 41/202 2024-03-04T09:18:40.350 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 98/202 2024-03-04T09:18:40.393 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 99/202 2024-03-04T09:18:40.403 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 42/202 2024-03-04T09:18:40.427 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 100/202 2024-03-04T09:18:40.452 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : luarocks-3.9.2-1.el9.noarch 43/202 2024-03-04T09:18:40.468 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 101/202 2024-03-04T09:18:40.503 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 102/202 2024-03-04T09:18:40.529 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 44/202 2024-03-04T09:18:40.591 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 103/202 2024-03-04T09:18:40.670 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 45/202 2024-03-04T09:18:40.681 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 104/202 2024-03-04T09:18:40.724 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 46/202 2024-03-04T09:18:40.725 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 105/202 2024-03-04T09:18:40.765 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 106/202 2024-03-04T09:18:40.811 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 107/202 2024-03-04T09:18:40.908 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-packaging-20.9-5.el9.noarch 108/202 2024-03-04T09:18:40.958 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 109/202 2024-03-04T09:18:41.219 INFO:teuthology.orchestra.run.smithi106.stdout: Running scriptlet: ceph-common-2:19.0.0-1739.g1bc250c7.el9.x86_64 131/202 2024-03-04T09:18:41.263 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : ceph-common-2:19.0.0-1739.g1bc250c7.el9.x86_64 131/202 2024-03-04T09:18:41.283 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-rsa-4.9-2.el9.noarch 47/202 2024-03-04T09:18:41.362 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 48/202 2024-03-04T09:18:41.465 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 49/202 2024-03-04T09:18:41.542 INFO:teuthology.orchestra.run.smithi052.stdout: Upgrading : util-linux-core-2.37.4-18.el9.x86_64 50/202 2024-03-04T09:18:41.550 INFO:teuthology.orchestra.run.smithi052.stdout: Running scriptlet: util-linux-core-2.37.4-18.el9.x86_64 50/202 2024-03-04T09:18:41.593 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : qatzip-libs-1.1.2-2.el9.x86_64 51/202 2024-03-04T09:18:41.631 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : librados-devel-2:19.0.0-1739.g1bc250c7.el9.x86_6 52/202 2024-03-04T09:18:41.663 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 53/202 2024-03-04T09:18:41.680 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 110/202 2024-03-04T09:18:41.730 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 111/202 2024-03-04T09:18:41.764 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 112/202 2024-03-04T09:18:41.796 INFO:teuthology.orchestra.run.smithi052.stdout: Upgrading : elfutils-libs-0.190-2.el9.x86_64 54/202 2024-03-04T09:18:41.797 INFO:teuthology.orchestra.run.smithi106.stdout: Running scriptlet: ceph-common-2:19.0.0-1739.g1bc250c7.el9.x86_64 131/202 2024-03-04T09:18:41.811 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 113/202 2024-03-04T09:18:41.833 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : ceph-base-2:19.0.0-1739.g1bc250c7.el9.x86_64 132/202 2024-03-04T09:18:41.890 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 114/202 2024-03-04T09:18:41.897 INFO:teuthology.orchestra.run.smithi052.stdout: Upgrading : iproute-6.7.0-2.el9.x86_64 55/202 2024-03-04T09:18:41.913 INFO:teuthology.orchestra.run.smithi106.stdout: Running scriptlet: ceph-base-2:19.0.0-1739.g1bc250c7.el9.x86_64 132/202 2024-03-04T09:18:41.913 INFO:teuthology.orchestra.run.smithi106.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-03-04T09:18:41.913 INFO:teuthology.orchestra.run.smithi106.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-03-04T09:18:41.913 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-03-04T09:18:41.931 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 115/202 2024-03-04T09:18:41.942 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : ceph-selinux-2:19.0.0-1739.g1bc250c7.el9.x86_64 133/202 2024-03-04T09:18:42.116 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : openblas-0.3.21-2.el9.x86_64 116/202 2024-03-04T09:18:42.146 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 117/202 2024-03-04T09:18:42.165 INFO:teuthology.orchestra.run.smithi052.stdout: Upgrading : libfdisk-2.37.4-18.el9.x86_64 56/202 2024-03-04T09:18:42.224 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 118/202 2024-03-04T09:18:42.319 INFO:teuthology.orchestra.run.smithi052.stdout: Upgrading : util-linux-2.37.4-18.el9.x86_64 57/202 2024-03-04T09:18:42.448 INFO:teuthology.orchestra.run.smithi052.stdout: Upgrading : xfsprogs-6.3.0-1.el9.x86_64 58/202 2024-03-04T09:18:42.493 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 59/202 2024-03-04T09:18:42.637 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 60/202 2024-03-04T09:18:42.696 INFO:teuthology.orchestra.run.smithi052.stdout: Upgrading : openssl-1:3.0.7-27.el9.x86_64 61/202 2024-03-04T09:18:42.752 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 62/202 2024-03-04T09:18:42.824 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 63/202 2024-03-04T09:18:42.852 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 119/202 2024-03-04T09:18:42.854 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 64/202 2024-03-04T09:18:42.890 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 65/202 2024-03-04T09:18:42.946 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 66/202 2024-03-04T09:18:42.959 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 120/202 2024-03-04T09:18:42.982 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 67/202 2024-03-04T09:18:43.025 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 68/202 2024-03-04T09:18:43.067 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 69/202 2024-03-04T09:18:43.208 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 70/202 2024-03-04T09:18:43.273 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 71/202 2024-03-04T09:18:43.302 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 72/202 2024-03-04T09:18:43.344 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 73/202 2024-03-04T09:18:43.442 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 74/202 2024-03-04T09:18:43.477 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-google-auth-1:2.28.0-1.el9.noarch 75/202 2024-03-04T09:18:43.517 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 76/202 2024-03-04T09:18:43.542 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 77/202 2024-03-04T09:18:43.567 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 78/202 2024-03-04T09:18:43.650 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 79/202 2024-03-04T09:18:43.781 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 80/202 2024-03-04T09:18:43.895 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 81/202 2024-03-04T09:18:43.942 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 82/202 2024-03-04T09:18:44.000 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-ply-3.11-14.el9.noarch 83/202 2024-03-04T09:18:44.056 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 84/202 2024-03-04T09:18:44.157 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 121/202 2024-03-04T09:18:44.243 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 122/202 2024-03-04T09:18:44.259 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 85/202 2024-03-04T09:18:44.269 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 123/202 2024-03-04T09:18:44.313 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 86/202 2024-03-04T09:18:44.457 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 87/202 2024-03-04T09:18:44.494 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-asyncssh-2.13.2-4.el9.noarch 88/202 2024-03-04T09:18:44.526 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 89/202 2024-03-04T09:18:44.558 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 90/202 2024-03-04T09:18:44.582 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 124/202 2024-03-04T09:18:44.593 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : jq-1.6-16.el9.x86_64 91/202 2024-03-04T09:18:44.608 INFO:teuthology.orchestra.run.smithi191.stdout: Upgrading : librbd1-2:19.0.0-1739.g1bc250c7.el9.x86_64 125/202 2024-03-04T09:18:44.624 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : fuse-2.9.9-15.el9.x86_64 92/202 2024-03-04T09:18:44.675 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : zlib-devel-1.2.11-40.el9.x86_64 93/202 2024-03-04T09:18:44.682 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: librbd1-2:19.0.0-1739.g1bc250c7.el9.x86_64 125/202 2024-03-04T09:18:44.712 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 94/202 2024-03-04T09:18:44.712 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-rbd-2:19.0.0-1739.g1bc250c7.el9.x86_64 126/202 2024-03-04T09:18:44.748 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 127/202 2024-03-04T09:18:44.751 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : uuid-1.6.2-55.el9.x86_64 95/202 2024-03-04T09:18:44.782 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 96/202 2024-03-04T09:18:44.812 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 97/202 2024-03-04T09:18:44.892 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 98/202 2024-03-04T09:18:44.923 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 99/202 2024-03-04T09:18:44.956 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 100/202 2024-03-04T09:18:44.998 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 101/202 2024-03-04T09:18:45.032 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 102/202 2024-03-04T09:18:45.125 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 103/202 2024-03-04T09:18:45.207 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 128/202 2024-03-04T09:18:45.215 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 104/202 2024-03-04T09:18:45.244 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : librgw2-2:19.0.0-1739.g1bc250c7.el9.x86_64 129/202 2024-03-04T09:18:45.263 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 105/202 2024-03-04T09:18:45.294 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 106/202 2024-03-04T09:18:45.307 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: librgw2-2:19.0.0-1739.g1bc250c7.el9.x86_64 129/202 2024-03-04T09:18:45.338 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-rgw-2:19.0.0-1739.g1bc250c7.el9.x86_64 130/202 2024-03-04T09:18:45.343 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 107/202 2024-03-04T09:18:45.451 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-packaging-20.9-5.el9.noarch 108/202 2024-03-04T09:18:45.505 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 109/202 2024-03-04T09:18:46.270 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 110/202 2024-03-04T09:18:46.323 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 111/202 2024-03-04T09:18:46.357 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 112/202 2024-03-04T09:18:46.407 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 113/202 2024-03-04T09:18:46.497 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 114/202 2024-03-04T09:18:46.535 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 115/202 2024-03-04T09:18:46.717 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : openblas-0.3.21-2.el9.x86_64 116/202 2024-03-04T09:18:46.747 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 117/202 2024-03-04T09:18:46.827 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 118/202 2024-03-04T09:18:47.489 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: ceph-common-2:19.0.0-1739.g1bc250c7.el9.x86_64 131/202 2024-03-04T09:18:47.508 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 119/202 2024-03-04T09:18:47.525 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : ceph-common-2:19.0.0-1739.g1bc250c7.el9.x86_64 131/202 2024-03-04T09:18:47.614 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 120/202 2024-03-04T09:18:48.062 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: ceph-common-2:19.0.0-1739.g1bc250c7.el9.x86_64 131/202 2024-03-04T09:18:48.104 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : ceph-base-2:19.0.0-1739.g1bc250c7.el9.x86_64 132/202 2024-03-04T09:18:48.185 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: ceph-base-2:19.0.0-1739.g1bc250c7.el9.x86_64 132/202 2024-03-04T09:18:48.185 INFO:teuthology.orchestra.run.smithi191.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-03-04T09:18:48.186 INFO:teuthology.orchestra.run.smithi191.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-03-04T09:18:48.186 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-03-04T09:18:48.213 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : ceph-selinux-2:19.0.0-1739.g1bc250c7.el9.x86_64 133/202 2024-03-04T09:18:48.906 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 121/202 2024-03-04T09:18:48.975 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 122/202 2024-03-04T09:18:49.002 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 123/202 2024-03-04T09:18:49.331 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 124/202 2024-03-04T09:18:49.365 INFO:teuthology.orchestra.run.smithi052.stdout: Upgrading : librbd1-2:19.0.0-1739.g1bc250c7.el9.x86_64 125/202 2024-03-04T09:18:49.439 INFO:teuthology.orchestra.run.smithi052.stdout: Running scriptlet: librbd1-2:19.0.0-1739.g1bc250c7.el9.x86_64 125/202 2024-03-04T09:18:49.470 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-rbd-2:19.0.0-1739.g1bc250c7.el9.x86_64 126/202 2024-03-04T09:18:49.507 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 127/202 2024-03-04T09:18:49.994 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 128/202 2024-03-04T09:18:50.026 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : librgw2-2:19.0.0-1739.g1bc250c7.el9.x86_64 129/202 2024-03-04T09:18:50.080 INFO:teuthology.orchestra.run.smithi052.stdout: Running scriptlet: librgw2-2:19.0.0-1739.g1bc250c7.el9.x86_64 129/202 2024-03-04T09:18:50.103 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-rgw-2:19.0.0-1739.g1bc250c7.el9.x86_64 130/202 2024-03-04T09:18:52.445 INFO:teuthology.orchestra.run.smithi052.stdout: Running scriptlet: ceph-common-2:19.0.0-1739.g1bc250c7.el9.x86_64 131/202 2024-03-04T09:18:52.482 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : ceph-common-2:19.0.0-1739.g1bc250c7.el9.x86_64 131/202 2024-03-04T09:18:53.046 INFO:teuthology.orchestra.run.smithi052.stdout: Running scriptlet: ceph-common-2:19.0.0-1739.g1bc250c7.el9.x86_64 131/202 2024-03-04T09:18:53.077 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : ceph-base-2:19.0.0-1739.g1bc250c7.el9.x86_64 132/202 2024-03-04T09:18:53.153 INFO:teuthology.orchestra.run.smithi052.stdout: Running scriptlet: ceph-base-2:19.0.0-1739.g1bc250c7.el9.x86_64 132/202 2024-03-04T09:18:53.153 INFO:teuthology.orchestra.run.smithi052.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-03-04T09:18:53.153 INFO:teuthology.orchestra.run.smithi052.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-03-04T09:18:53.153 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-03-04T09:18:53.195 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : ceph-selinux-2:19.0.0-1739.g1bc250c7.el9.x86_64 133/202 2024-03-04T09:18:53.576 INFO:teuthology.orchestra.run.smithi106.stdout: Running scriptlet: ceph-selinux-2:19.0.0-1739.g1bc250c7.el9.x86_64 133/202 2024-03-04T09:18:53.576 INFO:teuthology.orchestra.run.smithi106.stdout:skipping the directory /sys 2024-03-04T09:18:53.576 INFO:teuthology.orchestra.run.smithi106.stdout:skipping the directory /proc 2024-03-04T09:18:53.576 INFO:teuthology.orchestra.run.smithi106.stdout:skipping the directory /mnt 2024-03-04T09:18:53.576 INFO:teuthology.orchestra.run.smithi106.stdout:skipping the directory /var/tmp 2024-03-04T09:18:53.576 INFO:teuthology.orchestra.run.smithi106.stdout:skipping the directory /home 2024-03-04T09:18:53.576 INFO:teuthology.orchestra.run.smithi106.stdout:skipping the directory /root 2024-03-04T09:18:53.576 INFO:teuthology.orchestra.run.smithi106.stdout:skipping the directory /tmp 2024-03-04T09:18:53.576 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-03-04T09:18:53.739 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : ceph-volume-2:19.0.0-1739.g1bc250c7.el9.noarch 134/202 2024-03-04T09:18:53.755 INFO:teuthology.orchestra.run.smithi106.stdout: Running scriptlet: ceph-volume-2:19.0.0-1739.g1bc250c7.el9.noarch 134/202 2024-03-04T09:18:53.755 INFO:teuthology.orchestra.run.smithi106.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-04T09:18:53.755 INFO:teuthology.orchestra.run.smithi106.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-03-04T09:18:53.755 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-03-04T09:18:55.234 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : ceph-osd-2:19.0.0-1739.g1bc250c7.el9.x86_64 135/202 2024-03-04T09:18:55.260 INFO:teuthology.orchestra.run.smithi106.stdout: Running scriptlet: ceph-osd-2:19.0.0-1739.g1bc250c7.el9.x86_64 135/202 2024-03-04T09:18:55.260 INFO:teuthology.orchestra.run.smithi106.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-04T09:18:55.260 INFO:teuthology.orchestra.run.smithi106.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-03-04T09:18:55.260 INFO:teuthology.orchestra.run.smithi106.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-04T09:18:55.260 INFO:teuthology.orchestra.run.smithi106.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-04T09:18:55.260 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-03-04T09:18:55.476 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : ceph-mds-2:19.0.0-1739.g1bc250c7.el9.x86_64 136/202 2024-03-04T09:18:55.499 INFO:teuthology.orchestra.run.smithi106.stdout: Running scriptlet: ceph-mds-2:19.0.0-1739.g1bc250c7.el9.x86_64 136/202 2024-03-04T09:18:55.499 INFO:teuthology.orchestra.run.smithi106.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-04T09:18:55.499 INFO:teuthology.orchestra.run.smithi106.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-03-04T09:18:55.499 INFO:teuthology.orchestra.run.smithi106.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-04T09:18:55.499 INFO:teuthology.orchestra.run.smithi106.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-04T09:18:55.499 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-03-04T09:18:55.943 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : ceph-mon-2:19.0.0-1739.g1bc250c7.el9.x86_64 137/202 2024-03-04T09:18:55.966 INFO:teuthology.orchestra.run.smithi106.stdout: Running scriptlet: ceph-mon-2:19.0.0-1739.g1bc250c7.el9.x86_64 137/202 2024-03-04T09:18:55.966 INFO:teuthology.orchestra.run.smithi106.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-04T09:18:55.966 INFO:teuthology.orchestra.run.smithi106.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-03-04T09:18:55.966 INFO:teuthology.orchestra.run.smithi106.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-04T09:18:55.966 INFO:teuthology.orchestra.run.smithi106.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-04T09:18:55.966 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-03-04T09:18:56.070 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : automake-1.16.2-8.el9.noarch 138/202 2024-03-04T09:18:56.488 INFO:teuthology.orchestra.run.smithi106.stdout: Running scriptlet: cephadm-2:19.0.0-1739.g1bc250c7.el9.noarch 139/202 2024-03-04T09:18:56.518 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : cephadm-2:19.0.0-1739.g1bc250c7.el9.noarch 139/202 2024-03-04T09:18:56.549 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-1739.g1bc250c7.e 140/202 2024-03-04T09:18:56.610 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-1739.g1bc250c7. 141/202 2024-03-04T09:18:56.633 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-1739.g1bc250c7.el9.noa 142/202 2024-03-04T09:18:56.863 INFO:teuthology.orchestra.run.smithi106.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-1739.g1bc250c7.el9.noa 142/202 2024-03-04T09:18:56.900 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-1739.g1bc250c7.el9.n 143/202 2024-03-04T09:18:57.852 INFO:teuthology.orchestra.run.smithi106.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-1739.g1bc250c7.el9.n 143/202 2024-03-04T09:18:57.878 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-1739.g1bc 144/202 2024-03-04T09:18:57.979 INFO:teuthology.orchestra.run.smithi106.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-1739.g1bc 144/202 2024-03-04T09:18:58.140 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-1739.g1bc250c7.el 145/202 2024-03-04T09:18:58.163 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : ceph-mgr-2:19.0.0-1739.g1bc250c7.el9.x86_64 146/202 2024-03-04T09:18:58.186 INFO:teuthology.orchestra.run.smithi106.stdout: Running scriptlet: ceph-mgr-2:19.0.0-1739.g1bc250c7.el9.x86_64 146/202 2024-03-04T09:18:58.186 INFO:teuthology.orchestra.run.smithi106.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-04T09:18:58.186 INFO:teuthology.orchestra.run.smithi106.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-03-04T09:18:58.186 INFO:teuthology.orchestra.run.smithi106.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-04T09:18:58.187 INFO:teuthology.orchestra.run.smithi106.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-04T09:18:58.187 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-03-04T09:18:58.230 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : ceph-mgr-rook-2:19.0.0-1739.g1bc250c7.el9.noarch 147/202 2024-03-04T09:18:58.242 INFO:teuthology.orchestra.run.smithi106.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-1739.g1bc250c7.el9.noarch 147/202 2024-03-04T09:18:58.287 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : ceph-2:19.0.0-1739.g1bc250c7.el9.x86_64 148/202 2024-03-04T09:18:59.140 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : libtool-2.4.6-45.el9.x86_64 149/202 2024-03-04T09:18:59.174 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : ceph-radosgw-2:19.0.0-1739.g1bc250c7.el9.x86_64 150/202 2024-03-04T09:18:59.197 INFO:teuthology.orchestra.run.smithi106.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-1739.g1bc250c7.el9.x86_64 150/202 2024-03-04T09:18:59.197 INFO:teuthology.orchestra.run.smithi106.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-04T09:18:59.197 INFO:teuthology.orchestra.run.smithi106.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-03-04T09:18:59.197 INFO:teuthology.orchestra.run.smithi106.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-04T09:18:59.197 INFO:teuthology.orchestra.run.smithi106.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-04T09:18:59.197 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-03-04T09:18:59.241 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-1739.g1bc25 151/202 2024-03-04T09:18:59.263 INFO:teuthology.orchestra.run.smithi106.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-1739.g1bc25 151/202 2024-03-04T09:18:59.264 INFO:teuthology.orchestra.run.smithi106.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-04T09:18:59.264 INFO:teuthology.orchestra.run.smithi106.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-03-04T09:18:59.264 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-03-04T09:18:59.317 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : cephfs-mirror-2:19.0.0-1739.g1bc250c7.el9.x86_64 152/202 2024-03-04T09:18:59.339 INFO:teuthology.orchestra.run.smithi106.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-1739.g1bc250c7.el9.x86_64 152/202 2024-03-04T09:18:59.340 INFO:teuthology.orchestra.run.smithi106.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-04T09:18:59.340 INFO:teuthology.orchestra.run.smithi106.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-03-04T09:18:59.340 INFO:teuthology.orchestra.run.smithi106.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-03-04T09:18:59.340 INFO:teuthology.orchestra.run.smithi106.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-03-04T09:18:59.340 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-03-04T09:18:59.643 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : rbd-mirror-2:19.0.0-1739.g1bc250c7.el9.x86_64 153/202 2024-03-04T09:18:59.665 INFO:teuthology.orchestra.run.smithi106.stdout: Running scriptlet: rbd-mirror-2:19.0.0-1739.g1bc250c7.el9.x86_64 153/202 2024-03-04T09:18:59.665 INFO:teuthology.orchestra.run.smithi106.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-04T09:18:59.665 INFO:teuthology.orchestra.run.smithi106.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-03-04T09:18:59.665 INFO:teuthology.orchestra.run.smithi106.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-04T09:18:59.666 INFO:teuthology.orchestra.run.smithi106.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-04T09:18:59.666 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-03-04T09:19:00.193 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: ceph-selinux-2:19.0.0-1739.g1bc250c7.el9.x86_64 133/202 2024-03-04T09:19:00.193 INFO:teuthology.orchestra.run.smithi191.stdout:skipping the directory /sys 2024-03-04T09:19:00.193 INFO:teuthology.orchestra.run.smithi191.stdout:skipping the directory /proc 2024-03-04T09:19:00.193 INFO:teuthology.orchestra.run.smithi191.stdout:skipping the directory /mnt 2024-03-04T09:19:00.193 INFO:teuthology.orchestra.run.smithi191.stdout:skipping the directory /var/tmp 2024-03-04T09:19:00.193 INFO:teuthology.orchestra.run.smithi191.stdout:skipping the directory /home 2024-03-04T09:19:00.193 INFO:teuthology.orchestra.run.smithi191.stdout:skipping the directory /root 2024-03-04T09:19:00.193 INFO:teuthology.orchestra.run.smithi191.stdout:skipping the directory /tmp 2024-03-04T09:19:00.193 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-03-04T09:19:00.358 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : ceph-volume-2:19.0.0-1739.g1bc250c7.el9.noarch 134/202 2024-03-04T09:19:00.373 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: ceph-volume-2:19.0.0-1739.g1bc250c7.el9.noarch 134/202 2024-03-04T09:19:00.373 INFO:teuthology.orchestra.run.smithi191.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-04T09:19:00.373 INFO:teuthology.orchestra.run.smithi191.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-03-04T09:19:00.373 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-03-04T09:19:01.863 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : ceph-osd-2:19.0.0-1739.g1bc250c7.el9.x86_64 135/202 2024-03-04T09:19:01.889 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: ceph-osd-2:19.0.0-1739.g1bc250c7.el9.x86_64 135/202 2024-03-04T09:19:01.889 INFO:teuthology.orchestra.run.smithi191.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-04T09:19:01.889 INFO:teuthology.orchestra.run.smithi191.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-03-04T09:19:01.889 INFO:teuthology.orchestra.run.smithi191.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-04T09:19:01.889 INFO:teuthology.orchestra.run.smithi191.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-04T09:19:01.889 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-03-04T09:19:02.118 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : ceph-mds-2:19.0.0-1739.g1bc250c7.el9.x86_64 136/202 2024-03-04T09:19:02.136 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: ceph-mds-2:19.0.0-1739.g1bc250c7.el9.x86_64 136/202 2024-03-04T09:19:02.136 INFO:teuthology.orchestra.run.smithi191.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-04T09:19:02.136 INFO:teuthology.orchestra.run.smithi191.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-03-04T09:19:02.136 INFO:teuthology.orchestra.run.smithi191.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-04T09:19:02.136 INFO:teuthology.orchestra.run.smithi191.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-04T09:19:02.136 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-03-04T09:19:02.589 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : ceph-mon-2:19.0.0-1739.g1bc250c7.el9.x86_64 137/202 2024-03-04T09:19:02.611 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: ceph-mon-2:19.0.0-1739.g1bc250c7.el9.x86_64 137/202 2024-03-04T09:19:02.611 INFO:teuthology.orchestra.run.smithi191.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-04T09:19:02.611 INFO:teuthology.orchestra.run.smithi191.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-03-04T09:19:02.611 INFO:teuthology.orchestra.run.smithi191.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-04T09:19:02.611 INFO:teuthology.orchestra.run.smithi191.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-04T09:19:02.611 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-03-04T09:19:02.699 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : automake-1.16.2-8.el9.noarch 138/202 2024-03-04T09:19:03.107 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: cephadm-2:19.0.0-1739.g1bc250c7.el9.noarch 139/202 2024-03-04T09:19:03.140 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : cephadm-2:19.0.0-1739.g1bc250c7.el9.noarch 139/202 2024-03-04T09:19:03.170 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-1739.g1bc250c7.e 140/202 2024-03-04T09:19:03.231 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-1739.g1bc250c7. 141/202 2024-03-04T09:19:03.254 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-1739.g1bc250c7.el9.noa 142/202 2024-03-04T09:19:03.482 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-1739.g1bc250c7.el9.noa 142/202 2024-03-04T09:19:03.513 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-1739.g1bc250c7.el9.n 143/202 2024-03-04T09:19:04.055 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : ceph-test-2:19.0.0-1739.g1bc250c7.el9.x86_64 154/202 2024-03-04T09:19:04.096 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : rbd-fuse-2:19.0.0-1739.g1bc250c7.el9.x86_64 155/202 2024-03-04T09:19:04.127 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : rbd-nbd-2:19.0.0-1739.g1bc250c7.el9.x86_64 156/202 2024-03-04T09:19:04.154 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 157/202 2024-03-04T09:19:04.354 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : cephfs-shell-2:19.0.0-1739.g1bc250c7.el9.x86_64 158/202 2024-03-04T09:19:04.477 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-1739.g1bc250c7.el9.n 143/202 2024-03-04T09:19:04.478 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : postgresql-contrib-13.11-1.el9.x86_64 159/202 2024-03-04T09:19:04.505 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-1739.g1bc 144/202 2024-03-04T09:19:04.519 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : ceph-fuse-2:19.0.0-1739.g1bc250c7.el9.x86_64 160/202 2024-03-04T09:19:04.558 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 161/202 2024-03-04T09:19:04.605 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-1739.g1bc 144/202 2024-03-04T09:19:04.619 INFO:teuthology.orchestra.run.smithi106.stdout: Upgrading : xfsdump-3.1.12-4.el9.x86_64 162/202 2024-03-04T09:19:04.692 INFO:teuthology.orchestra.run.smithi106.stdout: Upgrading : util-linux-user-2.37.4-18.el9.x86_64 163/202 2024-03-04T09:19:04.739 INFO:teuthology.orchestra.run.smithi106.stdout: Upgrading : iproute-tc-6.7.0-2.el9.x86_64 164/202 2024-03-04T09:19:04.759 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-1739.g1bc250c7.el 145/202 2024-03-04T09:19:04.781 INFO:teuthology.orchestra.run.smithi106.stdout: Upgrading : elfutils-debuginfod-client-0.190-2.el9.x86_64 165/202 2024-03-04T09:19:04.790 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : ceph-mgr-2:19.0.0-1739.g1bc250c7.el9.x86_64 146/202 2024-03-04T09:19:04.811 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : libcephfs-devel-2:19.0.0-1739.g1bc250c7.el9.x86_ 166/202 2024-03-04T09:19:04.813 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: ceph-mgr-2:19.0.0-1739.g1bc250c7.el9.x86_64 146/202 2024-03-04T09:19:04.813 INFO:teuthology.orchestra.run.smithi191.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-04T09:19:04.813 INFO:teuthology.orchestra.run.smithi191.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-03-04T09:19:04.813 INFO:teuthology.orchestra.run.smithi191.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-04T09:19:04.813 INFO:teuthology.orchestra.run.smithi191.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-04T09:19:04.813 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-03-04T09:19:04.849 INFO:teuthology.orchestra.run.smithi106.stdout: Upgrading : NetworkManager-team-1:1.46.0-1.el9.x86_64 167/202 2024-03-04T09:19:04.857 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : ceph-mgr-rook-2:19.0.0-1739.g1bc250c7.el9.noarch 147/202 2024-03-04T09:19:04.868 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-1739.g1bc250c7.el9.noarch 147/202 2024-03-04T09:19:04.905 INFO:teuthology.orchestra.run.smithi106.stdout: Upgrading : NetworkManager-tui-1:1.46.0-1.el9.x86_64 168/202 2024-03-04T09:19:04.913 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : ceph-2:19.0.0-1739.g1bc250c7.el9.x86_64 148/202 2024-03-04T09:19:04.938 INFO:teuthology.orchestra.run.smithi106.stdout: Upgrading : libacl-devel-2.3.1-4.el9.x86_64 169/202 2024-03-04T09:19:04.973 INFO:teuthology.orchestra.run.smithi106.stdout: Upgrading : acl-2.3.1-4.el9.x86_64 170/202 2024-03-04T09:19:05.248 INFO:teuthology.orchestra.run.smithi052.stdout: Running scriptlet: ceph-selinux-2:19.0.0-1739.g1bc250c7.el9.x86_64 133/202 2024-03-04T09:19:05.248 INFO:teuthology.orchestra.run.smithi052.stdout:skipping the directory /sys 2024-03-04T09:19:05.248 INFO:teuthology.orchestra.run.smithi052.stdout:skipping the directory /proc 2024-03-04T09:19:05.248 INFO:teuthology.orchestra.run.smithi052.stdout:skipping the directory /mnt 2024-03-04T09:19:05.248 INFO:teuthology.orchestra.run.smithi052.stdout:skipping the directory /var/tmp 2024-03-04T09:19:05.248 INFO:teuthology.orchestra.run.smithi052.stdout:skipping the directory /home 2024-03-04T09:19:05.248 INFO:teuthology.orchestra.run.smithi052.stdout:skipping the directory /root 2024-03-04T09:19:05.248 INFO:teuthology.orchestra.run.smithi052.stdout:skipping the directory /tmp 2024-03-04T09:19:05.248 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-03-04T09:19:05.431 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : ceph-volume-2:19.0.0-1739.g1bc250c7.el9.noarch 134/202 2024-03-04T09:19:05.447 INFO:teuthology.orchestra.run.smithi052.stdout: Running scriptlet: ceph-volume-2:19.0.0-1739.g1bc250c7.el9.noarch 134/202 2024-03-04T09:19:05.447 INFO:teuthology.orchestra.run.smithi052.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-04T09:19:05.447 INFO:teuthology.orchestra.run.smithi052.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-03-04T09:19:05.447 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-03-04T09:19:05.585 INFO:teuthology.orchestra.run.smithi106.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 171/202 2024-03-04T09:19:05.620 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : postgresql-server-13.11-1.el9.x86_64 171/202 2024-03-04T09:19:05.641 INFO:teuthology.orchestra.run.smithi106.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 171/202 2024-03-04T09:19:05.767 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : libtool-2.4.6-45.el9.x86_64 149/202 2024-03-04T09:19:05.793 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : ceph-radosgw-2:19.0.0-1739.g1bc250c7.el9.x86_64 150/202 2024-03-04T09:19:05.815 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-1739.g1bc250c7.el9.x86_64 150/202 2024-03-04T09:19:05.815 INFO:teuthology.orchestra.run.smithi191.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-04T09:19:05.815 INFO:teuthology.orchestra.run.smithi191.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-03-04T09:19:05.815 INFO:teuthology.orchestra.run.smithi191.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-04T09:19:05.815 INFO:teuthology.orchestra.run.smithi191.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-04T09:19:05.815 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-03-04T09:19:05.860 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-1739.g1bc25 151/202 2024-03-04T09:19:05.881 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-1739.g1bc25 151/202 2024-03-04T09:19:05.881 INFO:teuthology.orchestra.run.smithi191.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-04T09:19:05.881 INFO:teuthology.orchestra.run.smithi191.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-03-04T09:19:05.881 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-03-04T09:19:05.936 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : cephfs-mirror-2:19.0.0-1739.g1bc250c7.el9.x86_64 152/202 2024-03-04T09:19:05.957 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-1739.g1bc250c7.el9.x86_64 152/202 2024-03-04T09:19:05.958 INFO:teuthology.orchestra.run.smithi191.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-04T09:19:05.958 INFO:teuthology.orchestra.run.smithi191.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-03-04T09:19:05.958 INFO:teuthology.orchestra.run.smithi191.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-03-04T09:19:05.958 INFO:teuthology.orchestra.run.smithi191.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-03-04T09:19:05.958 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-03-04T09:19:06.261 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : rbd-mirror-2:19.0.0-1739.g1bc250c7.el9.x86_64 153/202 2024-03-04T09:19:06.283 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: rbd-mirror-2:19.0.0-1739.g1bc250c7.el9.x86_64 153/202 2024-03-04T09:19:06.284 INFO:teuthology.orchestra.run.smithi191.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-04T09:19:06.284 INFO:teuthology.orchestra.run.smithi191.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-03-04T09:19:06.284 INFO:teuthology.orchestra.run.smithi191.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-04T09:19:06.284 INFO:teuthology.orchestra.run.smithi191.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-04T09:19:06.284 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-03-04T09:19:06.462 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : cephfs-top-2:19.0.0-1739.g1bc250c7.el9.noarch 172/202 2024-03-04T09:19:06.515 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : openssl-devel-1:3.0.7-27.el9.x86_64 173/202 2024-03-04T09:19:06.548 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 174/202 2024-03-04T09:19:06.637 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : flex-2.6.4-9.el9.x86_64 175/202 2024-03-04T09:19:06.677 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : bison-3.7.4-5.el9.x86_64 176/202 2024-03-04T09:19:06.702 INFO:teuthology.orchestra.run.smithi106.stdout: Cleanup : NetworkManager-tui-1:1.44.0-2.el9.x86_64 177/202 2024-03-04T09:19:06.727 INFO:teuthology.orchestra.run.smithi106.stdout: Cleanup : elfutils-debuginfod-client-0.189-3.el9.x86_64 178/202 2024-03-04T09:19:06.754 INFO:teuthology.orchestra.run.smithi106.stdout: Cleanup : elfutils-libs-0.189-3.el9.x86_64 179/202 2024-03-04T09:19:06.754 INFO:teuthology.orchestra.run.smithi106.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 180/202 2024-03-04T09:19:06.822 INFO:teuthology.orchestra.run.smithi106.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 180/202 2024-03-04T09:19:06.849 INFO:teuthology.orchestra.run.smithi106.stdout: Cleanup : libacl-devel-2.3.1-3.el9.x86_64 181/202 2024-03-04T09:19:06.880 INFO:teuthology.orchestra.run.smithi106.stdout: Cleanup : openssl-1:3.0.7-24.el9.x86_64 182/202 2024-03-04T09:19:06.880 INFO:teuthology.orchestra.run.smithi106.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 183/202 2024-03-04T09:19:06.948 INFO:teuthology.orchestra.run.smithi106.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 183/202 2024-03-04T09:19:06.974 INFO:teuthology.orchestra.run.smithi106.stdout: Cleanup : xfsdump-3.1.12-3.el9.x86_64 184/202 2024-03-04T09:19:06.984 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : ceph-osd-2:19.0.0-1739.g1bc250c7.el9.x86_64 135/202 2024-03-04T09:19:07.008 INFO:teuthology.orchestra.run.smithi106.stdout: Cleanup : xfsprogs-5.19.0-4.el9.x86_64 185/202 2024-03-04T09:19:07.009 INFO:teuthology.orchestra.run.smithi052.stdout: Running scriptlet: ceph-osd-2:19.0.0-1739.g1bc250c7.el9.x86_64 135/202 2024-03-04T09:19:07.009 INFO:teuthology.orchestra.run.smithi052.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-04T09:19:07.009 INFO:teuthology.orchestra.run.smithi052.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-03-04T09:19:07.009 INFO:teuthology.orchestra.run.smithi052.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-04T09:19:07.010 INFO:teuthology.orchestra.run.smithi052.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-04T09:19:07.010 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-03-04T09:19:07.032 INFO:teuthology.orchestra.run.smithi106.stdout: Cleanup : iproute-tc-6.2.0-5.el9.x86_64 186/202 2024-03-04T09:19:07.059 INFO:teuthology.orchestra.run.smithi106.stdout: Cleanup : acl-2.3.1-3.el9.x86_64 187/202 2024-03-04T09:19:07.092 INFO:teuthology.orchestra.run.smithi106.stdout: Cleanup : iproute-6.2.0-5.el9.x86_64 188/202 2024-03-04T09:19:07.125 INFO:teuthology.orchestra.run.smithi106.stdout: Cleanup : util-linux-user-2.37.4-15.el9.x86_64 189/202 2024-03-04T09:19:07.164 INFO:teuthology.orchestra.run.smithi106.stdout: Cleanup : util-linux-2.37.4-15.el9.x86_64 190/202 2024-03-04T09:19:07.197 INFO:teuthology.orchestra.run.smithi106.stdout: Cleanup : util-linux-core-2.37.4-15.el9.x86_64 191/202 2024-03-04T09:19:07.222 INFO:teuthology.orchestra.run.smithi106.stdout: Cleanup : libfdisk-2.37.4-15.el9.x86_64 192/202 2024-03-04T09:19:07.235 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : ceph-mds-2:19.0.0-1739.g1bc250c7.el9.x86_64 136/202 2024-03-04T09:19:07.247 INFO:teuthology.orchestra.run.smithi106.stdout: Cleanup : libmount-2.37.4-15.el9.x86_64 193/202 2024-03-04T09:19:07.257 INFO:teuthology.orchestra.run.smithi052.stdout: Running scriptlet: ceph-mds-2:19.0.0-1739.g1bc250c7.el9.x86_64 136/202 2024-03-04T09:19:07.257 INFO:teuthology.orchestra.run.smithi052.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-04T09:19:07.257 INFO:teuthology.orchestra.run.smithi052.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-03-04T09:19:07.257 INFO:teuthology.orchestra.run.smithi052.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-04T09:19:07.257 INFO:teuthology.orchestra.run.smithi052.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-04T09:19:07.257 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-03-04T09:19:07.281 INFO:teuthology.orchestra.run.smithi106.stdout: Cleanup : libblkid-2.37.4-15.el9.x86_64 194/202 2024-03-04T09:19:07.306 INFO:teuthology.orchestra.run.smithi106.stdout: Cleanup : NetworkManager-team-1:1.44.0-2.el9.x86_64 195/202 2024-03-04T09:19:07.313 INFO:teuthology.orchestra.run.smithi106.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 196/202 2024-03-04T09:19:07.313 INFO:teuthology.orchestra.run.smithi106.stdout: Cleanup : NetworkManager-1:1.44.0-2.el9.x86_64 196/202 2024-03-04T09:19:07.492 INFO:teuthology.orchestra.run.smithi106.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 196/202 2024-03-04T09:19:07.492 INFO:teuthology.orchestra.run.smithi106.stdout: Cleanup : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 197/202 2024-03-04T09:19:07.585 INFO:teuthology.orchestra.run.smithi106.stdout: Running scriptlet: NetworkManager-libnm-1:1.44.0-2.el9.x86_64 197/202 2024-03-04T09:19:07.624 INFO:teuthology.orchestra.run.smithi106.stdout: Cleanup : libuuid-2.37.4-15.el9.x86_64 198/202 2024-03-04T09:19:07.650 INFO:teuthology.orchestra.run.smithi106.stdout: Cleanup : libsmartcols-2.37.4-15.el9.x86_64 199/202 2024-03-04T09:19:07.675 INFO:teuthology.orchestra.run.smithi106.stdout: Cleanup : elfutils-libelf-0.189-3.el9.x86_64 200/202 2024-03-04T09:19:07.701 INFO:teuthology.orchestra.run.smithi106.stdout: Cleanup : libacl-2.3.1-3.el9.x86_64 201/202 2024-03-04T09:19:07.719 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : ceph-mon-2:19.0.0-1739.g1bc250c7.el9.x86_64 137/202 2024-03-04T09:19:07.741 INFO:teuthology.orchestra.run.smithi052.stdout: Running scriptlet: ceph-mon-2:19.0.0-1739.g1bc250c7.el9.x86_64 137/202 2024-03-04T09:19:07.741 INFO:teuthology.orchestra.run.smithi052.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-04T09:19:07.741 INFO:teuthology.orchestra.run.smithi052.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-03-04T09:19:07.741 INFO:teuthology.orchestra.run.smithi052.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-04T09:19:07.741 INFO:teuthology.orchestra.run.smithi052.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-04T09:19:07.741 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-03-04T09:19:07.774 INFO:teuthology.orchestra.run.smithi106.stdout: Cleanup : openssl-libs-1:3.0.7-24.el9.x86_64 202/202 2024-03-04T09:19:07.837 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : automake-1.16.2-8.el9.noarch 138/202 2024-03-04T09:19:08.273 INFO:teuthology.orchestra.run.smithi052.stdout: Running scriptlet: cephadm-2:19.0.0-1739.g1bc250c7.el9.noarch 139/202 2024-03-04T09:19:08.302 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : cephadm-2:19.0.0-1739.g1bc250c7.el9.noarch 139/202 2024-03-04T09:19:08.334 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-1739.g1bc250c7.e 140/202 2024-03-04T09:19:08.397 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-1739.g1bc250c7. 141/202 2024-03-04T09:19:08.425 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-1739.g1bc250c7.el9.noa 142/202 2024-03-04T09:19:08.665 INFO:teuthology.orchestra.run.smithi052.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-1739.g1bc250c7.el9.noa 142/202 2024-03-04T09:19:08.693 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-1739.g1bc250c7.el9.n 143/202 2024-03-04T09:19:09.702 INFO:teuthology.orchestra.run.smithi052.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-1739.g1bc250c7.el9.n 143/202 2024-03-04T09:19:09.726 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-1739.g1bc 144/202 2024-03-04T09:19:09.832 INFO:teuthology.orchestra.run.smithi052.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-1739.g1bc 144/202 2024-03-04T09:19:10.005 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-1739.g1bc250c7.el 145/202 2024-03-04T09:19:10.045 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : ceph-mgr-2:19.0.0-1739.g1bc250c7.el9.x86_64 146/202 2024-03-04T09:19:10.068 INFO:teuthology.orchestra.run.smithi052.stdout: Running scriptlet: ceph-mgr-2:19.0.0-1739.g1bc250c7.el9.x86_64 146/202 2024-03-04T09:19:10.068 INFO:teuthology.orchestra.run.smithi052.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-04T09:19:10.068 INFO:teuthology.orchestra.run.smithi052.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-03-04T09:19:10.068 INFO:teuthology.orchestra.run.smithi052.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-04T09:19:10.068 INFO:teuthology.orchestra.run.smithi052.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-04T09:19:10.068 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-03-04T09:19:10.113 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : ceph-mgr-rook-2:19.0.0-1739.g1bc250c7.el9.noarch 147/202 2024-03-04T09:19:10.123 INFO:teuthology.orchestra.run.smithi052.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-1739.g1bc250c7.el9.noarch 147/202 2024-03-04T09:19:10.177 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : ceph-2:19.0.0-1739.g1bc250c7.el9.x86_64 148/202 2024-03-04T09:19:10.707 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : ceph-test-2:19.0.0-1739.g1bc250c7.el9.x86_64 154/202 2024-03-04T09:19:10.748 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : rbd-fuse-2:19.0.0-1739.g1bc250c7.el9.x86_64 155/202 2024-03-04T09:19:10.779 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : rbd-nbd-2:19.0.0-1739.g1bc250c7.el9.x86_64 156/202 2024-03-04T09:19:10.806 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 157/202 2024-03-04T09:19:10.985 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : cephfs-shell-2:19.0.0-1739.g1bc250c7.el9.x86_64 158/202 2024-03-04T09:19:11.084 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : libtool-2.4.6-45.el9.x86_64 149/202 2024-03-04T09:19:11.097 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : postgresql-contrib-13.11-1.el9.x86_64 159/202 2024-03-04T09:19:11.114 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : ceph-radosgw-2:19.0.0-1739.g1bc250c7.el9.x86_64 150/202 2024-03-04T09:19:11.136 INFO:teuthology.orchestra.run.smithi052.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-1739.g1bc250c7.el9.x86_64 150/202 2024-03-04T09:19:11.136 INFO:teuthology.orchestra.run.smithi052.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-04T09:19:11.136 INFO:teuthology.orchestra.run.smithi052.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-03-04T09:19:11.136 INFO:teuthology.orchestra.run.smithi052.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-04T09:19:11.137 INFO:teuthology.orchestra.run.smithi052.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-04T09:19:11.137 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-03-04T09:19:11.146 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : ceph-fuse-2:19.0.0-1739.g1bc250c7.el9.x86_64 160/202 2024-03-04T09:19:11.182 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-1739.g1bc25 151/202 2024-03-04T09:19:11.185 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 161/202 2024-03-04T09:19:11.202 INFO:teuthology.orchestra.run.smithi052.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-1739.g1bc25 151/202 2024-03-04T09:19:11.203 INFO:teuthology.orchestra.run.smithi052.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-04T09:19:11.203 INFO:teuthology.orchestra.run.smithi052.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-03-04T09:19:11.203 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-03-04T09:19:11.229 INFO:teuthology.orchestra.run.smithi191.stdout: Upgrading : xfsdump-3.1.12-4.el9.x86_64 162/202 2024-03-04T09:19:11.257 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : cephfs-mirror-2:19.0.0-1739.g1bc250c7.el9.x86_64 152/202 2024-03-04T09:19:11.279 INFO:teuthology.orchestra.run.smithi052.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-1739.g1bc250c7.el9.x86_64 152/202 2024-03-04T09:19:11.279 INFO:teuthology.orchestra.run.smithi052.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-04T09:19:11.279 INFO:teuthology.orchestra.run.smithi052.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-03-04T09:19:11.279 INFO:teuthology.orchestra.run.smithi052.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-03-04T09:19:11.279 INFO:teuthology.orchestra.run.smithi052.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-03-04T09:19:11.279 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-03-04T09:19:11.304 INFO:teuthology.orchestra.run.smithi191.stdout: Upgrading : util-linux-user-2.37.4-18.el9.x86_64 163/202 2024-03-04T09:19:11.339 INFO:teuthology.orchestra.run.smithi191.stdout: Upgrading : iproute-tc-6.7.0-2.el9.x86_64 164/202 2024-03-04T09:19:11.383 INFO:teuthology.orchestra.run.smithi191.stdout: Upgrading : elfutils-debuginfod-client-0.190-2.el9.x86_64 165/202 2024-03-04T09:19:11.413 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : libcephfs-devel-2:19.0.0-1739.g1bc250c7.el9.x86_ 166/202 2024-03-04T09:19:11.443 INFO:teuthology.orchestra.run.smithi191.stdout: Upgrading : NetworkManager-team-1:1.46.0-1.el9.x86_64 167/202 2024-03-04T09:19:11.499 INFO:teuthology.orchestra.run.smithi191.stdout: Upgrading : NetworkManager-tui-1:1.46.0-1.el9.x86_64 168/202 2024-03-04T09:19:11.548 INFO:teuthology.orchestra.run.smithi191.stdout: Upgrading : libacl-devel-2.3.1-4.el9.x86_64 169/202 2024-03-04T09:19:11.591 INFO:teuthology.orchestra.run.smithi191.stdout: Upgrading : acl-2.3.1-4.el9.x86_64 170/202 2024-03-04T09:19:11.599 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : rbd-mirror-2:19.0.0-1739.g1bc250c7.el9.x86_64 153/202 2024-03-04T09:19:11.621 INFO:teuthology.orchestra.run.smithi052.stdout: Running scriptlet: rbd-mirror-2:19.0.0-1739.g1bc250c7.el9.x86_64 153/202 2024-03-04T09:19:11.621 INFO:teuthology.orchestra.run.smithi052.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-04T09:19:11.622 INFO:teuthology.orchestra.run.smithi052.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-03-04T09:19:11.622 INFO:teuthology.orchestra.run.smithi052.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-04T09:19:11.622 INFO:teuthology.orchestra.run.smithi052.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-04T09:19:11.622 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-03-04T09:19:12.169 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 171/202 2024-03-04T09:19:12.206 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : postgresql-server-13.11-1.el9.x86_64 171/202 2024-03-04T09:19:12.226 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 171/202 2024-03-04T09:19:12.601 INFO:teuthology.orchestra.run.smithi106.stdout: Running scriptlet: openssl-libs-1:3.0.7-24.el9.x86_64 202/202 2024-03-04T09:19:12.601 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : ceph-2:19.0.0-1739.g1bc250c7.el9.x86_64 1/202 2024-03-04T09:19:12.602 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : ceph-base-2:19.0.0-1739.g1bc250c7.el9.x86_64 2/202 2024-03-04T09:19:12.602 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : ceph-common-2:19.0.0-1739.g1bc250c7.el9.x86_64 3/202 2024-03-04T09:19:12.602 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : ceph-fuse-2:19.0.0-1739.g1bc250c7.el9.x86_64 4/202 2024-03-04T09:19:12.602 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-1739.g1bc25 5/202 2024-03-04T09:19:12.602 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : ceph-mds-2:19.0.0-1739.g1bc250c7.el9.x86_64 6/202 2024-03-04T09:19:12.602 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : ceph-mgr-2:19.0.0-1739.g1bc250c7.el9.x86_64 7/202 2024-03-04T09:19:12.602 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : ceph-mon-2:19.0.0-1739.g1bc250c7.el9.x86_64 8/202 2024-03-04T09:19:12.602 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : ceph-osd-2:19.0.0-1739.g1bc250c7.el9.x86_64 9/202 2024-03-04T09:19:12.602 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : ceph-radosgw-2:19.0.0-1739.g1bc250c7.el9.x86_64 10/202 2024-03-04T09:19:12.602 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : ceph-selinux-2:19.0.0-1739.g1bc250c7.el9.x86_64 11/202 2024-03-04T09:19:12.603 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : ceph-test-2:19.0.0-1739.g1bc250c7.el9.x86_64 12/202 2024-03-04T09:19:12.603 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : cephfs-mirror-2:19.0.0-1739.g1bc250c7.el9.x86_64 13/202 2024-03-04T09:19:12.603 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : cephfs-shell-2:19.0.0-1739.g1bc250c7.el9.x86_64 14/202 2024-03-04T09:19:12.603 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : libcephfs-devel-2:19.0.0-1739.g1bc250c7.el9.x86_ 15/202 2024-03-04T09:19:12.603 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : libcephfs2-2:19.0.0-1739.g1bc250c7.el9.x86_64 16/202 2024-03-04T09:19:12.603 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : libcephsqlite-2:19.0.0-1739.g1bc250c7.el9.x86_64 17/202 2024-03-04T09:19:12.603 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : librados-devel-2:19.0.0-1739.g1bc250c7.el9.x86_6 18/202 2024-03-04T09:19:12.603 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : libradosstriper1-2:19.0.0-1739.g1bc250c7.el9.x86 19/202 2024-03-04T09:19:12.603 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : librgw2-2:19.0.0-1739.g1bc250c7.el9.x86_64 20/202 2024-03-04T09:19:12.603 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-ceph-argparse-2:19.0.0-1739.g1bc250c7.el 21/202 2024-03-04T09:19:12.603 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-ceph-common-2:19.0.0-1739.g1bc250c7.el9. 22/202 2024-03-04T09:19:12.604 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-cephfs-2:19.0.0-1739.g1bc250c7.el9.x86_6 23/202 2024-03-04T09:19:12.604 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-rados-2:19.0.0-1739.g1bc250c7.el9.x86_64 24/202 2024-03-04T09:19:12.604 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-rbd-2:19.0.0-1739.g1bc250c7.el9.x86_64 25/202 2024-03-04T09:19:12.604 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-rgw-2:19.0.0-1739.g1bc250c7.el9.x86_64 26/202 2024-03-04T09:19:12.604 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : rbd-fuse-2:19.0.0-1739.g1bc250c7.el9.x86_64 27/202 2024-03-04T09:19:12.604 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : rbd-mirror-2:19.0.0-1739.g1bc250c7.el9.x86_64 28/202 2024-03-04T09:19:12.604 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : rbd-nbd-2:19.0.0-1739.g1bc250c7.el9.x86_64 29/202 2024-03-04T09:19:12.604 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-1739.g1bc250c7. 30/202 2024-03-04T09:19:12.604 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-1739.g1bc250c7.el9.noa 31/202 2024-03-04T09:19:12.604 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-1739.g1bc250c7.el9.n 32/202 2024-03-04T09:19:12.605 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-1739.g1bc 33/202 2024-03-04T09:19:12.605 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-1739.g1bc250c7.el 34/202 2024-03-04T09:19:12.605 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : ceph-mgr-rook-2:19.0.0-1739.g1bc250c7.el9.noarch 35/202 2024-03-04T09:19:12.605 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-1739.g1bc250c7.e 36/202 2024-03-04T09:19:12.605 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : ceph-volume-2:19.0.0-1739.g1bc250c7.el9.noarch 37/202 2024-03-04T09:19:12.605 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : cephadm-2:19.0.0-1739.g1bc250c7.el9.noarch 38/202 2024-03-04T09:19:12.605 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : cephfs-top-2:19.0.0-1739.g1bc250c7.el9.noarch 39/202 2024-03-04T09:19:12.606 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : automake-1.16.2-8.el9.noarch 40/202 2024-03-04T09:19:12.606 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : bison-3.7.4-5.el9.x86_64 41/202 2024-03-04T09:19:12.606 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 42/202 2024-03-04T09:19:12.606 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 43/202 2024-03-04T09:19:12.606 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : flex-2.6.4-9.el9.x86_64 44/202 2024-03-04T09:19:12.606 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 45/202 2024-03-04T09:19:12.606 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 46/202 2024-03-04T09:19:12.607 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 47/202 2024-03-04T09:19:12.607 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 48/202 2024-03-04T09:19:12.607 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 49/202 2024-03-04T09:19:12.607 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 50/202 2024-03-04T09:19:12.607 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 51/202 2024-03-04T09:19:12.608 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 52/202 2024-03-04T09:19:12.608 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 53/202 2024-03-04T09:19:12.608 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 54/202 2024-03-04T09:19:12.608 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 55/202 2024-03-04T09:19:12.608 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : lua-5.4.4-4.el9.x86_64 56/202 2024-03-04T09:19:12.608 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 57/202 2024-03-04T09:19:12.609 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 58/202 2024-03-04T09:19:12.609 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : openssl-devel-1:3.0.7-27.el9.x86_64 59/202 2024-03-04T09:19:12.609 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : postgresql-13.11-1.el9.x86_64 60/202 2024-03-04T09:19:12.609 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : postgresql-contrib-13.11-1.el9.x86_64 61/202 2024-03-04T09:19:12.609 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : postgresql-private-libs-13.11-1.el9.x86_64 62/202 2024-03-04T09:19:12.609 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : postgresql-server-13.11-1.el9.x86_64 63/202 2024-03-04T09:19:12.610 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 64/202 2024-03-04T09:19:12.610 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 65/202 2024-03-04T09:19:12.610 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 66/202 2024-03-04T09:19:12.610 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 67/202 2024-03-04T09:19:12.611 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 68/202 2024-03-04T09:19:12.611 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 69/202 2024-03-04T09:19:12.611 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 70/202 2024-03-04T09:19:12.611 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 71/202 2024-03-04T09:19:12.611 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 72/202 2024-03-04T09:19:12.611 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 73/202 2024-03-04T09:19:12.611 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 74/202 2024-03-04T09:19:12.612 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 75/202 2024-03-04T09:19:12.612 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 76/202 2024-03-04T09:19:12.612 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 77/202 2024-03-04T09:19:12.612 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 78/202 2024-03-04T09:19:12.612 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 79/202 2024-03-04T09:19:12.612 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 80/202 2024-03-04T09:19:12.613 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 81/202 2024-03-04T09:19:12.613 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 82/202 2024-03-04T09:19:12.613 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 83/202 2024-03-04T09:19:12.613 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 84/202 2024-03-04T09:19:12.613 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : qatlib-23.11.0-1.el9.x86_64 85/202 2024-03-04T09:19:12.613 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : qatlib-service-23.11.0-1.el9.x86_64 86/202 2024-03-04T09:19:12.613 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : qatzip-libs-1.1.2-2.el9.x86_64 87/202 2024-03-04T09:19:12.614 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 88/202 2024-03-04T09:19:12.614 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 89/202 2024-03-04T09:19:12.614 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 90/202 2024-03-04T09:19:12.614 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 91/202 2024-03-04T09:19:12.614 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : zlib-devel-1.2.11-40.el9.x86_64 92/202 2024-03-04T09:19:12.615 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 93/202 2024-03-04T09:19:12.615 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : jq-1.6-16.el9.x86_64 94/202 2024-03-04T09:19:12.615 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 95/202 2024-03-04T09:19:12.615 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 96/202 2024-03-04T09:19:12.616 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 97/202 2024-03-04T09:19:12.616 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 98/202 2024-03-04T09:19:12.616 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 99/202 2024-03-04T09:19:12.616 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-ply-3.11-14.el9.noarch 100/202 2024-03-04T09:19:12.616 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 101/202 2024-03-04T09:19:12.617 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 102/202 2024-03-04T09:19:12.617 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 103/202 2024-03-04T09:19:12.617 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 104/202 2024-03-04T09:19:12.617 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 105/202 2024-03-04T09:19:12.617 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 106/202 2024-03-04T09:19:12.617 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 107/202 2024-03-04T09:19:12.618 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 108/202 2024-03-04T09:19:12.618 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 109/202 2024-03-04T09:19:12.618 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 110/202 2024-03-04T09:19:12.618 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 111/202 2024-03-04T09:19:12.618 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 112/202 2024-03-04T09:19:12.618 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 113/202 2024-03-04T09:19:12.618 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 114/202 2024-03-04T09:19:12.619 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 115/202 2024-03-04T09:19:12.619 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-asyncssh-2.13.2-4.el9.noarch 116/202 2024-03-04T09:19:12.619 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 117/202 2024-03-04T09:19:12.619 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 118/202 2024-03-04T09:19:12.619 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 119/202 2024-03-04T09:19:12.619 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 120/202 2024-03-04T09:19:12.620 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 121/202 2024-03-04T09:19:12.620 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 122/202 2024-03-04T09:19:12.620 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 123/202 2024-03-04T09:19:12.620 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-google-auth-1:2.28.0-1.el9.noarch 124/202 2024-03-04T09:19:12.620 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 125/202 2024-03-04T09:19:12.620 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 126/202 2024-03-04T09:19:12.620 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 127/202 2024-03-04T09:19:12.621 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 128/202 2024-03-04T09:19:12.621 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 129/202 2024-03-04T09:19:12.621 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 130/202 2024-03-04T09:19:12.621 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 131/202 2024-03-04T09:19:12.621 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 132/202 2024-03-04T09:19:12.621 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 133/202 2024-03-04T09:19:12.622 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 134/202 2024-03-04T09:19:12.622 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 135/202 2024-03-04T09:19:12.622 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 136/202 2024-03-04T09:19:12.622 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 137/202 2024-03-04T09:19:12.622 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 138/202 2024-03-04T09:19:12.622 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 139/202 2024-03-04T09:19:12.622 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 140/202 2024-03-04T09:19:12.623 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 141/202 2024-03-04T09:19:12.623 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 142/202 2024-03-04T09:19:12.623 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 143/202 2024-03-04T09:19:12.623 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 144/202 2024-03-04T09:19:12.623 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 145/202 2024-03-04T09:19:12.623 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 146/202 2024-03-04T09:19:12.623 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 147/202 2024-03-04T09:19:12.623 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 148/202 2024-03-04T09:19:12.624 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : re2-1:20211101-3.el9.x86_64 149/202 2024-03-04T09:19:12.624 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 150/202 2024-03-04T09:19:12.624 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : librados2-2:19.0.0-1739.g1bc250c7.el9.x86_64 151/202 2024-03-04T09:19:12.624 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 152/202 2024-03-04T09:19:12.624 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : librbd1-2:19.0.0-1739.g1bc250c7.el9.x86_64 153/202 2024-03-04T09:19:12.624 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 154/202 2024-03-04T09:19:12.624 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : libacl-devel-2.3.1-4.el9.x86_64 155/202 2024-03-04T09:19:12.625 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : libacl-devel-2.3.1-3.el9.x86_64 156/202 2024-03-04T09:19:12.625 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : NetworkManager-1:1.46.0-1.el9.x86_64 157/202 2024-03-04T09:19:12.625 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : NetworkManager-1:1.44.0-2.el9.x86_64 158/202 2024-03-04T09:19:12.625 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : NetworkManager-libnm-1:1.46.0-1.el9.x86_64 159/202 2024-03-04T09:19:12.625 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 160/202 2024-03-04T09:19:12.625 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : NetworkManager-team-1:1.46.0-1.el9.x86_64 161/202 2024-03-04T09:19:12.626 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : NetworkManager-team-1:1.44.0-2.el9.x86_64 162/202 2024-03-04T09:19:12.626 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : NetworkManager-tui-1:1.46.0-1.el9.x86_64 163/202 2024-03-04T09:19:12.626 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : NetworkManager-tui-1:1.44.0-2.el9.x86_64 164/202 2024-03-04T09:19:12.626 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : acl-2.3.1-4.el9.x86_64 165/202 2024-03-04T09:19:12.626 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : acl-2.3.1-3.el9.x86_64 166/202 2024-03-04T09:19:12.626 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 167/202 2024-03-04T09:19:12.626 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : elfutils-debuginfod-client-0.189-3.el9.x86_64 168/202 2024-03-04T09:19:12.627 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 169/202 2024-03-04T09:19:12.627 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : elfutils-libelf-0.189-3.el9.x86_64 170/202 2024-03-04T09:19:12.627 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 171/202 2024-03-04T09:19:12.627 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : elfutils-libs-0.189-3.el9.x86_64 172/202 2024-03-04T09:19:12.627 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : iproute-6.7.0-2.el9.x86_64 173/202 2024-03-04T09:19:12.627 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : iproute-6.2.0-5.el9.x86_64 174/202 2024-03-04T09:19:12.628 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : iproute-tc-6.7.0-2.el9.x86_64 175/202 2024-03-04T09:19:12.628 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : iproute-tc-6.2.0-5.el9.x86_64 176/202 2024-03-04T09:19:12.628 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : libacl-2.3.1-4.el9.x86_64 177/202 2024-03-04T09:19:12.628 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : libacl-2.3.1-3.el9.x86_64 178/202 2024-03-04T09:19:12.628 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : libblkid-2.37.4-18.el9.x86_64 179/202 2024-03-04T09:19:12.629 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : libblkid-2.37.4-15.el9.x86_64 180/202 2024-03-04T09:19:12.629 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : libfdisk-2.37.4-18.el9.x86_64 181/202 2024-03-04T09:19:12.629 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : libfdisk-2.37.4-15.el9.x86_64 182/202 2024-03-04T09:19:12.629 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : libmount-2.37.4-18.el9.x86_64 183/202 2024-03-04T09:19:12.629 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : libmount-2.37.4-15.el9.x86_64 184/202 2024-03-04T09:19:12.629 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : libsmartcols-2.37.4-18.el9.x86_64 185/202 2024-03-04T09:19:12.630 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : libsmartcols-2.37.4-15.el9.x86_64 186/202 2024-03-04T09:19:12.630 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : libuuid-2.37.4-18.el9.x86_64 187/202 2024-03-04T09:19:12.630 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : libuuid-2.37.4-15.el9.x86_64 188/202 2024-03-04T09:19:12.630 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 189/202 2024-03-04T09:19:12.630 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : openssl-1:3.0.7-24.el9.x86_64 190/202 2024-03-04T09:19:12.630 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 191/202 2024-03-04T09:19:12.631 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : openssl-libs-1:3.0.7-24.el9.x86_64 192/202 2024-03-04T09:19:12.631 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : util-linux-2.37.4-18.el9.x86_64 193/202 2024-03-04T09:19:12.631 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : util-linux-2.37.4-15.el9.x86_64 194/202 2024-03-04T09:19:12.631 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : util-linux-core-2.37.4-18.el9.x86_64 195/202 2024-03-04T09:19:12.631 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : util-linux-core-2.37.4-15.el9.x86_64 196/202 2024-03-04T09:19:12.631 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : util-linux-user-2.37.4-18.el9.x86_64 197/202 2024-03-04T09:19:12.631 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : util-linux-user-2.37.4-15.el9.x86_64 198/202 2024-03-04T09:19:12.632 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : xfsdump-3.1.12-4.el9.x86_64 199/202 2024-03-04T09:19:12.632 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : xfsdump-3.1.12-3.el9.x86_64 200/202 2024-03-04T09:19:12.632 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : xfsprogs-6.3.0-1.el9.x86_64 201/202 2024-03-04T09:19:13.044 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : cephfs-top-2:19.0.0-1739.g1bc250c7.el9.noarch 172/202 2024-03-04T09:19:13.109 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : openssl-devel-1:3.0.7-27.el9.x86_64 173/202 2024-03-04T09:19:13.150 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 174/202 2024-03-04T09:19:13.230 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : flex-2.6.4-9.el9.x86_64 175/202 2024-03-04T09:19:13.271 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : bison-3.7.4-5.el9.x86_64 176/202 2024-03-04T09:19:13.296 INFO:teuthology.orchestra.run.smithi191.stdout: Cleanup : NetworkManager-tui-1:1.44.0-2.el9.x86_64 177/202 2024-03-04T09:19:13.322 INFO:teuthology.orchestra.run.smithi191.stdout: Cleanup : elfutils-debuginfod-client-0.189-3.el9.x86_64 178/202 2024-03-04T09:19:13.348 INFO:teuthology.orchestra.run.smithi191.stdout: Cleanup : elfutils-libs-0.189-3.el9.x86_64 179/202 2024-03-04T09:19:13.348 INFO:teuthology.orchestra.run.smithi191.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 180/202 2024-03-04T09:19:13.432 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 180/202 2024-03-04T09:19:13.460 INFO:teuthology.orchestra.run.smithi191.stdout: Cleanup : libacl-devel-2.3.1-3.el9.x86_64 181/202 2024-03-04T09:19:13.490 INFO:teuthology.orchestra.run.smithi191.stdout: Cleanup : openssl-1:3.0.7-24.el9.x86_64 182/202 2024-03-04T09:19:13.490 INFO:teuthology.orchestra.run.smithi191.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 183/202 2024-03-04T09:19:13.575 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 183/202 2024-03-04T09:19:13.601 INFO:teuthology.orchestra.run.smithi191.stdout: Cleanup : xfsdump-3.1.12-3.el9.x86_64 184/202 2024-03-04T09:19:13.635 INFO:teuthology.orchestra.run.smithi191.stdout: Cleanup : xfsprogs-5.19.0-4.el9.x86_64 185/202 2024-03-04T09:19:13.668 INFO:teuthology.orchestra.run.smithi191.stdout: Cleanup : iproute-tc-6.2.0-5.el9.x86_64 186/202 2024-03-04T09:19:13.696 INFO:teuthology.orchestra.run.smithi191.stdout: Cleanup : acl-2.3.1-3.el9.x86_64 187/202 2024-03-04T09:19:13.728 INFO:teuthology.orchestra.run.smithi191.stdout: Cleanup : iproute-6.2.0-5.el9.x86_64 188/202 2024-03-04T09:19:13.769 INFO:teuthology.orchestra.run.smithi191.stdout: Cleanup : util-linux-user-2.37.4-15.el9.x86_64 189/202 2024-03-04T09:19:13.817 INFO:teuthology.orchestra.run.smithi191.stdout: Cleanup : util-linux-2.37.4-15.el9.x86_64 190/202 2024-03-04T09:19:13.858 INFO:teuthology.orchestra.run.smithi191.stdout: Cleanup : util-linux-core-2.37.4-15.el9.x86_64 191/202 2024-03-04T09:19:13.883 INFO:teuthology.orchestra.run.smithi191.stdout: Cleanup : libfdisk-2.37.4-15.el9.x86_64 192/202 2024-03-04T09:19:13.937 INFO:teuthology.orchestra.run.smithi191.stdout: Cleanup : libmount-2.37.4-15.el9.x86_64 193/202 2024-03-04T09:19:13.967 INFO:teuthology.orchestra.run.smithi191.stdout: Cleanup : libblkid-2.37.4-15.el9.x86_64 194/202 2024-03-04T09:19:13.992 INFO:teuthology.orchestra.run.smithi191.stdout: Cleanup : NetworkManager-team-1:1.44.0-2.el9.x86_64 195/202 2024-03-04T09:19:13.999 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 196/202 2024-03-04T09:19:13.999 INFO:teuthology.orchestra.run.smithi191.stdout: Cleanup : NetworkManager-1:1.44.0-2.el9.x86_64 196/202 2024-03-04T09:19:14.178 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 196/202 2024-03-04T09:19:14.179 INFO:teuthology.orchestra.run.smithi191.stdout: Cleanup : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 197/202 2024-03-04T09:19:14.269 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: NetworkManager-libnm-1:1.44.0-2.el9.x86_64 197/202 2024-03-04T09:19:14.301 INFO:teuthology.orchestra.run.smithi191.stdout: Cleanup : libuuid-2.37.4-15.el9.x86_64 198/202 2024-03-04T09:19:14.328 INFO:teuthology.orchestra.run.smithi191.stdout: Cleanup : libsmartcols-2.37.4-15.el9.x86_64 199/202 2024-03-04T09:19:14.353 INFO:teuthology.orchestra.run.smithi191.stdout: Cleanup : elfutils-libelf-0.189-3.el9.x86_64 200/202 2024-03-04T09:19:14.379 INFO:teuthology.orchestra.run.smithi191.stdout: Cleanup : libacl-2.3.1-3.el9.x86_64 201/202 2024-03-04T09:19:14.442 INFO:teuthology.orchestra.run.smithi191.stdout: Cleanup : openssl-libs-1:3.0.7-24.el9.x86_64 202/202 2024-03-04T09:19:15.555 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : xfsprogs-5.19.0-4.el9.x86_64 202/202 2024-03-04T09:19:15.555 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-03-04T09:19:15.556 INFO:teuthology.orchestra.run.smithi106.stdout:Upgraded: 2024-03-04T09:19:15.556 INFO:teuthology.orchestra.run.smithi106.stdout: NetworkManager-1:1.46.0-1.el9.x86_64 2024-03-04T09:19:15.556 INFO:teuthology.orchestra.run.smithi106.stdout: NetworkManager-libnm-1:1.46.0-1.el9.x86_64 2024-03-04T09:19:15.556 INFO:teuthology.orchestra.run.smithi106.stdout: NetworkManager-team-1:1.46.0-1.el9.x86_64 2024-03-04T09:19:15.556 INFO:teuthology.orchestra.run.smithi106.stdout: NetworkManager-tui-1:1.46.0-1.el9.x86_64 2024-03-04T09:19:15.556 INFO:teuthology.orchestra.run.smithi106.stdout: acl-2.3.1-4.el9.x86_64 2024-03-04T09:19:15.556 INFO:teuthology.orchestra.run.smithi106.stdout: elfutils-debuginfod-client-0.190-2.el9.x86_64 2024-03-04T09:19:15.556 INFO:teuthology.orchestra.run.smithi106.stdout: elfutils-libelf-0.190-2.el9.x86_64 2024-03-04T09:19:15.556 INFO:teuthology.orchestra.run.smithi106.stdout: elfutils-libs-0.190-2.el9.x86_64 2024-03-04T09:19:15.556 INFO:teuthology.orchestra.run.smithi106.stdout: iproute-6.7.0-2.el9.x86_64 2024-03-04T09:19:15.556 INFO:teuthology.orchestra.run.smithi106.stdout: iproute-tc-6.7.0-2.el9.x86_64 2024-03-04T09:19:15.556 INFO:teuthology.orchestra.run.smithi106.stdout: libacl-2.3.1-4.el9.x86_64 2024-03-04T09:19:15.557 INFO:teuthology.orchestra.run.smithi106.stdout: libacl-devel-2.3.1-4.el9.x86_64 2024-03-04T09:19:15.557 INFO:teuthology.orchestra.run.smithi106.stdout: libblkid-2.37.4-18.el9.x86_64 2024-03-04T09:19:15.557 INFO:teuthology.orchestra.run.smithi106.stdout: libfdisk-2.37.4-18.el9.x86_64 2024-03-04T09:19:15.557 INFO:teuthology.orchestra.run.smithi106.stdout: libmount-2.37.4-18.el9.x86_64 2024-03-04T09:19:15.557 INFO:teuthology.orchestra.run.smithi106.stdout: librados2-2:19.0.0-1739.g1bc250c7.el9.x86_64 2024-03-04T09:19:15.557 INFO:teuthology.orchestra.run.smithi106.stdout: librbd1-2:19.0.0-1739.g1bc250c7.el9.x86_64 2024-03-04T09:19:15.557 INFO:teuthology.orchestra.run.smithi106.stdout: libsmartcols-2.37.4-18.el9.x86_64 2024-03-04T09:19:15.557 INFO:teuthology.orchestra.run.smithi106.stdout: libuuid-2.37.4-18.el9.x86_64 2024-03-04T09:19:15.557 INFO:teuthology.orchestra.run.smithi106.stdout: openssl-1:3.0.7-27.el9.x86_64 2024-03-04T09:19:15.557 INFO:teuthology.orchestra.run.smithi106.stdout: openssl-libs-1:3.0.7-27.el9.x86_64 2024-03-04T09:19:15.557 INFO:teuthology.orchestra.run.smithi106.stdout: util-linux-2.37.4-18.el9.x86_64 2024-03-04T09:19:15.557 INFO:teuthology.orchestra.run.smithi106.stdout: util-linux-core-2.37.4-18.el9.x86_64 2024-03-04T09:19:15.557 INFO:teuthology.orchestra.run.smithi106.stdout: util-linux-user-2.37.4-18.el9.x86_64 2024-03-04T09:19:15.557 INFO:teuthology.orchestra.run.smithi106.stdout: xfsdump-3.1.12-4.el9.x86_64 2024-03-04T09:19:15.557 INFO:teuthology.orchestra.run.smithi106.stdout: xfsprogs-6.3.0-1.el9.x86_64 2024-03-04T09:19:15.558 INFO:teuthology.orchestra.run.smithi106.stdout:Installed: 2024-03-04T09:19:15.558 INFO:teuthology.orchestra.run.smithi106.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-03-04T09:19:15.558 INFO:teuthology.orchestra.run.smithi106.stdout: automake-1.16.2-8.el9.noarch 2024-03-04T09:19:15.558 INFO:teuthology.orchestra.run.smithi106.stdout: bison-3.7.4-5.el9.x86_64 2024-03-04T09:19:15.558 INFO:teuthology.orchestra.run.smithi106.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-03-04T09:19:15.558 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-2:19.0.0-1739.g1bc250c7.el9.x86_64 2024-03-04T09:19:15.558 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-base-2:19.0.0-1739.g1bc250c7.el9.x86_64 2024-03-04T09:19:15.558 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-common-2:19.0.0-1739.g1bc250c7.el9.x86_64 2024-03-04T09:19:15.558 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-fuse-2:19.0.0-1739.g1bc250c7.el9.x86_64 2024-03-04T09:19:15.558 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-grafana-dashboards-2:19.0.0-1739.g1bc250c7.el9.noarch 2024-03-04T09:19:15.558 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-immutable-object-cache-2:19.0.0-1739.g1bc250c7.el9.x86_64 2024-03-04T09:19:15.558 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-mds-2:19.0.0-1739.g1bc250c7.el9.x86_64 2024-03-04T09:19:15.558 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-mgr-2:19.0.0-1739.g1bc250c7.el9.x86_64 2024-03-04T09:19:15.558 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-mgr-cephadm-2:19.0.0-1739.g1bc250c7.el9.noarch 2024-03-04T09:19:15.559 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-mgr-dashboard-2:19.0.0-1739.g1bc250c7.el9.noarch 2024-03-04T09:19:15.559 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-mgr-diskprediction-local-2:19.0.0-1739.g1bc250c7.el9.noarch 2024-03-04T09:19:15.559 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-mgr-modules-core-2:19.0.0-1739.g1bc250c7.el9.noarch 2024-03-04T09:19:15.559 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-mgr-rook-2:19.0.0-1739.g1bc250c7.el9.noarch 2024-03-04T09:19:15.559 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-mon-2:19.0.0-1739.g1bc250c7.el9.x86_64 2024-03-04T09:19:15.559 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-osd-2:19.0.0-1739.g1bc250c7.el9.x86_64 2024-03-04T09:19:15.559 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-prometheus-alerts-2:19.0.0-1739.g1bc250c7.el9.noarch 2024-03-04T09:19:15.559 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-radosgw-2:19.0.0-1739.g1bc250c7.el9.x86_64 2024-03-04T09:19:15.559 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-selinux-2:19.0.0-1739.g1bc250c7.el9.x86_64 2024-03-04T09:19:15.559 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-test-2:19.0.0-1739.g1bc250c7.el9.x86_64 2024-03-04T09:19:15.559 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-volume-2:19.0.0-1739.g1bc250c7.el9.noarch 2024-03-04T09:19:15.559 INFO:teuthology.orchestra.run.smithi106.stdout: cephadm-2:19.0.0-1739.g1bc250c7.el9.noarch 2024-03-04T09:19:15.559 INFO:teuthology.orchestra.run.smithi106.stdout: cephfs-mirror-2:19.0.0-1739.g1bc250c7.el9.x86_64 2024-03-04T09:19:15.560 INFO:teuthology.orchestra.run.smithi106.stdout: cephfs-shell-2:19.0.0-1739.g1bc250c7.el9.x86_64 2024-03-04T09:19:15.560 INFO:teuthology.orchestra.run.smithi106.stdout: cephfs-top-2:19.0.0-1739.g1bc250c7.el9.noarch 2024-03-04T09:19:15.560 INFO:teuthology.orchestra.run.smithi106.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-03-04T09:19:15.560 INFO:teuthology.orchestra.run.smithi106.stdout: flex-2.6.4-9.el9.x86_64 2024-03-04T09:19:15.560 INFO:teuthology.orchestra.run.smithi106.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-03-04T09:19:15.560 INFO:teuthology.orchestra.run.smithi106.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-03-04T09:19:15.560 INFO:teuthology.orchestra.run.smithi106.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-03-04T09:19:15.560 INFO:teuthology.orchestra.run.smithi106.stdout: fuse-2.9.9-15.el9.x86_64 2024-03-04T09:19:15.560 INFO:teuthology.orchestra.run.smithi106.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-03-04T09:19:15.560 INFO:teuthology.orchestra.run.smithi106.stdout: grpc-data-1.46.7-10.el9.noarch 2024-03-04T09:19:15.560 INFO:teuthology.orchestra.run.smithi106.stdout: jq-1.6-16.el9.x86_64 2024-03-04T09:19:15.560 INFO:teuthology.orchestra.run.smithi106.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-03-04T09:19:15.560 INFO:teuthology.orchestra.run.smithi106.stdout: libarrow-9.0.0-10.el9.x86_64 2024-03-04T09:19:15.561 INFO:teuthology.orchestra.run.smithi106.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-03-04T09:19:15.561 INFO:teuthology.orchestra.run.smithi106.stdout: libcephfs-devel-2:19.0.0-1739.g1bc250c7.el9.x86_64 2024-03-04T09:19:15.561 INFO:teuthology.orchestra.run.smithi106.stdout: libcephfs2-2:19.0.0-1739.g1bc250c7.el9.x86_64 2024-03-04T09:19:15.561 INFO:teuthology.orchestra.run.smithi106.stdout: libcephsqlite-2:19.0.0-1739.g1bc250c7.el9.x86_64 2024-03-04T09:19:15.561 INFO:teuthology.orchestra.run.smithi106.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-03-04T09:19:15.561 INFO:teuthology.orchestra.run.smithi106.stdout: liboath-2.6.7-2.el9.x86_64 2024-03-04T09:19:15.561 INFO:teuthology.orchestra.run.smithi106.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-03-04T09:19:15.561 INFO:teuthology.orchestra.run.smithi106.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-03-04T09:19:15.561 INFO:teuthology.orchestra.run.smithi106.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-03-04T09:19:15.561 INFO:teuthology.orchestra.run.smithi106.stdout: librados-devel-2:19.0.0-1739.g1bc250c7.el9.x86_64 2024-03-04T09:19:15.561 INFO:teuthology.orchestra.run.smithi106.stdout: libradosstriper1-2:19.0.0-1739.g1bc250c7.el9.x86_64 2024-03-04T09:19:15.561 INFO:teuthology.orchestra.run.smithi106.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-03-04T09:19:15.561 INFO:teuthology.orchestra.run.smithi106.stdout: librgw2-2:19.0.0-1739.g1bc250c7.el9.x86_64 2024-03-04T09:19:15.562 INFO:teuthology.orchestra.run.smithi106.stdout: libtool-2.4.6-45.el9.x86_64 2024-03-04T09:19:15.562 INFO:teuthology.orchestra.run.smithi106.stdout: libunwind-1.6.2-1.el9.x86_64 2024-03-04T09:19:15.562 INFO:teuthology.orchestra.run.smithi106.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-03-04T09:19:15.562 INFO:teuthology.orchestra.run.smithi106.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-03-04T09:19:15.562 INFO:teuthology.orchestra.run.smithi106.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-03-04T09:19:15.562 INFO:teuthology.orchestra.run.smithi106.stdout: lua-5.4.4-4.el9.x86_64 2024-03-04T09:19:15.562 INFO:teuthology.orchestra.run.smithi106.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-03-04T09:19:15.562 INFO:teuthology.orchestra.run.smithi106.stdout: luarocks-3.9.2-1.el9.noarch 2024-03-04T09:19:15.562 INFO:teuthology.orchestra.run.smithi106.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2024-03-04T09:19:15.562 INFO:teuthology.orchestra.run.smithi106.stdout: openblas-0.3.21-2.el9.x86_64 2024-03-04T09:19:15.562 INFO:teuthology.orchestra.run.smithi106.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-03-04T09:19:15.562 INFO:teuthology.orchestra.run.smithi106.stdout: openssl-devel-1:3.0.7-27.el9.x86_64 2024-03-04T09:19:15.562 INFO:teuthology.orchestra.run.smithi106.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-03-04T09:19:15.562 INFO:teuthology.orchestra.run.smithi106.stdout: postgresql-13.11-1.el9.x86_64 2024-03-04T09:19:15.563 INFO:teuthology.orchestra.run.smithi106.stdout: postgresql-contrib-13.11-1.el9.x86_64 2024-03-04T09:19:15.563 INFO:teuthology.orchestra.run.smithi106.stdout: postgresql-private-libs-13.11-1.el9.x86_64 2024-03-04T09:19:15.563 INFO:teuthology.orchestra.run.smithi106.stdout: postgresql-server-13.11-1.el9.x86_64 2024-03-04T09:19:15.563 INFO:teuthology.orchestra.run.smithi106.stdout: protobuf-3.14.0-13.el9.x86_64 2024-03-04T09:19:15.563 INFO:teuthology.orchestra.run.smithi106.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-03-04T09:19:15.563 INFO:teuthology.orchestra.run.smithi106.stdout: python3-asyncssh-2.13.2-4.el9.noarch 2024-03-04T09:19:15.563 INFO:teuthology.orchestra.run.smithi106.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-03-04T09:19:15.563 INFO:teuthology.orchestra.run.smithi106.stdout: python3-babel-2.9.1-2.el9.noarch 2024-03-04T09:19:15.563 INFO:teuthology.orchestra.run.smithi106.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-03-04T09:19:15.563 INFO:teuthology.orchestra.run.smithi106.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-03-04T09:19:15.563 INFO:teuthology.orchestra.run.smithi106.stdout: python3-ceph-argparse-2:19.0.0-1739.g1bc250c7.el9.x86_64 2024-03-04T09:19:15.563 INFO:teuthology.orchestra.run.smithi106.stdout: python3-ceph-common-2:19.0.0-1739.g1bc250c7.el9.x86_64 2024-03-04T09:19:15.563 INFO:teuthology.orchestra.run.smithi106.stdout: python3-cephfs-2:19.0.0-1739.g1bc250c7.el9.x86_64 2024-03-04T09:19:15.563 INFO:teuthology.orchestra.run.smithi106.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-03-04T09:19:15.564 INFO:teuthology.orchestra.run.smithi106.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-03-04T09:19:15.564 INFO:teuthology.orchestra.run.smithi106.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-03-04T09:19:15.564 INFO:teuthology.orchestra.run.smithi106.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-03-04T09:19:15.564 INFO:teuthology.orchestra.run.smithi106.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-03-04T09:19:15.564 INFO:teuthology.orchestra.run.smithi106.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-03-04T09:19:15.564 INFO:teuthology.orchestra.run.smithi106.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-03-04T09:19:15.564 INFO:teuthology.orchestra.run.smithi106.stdout: python3-devel-3.9.17-2.el9.x86_64 2024-03-04T09:19:15.564 INFO:teuthology.orchestra.run.smithi106.stdout: python3-google-auth-1:2.28.0-1.el9.noarch 2024-03-04T09:19:15.564 INFO:teuthology.orchestra.run.smithi106.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-03-04T09:19:15.564 INFO:teuthology.orchestra.run.smithi106.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-03-04T09:19:15.564 INFO:teuthology.orchestra.run.smithi106.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-03-04T09:19:15.564 INFO:teuthology.orchestra.run.smithi106.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-03-04T09:19:15.565 INFO:teuthology.orchestra.run.smithi106.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-03-04T09:19:15.565 INFO:teuthology.orchestra.run.smithi106.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-03-04T09:19:15.565 INFO:teuthology.orchestra.run.smithi106.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-03-04T09:19:15.565 INFO:teuthology.orchestra.run.smithi106.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-03-04T09:19:15.565 INFO:teuthology.orchestra.run.smithi106.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-03-04T09:19:15.565 INFO:teuthology.orchestra.run.smithi106.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-03-04T09:19:15.565 INFO:teuthology.orchestra.run.smithi106.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-03-04T09:19:15.565 INFO:teuthology.orchestra.run.smithi106.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-03-04T09:19:15.565 INFO:teuthology.orchestra.run.smithi106.stdout: python3-mako-1.1.4-6.el9.noarch 2024-03-04T09:19:15.565 INFO:teuthology.orchestra.run.smithi106.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-03-04T09:19:15.565 INFO:teuthology.orchestra.run.smithi106.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-03-04T09:19:15.565 INFO:teuthology.orchestra.run.smithi106.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-03-04T09:19:15.565 INFO:teuthology.orchestra.run.smithi106.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-03-04T09:19:15.566 INFO:teuthology.orchestra.run.smithi106.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-03-04T09:19:15.566 INFO:teuthology.orchestra.run.smithi106.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-03-04T09:19:15.566 INFO:teuthology.orchestra.run.smithi106.stdout: python3-packaging-20.9-5.el9.noarch 2024-03-04T09:19:15.566 INFO:teuthology.orchestra.run.smithi106.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-03-04T09:19:15.566 INFO:teuthology.orchestra.run.smithi106.stdout: python3-ply-3.11-14.el9.noarch 2024-03-04T09:19:15.566 INFO:teuthology.orchestra.run.smithi106.stdout: python3-portend-3.1.0-2.el9.noarch 2024-03-04T09:19:15.566 INFO:teuthology.orchestra.run.smithi106.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-03-04T09:19:15.566 INFO:teuthology.orchestra.run.smithi106.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-03-04T09:19:15.566 INFO:teuthology.orchestra.run.smithi106.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-03-04T09:19:15.566 INFO:teuthology.orchestra.run.smithi106.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-03-04T09:19:15.566 INFO:teuthology.orchestra.run.smithi106.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-03-04T09:19:15.566 INFO:teuthology.orchestra.run.smithi106.stdout: python3-pycparser-2.20-6.el9.noarch 2024-03-04T09:19:15.566 INFO:teuthology.orchestra.run.smithi106.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-03-04T09:19:15.567 INFO:teuthology.orchestra.run.smithi106.stdout: python3-pytz-2021.1-5.el9.noarch 2024-03-04T09:19:15.567 INFO:teuthology.orchestra.run.smithi106.stdout: python3-rados-2:19.0.0-1739.g1bc250c7.el9.x86_64 2024-03-04T09:19:15.567 INFO:teuthology.orchestra.run.smithi106.stdout: python3-rbd-2:19.0.0-1739.g1bc250c7.el9.x86_64 2024-03-04T09:19:15.567 INFO:teuthology.orchestra.run.smithi106.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-03-04T09:19:15.567 INFO:teuthology.orchestra.run.smithi106.stdout: python3-requests-2.25.1-8.el9.noarch 2024-03-04T09:19:15.567 INFO:teuthology.orchestra.run.smithi106.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-03-04T09:19:15.567 INFO:teuthology.orchestra.run.smithi106.stdout: python3-rgw-2:19.0.0-1739.g1bc250c7.el9.x86_64 2024-03-04T09:19:15.567 INFO:teuthology.orchestra.run.smithi106.stdout: python3-routes-2.5.1-5.el9.noarch 2024-03-04T09:19:15.567 INFO:teuthology.orchestra.run.smithi106.stdout: python3-rsa-4.9-2.el9.noarch 2024-03-04T09:19:15.567 INFO:teuthology.orchestra.run.smithi106.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-03-04T09:19:15.567 INFO:teuthology.orchestra.run.smithi106.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-03-04T09:19:15.567 INFO:teuthology.orchestra.run.smithi106.stdout: python3-toml-0.10.2-6.el9.noarch 2024-03-04T09:19:15.567 INFO:teuthology.orchestra.run.smithi106.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-03-04T09:19:15.568 INFO:teuthology.orchestra.run.smithi106.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-03-04T09:19:15.568 INFO:teuthology.orchestra.run.smithi106.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-03-04T09:19:15.568 INFO:teuthology.orchestra.run.smithi106.stdout: python3-webob-1.8.7-6.el9.noarch 2024-03-04T09:19:15.568 INFO:teuthology.orchestra.run.smithi106.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-03-04T09:19:15.568 INFO:teuthology.orchestra.run.smithi106.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-03-04T09:19:15.568 INFO:teuthology.orchestra.run.smithi106.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-03-04T09:19:15.568 INFO:teuthology.orchestra.run.smithi106.stdout: qatlib-23.11.0-1.el9.x86_64 2024-03-04T09:19:15.568 INFO:teuthology.orchestra.run.smithi106.stdout: qatlib-service-23.11.0-1.el9.x86_64 2024-03-04T09:19:15.568 INFO:teuthology.orchestra.run.smithi106.stdout: qatzip-libs-1.1.2-2.el9.x86_64 2024-03-04T09:19:15.568 INFO:teuthology.orchestra.run.smithi106.stdout: rbd-fuse-2:19.0.0-1739.g1bc250c7.el9.x86_64 2024-03-04T09:19:15.568 INFO:teuthology.orchestra.run.smithi106.stdout: rbd-mirror-2:19.0.0-1739.g1bc250c7.el9.x86_64 2024-03-04T09:19:15.568 INFO:teuthology.orchestra.run.smithi106.stdout: rbd-nbd-2:19.0.0-1739.g1bc250c7.el9.x86_64 2024-03-04T09:19:15.568 INFO:teuthology.orchestra.run.smithi106.stdout: re2-1:20211101-3.el9.x86_64 2024-03-04T09:19:15.568 INFO:teuthology.orchestra.run.smithi106.stdout: socat-1.7.4.1-5.el9.x86_64 2024-03-04T09:19:15.569 INFO:teuthology.orchestra.run.smithi106.stdout: thrift-0.15.0-2.el9.x86_64 2024-03-04T09:19:15.569 INFO:teuthology.orchestra.run.smithi106.stdout: uuid-1.6.2-55.el9.x86_64 2024-03-04T09:19:15.569 INFO:teuthology.orchestra.run.smithi106.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-03-04T09:19:15.569 INFO:teuthology.orchestra.run.smithi106.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-03-04T09:19:15.569 INFO:teuthology.orchestra.run.smithi106.stdout: zlib-devel-1.2.11-40.el9.x86_64 2024-03-04T09:19:15.569 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-03-04T09:19:15.569 INFO:teuthology.orchestra.run.smithi106.stdout:Complete! 2024-03-04T09:19:16.132 DEBUG:teuthology.parallel:result is None 2024-03-04T09:19:16.296 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : ceph-test-2:19.0.0-1739.g1bc250c7.el9.x86_64 154/202 2024-03-04T09:19:16.337 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : rbd-fuse-2:19.0.0-1739.g1bc250c7.el9.x86_64 155/202 2024-03-04T09:19:16.367 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : rbd-nbd-2:19.0.0-1739.g1bc250c7.el9.x86_64 156/202 2024-03-04T09:19:16.395 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 157/202 2024-03-04T09:19:16.580 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : cephfs-shell-2:19.0.0-1739.g1bc250c7.el9.x86_64 158/202 2024-03-04T09:19:16.697 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : postgresql-contrib-13.11-1.el9.x86_64 159/202 2024-03-04T09:19:16.742 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : ceph-fuse-2:19.0.0-1739.g1bc250c7.el9.x86_64 160/202 2024-03-04T09:19:16.799 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 161/202 2024-03-04T09:19:16.858 INFO:teuthology.orchestra.run.smithi052.stdout: Upgrading : xfsdump-3.1.12-4.el9.x86_64 162/202 2024-03-04T09:19:16.941 INFO:teuthology.orchestra.run.smithi052.stdout: Upgrading : util-linux-user-2.37.4-18.el9.x86_64 163/202 2024-03-04T09:19:16.977 INFO:teuthology.orchestra.run.smithi052.stdout: Upgrading : iproute-tc-6.7.0-2.el9.x86_64 164/202 2024-03-04T09:19:17.012 INFO:teuthology.orchestra.run.smithi052.stdout: Upgrading : elfutils-debuginfod-client-0.190-2.el9.x86_64 165/202 2024-03-04T09:19:17.043 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : libcephfs-devel-2:19.0.0-1739.g1bc250c7.el9.x86_ 166/202 2024-03-04T09:19:17.073 INFO:teuthology.orchestra.run.smithi052.stdout: Upgrading : NetworkManager-team-1:1.46.0-1.el9.x86_64 167/202 2024-03-04T09:19:17.144 INFO:teuthology.orchestra.run.smithi052.stdout: Upgrading : NetworkManager-tui-1:1.46.0-1.el9.x86_64 168/202 2024-03-04T09:19:17.205 INFO:teuthology.orchestra.run.smithi052.stdout: Upgrading : libacl-devel-2.3.1-4.el9.x86_64 169/202 2024-03-04T09:19:17.238 INFO:teuthology.orchestra.run.smithi052.stdout: Upgrading : acl-2.3.1-4.el9.x86_64 170/202 2024-03-04T09:19:17.884 INFO:teuthology.orchestra.run.smithi052.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 171/202 2024-03-04T09:19:17.919 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : postgresql-server-13.11-1.el9.x86_64 171/202 2024-03-04T09:19:17.937 INFO:teuthology.orchestra.run.smithi052.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 171/202 2024-03-04T09:19:18.799 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : cephfs-top-2:19.0.0-1739.g1bc250c7.el9.noarch 172/202 2024-03-04T09:19:18.863 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : openssl-devel-1:3.0.7-27.el9.x86_64 173/202 2024-03-04T09:19:18.896 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 174/202 2024-03-04T09:19:18.995 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : flex-2.6.4-9.el9.x86_64 175/202 2024-03-04T09:19:19.033 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : bison-3.7.4-5.el9.x86_64 176/202 2024-03-04T09:19:19.058 INFO:teuthology.orchestra.run.smithi052.stdout: Cleanup : NetworkManager-tui-1:1.44.0-2.el9.x86_64 177/202 2024-03-04T09:19:19.083 INFO:teuthology.orchestra.run.smithi052.stdout: Cleanup : elfutils-debuginfod-client-0.189-3.el9.x86_64 178/202 2024-03-04T09:19:19.110 INFO:teuthology.orchestra.run.smithi052.stdout: Cleanup : elfutils-libs-0.189-3.el9.x86_64 179/202 2024-03-04T09:19:19.110 INFO:teuthology.orchestra.run.smithi052.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 180/202 2024-03-04T09:19:19.196 INFO:teuthology.orchestra.run.smithi052.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 180/202 2024-03-04T09:19:19.224 INFO:teuthology.orchestra.run.smithi052.stdout: Cleanup : libacl-devel-2.3.1-3.el9.x86_64 181/202 2024-03-04T09:19:19.254 INFO:teuthology.orchestra.run.smithi052.stdout: Cleanup : openssl-1:3.0.7-24.el9.x86_64 182/202 2024-03-04T09:19:19.254 INFO:teuthology.orchestra.run.smithi052.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 183/202 2024-03-04T09:19:19.364 INFO:teuthology.orchestra.run.smithi052.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 183/202 2024-03-04T09:19:19.390 INFO:teuthology.orchestra.run.smithi052.stdout: Cleanup : xfsdump-3.1.12-3.el9.x86_64 184/202 2024-03-04T09:19:19.424 INFO:teuthology.orchestra.run.smithi052.stdout: Cleanup : xfsprogs-5.19.0-4.el9.x86_64 185/202 2024-03-04T09:19:19.457 INFO:teuthology.orchestra.run.smithi052.stdout: Cleanup : iproute-tc-6.2.0-5.el9.x86_64 186/202 2024-03-04T09:19:19.492 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: openssl-libs-1:3.0.7-24.el9.x86_64 202/202 2024-03-04T09:19:19.492 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : ceph-2:19.0.0-1739.g1bc250c7.el9.x86_64 1/202 2024-03-04T09:19:19.492 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : ceph-base-2:19.0.0-1739.g1bc250c7.el9.x86_64 2/202 2024-03-04T09:19:19.492 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : ceph-common-2:19.0.0-1739.g1bc250c7.el9.x86_64 3/202 2024-03-04T09:19:19.492 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : ceph-fuse-2:19.0.0-1739.g1bc250c7.el9.x86_64 4/202 2024-03-04T09:19:19.492 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-1739.g1bc25 5/202 2024-03-04T09:19:19.492 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : ceph-mds-2:19.0.0-1739.g1bc250c7.el9.x86_64 6/202 2024-03-04T09:19:19.493 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : ceph-mgr-2:19.0.0-1739.g1bc250c7.el9.x86_64 7/202 2024-03-04T09:19:19.493 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : ceph-mon-2:19.0.0-1739.g1bc250c7.el9.x86_64 8/202 2024-03-04T09:19:19.493 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : ceph-osd-2:19.0.0-1739.g1bc250c7.el9.x86_64 9/202 2024-03-04T09:19:19.493 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : ceph-radosgw-2:19.0.0-1739.g1bc250c7.el9.x86_64 10/202 2024-03-04T09:19:19.493 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : ceph-selinux-2:19.0.0-1739.g1bc250c7.el9.x86_64 11/202 2024-03-04T09:19:19.493 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : ceph-test-2:19.0.0-1739.g1bc250c7.el9.x86_64 12/202 2024-03-04T09:19:19.493 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : cephfs-mirror-2:19.0.0-1739.g1bc250c7.el9.x86_64 13/202 2024-03-04T09:19:19.493 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : cephfs-shell-2:19.0.0-1739.g1bc250c7.el9.x86_64 14/202 2024-03-04T09:19:19.493 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : libcephfs-devel-2:19.0.0-1739.g1bc250c7.el9.x86_ 15/202 2024-03-04T09:19:19.493 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : libcephfs2-2:19.0.0-1739.g1bc250c7.el9.x86_64 16/202 2024-03-04T09:19:19.493 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : libcephsqlite-2:19.0.0-1739.g1bc250c7.el9.x86_64 17/202 2024-03-04T09:19:19.493 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : librados-devel-2:19.0.0-1739.g1bc250c7.el9.x86_6 18/202 2024-03-04T09:19:19.493 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : libradosstriper1-2:19.0.0-1739.g1bc250c7.el9.x86 19/202 2024-03-04T09:19:19.493 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : librgw2-2:19.0.0-1739.g1bc250c7.el9.x86_64 20/202 2024-03-04T09:19:19.494 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-ceph-argparse-2:19.0.0-1739.g1bc250c7.el 21/202 2024-03-04T09:19:19.494 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-ceph-common-2:19.0.0-1739.g1bc250c7.el9. 22/202 2024-03-04T09:19:19.494 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-cephfs-2:19.0.0-1739.g1bc250c7.el9.x86_6 23/202 2024-03-04T09:19:19.494 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-rados-2:19.0.0-1739.g1bc250c7.el9.x86_64 24/202 2024-03-04T09:19:19.494 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-rbd-2:19.0.0-1739.g1bc250c7.el9.x86_64 25/202 2024-03-04T09:19:19.494 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-rgw-2:19.0.0-1739.g1bc250c7.el9.x86_64 26/202 2024-03-04T09:19:19.494 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : rbd-fuse-2:19.0.0-1739.g1bc250c7.el9.x86_64 27/202 2024-03-04T09:19:19.494 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : rbd-mirror-2:19.0.0-1739.g1bc250c7.el9.x86_64 28/202 2024-03-04T09:19:19.494 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : rbd-nbd-2:19.0.0-1739.g1bc250c7.el9.x86_64 29/202 2024-03-04T09:19:19.494 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-1739.g1bc250c7. 30/202 2024-03-04T09:19:19.494 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-1739.g1bc250c7.el9.noa 31/202 2024-03-04T09:19:19.494 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-1739.g1bc250c7.el9.n 32/202 2024-03-04T09:19:19.495 INFO:teuthology.orchestra.run.smithi052.stdout: Cleanup : acl-2.3.1-3.el9.x86_64 187/202 2024-03-04T09:19:19.511 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-1739.g1bc 33/202 2024-03-04T09:19:19.511 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-1739.g1bc250c7.el 34/202 2024-03-04T09:19:19.511 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : ceph-mgr-rook-2:19.0.0-1739.g1bc250c7.el9.noarch 35/202 2024-03-04T09:19:19.511 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-1739.g1bc250c7.e 36/202 2024-03-04T09:19:19.511 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : ceph-volume-2:19.0.0-1739.g1bc250c7.el9.noarch 37/202 2024-03-04T09:19:19.511 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : cephadm-2:19.0.0-1739.g1bc250c7.el9.noarch 38/202 2024-03-04T09:19:19.511 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : cephfs-top-2:19.0.0-1739.g1bc250c7.el9.noarch 39/202 2024-03-04T09:19:19.511 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : automake-1.16.2-8.el9.noarch 40/202 2024-03-04T09:19:19.511 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : bison-3.7.4-5.el9.x86_64 41/202 2024-03-04T09:19:19.511 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 42/202 2024-03-04T09:19:19.511 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 43/202 2024-03-04T09:19:19.511 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : flex-2.6.4-9.el9.x86_64 44/202 2024-03-04T09:19:19.512 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 45/202 2024-03-04T09:19:19.512 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 46/202 2024-03-04T09:19:19.512 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 47/202 2024-03-04T09:19:19.512 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 48/202 2024-03-04T09:19:19.512 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 49/202 2024-03-04T09:19:19.512 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 50/202 2024-03-04T09:19:19.512 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 51/202 2024-03-04T09:19:19.512 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 52/202 2024-03-04T09:19:19.512 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 53/202 2024-03-04T09:19:19.512 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 54/202 2024-03-04T09:19:19.512 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 55/202 2024-03-04T09:19:19.512 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : lua-5.4.4-4.el9.x86_64 56/202 2024-03-04T09:19:19.512 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 57/202 2024-03-04T09:19:19.512 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 58/202 2024-03-04T09:19:19.513 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : openssl-devel-1:3.0.7-27.el9.x86_64 59/202 2024-03-04T09:19:19.513 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : postgresql-13.11-1.el9.x86_64 60/202 2024-03-04T09:19:19.513 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : postgresql-contrib-13.11-1.el9.x86_64 61/202 2024-03-04T09:19:19.513 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : postgresql-private-libs-13.11-1.el9.x86_64 62/202 2024-03-04T09:19:19.513 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : postgresql-server-13.11-1.el9.x86_64 63/202 2024-03-04T09:19:19.513 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 64/202 2024-03-04T09:19:19.513 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 65/202 2024-03-04T09:19:19.513 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 66/202 2024-03-04T09:19:19.513 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 67/202 2024-03-04T09:19:19.513 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 68/202 2024-03-04T09:19:19.513 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 69/202 2024-03-04T09:19:19.513 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 70/202 2024-03-04T09:19:19.513 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 71/202 2024-03-04T09:19:19.513 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 72/202 2024-03-04T09:19:19.514 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 73/202 2024-03-04T09:19:19.514 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 74/202 2024-03-04T09:19:19.514 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 75/202 2024-03-04T09:19:19.514 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 76/202 2024-03-04T09:19:19.514 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 77/202 2024-03-04T09:19:19.514 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 78/202 2024-03-04T09:19:19.514 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 79/202 2024-03-04T09:19:19.514 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 80/202 2024-03-04T09:19:19.514 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 81/202 2024-03-04T09:19:19.514 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 82/202 2024-03-04T09:19:19.514 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 83/202 2024-03-04T09:19:19.514 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 84/202 2024-03-04T09:19:19.514 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : qatlib-23.11.0-1.el9.x86_64 85/202 2024-03-04T09:19:19.514 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : qatlib-service-23.11.0-1.el9.x86_64 86/202 2024-03-04T09:19:19.515 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : qatzip-libs-1.1.2-2.el9.x86_64 87/202 2024-03-04T09:19:19.515 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 88/202 2024-03-04T09:19:19.515 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 89/202 2024-03-04T09:19:19.515 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 90/202 2024-03-04T09:19:19.515 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 91/202 2024-03-04T09:19:19.515 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : zlib-devel-1.2.11-40.el9.x86_64 92/202 2024-03-04T09:19:19.515 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 93/202 2024-03-04T09:19:19.515 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : jq-1.6-16.el9.x86_64 94/202 2024-03-04T09:19:19.515 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 95/202 2024-03-04T09:19:19.515 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 96/202 2024-03-04T09:19:19.515 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 97/202 2024-03-04T09:19:19.515 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 98/202 2024-03-04T09:19:19.515 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 99/202 2024-03-04T09:19:19.515 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-ply-3.11-14.el9.noarch 100/202 2024-03-04T09:19:19.516 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 101/202 2024-03-04T09:19:19.516 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 102/202 2024-03-04T09:19:19.516 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 103/202 2024-03-04T09:19:19.516 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 104/202 2024-03-04T09:19:19.516 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 105/202 2024-03-04T09:19:19.516 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 106/202 2024-03-04T09:19:19.516 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 107/202 2024-03-04T09:19:19.516 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 108/202 2024-03-04T09:19:19.516 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 109/202 2024-03-04T09:19:19.516 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 110/202 2024-03-04T09:19:19.516 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 111/202 2024-03-04T09:19:19.516 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 112/202 2024-03-04T09:19:19.516 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 113/202 2024-03-04T09:19:19.516 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 114/202 2024-03-04T09:19:19.516 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 115/202 2024-03-04T09:19:19.517 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-asyncssh-2.13.2-4.el9.noarch 116/202 2024-03-04T09:19:19.517 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 117/202 2024-03-04T09:19:19.517 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 118/202 2024-03-04T09:19:19.517 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 119/202 2024-03-04T09:19:19.517 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 120/202 2024-03-04T09:19:19.517 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 121/202 2024-03-04T09:19:19.517 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 122/202 2024-03-04T09:19:19.517 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 123/202 2024-03-04T09:19:19.517 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-google-auth-1:2.28.0-1.el9.noarch 124/202 2024-03-04T09:19:19.517 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 125/202 2024-03-04T09:19:19.517 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 126/202 2024-03-04T09:19:19.517 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 127/202 2024-03-04T09:19:19.517 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 128/202 2024-03-04T09:19:19.517 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 129/202 2024-03-04T09:19:19.517 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 130/202 2024-03-04T09:19:19.518 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 131/202 2024-03-04T09:19:19.518 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 132/202 2024-03-04T09:19:19.518 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 133/202 2024-03-04T09:19:19.518 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 134/202 2024-03-04T09:19:19.518 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 135/202 2024-03-04T09:19:19.518 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 136/202 2024-03-04T09:19:19.518 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 137/202 2024-03-04T09:19:19.518 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 138/202 2024-03-04T09:19:19.518 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 139/202 2024-03-04T09:19:19.518 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 140/202 2024-03-04T09:19:19.518 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 141/202 2024-03-04T09:19:19.518 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 142/202 2024-03-04T09:19:19.518 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 143/202 2024-03-04T09:19:19.518 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 144/202 2024-03-04T09:19:19.519 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 145/202 2024-03-04T09:19:19.519 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 146/202 2024-03-04T09:19:19.519 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 147/202 2024-03-04T09:19:19.519 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 148/202 2024-03-04T09:19:19.519 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : re2-1:20211101-3.el9.x86_64 149/202 2024-03-04T09:19:19.519 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 150/202 2024-03-04T09:19:19.519 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : librados2-2:19.0.0-1739.g1bc250c7.el9.x86_64 151/202 2024-03-04T09:19:19.519 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 152/202 2024-03-04T09:19:19.519 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : librbd1-2:19.0.0-1739.g1bc250c7.el9.x86_64 153/202 2024-03-04T09:19:19.519 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 154/202 2024-03-04T09:19:19.519 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : libacl-devel-2.3.1-4.el9.x86_64 155/202 2024-03-04T09:19:19.519 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : libacl-devel-2.3.1-3.el9.x86_64 156/202 2024-03-04T09:19:19.519 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : NetworkManager-1:1.46.0-1.el9.x86_64 157/202 2024-03-04T09:19:19.519 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : NetworkManager-1:1.44.0-2.el9.x86_64 158/202 2024-03-04T09:19:19.519 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : NetworkManager-libnm-1:1.46.0-1.el9.x86_64 159/202 2024-03-04T09:19:19.520 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 160/202 2024-03-04T09:19:19.520 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : NetworkManager-team-1:1.46.0-1.el9.x86_64 161/202 2024-03-04T09:19:19.520 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : NetworkManager-team-1:1.44.0-2.el9.x86_64 162/202 2024-03-04T09:19:19.520 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : NetworkManager-tui-1:1.46.0-1.el9.x86_64 163/202 2024-03-04T09:19:19.520 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : NetworkManager-tui-1:1.44.0-2.el9.x86_64 164/202 2024-03-04T09:19:19.520 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : acl-2.3.1-4.el9.x86_64 165/202 2024-03-04T09:19:19.520 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : acl-2.3.1-3.el9.x86_64 166/202 2024-03-04T09:19:19.520 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 167/202 2024-03-04T09:19:19.520 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : elfutils-debuginfod-client-0.189-3.el9.x86_64 168/202 2024-03-04T09:19:19.520 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 169/202 2024-03-04T09:19:19.520 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : elfutils-libelf-0.189-3.el9.x86_64 170/202 2024-03-04T09:19:19.520 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 171/202 2024-03-04T09:19:19.520 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : elfutils-libs-0.189-3.el9.x86_64 172/202 2024-03-04T09:19:19.521 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : iproute-6.7.0-2.el9.x86_64 173/202 2024-03-04T09:19:19.521 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : iproute-6.2.0-5.el9.x86_64 174/202 2024-03-04T09:19:19.521 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : iproute-tc-6.7.0-2.el9.x86_64 175/202 2024-03-04T09:19:19.521 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : iproute-tc-6.2.0-5.el9.x86_64 176/202 2024-03-04T09:19:19.521 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : libacl-2.3.1-4.el9.x86_64 177/202 2024-03-04T09:19:19.521 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : libacl-2.3.1-3.el9.x86_64 178/202 2024-03-04T09:19:19.521 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : libblkid-2.37.4-18.el9.x86_64 179/202 2024-03-04T09:19:19.521 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : libblkid-2.37.4-15.el9.x86_64 180/202 2024-03-04T09:19:19.521 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : libfdisk-2.37.4-18.el9.x86_64 181/202 2024-03-04T09:19:19.521 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : libfdisk-2.37.4-15.el9.x86_64 182/202 2024-03-04T09:19:19.521 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : libmount-2.37.4-18.el9.x86_64 183/202 2024-03-04T09:19:19.522 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : libmount-2.37.4-15.el9.x86_64 184/202 2024-03-04T09:19:19.522 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : libsmartcols-2.37.4-18.el9.x86_64 185/202 2024-03-04T09:19:19.522 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : libsmartcols-2.37.4-15.el9.x86_64 186/202 2024-03-04T09:19:19.522 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : libuuid-2.37.4-18.el9.x86_64 187/202 2024-03-04T09:19:19.522 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : libuuid-2.37.4-15.el9.x86_64 188/202 2024-03-04T09:19:19.522 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 189/202 2024-03-04T09:19:19.522 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : openssl-1:3.0.7-24.el9.x86_64 190/202 2024-03-04T09:19:19.522 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 191/202 2024-03-04T09:19:19.522 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : openssl-libs-1:3.0.7-24.el9.x86_64 192/202 2024-03-04T09:19:19.522 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : util-linux-2.37.4-18.el9.x86_64 193/202 2024-03-04T09:19:19.522 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : util-linux-2.37.4-15.el9.x86_64 194/202 2024-03-04T09:19:19.522 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : util-linux-core-2.37.4-18.el9.x86_64 195/202 2024-03-04T09:19:19.522 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : util-linux-core-2.37.4-15.el9.x86_64 196/202 2024-03-04T09:19:19.522 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : util-linux-user-2.37.4-18.el9.x86_64 197/202 2024-03-04T09:19:19.523 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : util-linux-user-2.37.4-15.el9.x86_64 198/202 2024-03-04T09:19:19.523 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : xfsdump-3.1.12-4.el9.x86_64 199/202 2024-03-04T09:19:19.523 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : xfsdump-3.1.12-3.el9.x86_64 200/202 2024-03-04T09:19:19.523 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : xfsprogs-6.3.0-1.el9.x86_64 201/202 2024-03-04T09:19:19.525 INFO:teuthology.orchestra.run.smithi052.stdout: Cleanup : iproute-6.2.0-5.el9.x86_64 188/202 2024-03-04T09:19:19.558 INFO:teuthology.orchestra.run.smithi052.stdout: Cleanup : util-linux-user-2.37.4-15.el9.x86_64 189/202 2024-03-04T09:19:19.597 INFO:teuthology.orchestra.run.smithi052.stdout: Cleanup : util-linux-2.37.4-15.el9.x86_64 190/202 2024-03-04T09:19:19.638 INFO:teuthology.orchestra.run.smithi052.stdout: Cleanup : util-linux-core-2.37.4-15.el9.x86_64 191/202 2024-03-04T09:19:19.663 INFO:teuthology.orchestra.run.smithi052.stdout: Cleanup : libfdisk-2.37.4-15.el9.x86_64 192/202 2024-03-04T09:19:19.689 INFO:teuthology.orchestra.run.smithi052.stdout: Cleanup : libmount-2.37.4-15.el9.x86_64 193/202 2024-03-04T09:19:19.705 INFO:teuthology.orchestra.run.smithi052.stdout: Cleanup : libblkid-2.37.4-15.el9.x86_64 194/202 2024-03-04T09:19:19.730 INFO:teuthology.orchestra.run.smithi052.stdout: Cleanup : NetworkManager-team-1:1.44.0-2.el9.x86_64 195/202 2024-03-04T09:19:19.737 INFO:teuthology.orchestra.run.smithi052.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 196/202 2024-03-04T09:19:19.737 INFO:teuthology.orchestra.run.smithi052.stdout: Cleanup : NetworkManager-1:1.44.0-2.el9.x86_64 196/202 2024-03-04T09:19:19.917 INFO:teuthology.orchestra.run.smithi052.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 196/202 2024-03-04T09:19:19.917 INFO:teuthology.orchestra.run.smithi052.stdout: Cleanup : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 197/202 2024-03-04T09:19:19.999 INFO:teuthology.orchestra.run.smithi052.stdout: Running scriptlet: NetworkManager-libnm-1:1.44.0-2.el9.x86_64 197/202 2024-03-04T09:19:20.031 INFO:teuthology.orchestra.run.smithi052.stdout: Cleanup : libuuid-2.37.4-15.el9.x86_64 198/202 2024-03-04T09:19:20.058 INFO:teuthology.orchestra.run.smithi052.stdout: Cleanup : libsmartcols-2.37.4-15.el9.x86_64 199/202 2024-03-04T09:19:20.083 INFO:teuthology.orchestra.run.smithi052.stdout: Cleanup : elfutils-libelf-0.189-3.el9.x86_64 200/202 2024-03-04T09:19:20.109 INFO:teuthology.orchestra.run.smithi052.stdout: Cleanup : libacl-2.3.1-3.el9.x86_64 201/202 2024-03-04T09:19:20.177 INFO:teuthology.orchestra.run.smithi052.stdout: Cleanup : openssl-libs-1:3.0.7-24.el9.x86_64 202/202 2024-03-04T09:19:22.492 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : xfsprogs-5.19.0-4.el9.x86_64 202/202 2024-03-04T09:19:22.493 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-03-04T09:19:22.493 INFO:teuthology.orchestra.run.smithi191.stdout:Upgraded: 2024-03-04T09:19:22.493 INFO:teuthology.orchestra.run.smithi191.stdout: NetworkManager-1:1.46.0-1.el9.x86_64 2024-03-04T09:19:22.493 INFO:teuthology.orchestra.run.smithi191.stdout: NetworkManager-libnm-1:1.46.0-1.el9.x86_64 2024-03-04T09:19:22.493 INFO:teuthology.orchestra.run.smithi191.stdout: NetworkManager-team-1:1.46.0-1.el9.x86_64 2024-03-04T09:19:22.493 INFO:teuthology.orchestra.run.smithi191.stdout: NetworkManager-tui-1:1.46.0-1.el9.x86_64 2024-03-04T09:19:22.493 INFO:teuthology.orchestra.run.smithi191.stdout: acl-2.3.1-4.el9.x86_64 2024-03-04T09:19:22.493 INFO:teuthology.orchestra.run.smithi191.stdout: elfutils-debuginfod-client-0.190-2.el9.x86_64 2024-03-04T09:19:22.493 INFO:teuthology.orchestra.run.smithi191.stdout: elfutils-libelf-0.190-2.el9.x86_64 2024-03-04T09:19:22.493 INFO:teuthology.orchestra.run.smithi191.stdout: elfutils-libs-0.190-2.el9.x86_64 2024-03-04T09:19:22.493 INFO:teuthology.orchestra.run.smithi191.stdout: iproute-6.7.0-2.el9.x86_64 2024-03-04T09:19:22.493 INFO:teuthology.orchestra.run.smithi191.stdout: iproute-tc-6.7.0-2.el9.x86_64 2024-03-04T09:19:22.494 INFO:teuthology.orchestra.run.smithi191.stdout: libacl-2.3.1-4.el9.x86_64 2024-03-04T09:19:22.494 INFO:teuthology.orchestra.run.smithi191.stdout: libacl-devel-2.3.1-4.el9.x86_64 2024-03-04T09:19:22.494 INFO:teuthology.orchestra.run.smithi191.stdout: libblkid-2.37.4-18.el9.x86_64 2024-03-04T09:19:22.494 INFO:teuthology.orchestra.run.smithi191.stdout: libfdisk-2.37.4-18.el9.x86_64 2024-03-04T09:19:22.494 INFO:teuthology.orchestra.run.smithi191.stdout: libmount-2.37.4-18.el9.x86_64 2024-03-04T09:19:22.494 INFO:teuthology.orchestra.run.smithi191.stdout: librados2-2:19.0.0-1739.g1bc250c7.el9.x86_64 2024-03-04T09:19:22.494 INFO:teuthology.orchestra.run.smithi191.stdout: librbd1-2:19.0.0-1739.g1bc250c7.el9.x86_64 2024-03-04T09:19:22.494 INFO:teuthology.orchestra.run.smithi191.stdout: libsmartcols-2.37.4-18.el9.x86_64 2024-03-04T09:19:22.494 INFO:teuthology.orchestra.run.smithi191.stdout: libuuid-2.37.4-18.el9.x86_64 2024-03-04T09:19:22.494 INFO:teuthology.orchestra.run.smithi191.stdout: openssl-1:3.0.7-27.el9.x86_64 2024-03-04T09:19:22.494 INFO:teuthology.orchestra.run.smithi191.stdout: openssl-libs-1:3.0.7-27.el9.x86_64 2024-03-04T09:19:22.495 INFO:teuthology.orchestra.run.smithi191.stdout: util-linux-2.37.4-18.el9.x86_64 2024-03-04T09:19:22.495 INFO:teuthology.orchestra.run.smithi191.stdout: util-linux-core-2.37.4-18.el9.x86_64 2024-03-04T09:19:22.495 INFO:teuthology.orchestra.run.smithi191.stdout: util-linux-user-2.37.4-18.el9.x86_64 2024-03-04T09:19:22.495 INFO:teuthology.orchestra.run.smithi191.stdout: xfsdump-3.1.12-4.el9.x86_64 2024-03-04T09:19:22.495 INFO:teuthology.orchestra.run.smithi191.stdout: xfsprogs-6.3.0-1.el9.x86_64 2024-03-04T09:19:22.495 INFO:teuthology.orchestra.run.smithi191.stdout:Installed: 2024-03-04T09:19:22.495 INFO:teuthology.orchestra.run.smithi191.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-03-04T09:19:22.495 INFO:teuthology.orchestra.run.smithi191.stdout: automake-1.16.2-8.el9.noarch 2024-03-04T09:19:22.495 INFO:teuthology.orchestra.run.smithi191.stdout: bison-3.7.4-5.el9.x86_64 2024-03-04T09:19:22.495 INFO:teuthology.orchestra.run.smithi191.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-03-04T09:19:22.495 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-2:19.0.0-1739.g1bc250c7.el9.x86_64 2024-03-04T09:19:22.495 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-base-2:19.0.0-1739.g1bc250c7.el9.x86_64 2024-03-04T09:19:22.495 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-common-2:19.0.0-1739.g1bc250c7.el9.x86_64 2024-03-04T09:19:22.495 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-fuse-2:19.0.0-1739.g1bc250c7.el9.x86_64 2024-03-04T09:19:22.496 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-grafana-dashboards-2:19.0.0-1739.g1bc250c7.el9.noarch 2024-03-04T09:19:22.496 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-immutable-object-cache-2:19.0.0-1739.g1bc250c7.el9.x86_64 2024-03-04T09:19:22.496 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-mds-2:19.0.0-1739.g1bc250c7.el9.x86_64 2024-03-04T09:19:22.496 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-mgr-2:19.0.0-1739.g1bc250c7.el9.x86_64 2024-03-04T09:19:22.496 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-mgr-cephadm-2:19.0.0-1739.g1bc250c7.el9.noarch 2024-03-04T09:19:22.496 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-mgr-dashboard-2:19.0.0-1739.g1bc250c7.el9.noarch 2024-03-04T09:19:22.496 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-mgr-diskprediction-local-2:19.0.0-1739.g1bc250c7.el9.noarch 2024-03-04T09:19:22.496 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-mgr-modules-core-2:19.0.0-1739.g1bc250c7.el9.noarch 2024-03-04T09:19:22.496 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-mgr-rook-2:19.0.0-1739.g1bc250c7.el9.noarch 2024-03-04T09:19:22.496 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-mon-2:19.0.0-1739.g1bc250c7.el9.x86_64 2024-03-04T09:19:22.496 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-osd-2:19.0.0-1739.g1bc250c7.el9.x86_64 2024-03-04T09:19:22.496 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-prometheus-alerts-2:19.0.0-1739.g1bc250c7.el9.noarch 2024-03-04T09:19:22.497 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-radosgw-2:19.0.0-1739.g1bc250c7.el9.x86_64 2024-03-04T09:19:22.497 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-selinux-2:19.0.0-1739.g1bc250c7.el9.x86_64 2024-03-04T09:19:22.497 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-test-2:19.0.0-1739.g1bc250c7.el9.x86_64 2024-03-04T09:19:22.497 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-volume-2:19.0.0-1739.g1bc250c7.el9.noarch 2024-03-04T09:19:22.497 INFO:teuthology.orchestra.run.smithi191.stdout: cephadm-2:19.0.0-1739.g1bc250c7.el9.noarch 2024-03-04T09:19:22.497 INFO:teuthology.orchestra.run.smithi191.stdout: cephfs-mirror-2:19.0.0-1739.g1bc250c7.el9.x86_64 2024-03-04T09:19:22.497 INFO:teuthology.orchestra.run.smithi191.stdout: cephfs-shell-2:19.0.0-1739.g1bc250c7.el9.x86_64 2024-03-04T09:19:22.497 INFO:teuthology.orchestra.run.smithi191.stdout: cephfs-top-2:19.0.0-1739.g1bc250c7.el9.noarch 2024-03-04T09:19:22.497 INFO:teuthology.orchestra.run.smithi191.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-03-04T09:19:22.497 INFO:teuthology.orchestra.run.smithi191.stdout: flex-2.6.4-9.el9.x86_64 2024-03-04T09:19:22.497 INFO:teuthology.orchestra.run.smithi191.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-03-04T09:19:22.497 INFO:teuthology.orchestra.run.smithi191.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-03-04T09:19:22.497 INFO:teuthology.orchestra.run.smithi191.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-03-04T09:19:22.497 INFO:teuthology.orchestra.run.smithi191.stdout: fuse-2.9.9-15.el9.x86_64 2024-03-04T09:19:22.498 INFO:teuthology.orchestra.run.smithi191.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-03-04T09:19:22.498 INFO:teuthology.orchestra.run.smithi191.stdout: grpc-data-1.46.7-10.el9.noarch 2024-03-04T09:19:22.498 INFO:teuthology.orchestra.run.smithi191.stdout: jq-1.6-16.el9.x86_64 2024-03-04T09:19:22.498 INFO:teuthology.orchestra.run.smithi191.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-03-04T09:19:22.498 INFO:teuthology.orchestra.run.smithi191.stdout: libarrow-9.0.0-10.el9.x86_64 2024-03-04T09:19:22.498 INFO:teuthology.orchestra.run.smithi191.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-03-04T09:19:22.498 INFO:teuthology.orchestra.run.smithi191.stdout: libcephfs-devel-2:19.0.0-1739.g1bc250c7.el9.x86_64 2024-03-04T09:19:22.498 INFO:teuthology.orchestra.run.smithi191.stdout: libcephfs2-2:19.0.0-1739.g1bc250c7.el9.x86_64 2024-03-04T09:19:22.498 INFO:teuthology.orchestra.run.smithi191.stdout: libcephsqlite-2:19.0.0-1739.g1bc250c7.el9.x86_64 2024-03-04T09:19:22.498 INFO:teuthology.orchestra.run.smithi191.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-03-04T09:19:22.498 INFO:teuthology.orchestra.run.smithi191.stdout: liboath-2.6.7-2.el9.x86_64 2024-03-04T09:19:22.498 INFO:teuthology.orchestra.run.smithi191.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-03-04T09:19:22.498 INFO:teuthology.orchestra.run.smithi191.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-03-04T09:19:22.498 INFO:teuthology.orchestra.run.smithi191.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-03-04T09:19:22.499 INFO:teuthology.orchestra.run.smithi191.stdout: librados-devel-2:19.0.0-1739.g1bc250c7.el9.x86_64 2024-03-04T09:19:22.499 INFO:teuthology.orchestra.run.smithi191.stdout: libradosstriper1-2:19.0.0-1739.g1bc250c7.el9.x86_64 2024-03-04T09:19:22.499 INFO:teuthology.orchestra.run.smithi191.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-03-04T09:19:22.499 INFO:teuthology.orchestra.run.smithi191.stdout: librgw2-2:19.0.0-1739.g1bc250c7.el9.x86_64 2024-03-04T09:19:22.499 INFO:teuthology.orchestra.run.smithi191.stdout: libtool-2.4.6-45.el9.x86_64 2024-03-04T09:19:22.499 INFO:teuthology.orchestra.run.smithi191.stdout: libunwind-1.6.2-1.el9.x86_64 2024-03-04T09:19:22.499 INFO:teuthology.orchestra.run.smithi191.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-03-04T09:19:22.499 INFO:teuthology.orchestra.run.smithi191.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-03-04T09:19:22.499 INFO:teuthology.orchestra.run.smithi191.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-03-04T09:19:22.499 INFO:teuthology.orchestra.run.smithi191.stdout: lua-5.4.4-4.el9.x86_64 2024-03-04T09:19:22.499 INFO:teuthology.orchestra.run.smithi191.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-03-04T09:19:22.499 INFO:teuthology.orchestra.run.smithi191.stdout: luarocks-3.9.2-1.el9.noarch 2024-03-04T09:19:22.500 INFO:teuthology.orchestra.run.smithi191.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2024-03-04T09:19:22.500 INFO:teuthology.orchestra.run.smithi191.stdout: openblas-0.3.21-2.el9.x86_64 2024-03-04T09:19:22.500 INFO:teuthology.orchestra.run.smithi191.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-03-04T09:19:22.500 INFO:teuthology.orchestra.run.smithi191.stdout: openssl-devel-1:3.0.7-27.el9.x86_64 2024-03-04T09:19:22.501 INFO:teuthology.orchestra.run.smithi191.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-03-04T09:19:22.501 INFO:teuthology.orchestra.run.smithi191.stdout: postgresql-13.11-1.el9.x86_64 2024-03-04T09:19:22.501 INFO:teuthology.orchestra.run.smithi191.stdout: postgresql-contrib-13.11-1.el9.x86_64 2024-03-04T09:19:22.501 INFO:teuthology.orchestra.run.smithi191.stdout: postgresql-private-libs-13.11-1.el9.x86_64 2024-03-04T09:19:22.501 INFO:teuthology.orchestra.run.smithi191.stdout: postgresql-server-13.11-1.el9.x86_64 2024-03-04T09:19:22.501 INFO:teuthology.orchestra.run.smithi191.stdout: protobuf-3.14.0-13.el9.x86_64 2024-03-04T09:19:22.501 INFO:teuthology.orchestra.run.smithi191.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-03-04T09:19:22.501 INFO:teuthology.orchestra.run.smithi191.stdout: python3-asyncssh-2.13.2-4.el9.noarch 2024-03-04T09:19:22.501 INFO:teuthology.orchestra.run.smithi191.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-03-04T09:19:22.501 INFO:teuthology.orchestra.run.smithi191.stdout: python3-babel-2.9.1-2.el9.noarch 2024-03-04T09:19:22.501 INFO:teuthology.orchestra.run.smithi191.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-03-04T09:19:22.501 INFO:teuthology.orchestra.run.smithi191.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-03-04T09:19:22.501 INFO:teuthology.orchestra.run.smithi191.stdout: python3-ceph-argparse-2:19.0.0-1739.g1bc250c7.el9.x86_64 2024-03-04T09:19:22.502 INFO:teuthology.orchestra.run.smithi191.stdout: python3-ceph-common-2:19.0.0-1739.g1bc250c7.el9.x86_64 2024-03-04T09:19:22.502 INFO:teuthology.orchestra.run.smithi191.stdout: python3-cephfs-2:19.0.0-1739.g1bc250c7.el9.x86_64 2024-03-04T09:19:22.502 INFO:teuthology.orchestra.run.smithi191.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-03-04T09:19:22.502 INFO:teuthology.orchestra.run.smithi191.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-03-04T09:19:22.502 INFO:teuthology.orchestra.run.smithi191.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-03-04T09:19:22.502 INFO:teuthology.orchestra.run.smithi191.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-03-04T09:19:22.502 INFO:teuthology.orchestra.run.smithi191.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-03-04T09:19:22.503 INFO:teuthology.orchestra.run.smithi191.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-03-04T09:19:22.503 INFO:teuthology.orchestra.run.smithi191.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-03-04T09:19:22.503 INFO:teuthology.orchestra.run.smithi191.stdout: python3-devel-3.9.17-2.el9.x86_64 2024-03-04T09:19:22.503 INFO:teuthology.orchestra.run.smithi191.stdout: python3-google-auth-1:2.28.0-1.el9.noarch 2024-03-04T09:19:22.503 INFO:teuthology.orchestra.run.smithi191.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-03-04T09:19:22.504 INFO:teuthology.orchestra.run.smithi191.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-03-04T09:19:22.504 INFO:teuthology.orchestra.run.smithi191.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-03-04T09:19:22.504 INFO:teuthology.orchestra.run.smithi191.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-03-04T09:19:22.504 INFO:teuthology.orchestra.run.smithi191.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-03-04T09:19:22.504 INFO:teuthology.orchestra.run.smithi191.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-03-04T09:19:22.504 INFO:teuthology.orchestra.run.smithi191.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-03-04T09:19:22.504 INFO:teuthology.orchestra.run.smithi191.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-03-04T09:19:22.504 INFO:teuthology.orchestra.run.smithi191.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-03-04T09:19:22.505 INFO:teuthology.orchestra.run.smithi191.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-03-04T09:19:22.505 INFO:teuthology.orchestra.run.smithi191.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-03-04T09:19:22.505 INFO:teuthology.orchestra.run.smithi191.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-03-04T09:19:22.505 INFO:teuthology.orchestra.run.smithi191.stdout: python3-mako-1.1.4-6.el9.noarch 2024-03-04T09:19:22.505 INFO:teuthology.orchestra.run.smithi191.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-03-04T09:19:22.505 INFO:teuthology.orchestra.run.smithi191.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-03-04T09:19:22.505 INFO:teuthology.orchestra.run.smithi191.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-03-04T09:19:22.505 INFO:teuthology.orchestra.run.smithi191.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-03-04T09:19:22.505 INFO:teuthology.orchestra.run.smithi191.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-03-04T09:19:22.505 INFO:teuthology.orchestra.run.smithi191.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-03-04T09:19:22.505 INFO:teuthology.orchestra.run.smithi191.stdout: python3-packaging-20.9-5.el9.noarch 2024-03-04T09:19:22.505 INFO:teuthology.orchestra.run.smithi191.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-03-04T09:19:22.505 INFO:teuthology.orchestra.run.smithi191.stdout: python3-ply-3.11-14.el9.noarch 2024-03-04T09:19:22.505 INFO:teuthology.orchestra.run.smithi191.stdout: python3-portend-3.1.0-2.el9.noarch 2024-03-04T09:19:22.506 INFO:teuthology.orchestra.run.smithi191.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-03-04T09:19:22.506 INFO:teuthology.orchestra.run.smithi191.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-03-04T09:19:22.506 INFO:teuthology.orchestra.run.smithi191.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-03-04T09:19:22.506 INFO:teuthology.orchestra.run.smithi191.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-03-04T09:19:22.506 INFO:teuthology.orchestra.run.smithi191.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-03-04T09:19:22.506 INFO:teuthology.orchestra.run.smithi191.stdout: python3-pycparser-2.20-6.el9.noarch 2024-03-04T09:19:22.506 INFO:teuthology.orchestra.run.smithi191.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-03-04T09:19:22.506 INFO:teuthology.orchestra.run.smithi191.stdout: python3-pytz-2021.1-5.el9.noarch 2024-03-04T09:19:22.506 INFO:teuthology.orchestra.run.smithi191.stdout: python3-rados-2:19.0.0-1739.g1bc250c7.el9.x86_64 2024-03-04T09:19:22.506 INFO:teuthology.orchestra.run.smithi191.stdout: python3-rbd-2:19.0.0-1739.g1bc250c7.el9.x86_64 2024-03-04T09:19:22.506 INFO:teuthology.orchestra.run.smithi191.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-03-04T09:19:22.506 INFO:teuthology.orchestra.run.smithi191.stdout: python3-requests-2.25.1-8.el9.noarch 2024-03-04T09:19:22.507 INFO:teuthology.orchestra.run.smithi191.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-03-04T09:19:22.507 INFO:teuthology.orchestra.run.smithi191.stdout: python3-rgw-2:19.0.0-1739.g1bc250c7.el9.x86_64 2024-03-04T09:19:22.507 INFO:teuthology.orchestra.run.smithi191.stdout: python3-routes-2.5.1-5.el9.noarch 2024-03-04T09:19:22.507 INFO:teuthology.orchestra.run.smithi191.stdout: python3-rsa-4.9-2.el9.noarch 2024-03-04T09:19:22.507 INFO:teuthology.orchestra.run.smithi191.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-03-04T09:19:22.507 INFO:teuthology.orchestra.run.smithi191.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-03-04T09:19:22.508 INFO:teuthology.orchestra.run.smithi191.stdout: python3-toml-0.10.2-6.el9.noarch 2024-03-04T09:19:22.508 INFO:teuthology.orchestra.run.smithi191.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-03-04T09:19:22.508 INFO:teuthology.orchestra.run.smithi191.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-03-04T09:19:22.508 INFO:teuthology.orchestra.run.smithi191.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-03-04T09:19:22.508 INFO:teuthology.orchestra.run.smithi191.stdout: python3-webob-1.8.7-6.el9.noarch 2024-03-04T09:19:22.508 INFO:teuthology.orchestra.run.smithi191.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-03-04T09:19:22.509 INFO:teuthology.orchestra.run.smithi191.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-03-04T09:19:22.509 INFO:teuthology.orchestra.run.smithi191.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-03-04T09:19:22.509 INFO:teuthology.orchestra.run.smithi191.stdout: qatlib-23.11.0-1.el9.x86_64 2024-03-04T09:19:22.509 INFO:teuthology.orchestra.run.smithi191.stdout: qatlib-service-23.11.0-1.el9.x86_64 2024-03-04T09:19:22.509 INFO:teuthology.orchestra.run.smithi191.stdout: qatzip-libs-1.1.2-2.el9.x86_64 2024-03-04T09:19:22.509 INFO:teuthology.orchestra.run.smithi191.stdout: rbd-fuse-2:19.0.0-1739.g1bc250c7.el9.x86_64 2024-03-04T09:19:22.510 INFO:teuthology.orchestra.run.smithi191.stdout: rbd-mirror-2:19.0.0-1739.g1bc250c7.el9.x86_64 2024-03-04T09:19:22.510 INFO:teuthology.orchestra.run.smithi191.stdout: rbd-nbd-2:19.0.0-1739.g1bc250c7.el9.x86_64 2024-03-04T09:19:22.510 INFO:teuthology.orchestra.run.smithi191.stdout: re2-1:20211101-3.el9.x86_64 2024-03-04T09:19:22.510 INFO:teuthology.orchestra.run.smithi191.stdout: socat-1.7.4.1-5.el9.x86_64 2024-03-04T09:19:22.510 INFO:teuthology.orchestra.run.smithi191.stdout: thrift-0.15.0-2.el9.x86_64 2024-03-04T09:19:22.511 INFO:teuthology.orchestra.run.smithi191.stdout: uuid-1.6.2-55.el9.x86_64 2024-03-04T09:19:22.511 INFO:teuthology.orchestra.run.smithi191.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-03-04T09:19:22.511 INFO:teuthology.orchestra.run.smithi191.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-03-04T09:19:22.512 INFO:teuthology.orchestra.run.smithi191.stdout: zlib-devel-1.2.11-40.el9.x86_64 2024-03-04T09:19:22.512 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-03-04T09:19:22.512 INFO:teuthology.orchestra.run.smithi191.stdout:Complete! 2024-03-04T09:19:23.115 DEBUG:teuthology.parallel:result is None 2024-03-04T09:19:24.781 INFO:teuthology.orchestra.run.smithi052.stdout: Running scriptlet: openssl-libs-1:3.0.7-24.el9.x86_64 202/202 2024-03-04T09:19:24.781 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : ceph-2:19.0.0-1739.g1bc250c7.el9.x86_64 1/202 2024-03-04T09:19:24.781 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : ceph-base-2:19.0.0-1739.g1bc250c7.el9.x86_64 2/202 2024-03-04T09:19:24.781 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : ceph-common-2:19.0.0-1739.g1bc250c7.el9.x86_64 3/202 2024-03-04T09:19:24.781 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : ceph-fuse-2:19.0.0-1739.g1bc250c7.el9.x86_64 4/202 2024-03-04T09:19:24.781 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-1739.g1bc25 5/202 2024-03-04T09:19:24.781 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : ceph-mds-2:19.0.0-1739.g1bc250c7.el9.x86_64 6/202 2024-03-04T09:19:24.781 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : ceph-mgr-2:19.0.0-1739.g1bc250c7.el9.x86_64 7/202 2024-03-04T09:19:24.781 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : ceph-mon-2:19.0.0-1739.g1bc250c7.el9.x86_64 8/202 2024-03-04T09:19:24.781 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : ceph-osd-2:19.0.0-1739.g1bc250c7.el9.x86_64 9/202 2024-03-04T09:19:24.781 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : ceph-radosgw-2:19.0.0-1739.g1bc250c7.el9.x86_64 10/202 2024-03-04T09:19:24.781 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : ceph-selinux-2:19.0.0-1739.g1bc250c7.el9.x86_64 11/202 2024-03-04T09:19:24.781 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : ceph-test-2:19.0.0-1739.g1bc250c7.el9.x86_64 12/202 2024-03-04T09:19:24.782 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : cephfs-mirror-2:19.0.0-1739.g1bc250c7.el9.x86_64 13/202 2024-03-04T09:19:24.782 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : cephfs-shell-2:19.0.0-1739.g1bc250c7.el9.x86_64 14/202 2024-03-04T09:19:24.782 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : libcephfs-devel-2:19.0.0-1739.g1bc250c7.el9.x86_ 15/202 2024-03-04T09:19:24.782 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : libcephfs2-2:19.0.0-1739.g1bc250c7.el9.x86_64 16/202 2024-03-04T09:19:24.782 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : libcephsqlite-2:19.0.0-1739.g1bc250c7.el9.x86_64 17/202 2024-03-04T09:19:24.782 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : librados-devel-2:19.0.0-1739.g1bc250c7.el9.x86_6 18/202 2024-03-04T09:19:24.782 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : libradosstriper1-2:19.0.0-1739.g1bc250c7.el9.x86 19/202 2024-03-04T09:19:24.782 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : librgw2-2:19.0.0-1739.g1bc250c7.el9.x86_64 20/202 2024-03-04T09:19:24.782 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-ceph-argparse-2:19.0.0-1739.g1bc250c7.el 21/202 2024-03-04T09:19:24.783 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-ceph-common-2:19.0.0-1739.g1bc250c7.el9. 22/202 2024-03-04T09:19:24.783 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-cephfs-2:19.0.0-1739.g1bc250c7.el9.x86_6 23/202 2024-03-04T09:19:24.783 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-rados-2:19.0.0-1739.g1bc250c7.el9.x86_64 24/202 2024-03-04T09:19:24.783 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-rbd-2:19.0.0-1739.g1bc250c7.el9.x86_64 25/202 2024-03-04T09:19:24.783 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-rgw-2:19.0.0-1739.g1bc250c7.el9.x86_64 26/202 2024-03-04T09:19:24.801 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : rbd-fuse-2:19.0.0-1739.g1bc250c7.el9.x86_64 27/202 2024-03-04T09:19:24.801 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : rbd-mirror-2:19.0.0-1739.g1bc250c7.el9.x86_64 28/202 2024-03-04T09:19:24.801 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : rbd-nbd-2:19.0.0-1739.g1bc250c7.el9.x86_64 29/202 2024-03-04T09:19:24.801 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-1739.g1bc250c7. 30/202 2024-03-04T09:19:24.801 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-1739.g1bc250c7.el9.noa 31/202 2024-03-04T09:19:24.801 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-1739.g1bc250c7.el9.n 32/202 2024-03-04T09:19:24.802 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-1739.g1bc 33/202 2024-03-04T09:19:24.802 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-1739.g1bc250c7.el 34/202 2024-03-04T09:19:24.802 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : ceph-mgr-rook-2:19.0.0-1739.g1bc250c7.el9.noarch 35/202 2024-03-04T09:19:24.802 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-1739.g1bc250c7.e 36/202 2024-03-04T09:19:24.802 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : ceph-volume-2:19.0.0-1739.g1bc250c7.el9.noarch 37/202 2024-03-04T09:19:24.802 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : cephadm-2:19.0.0-1739.g1bc250c7.el9.noarch 38/202 2024-03-04T09:19:24.802 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : cephfs-top-2:19.0.0-1739.g1bc250c7.el9.noarch 39/202 2024-03-04T09:19:24.802 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : automake-1.16.2-8.el9.noarch 40/202 2024-03-04T09:19:24.802 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : bison-3.7.4-5.el9.x86_64 41/202 2024-03-04T09:19:24.802 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 42/202 2024-03-04T09:19:24.802 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 43/202 2024-03-04T09:19:24.802 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : flex-2.6.4-9.el9.x86_64 44/202 2024-03-04T09:19:24.802 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 45/202 2024-03-04T09:19:24.802 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 46/202 2024-03-04T09:19:24.803 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 47/202 2024-03-04T09:19:24.803 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 48/202 2024-03-04T09:19:24.803 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 49/202 2024-03-04T09:19:24.803 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 50/202 2024-03-04T09:19:24.803 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 51/202 2024-03-04T09:19:24.803 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 52/202 2024-03-04T09:19:24.803 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 53/202 2024-03-04T09:19:24.804 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 54/202 2024-03-04T09:19:24.804 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 55/202 2024-03-04T09:19:24.804 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : lua-5.4.4-4.el9.x86_64 56/202 2024-03-04T09:19:24.804 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 57/202 2024-03-04T09:19:24.804 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 58/202 2024-03-04T09:19:24.804 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : openssl-devel-1:3.0.7-27.el9.x86_64 59/202 2024-03-04T09:19:24.804 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : postgresql-13.11-1.el9.x86_64 60/202 2024-03-04T09:19:24.804 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : postgresql-contrib-13.11-1.el9.x86_64 61/202 2024-03-04T09:19:24.805 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : postgresql-private-libs-13.11-1.el9.x86_64 62/202 2024-03-04T09:19:24.805 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : postgresql-server-13.11-1.el9.x86_64 63/202 2024-03-04T09:19:24.805 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 64/202 2024-03-04T09:19:24.805 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 65/202 2024-03-04T09:19:24.805 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 66/202 2024-03-04T09:19:24.805 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 67/202 2024-03-04T09:19:24.805 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 68/202 2024-03-04T09:19:24.805 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 69/202 2024-03-04T09:19:24.805 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 70/202 2024-03-04T09:19:24.805 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 71/202 2024-03-04T09:19:24.805 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 72/202 2024-03-04T09:19:24.805 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 73/202 2024-03-04T09:19:24.805 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 74/202 2024-03-04T09:19:24.806 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 75/202 2024-03-04T09:19:24.806 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 76/202 2024-03-04T09:19:24.806 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 77/202 2024-03-04T09:19:24.806 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 78/202 2024-03-04T09:19:24.806 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 79/202 2024-03-04T09:19:24.806 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 80/202 2024-03-04T09:19:24.806 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 81/202 2024-03-04T09:19:24.806 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 82/202 2024-03-04T09:19:24.806 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 83/202 2024-03-04T09:19:24.806 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 84/202 2024-03-04T09:19:24.806 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : qatlib-23.11.0-1.el9.x86_64 85/202 2024-03-04T09:19:24.806 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : qatlib-service-23.11.0-1.el9.x86_64 86/202 2024-03-04T09:19:24.806 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : qatzip-libs-1.1.2-2.el9.x86_64 87/202 2024-03-04T09:19:24.807 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 88/202 2024-03-04T09:19:24.807 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 89/202 2024-03-04T09:19:24.807 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 90/202 2024-03-04T09:19:24.807 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 91/202 2024-03-04T09:19:24.807 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : zlib-devel-1.2.11-40.el9.x86_64 92/202 2024-03-04T09:19:24.807 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 93/202 2024-03-04T09:19:24.807 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : jq-1.6-16.el9.x86_64 94/202 2024-03-04T09:19:24.807 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 95/202 2024-03-04T09:19:24.807 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 96/202 2024-03-04T09:19:24.808 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 97/202 2024-03-04T09:19:24.808 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 98/202 2024-03-04T09:19:24.808 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 99/202 2024-03-04T09:19:24.808 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-ply-3.11-14.el9.noarch 100/202 2024-03-04T09:19:24.808 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 101/202 2024-03-04T09:19:24.808 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 102/202 2024-03-04T09:19:24.808 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 103/202 2024-03-04T09:19:24.808 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 104/202 2024-03-04T09:19:24.808 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 105/202 2024-03-04T09:19:24.808 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 106/202 2024-03-04T09:19:24.808 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 107/202 2024-03-04T09:19:24.808 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 108/202 2024-03-04T09:19:24.808 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 109/202 2024-03-04T09:19:24.809 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 110/202 2024-03-04T09:19:24.809 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 111/202 2024-03-04T09:19:24.809 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 112/202 2024-03-04T09:19:24.809 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 113/202 2024-03-04T09:19:24.809 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 114/202 2024-03-04T09:19:24.809 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 115/202 2024-03-04T09:19:24.809 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-asyncssh-2.13.2-4.el9.noarch 116/202 2024-03-04T09:19:24.809 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 117/202 2024-03-04T09:19:24.809 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 118/202 2024-03-04T09:19:24.809 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 119/202 2024-03-04T09:19:24.809 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 120/202 2024-03-04T09:19:24.809 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 121/202 2024-03-04T09:19:24.809 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 122/202 2024-03-04T09:19:24.809 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 123/202 2024-03-04T09:19:24.810 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-google-auth-1:2.28.0-1.el9.noarch 124/202 2024-03-04T09:19:24.810 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 125/202 2024-03-04T09:19:24.810 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 126/202 2024-03-04T09:19:24.810 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 127/202 2024-03-04T09:19:24.810 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 128/202 2024-03-04T09:19:24.810 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 129/202 2024-03-04T09:19:24.810 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 130/202 2024-03-04T09:19:24.810 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 131/202 2024-03-04T09:19:24.810 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 132/202 2024-03-04T09:19:24.810 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 133/202 2024-03-04T09:19:24.810 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 134/202 2024-03-04T09:19:24.810 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 135/202 2024-03-04T09:19:24.810 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 136/202 2024-03-04T09:19:24.811 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 137/202 2024-03-04T09:19:24.811 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 138/202 2024-03-04T09:19:24.811 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 139/202 2024-03-04T09:19:24.811 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 140/202 2024-03-04T09:19:24.811 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 141/202 2024-03-04T09:19:24.811 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 142/202 2024-03-04T09:19:24.811 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 143/202 2024-03-04T09:19:24.811 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 144/202 2024-03-04T09:19:24.811 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 145/202 2024-03-04T09:19:24.811 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 146/202 2024-03-04T09:19:24.811 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 147/202 2024-03-04T09:19:24.811 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 148/202 2024-03-04T09:19:24.811 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : re2-1:20211101-3.el9.x86_64 149/202 2024-03-04T09:19:24.811 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 150/202 2024-03-04T09:19:24.812 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : librados2-2:19.0.0-1739.g1bc250c7.el9.x86_64 151/202 2024-03-04T09:19:24.812 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 152/202 2024-03-04T09:19:24.812 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : librbd1-2:19.0.0-1739.g1bc250c7.el9.x86_64 153/202 2024-03-04T09:19:24.812 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 154/202 2024-03-04T09:19:24.812 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : libacl-devel-2.3.1-4.el9.x86_64 155/202 2024-03-04T09:19:24.812 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : libacl-devel-2.3.1-3.el9.x86_64 156/202 2024-03-04T09:19:24.812 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : NetworkManager-1:1.46.0-1.el9.x86_64 157/202 2024-03-04T09:19:24.812 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : NetworkManager-1:1.44.0-2.el9.x86_64 158/202 2024-03-04T09:19:24.812 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : NetworkManager-libnm-1:1.46.0-1.el9.x86_64 159/202 2024-03-04T09:19:24.812 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 160/202 2024-03-04T09:19:24.813 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : NetworkManager-team-1:1.46.0-1.el9.x86_64 161/202 2024-03-04T09:19:24.813 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : NetworkManager-team-1:1.44.0-2.el9.x86_64 162/202 2024-03-04T09:19:24.813 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : NetworkManager-tui-1:1.46.0-1.el9.x86_64 163/202 2024-03-04T09:19:24.813 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : NetworkManager-tui-1:1.44.0-2.el9.x86_64 164/202 2024-03-04T09:19:24.813 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : acl-2.3.1-4.el9.x86_64 165/202 2024-03-04T09:19:24.813 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : acl-2.3.1-3.el9.x86_64 166/202 2024-03-04T09:19:24.813 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 167/202 2024-03-04T09:19:24.813 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : elfutils-debuginfod-client-0.189-3.el9.x86_64 168/202 2024-03-04T09:19:24.813 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 169/202 2024-03-04T09:19:24.814 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : elfutils-libelf-0.189-3.el9.x86_64 170/202 2024-03-04T09:19:24.814 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 171/202 2024-03-04T09:19:24.814 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : elfutils-libs-0.189-3.el9.x86_64 172/202 2024-03-04T09:19:24.814 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : iproute-6.7.0-2.el9.x86_64 173/202 2024-03-04T09:19:24.814 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : iproute-6.2.0-5.el9.x86_64 174/202 2024-03-04T09:19:24.814 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : iproute-tc-6.7.0-2.el9.x86_64 175/202 2024-03-04T09:19:24.814 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : iproute-tc-6.2.0-5.el9.x86_64 176/202 2024-03-04T09:19:24.814 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : libacl-2.3.1-4.el9.x86_64 177/202 2024-03-04T09:19:24.815 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : libacl-2.3.1-3.el9.x86_64 178/202 2024-03-04T09:19:24.815 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : libblkid-2.37.4-18.el9.x86_64 179/202 2024-03-04T09:19:24.815 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : libblkid-2.37.4-15.el9.x86_64 180/202 2024-03-04T09:19:24.815 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : libfdisk-2.37.4-18.el9.x86_64 181/202 2024-03-04T09:19:24.815 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : libfdisk-2.37.4-15.el9.x86_64 182/202 2024-03-04T09:19:24.815 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : libmount-2.37.4-18.el9.x86_64 183/202 2024-03-04T09:19:24.815 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : libmount-2.37.4-15.el9.x86_64 184/202 2024-03-04T09:19:24.815 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : libsmartcols-2.37.4-18.el9.x86_64 185/202 2024-03-04T09:19:24.815 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : libsmartcols-2.37.4-15.el9.x86_64 186/202 2024-03-04T09:19:24.815 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : libuuid-2.37.4-18.el9.x86_64 187/202 2024-03-04T09:19:24.815 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : libuuid-2.37.4-15.el9.x86_64 188/202 2024-03-04T09:19:24.815 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 189/202 2024-03-04T09:19:24.815 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : openssl-1:3.0.7-24.el9.x86_64 190/202 2024-03-04T09:19:24.815 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 191/202 2024-03-04T09:19:24.816 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : openssl-libs-1:3.0.7-24.el9.x86_64 192/202 2024-03-04T09:19:24.816 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : util-linux-2.37.4-18.el9.x86_64 193/202 2024-03-04T09:19:24.816 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : util-linux-2.37.4-15.el9.x86_64 194/202 2024-03-04T09:19:24.816 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : util-linux-core-2.37.4-18.el9.x86_64 195/202 2024-03-04T09:19:24.816 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : util-linux-core-2.37.4-15.el9.x86_64 196/202 2024-03-04T09:19:24.816 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : util-linux-user-2.37.4-18.el9.x86_64 197/202 2024-03-04T09:19:24.816 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : util-linux-user-2.37.4-15.el9.x86_64 198/202 2024-03-04T09:19:24.816 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : xfsdump-3.1.12-4.el9.x86_64 199/202 2024-03-04T09:19:24.816 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : xfsdump-3.1.12-3.el9.x86_64 200/202 2024-03-04T09:19:24.816 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : xfsprogs-6.3.0-1.el9.x86_64 201/202 2024-03-04T09:19:27.561 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : xfsprogs-5.19.0-4.el9.x86_64 202/202 2024-03-04T09:19:27.561 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-03-04T09:19:27.561 INFO:teuthology.orchestra.run.smithi052.stdout:Upgraded: 2024-03-04T09:19:27.561 INFO:teuthology.orchestra.run.smithi052.stdout: NetworkManager-1:1.46.0-1.el9.x86_64 2024-03-04T09:19:27.562 INFO:teuthology.orchestra.run.smithi052.stdout: NetworkManager-libnm-1:1.46.0-1.el9.x86_64 2024-03-04T09:19:27.562 INFO:teuthology.orchestra.run.smithi052.stdout: NetworkManager-team-1:1.46.0-1.el9.x86_64 2024-03-04T09:19:27.562 INFO:teuthology.orchestra.run.smithi052.stdout: NetworkManager-tui-1:1.46.0-1.el9.x86_64 2024-03-04T09:19:27.562 INFO:teuthology.orchestra.run.smithi052.stdout: acl-2.3.1-4.el9.x86_64 2024-03-04T09:19:27.562 INFO:teuthology.orchestra.run.smithi052.stdout: elfutils-debuginfod-client-0.190-2.el9.x86_64 2024-03-04T09:19:27.562 INFO:teuthology.orchestra.run.smithi052.stdout: elfutils-libelf-0.190-2.el9.x86_64 2024-03-04T09:19:27.562 INFO:teuthology.orchestra.run.smithi052.stdout: elfutils-libs-0.190-2.el9.x86_64 2024-03-04T09:19:27.562 INFO:teuthology.orchestra.run.smithi052.stdout: iproute-6.7.0-2.el9.x86_64 2024-03-04T09:19:27.562 INFO:teuthology.orchestra.run.smithi052.stdout: iproute-tc-6.7.0-2.el9.x86_64 2024-03-04T09:19:27.562 INFO:teuthology.orchestra.run.smithi052.stdout: libacl-2.3.1-4.el9.x86_64 2024-03-04T09:19:27.562 INFO:teuthology.orchestra.run.smithi052.stdout: libacl-devel-2.3.1-4.el9.x86_64 2024-03-04T09:19:27.563 INFO:teuthology.orchestra.run.smithi052.stdout: libblkid-2.37.4-18.el9.x86_64 2024-03-04T09:19:27.563 INFO:teuthology.orchestra.run.smithi052.stdout: libfdisk-2.37.4-18.el9.x86_64 2024-03-04T09:19:27.563 INFO:teuthology.orchestra.run.smithi052.stdout: libmount-2.37.4-18.el9.x86_64 2024-03-04T09:19:27.563 INFO:teuthology.orchestra.run.smithi052.stdout: librados2-2:19.0.0-1739.g1bc250c7.el9.x86_64 2024-03-04T09:19:27.563 INFO:teuthology.orchestra.run.smithi052.stdout: librbd1-2:19.0.0-1739.g1bc250c7.el9.x86_64 2024-03-04T09:19:27.563 INFO:teuthology.orchestra.run.smithi052.stdout: libsmartcols-2.37.4-18.el9.x86_64 2024-03-04T09:19:27.563 INFO:teuthology.orchestra.run.smithi052.stdout: libuuid-2.37.4-18.el9.x86_64 2024-03-04T09:19:27.563 INFO:teuthology.orchestra.run.smithi052.stdout: openssl-1:3.0.7-27.el9.x86_64 2024-03-04T09:19:27.563 INFO:teuthology.orchestra.run.smithi052.stdout: openssl-libs-1:3.0.7-27.el9.x86_64 2024-03-04T09:19:27.563 INFO:teuthology.orchestra.run.smithi052.stdout: util-linux-2.37.4-18.el9.x86_64 2024-03-04T09:19:27.563 INFO:teuthology.orchestra.run.smithi052.stdout: util-linux-core-2.37.4-18.el9.x86_64 2024-03-04T09:19:27.564 INFO:teuthology.orchestra.run.smithi052.stdout: util-linux-user-2.37.4-18.el9.x86_64 2024-03-04T09:19:27.564 INFO:teuthology.orchestra.run.smithi052.stdout: xfsdump-3.1.12-4.el9.x86_64 2024-03-04T09:19:27.564 INFO:teuthology.orchestra.run.smithi052.stdout: xfsprogs-6.3.0-1.el9.x86_64 2024-03-04T09:19:27.564 INFO:teuthology.orchestra.run.smithi052.stdout:Installed: 2024-03-04T09:19:27.564 INFO:teuthology.orchestra.run.smithi052.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-03-04T09:19:27.564 INFO:teuthology.orchestra.run.smithi052.stdout: automake-1.16.2-8.el9.noarch 2024-03-04T09:19:27.564 INFO:teuthology.orchestra.run.smithi052.stdout: bison-3.7.4-5.el9.x86_64 2024-03-04T09:19:27.564 INFO:teuthology.orchestra.run.smithi052.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-03-04T09:19:27.564 INFO:teuthology.orchestra.run.smithi052.stdout: ceph-2:19.0.0-1739.g1bc250c7.el9.x86_64 2024-03-04T09:19:27.564 INFO:teuthology.orchestra.run.smithi052.stdout: ceph-base-2:19.0.0-1739.g1bc250c7.el9.x86_64 2024-03-04T09:19:27.565 INFO:teuthology.orchestra.run.smithi052.stdout: ceph-common-2:19.0.0-1739.g1bc250c7.el9.x86_64 2024-03-04T09:19:27.565 INFO:teuthology.orchestra.run.smithi052.stdout: ceph-fuse-2:19.0.0-1739.g1bc250c7.el9.x86_64 2024-03-04T09:19:27.565 INFO:teuthology.orchestra.run.smithi052.stdout: ceph-grafana-dashboards-2:19.0.0-1739.g1bc250c7.el9.noarch 2024-03-04T09:19:27.565 INFO:teuthology.orchestra.run.smithi052.stdout: ceph-immutable-object-cache-2:19.0.0-1739.g1bc250c7.el9.x86_64 2024-03-04T09:19:27.565 INFO:teuthology.orchestra.run.smithi052.stdout: ceph-mds-2:19.0.0-1739.g1bc250c7.el9.x86_64 2024-03-04T09:19:27.565 INFO:teuthology.orchestra.run.smithi052.stdout: ceph-mgr-2:19.0.0-1739.g1bc250c7.el9.x86_64 2024-03-04T09:19:27.565 INFO:teuthology.orchestra.run.smithi052.stdout: ceph-mgr-cephadm-2:19.0.0-1739.g1bc250c7.el9.noarch 2024-03-04T09:19:27.565 INFO:teuthology.orchestra.run.smithi052.stdout: ceph-mgr-dashboard-2:19.0.0-1739.g1bc250c7.el9.noarch 2024-03-04T09:19:27.565 INFO:teuthology.orchestra.run.smithi052.stdout: ceph-mgr-diskprediction-local-2:19.0.0-1739.g1bc250c7.el9.noarch 2024-03-04T09:19:27.565 INFO:teuthology.orchestra.run.smithi052.stdout: ceph-mgr-modules-core-2:19.0.0-1739.g1bc250c7.el9.noarch 2024-03-04T09:19:27.566 INFO:teuthology.orchestra.run.smithi052.stdout: ceph-mgr-rook-2:19.0.0-1739.g1bc250c7.el9.noarch 2024-03-04T09:19:27.566 INFO:teuthology.orchestra.run.smithi052.stdout: ceph-mon-2:19.0.0-1739.g1bc250c7.el9.x86_64 2024-03-04T09:19:27.566 INFO:teuthology.orchestra.run.smithi052.stdout: ceph-osd-2:19.0.0-1739.g1bc250c7.el9.x86_64 2024-03-04T09:19:27.566 INFO:teuthology.orchestra.run.smithi052.stdout: ceph-prometheus-alerts-2:19.0.0-1739.g1bc250c7.el9.noarch 2024-03-04T09:19:27.566 INFO:teuthology.orchestra.run.smithi052.stdout: ceph-radosgw-2:19.0.0-1739.g1bc250c7.el9.x86_64 2024-03-04T09:19:27.566 INFO:teuthology.orchestra.run.smithi052.stdout: ceph-selinux-2:19.0.0-1739.g1bc250c7.el9.x86_64 2024-03-04T09:19:27.566 INFO:teuthology.orchestra.run.smithi052.stdout: ceph-test-2:19.0.0-1739.g1bc250c7.el9.x86_64 2024-03-04T09:19:27.567 INFO:teuthology.orchestra.run.smithi052.stdout: ceph-volume-2:19.0.0-1739.g1bc250c7.el9.noarch 2024-03-04T09:19:27.567 INFO:teuthology.orchestra.run.smithi052.stdout: cephadm-2:19.0.0-1739.g1bc250c7.el9.noarch 2024-03-04T09:19:27.567 INFO:teuthology.orchestra.run.smithi052.stdout: cephfs-mirror-2:19.0.0-1739.g1bc250c7.el9.x86_64 2024-03-04T09:19:27.567 INFO:teuthology.orchestra.run.smithi052.stdout: cephfs-shell-2:19.0.0-1739.g1bc250c7.el9.x86_64 2024-03-04T09:19:27.567 INFO:teuthology.orchestra.run.smithi052.stdout: cephfs-top-2:19.0.0-1739.g1bc250c7.el9.noarch 2024-03-04T09:19:27.567 INFO:teuthology.orchestra.run.smithi052.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-03-04T09:19:27.567 INFO:teuthology.orchestra.run.smithi052.stdout: flex-2.6.4-9.el9.x86_64 2024-03-04T09:19:27.567 INFO:teuthology.orchestra.run.smithi052.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-03-04T09:19:27.567 INFO:teuthology.orchestra.run.smithi052.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-03-04T09:19:27.567 INFO:teuthology.orchestra.run.smithi052.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-03-04T09:19:27.567 INFO:teuthology.orchestra.run.smithi052.stdout: fuse-2.9.9-15.el9.x86_64 2024-03-04T09:19:27.568 INFO:teuthology.orchestra.run.smithi052.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-03-04T09:19:27.568 INFO:teuthology.orchestra.run.smithi052.stdout: grpc-data-1.46.7-10.el9.noarch 2024-03-04T09:19:27.568 INFO:teuthology.orchestra.run.smithi052.stdout: jq-1.6-16.el9.x86_64 2024-03-04T09:19:27.568 INFO:teuthology.orchestra.run.smithi052.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-03-04T09:19:27.568 INFO:teuthology.orchestra.run.smithi052.stdout: libarrow-9.0.0-10.el9.x86_64 2024-03-04T09:19:27.568 INFO:teuthology.orchestra.run.smithi052.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-03-04T09:19:27.568 INFO:teuthology.orchestra.run.smithi052.stdout: libcephfs-devel-2:19.0.0-1739.g1bc250c7.el9.x86_64 2024-03-04T09:19:27.568 INFO:teuthology.orchestra.run.smithi052.stdout: libcephfs2-2:19.0.0-1739.g1bc250c7.el9.x86_64 2024-03-04T09:19:27.568 INFO:teuthology.orchestra.run.smithi052.stdout: libcephsqlite-2:19.0.0-1739.g1bc250c7.el9.x86_64 2024-03-04T09:19:27.568 INFO:teuthology.orchestra.run.smithi052.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-03-04T09:19:27.568 INFO:teuthology.orchestra.run.smithi052.stdout: liboath-2.6.7-2.el9.x86_64 2024-03-04T09:19:27.568 INFO:teuthology.orchestra.run.smithi052.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-03-04T09:19:27.569 INFO:teuthology.orchestra.run.smithi052.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-03-04T09:19:27.569 INFO:teuthology.orchestra.run.smithi052.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-03-04T09:19:27.569 INFO:teuthology.orchestra.run.smithi052.stdout: librados-devel-2:19.0.0-1739.g1bc250c7.el9.x86_64 2024-03-04T09:19:27.569 INFO:teuthology.orchestra.run.smithi052.stdout: libradosstriper1-2:19.0.0-1739.g1bc250c7.el9.x86_64 2024-03-04T09:19:27.569 INFO:teuthology.orchestra.run.smithi052.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-03-04T09:19:27.569 INFO:teuthology.orchestra.run.smithi052.stdout: librgw2-2:19.0.0-1739.g1bc250c7.el9.x86_64 2024-03-04T09:19:27.569 INFO:teuthology.orchestra.run.smithi052.stdout: libtool-2.4.6-45.el9.x86_64 2024-03-04T09:19:27.569 INFO:teuthology.orchestra.run.smithi052.stdout: libunwind-1.6.2-1.el9.x86_64 2024-03-04T09:19:27.569 INFO:teuthology.orchestra.run.smithi052.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-03-04T09:19:27.569 INFO:teuthology.orchestra.run.smithi052.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-03-04T09:19:27.569 INFO:teuthology.orchestra.run.smithi052.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-03-04T09:19:27.570 INFO:teuthology.orchestra.run.smithi052.stdout: lua-5.4.4-4.el9.x86_64 2024-03-04T09:19:27.570 INFO:teuthology.orchestra.run.smithi052.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-03-04T09:19:27.570 INFO:teuthology.orchestra.run.smithi052.stdout: luarocks-3.9.2-1.el9.noarch 2024-03-04T09:19:27.570 INFO:teuthology.orchestra.run.smithi052.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2024-03-04T09:19:27.570 INFO:teuthology.orchestra.run.smithi052.stdout: openblas-0.3.21-2.el9.x86_64 2024-03-04T09:19:27.570 INFO:teuthology.orchestra.run.smithi052.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-03-04T09:19:27.570 INFO:teuthology.orchestra.run.smithi052.stdout: openssl-devel-1:3.0.7-27.el9.x86_64 2024-03-04T09:19:27.570 INFO:teuthology.orchestra.run.smithi052.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-03-04T09:19:27.570 INFO:teuthology.orchestra.run.smithi052.stdout: postgresql-13.11-1.el9.x86_64 2024-03-04T09:19:27.570 INFO:teuthology.orchestra.run.smithi052.stdout: postgresql-contrib-13.11-1.el9.x86_64 2024-03-04T09:19:27.570 INFO:teuthology.orchestra.run.smithi052.stdout: postgresql-private-libs-13.11-1.el9.x86_64 2024-03-04T09:19:27.570 INFO:teuthology.orchestra.run.smithi052.stdout: postgresql-server-13.11-1.el9.x86_64 2024-03-04T09:19:27.571 INFO:teuthology.orchestra.run.smithi052.stdout: protobuf-3.14.0-13.el9.x86_64 2024-03-04T09:19:27.571 INFO:teuthology.orchestra.run.smithi052.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-03-04T09:19:27.571 INFO:teuthology.orchestra.run.smithi052.stdout: python3-asyncssh-2.13.2-4.el9.noarch 2024-03-04T09:19:27.571 INFO:teuthology.orchestra.run.smithi052.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-03-04T09:19:27.571 INFO:teuthology.orchestra.run.smithi052.stdout: python3-babel-2.9.1-2.el9.noarch 2024-03-04T09:19:27.571 INFO:teuthology.orchestra.run.smithi052.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-03-04T09:19:27.571 INFO:teuthology.orchestra.run.smithi052.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-03-04T09:19:27.571 INFO:teuthology.orchestra.run.smithi052.stdout: python3-ceph-argparse-2:19.0.0-1739.g1bc250c7.el9.x86_64 2024-03-04T09:19:27.571 INFO:teuthology.orchestra.run.smithi052.stdout: python3-ceph-common-2:19.0.0-1739.g1bc250c7.el9.x86_64 2024-03-04T09:19:27.571 INFO:teuthology.orchestra.run.smithi052.stdout: python3-cephfs-2:19.0.0-1739.g1bc250c7.el9.x86_64 2024-03-04T09:19:27.571 INFO:teuthology.orchestra.run.smithi052.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-03-04T09:19:27.572 INFO:teuthology.orchestra.run.smithi052.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-03-04T09:19:27.572 INFO:teuthology.orchestra.run.smithi052.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-03-04T09:19:27.572 INFO:teuthology.orchestra.run.smithi052.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-03-04T09:19:27.572 INFO:teuthology.orchestra.run.smithi052.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-03-04T09:19:27.572 INFO:teuthology.orchestra.run.smithi052.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-03-04T09:19:27.572 INFO:teuthology.orchestra.run.smithi052.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-03-04T09:19:27.572 INFO:teuthology.orchestra.run.smithi052.stdout: python3-devel-3.9.17-2.el9.x86_64 2024-03-04T09:19:27.572 INFO:teuthology.orchestra.run.smithi052.stdout: python3-google-auth-1:2.28.0-1.el9.noarch 2024-03-04T09:19:27.572 INFO:teuthology.orchestra.run.smithi052.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-03-04T09:19:27.572 INFO:teuthology.orchestra.run.smithi052.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-03-04T09:19:27.572 INFO:teuthology.orchestra.run.smithi052.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-03-04T09:19:27.572 INFO:teuthology.orchestra.run.smithi052.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-03-04T09:19:27.573 INFO:teuthology.orchestra.run.smithi052.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-03-04T09:19:27.573 INFO:teuthology.orchestra.run.smithi052.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-03-04T09:19:27.573 INFO:teuthology.orchestra.run.smithi052.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-03-04T09:19:27.573 INFO:teuthology.orchestra.run.smithi052.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-03-04T09:19:27.573 INFO:teuthology.orchestra.run.smithi052.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-03-04T09:19:27.573 INFO:teuthology.orchestra.run.smithi052.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-03-04T09:19:27.573 INFO:teuthology.orchestra.run.smithi052.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-03-04T09:19:27.573 INFO:teuthology.orchestra.run.smithi052.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-03-04T09:19:27.573 INFO:teuthology.orchestra.run.smithi052.stdout: python3-mako-1.1.4-6.el9.noarch 2024-03-04T09:19:27.573 INFO:teuthology.orchestra.run.smithi052.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-03-04T09:19:27.573 INFO:teuthology.orchestra.run.smithi052.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-03-04T09:19:27.573 INFO:teuthology.orchestra.run.smithi052.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-03-04T09:19:27.574 INFO:teuthology.orchestra.run.smithi052.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-03-04T09:19:27.574 INFO:teuthology.orchestra.run.smithi052.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-03-04T09:19:27.574 INFO:teuthology.orchestra.run.smithi052.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-03-04T09:19:27.574 INFO:teuthology.orchestra.run.smithi052.stdout: python3-packaging-20.9-5.el9.noarch 2024-03-04T09:19:27.574 INFO:teuthology.orchestra.run.smithi052.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-03-04T09:19:27.574 INFO:teuthology.orchestra.run.smithi052.stdout: python3-ply-3.11-14.el9.noarch 2024-03-04T09:19:27.574 INFO:teuthology.orchestra.run.smithi052.stdout: python3-portend-3.1.0-2.el9.noarch 2024-03-04T09:19:27.574 INFO:teuthology.orchestra.run.smithi052.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-03-04T09:19:27.574 INFO:teuthology.orchestra.run.smithi052.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-03-04T09:19:27.574 INFO:teuthology.orchestra.run.smithi052.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-03-04T09:19:27.575 INFO:teuthology.orchestra.run.smithi052.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-03-04T09:19:27.575 INFO:teuthology.orchestra.run.smithi052.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-03-04T09:19:27.575 INFO:teuthology.orchestra.run.smithi052.stdout: python3-pycparser-2.20-6.el9.noarch 2024-03-04T09:19:27.575 INFO:teuthology.orchestra.run.smithi052.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-03-04T09:19:27.575 INFO:teuthology.orchestra.run.smithi052.stdout: python3-pytz-2021.1-5.el9.noarch 2024-03-04T09:19:27.575 INFO:teuthology.orchestra.run.smithi052.stdout: python3-rados-2:19.0.0-1739.g1bc250c7.el9.x86_64 2024-03-04T09:19:27.575 INFO:teuthology.orchestra.run.smithi052.stdout: python3-rbd-2:19.0.0-1739.g1bc250c7.el9.x86_64 2024-03-04T09:19:27.576 INFO:teuthology.orchestra.run.smithi052.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-03-04T09:19:27.576 INFO:teuthology.orchestra.run.smithi052.stdout: python3-requests-2.25.1-8.el9.noarch 2024-03-04T09:19:27.576 INFO:teuthology.orchestra.run.smithi052.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-03-04T09:19:27.576 INFO:teuthology.orchestra.run.smithi052.stdout: python3-rgw-2:19.0.0-1739.g1bc250c7.el9.x86_64 2024-03-04T09:19:27.576 INFO:teuthology.orchestra.run.smithi052.stdout: python3-routes-2.5.1-5.el9.noarch 2024-03-04T09:19:27.576 INFO:teuthology.orchestra.run.smithi052.stdout: python3-rsa-4.9-2.el9.noarch 2024-03-04T09:19:27.576 INFO:teuthology.orchestra.run.smithi052.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-03-04T09:19:27.576 INFO:teuthology.orchestra.run.smithi052.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-03-04T09:19:27.576 INFO:teuthology.orchestra.run.smithi052.stdout: python3-toml-0.10.2-6.el9.noarch 2024-03-04T09:19:27.576 INFO:teuthology.orchestra.run.smithi052.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-03-04T09:19:27.576 INFO:teuthology.orchestra.run.smithi052.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-03-04T09:19:27.577 INFO:teuthology.orchestra.run.smithi052.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-03-04T09:19:27.577 INFO:teuthology.orchestra.run.smithi052.stdout: python3-webob-1.8.7-6.el9.noarch 2024-03-04T09:19:27.577 INFO:teuthology.orchestra.run.smithi052.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-03-04T09:19:27.577 INFO:teuthology.orchestra.run.smithi052.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-03-04T09:19:27.577 INFO:teuthology.orchestra.run.smithi052.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-03-04T09:19:27.577 INFO:teuthology.orchestra.run.smithi052.stdout: qatlib-23.11.0-1.el9.x86_64 2024-03-04T09:19:27.577 INFO:teuthology.orchestra.run.smithi052.stdout: qatlib-service-23.11.0-1.el9.x86_64 2024-03-04T09:19:27.577 INFO:teuthology.orchestra.run.smithi052.stdout: qatzip-libs-1.1.2-2.el9.x86_64 2024-03-04T09:19:27.577 INFO:teuthology.orchestra.run.smithi052.stdout: rbd-fuse-2:19.0.0-1739.g1bc250c7.el9.x86_64 2024-03-04T09:19:27.577 INFO:teuthology.orchestra.run.smithi052.stdout: rbd-mirror-2:19.0.0-1739.g1bc250c7.el9.x86_64 2024-03-04T09:19:27.577 INFO:teuthology.orchestra.run.smithi052.stdout: rbd-nbd-2:19.0.0-1739.g1bc250c7.el9.x86_64 2024-03-04T09:19:27.577 INFO:teuthology.orchestra.run.smithi052.stdout: re2-1:20211101-3.el9.x86_64 2024-03-04T09:19:27.578 INFO:teuthology.orchestra.run.smithi052.stdout: socat-1.7.4.1-5.el9.x86_64 2024-03-04T09:19:27.578 INFO:teuthology.orchestra.run.smithi052.stdout: thrift-0.15.0-2.el9.x86_64 2024-03-04T09:19:27.578 INFO:teuthology.orchestra.run.smithi052.stdout: uuid-1.6.2-55.el9.x86_64 2024-03-04T09:19:27.578 INFO:teuthology.orchestra.run.smithi052.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-03-04T09:19:27.578 INFO:teuthology.orchestra.run.smithi052.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-03-04T09:19:27.578 INFO:teuthology.orchestra.run.smithi052.stdout: zlib-devel-1.2.11-40.el9.x86_64 2024-03-04T09:19:27.578 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-03-04T09:19:27.579 INFO:teuthology.orchestra.run.smithi052.stdout:Complete! 2024-03-04T09:19:28.200 DEBUG:teuthology.parallel:result is None 2024-03-04T09:19:28.201 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd 2024-03-04T09:19:28.355 DEBUG:teuthology.orchestra.run.smithi052:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-03-04T09:19:28.377 INFO:teuthology.orchestra.run.smithi052.stdout:19.0.0-1739.g1bc250c7.el9 2024-03-04T09:19:28.378 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-1739.g1bc250c7.el9 2024-03-04T09:19:28.378 INFO:teuthology.task.install:The correct ceph version 19.0.0-1739.g1bc250c7 is installed. 2024-03-04T09:19:28.380 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd 2024-03-04T09:19:28.531 DEBUG:teuthology.orchestra.run.smithi106:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-03-04T09:19:28.557 INFO:teuthology.orchestra.run.smithi106.stdout:19.0.0-1739.g1bc250c7.el9 2024-03-04T09:19:28.558 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-1739.g1bc250c7.el9 2024-03-04T09:19:28.558 INFO:teuthology.task.install:The correct ceph version 19.0.0-1739.g1bc250c7 is installed. 2024-03-04T09:19:28.560 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd 2024-03-04T09:19:28.737 DEBUG:teuthology.orchestra.run.smithi191:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-03-04T09:19:28.763 INFO:teuthology.orchestra.run.smithi191.stdout:19.0.0-1739.g1bc250c7.el9 2024-03-04T09:19:28.763 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-1739.g1bc250c7.el9 2024-03-04T09:19:28.763 INFO:teuthology.task.install:The correct ceph version 19.0.0-1739.g1bc250c7 is installed. 2024-03-04T09:19:28.765 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-03-04T09:19:28.765 DEBUG:teuthology.orchestra.run.smithi052:> set -ex 2024-03-04T09:19:28.765 DEBUG:teuthology.orchestra.run.smithi052:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-03-04T09:19:28.797 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2024-03-04T09:19:28.797 DEBUG:teuthology.orchestra.run.smithi106:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-03-04T09:19:28.832 DEBUG:teuthology.orchestra.run.smithi191:> set -ex 2024-03-04T09:19:28.832 DEBUG:teuthology.orchestra.run.smithi191:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-03-04T09:19:28.865 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-03-04T09:19:28.866 DEBUG:teuthology.orchestra.run.smithi052:> set -ex 2024-03-04T09:19:28.866 DEBUG:teuthology.orchestra.run.smithi052:> sudo dd of=/usr/bin/daemon-helper 2024-03-04T09:19:28.894 DEBUG:teuthology.orchestra.run.smithi052:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-03-04T09:19:28.963 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2024-03-04T09:19:28.963 DEBUG:teuthology.orchestra.run.smithi106:> sudo dd of=/usr/bin/daemon-helper 2024-03-04T09:19:28.993 DEBUG:teuthology.orchestra.run.smithi106:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-03-04T09:19:29.064 DEBUG:teuthology.orchestra.run.smithi191:> set -ex 2024-03-04T09:19:29.064 DEBUG:teuthology.orchestra.run.smithi191:> sudo dd of=/usr/bin/daemon-helper 2024-03-04T09:19:29.095 DEBUG:teuthology.orchestra.run.smithi191:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-03-04T09:19:29.164 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-03-04T09:19:29.165 DEBUG:teuthology.orchestra.run.smithi052:> set -ex 2024-03-04T09:19:29.165 DEBUG:teuthology.orchestra.run.smithi052:> sudo dd of=/usr/bin/adjust-ulimits 2024-03-04T09:19:29.193 DEBUG:teuthology.orchestra.run.smithi052:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-03-04T09:19:29.261 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2024-03-04T09:19:29.261 DEBUG:teuthology.orchestra.run.smithi106:> sudo dd of=/usr/bin/adjust-ulimits 2024-03-04T09:19:29.288 DEBUG:teuthology.orchestra.run.smithi106:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-03-04T09:19:29.356 DEBUG:teuthology.orchestra.run.smithi191:> set -ex 2024-03-04T09:19:29.356 DEBUG:teuthology.orchestra.run.smithi191:> sudo dd of=/usr/bin/adjust-ulimits 2024-03-04T09:19:29.388 DEBUG:teuthology.orchestra.run.smithi191:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-03-04T09:19:29.458 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-03-04T09:19:29.458 DEBUG:teuthology.orchestra.run.smithi052:> set -ex 2024-03-04T09:19:29.458 DEBUG:teuthology.orchestra.run.smithi052:> sudo dd of=/usr/bin/stdin-killer 2024-03-04T09:19:29.487 DEBUG:teuthology.orchestra.run.smithi052:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-03-04T09:19:29.557 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2024-03-04T09:19:29.557 DEBUG:teuthology.orchestra.run.smithi106:> sudo dd of=/usr/bin/stdin-killer 2024-03-04T09:19:29.584 DEBUG:teuthology.orchestra.run.smithi106:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-03-04T09:19:29.651 DEBUG:teuthology.orchestra.run.smithi191:> set -ex 2024-03-04T09:19:29.651 DEBUG:teuthology.orchestra.run.smithi191:> sudo dd of=/usr/bin/stdin-killer 2024-03-04T09:19:29.682 DEBUG:teuthology.orchestra.run.smithi191:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-03-04T09:19:29.753 INFO:teuthology.run_tasks:Running task cephadm... 2024-03-04T09:19:29.863 INFO:tasks.cephadm:Config: {'roleless': False, 'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'max_mds': 3, 'session_timeout': 300, 'standby_replay': True}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'mds_export_ephemeral_random_max': 0.1, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'cephsqlite lock renewal timeout': 900000, 'debug cephsqlite': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180, 'osd shutdown pgref assert': True, 'osd_max_omap_entries_per_request': 10}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding'], 'sha1': '1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd', 'subvols': {'create': 2, 'subvol_options': '--size 25000000000'}} 2024-03-04T09:19:29.864 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd 2024-03-04T09:19:29.865 INFO:tasks.cephadm:Cluster fsid is 4e059f08-da08-11ee-95c6-87774f69a715 2024-03-04T09:19:29.865 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-03-04T09:19:29.865 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.52', 'mon.b': '172.21.15.106', 'mon.c': '172.21.15.191'} 2024-03-04T09:19:29.865 INFO:tasks.cephadm:First mon is mon.a on smithi052 2024-03-04T09:19:29.865 INFO:tasks.cephadm:First mgr is x 2024-03-04T09:19:29.865 INFO:tasks.cephadm:Normalizing hostnames... 2024-03-04T09:19:29.865 DEBUG:teuthology.orchestra.run.smithi052:> sudo hostname $(hostname -s) 2024-03-04T09:19:29.897 DEBUG:teuthology.orchestra.run.smithi106:> sudo hostname $(hostname -s) 2024-03-04T09:19:29.925 DEBUG:teuthology.orchestra.run.smithi191:> sudo hostname $(hostname -s) 2024-03-04T09:19:29.954 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2024-03-04T09:19:29.955 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd 2024-03-04T09:19:30.281 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': '1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/77279/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.0.0-1739-g1bc250c7', 'node_name': '172.21.5.36+adami06', 'job_name': 'ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.0.0-1739.g1bc250c7'}, 'url': 'https://3.chacra.ceph.com/r/ceph/wip-lusov-qdb/1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-03-04 00:06:26.635980', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'wip-lusov-qdb', 'chacra_url': 'https://3.chacra.ceph.com/repos/ceph/wip-lusov-qdb/1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd/centos/9/flavors/default/', 'archs': ['x86_64', 'source'], 'distro': 'centos'}] 2024-03-04T09:19:30.431 INFO:tasks.util.chacra:got chacra host 3.chacra.ceph.com, ref wip-lusov-qdb, sha1 1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F9%2Fx86_64&flavor=default&sha1=1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd 2024-03-04T09:19:30.433 INFO:tasks.cephadm:Discovered cachra url: https://3.chacra.ceph.com/binaries/ceph/wip-lusov-qdb/1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd/centos/9/x86_64/flavors/default/cephadm 2024-03-04T09:19:30.433 DEBUG:teuthology.orchestra.run.smithi052:> curl --silent -L https://3.chacra.ceph.com/binaries/ceph/wip-lusov-qdb/1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-03-04T09:19:31.205 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r--. 1 ubuntu ubuntu 766895 Mar 4 09:19 /home/ubuntu/cephtest/cephadm 2024-03-04T09:19:31.206 DEBUG:teuthology.orchestra.run.smithi106:> curl --silent -L https://3.chacra.ceph.com/binaries/ceph/wip-lusov-qdb/1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-03-04T09:19:31.880 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r--. 1 ubuntu ubuntu 766895 Mar 4 09:19 /home/ubuntu/cephtest/cephadm 2024-03-04T09:19:31.880 DEBUG:teuthology.orchestra.run.smithi191:> curl --silent -L https://3.chacra.ceph.com/binaries/ceph/wip-lusov-qdb/1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-03-04T09:19:33.156 INFO:teuthology.orchestra.run.smithi191.stdout:-rw-r--r--. 1 ubuntu ubuntu 766895 Mar 4 09:19 /home/ubuntu/cephtest/cephadm 2024-03-04T09:19:33.157 DEBUG:teuthology.orchestra.run.smithi052:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-03-04T09:19:33.175 DEBUG:teuthology.orchestra.run.smithi106:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-03-04T09:19:33.194 DEBUG:teuthology.orchestra.run.smithi191:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-03-04T09:19:33.231 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd on all hosts... 2024-03-04T09:19:33.231 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd pull 2024-03-04T09:19:33.234 DEBUG:teuthology.orchestra.run.smithi106:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd pull 2024-03-04T09:19:33.236 DEBUG:teuthology.orchestra.run.smithi191:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd pull 2024-03-04T09:19:34.079 INFO:teuthology.orchestra.run.smithi106.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd... 2024-03-04T09:19:34.171 INFO:teuthology.orchestra.run.smithi052.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd... 2024-03-04T09:19:34.174 INFO:teuthology.orchestra.run.smithi191.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd... 2024-03-04T09:20:10.516 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2024-03-04T09:20:10.516 INFO:teuthology.orchestra.run.smithi052.stdout: "ceph_version": "ceph version 19.0.0-1739-g1bc250c7 (1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd) squid (dev)", 2024-03-04T09:20:10.516 INFO:teuthology.orchestra.run.smithi052.stdout: "image_id": "f88ffb32aa6a7d5dfdc954d9f096f0c9cc47981d05ec8d117ba859a22d8185ba", 2024-03-04T09:20:10.516 INFO:teuthology.orchestra.run.smithi052.stdout: "repo_digests": [ 2024-03-04T09:20:10.516 INFO:teuthology.orchestra.run.smithi052.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5" 2024-03-04T09:20:10.516 INFO:teuthology.orchestra.run.smithi052.stdout: ] 2024-03-04T09:20:10.516 INFO:teuthology.orchestra.run.smithi052.stdout:} 2024-03-04T09:20:24.710 INFO:teuthology.orchestra.run.smithi191.stdout:{ 2024-03-04T09:20:24.710 INFO:teuthology.orchestra.run.smithi191.stdout: "ceph_version": "ceph version 19.0.0-1739-g1bc250c7 (1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd) squid (dev)", 2024-03-04T09:20:24.710 INFO:teuthology.orchestra.run.smithi191.stdout: "image_id": "f88ffb32aa6a7d5dfdc954d9f096f0c9cc47981d05ec8d117ba859a22d8185ba", 2024-03-04T09:20:24.710 INFO:teuthology.orchestra.run.smithi191.stdout: "repo_digests": [ 2024-03-04T09:20:24.711 INFO:teuthology.orchestra.run.smithi191.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5" 2024-03-04T09:20:24.711 INFO:teuthology.orchestra.run.smithi191.stdout: ] 2024-03-04T09:20:24.711 INFO:teuthology.orchestra.run.smithi191.stdout:} 2024-03-04T09:20:31.330 INFO:teuthology.orchestra.run.smithi106.stdout:{ 2024-03-04T09:20:31.330 INFO:teuthology.orchestra.run.smithi106.stdout: "ceph_version": "ceph version 19.0.0-1739-g1bc250c7 (1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd) squid (dev)", 2024-03-04T09:20:31.330 INFO:teuthology.orchestra.run.smithi106.stdout: "image_id": "f88ffb32aa6a7d5dfdc954d9f096f0c9cc47981d05ec8d117ba859a22d8185ba", 2024-03-04T09:20:31.330 INFO:teuthology.orchestra.run.smithi106.stdout: "repo_digests": [ 2024-03-04T09:20:31.330 INFO:teuthology.orchestra.run.smithi106.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5" 2024-03-04T09:20:31.330 INFO:teuthology.orchestra.run.smithi106.stdout: ] 2024-03-04T09:20:31.330 INFO:teuthology.orchestra.run.smithi106.stdout:} 2024-03-04T09:20:31.354 DEBUG:teuthology.orchestra.run.smithi052:> sudo mkdir -p /etc/ceph 2024-03-04T09:20:31.390 DEBUG:teuthology.orchestra.run.smithi106:> sudo mkdir -p /etc/ceph 2024-03-04T09:20:31.420 DEBUG:teuthology.orchestra.run.smithi191:> sudo mkdir -p /etc/ceph 2024-03-04T09:20:31.455 DEBUG:teuthology.orchestra.run.smithi052:> sudo chmod 777 /etc/ceph 2024-03-04T09:20:31.486 DEBUG:teuthology.orchestra.run.smithi106:> sudo chmod 777 /etc/ceph 2024-03-04T09:20:31.513 DEBUG:teuthology.orchestra.run.smithi191:> sudo chmod 777 /etc/ceph 2024-03-04T09:20:31.542 INFO:tasks.cephadm:Writing seed config... 2024-03-04T09:20:31.544 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2024-03-04T09:20:31.544 INFO:tasks.cephadm: override: [client] debug client = 20 2024-03-04T09:20:31.544 INFO:tasks.cephadm: override: [client] debug ms = 1 2024-03-04T09:20:31.544 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2024-03-04T09:20:31.544 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2024-03-04T09:20:31.544 INFO:tasks.cephadm: override: [global] ms die on skipped message = False 2024-03-04T09:20:31.544 INFO:tasks.cephadm: override: [mds] debug mds = 20 2024-03-04T09:20:31.544 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2024-03-04T09:20:31.545 INFO:tasks.cephadm: override: [mds] debug ms = 1 2024-03-04T09:20:31.545 INFO:tasks.cephadm: override: [mds] mds bal fragment size max = 10000 2024-03-04T09:20:31.545 INFO:tasks.cephadm: override: [mds] mds bal merge size = 5 2024-03-04T09:20:31.545 INFO:tasks.cephadm: override: [mds] mds bal split bits = 3 2024-03-04T09:20:31.545 INFO:tasks.cephadm: override: [mds] mds bal split size = 100 2024-03-04T09:20:31.545 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2024-03-04T09:20:31.545 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2024-03-04T09:20:31.545 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2024-03-04T09:20:31.545 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2024-03-04T09:20:31.545 INFO:tasks.cephadm: override: [mds] mds_export_ephemeral_random_max = 0.1 2024-03-04T09:20:31.545 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2024-03-04T09:20:31.545 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2024-03-04T09:20:31.545 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2024-03-04T09:20:31.545 INFO:tasks.cephadm: override: [mgr] cephsqlite lock renewal timeout = 900000 2024-03-04T09:20:31.546 INFO:tasks.cephadm: override: [mgr] debug cephsqlite = 20 2024-03-04T09:20:31.546 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-03-04T09:20:31.546 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-03-04T09:20:31.546 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-03-04T09:20:31.546 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-03-04T09:20:31.546 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-03-04T09:20:31.546 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2024-03-04T09:20:31.546 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-03-04T09:20:31.546 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-03-04T09:20:31.546 INFO:tasks.cephadm: override: [osd] debug bluefs = 20 2024-03-04T09:20:31.546 INFO:tasks.cephadm: override: [osd] debug bluestore = 20 2024-03-04T09:20:31.546 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-03-04T09:20:31.546 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-03-04T09:20:31.546 INFO:tasks.cephadm: override: [osd] debug rocksdb = 10 2024-03-04T09:20:31.547 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-03-04T09:20:31.547 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-03-04T09:20:31.547 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-03-04T09:20:31.547 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-03-04T09:20:31.547 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-03-04T09:20:31.547 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2024-03-04T09:20:31.547 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-03-04T09:20:31.547 INFO:tasks.cephadm: override: [osd] osd_max_omap_entries_per_request = 10 2024-03-04T09:20:31.548 DEBUG:teuthology.orchestra.run.smithi052:> set -ex 2024-03-04T09:20:31.548 DEBUG:teuthology.orchestra.run.smithi052:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-03-04T09:20:31.566 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = 4e059f08-da08-11ee-95c6-87774f69a715 ms die on skipped message = False [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true bluestore block size = 96636764160 bluestore fsck on mount = True debug bluefs = 20 debug bluestore = 20 debug ms = 1 debug osd = 20 debug rocksdb = 10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd op complaint time = 180 osd_max_omap_entries_per_request = 10 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false cephsqlite lock renewal timeout = 900000 debug cephsqlite = 20 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon op complaint time = 120 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds bal fragment size max = 10000 mds bal merge size = 5 mds bal split bits = 3 mds bal split size = 100 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True mds_export_ephemeral_random_max = 0.1 osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2024-03-04T09:20:31.567 DEBUG:teuthology.orchestra.run.smithi052:mon.a> sudo journalctl -f -n 0 -u ceph-4e059f08-da08-11ee-95c6-87774f69a715@mon.a.service 2024-03-04T09:20:31.609 DEBUG:teuthology.orchestra.run.smithi052:mgr.x> sudo journalctl -f -n 0 -u ceph-4e059f08-da08-11ee-95c6-87774f69a715@mgr.x.service 2024-03-04T09:20:31.652 INFO:tasks.cephadm:Bootstrapping... 2024-03-04T09:20:31.652 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd -v bootstrap --fsid 4e059f08-da08-11ee-95c6-87774f69a715 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.52 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-03-04T09:20:31.850 INFO:teuthology.orchestra.run.smithi052.stdout:-------------------------------------------------------------------------------- 2024-03-04T09:20:31.851 INFO:teuthology.orchestra.run.smithi052.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd', '-v', 'bootstrap', '--fsid', '4e059f08-da08-11ee-95c6-87774f69a715', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.52', '--skip-admin-label'] 2024-03-04T09:20:31.879 INFO:teuthology.orchestra.run.smithi052.stdout:/bin/podman: stdout 4.9.3-dev 2024-03-04T09:20:31.880 INFO:teuthology.orchestra.run.smithi052.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-03-04T09:20:31.880 INFO:teuthology.orchestra.run.smithi052.stdout:Verifying podman|docker is present... 2024-03-04T09:20:31.907 INFO:teuthology.orchestra.run.smithi052.stdout:/bin/podman: stdout 4.9.3-dev 2024-03-04T09:20:31.907 INFO:teuthology.orchestra.run.smithi052.stdout:Verifying lvm2 is present... 2024-03-04T09:20:31.908 INFO:teuthology.orchestra.run.smithi052.stdout:Verifying time synchronization is in place... 2024-03-04T09:20:31.916 INFO:teuthology.orchestra.run.smithi052.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-03-04T09:20:31.916 INFO:teuthology.orchestra.run.smithi052.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-03-04T09:20:31.922 INFO:teuthology.orchestra.run.smithi052.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-03-04T09:20:31.922 INFO:teuthology.orchestra.run.smithi052.stdout:systemctl: stdout inactive 2024-03-04T09:20:31.930 INFO:teuthology.orchestra.run.smithi052.stdout:systemctl: stdout enabled 2024-03-04T09:20:31.936 INFO:teuthology.orchestra.run.smithi052.stdout:systemctl: stdout active 2024-03-04T09:20:31.936 INFO:teuthology.orchestra.run.smithi052.stdout:Unit chronyd.service is enabled and running 2024-03-04T09:20:31.936 INFO:teuthology.orchestra.run.smithi052.stdout:Repeating the final host check... 2024-03-04T09:20:31.960 INFO:teuthology.orchestra.run.smithi052.stdout:/bin/podman: stdout 4.9.3-dev 2024-03-04T09:20:31.960 INFO:teuthology.orchestra.run.smithi052.stdout:podman (/bin/podman) version 4.9.3 is present 2024-03-04T09:20:31.960 INFO:teuthology.orchestra.run.smithi052.stdout:systemctl is present 2024-03-04T09:20:31.960 INFO:teuthology.orchestra.run.smithi052.stdout:lvcreate is present 2024-03-04T09:20:31.966 INFO:teuthology.orchestra.run.smithi052.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-03-04T09:20:31.967 INFO:teuthology.orchestra.run.smithi052.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-03-04T09:20:31.974 INFO:teuthology.orchestra.run.smithi052.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-03-04T09:20:31.974 INFO:teuthology.orchestra.run.smithi052.stdout:systemctl: stdout inactive 2024-03-04T09:20:31.981 INFO:teuthology.orchestra.run.smithi052.stdout:systemctl: stdout enabled 2024-03-04T09:20:31.988 INFO:teuthology.orchestra.run.smithi052.stdout:systemctl: stdout active 2024-03-04T09:20:31.988 INFO:teuthology.orchestra.run.smithi052.stdout:Unit chronyd.service is enabled and running 2024-03-04T09:20:31.988 INFO:teuthology.orchestra.run.smithi052.stdout:Host looks OK 2024-03-04T09:20:31.988 INFO:teuthology.orchestra.run.smithi052.stdout:Cluster fsid: 4e059f08-da08-11ee-95c6-87774f69a715 2024-03-04T09:20:31.988 INFO:teuthology.orchestra.run.smithi052.stdout:Acquiring lock 140135722344400 on /run/cephadm/4e059f08-da08-11ee-95c6-87774f69a715.lock 2024-03-04T09:20:31.989 INFO:teuthology.orchestra.run.smithi052.stdout:Lock 140135722344400 acquired on /run/cephadm/4e059f08-da08-11ee-95c6-87774f69a715.lock 2024-03-04T09:20:31.989 INFO:teuthology.orchestra.run.smithi052.stdout:Verifying IP 172.21.15.52 port 3300 ... 2024-03-04T09:20:31.990 INFO:teuthology.orchestra.run.smithi052.stdout:Verifying IP 172.21.15.52 port 6789 ... 2024-03-04T09:20:31.990 INFO:teuthology.orchestra.run.smithi052.stdout:Base mon IP(s) is [172.21.15.52:3300, 172.21.15.52:6789], mon addrv is [v2:172.21.15.52:3300,v1:172.21.15.52:6789] 2024-03-04T09:20:31.994 INFO:teuthology.orchestra.run.smithi052.stdout:/sbin/ip: stdout default via 172.21.15.254 dev ens1f0 proto dhcp src 172.21.15.52 metric 100 2024-03-04T09:20:31.994 INFO:teuthology.orchestra.run.smithi052.stdout:/sbin/ip: stdout 172.21.0.0/20 dev ens1f0 proto kernel scope link src 172.21.15.52 metric 100 2024-03-04T09:20:31.997 INFO:teuthology.orchestra.run.smithi052.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-03-04T09:20:31.997 INFO:teuthology.orchestra.run.smithi052.stdout:/sbin/ip: stdout fe80::/64 dev ens1f0 proto kernel metric 1024 pref medium 2024-03-04T09:20:31.998 INFO:teuthology.orchestra.run.smithi052.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev ens1f0 proto ra metric 100 pref medium 2024-03-04T09:20:32.000 INFO:teuthology.orchestra.run.smithi052.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-03-04T09:20:32.001 INFO:teuthology.orchestra.run.smithi052.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-03-04T09:20:32.001 INFO:teuthology.orchestra.run.smithi052.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-03-04T09:20:32.001 INFO:teuthology.orchestra.run.smithi052.stdout:/sbin/ip: stdout 4: ens1f0: mtu 1500 state UP qlen 1000 2024-03-04T09:20:32.001 INFO:teuthology.orchestra.run.smithi052.stdout:/sbin/ip: stdout inet6 fe80::ec4:7aff:febd:15de/64 scope link noprefixroute 2024-03-04T09:20:32.001 INFO:teuthology.orchestra.run.smithi052.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-03-04T09:20:32.002 INFO:teuthology.orchestra.run.smithi052.stdout:Mon IP `172.21.15.52` is in CIDR network `172.21.0.0/20` 2024-03-04T09:20:32.002 INFO:teuthology.orchestra.run.smithi052.stdout:Mon IP `172.21.15.52` is in CIDR network `172.21.0.0/20` 2024-03-04T09:20:32.002 INFO:teuthology.orchestra.run.smithi052.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-03-04T09:20:32.003 INFO:teuthology.orchestra.run.smithi052.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-03-04T09:20:32.005 INFO:teuthology.orchestra.run.smithi052.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd... 2024-03-04T09:20:34.805 INFO:teuthology.orchestra.run.smithi052.stdout:/bin/podman: stdout f88ffb32aa6a7d5dfdc954d9f096f0c9cc47981d05ec8d117ba859a22d8185ba 2024-03-04T09:20:34.806 INFO:teuthology.orchestra.run.smithi052.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd... 2024-03-04T09:20:34.806 INFO:teuthology.orchestra.run.smithi052.stdout:/bin/podman: stderr Getting image source signatures 2024-03-04T09:20:34.806 INFO:teuthology.orchestra.run.smithi052.stdout:/bin/podman: stderr Copying blob sha256:5d74734bfa4d1f8e0a72f8c221d93ca0d3a68fb40764a44763479cacf5cffcb4 2024-03-04T09:20:34.806 INFO:teuthology.orchestra.run.smithi052.stdout:/bin/podman: stderr Copying blob sha256:7feca07754707458c3945cf0062cf4dabc512f6d90fe1a9a1370b362b6011124 2024-03-04T09:20:34.806 INFO:teuthology.orchestra.run.smithi052.stdout:/bin/podman: stderr Copying config sha256:f88ffb32aa6a7d5dfdc954d9f096f0c9cc47981d05ec8d117ba859a22d8185ba 2024-03-04T09:20:34.806 INFO:teuthology.orchestra.run.smithi052.stdout:/bin/podman: stderr Writing manifest to image destination 2024-03-04T09:20:38.504 INFO:teuthology.orchestra.run.smithi052.stdout:ceph: stdout ceph version 19.0.0-1739-g1bc250c7 (1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd) squid (dev) 2024-03-04T09:20:38.504 INFO:teuthology.orchestra.run.smithi052.stdout:Ceph version: ceph version 19.0.0-1739-g1bc250c7 (1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd) squid (dev) 2024-03-04T09:20:38.504 INFO:teuthology.orchestra.run.smithi052.stdout:Extracting ceph user uid/gid from container image... 2024-03-04T09:20:39.476 INFO:teuthology.orchestra.run.smithi052.stdout:stat: stdout 167 167 2024-03-04T09:20:39.476 INFO:teuthology.orchestra.run.smithi052.stdout:Creating initial keys... 2024-03-04T09:20:40.426 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph-authtool: stdout AQBnkuVlOwWJMRAAbaumcN/rga6VSCCu8b94AA== 2024-03-04T09:20:41.323 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph-authtool: stdout AQBokuVlz/oJLBAAECYbkJneFSUo6wdFSS5zkw== 2024-03-04T09:20:42.329 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph-authtool: stdout AQBpkuVlzgvrJxAAiJxVEkvgUMRXNsqsTKbjGg== 2024-03-04T09:20:42.330 INFO:teuthology.orchestra.run.smithi052.stdout:Creating initial monmap... 2024-03-04T09:20:43.678 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-03-04T09:20:43.678 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = quincy 2024-03-04T09:20:43.678 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to 4e059f08-da08-11ee-95c6-87774f69a715 2024-03-04T09:20:43.678 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-03-04T09:20:43.678 INFO:teuthology.orchestra.run.smithi052.stdout:monmaptool for a [v2:172.21.15.52:3300,v1:172.21.15.52:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-03-04T09:20:43.678 INFO:teuthology.orchestra.run.smithi052.stdout:setting min_mon_release = quincy 2024-03-04T09:20:43.678 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/monmaptool: set fsid to 4e059f08-da08-11ee-95c6-87774f69a715 2024-03-04T09:20:43.678 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-03-04T09:20:43.678 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-03-04T09:20:43.679 INFO:teuthology.orchestra.run.smithi052.stdout:Creating mon... 2024-03-04T09:20:44.691 INFO:teuthology.orchestra.run.smithi052.stdout:create mon.a on 2024-03-04T09:20:44.890 INFO:teuthology.orchestra.run.smithi052.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-03-04T09:20:45.062 INFO:teuthology.orchestra.run.smithi052.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-03-04T09:20:45.232 INFO:teuthology.orchestra.run.smithi052.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-4e059f08-da08-11ee-95c6-87774f69a715.target → /etc/systemd/system/ceph-4e059f08-da08-11ee-95c6-87774f69a715.target. 2024-03-04T09:20:45.232 INFO:teuthology.orchestra.run.smithi052.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-4e059f08-da08-11ee-95c6-87774f69a715.target → /etc/systemd/system/ceph-4e059f08-da08-11ee-95c6-87774f69a715.target. 2024-03-04T09:20:45.465 INFO:teuthology.orchestra.run.smithi052.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-4e059f08-da08-11ee-95c6-87774f69a715@mon.a 2024-03-04T09:20:45.465 INFO:teuthology.orchestra.run.smithi052.stdout:systemctl: stderr Failed to reset failed state of unit ceph-4e059f08-da08-11ee-95c6-87774f69a715@mon.a.service: Unit ceph-4e059f08-da08-11ee-95c6-87774f69a715@mon.a.service not loaded. 2024-03-04T09:20:45.625 INFO:teuthology.orchestra.run.smithi052.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-4e059f08-da08-11ee-95c6-87774f69a715.target.wants/ceph-4e059f08-da08-11ee-95c6-87774f69a715@mon.a.service → /etc/systemd/system/ceph-4e059f08-da08-11ee-95c6-87774f69a715@.service. 2024-03-04T09:20:46.132 INFO:teuthology.orchestra.run.smithi052.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-03-04T09:20:46.132 INFO:teuthology.orchestra.run.smithi052.stdout:systemctl: stdout disabled 2024-03-04T09:20:46.138 INFO:teuthology.orchestra.run.smithi052.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-03-04T09:20:46.138 INFO:teuthology.orchestra.run.smithi052.stdout:systemctl: stdout inactive 2024-03-04T09:20:46.139 INFO:teuthology.orchestra.run.smithi052.stdout:firewalld.service is not enabled 2024-03-04T09:20:46.139 INFO:teuthology.orchestra.run.smithi052.stdout:Not possible to enable service . firewalld.service is not available 2024-03-04T09:20:46.139 INFO:teuthology.orchestra.run.smithi052.stdout:Waiting for mon to start... 2024-03-04T09:20:46.139 INFO:teuthology.orchestra.run.smithi052.stdout:Waiting for mon... 2024-03-04T09:20:47.153 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout cluster: 2024-03-04T09:20:47.153 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout id: 4e059f08-da08-11ee-95c6-87774f69a715 2024-03-04T09:20:47.153 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-03-04T09:20:47.153 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout 2024-03-04T09:20:47.154 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout services: 2024-03-04T09:20:47.154 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.543129s) 2024-03-04T09:20:47.154 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-03-04T09:20:47.154 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-03-04T09:20:47.154 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout 2024-03-04T09:20:47.154 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout data: 2024-03-04T09:20:47.154 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-03-04T09:20:47.154 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-03-04T09:20:47.154 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-03-04T09:20:47.154 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout pgs: 2024-03-04T09:20:47.154 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout 2024-03-04T09:20:47.154 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:20:46.566+0000 7ff15496b700 1 Processor -- start 2024-03-04T09:20:47.154 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:20:46.566+0000 7ff15496b700 1 -- start start 2024-03-04T09:20:47.154 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:20:46.567+0000 7ff15496b700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff150106920 0x7ff150106d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:20:47.154 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:20:46.567+0000 7ff15496b700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff150107310 con 0x7ff150106920 2024-03-04T09:20:47.155 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:20:46.567+0000 7ff14e59c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff150106920 0x7ff150106d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:20:47.155 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:20:46.567+0000 7ff14e59c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff150106920 0x7ff150106d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:55456/0 (socket says 172.21.15.52:55456) 2024-03-04T09:20:47.155 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:20:46.567+0000 7ff14e59c700 1 -- 172.21.15.52:0/3578655425 learned_addr learned my addr 172.21.15.52:0/3578655425 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-04T09:20:47.155 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:20:46.567+0000 7ff14e59c700 1 -- 172.21.15.52:0/3578655425 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff150107b20 con 0x7ff150106920 2024-03-04T09:20:47.155 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:20:46.568+0000 7ff14e59c700 1 --2- 172.21.15.52:0/3578655425 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff150106920 0x7ff150106d40 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7ff1400098a0 tx=0x7ff140009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27483d96d2667c98 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:20:47.155 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:20:46.568+0000 7ff14d59a700 1 -- 172.21.15.52:0/3578655425 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff140004160 con 0x7ff150106920 2024-03-04T09:20:47.155 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:20:46.568+0000 7ff14d59a700 1 -- 172.21.15.52:0/3578655425 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff140004300 con 0x7ff150106920 2024-03-04T09:20:47.155 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:20:46.568+0000 7ff14d59a700 1 -- 172.21.15.52:0/3578655425 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff1400036a0 con 0x7ff150106920 2024-03-04T09:20:47.155 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:20:46.568+0000 7ff15496b700 1 -- 172.21.15.52:0/3578655425 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff150106920 msgr2=0x7ff150106d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:20:47.155 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:20:46.568+0000 7ff15496b700 1 --2- 172.21.15.52:0/3578655425 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff150106920 0x7ff150106d40 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7ff1400098a0 tx=0x7ff140009bb0 comp rx=0 tx=0).stop 2024-03-04T09:20:47.155 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:20:46.568+0000 7ff15496b700 1 -- 172.21.15.52:0/3578655425 shutdown_connections 2024-03-04T09:20:47.155 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:20:46.568+0000 7ff15496b700 1 --2- 172.21.15.52:0/3578655425 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff150106920 0x7ff150106d40 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:20:47.155 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:20:46.569+0000 7ff15496b700 1 -- 172.21.15.52:0/3578655425 >> 172.21.15.52:0/3578655425 conn(0x7ff150101fc0 msgr2=0x7ff1501043e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:20:47.155 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:20:46.569+0000 7ff15496b700 1 -- 172.21.15.52:0/3578655425 shutdown_connections 2024-03-04T09:20:47.156 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:20:46.569+0000 7ff15496b700 1 -- 172.21.15.52:0/3578655425 wait complete. 2024-03-04T09:20:47.156 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:20:46.569+0000 7ff15496b700 1 Processor -- start 2024-03-04T09:20:47.156 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:20:46.569+0000 7ff15496b700 1 -- start start 2024-03-04T09:20:47.156 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:20:46.569+0000 7ff15496b700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff150106920 0x7ff150199fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:20:47.156 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:20:46.569+0000 7ff15496b700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff150108050 con 0x7ff150106920 2024-03-04T09:20:47.156 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:20:46.570+0000 7ff14e59c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff150106920 0x7ff150199fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:20:47.156 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:20:46.570+0000 7ff14e59c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff150106920 0x7ff150199fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:55460/0 (socket says 172.21.15.52:55460) 2024-03-04T09:20:47.156 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:20:46.570+0000 7ff14e59c700 1 -- 172.21.15.52:0/2931024635 learned_addr learned my addr 172.21.15.52:0/2931024635 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-04T09:20:47.156 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:20:46.570+0000 7ff14e59c700 1 -- 172.21.15.52:0/2931024635 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff15019a500 con 0x7ff150106920 2024-03-04T09:20:47.156 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:20:46.570+0000 7ff14e59c700 1 --2- 172.21.15.52:0/2931024635 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff150106920 0x7ff150199fc0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7ff140000c00 tx=0x7ff140004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:20:47.156 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:20:46.570+0000 7ff13f7fe700 1 -- 172.21.15.52:0/2931024635 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff140003db0 con 0x7ff150106920 2024-03-04T09:20:47.156 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:20:46.570+0000 7ff13f7fe700 1 -- 172.21.15.52:0/2931024635 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff140011630 con 0x7ff150106920 2024-03-04T09:20:47.156 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:20:46.570+0000 7ff15496b700 1 -- 172.21.15.52:0/2931024635 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff15019a790 con 0x7ff150106920 2024-03-04T09:20:47.157 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:20:46.570+0000 7ff13f7fe700 1 -- 172.21.15.52:0/2931024635 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff140011950 con 0x7ff150106920 2024-03-04T09:20:47.157 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:20:46.570+0000 7ff15496b700 1 -- 172.21.15.52:0/2931024635 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff15019d480 con 0x7ff150106920 2024-03-04T09:20:47.157 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:20:46.570+0000 7ff13f7fe700 1 -- 172.21.15.52:0/2931024635 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7ff14001a460 con 0x7ff150106920 2024-03-04T09:20:47.157 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:20:46.571+0000 7ff13f7fe700 1 -- 172.21.15.52:0/2931024635 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7ff14001b780 con 0x7ff150106920 2024-03-04T09:20:47.157 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:20:46.571+0000 7ff15496b700 1 -- 172.21.15.52:0/2931024635 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff150069170 con 0x7ff150106920 2024-03-04T09:20:47.157 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:20:46.574+0000 7ff13f7fe700 1 -- 172.21.15.52:0/2931024635 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7ff14001b980 con 0x7ff150106920 2024-03-04T09:20:47.157 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:20:46.646+0000 7ff15496b700 1 -- 172.21.15.52:0/2931024635 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "status"} v 0) v1 -- 0x7ff15002dc00 con 0x7ff150106920 2024-03-04T09:20:47.157 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:20:46.646+0000 7ff13f7fe700 1 -- 172.21.15.52:0/2931024635 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) v1 ==== 54+0+317 (secure 0 0 0) 0x7ff14001bb60 con 0x7ff150106920 2024-03-04T09:20:47.157 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:20:46.647+0000 7ff15496b700 1 -- 172.21.15.52:0/2931024635 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff150106920 msgr2=0x7ff150199fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:20:47.157 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:20:46.647+0000 7ff15496b700 1 --2- 172.21.15.52:0/2931024635 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff150106920 0x7ff150199fc0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7ff140000c00 tx=0x7ff140004750 comp rx=0 tx=0).stop 2024-03-04T09:20:47.157 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:20:46.647+0000 7ff15496b700 1 -- 172.21.15.52:0/2931024635 shutdown_connections 2024-03-04T09:20:47.157 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:20:46.647+0000 7ff15496b700 1 --2- 172.21.15.52:0/2931024635 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff150106920 0x7ff150199fc0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:20:47.157 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:20:46.647+0000 7ff15496b700 1 -- 172.21.15.52:0/2931024635 >> 172.21.15.52:0/2931024635 conn(0x7ff150101fc0 msgr2=0x7ff150190c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:20:47.157 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:20:46.647+0000 7ff15496b700 1 -- 172.21.15.52:0/2931024635 shutdown_connections 2024-03-04T09:20:47.158 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:20:46.647+0000 7ff15496b700 1 -- 172.21.15.52:0/2931024635 wait complete. 2024-03-04T09:20:47.158 INFO:teuthology.orchestra.run.smithi052.stdout:mon is available 2024-03-04T09:20:47.158 INFO:teuthology.orchestra.run.smithi052.stdout:Assimilating anything we can from ceph.conf... 2024-03-04T09:20:48.150 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout 2024-03-04T09:20:48.150 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout [global] 2024-03-04T09:20:48.150 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout fsid = 4e059f08-da08-11ee-95c6-87774f69a715 2024-03-04T09:20:48.150 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.52:3300,v1:172.21.15.52:6789] 2024-03-04T09:20:48.151 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-03-04T09:20:48.151 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-03-04T09:20:48.151 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-03-04T09:20:48.151 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-03-04T09:20:48.151 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout 2024-03-04T09:20:48.151 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout [mgr] 2024-03-04T09:20:48.151 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-03-04T09:20:48.151 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout 2024-03-04T09:20:48.151 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout [osd] 2024-03-04T09:20:48.151 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-03-04T09:20:48.151 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-03-04T09:20:48.151 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-03-04T09:20:48.152 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-03-04T09:20:48.152 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-03-04T09:20:48.152 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:20:47.583+0000 7f153ba2f700 1 Processor -- start 2024-03-04T09:20:48.152 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:20:47.583+0000 7f153ba2f700 1 -- start start 2024-03-04T09:20:48.152 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:20:47.583+0000 7f153ba2f700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1534106280 0x7f15341066a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:20:48.152 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:20:47.583+0000 7f153ba2f700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f153407ae60 con 0x7f1534106280 2024-03-04T09:20:48.152 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:20:47.583+0000 7f15397cb700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1534106280 0x7f15341066a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:20:48.152 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:20:47.583+0000 7f15397cb700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1534106280 0x7f15341066a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:55472/0 (socket says 172.21.15.52:55472) 2024-03-04T09:20:48.152 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:20:47.583+0000 7f15397cb700 1 -- 172.21.15.52:0/1043544052 learned_addr learned my addr 172.21.15.52:0/1043544052 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-04T09:20:48.152 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:20:47.583+0000 7f15397cb700 1 -- 172.21.15.52:0/1043544052 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1534106be0 con 0x7f1534106280 2024-03-04T09:20:48.152 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:20:47.584+0000 7f15397cb700 1 --2- 172.21.15.52:0/1043544052 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1534106280 0x7f15341066a0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f15300098a0 tx=0x7f1530009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=308b26993cfa2851 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:20:48.152 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:20:47.584+0000 7f1527fff700 1 -- 172.21.15.52:0/1043544052 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1530004160 con 0x7f1534106280 2024-03-04T09:20:48.153 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:20:47.584+0000 7f1527fff700 1 -- 172.21.15.52:0/1043544052 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1530004300 con 0x7f1534106280 2024-03-04T09:20:48.153 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:20:47.584+0000 7f1527fff700 1 -- 172.21.15.52:0/1043544052 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f15300036a0 con 0x7f1534106280 2024-03-04T09:20:48.153 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:20:47.584+0000 7f153ba2f700 1 -- 172.21.15.52:0/1043544052 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1534106280 msgr2=0x7f15341066a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:20:48.153 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:20:47.584+0000 7f153ba2f700 1 --2- 172.21.15.52:0/1043544052 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1534106280 0x7f15341066a0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f15300098a0 tx=0x7f1530009bb0 comp rx=0 tx=0).stop 2024-03-04T09:20:48.153 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:20:47.584+0000 7f153ba2f700 1 -- 172.21.15.52:0/1043544052 shutdown_connections 2024-03-04T09:20:48.153 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:20:47.584+0000 7f153ba2f700 1 --2- 172.21.15.52:0/1043544052 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1534106280 0x7f15341066a0 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:20:48.153 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:20:47.584+0000 7f153ba2f700 1 -- 172.21.15.52:0/1043544052 >> 172.21.15.52:0/1043544052 conn(0x7f1534101f20 msgr2=0x7f1534104340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:20:48.153 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:20:47.584+0000 7f153ba2f700 1 -- 172.21.15.52:0/1043544052 shutdown_connections 2024-03-04T09:20:48.153 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:20:47.585+0000 7f153ba2f700 1 -- 172.21.15.52:0/1043544052 wait complete. 2024-03-04T09:20:48.153 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:20:47.585+0000 7f153ba2f700 1 Processor -- start 2024-03-04T09:20:48.153 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:20:47.585+0000 7f153ba2f700 1 -- start start 2024-03-04T09:20:48.154 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:20:47.585+0000 7f153ba2f700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1534106280 0x7f153419d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:20:48.154 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:20:47.585+0000 7f153ba2f700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1534107110 con 0x7f1534106280 2024-03-04T09:20:48.154 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:20:47.585+0000 7f15397cb700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1534106280 0x7f153419d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:20:48.154 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:20:47.585+0000 7f15397cb700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1534106280 0x7f153419d300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:55474/0 (socket says 172.21.15.52:55474) 2024-03-04T09:20:48.154 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:20:47.585+0000 7f15397cb700 1 -- 172.21.15.52:0/172050563 learned_addr learned my addr 172.21.15.52:0/172050563 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-04T09:20:48.154 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:20:47.586+0000 7f15397cb700 1 -- 172.21.15.52:0/172050563 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f153419d840 con 0x7f1534106280 2024-03-04T09:20:48.154 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:20:47.586+0000 7f15397cb700 1 --2- 172.21.15.52:0/172050563 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1534106280 0x7f153419d300 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f1530000c00 tx=0x7f1530004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:20:48.154 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:20:47.586+0000 7f15267fc700 1 -- 172.21.15.52:0/172050563 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1530003db0 con 0x7f1534106280 2024-03-04T09:20:48.154 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:20:47.586+0000 7f15267fc700 1 -- 172.21.15.52:0/172050563 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1530011630 con 0x7f1534106280 2024-03-04T09:20:48.154 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:20:47.586+0000 7f15267fc700 1 -- 172.21.15.52:0/172050563 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1530011950 con 0x7f1534106280 2024-03-04T09:20:48.154 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:20:47.586+0000 7f153ba2f700 1 -- 172.21.15.52:0/172050563 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f153419dad0 con 0x7f1534106280 2024-03-04T09:20:48.154 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:20:47.586+0000 7f153ba2f700 1 -- 172.21.15.52:0/172050563 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15341a07c0 con 0x7f1534106280 2024-03-04T09:20:48.154 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:20:47.586+0000 7f15267fc700 1 -- 172.21.15.52:0/172050563 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f153001a460 con 0x7f1534106280 2024-03-04T09:20:48.155 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:20:47.586+0000 7f15267fc700 1 -- 172.21.15.52:0/172050563 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f153001b780 con 0x7f1534106280 2024-03-04T09:20:48.155 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:20:47.587+0000 7f153ba2f700 1 -- 172.21.15.52:0/172050563 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1534069170 con 0x7f1534106280 2024-03-04T09:20:48.155 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:20:47.589+0000 7f15267fc700 1 -- 172.21.15.52:0/172050563 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f153001ba20 con 0x7f1534106280 2024-03-04T09:20:48.155 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:20:47.660+0000 7f153ba2f700 1 -- 172.21.15.52:0/172050563 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) v1 -- 0x7f15341a0990 con 0x7f1534106280 2024-03-04T09:20:48.155 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:20:47.662+0000 7f15267fc700 1 -- 172.21.15.52:0/172050563 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) v1 ==== 70+0+429 (secure 0 0 0) 0x7f153001bc00 con 0x7f1534106280 2024-03-04T09:20:48.155 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:20:47.663+0000 7f153ba2f700 1 -- 172.21.15.52:0/172050563 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1534106280 msgr2=0x7f153419d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:20:48.155 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:20:47.663+0000 7f153ba2f700 1 --2- 172.21.15.52:0/172050563 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1534106280 0x7f153419d300 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f1530000c00 tx=0x7f1530004750 comp rx=0 tx=0).stop 2024-03-04T09:20:48.155 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:20:47.663+0000 7f153ba2f700 1 -- 172.21.15.52:0/172050563 shutdown_connections 2024-03-04T09:20:48.155 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:20:47.663+0000 7f153ba2f700 1 --2- 172.21.15.52:0/172050563 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1534106280 0x7f153419d300 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:20:48.155 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:20:47.663+0000 7f153ba2f700 1 -- 172.21.15.52:0/172050563 >> 172.21.15.52:0/172050563 conn(0x7f1534101f20 msgr2=0x7f1534193920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:20:48.155 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:20:47.663+0000 7f153ba2f700 1 -- 172.21.15.52:0/172050563 shutdown_connections 2024-03-04T09:20:48.156 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:20:47.663+0000 7f153ba2f700 1 -- 172.21.15.52:0/172050563 wait complete. 2024-03-04T09:20:48.156 INFO:teuthology.orchestra.run.smithi052.stdout:Generating new minimal ceph.conf... 2024-03-04T09:20:49.188 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:20:49 smithi052 ceph-mon[32103]: from='client.? 172.21.15.52:0/1316413341' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T09:20:49.195 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:20:48.554+0000 7f6a93f2a700 1 Processor -- start 2024-03-04T09:20:49.195 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:20:48.555+0000 7f6a93f2a700 1 -- start start 2024-03-04T09:20:49.195 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:20:48.555+0000 7f6a93f2a700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6a8c105cd0 0x7f6a8c1060f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:20:49.195 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:20:48.555+0000 7f6a93f2a700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a8c1066c0 con 0x7f6a8c105cd0 2024-03-04T09:20:49.195 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:20:48.555+0000 7f6a91cc6700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6a8c105cd0 0x7f6a8c1060f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:20:49.196 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:20:48.555+0000 7f6a91cc6700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6a8c105cd0 0x7f6a8c1060f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:55486/0 (socket says 172.21.15.52:55486) 2024-03-04T09:20:49.196 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:20:48.555+0000 7f6a91cc6700 1 -- 172.21.15.52:0/2827746403 learned_addr learned my addr 172.21.15.52:0/2827746403 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-04T09:20:49.196 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:20:48.555+0000 7f6a91cc6700 1 -- 172.21.15.52:0/2827746403 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a8c108c60 con 0x7f6a8c105cd0 2024-03-04T09:20:49.196 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:20:48.556+0000 7f6a91cc6700 1 --2- 172.21.15.52:0/2827746403 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6a8c105cd0 0x7f6a8c1060f0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f6a7c0098a0 tx=0x7f6a7c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4df6774aa84af19a server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:20:49.196 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:20:48.556+0000 7f6a90cc4700 1 -- 172.21.15.52:0/2827746403 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6a7c004160 con 0x7f6a8c105cd0 2024-03-04T09:20:49.196 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:20:48.556+0000 7f6a90cc4700 1 -- 172.21.15.52:0/2827746403 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6a7c004300 con 0x7f6a8c105cd0 2024-03-04T09:20:49.196 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:20:48.556+0000 7f6a90cc4700 1 -- 172.21.15.52:0/2827746403 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6a7c011b50 con 0x7f6a8c105cd0 2024-03-04T09:20:49.196 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:20:48.556+0000 7f6a93f2a700 1 -- 172.21.15.52:0/2827746403 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6a8c105cd0 msgr2=0x7f6a8c1060f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:20:49.196 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:20:48.556+0000 7f6a93f2a700 1 --2- 172.21.15.52:0/2827746403 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6a8c105cd0 0x7f6a8c1060f0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f6a7c0098a0 tx=0x7f6a7c009bb0 comp rx=0 tx=0).stop 2024-03-04T09:20:49.196 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:20:48.556+0000 7f6a93f2a700 1 -- 172.21.15.52:0/2827746403 shutdown_connections 2024-03-04T09:20:49.196 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:20:48.556+0000 7f6a93f2a700 1 --2- 172.21.15.52:0/2827746403 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6a8c105cd0 0x7f6a8c1060f0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:20:49.196 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:20:48.556+0000 7f6a93f2a700 1 -- 172.21.15.52:0/2827746403 >> 172.21.15.52:0/2827746403 conn(0x7f6a8c101640 msgr2=0x7f6a8c103a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:20:49.197 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:20:48.557+0000 7f6a93f2a700 1 -- 172.21.15.52:0/2827746403 shutdown_connections 2024-03-04T09:20:49.197 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:20:48.557+0000 7f6a93f2a700 1 -- 172.21.15.52:0/2827746403 wait complete. 2024-03-04T09:20:49.197 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:20:48.557+0000 7f6a93f2a700 1 Processor -- start 2024-03-04T09:20:49.197 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:20:48.557+0000 7f6a93f2a700 1 -- start start 2024-03-04T09:20:49.197 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:20:48.557+0000 7f6a93f2a700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6a8c105cd0 0x7f6a8c19ba10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:20:49.197 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:20:48.557+0000 7f6a93f2a700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a8c10c010 con 0x7f6a8c105cd0 2024-03-04T09:20:49.197 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:20:48.557+0000 7f6a91cc6700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6a8c105cd0 0x7f6a8c19ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:20:49.197 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:20:48.557+0000 7f6a91cc6700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6a8c105cd0 0x7f6a8c19ba10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:55496/0 (socket says 172.21.15.52:55496) 2024-03-04T09:20:49.197 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:20:48.557+0000 7f6a91cc6700 1 -- 172.21.15.52:0/1316413341 learned_addr learned my addr 172.21.15.52:0/1316413341 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-04T09:20:49.197 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:20:48.558+0000 7f6a91cc6700 1 -- 172.21.15.52:0/1316413341 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a8c19bf50 con 0x7f6a8c105cd0 2024-03-04T09:20:49.197 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:20:48.558+0000 7f6a91cc6700 1 --2- 172.21.15.52:0/1316413341 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6a8c105cd0 0x7f6a8c19ba10 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f6a7c009fb0 tx=0x7f6a7c011bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:20:49.197 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:20:48.558+0000 7f6a82ffd700 1 -- 172.21.15.52:0/1316413341 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6a7c0038b0 con 0x7f6a8c105cd0 2024-03-04T09:20:49.197 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:20:48.558+0000 7f6a82ffd700 1 -- 172.21.15.52:0/1316413341 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6a7c011e50 con 0x7f6a8c105cd0 2024-03-04T09:20:49.198 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:20:48.558+0000 7f6a82ffd700 1 -- 172.21.15.52:0/1316413341 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6a7c019550 con 0x7f6a8c105cd0 2024-03-04T09:20:49.198 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:20:48.558+0000 7f6a93f2a700 1 -- 172.21.15.52:0/1316413341 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a8c19c1e0 con 0x7f6a8c105cd0 2024-03-04T09:20:49.198 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:20:48.558+0000 7f6a93f2a700 1 -- 172.21.15.52:0/1316413341 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a8c19eed0 con 0x7f6a8c105cd0 2024-03-04T09:20:49.198 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:20:48.558+0000 7f6a82ffd700 1 -- 172.21.15.52:0/1316413341 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f6a7c019be0 con 0x7f6a8c105cd0 2024-03-04T09:20:49.198 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:20:48.558+0000 7f6a82ffd700 1 -- 172.21.15.52:0/1316413341 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f6a7c024730 con 0x7f6a8c105cd0 2024-03-04T09:20:49.198 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:20:48.559+0000 7f6a93f2a700 1 -- 172.21.15.52:0/1316413341 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a8c069170 con 0x7f6a8c105cd0 2024-03-04T09:20:49.198 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:20:48.561+0000 7f6a82ffd700 1 -- 172.21.15.52:0/1316413341 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f6a7c003a50 con 0x7f6a8c105cd0 2024-03-04T09:20:49.198 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:20:48.632+0000 7f6a93f2a700 1 -- 172.21.15.52:0/1316413341 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) v1 -- 0x7f6a8c19f480 con 0x7f6a8c105cd0 2024-03-04T09:20:49.198 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:20:48.632+0000 7f6a82ffd700 1 -- 172.21.15.52:0/1316413341 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) v1 ==== 76+0+175 (secure 0 0 0) 0x7f6a7c0249d0 con 0x7f6a8c105cd0 2024-03-04T09:20:49.198 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:20:48.633+0000 7f6a93f2a700 1 -- 172.21.15.52:0/1316413341 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6a8c105cd0 msgr2=0x7f6a8c19ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:20:49.198 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:20:48.633+0000 7f6a93f2a700 1 --2- 172.21.15.52:0/1316413341 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6a8c105cd0 0x7f6a8c19ba10 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f6a7c009fb0 tx=0x7f6a7c011bf0 comp rx=0 tx=0).stop 2024-03-04T09:20:49.198 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:20:48.633+0000 7f6a93f2a700 1 -- 172.21.15.52:0/1316413341 shutdown_connections 2024-03-04T09:20:49.198 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:20:48.633+0000 7f6a93f2a700 1 --2- 172.21.15.52:0/1316413341 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6a8c105cd0 0x7f6a8c19ba10 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:20:49.199 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:20:48.633+0000 7f6a93f2a700 1 -- 172.21.15.52:0/1316413341 >> 172.21.15.52:0/1316413341 conn(0x7f6a8c101640 msgr2=0x7f6a8c102180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:20:49.199 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:20:48.633+0000 7f6a93f2a700 1 -- 172.21.15.52:0/1316413341 shutdown_connections 2024-03-04T09:20:49.199 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:20:48.633+0000 7f6a93f2a700 1 -- 172.21.15.52:0/1316413341 wait complete. 2024-03-04T09:20:49.199 INFO:teuthology.orchestra.run.smithi052.stdout:Restarting the monitor... 2024-03-04T09:20:49.443 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:20:49 smithi052 systemd[1]: Stopping Ceph mon.a for 4e059f08-da08-11ee-95c6-87774f69a715... 2024-03-04T09:20:49.801 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:20:49 smithi052 ceph-4e059f08-da08-11ee-95c6-87774f69a715-mon-a[32099]: 2024-03-04T09:20:49.445+0000 7f0c59b1f700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-03-04T09:20:49.801 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:20:49 smithi052 ceph-4e059f08-da08-11ee-95c6-87774f69a715-mon-a[32099]: 2024-03-04T09:20:49.445+0000 7f0c59b1f700 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-03-04T09:20:49.801 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:20:49 smithi052 podman[32346]: 2024-03-04 09:20:49.579201191 +0000 UTC m=+0.220121653 container died c2cd46c36c3fa3f30786dc8d6afcc1601ea7e32d7760abb614dc9540c1a474f7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd, name=ceph-4e059f08-da08-11ee-95c6-87774f69a715-mon-a, RELEASE=wip-lusov-qdb-1bc250c, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240222, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-03-04T09:20:50.084 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:20:49 smithi052 podman[32346]: 2024-03-04 09:20:49.887137743 +0000 UTC m=+0.528058214 container cleanup c2cd46c36c3fa3f30786dc8d6afcc1601ea7e32d7760abb614dc9540c1a474f7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd, name=ceph-4e059f08-da08-11ee-95c6-87774f69a715-mon-a, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-qdb-1bc250c, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240222, org.label-schema.schema-version=1.0) 2024-03-04T09:20:50.085 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:20:49 smithi052 bash[32346]: ceph-4e059f08-da08-11ee-95c6-87774f69a715-mon-a 2024-03-04T09:20:50.443 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:20:50 smithi052 podman[32360]: 2024-03-04 09:20:50.104849932 +0000 UTC m=+0.518946405 container remove c2cd46c36c3fa3f30786dc8d6afcc1601ea7e32d7760abb614dc9540c1a474f7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd, name=ceph-4e059f08-da08-11ee-95c6-87774f69a715-mon-a, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, org.label-schema.build-date=20240222, RELEASE=wip-lusov-qdb-1bc250c, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-03-04T09:20:50.443 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:20:50 smithi052 systemd[1]: ceph-4e059f08-da08-11ee-95c6-87774f69a715@mon.a.service: Deactivated successfully. 2024-03-04T09:20:50.443 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:20:50 smithi052 systemd[1]: Stopped Ceph mon.a for 4e059f08-da08-11ee-95c6-87774f69a715. 2024-03-04T09:20:50.443 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:20:50 smithi052 systemd[1]: Starting Ceph mon.a for 4e059f08-da08-11ee-95c6-87774f69a715... 2024-03-04T09:20:50.715 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:20:50 smithi052 podman[32454]: 2024-03-04 09:20:50.447306489 +0000 UTC m=+0.114971159 container create b840a3470a2b8f2bf6ea9a6f5ee27a387058780843cce4dbfba12d501c25b2b0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd, name=ceph-4e059f08-da08-11ee-95c6-87774f69a715-mon-a, maintainer=Guillaume Abrioux , ceph=True, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, org.label-schema.build-date=20240222, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, RELEASE=wip-lusov-qdb-1bc250c, GIT_CLEAN=True) 2024-03-04T09:20:50.715 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:20:50 smithi052 podman[32454]: 2024-03-04 09:20:50.35008993 +0000 UTC m=+0.017754597 image pull f88ffb32aa6a7d5dfdc954d9f096f0c9cc47981d05ec8d117ba859a22d8185ba quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd 2024-03-04T09:20:50.715 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:20:50 smithi052 podman[32454]: 2024-03-04 09:20:50.69164328 +0000 UTC m=+0.359307960 container init b840a3470a2b8f2bf6ea9a6f5ee27a387058780843cce4dbfba12d501c25b2b0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd, name=ceph-4e059f08-da08-11ee-95c6-87774f69a715-mon-a, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, RELEASE=wip-lusov-qdb-1bc250c, org.label-schema.build-date=20240222, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.33.5, GIT_CLEAN=True) 2024-03-04T09:20:50.715 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:20:50 smithi052 podman[32454]: 2024-03-04 09:20:50.694983724 +0000 UTC m=+0.362648391 container start b840a3470a2b8f2bf6ea9a6f5ee27a387058780843cce4dbfba12d501c25b2b0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd, name=ceph-4e059f08-da08-11ee-95c6-87774f69a715-mon-a, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, RELEASE=wip-lusov-qdb-1bc250c, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240222, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, GIT_CLEAN=True) 2024-03-04T09:20:50.750 INFO:teuthology.orchestra.run.smithi052.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-03-04T09:20:50.982 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:20:50 smithi052 ceph-mon[32473]: set uid:gid to 167:167 (ceph:ceph) 2024-03-04T09:20:50.982 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:20:50 smithi052 ceph-mon[32473]: ceph version 19.0.0-1739-g1bc250c7 (1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd) squid (dev), process ceph-mon, pid 2 2024-03-04T09:20:50.982 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:20:50 smithi052 ceph-mon[32473]: pidfile_write: ignore empty --pid-file 2024-03-04T09:20:50.983 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:20:50 smithi052 ceph-mon[32473]: load: jerasure load: lrc 2024-03-04T09:20:50.983 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:20:50 smithi052 ceph-mon[32473]: rocksdb: RocksDB version: 7.9.2 2024-03-04T09:20:50.983 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:20:50 smithi052 ceph-mon[32473]: rocksdb: Git sha 0 2024-03-04T09:20:50.983 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:20:50 smithi052 ceph-mon[32473]: rocksdb: Compile date 2024-03-03 23:16:35 2024-03-04T09:20:50.983 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:20:50 smithi052 ceph-mon[32473]: rocksdb: DB SUMMARY 2024-03-04T09:20:50.983 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:20:50 smithi052 ceph-mon[32473]: rocksdb: DB Session ID: 1YA4CCURFWDCIBO1PAN1 2024-03-04T09:20:50.983 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:20:50 smithi052 ceph-mon[32473]: rocksdb: CURRENT file: CURRENT 2024-03-04T09:20:50.983 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:20:50 smithi052 ceph-mon[32473]: rocksdb: IDENTITY file: IDENTITY 2024-03-04T09:20:50.983 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:20:50 smithi052 ceph-mon[32473]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-03-04T09:20:50.983 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:20:50 smithi052 ceph-mon[32473]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-03-04T09:20:50.983 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:20:50 smithi052 ceph-mon[32473]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 99732 ; 2024-03-04T09:20:50.983 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:20:50 smithi052 ceph-mon[32473]: rocksdb: Options.error_if_exists: 0 2024-03-04T09:20:50.983 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:20:50 smithi052 ceph-mon[32473]: rocksdb: Options.create_if_missing: 0 2024-03-04T09:20:50.984 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:20:50 smithi052 ceph-mon[32473]: rocksdb: Options.paranoid_checks: 1 2024-03-04T09:20:50.984 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:20:50 smithi052 ceph-mon[32473]: rocksdb: Options.flush_verify_memtable_count: 1 2024-03-04T09:20:50.984 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:20:50 smithi052 ceph-mon[32473]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-03-04T09:20:50.984 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:20:50 smithi052 ceph-mon[32473]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-03-04T09:20:50.984 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:20:50 smithi052 ceph-mon[32473]: rocksdb: Options.env: 0x560a4cd90b20 2024-03-04T09:20:50.984 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:20:50 smithi052 ceph-mon[32473]: rocksdb: Options.fs: PosixFileSystem 2024-03-04T09:20:50.984 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:20:50 smithi052 ceph-mon[32473]: rocksdb: Options.info_log: 0x560a4eeaf920 2024-03-04T09:20:50.984 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:20:50 smithi052 ceph-mon[32473]: rocksdb: Options.max_file_opening_threads: 16 2024-03-04T09:20:50.984 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:20:50 smithi052 ceph-mon[32473]: rocksdb: Options.statistics: (nil) 2024-03-04T09:20:50.984 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:20:50 smithi052 ceph-mon[32473]: rocksdb: Options.use_fsync: 0 2024-03-04T09:20:50.984 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:20:50 smithi052 ceph-mon[32473]: rocksdb: Options.max_log_file_size: 0 2024-03-04T09:20:50.984 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:20:50 smithi052 ceph-mon[32473]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-03-04T09:20:50.984 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:20:50 smithi052 ceph-mon[32473]: rocksdb: Options.log_file_time_to_roll: 0 2024-03-04T09:20:50.985 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:20:50 smithi052 ceph-mon[32473]: rocksdb: Options.keep_log_file_num: 1000 2024-03-04T09:20:50.985 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:20:50 smithi052 ceph-mon[32473]: rocksdb: Options.recycle_log_file_num: 0 2024-03-04T09:20:50.985 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:20:50 smithi052 ceph-mon[32473]: rocksdb: Options.allow_fallocate: 1 2024-03-04T09:20:50.985 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:20:50 smithi052 ceph-mon[32473]: rocksdb: Options.allow_mmap_reads: 0 2024-03-04T09:20:50.985 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:20:50 smithi052 ceph-mon[32473]: rocksdb: Options.allow_mmap_writes: 0 2024-03-04T09:20:50.985 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:20:50 smithi052 ceph-mon[32473]: rocksdb: Options.use_direct_reads: 0 2024-03-04T09:20:50.985 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:20:50 smithi052 ceph-mon[32473]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-03-04T09:20:50.985 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:20:50 smithi052 ceph-mon[32473]: rocksdb: Options.create_missing_column_families: 0 2024-03-04T09:20:50.985 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:20:50 smithi052 ceph-mon[32473]: rocksdb: Options.db_log_dir: 2024-03-04T09:20:50.985 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:20:50 smithi052 ceph-mon[32473]: rocksdb: Options.wal_dir: 2024-03-04T09:20:50.985 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:20:50 smithi052 ceph-mon[32473]: rocksdb: Options.table_cache_numshardbits: 6 2024-03-04T09:20:50.987 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:20:50 smithi052 ceph-mon[32473]: rocksdb: Options.WAL_ttl_seconds: 0 2024-03-04T09:20:50.987 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:20:50 smithi052 ceph-mon[32473]: rocksdb: Options.WAL_size_limit_MB: 0 2024-03-04T09:20:50.987 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:20:50 smithi052 ceph-mon[32473]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-03-04T09:20:50.987 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:20:50 smithi052 ceph-mon[32473]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-03-04T09:20:50.987 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:20:50 smithi052 ceph-mon[32473]: rocksdb: Options.is_fd_close_on_exec: 1 2024-03-04T09:20:50.987 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:20:50 smithi052 ceph-mon[32473]: rocksdb: Options.advise_random_on_open: 1 2024-03-04T09:20:50.987 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:20:50 smithi052 ceph-mon[32473]: rocksdb: Options.db_write_buffer_size: 0 2024-03-04T09:20:50.987 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:20:50 smithi052 ceph-mon[32473]: rocksdb: Options.write_buffer_manager: 0x560a4ee83400 2024-03-04T09:20:50.987 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:20:50 smithi052 ceph-mon[32473]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-03-04T09:20:50.987 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:20:50 smithi052 ceph-mon[32473]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-03-04T09:20:50.987 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:20:50 smithi052 ceph-mon[32473]: rocksdb: Options.use_adaptive_mutex: 0 2024-03-04T09:20:50.988 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:20:50 smithi052 ceph-mon[32473]: rocksdb: Options.rate_limiter: (nil) 2024-03-04T09:20:50.988 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:20:50 smithi052 ceph-mon[32473]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-03-04T09:20:50.988 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:20:50 smithi052 ceph-mon[32473]: rocksdb: Options.wal_recovery_mode: 2 2024-03-04T09:20:50.988 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:20:50 smithi052 ceph-mon[32473]: rocksdb: Options.enable_thread_tracking: 0 2024-03-04T09:20:50.988 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:20:50 smithi052 ceph-mon[32473]: rocksdb: Options.enable_pipelined_write: 0 2024-03-04T09:20:50.988 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:20:50 smithi052 ceph-mon[32473]: rocksdb: Options.unordered_write: 0 2024-03-04T09:20:50.988 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:20:50 smithi052 ceph-mon[32473]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-03-04T09:20:50.988 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:20:50 smithi052 ceph-mon[32473]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-03-04T09:20:50.988 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:20:50 smithi052 ceph-mon[32473]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-03-04T09:20:50.988 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:20:50 smithi052 ceph-mon[32473]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-03-04T09:20:50.988 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:20:50 smithi052 ceph-mon[32473]: rocksdb: Options.row_cache: None 2024-03-04T09:20:50.988 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:20:50 smithi052 ceph-mon[32473]: rocksdb: Options.wal_filter: None 2024-03-04T09:20:50.988 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:20:50 smithi052 ceph-mon[32473]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-03-04T09:20:50.989 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:20:50 smithi052 ceph-mon[32473]: rocksdb: Options.allow_ingest_behind: 0 2024-03-04T09:20:50.989 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:20:50 smithi052 ceph-mon[32473]: rocksdb: Options.two_write_queues: 0 2024-03-04T09:20:50.989 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:20:50 smithi052 ceph-mon[32473]: rocksdb: Options.manual_wal_flush: 0 2024-03-04T09:20:50.989 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:20:50 smithi052 ceph-mon[32473]: rocksdb: Options.wal_compression: 0 2024-03-04T09:20:50.989 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:20:50 smithi052 ceph-mon[32473]: rocksdb: Options.atomic_flush: 0 2024-03-04T09:20:50.989 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:20:50 smithi052 ceph-mon[32473]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-03-04T09:20:50.989 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:20:50 smithi052 ceph-mon[32473]: rocksdb: Options.persist_stats_to_disk: 0 2024-03-04T09:20:50.989 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:20:50 smithi052 ceph-mon[32473]: rocksdb: Options.write_dbid_to_manifest: 0 2024-03-04T09:20:50.989 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:20:50 smithi052 ceph-mon[32473]: rocksdb: Options.log_readahead_size: 0 2024-03-04T09:20:50.989 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:20:50 smithi052 ceph-mon[32473]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-03-04T09:20:50.989 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:20:50 smithi052 ceph-mon[32473]: rocksdb: Options.best_efforts_recovery: 0 2024-03-04T09:20:50.989 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:20:50 smithi052 ceph-mon[32473]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-03-04T09:20:50.990 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:20:50 smithi052 ceph-mon[32473]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-03-04T09:20:50.990 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:20:50 smithi052 ceph-mon[32473]: rocksdb: Options.allow_data_in_errors: 0 2024-03-04T09:20:50.990 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:20:50 smithi052 ceph-mon[32473]: rocksdb: Options.db_host_id: __hostname__ 2024-03-04T09:20:50.990 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:20:50 smithi052 ceph-mon[32473]: rocksdb: Options.enforce_single_del_contracts: true 2024-03-04T09:20:50.990 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:20:50 smithi052 ceph-mon[32473]: rocksdb: Options.max_background_jobs: 2 2024-03-04T09:20:50.990 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:20:50 smithi052 ceph-mon[32473]: rocksdb: Options.max_background_compactions: -1 2024-03-04T09:20:50.990 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:20:50 smithi052 ceph-mon[32473]: rocksdb: Options.max_subcompactions: 1 2024-03-04T09:20:50.990 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:20:50 smithi052 ceph-mon[32473]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-03-04T09:20:50.990 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:20:50 smithi052 ceph-mon[32473]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-03-04T09:20:50.990 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:20:50 smithi052 ceph-mon[32473]: rocksdb: Options.delayed_write_rate : 16777216 2024-03-04T09:20:50.990 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:20:50 smithi052 ceph-mon[32473]: rocksdb: Options.max_total_wal_size: 0 2024-03-04T09:20:50.990 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:20:50 smithi052 ceph-mon[32473]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-03-04T09:20:50.990 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:20:50 smithi052 ceph-mon[32473]: rocksdb: Options.stats_dump_period_sec: 600 2024-03-04T09:20:50.991 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:20:50 smithi052 ceph-mon[32473]: rocksdb: Options.stats_persist_period_sec: 600 2024-03-04T09:20:50.991 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:20:50 smithi052 ceph-mon[32473]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-03-04T09:20:50.991 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:20:50 smithi052 ceph-mon[32473]: rocksdb: Options.max_open_files: -1 2024-03-04T09:20:50.991 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:20:50 smithi052 ceph-mon[32473]: rocksdb: Options.bytes_per_sync: 0 2024-03-04T09:20:50.991 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:20:50 smithi052 ceph-mon[32473]: rocksdb: Options.wal_bytes_per_sync: 0 2024-03-04T09:20:50.991 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:20:50 smithi052 ceph-mon[32473]: rocksdb: Options.strict_bytes_per_sync: 0 2024-03-04T09:20:50.991 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:20:50 smithi052 ceph-mon[32473]: rocksdb: Options.compaction_readahead_size: 0 2024-03-04T09:20:50.991 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:20:50 smithi052 ceph-mon[32473]: rocksdb: Options.max_background_flushes: -1 2024-03-04T09:20:50.991 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:20:50 smithi052 ceph-mon[32473]: rocksdb: Compression algorithms supported: 2024-03-04T09:20:50.991 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:20:50 smithi052 ceph-mon[32473]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-03-04T09:20:50.991 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:20:50 smithi052 ceph-mon[32473]: rocksdb: kZSTD supported: 0 2024-03-04T09:20:50.991 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:20:50 smithi052 ceph-mon[32473]: rocksdb: kXpressCompression supported: 0 2024-03-04T09:20:50.991 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:20:50 smithi052 ceph-mon[32473]: rocksdb: kLZ4HCCompression supported: 1 2024-03-04T09:20:50.992 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:20:50 smithi052 ceph-mon[32473]: rocksdb: kZlibCompression supported: 1 2024-03-04T09:20:50.992 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:20:50 smithi052 ceph-mon[32473]: rocksdb: kSnappyCompression supported: 1 2024-03-04T09:20:50.992 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:20:50 smithi052 ceph-mon[32473]: rocksdb: kLZ4Compression supported: 1 2024-03-04T09:20:50.992 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:20:50 smithi052 ceph-mon[32473]: rocksdb: kBZip2Compression supported: 0 2024-03-04T09:20:50.992 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:20:50 smithi052 ceph-mon[32473]: rocksdb: Fast CRC32 supported: Supported on x86 2024-03-04T09:20:50.992 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:20:50 smithi052 ceph-mon[32473]: rocksdb: DMutex implementation: pthread_mutex_t 2024-03-04T09:20:50.992 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:20:50 smithi052 ceph-mon[32473]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-03-04T09:20:50.992 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:20:50 smithi052 ceph-mon[32473]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-03-04T09:20:50.992 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:20:50 smithi052 ceph-mon[32473]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-03-04T09:20:50.992 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:20:50 smithi052 ceph-mon[32473]: rocksdb: Options.merge_operator: 2024-03-04T09:20:50.992 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:20:50 smithi052 ceph-mon[32473]: rocksdb: Options.compaction_filter: None 2024-03-04T09:20:50.992 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:20:50 smithi052 ceph-mon[32473]: rocksdb: Options.compaction_filter_factory: None 2024-03-04T09:20:50.992 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:20:50 smithi052 ceph-mon[32473]: rocksdb: Options.sst_partitioner_factory: None 2024-03-04T09:20:50.992 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:20:50 smithi052 ceph-mon[32473]: rocksdb: Options.memtable_factory: SkipListFactory 2024-03-04T09:20:50.993 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:20:50 smithi052 ceph-mon[32473]: rocksdb: Options.table_factory: BlockBasedTable 2024-03-04T09:20:50.993 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:20:50 smithi052 ceph-mon[32473]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x560a4eeafa20) 2024-03-04T09:20:50.993 INFO:journalctl@ceph.mon.a.smithi052.stdout: cache_index_and_filter_blocks: 1 2024-03-04T09:20:50.993 INFO:journalctl@ceph.mon.a.smithi052.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-03-04T09:20:50.993 INFO:journalctl@ceph.mon.a.smithi052.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-03-04T09:20:50.993 INFO:journalctl@ceph.mon.a.smithi052.stdout: pin_top_level_index_and_filter: 1 2024-03-04T09:20:50.993 INFO:journalctl@ceph.mon.a.smithi052.stdout: index_type: 0 2024-03-04T09:20:50.993 INFO:journalctl@ceph.mon.a.smithi052.stdout: data_block_index_type: 0 2024-03-04T09:20:50.993 INFO:journalctl@ceph.mon.a.smithi052.stdout: index_shortening: 1 2024-03-04T09:20:50.993 INFO:journalctl@ceph.mon.a.smithi052.stdout: data_block_hash_table_util_ratio: 0.750000 2024-03-04T09:20:50.993 INFO:journalctl@ceph.mon.a.smithi052.stdout: checksum: 4 2024-03-04T09:20:50.993 INFO:journalctl@ceph.mon.a.smithi052.stdout: no_block_cache: 0 2024-03-04T09:20:50.993 INFO:journalctl@ceph.mon.a.smithi052.stdout: block_cache: 0x560a4eeb3090 2024-03-04T09:20:50.993 INFO:journalctl@ceph.mon.a.smithi052.stdout: block_cache_name: BinnedLRUCache 2024-03-04T09:20:50.994 INFO:journalctl@ceph.mon.a.smithi052.stdout: block_cache_options: 2024-03-04T09:20:50.994 INFO:journalctl@ceph.mon.a.smithi052.stdout: capacity : 536870912 2024-03-04T09:20:50.994 INFO:journalctl@ceph.mon.a.smithi052.stdout: num_shard_bits : 4 2024-03-04T09:20:50.994 INFO:journalctl@ceph.mon.a.smithi052.stdout: strict_capacity_limit : 0 2024-03-04T09:20:50.994 INFO:journalctl@ceph.mon.a.smithi052.stdout: high_pri_pool_ratio: 0.000 2024-03-04T09:20:50.994 INFO:journalctl@ceph.mon.a.smithi052.stdout: block_cache_compressed: (nil) 2024-03-04T09:20:50.994 INFO:journalctl@ceph.mon.a.smithi052.stdout: persistent_cache: (nil) 2024-03-04T09:20:50.994 INFO:journalctl@ceph.mon.a.smithi052.stdout: block_size: 4096 2024-03-04T09:20:50.994 INFO:journalctl@ceph.mon.a.smithi052.stdout: block_size_deviation: 10 2024-03-04T09:20:50.994 INFO:journalctl@ceph.mon.a.smithi052.stdout: block_restart_interval: 16 2024-03-04T09:20:50.994 INFO:journalctl@ceph.mon.a.smithi052.stdout: index_block_restart_interval: 1 2024-03-04T09:20:50.994 INFO:journalctl@ceph.mon.a.smithi052.stdout: metadata_block_size: 4096 2024-03-04T09:20:50.994 INFO:journalctl@ceph.mon.a.smithi052.stdout: partition_filters: 0 2024-03-04T09:20:50.994 INFO:journalctl@ceph.mon.a.smithi052.stdout: use_delta_encoding: 1 2024-03-04T09:20:50.994 INFO:journalctl@ceph.mon.a.smithi052.stdout: filter_policy: bloomfilter 2024-03-04T09:20:50.995 INFO:journalctl@ceph.mon.a.smithi052.stdout: whole_key_filtering: 1 2024-03-04T09:20:50.995 INFO:journalctl@ceph.mon.a.smithi052.stdout: verify_compression: 0 2024-03-04T09:20:50.995 INFO:journalctl@ceph.mon.a.smithi052.stdout: read_amp_bytes_per_bit: 0 2024-03-04T09:20:50.995 INFO:journalctl@ceph.mon.a.smithi052.stdout: format_version: 5 2024-03-04T09:20:50.995 INFO:journalctl@ceph.mon.a.smithi052.stdout: enable_index_compression: 1 2024-03-04T09:20:50.995 INFO:journalctl@ceph.mon.a.smithi052.stdout: block_align: 0 2024-03-04T09:20:50.995 INFO:journalctl@ceph.mon.a.smithi052.stdout: max_auto_readahead_size: 262144 2024-03-04T09:20:50.995 INFO:journalctl@ceph.mon.a.smithi052.stdout: prepopulate_block_cache: 0 2024-03-04T09:20:50.995 INFO:journalctl@ceph.mon.a.smithi052.stdout: initial_auto_readahead_size: 8192 2024-03-04T09:20:50.995 INFO:journalctl@ceph.mon.a.smithi052.stdout: num_file_reads_for_auto_readahead: 2 2024-03-04T09:20:50.995 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:20:50 smithi052 ceph-mon[32473]: rocksdb: Options.write_buffer_size: 33554432 2024-03-04T09:20:50.995 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:20:50 smithi052 ceph-mon[32473]: rocksdb: Options.max_write_buffer_number: 2 2024-03-04T09:20:50.995 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:20:50 smithi052 ceph-mon[32473]: rocksdb: Options.compression: NoCompression 2024-03-04T09:20:50.996 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:20:50 smithi052 ceph-mon[32473]: rocksdb: Options.bottommost_compression: Disabled 2024-03-04T09:20:50.996 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:20:50 smithi052 ceph-mon[32473]: rocksdb: Options.prefix_extractor: nullptr 2024-03-04T09:20:50.996 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:20:50 smithi052 ceph-mon[32473]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-03-04T09:20:50.996 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:20:50 smithi052 ceph-mon[32473]: rocksdb: Options.num_levels: 7 2024-03-04T09:20:50.996 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:20:50 smithi052 ceph-mon[32473]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-03-04T09:20:50.996 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:20:50 smithi052 ceph-mon[32473]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-03-04T09:20:50.996 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:20:50 smithi052 ceph-mon[32473]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-03-04T09:20:50.996 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:20:50 smithi052 ceph-mon[32473]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-03-04T09:20:50.996 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:20:50 smithi052 ceph-mon[32473]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-03-04T09:20:50.996 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:20:50 smithi052 ceph-mon[32473]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-03-04T09:20:50.996 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:20:50 smithi052 ceph-mon[32473]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-03-04T09:20:50.996 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:20:50 smithi052 ceph-mon[32473]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-03-04T09:20:50.996 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:20:50 smithi052 ceph-mon[32473]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-03-04T09:20:50.996 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:20:50 smithi052 ceph-mon[32473]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-03-04T09:20:50.997 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:20:50 smithi052 ceph-mon[32473]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-03-04T09:20:50.997 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:20:50 smithi052 ceph-mon[32473]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-03-04T09:20:50.997 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:20:50 smithi052 ceph-mon[32473]: rocksdb: Options.compression_opts.window_bits: -14 2024-03-04T09:20:50.997 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:20:50 smithi052 ceph-mon[32473]: rocksdb: Options.compression_opts.level: 32767 2024-03-04T09:20:50.997 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:20:50 smithi052 ceph-mon[32473]: rocksdb: Options.compression_opts.strategy: 0 2024-03-04T09:20:50.997 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:20:50 smithi052 ceph-mon[32473]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-03-04T09:20:50.997 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:20:50 smithi052 ceph-mon[32473]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-03-04T09:20:50.997 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:20:50 smithi052 ceph-mon[32473]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-03-04T09:20:50.997 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:20:50 smithi052 ceph-mon[32473]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-03-04T09:20:50.997 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:20:50 smithi052 ceph-mon[32473]: rocksdb: Options.compression_opts.enabled: false 2024-03-04T09:20:50.997 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:20:50 smithi052 ceph-mon[32473]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-03-04T09:20:50.997 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:20:50 smithi052 ceph-mon[32473]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-03-04T09:20:50.997 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:20:50 smithi052 ceph-mon[32473]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-03-04T09:20:50.998 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:20:50 smithi052 ceph-mon[32473]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-03-04T09:20:50.998 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:20:50 smithi052 ceph-mon[32473]: rocksdb: Options.target_file_size_base: 67108864 2024-03-04T09:20:50.998 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:20:50 smithi052 ceph-mon[32473]: rocksdb: Options.target_file_size_multiplier: 1 2024-03-04T09:20:50.998 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:20:50 smithi052 ceph-mon[32473]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-03-04T09:20:50.998 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:20:50 smithi052 ceph-mon[32473]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-03-04T09:20:50.998 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:20:50 smithi052 ceph-mon[32473]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-03-04T09:20:50.998 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:20:50 smithi052 ceph-mon[32473]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-03-04T09:20:50.998 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:20:50 smithi052 ceph-mon[32473]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-03-04T09:20:50.998 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:20:50 smithi052 ceph-mon[32473]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-03-04T09:20:50.998 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:20:50 smithi052 ceph-mon[32473]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-03-04T09:20:50.998 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:20:50 smithi052 ceph-mon[32473]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-03-04T09:20:50.998 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:20:50 smithi052 ceph-mon[32473]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-03-04T09:20:50.998 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:20:50 smithi052 ceph-mon[32473]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-03-04T09:20:50.998 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:20:50 smithi052 ceph-mon[32473]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-03-04T09:20:50.998 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:20:50 smithi052 ceph-mon[32473]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-03-04T09:20:50.999 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:20:50 smithi052 ceph-mon[32473]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-03-04T09:20:50.999 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:20:50 smithi052 ceph-mon[32473]: rocksdb: Options.arena_block_size: 1048576 2024-03-04T09:20:50.999 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:20:50 smithi052 ceph-mon[32473]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-03-04T09:20:50.999 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:20:50 smithi052 ceph-mon[32473]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-03-04T09:20:50.999 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:20:50 smithi052 ceph-mon[32473]: rocksdb: Options.disable_auto_compactions: 0 2024-03-04T09:20:50.999 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:20:50 smithi052 ceph-mon[32473]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-03-04T09:20:50.999 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:20:50 smithi052 ceph-mon[32473]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-03-04T09:20:50.999 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:20:50 smithi052 ceph-mon[32473]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-03-04T09:20:50.999 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:20:50 smithi052 ceph-mon[32473]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-03-04T09:20:50.999 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:20:50 smithi052 ceph-mon[32473]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-03-04T09:20:50.999 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:20:50 smithi052 ceph-mon[32473]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-03-04T09:20:50.999 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:20:50 smithi052 ceph-mon[32473]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-03-04T09:20:50.999 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:20:50 smithi052 ceph-mon[32473]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-03-04T09:20:51.000 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:20:50 smithi052 ceph-mon[32473]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-03-04T09:20:51.000 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:20:50 smithi052 ceph-mon[32473]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-03-04T09:20:51.000 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:20:50 smithi052 ceph-mon[32473]: rocksdb: Options.table_properties_collectors: 2024-03-04T09:20:51.000 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:20:50 smithi052 ceph-mon[32473]: rocksdb: Options.inplace_update_support: 0 2024-03-04T09:20:51.000 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:20:50 smithi052 ceph-mon[32473]: rocksdb: Options.inplace_update_num_locks: 10000 2024-03-04T09:20:51.000 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:20:50 smithi052 ceph-mon[32473]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-03-04T09:20:51.000 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:20:50 smithi052 ceph-mon[32473]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-03-04T09:20:51.000 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:20:50 smithi052 ceph-mon[32473]: rocksdb: Options.memtable_huge_page_size: 0 2024-03-04T09:20:51.000 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:20:50 smithi052 ceph-mon[32473]: rocksdb: Options.bloom_locality: 0 2024-03-04T09:20:51.000 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:20:50 smithi052 ceph-mon[32473]: rocksdb: Options.max_successive_merges: 0 2024-03-04T09:20:51.000 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:20:50 smithi052 ceph-mon[32473]: rocksdb: Options.optimize_filters_for_hits: 0 2024-03-04T09:20:51.000 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:20:50 smithi052 ceph-mon[32473]: rocksdb: Options.paranoid_file_checks: 0 2024-03-04T09:20:51.001 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:20:50 smithi052 ceph-mon[32473]: rocksdb: Options.force_consistency_checks: 1 2024-03-04T09:20:51.001 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:20:50 smithi052 ceph-mon[32473]: rocksdb: Options.report_bg_io_stats: 0 2024-03-04T09:20:51.001 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:20:50 smithi052 ceph-mon[32473]: rocksdb: Options.ttl: 2592000 2024-03-04T09:20:51.001 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:20:50 smithi052 ceph-mon[32473]: rocksdb: Options.periodic_compaction_seconds: 0 2024-03-04T09:20:51.001 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:20:50 smithi052 ceph-mon[32473]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-03-04T09:20:51.001 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:20:50 smithi052 ceph-mon[32473]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-03-04T09:20:51.001 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:20:50 smithi052 ceph-mon[32473]: rocksdb: Options.enable_blob_files: false 2024-03-04T09:20:51.001 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:20:50 smithi052 ceph-mon[32473]: rocksdb: Options.min_blob_size: 0 2024-03-04T09:20:51.001 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:20:50 smithi052 ceph-mon[32473]: rocksdb: Options.blob_file_size: 268435456 2024-03-04T09:20:51.001 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:20:50 smithi052 ceph-mon[32473]: rocksdb: Options.blob_compression_type: NoCompression 2024-03-04T09:20:51.001 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:20:50 smithi052 ceph-mon[32473]: rocksdb: Options.enable_blob_garbage_collection: false 2024-03-04T09:20:51.001 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:20:50 smithi052 ceph-mon[32473]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-03-04T09:20:51.001 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:20:50 smithi052 ceph-mon[32473]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-03-04T09:20:51.002 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:20:50 smithi052 ceph-mon[32473]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-03-04T09:20:51.002 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:20:50 smithi052 ceph-mon[32473]: rocksdb: Options.blob_file_starting_level: 0 2024-03-04T09:20:51.002 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:20:50 smithi052 ceph-mon[32473]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-03-04T09:20:51.002 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:20:50 smithi052 ceph-mon[32473]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-03-04T09:20:51.002 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:20:50 smithi052 ceph-mon[32473]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-03-04T09:20:51.002 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:20:50 smithi052 ceph-mon[32473]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 5d4862b7-f0a4-49e8-8280-439958875eca 2024-03-04T09:20:51.002 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:20:50 smithi052 ceph-mon[32473]: rocksdb: EVENT_LOG_v1 {"time_micros": 1709544050725017, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-03-04T09:20:51.002 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:20:50 smithi052 ceph-mon[32473]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-03-04T09:20:51.002 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:20:50 smithi052 ceph-mon[32473]: rocksdb: EVENT_LOG_v1 {"time_micros": 1709544050725944, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 94913, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 316, "table_properties": {"data_size": 92911, "index_size": 284, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 709, "raw_key_size": 13781, "raw_average_key_size": 50, "raw_value_size": 85412, "raw_average_value_size": 310, "num_data_blocks": 11, "num_entries": 275, "num_filter_entries": 275, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1709544050, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "5d4862b7-f0a4-49e8-8280-439958875eca", "db_session_id": "1YA4CCURFWDCIBO1PAN1", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-03-04T09:20:51.002 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:20:50 smithi052 ceph-mon[32473]: rocksdb: EVENT_LOG_v1 {"time_micros": 1709544050726033, "job": 1, "event": "recovery_finished"} 2024-03-04T09:20:51.002 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:20:50 smithi052 ceph-mon[32473]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-03-04T09:20:51.002 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:20:50 smithi052 ceph-mon[32473]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-03-04T09:20:51.002 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:20:50 smithi052 ceph-mon[32473]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x560a4efa4000 2024-03-04T09:20:51.003 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:20:50 smithi052 ceph-mon[32473]: rocksdb: DB pointer 0x560a4ef8e000 2024-03-04T09:20:51.003 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:20:50 smithi052 ceph-mon[32473]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-03-04T09:20:51.003 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:20:50 smithi052 ceph-mon[32473]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-03-04T09:20:51.003 INFO:journalctl@ceph.mon.a.smithi052.stdout: ** DB Stats ** 2024-03-04T09:20:51.003 INFO:journalctl@ceph.mon.a.smithi052.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-03-04T09:20:51.003 INFO:journalctl@ceph.mon.a.smithi052.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-03-04T09:20:51.003 INFO:journalctl@ceph.mon.a.smithi052.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-03-04T09:20:51.003 INFO:journalctl@ceph.mon.a.smithi052.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-03-04T09:20:51.003 INFO:journalctl@ceph.mon.a.smithi052.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-03-04T09:20:51.003 INFO:journalctl@ceph.mon.a.smithi052.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-03-04T09:20:51.003 INFO:journalctl@ceph.mon.a.smithi052.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-03-04T09:20:51.003 INFO:journalctl@ceph.mon.a.smithi052.stdout: 2024-03-04T09:20:51.003 INFO:journalctl@ceph.mon.a.smithi052.stdout: ** Compaction Stats [default] ** 2024-03-04T09:20:51.003 INFO:journalctl@ceph.mon.a.smithi052.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-03-04T09:20:51.004 INFO:journalctl@ceph.mon.a.smithi052.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-03-04T09:20:51.004 INFO:journalctl@ceph.mon.a.smithi052.stdout: L0 2/0 94.52 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 158.2 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-03-04T09:20:51.004 INFO:journalctl@ceph.mon.a.smithi052.stdout: Sum 2/0 94.52 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 158.2 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-03-04T09:20:51.004 INFO:journalctl@ceph.mon.a.smithi052.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 158.2 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-03-04T09:20:51.004 INFO:journalctl@ceph.mon.a.smithi052.stdout: 2024-03-04T09:20:51.004 INFO:journalctl@ceph.mon.a.smithi052.stdout: ** Compaction Stats [default] ** 2024-03-04T09:20:51.004 INFO:journalctl@ceph.mon.a.smithi052.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-03-04T09:20:51.004 INFO:journalctl@ceph.mon.a.smithi052.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-03-04T09:20:51.004 INFO:journalctl@ceph.mon.a.smithi052.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 158.2 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-03-04T09:20:51.004 INFO:journalctl@ceph.mon.a.smithi052.stdout: 2024-03-04T09:20:51.004 INFO:journalctl@ceph.mon.a.smithi052.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-03-04T09:20:51.004 INFO:journalctl@ceph.mon.a.smithi052.stdout: 2024-03-04T09:20:51.005 INFO:journalctl@ceph.mon.a.smithi052.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-03-04T09:20:51.005 INFO:journalctl@ceph.mon.a.smithi052.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-03-04T09:20:51.005 INFO:journalctl@ceph.mon.a.smithi052.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-03-04T09:20:51.005 INFO:journalctl@ceph.mon.a.smithi052.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-03-04T09:20:51.005 INFO:journalctl@ceph.mon.a.smithi052.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-03-04T09:20:51.005 INFO:journalctl@ceph.mon.a.smithi052.stdout: AddFile(Keys): cumulative 0, interval 0 2024-03-04T09:20:51.005 INFO:journalctl@ceph.mon.a.smithi052.stdout: Cumulative compaction: 0.00 GB write, 31.16 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-03-04T09:20:51.005 INFO:journalctl@ceph.mon.a.smithi052.stdout: Interval compaction: 0.00 GB write, 31.16 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-03-04T09:20:51.005 INFO:journalctl@ceph.mon.a.smithi052.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-03-04T09:20:51.005 INFO:journalctl@ceph.mon.a.smithi052.stdout: Block cache BinnedLRUCache@0x560a4eeb3090#2 capacity: 512.00 MB usage: 1.36 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 8e-06 secs_since: 0 2024-03-04T09:20:51.005 INFO:journalctl@ceph.mon.a.smithi052.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,0.89 KB,0.000169873%) IndexBlock(2,0.47 KB,8.9407e-05%) Misc(1,0.00 KB,0%) 2024-03-04T09:20:51.005 INFO:journalctl@ceph.mon.a.smithi052.stdout: 2024-03-04T09:20:51.006 INFO:journalctl@ceph.mon.a.smithi052.stdout: ** File Read Latency Histogram By Level [default] ** 2024-03-04T09:20:51.006 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:20:50 smithi052 ceph-mon[32473]: starting mon.a rank 0 at public addrs [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] at bind addrs [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid 4e059f08-da08-11ee-95c6-87774f69a715 2024-03-04T09:20:51.006 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:20:50 smithi052 ceph-mon[32473]: mon.a@-1(???) e1 preinit fsid 4e059f08-da08-11ee-95c6-87774f69a715 2024-03-04T09:20:51.006 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:20:50 smithi052 ceph-mon[32473]: mon.a@-1(???).mds e1 new map 2024-03-04T09:20:51.006 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:20:50 smithi052 ceph-mon[32473]: mon.a@-1(???).mds e1 print_map 2024-03-04T09:20:51.006 INFO:journalctl@ceph.mon.a.smithi052.stdout: e1 2024-03-04T09:20:51.006 INFO:journalctl@ceph.mon.a.smithi052.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-03-04T09:20:51.006 INFO:journalctl@ceph.mon.a.smithi052.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments} 2024-03-04T09:20:51.006 INFO:journalctl@ceph.mon.a.smithi052.stdout: legacy client fscid: -1 2024-03-04T09:20:51.006 INFO:journalctl@ceph.mon.a.smithi052.stdout: 2024-03-04T09:20:51.006 INFO:journalctl@ceph.mon.a.smithi052.stdout: No filesystems configured 2024-03-04T09:20:51.006 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:20:50 smithi052 ceph-mon[32473]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-03-04T09:20:51.006 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:20:50 smithi052 ceph-mon[32473]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-03-04T09:20:51.007 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:20:50 smithi052 ceph-mon[32473]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-03-04T09:20:51.007 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:20:50 smithi052 ceph-mon[32473]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-03-04T09:20:51.007 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:20:50 smithi052 ceph-mon[32473]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-03-04T09:20:51.007 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:20:50 smithi052 bash[32454]: b840a3470a2b8f2bf6ea9a6f5ee27a387058780843cce4dbfba12d501c25b2b0 2024-03-04T09:20:51.007 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:20:50 smithi052 systemd[1]: Started Ceph mon.a for 4e059f08-da08-11ee-95c6-87774f69a715. 2024-03-04T09:20:51.007 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:20:50 smithi052 ceph-mon[32473]: mon.a is new leader, mons a in quorum (ranks 0) 2024-03-04T09:20:51.007 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:20:50 smithi052 ceph-mon[32473]: monmap e1: 1 mons at {a=[v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-04T09:20:51.007 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:20:50 smithi052 ceph-mon[32473]: fsmap 2024-03-04T09:20:51.007 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:20:50 smithi052 ceph-mon[32473]: osdmap e1: 0 total, 0 up, 0 in 2024-03-04T09:20:51.007 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:20:50 smithi052 ceph-mon[32473]: mgrmap e1: no daemons active 2024-03-04T09:20:51.826 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:20:51.189+0000 7fd806408700 1 Processor -- start 2024-03-04T09:20:51.826 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:20:51.189+0000 7fd806408700 1 -- start start 2024-03-04T09:20:51.826 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:20:51.189+0000 7fd806408700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd80007ce20 0x7fd80007b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:20:51.826 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:20:51.189+0000 7fd806408700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd80007b810 con 0x7fd80007ce20 2024-03-04T09:20:51.826 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:20:51.190+0000 7fd7fffff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd80007ce20 0x7fd80007b2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:20:51.826 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:20:51.190+0000 7fd7fffff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd80007ce20 0x7fd80007b2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:55508/0 (socket says 172.21.15.52:55508) 2024-03-04T09:20:51.826 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:20:51.190+0000 7fd7fffff700 1 -- 172.21.15.52:0/3085143730 learned_addr learned my addr 172.21.15.52:0/3085143730 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-04T09:20:51.826 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:20:51.190+0000 7fd7fffff700 1 -- 172.21.15.52:0/3085143730 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd80007b990 con 0x7fd80007ce20 2024-03-04T09:20:51.826 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:20:51.190+0000 7fd7fffff700 1 --2- 172.21.15.52:0/3085143730 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd80007ce20 0x7fd80007b2d0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fd7f00098a0 tx=0x7fd7f0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f11711fa7ee22b58 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:20:51.827 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:20:51.190+0000 7fd7feffd700 1 -- 172.21.15.52:0/3085143730 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd7f0004160 con 0x7fd80007ce20 2024-03-04T09:20:51.827 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:20:51.190+0000 7fd7feffd700 1 -- 172.21.15.52:0/3085143730 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd7f0004300 con 0x7fd80007ce20 2024-03-04T09:20:51.827 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:20:51.190+0000 7fd7feffd700 1 -- 172.21.15.52:0/3085143730 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd7f0011b50 con 0x7fd80007ce20 2024-03-04T09:20:51.827 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:20:51.191+0000 7fd806408700 1 -- 172.21.15.52:0/3085143730 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd80007ce20 msgr2=0x7fd80007b2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:20:51.827 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:20:51.191+0000 7fd806408700 1 --2- 172.21.15.52:0/3085143730 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd80007ce20 0x7fd80007b2d0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fd7f00098a0 tx=0x7fd7f0009bb0 comp rx=0 tx=0).stop 2024-03-04T09:20:51.827 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:20:51.191+0000 7fd806408700 1 -- 172.21.15.52:0/3085143730 shutdown_connections 2024-03-04T09:20:51.827 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:20:51.191+0000 7fd806408700 1 --2- 172.21.15.52:0/3085143730 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd80007ce20 0x7fd80007b2d0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:20:51.827 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:20:51.191+0000 7fd806408700 1 -- 172.21.15.52:0/3085143730 >> 172.21.15.52:0/3085143730 conn(0x7fd800103060 msgr2=0x7fd800105480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:20:51.827 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:20:51.191+0000 7fd806408700 1 -- 172.21.15.52:0/3085143730 shutdown_connections 2024-03-04T09:20:51.827 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:20:51.191+0000 7fd806408700 1 -- 172.21.15.52:0/3085143730 wait complete. 2024-03-04T09:20:51.827 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:20:51.191+0000 7fd806408700 1 Processor -- start 2024-03-04T09:20:51.827 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:20:51.191+0000 7fd806408700 1 -- start start 2024-03-04T09:20:51.827 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:20:51.192+0000 7fd806408700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd80007ce20 0x7fd80019db60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:20:51.827 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:20:51.192+0000 7fd806408700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd80010df90 con 0x7fd80007ce20 2024-03-04T09:20:51.828 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:20:51.192+0000 7fd7fffff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd80007ce20 0x7fd80019db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:20:51.828 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:20:51.192+0000 7fd7fffff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd80007ce20 0x7fd80019db60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:55522/0 (socket says 172.21.15.52:55522) 2024-03-04T09:20:51.828 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:20:51.192+0000 7fd7fffff700 1 -- 172.21.15.52:0/123207004 learned_addr learned my addr 172.21.15.52:0/123207004 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-04T09:20:51.828 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:20:51.192+0000 7fd7fffff700 1 -- 172.21.15.52:0/123207004 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd80019e0a0 con 0x7fd80007ce20 2024-03-04T09:20:51.828 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:20:51.192+0000 7fd7fffff700 1 --2- 172.21.15.52:0/123207004 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd80007ce20 0x7fd80019db60 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fd7f0011b60 tx=0x7fd7f0011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:20:51.828 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:20:51.192+0000 7fd7fd7fa700 1 -- 172.21.15.52:0/123207004 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd7f0003b20 con 0x7fd80007ce20 2024-03-04T09:20:51.828 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:20:51.192+0000 7fd7fd7fa700 1 -- 172.21.15.52:0/123207004 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd7f0003cc0 con 0x7fd80007ce20 2024-03-04T09:20:51.828 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:20:51.192+0000 7fd806408700 1 -- 172.21.15.52:0/123207004 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd80019e330 con 0x7fd80007ce20 2024-03-04T09:20:51.828 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:20:51.192+0000 7fd7fd7fa700 1 -- 172.21.15.52:0/123207004 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd7f0024840 con 0x7fd80007ce20 2024-03-04T09:20:51.828 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:20:51.193+0000 7fd806408700 1 -- 172.21.15.52:0/123207004 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8001a1020 con 0x7fd80007ce20 2024-03-04T09:20:51.828 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:20:51.193+0000 7fd7fd7fa700 1 -- 172.21.15.52:0/123207004 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7fd7f0022020 con 0x7fd80007ce20 2024-03-04T09:20:51.828 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:20:51.193+0000 7fd7fd7fa700 1 -- 172.21.15.52:0/123207004 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fd7f0011e00 con 0x7fd80007ce20 2024-03-04T09:20:51.828 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:20:51.193+0000 7fd806408700 1 -- 172.21.15.52:0/123207004 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd8000636f0 con 0x7fd80007ce20 2024-03-04T09:20:51.828 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:20:51.196+0000 7fd7fd7fa700 1 -- 172.21.15.52:0/123207004 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7fd7f0043dd0 con 0x7fd80007ce20 2024-03-04T09:20:51.829 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:20:51.266+0000 7fd806408700 1 -- 172.21.15.52:0/123207004 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) v1 -- 0x7fd8001a11f0 con 0x7fd80007ce20 2024-03-04T09:20:51.829 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:20:51.267+0000 7fd7fd7fa700 1 -- 172.21.15.52:0/123207004 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) v1 ==== 124+0+0 (secure 0 0 0) 0x7fd7f0032020 con 0x7fd80007ce20 2024-03-04T09:20:51.829 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:20:51.268+0000 7fd806408700 1 -- 172.21.15.52:0/123207004 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd80007ce20 msgr2=0x7fd80019db60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:20:51.829 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:20:51.268+0000 7fd806408700 1 --2- 172.21.15.52:0/123207004 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd80007ce20 0x7fd80019db60 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fd7f0011b60 tx=0x7fd7f0011c40 comp rx=0 tx=0).stop 2024-03-04T09:20:51.829 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:20:51.268+0000 7fd806408700 1 -- 172.21.15.52:0/123207004 shutdown_connections 2024-03-04T09:20:51.829 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:20:51.268+0000 7fd806408700 1 --2- 172.21.15.52:0/123207004 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd80007ce20 0x7fd80019db60 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:20:51.829 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:20:51.268+0000 7fd806408700 1 -- 172.21.15.52:0/123207004 >> 172.21.15.52:0/123207004 conn(0x7fd800103060 msgr2=0x7fd800103c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:20:51.829 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:20:51.269+0000 7fd806408700 1 -- 172.21.15.52:0/123207004 shutdown_connections 2024-03-04T09:20:51.829 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:20:51.269+0000 7fd806408700 1 -- 172.21.15.52:0/123207004 wait complete. 2024-03-04T09:20:51.829 INFO:teuthology.orchestra.run.smithi052.stdout:Wrote config to /etc/ceph/ceph.conf 2024-03-04T09:20:51.852 INFO:teuthology.orchestra.run.smithi052.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-03-04T09:20:51.853 INFO:teuthology.orchestra.run.smithi052.stdout:Creating mgr... 2024-03-04T09:20:51.853 INFO:teuthology.orchestra.run.smithi052.stdout:Verifying port 0.0.0.0:9283 ... 2024-03-04T09:20:51.853 INFO:teuthology.orchestra.run.smithi052.stdout:Verifying port 0.0.0.0:8765 ... 2024-03-04T09:20:52.104 INFO:teuthology.orchestra.run.smithi052.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-4e059f08-da08-11ee-95c6-87774f69a715@mgr.x 2024-03-04T09:20:52.104 INFO:teuthology.orchestra.run.smithi052.stdout:systemctl: stderr Failed to reset failed state of unit ceph-4e059f08-da08-11ee-95c6-87774f69a715@mgr.x.service: Unit ceph-4e059f08-da08-11ee-95c6-87774f69a715@mgr.x.service not loaded. 2024-03-04T09:20:52.261 INFO:teuthology.orchestra.run.smithi052.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-4e059f08-da08-11ee-95c6-87774f69a715.target.wants/ceph-4e059f08-da08-11ee-95c6-87774f69a715@mgr.x.service → /etc/systemd/system/ceph-4e059f08-da08-11ee-95c6-87774f69a715@.service. 2024-03-04T09:20:52.488 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:20:52 smithi052 ceph-mon[32473]: from='client.? 172.21.15.52:0/123207004' entity='client.admin' 2024-03-04T09:20:52.489 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Mar 04 09:20:52 smithi052 systemd[1]: Starting Ceph mgr.x for 4e059f08-da08-11ee-95c6-87774f69a715... 2024-03-04T09:20:52.757 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Mar 04 09:20:52 smithi052 podman[32691]: 2024-03-04 09:20:52.492267523 +0000 UTC m=+0.105774755 container create 4852946a56df23e775c98f79c6a3ff5f22eecdc45a7f257119b0a1b7426f11c6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd, name=ceph-4e059f08-da08-11ee-95c6-87774f69a715-mgr-x, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.build-date=20240222, RELEASE=wip-lusov-qdb-1bc250c, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8) 2024-03-04T09:20:52.757 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Mar 04 09:20:52 smithi052 podman[32691]: 2024-03-04 09:20:52.404058953 +0000 UTC m=+0.017566197 image pull f88ffb32aa6a7d5dfdc954d9f096f0c9cc47981d05ec8d117ba859a22d8185ba quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd 2024-03-04T09:20:52.757 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Mar 04 09:20:52 smithi052 podman[32691]: 2024-03-04 09:20:52.694591935 +0000 UTC m=+0.308099167 container init 4852946a56df23e775c98f79c6a3ff5f22eecdc45a7f257119b0a1b7426f11c6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd, name=ceph-4e059f08-da08-11ee-95c6-87774f69a715-mgr-x, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-qdb-1bc250c, io.buildah.version=1.33.5, org.label-schema.build-date=20240222, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8) 2024-03-04T09:20:52.757 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Mar 04 09:20:52 smithi052 podman[32691]: 2024-03-04 09:20:52.696966697 +0000 UTC m=+0.310473939 container start 4852946a56df23e775c98f79c6a3ff5f22eecdc45a7f257119b0a1b7426f11c6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd, name=ceph-4e059f08-da08-11ee-95c6-87774f69a715-mgr-x, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=wip-lusov-qdb-1bc250c, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240222, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.33.5) 2024-03-04T09:20:52.757 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Mar 04 09:20:52 smithi052 bash[32691]: 4852946a56df23e775c98f79c6a3ff5f22eecdc45a7f257119b0a1b7426f11c6 2024-03-04T09:20:52.772 INFO:teuthology.orchestra.run.smithi052.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-03-04T09:20:52.772 INFO:teuthology.orchestra.run.smithi052.stdout:systemctl: stdout disabled 2024-03-04T09:20:52.779 INFO:teuthology.orchestra.run.smithi052.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-03-04T09:20:52.779 INFO:teuthology.orchestra.run.smithi052.stdout:systemctl: stdout inactive 2024-03-04T09:20:52.779 INFO:teuthology.orchestra.run.smithi052.stdout:firewalld.service is not enabled 2024-03-04T09:20:52.780 INFO:teuthology.orchestra.run.smithi052.stdout:Not possible to enable service . firewalld.service is not available 2024-03-04T09:20:52.787 INFO:teuthology.orchestra.run.smithi052.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-03-04T09:20:52.788 INFO:teuthology.orchestra.run.smithi052.stdout:systemctl: stdout disabled 2024-03-04T09:20:52.794 INFO:teuthology.orchestra.run.smithi052.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-03-04T09:20:52.794 INFO:teuthology.orchestra.run.smithi052.stdout:systemctl: stdout inactive 2024-03-04T09:20:52.795 INFO:teuthology.orchestra.run.smithi052.stdout:firewalld.service is not enabled 2024-03-04T09:20:52.795 INFO:teuthology.orchestra.run.smithi052.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-03-04T09:20:52.795 INFO:teuthology.orchestra.run.smithi052.stdout:Waiting for mgr to start... 2024-03-04T09:20:52.795 INFO:teuthology.orchestra.run.smithi052.stdout:Waiting for mgr... 2024-03-04T09:20:53.013 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Mar 04 09:20:52 smithi052 systemd[1]: Started Ceph mgr.x for 4e059f08-da08-11ee-95c6-87774f69a715. 2024-03-04T09:20:53.584 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:20:53 smithi052 ceph-mon[32473]: from='client.? 172.21.15.52:0/418173470' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-03-04T09:20:53.585 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Mar 04 09:20:53 smithi052 ceph-4e059f08-da08-11ee-95c6-87774f69a715-mgr-x[32705]: 2024-03-04T09:20:53.269+0000 7fe55911a200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-04T09:20:53.585 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Mar 04 09:20:53 smithi052 ceph-4e059f08-da08-11ee-95c6-87774f69a715-mgr-x[32705]: 2024-03-04T09:20:53.369+0000 7fe55911a200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-04T09:20:53.851 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout 2024-03-04T09:20:53.851 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout { 2024-03-04T09:20:53.852 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "fsid": "4e059f08-da08-11ee-95c6-87774f69a715", 2024-03-04T09:20:53.852 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "health": { 2024-03-04T09:20:53.852 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-03-04T09:20:53.852 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-03-04T09:20:53.852 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-03-04T09:20:53.852 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout }, 2024-03-04T09:20:53.852 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-03-04T09:20:53.852 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-03-04T09:20:53.852 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout 0 2024-03-04T09:20:53.852 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout ], 2024-03-04T09:20:53.852 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-03-04T09:20:53.852 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "a" 2024-03-04T09:20:53.853 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout ], 2024-03-04T09:20:53.853 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "quorum_age": 2, 2024-03-04T09:20:53.853 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "monmap": { 2024-03-04T09:20:53.853 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-04T09:20:53.853 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-03-04T09:20:53.853 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-03-04T09:20:53.853 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout }, 2024-03-04T09:20:53.853 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-03-04T09:20:53.853 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-04T09:20:53.853 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-03-04T09:20:53.853 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-03-04T09:20:53.853 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-03-04T09:20:53.853 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-03-04T09:20:53.853 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-03-04T09:20:53.853 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-03-04T09:20:53.854 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout }, 2024-03-04T09:20:53.854 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-03-04T09:20:53.854 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-03-04T09:20:53.854 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-03-04T09:20:53.854 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-03-04T09:20:53.854 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-03-04T09:20:53.854 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-03-04T09:20:53.854 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-03-04T09:20:53.854 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-03-04T09:20:53.854 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-03-04T09:20:53.854 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout }, 2024-03-04T09:20:53.854 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-03-04T09:20:53.854 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-04T09:20:53.855 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-03-04T09:20:53.855 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-03-04T09:20:53.855 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout }, 2024-03-04T09:20:53.855 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-03-04T09:20:53.855 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "available": false, 2024-03-04T09:20:53.855 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-03-04T09:20:53.855 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "modules": [ 2024-03-04T09:20:53.855 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "iostat", 2024-03-04T09:20:53.855 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "nfs", 2024-03-04T09:20:53.855 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "restful" 2024-03-04T09:20:53.855 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout ], 2024-03-04T09:20:53.855 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-04T09:20:53.855 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout }, 2024-03-04T09:20:53.855 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-03-04T09:20:53.855 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-04T09:20:53.856 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "modified": "2024-03-04T09:20:46.105424+0000", 2024-03-04T09:20:53.856 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-04T09:20:53.856 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout }, 2024-03-04T09:20:53.856 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-03-04T09:20:53.856 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout } 2024-03-04T09:20:53.856 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:20:53.198+0000 7fab7225a700 1 Processor -- start 2024-03-04T09:20:53.856 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:20:53.198+0000 7fab7225a700 1 -- start start 2024-03-04T09:20:53.856 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:20:53.198+0000 7fab7225a700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fab6c1068d0 0x7fab6c106cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:20:53.856 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:20:53.198+0000 7fab7225a700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab6c1072c0 con 0x7fab6c1068d0 2024-03-04T09:20:53.856 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:20:53.199+0000 7fab6b7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fab6c1068d0 0x7fab6c106cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:20:53.856 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:20:53.199+0000 7fab6b7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fab6c1068d0 0x7fab6c106cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:55562/0 (socket says 172.21.15.52:55562) 2024-03-04T09:20:53.856 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:20:53.199+0000 7fab6b7fe700 1 -- 172.21.15.52:0/4253570436 learned_addr learned my addr 172.21.15.52:0/4253570436 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-04T09:20:53.856 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:20:53.199+0000 7fab6b7fe700 1 -- 172.21.15.52:0/4253570436 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab6c107ad0 con 0x7fab6c1068d0 2024-03-04T09:20:53.857 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:20:53.199+0000 7fab6b7fe700 1 --2- 172.21.15.52:0/4253570436 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fab6c1068d0 0x7fab6c106cf0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fab5c0098a0 tx=0x7fab5c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf264899f83b2f7d server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:20:53.857 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:20:53.199+0000 7fab6a7fc700 1 -- 172.21.15.52:0/4253570436 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fab5c004160 con 0x7fab6c1068d0 2024-03-04T09:20:53.857 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:20:53.200+0000 7fab6a7fc700 1 -- 172.21.15.52:0/4253570436 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fab5c004300 con 0x7fab6c1068d0 2024-03-04T09:20:53.857 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:20:53.200+0000 7fab6a7fc700 1 -- 172.21.15.52:0/4253570436 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fab5c011b50 con 0x7fab6c1068d0 2024-03-04T09:20:53.857 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:20:53.200+0000 7fab7225a700 1 -- 172.21.15.52:0/4253570436 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fab6c1068d0 msgr2=0x7fab6c106cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:20:53.857 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:20:53.200+0000 7fab7225a700 1 --2- 172.21.15.52:0/4253570436 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fab6c1068d0 0x7fab6c106cf0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fab5c0098a0 tx=0x7fab5c009bb0 comp rx=0 tx=0).stop 2024-03-04T09:20:53.857 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:20:53.200+0000 7fab7225a700 1 -- 172.21.15.52:0/4253570436 shutdown_connections 2024-03-04T09:20:53.857 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:20:53.200+0000 7fab7225a700 1 --2- 172.21.15.52:0/4253570436 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fab6c1068d0 0x7fab6c106cf0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:20:53.857 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:20:53.200+0000 7fab7225a700 1 -- 172.21.15.52:0/4253570436 >> 172.21.15.52:0/4253570436 conn(0x7fab6c101f70 msgr2=0x7fab6c104390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:20:53.857 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:20:53.200+0000 7fab7225a700 1 -- 172.21.15.52:0/4253570436 shutdown_connections 2024-03-04T09:20:53.857 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:20:53.200+0000 7fab7225a700 1 -- 172.21.15.52:0/4253570436 wait complete. 2024-03-04T09:20:53.857 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:20:53.201+0000 7fab7225a700 1 Processor -- start 2024-03-04T09:20:53.857 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:20:53.201+0000 7fab7225a700 1 -- start start 2024-03-04T09:20:53.857 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:20:53.201+0000 7fab7225a700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fab6c1068d0 0x7fab6c199830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:20:53.858 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:20:53.201+0000 7fab7225a700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab6c079fe0 con 0x7fab6c1068d0 2024-03-04T09:20:53.858 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:20:53.201+0000 7fab6b7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fab6c1068d0 0x7fab6c199830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:20:53.858 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:20:53.201+0000 7fab6b7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fab6c1068d0 0x7fab6c199830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:55566/0 (socket says 172.21.15.52:55566) 2024-03-04T09:20:53.858 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:20:53.201+0000 7fab6b7fe700 1 -- 172.21.15.52:0/418173470 learned_addr learned my addr 172.21.15.52:0/418173470 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-04T09:20:53.858 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:20:53.202+0000 7fab6b7fe700 1 -- 172.21.15.52:0/418173470 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab6c199d70 con 0x7fab6c1068d0 2024-03-04T09:20:53.858 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:20:53.202+0000 7fab6b7fe700 1 --2- 172.21.15.52:0/418173470 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fab6c1068d0 0x7fab6c199830 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fab5c009fb0 tx=0x7fab5c011bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:20:53.858 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:20:53.202+0000 7fab68ff9700 1 -- 172.21.15.52:0/418173470 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fab5c0038b0 con 0x7fab6c1068d0 2024-03-04T09:20:53.858 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:20:53.202+0000 7fab68ff9700 1 -- 172.21.15.52:0/418173470 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fab5c011e50 con 0x7fab6c1068d0 2024-03-04T09:20:53.858 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:20:53.202+0000 7fab7225a700 1 -- 172.21.15.52:0/418173470 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab6c19a000 con 0x7fab6c1068d0 2024-03-04T09:20:53.858 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:20:53.202+0000 7fab68ff9700 1 -- 172.21.15.52:0/418173470 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fab5c019550 con 0x7fab6c1068d0 2024-03-04T09:20:53.858 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:20:53.202+0000 7fab7225a700 1 -- 172.21.15.52:0/418173470 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab6c19ccf0 con 0x7fab6c1068d0 2024-03-04T09:20:53.858 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:20:53.202+0000 7fab68ff9700 1 -- 172.21.15.52:0/418173470 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7fab5c019b90 con 0x7fab6c1068d0 2024-03-04T09:20:53.858 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:20:53.202+0000 7fab68ff9700 1 -- 172.21.15.52:0/418173470 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fab5c024730 con 0x7fab6c1068d0 2024-03-04T09:20:53.859 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:20:53.203+0000 7fab7225a700 1 -- 172.21.15.52:0/418173470 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab6c069170 con 0x7fab6c1068d0 2024-03-04T09:20:53.859 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:20:53.205+0000 7fab68ff9700 1 -- 172.21.15.52:0/418173470 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7fab5c003a50 con 0x7fab6c1068d0 2024-03-04T09:20:53.859 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:20:53.279+0000 7fab7225a700 1 -- 172.21.15.52:0/418173470 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7fab6c192d90 con 0x7fab6c1068d0 2024-03-04T09:20:53.859 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:20:53.279+0000 7fab68ff9700 1 -- 172.21.15.52:0/418173470 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1239 (secure 0 0 0) 0x7fab6c192d90 con 0x7fab6c1068d0 2024-03-04T09:20:53.859 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:20:53.280+0000 7fab7225a700 1 -- 172.21.15.52:0/418173470 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fab6c1068d0 msgr2=0x7fab6c199830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:20:53.859 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:20:53.280+0000 7fab7225a700 1 --2- 172.21.15.52:0/418173470 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fab6c1068d0 0x7fab6c199830 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fab5c009fb0 tx=0x7fab5c011bf0 comp rx=0 tx=0).stop 2024-03-04T09:20:53.859 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:20:53.280+0000 7fab7225a700 1 -- 172.21.15.52:0/418173470 shutdown_connections 2024-03-04T09:20:53.859 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:20:53.280+0000 7fab7225a700 1 --2- 172.21.15.52:0/418173470 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fab6c1068d0 0x7fab6c199830 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:20:53.859 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:20:53.280+0000 7fab7225a700 1 -- 172.21.15.52:0/418173470 >> 172.21.15.52:0/418173470 conn(0x7fab6c101f70 msgr2=0x7fab6c18fe50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:20:53.859 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:20:53.281+0000 7fab7225a700 1 -- 172.21.15.52:0/418173470 shutdown_connections 2024-03-04T09:20:53.859 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:20:53.281+0000 7fab7225a700 1 -- 172.21.15.52:0/418173470 wait complete. 2024-03-04T09:20:53.859 INFO:teuthology.orchestra.run.smithi052.stdout:mgr not available, waiting (1/15)... 2024-03-04T09:20:54.127 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Mar 04 09:20:54 smithi052 ceph-4e059f08-da08-11ee-95c6-87774f69a715-mgr-x[32705]: 2024-03-04T09:20:54.034+0000 7fe55911a200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-04T09:20:54.461 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Mar 04 09:20:54 smithi052 ceph-4e059f08-da08-11ee-95c6-87774f69a715-mgr-x[32705]: 2024-03-04T09:20:54.130+0000 7fe55911a200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-04T09:20:54.461 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Mar 04 09:20:54 smithi052 ceph-4e059f08-da08-11ee-95c6-87774f69a715-mgr-x[32705]: 2024-03-04T09:20:54.282+0000 7fe55911a200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-04T09:20:54.461 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Mar 04 09:20:54 smithi052 ceph-4e059f08-da08-11ee-95c6-87774f69a715-mgr-x[32705]: 2024-03-04T09:20:54.379+0000 7fe55911a200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-04T09:20:54.462 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Mar 04 09:20:54 smithi052 ceph-4e059f08-da08-11ee-95c6-87774f69a715-mgr-x[32705]: 2024-03-04T09:20:54.464+0000 7fe55911a200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-04T09:20:55.026 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Mar 04 09:20:54 smithi052 ceph-4e059f08-da08-11ee-95c6-87774f69a715-mgr-x[32705]: 2024-03-04T09:20:54.730+0000 7fe55911a200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-04T09:20:55.026 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Mar 04 09:20:54 smithi052 ceph-4e059f08-da08-11ee-95c6-87774f69a715-mgr-x[32705]: 2024-03-04T09:20:54.844+0000 7fe55911a200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-04T09:20:55.026 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Mar 04 09:20:54 smithi052 ceph-4e059f08-da08-11ee-95c6-87774f69a715-mgr-x[32705]: 2024-03-04T09:20:54.932+0000 7fe55911a200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-04T09:20:55.026 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Mar 04 09:20:55 smithi052 ceph-4e059f08-da08-11ee-95c6-87774f69a715-mgr-x[32705]: 2024-03-04T09:20:55.028+0000 7fe55911a200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-04T09:20:55.334 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Mar 04 09:20:55 smithi052 ceph-4e059f08-da08-11ee-95c6-87774f69a715-mgr-x[32705]: 2024-03-04T09:20:55.206+0000 7fe55911a200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-04T09:20:56.585 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:20:56 smithi052 ceph-mon[32473]: from='client.? 172.21.15.52:0/2425951143' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-03-04T09:20:56.585 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Mar 04 09:20:56 smithi052 ceph-4e059f08-da08-11ee-95c6-87774f69a715-mgr-x[32705]: 2024-03-04T09:20:56.218+0000 7fe55911a200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-04T09:20:56.585 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Mar 04 09:20:56 smithi052 ceph-4e059f08-da08-11ee-95c6-87774f69a715-mgr-x[32705]: 2024-03-04T09:20:56.359+0000 7fe55911a200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-04T09:20:56.901 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout 2024-03-04T09:20:56.901 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout { 2024-03-04T09:20:56.901 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "fsid": "4e059f08-da08-11ee-95c6-87774f69a715", 2024-03-04T09:20:56.901 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "health": { 2024-03-04T09:20:56.901 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-03-04T09:20:56.901 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-03-04T09:20:56.901 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-03-04T09:20:56.901 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout }, 2024-03-04T09:20:56.901 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-03-04T09:20:56.901 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-03-04T09:20:56.901 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout 0 2024-03-04T09:20:56.902 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout ], 2024-03-04T09:20:56.902 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-03-04T09:20:56.902 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "a" 2024-03-04T09:20:56.902 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout ], 2024-03-04T09:20:56.902 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "quorum_age": 5, 2024-03-04T09:20:56.902 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "monmap": { 2024-03-04T09:20:56.902 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-04T09:20:56.902 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-03-04T09:20:56.902 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-03-04T09:20:56.902 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout }, 2024-03-04T09:20:56.902 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-03-04T09:20:56.902 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-04T09:20:56.902 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-03-04T09:20:56.902 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-03-04T09:20:56.903 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-03-04T09:20:56.910 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-03-04T09:20:56.910 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-03-04T09:20:56.911 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-03-04T09:20:56.911 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout }, 2024-03-04T09:20:56.911 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-03-04T09:20:56.911 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-03-04T09:20:56.911 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-03-04T09:20:56.911 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-03-04T09:20:56.911 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-03-04T09:20:56.911 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-03-04T09:20:56.911 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-03-04T09:20:56.912 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-03-04T09:20:56.912 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-03-04T09:20:56.912 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout }, 2024-03-04T09:20:56.912 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-03-04T09:20:56.912 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-04T09:20:56.912 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-03-04T09:20:56.912 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-03-04T09:20:56.912 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout }, 2024-03-04T09:20:56.912 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-03-04T09:20:56.912 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "available": false, 2024-03-04T09:20:56.913 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-03-04T09:20:56.913 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "modules": [ 2024-03-04T09:20:56.913 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "iostat", 2024-03-04T09:20:56.913 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "nfs", 2024-03-04T09:20:56.913 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "restful" 2024-03-04T09:20:56.913 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout ], 2024-03-04T09:20:56.913 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-04T09:20:56.913 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout }, 2024-03-04T09:20:56.913 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-03-04T09:20:56.913 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-04T09:20:56.913 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "modified": "2024-03-04T09:20:46.105424+0000", 2024-03-04T09:20:56.913 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-04T09:20:56.913 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout }, 2024-03-04T09:20:56.913 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-03-04T09:20:56.913 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout } 2024-03-04T09:20:56.914 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:20:56.252+0000 7f60138da700 1 Processor -- start 2024-03-04T09:20:56.914 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:20:56.252+0000 7f60138da700 1 -- start start 2024-03-04T09:20:56.914 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:20:56.252+0000 7f60138da700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f600c1062b0 0x7f600c1066d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:20:56.914 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:20:56.252+0000 7f60138da700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f600c07ae40 con 0x7f600c1062b0 2024-03-04T09:20:56.914 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:20:56.252+0000 7f6011676700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f600c1062b0 0x7f600c1066d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:20:56.914 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:20:56.252+0000 7f6011676700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f600c1062b0 0x7f600c1066d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:37462/0 (socket says 172.21.15.52:37462) 2024-03-04T09:20:56.914 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:20:56.252+0000 7f6011676700 1 -- 172.21.15.52:0/722221809 learned_addr learned my addr 172.21.15.52:0/722221809 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-04T09:20:56.914 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:20:56.252+0000 7f6011676700 1 -- 172.21.15.52:0/722221809 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f600c106c10 con 0x7f600c1062b0 2024-03-04T09:20:56.914 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:20:56.253+0000 7f6011676700 1 --2- 172.21.15.52:0/722221809 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f600c1062b0 0x7f600c1066d0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f5ffc0098a0 tx=0x7f5ffc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a50c1d38e87f057 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:20:56.914 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:20:56.253+0000 7f6003fff700 1 -- 172.21.15.52:0/722221809 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5ffc004160 con 0x7f600c1062b0 2024-03-04T09:20:56.914 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:20:56.253+0000 7f6003fff700 1 -- 172.21.15.52:0/722221809 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5ffc004300 con 0x7f600c1062b0 2024-03-04T09:20:56.914 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:20:56.253+0000 7f6003fff700 1 -- 172.21.15.52:0/722221809 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5ffc011b50 con 0x7f600c1062b0 2024-03-04T09:20:56.914 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:20:56.253+0000 7f60138da700 1 -- 172.21.15.52:0/722221809 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f600c1062b0 msgr2=0x7f600c1066d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:20:56.915 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:20:56.253+0000 7f60138da700 1 --2- 172.21.15.52:0/722221809 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f600c1062b0 0x7f600c1066d0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f5ffc0098a0 tx=0x7f5ffc009bb0 comp rx=0 tx=0).stop 2024-03-04T09:20:56.915 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:20:56.253+0000 7f60138da700 1 -- 172.21.15.52:0/722221809 shutdown_connections 2024-03-04T09:20:56.915 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:20:56.254+0000 7f60138da700 1 --2- 172.21.15.52:0/722221809 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f600c1062b0 0x7f600c1066d0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:20:56.915 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:20:56.254+0000 7f60138da700 1 -- 172.21.15.52:0/722221809 >> 172.21.15.52:0/722221809 conn(0x7f600c101f70 msgr2=0x7f600c104390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:20:56.915 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:20:56.254+0000 7f60138da700 1 -- 172.21.15.52:0/722221809 shutdown_connections 2024-03-04T09:20:56.915 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:20:56.254+0000 7f60138da700 1 -- 172.21.15.52:0/722221809 wait complete. 2024-03-04T09:20:56.915 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:20:56.254+0000 7f60138da700 1 Processor -- start 2024-03-04T09:20:56.915 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:20:56.254+0000 7f60138da700 1 -- start start 2024-03-04T09:20:56.915 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:20:56.255+0000 7f60138da700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f600c1062b0 0x7f600c19dba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:20:56.915 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:20:56.255+0000 7f60138da700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f600c107ca0 con 0x7f600c1062b0 2024-03-04T09:20:56.915 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:20:56.255+0000 7f6011676700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f600c1062b0 0x7f600c19dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:20:56.915 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:20:56.255+0000 7f6011676700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f600c1062b0 0x7f600c19dba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:37478/0 (socket says 172.21.15.52:37478) 2024-03-04T09:20:56.915 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:20:56.255+0000 7f6011676700 1 -- 172.21.15.52:0/2425951143 learned_addr learned my addr 172.21.15.52:0/2425951143 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-04T09:20:56.915 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:20:56.255+0000 7f6011676700 1 -- 172.21.15.52:0/2425951143 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f600c19e0e0 con 0x7f600c1062b0 2024-03-04T09:20:56.916 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:20:56.255+0000 7f6011676700 1 --2- 172.21.15.52:0/2425951143 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f600c1062b0 0x7f600c19dba0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f5ffc009fb0 tx=0x7f5ffc011bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:20:56.916 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:20:56.255+0000 7f60027fc700 1 -- 172.21.15.52:0/2425951143 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5ffc0038b0 con 0x7f600c1062b0 2024-03-04T09:20:56.916 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:20:56.255+0000 7f60027fc700 1 -- 172.21.15.52:0/2425951143 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5ffc011e50 con 0x7f600c1062b0 2024-03-04T09:20:56.916 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:20:56.255+0000 7f60027fc700 1 -- 172.21.15.52:0/2425951143 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5ffc019420 con 0x7f600c1062b0 2024-03-04T09:20:56.916 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:20:56.255+0000 7f60138da700 1 -- 172.21.15.52:0/2425951143 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f600c19e370 con 0x7f600c1062b0 2024-03-04T09:20:56.916 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:20:56.256+0000 7f60138da700 1 -- 172.21.15.52:0/2425951143 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f600c1a1060 con 0x7f600c1062b0 2024-03-04T09:20:56.916 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:20:56.256+0000 7f60027fc700 1 -- 172.21.15.52:0/2425951143 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f5ffc019950 con 0x7f600c1062b0 2024-03-04T09:20:56.916 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:20:56.256+0000 7f60027fc700 1 -- 172.21.15.52:0/2425951143 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f5ffc024730 con 0x7f600c1062b0 2024-03-04T09:20:56.916 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:20:56.256+0000 7f60138da700 1 -- 172.21.15.52:0/2425951143 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f600c069170 con 0x7f600c1062b0 2024-03-04T09:20:56.916 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:20:56.259+0000 7f60027fc700 1 -- 172.21.15.52:0/2425951143 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f5ffc043dd0 con 0x7f600c1062b0 2024-03-04T09:20:56.916 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:20:56.332+0000 7f60138da700 1 -- 172.21.15.52:0/2425951143 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f600c02dbf0 con 0x7f600c1062b0 2024-03-04T09:20:56.916 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:20:56.332+0000 7f60027fc700 1 -- 172.21.15.52:0/2425951143 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1239 (secure 0 0 0) 0x7f5ffc032030 con 0x7f600c1062b0 2024-03-04T09:20:56.916 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:20:56.333+0000 7f60138da700 1 -- 172.21.15.52:0/2425951143 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f600c1062b0 msgr2=0x7f600c19dba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:20:56.917 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:20:56.333+0000 7f60138da700 1 --2- 172.21.15.52:0/2425951143 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f600c1062b0 0x7f600c19dba0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f5ffc009fb0 tx=0x7f5ffc011bf0 comp rx=0 tx=0).stop 2024-03-04T09:20:56.917 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:20:56.333+0000 7f60138da700 1 -- 172.21.15.52:0/2425951143 shutdown_connections 2024-03-04T09:20:56.917 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:20:56.333+0000 7f60138da700 1 --2- 172.21.15.52:0/2425951143 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f600c1062b0 0x7f600c19dba0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:20:56.917 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:20:56.333+0000 7f60138da700 1 -- 172.21.15.52:0/2425951143 >> 172.21.15.52:0/2425951143 conn(0x7f600c101f70 msgr2=0x7f600c194810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:20:56.917 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:20:56.333+0000 7f60138da700 1 -- 172.21.15.52:0/2425951143 shutdown_connections 2024-03-04T09:20:56.917 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:20:56.333+0000 7f60138da700 1 -- 172.21.15.52:0/2425951143 wait complete. 2024-03-04T09:20:56.917 INFO:teuthology.orchestra.run.smithi052.stdout:mgr not available, waiting (2/15)... 2024-03-04T09:20:56.967 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Mar 04 09:20:56 smithi052 ceph-4e059f08-da08-11ee-95c6-87774f69a715-mgr-x[32705]: 2024-03-04T09:20:56.701+0000 7fe55911a200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-04T09:20:56.967 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Mar 04 09:20:56 smithi052 ceph-4e059f08-da08-11ee-95c6-87774f69a715-mgr-x[32705]: 2024-03-04T09:20:56.792+0000 7fe55911a200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-04T09:20:57.334 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Mar 04 09:20:56 smithi052 ceph-4e059f08-da08-11ee-95c6-87774f69a715-mgr-x[32705]: 2024-03-04T09:20:56.996+0000 7fe55911a200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-04T09:20:57.335 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Mar 04 09:20:57 smithi052 ceph-4e059f08-da08-11ee-95c6-87774f69a715-mgr-x[32705]: 2024-03-04T09:20:57.185+0000 7fe55911a200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-04T09:20:57.734 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Mar 04 09:20:57 smithi052 ceph-4e059f08-da08-11ee-95c6-87774f69a715-mgr-x[32705]: 2024-03-04T09:20:57.475+0000 7fe55911a200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-04T09:20:58.084 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Mar 04 09:20:57 smithi052 ceph-4e059f08-da08-11ee-95c6-87774f69a715-mgr-x[32705]: 2024-03-04T09:20:57.737+0000 7fe55911a200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-04T09:20:58.085 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Mar 04 09:20:57 smithi052 ceph-4e059f08-da08-11ee-95c6-87774f69a715-mgr-x[32705]: 2024-03-04T09:20:57.980+0000 7fe55911a200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-04T09:20:59.084 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Mar 04 09:20:58 smithi052 ceph-4e059f08-da08-11ee-95c6-87774f69a715-mgr-x[32705]: 2024-03-04T09:20:58.806+0000 7fe55911a200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-04T09:20:59.434 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Mar 04 09:20:59 smithi052 ceph-4e059f08-da08-11ee-95c6-87774f69a715-mgr-x[32705]: 2024-03-04T09:20:59.394+0000 7fe55911a200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-04T09:20:59.717 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:20:59 smithi052 ceph-mon[32473]: from='client.? 172.21.15.52:0/615578744' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-03-04T09:20:59.717 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Mar 04 09:20:59 smithi052 ceph-4e059f08-da08-11ee-95c6-87774f69a715-mgr-x[32705]: 2024-03-04T09:20:59.499+0000 7fe55911a200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-04T09:20:59.717 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Mar 04 09:20:59 smithi052 ceph-4e059f08-da08-11ee-95c6-87774f69a715-mgr-x[32705]: 2024-03-04T09:20:59.720+0000 7fe55911a200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-04T09:21:00.090 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout 2024-03-04T09:21:00.090 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout { 2024-03-04T09:21:00.090 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "fsid": "4e059f08-da08-11ee-95c6-87774f69a715", 2024-03-04T09:21:00.090 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "health": { 2024-03-04T09:21:00.090 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-03-04T09:21:00.091 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-03-04T09:21:00.091 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-03-04T09:21:00.091 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout }, 2024-03-04T09:21:00.091 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-03-04T09:21:00.091 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-03-04T09:21:00.091 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout 0 2024-03-04T09:21:00.091 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout ], 2024-03-04T09:21:00.091 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-03-04T09:21:00.091 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "a" 2024-03-04T09:21:00.091 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout ], 2024-03-04T09:21:00.091 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "quorum_age": 8, 2024-03-04T09:21:00.091 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "monmap": { 2024-03-04T09:21:00.092 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-04T09:21:00.092 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-03-04T09:21:00.092 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-03-04T09:21:00.092 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout }, 2024-03-04T09:21:00.092 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-03-04T09:21:00.092 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-04T09:21:00.092 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-03-04T09:21:00.092 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-03-04T09:21:00.092 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-03-04T09:21:00.092 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-03-04T09:21:00.092 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-03-04T09:21:00.093 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-03-04T09:21:00.093 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout }, 2024-03-04T09:21:00.093 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-03-04T09:21:00.093 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-03-04T09:21:00.093 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-03-04T09:21:00.093 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-03-04T09:21:00.093 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-03-04T09:21:00.093 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-03-04T09:21:00.093 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-03-04T09:21:00.093 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-03-04T09:21:00.093 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-03-04T09:21:00.093 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout }, 2024-03-04T09:21:00.093 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-03-04T09:21:00.094 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-04T09:21:00.094 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-03-04T09:21:00.094 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-03-04T09:21:00.094 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout }, 2024-03-04T09:21:00.094 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-03-04T09:21:00.094 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "available": false, 2024-03-04T09:21:00.094 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-03-04T09:21:00.094 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "modules": [ 2024-03-04T09:21:00.094 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "iostat", 2024-03-04T09:21:00.094 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "nfs", 2024-03-04T09:21:00.094 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "restful" 2024-03-04T09:21:00.094 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout ], 2024-03-04T09:21:00.094 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-04T09:21:00.094 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout }, 2024-03-04T09:21:00.095 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-03-04T09:21:00.095 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-04T09:21:00.095 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "modified": "2024-03-04T09:20:46.105424+0000", 2024-03-04T09:21:00.095 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-04T09:21:00.095 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout }, 2024-03-04T09:21:00.095 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-03-04T09:21:00.095 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout } 2024-03-04T09:21:00.095 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:20:59.358+0000 7ff66b5ae700 1 Processor -- start 2024-03-04T09:21:00.095 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:20:59.358+0000 7ff66b5ae700 1 -- start start 2024-03-04T09:21:00.095 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:20:59.358+0000 7ff66b5ae700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff664107a60 0x7ff664107e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:21:00.095 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:20:59.358+0000 7ff66b5ae700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff664108450 con 0x7ff664107a60 2024-03-04T09:21:00.095 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:20:59.358+0000 7ff66934a700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff664107a60 0x7ff664107e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:21:00.096 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:20:59.358+0000 7ff66934a700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff664107a60 0x7ff664107e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:37494/0 (socket says 172.21.15.52:37494) 2024-03-04T09:21:00.096 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:20:59.358+0000 7ff66934a700 1 -- 172.21.15.52:0/1935558510 learned_addr learned my addr 172.21.15.52:0/1935558510 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-04T09:21:00.096 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:20:59.358+0000 7ff66934a700 1 -- 172.21.15.52:0/1935558510 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff664108c10 con 0x7ff664107a60 2024-03-04T09:21:00.096 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:20:59.359+0000 7ff66934a700 1 --2- 172.21.15.52:0/1935558510 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff664107a60 0x7ff664107e80 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7ff6600098a0 tx=0x7ff660009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31d51f0e76b9d0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:21:00.096 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:20:59.359+0000 7ff65bfff700 1 -- 172.21.15.52:0/1935558510 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff660004160 con 0x7ff664107a60 2024-03-04T09:21:00.096 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:20:59.359+0000 7ff65bfff700 1 -- 172.21.15.52:0/1935558510 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff660004300 con 0x7ff664107a60 2024-03-04T09:21:00.096 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:20:59.359+0000 7ff65bfff700 1 -- 172.21.15.52:0/1935558510 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff660011b50 con 0x7ff664107a60 2024-03-04T09:21:00.096 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:20:59.359+0000 7ff66b5ae700 1 -- 172.21.15.52:0/1935558510 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff664107a60 msgr2=0x7ff664107e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:21:00.096 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:20:59.359+0000 7ff66b5ae700 1 --2- 172.21.15.52:0/1935558510 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff664107a60 0x7ff664107e80 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7ff6600098a0 tx=0x7ff660009bb0 comp rx=0 tx=0).stop 2024-03-04T09:21:00.096 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:20:59.359+0000 7ff66b5ae700 1 -- 172.21.15.52:0/1935558510 shutdown_connections 2024-03-04T09:21:00.097 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:20:59.359+0000 7ff66b5ae700 1 --2- 172.21.15.52:0/1935558510 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff664107a60 0x7ff664107e80 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:21:00.097 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:20:59.359+0000 7ff66b5ae700 1 -- 172.21.15.52:0/1935558510 >> 172.21.15.52:0/1935558510 conn(0x7ff664103060 msgr2=0x7ff664105480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:21:00.097 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:20:59.360+0000 7ff66b5ae700 1 -- 172.21.15.52:0/1935558510 shutdown_connections 2024-03-04T09:21:00.097 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:20:59.360+0000 7ff66b5ae700 1 -- 172.21.15.52:0/1935558510 wait complete. 2024-03-04T09:21:00.097 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:20:59.360+0000 7ff66b5ae700 1 Processor -- start 2024-03-04T09:21:00.097 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:20:59.360+0000 7ff66b5ae700 1 -- start start 2024-03-04T09:21:00.097 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:20:59.360+0000 7ff66b5ae700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff664107a60 0x7ff66419b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:21:00.097 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:20:59.360+0000 7ff66b5ae700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff66410bfc0 con 0x7ff664107a60 2024-03-04T09:21:00.097 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:20:59.360+0000 7ff66934a700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff664107a60 0x7ff66419b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:21:00.097 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:20:59.361+0000 7ff66934a700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff664107a60 0x7ff66419b9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:37498/0 (socket says 172.21.15.52:37498) 2024-03-04T09:21:00.097 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:20:59.361+0000 7ff66934a700 1 -- 172.21.15.52:0/615578744 learned_addr learned my addr 172.21.15.52:0/615578744 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-04T09:21:00.097 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:20:59.361+0000 7ff66934a700 1 -- 172.21.15.52:0/615578744 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff66419bee0 con 0x7ff664107a60 2024-03-04T09:21:00.098 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:20:59.361+0000 7ff66934a700 1 --2- 172.21.15.52:0/615578744 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff664107a60 0x7ff66419b9a0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7ff660011670 tx=0x7ff660011b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:21:00.098 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:20:59.361+0000 7ff65a7fc700 1 -- 172.21.15.52:0/615578744 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff6600039c0 con 0x7ff664107a60 2024-03-04T09:21:00.098 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:20:59.361+0000 7ff65a7fc700 1 -- 172.21.15.52:0/615578744 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff660011e50 con 0x7ff664107a60 2024-03-04T09:21:00.098 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:20:59.361+0000 7ff66b5ae700 1 -- 172.21.15.52:0/615578744 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff66419c170 con 0x7ff664107a60 2024-03-04T09:21:00.098 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:20:59.361+0000 7ff65a7fc700 1 -- 172.21.15.52:0/615578744 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff66001a5c0 con 0x7ff664107a60 2024-03-04T09:21:00.098 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:20:59.361+0000 7ff66b5ae700 1 -- 172.21.15.52:0/615578744 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff66419ee60 con 0x7ff664107a60 2024-03-04T09:21:00.098 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:20:59.362+0000 7ff65a7fc700 1 -- 172.21.15.52:0/615578744 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7ff66001abc0 con 0x7ff664107a60 2024-03-04T09:21:00.098 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:20:59.362+0000 7ff65a7fc700 1 -- 172.21.15.52:0/615578744 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7ff660019700 con 0x7ff664107a60 2024-03-04T09:21:00.098 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:20:59.362+0000 7ff66b5ae700 1 -- 172.21.15.52:0/615578744 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff664069170 con 0x7ff664107a60 2024-03-04T09:21:00.098 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:20:59.364+0000 7ff65a7fc700 1 -- 172.21.15.52:0/615578744 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7ff660003b60 con 0x7ff664107a60 2024-03-04T09:21:00.098 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:20:59.437+0000 7ff66b5ae700 1 -- 172.21.15.52:0/615578744 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7ff66419f3a0 con 0x7ff664107a60 2024-03-04T09:21:00.098 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:20:59.437+0000 7ff65a7fc700 1 -- 172.21.15.52:0/615578744 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1239 (secure 0 0 0) 0x7ff660019950 con 0x7ff664107a60 2024-03-04T09:21:00.099 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:20:59.438+0000 7ff66b5ae700 1 -- 172.21.15.52:0/615578744 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff664107a60 msgr2=0x7ff66419b9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:21:00.099 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:20:59.438+0000 7ff66b5ae700 1 --2- 172.21.15.52:0/615578744 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff664107a60 0x7ff66419b9a0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7ff660011670 tx=0x7ff660011b60 comp rx=0 tx=0).stop 2024-03-04T09:21:00.099 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:20:59.439+0000 7ff66b5ae700 1 -- 172.21.15.52:0/615578744 shutdown_connections 2024-03-04T09:21:00.099 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:20:59.439+0000 7ff66b5ae700 1 --2- 172.21.15.52:0/615578744 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff664107a60 0x7ff66419b9a0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:21:00.099 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:20:59.439+0000 7ff66b5ae700 1 -- 172.21.15.52:0/615578744 >> 172.21.15.52:0/615578744 conn(0x7ff664103060 msgr2=0x7ff664103ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:21:00.099 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:20:59.439+0000 7ff66b5ae700 1 -- 172.21.15.52:0/615578744 shutdown_connections 2024-03-04T09:21:00.099 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:20:59.439+0000 7ff66b5ae700 1 -- 172.21.15.52:0/615578744 wait complete. 2024-03-04T09:21:00.099 INFO:teuthology.orchestra.run.smithi052.stdout:mgr not available, waiting (3/15)... 2024-03-04T09:21:00.835 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:21:00 smithi052 ceph-mon[32473]: Activating manager daemon x 2024-03-04T09:21:00.835 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:21:00 smithi052 ceph-mon[32473]: mgrmap e2: x(active, starting, since 0.00291033s) 2024-03-04T09:21:00.835 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:21:00 smithi052 ceph-mon[32473]: from='mgr.14100 172.21.15.52:0/2110597076' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-04T09:21:00.835 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:21:00 smithi052 ceph-mon[32473]: from='mgr.14100 172.21.15.52:0/2110597076' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-04T09:21:00.835 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:21:00 smithi052 ceph-mon[32473]: from='mgr.14100 172.21.15.52:0/2110597076' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-04T09:21:00.835 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:21:00 smithi052 ceph-mon[32473]: from='mgr.14100 172.21.15.52:0/2110597076' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-04T09:21:00.835 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:21:00 smithi052 ceph-mon[32473]: from='mgr.14100 172.21.15.52:0/2110597076' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-03-04T09:21:00.835 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:21:00 smithi052 ceph-mon[32473]: Manager daemon x is now available 2024-03-04T09:21:00.835 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:21:00 smithi052 ceph-mon[32473]: from='mgr.14100 172.21.15.52:0/2110597076' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-03-04T09:21:00.836 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:21:00 smithi052 ceph-mon[32473]: from='mgr.14100 172.21.15.52:0/2110597076' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-03-04T09:21:00.836 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:21:00 smithi052 ceph-mon[32473]: from='mgr.14100 172.21.15.52:0/2110597076' entity='mgr.x' 2024-03-04T09:21:00.836 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:21:00 smithi052 ceph-mon[32473]: from='mgr.14100 172.21.15.52:0/2110597076' entity='mgr.x' 2024-03-04T09:21:00.836 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:21:00 smithi052 ceph-mon[32473]: from='mgr.14100 172.21.15.52:0/2110597076' entity='mgr.x' 2024-03-04T09:21:02.021 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:21:01 smithi052 ceph-mon[32473]: mgrmap e3: x(active, since 1.00732s) 2024-03-04T09:21:03.367 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout 2024-03-04T09:21:03.367 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout { 2024-03-04T09:21:03.367 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "fsid": "4e059f08-da08-11ee-95c6-87774f69a715", 2024-03-04T09:21:03.367 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "health": { 2024-03-04T09:21:03.367 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-03-04T09:21:03.367 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-03-04T09:21:03.367 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-03-04T09:21:03.367 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout }, 2024-03-04T09:21:03.367 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-03-04T09:21:03.367 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-03-04T09:21:03.367 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout 0 2024-03-04T09:21:03.367 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout ], 2024-03-04T09:21:03.368 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-03-04T09:21:03.368 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "a" 2024-03-04T09:21:03.368 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout ], 2024-03-04T09:21:03.368 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "quorum_age": 12, 2024-03-04T09:21:03.368 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "monmap": { 2024-03-04T09:21:03.368 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-04T09:21:03.368 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-03-04T09:21:03.368 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-03-04T09:21:03.368 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout }, 2024-03-04T09:21:03.368 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-03-04T09:21:03.368 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-04T09:21:03.368 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-03-04T09:21:03.368 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-03-04T09:21:03.368 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-03-04T09:21:03.369 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-03-04T09:21:03.369 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-03-04T09:21:03.369 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-03-04T09:21:03.369 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout }, 2024-03-04T09:21:03.369 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-03-04T09:21:03.369 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-03-04T09:21:03.369 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-03-04T09:21:03.369 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-03-04T09:21:03.369 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-03-04T09:21:03.369 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-03-04T09:21:03.369 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-03-04T09:21:03.369 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-03-04T09:21:03.369 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-03-04T09:21:03.369 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout }, 2024-03-04T09:21:03.370 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-03-04T09:21:03.370 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-04T09:21:03.370 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-03-04T09:21:03.370 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-03-04T09:21:03.370 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout }, 2024-03-04T09:21:03.370 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-03-04T09:21:03.370 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "available": true, 2024-03-04T09:21:03.370 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-03-04T09:21:03.370 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "modules": [ 2024-03-04T09:21:03.370 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "iostat", 2024-03-04T09:21:03.370 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "nfs", 2024-03-04T09:21:03.370 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "restful" 2024-03-04T09:21:03.370 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout ], 2024-03-04T09:21:03.370 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-04T09:21:03.371 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout }, 2024-03-04T09:21:03.371 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-03-04T09:21:03.371 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-04T09:21:03.371 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "modified": "2024-03-04T09:20:46.105424+0000", 2024-03-04T09:21:03.371 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-04T09:21:03.371 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout }, 2024-03-04T09:21:03.371 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-03-04T09:21:03.371 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout } 2024-03-04T09:21:03.371 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:02.500+0000 7ff9fa278700 1 Processor -- start 2024-03-04T09:21:03.371 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:02.500+0000 7ff9fa278700 1 -- start start 2024-03-04T09:21:03.371 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:02.500+0000 7ff9fa278700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff9f41068a0 0x7ff9f4106cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:21:03.371 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:02.500+0000 7ff9fa278700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9f4107290 con 0x7ff9f41068a0 2024-03-04T09:21:03.371 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:02.500+0000 7ff9f3fff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff9f41068a0 0x7ff9f4106cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:21:03.371 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:02.500+0000 7ff9f3fff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff9f41068a0 0x7ff9f4106cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:37580/0 (socket says 172.21.15.52:37580) 2024-03-04T09:21:03.372 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:02.501+0000 7ff9f3fff700 1 -- 172.21.15.52:0/1114554437 learned_addr learned my addr 172.21.15.52:0/1114554437 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-04T09:21:03.372 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:02.501+0000 7ff9f3fff700 1 -- 172.21.15.52:0/1114554437 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9f4107aa0 con 0x7ff9f41068a0 2024-03-04T09:21:03.372 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:02.501+0000 7ff9f3fff700 1 --2- 172.21.15.52:0/1114554437 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff9f41068a0 0x7ff9f4106cc0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7ff9e40098a0 tx=0x7ff9e4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ddd13996aef65eab server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:21:03.372 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:02.501+0000 7ff9f2ffd700 1 -- 172.21.15.52:0/1114554437 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff9e4004160 con 0x7ff9f41068a0 2024-03-04T09:21:03.372 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:02.501+0000 7ff9f2ffd700 1 -- 172.21.15.52:0/1114554437 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff9e4004300 con 0x7ff9f41068a0 2024-03-04T09:21:03.372 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:02.501+0000 7ff9f2ffd700 1 -- 172.21.15.52:0/1114554437 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff9e4011b50 con 0x7ff9f41068a0 2024-03-04T09:21:03.372 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:02.502+0000 7ff9fa278700 1 -- 172.21.15.52:0/1114554437 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff9f41068a0 msgr2=0x7ff9f4106cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:21:03.372 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:02.502+0000 7ff9fa278700 1 --2- 172.21.15.52:0/1114554437 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff9f41068a0 0x7ff9f4106cc0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7ff9e40098a0 tx=0x7ff9e4009bb0 comp rx=0 tx=0).stop 2024-03-04T09:21:03.372 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:02.502+0000 7ff9fa278700 1 -- 172.21.15.52:0/1114554437 shutdown_connections 2024-03-04T09:21:03.372 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:02.502+0000 7ff9fa278700 1 --2- 172.21.15.52:0/1114554437 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff9f41068a0 0x7ff9f4106cc0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:21:03.372 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:02.502+0000 7ff9fa278700 1 -- 172.21.15.52:0/1114554437 >> 172.21.15.52:0/1114554437 conn(0x7ff9f4101ee0 msgr2=0x7ff9f4104300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:21:03.372 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:02.502+0000 7ff9fa278700 1 -- 172.21.15.52:0/1114554437 shutdown_connections 2024-03-04T09:21:03.372 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:02.502+0000 7ff9fa278700 1 -- 172.21.15.52:0/1114554437 wait complete. 2024-03-04T09:21:03.372 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:02.502+0000 7ff9fa278700 1 Processor -- start 2024-03-04T09:21:03.373 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:02.502+0000 7ff9fa278700 1 -- start start 2024-03-04T09:21:03.373 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:02.503+0000 7ff9fa278700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff9f41068a0 0x7ff9f407aac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:21:03.373 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:02.503+0000 7ff9fa278700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9f410ae50 con 0x7ff9f41068a0 2024-03-04T09:21:03.373 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:02.503+0000 7ff9f3fff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff9f41068a0 0x7ff9f407aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:21:03.373 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:02.503+0000 7ff9f3fff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff9f41068a0 0x7ff9f407aac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:37594/0 (socket says 172.21.15.52:37594) 2024-03-04T09:21:03.373 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:02.503+0000 7ff9f3fff700 1 -- 172.21.15.52:0/395247832 learned_addr learned my addr 172.21.15.52:0/395247832 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-04T09:21:03.373 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:02.503+0000 7ff9f3fff700 1 -- 172.21.15.52:0/395247832 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9f419e6e0 con 0x7ff9f41068a0 2024-03-04T09:21:03.373 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:02.503+0000 7ff9f3fff700 1 --2- 172.21.15.52:0/395247832 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff9f41068a0 0x7ff9f407aac0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7ff9e4011850 tx=0x7ff9e4011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:21:03.373 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:02.503+0000 7ff9f17fa700 1 -- 172.21.15.52:0/395247832 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff9e4003af0 con 0x7ff9f41068a0 2024-03-04T09:21:03.373 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:02.503+0000 7ff9f17fa700 1 -- 172.21.15.52:0/395247832 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff9e4020070 con 0x7ff9f41068a0 2024-03-04T09:21:03.373 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:02.503+0000 7ff9fa278700 1 -- 172.21.15.52:0/395247832 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9f419e8b0 con 0x7ff9f41068a0 2024-03-04T09:21:03.373 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:02.503+0000 7ff9f17fa700 1 -- 172.21.15.52:0/395247832 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff9e40195a0 con 0x7ff9f41068a0 2024-03-04T09:21:03.373 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:02.504+0000 7ff9fa278700 1 -- 172.21.15.52:0/395247832 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9f419ecd0 con 0x7ff9f41068a0 2024-03-04T09:21:03.374 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:02.504+0000 7ff9f17fa700 1 -- 172.21.15.52:0/395247832 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 3) v1 ==== 45811+0+0 (secure 0 0 0) 0x7ff9e4019850 con 0x7ff9f41068a0 2024-03-04T09:21:03.374 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:02.505+0000 7ff9fa278700 1 -- 172.21.15.52:0/395247832 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff9f4069170 con 0x7ff9f41068a0 2024-03-04T09:21:03.374 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:02.505+0000 7ff9f17fa700 1 --2- 172.21.15.52:0/395247832 >> [v2:172.21.15.52:6800/2379361333,v1:172.21.15.52:6801/2379361333] conn(0x7ff9e0038a80 0x7ff9e003af40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:21:03.374 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:02.505+0000 7ff9f17fa700 1 -- 172.21.15.52:0/395247832 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7ff9e4011e00 con 0x7ff9f41068a0 2024-03-04T09:21:03.374 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:02.505+0000 7ff9f37fe700 1 --2- 172.21.15.52:0/395247832 >> [v2:172.21.15.52:6800/2379361333,v1:172.21.15.52:6801/2379361333] conn(0x7ff9e0038a80 0x7ff9e003af40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:21:03.374 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:02.505+0000 7ff9f37fe700 1 --2- 172.21.15.52:0/395247832 >> [v2:172.21.15.52:6800/2379361333,v1:172.21.15.52:6801/2379361333] conn(0x7ff9e0038a80 0x7ff9e003af40 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7ff9dc006fd0 tx=0x7ff9dc006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:21:03.374 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:02.510+0000 7ff9f17fa700 1 -- 172.21.15.52:0/395247832 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7ff9e4003c90 con 0x7ff9f41068a0 2024-03-04T09:21:03.374 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:02.739+0000 7ff9f17fa700 1 -- 172.21.15.52:0/395247832 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mgrmap(e 4) v1 ==== 45917+0+0 (secure 0 0 0) 0x7ff9e400b430 con 0x7ff9f41068a0 2024-03-04T09:21:03.374 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:02.802+0000 7ff9fa278700 1 -- 172.21.15.52:0/395247832 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7ff9f419f4b0 con 0x7ff9f41068a0 2024-03-04T09:21:03.374 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:02.803+0000 7ff9f17fa700 1 -- 172.21.15.52:0/395247832 <== mon.0 v2:172.21.15.52:3300/0 8 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1239 (secure 0 0 0) 0x7ff9e40443d0 con 0x7ff9f41068a0 2024-03-04T09:21:03.374 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:02.805+0000 7ff9fa278700 1 -- 172.21.15.52:0/395247832 >> [v2:172.21.15.52:6800/2379361333,v1:172.21.15.52:6801/2379361333] conn(0x7ff9e0038a80 msgr2=0x7ff9e003af40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:21:03.374 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:02.805+0000 7ff9fa278700 1 --2- 172.21.15.52:0/395247832 >> [v2:172.21.15.52:6800/2379361333,v1:172.21.15.52:6801/2379361333] conn(0x7ff9e0038a80 0x7ff9e003af40 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7ff9dc006fd0 tx=0x7ff9dc006e40 comp rx=0 tx=0).stop 2024-03-04T09:21:03.374 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:02.805+0000 7ff9fa278700 1 -- 172.21.15.52:0/395247832 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff9f41068a0 msgr2=0x7ff9f407aac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:21:03.374 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:02.805+0000 7ff9fa278700 1 --2- 172.21.15.52:0/395247832 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff9f41068a0 0x7ff9f407aac0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7ff9e4011850 tx=0x7ff9e4011c10 comp rx=0 tx=0).stop 2024-03-04T09:21:03.374 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:02.805+0000 7ff9fa278700 1 -- 172.21.15.52:0/395247832 shutdown_connections 2024-03-04T09:21:03.375 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:02.805+0000 7ff9fa278700 1 --2- 172.21.15.52:0/395247832 >> [v2:172.21.15.52:6800/2379361333,v1:172.21.15.52:6801/2379361333] conn(0x7ff9e0038a80 0x7ff9e003af40 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:21:03.375 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:02.805+0000 7ff9fa278700 1 --2- 172.21.15.52:0/395247832 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff9f41068a0 0x7ff9f407aac0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:21:03.375 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:02.805+0000 7ff9fa278700 1 -- 172.21.15.52:0/395247832 >> 172.21.15.52:0/395247832 conn(0x7ff9f4101ee0 msgr2=0x7ff9f4102a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:21:03.375 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:02.805+0000 7ff9fa278700 1 -- 172.21.15.52:0/395247832 shutdown_connections 2024-03-04T09:21:03.375 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:02.805+0000 7ff9fa278700 1 -- 172.21.15.52:0/395247832 wait complete. 2024-03-04T09:21:03.375 INFO:teuthology.orchestra.run.smithi052.stdout:mgr is available 2024-03-04T09:21:03.989 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:21:03 smithi052 ceph-mon[32473]: mgrmap e4: x(active, since 3s) 2024-03-04T09:21:03.989 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:21:03 smithi052 ceph-mon[32473]: from='client.? 172.21.15.52:0/395247832' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-03-04T09:21:04.509 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout 2024-03-04T09:21:04.509 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout [global] 2024-03-04T09:21:04.509 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout fsid = 4e059f08-da08-11ee-95c6-87774f69a715 2024-03-04T09:21:04.509 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-03-04T09:21:04.509 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-03-04T09:21:04.509 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-03-04T09:21:04.509 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-03-04T09:21:04.509 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout 2024-03-04T09:21:04.509 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout [mgr] 2024-03-04T09:21:04.510 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-03-04T09:21:04.510 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout 2024-03-04T09:21:04.510 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout [osd] 2024-03-04T09:21:04.510 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-03-04T09:21:04.510 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-03-04T09:21:04.510 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-03-04T09:21:04.510 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-03-04T09:21:04.510 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-03-04T09:21:04.510 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:03.782+0000 7f84b9985700 1 Processor -- start 2024-03-04T09:21:04.510 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:03.783+0000 7f84b9985700 1 -- start start 2024-03-04T09:21:04.510 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:03.783+0000 7f84b9985700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f84b4107a90 0x7f84b4107eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:21:04.510 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:03.783+0000 7f84b9985700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84b4108480 con 0x7f84b4107a90 2024-03-04T09:21:04.510 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:03.783+0000 7f84b2ffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f84b4107a90 0x7f84b4107eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:21:04.511 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:03.783+0000 7f84b2ffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f84b4107a90 0x7f84b4107eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:37600/0 (socket says 172.21.15.52:37600) 2024-03-04T09:21:04.511 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:03.783+0000 7f84b2ffd700 1 -- 172.21.15.52:0/1482228377 learned_addr learned my addr 172.21.15.52:0/1482228377 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-04T09:21:04.511 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:03.783+0000 7f84b2ffd700 1 -- 172.21.15.52:0/1482228377 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84b4108c40 con 0x7f84b4107a90 2024-03-04T09:21:04.511 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:03.783+0000 7f84b2ffd700 1 --2- 172.21.15.52:0/1482228377 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f84b4107a90 0x7f84b4107eb0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f84a40098a0 tx=0x7f84a4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5db9b6cd2062a2e server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:21:04.511 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:03.783+0000 7f84b1ffb700 1 -- 172.21.15.52:0/1482228377 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f84a4004160 con 0x7f84b4107a90 2024-03-04T09:21:04.511 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:03.784+0000 7f84b1ffb700 1 -- 172.21.15.52:0/1482228377 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f84a4004300 con 0x7f84b4107a90 2024-03-04T09:21:04.511 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:03.784+0000 7f84b1ffb700 1 -- 172.21.15.52:0/1482228377 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f84a4011b50 con 0x7f84b4107a90 2024-03-04T09:21:04.511 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:03.784+0000 7f84b9985700 1 -- 172.21.15.52:0/1482228377 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f84b4107a90 msgr2=0x7f84b4107eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:21:04.511 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:03.784+0000 7f84b9985700 1 --2- 172.21.15.52:0/1482228377 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f84b4107a90 0x7f84b4107eb0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f84a40098a0 tx=0x7f84a4009bb0 comp rx=0 tx=0).stop 2024-03-04T09:21:04.511 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:03.784+0000 7f84b9985700 1 -- 172.21.15.52:0/1482228377 shutdown_connections 2024-03-04T09:21:04.511 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:03.784+0000 7f84b9985700 1 --2- 172.21.15.52:0/1482228377 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f84b4107a90 0x7f84b4107eb0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:21:04.511 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:03.784+0000 7f84b9985700 1 -- 172.21.15.52:0/1482228377 >> 172.21.15.52:0/1482228377 conn(0x7f84b4103090 msgr2=0x7f84b41054b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:21:04.511 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:03.784+0000 7f84b9985700 1 -- 172.21.15.52:0/1482228377 shutdown_connections 2024-03-04T09:21:04.511 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:03.784+0000 7f84b9985700 1 -- 172.21.15.52:0/1482228377 wait complete. 2024-03-04T09:21:04.512 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:03.785+0000 7f84b9985700 1 Processor -- start 2024-03-04T09:21:04.512 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:03.785+0000 7f84b9985700 1 -- start start 2024-03-04T09:21:04.512 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:03.785+0000 7f84b9985700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f84b4107a90 0x7f84b419b9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:21:04.512 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:03.785+0000 7f84b9985700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84b410bff0 con 0x7f84b4107a90 2024-03-04T09:21:04.512 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:03.785+0000 7f84b2ffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f84b4107a90 0x7f84b419b9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:21:04.512 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:03.785+0000 7f84b2ffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f84b4107a90 0x7f84b419b9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:37616/0 (socket says 172.21.15.52:37616) 2024-03-04T09:21:04.512 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:03.785+0000 7f84b2ffd700 1 -- 172.21.15.52:0/1843605130 learned_addr learned my addr 172.21.15.52:0/1843605130 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-04T09:21:04.512 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:03.785+0000 7f84b2ffd700 1 -- 172.21.15.52:0/1843605130 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84b419bf30 con 0x7f84b4107a90 2024-03-04T09:21:04.512 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:03.786+0000 7f84b2ffd700 1 --2- 172.21.15.52:0/1843605130 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f84b4107a90 0x7f84b419b9f0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f84a4009fb0 tx=0x7f84a4011bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:21:04.512 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:03.786+0000 7f84b8983700 1 -- 172.21.15.52:0/1843605130 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f84a40038b0 con 0x7f84b4107a90 2024-03-04T09:21:04.512 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:03.786+0000 7f84b9985700 1 -- 172.21.15.52:0/1843605130 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84b419c1c0 con 0x7f84b4107a90 2024-03-04T09:21:04.512 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:03.786+0000 7f84b8983700 1 -- 172.21.15.52:0/1843605130 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f84a4011e50 con 0x7f84b4107a90 2024-03-04T09:21:04.512 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:03.786+0000 7f84b8983700 1 -- 172.21.15.52:0/1843605130 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f84a4019550 con 0x7f84b4107a90 2024-03-04T09:21:04.513 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:03.786+0000 7f84b9985700 1 -- 172.21.15.52:0/1843605130 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84b419eeb0 con 0x7f84b4107a90 2024-03-04T09:21:04.513 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:03.787+0000 7f84b8983700 1 -- 172.21.15.52:0/1843605130 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 4) v1 ==== 45917+0+0 (secure 0 0 0) 0x7f84a4019850 con 0x7f84b4107a90 2024-03-04T09:21:04.513 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:03.787+0000 7f84b8983700 1 --2- 172.21.15.52:0/1843605130 >> [v2:172.21.15.52:6800/2379361333,v1:172.21.15.52:6801/2379361333] conn(0x7f84a0038b00 0x7f84a003afc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:21:04.513 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:03.787+0000 7f84b9985700 1 -- 172.21.15.52:0/1843605130 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f84b4069170 con 0x7f84b4107a90 2024-03-04T09:21:04.513 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:03.787+0000 7f84b8983700 1 -- 172.21.15.52:0/1843605130 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f84a404cc20 con 0x7f84b4107a90 2024-03-04T09:21:04.513 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:03.787+0000 7f84b27fc700 1 --2- 172.21.15.52:0/1843605130 >> [v2:172.21.15.52:6800/2379361333,v1:172.21.15.52:6801/2379361333] conn(0x7f84a0038b00 0x7f84a003afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:21:04.513 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:03.787+0000 7f84b27fc700 1 --2- 172.21.15.52:0/1843605130 >> [v2:172.21.15.52:6800/2379361333,v1:172.21.15.52:6801/2379361333] conn(0x7f84a0038b00 0x7f84a003afc0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f849c006fd0 tx=0x7f849c006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:21:04.513 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:03.792+0000 7f84b8983700 1 -- 172.21.15.52:0/1843605130 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f84a400f2e0 con 0x7f84b4107a90 2024-03-04T09:21:04.513 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:03.992+0000 7f84b9985700 1 -- 172.21.15.52:0/1843605130 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) v1 -- 0x7f84b419f080 con 0x7f84b4107a90 2024-03-04T09:21:04.513 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:03.993+0000 7f84b8983700 1 -- 172.21.15.52:0/1843605130 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) v1 ==== 70+0+373 (secure 0 0 0) 0x7f84b419f080 con 0x7f84b4107a90 2024-03-04T09:21:04.513 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:03.996+0000 7f84b9985700 1 -- 172.21.15.52:0/1843605130 >> [v2:172.21.15.52:6800/2379361333,v1:172.21.15.52:6801/2379361333] conn(0x7f84a0038b00 msgr2=0x7f84a003afc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:21:04.513 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:03.996+0000 7f84b9985700 1 --2- 172.21.15.52:0/1843605130 >> [v2:172.21.15.52:6800/2379361333,v1:172.21.15.52:6801/2379361333] conn(0x7f84a0038b00 0x7f84a003afc0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f849c006fd0 tx=0x7f849c006e40 comp rx=0 tx=0).stop 2024-03-04T09:21:04.513 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:03.996+0000 7f84b9985700 1 -- 172.21.15.52:0/1843605130 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f84b4107a90 msgr2=0x7f84b419b9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:21:04.514 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:03.996+0000 7f84b9985700 1 --2- 172.21.15.52:0/1843605130 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f84b4107a90 0x7f84b419b9f0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f84a4009fb0 tx=0x7f84a4011bf0 comp rx=0 tx=0).stop 2024-03-04T09:21:04.514 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:03.996+0000 7f84b9985700 1 -- 172.21.15.52:0/1843605130 shutdown_connections 2024-03-04T09:21:04.514 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:03.996+0000 7f84b9985700 1 --2- 172.21.15.52:0/1843605130 >> [v2:172.21.15.52:6800/2379361333,v1:172.21.15.52:6801/2379361333] conn(0x7f84a0038b00 0x7f84a003afc0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:21:04.514 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:03.996+0000 7f84b9985700 1 --2- 172.21.15.52:0/1843605130 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f84b4107a90 0x7f84b419b9f0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:21:04.514 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:03.996+0000 7f84b9985700 1 -- 172.21.15.52:0/1843605130 >> 172.21.15.52:0/1843605130 conn(0x7f84b4103090 msgr2=0x7f84b4103c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:21:04.514 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:03.996+0000 7f84b9985700 1 -- 172.21.15.52:0/1843605130 shutdown_connections 2024-03-04T09:21:04.514 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:03.996+0000 7f84b9985700 1 -- 172.21.15.52:0/1843605130 wait complete. 2024-03-04T09:21:04.514 INFO:teuthology.orchestra.run.smithi052.stdout:Enabling cephadm module... 2024-03-04T09:21:04.834 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:21:04 smithi052 ceph-mon[32473]: from='client.? 172.21.15.52:0/1843605130' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-03-04T09:21:06.038 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:21:05 smithi052 ceph-mon[32473]: from='client.? 172.21.15.52:0/4218323885' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-03-04T09:21:06.038 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Mar 04 09:21:05 smithi052 ceph-4e059f08-da08-11ee-95c6-87774f69a715-mgr-x[32705]: ignoring --setuser ceph since I am not root 2024-03-04T09:21:06.038 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Mar 04 09:21:05 smithi052 ceph-4e059f08-da08-11ee-95c6-87774f69a715-mgr-x[32705]: ignoring --setgroup ceph since I am not root 2024-03-04T09:21:06.329 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:04.959+0000 7f1f2cea7700 1 Processor -- start 2024-03-04T09:21:06.329 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:04.959+0000 7f1f2cea7700 1 -- start start 2024-03-04T09:21:06.329 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:04.959+0000 7f1f2cea7700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1f28107a60 0x7f1f28107e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:21:06.330 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:04.959+0000 7f1f2cea7700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f28108450 con 0x7f1f28107a60 2024-03-04T09:21:06.330 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:04.959+0000 7f1f2659c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1f28107a60 0x7f1f28107e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:21:06.330 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:04.959+0000 7f1f2659c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1f28107a60 0x7f1f28107e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:60180/0 (socket says 172.21.15.52:60180) 2024-03-04T09:21:06.330 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:04.959+0000 7f1f2659c700 1 -- 172.21.15.52:0/1907071556 learned_addr learned my addr 172.21.15.52:0/1907071556 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-04T09:21:06.330 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:04.960+0000 7f1f2659c700 1 -- 172.21.15.52:0/1907071556 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f28108c10 con 0x7f1f28107a60 2024-03-04T09:21:06.330 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:04.960+0000 7f1f2659c700 1 --2- 172.21.15.52:0/1907071556 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1f28107a60 0x7f1f28107e80 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f1f180098a0 tx=0x7f1f18009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d3915b7c89ee5d2 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:21:06.331 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:04.960+0000 7f1f2559a700 1 -- 172.21.15.52:0/1907071556 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1f18004160 con 0x7f1f28107a60 2024-03-04T09:21:06.331 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:04.960+0000 7f1f2559a700 1 -- 172.21.15.52:0/1907071556 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1f18004300 con 0x7f1f28107a60 2024-03-04T09:21:06.331 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:04.960+0000 7f1f2559a700 1 -- 172.21.15.52:0/1907071556 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1f18011b50 con 0x7f1f28107a60 2024-03-04T09:21:06.331 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:04.960+0000 7f1f2cea7700 1 -- 172.21.15.52:0/1907071556 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1f28107a60 msgr2=0x7f1f28107e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:21:06.331 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:04.960+0000 7f1f2cea7700 1 --2- 172.21.15.52:0/1907071556 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1f28107a60 0x7f1f28107e80 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f1f180098a0 tx=0x7f1f18009bb0 comp rx=0 tx=0).stop 2024-03-04T09:21:06.331 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:04.961+0000 7f1f2cea7700 1 -- 172.21.15.52:0/1907071556 shutdown_connections 2024-03-04T09:21:06.331 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:04.961+0000 7f1f2cea7700 1 --2- 172.21.15.52:0/1907071556 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1f28107a60 0x7f1f28107e80 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:21:06.331 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:04.961+0000 7f1f2cea7700 1 -- 172.21.15.52:0/1907071556 >> 172.21.15.52:0/1907071556 conn(0x7f1f28103060 msgr2=0x7f1f28105480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:21:06.331 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:04.961+0000 7f1f2cea7700 1 -- 172.21.15.52:0/1907071556 shutdown_connections 2024-03-04T09:21:06.331 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:04.961+0000 7f1f2cea7700 1 -- 172.21.15.52:0/1907071556 wait complete. 2024-03-04T09:21:06.331 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:04.961+0000 7f1f2cea7700 1 Processor -- start 2024-03-04T09:21:06.331 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:04.961+0000 7f1f2cea7700 1 -- start start 2024-03-04T09:21:06.331 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:04.961+0000 7f1f2cea7700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1f28107a60 0x7f1f2819b970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:21:06.331 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:04.961+0000 7f1f2cea7700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f2810bfc0 con 0x7f1f28107a60 2024-03-04T09:21:06.332 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:04.962+0000 7f1f2659c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1f28107a60 0x7f1f2819b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:21:06.332 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:04.962+0000 7f1f2659c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1f28107a60 0x7f1f2819b970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:60194/0 (socket says 172.21.15.52:60194) 2024-03-04T09:21:06.332 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:04.962+0000 7f1f2659c700 1 -- 172.21.15.52:0/4218323885 learned_addr learned my addr 172.21.15.52:0/4218323885 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-04T09:21:06.332 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:04.962+0000 7f1f2659c700 1 -- 172.21.15.52:0/4218323885 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f2819beb0 con 0x7f1f28107a60 2024-03-04T09:21:06.332 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:04.962+0000 7f1f2659c700 1 --2- 172.21.15.52:0/4218323885 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1f28107a60 0x7f1f2819b970 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f1f18011670 tx=0x7f1f18011b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:21:06.332 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:04.962+0000 7f1f177fe700 1 -- 172.21.15.52:0/4218323885 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1f180039c0 con 0x7f1f28107a60 2024-03-04T09:21:06.332 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:04.962+0000 7f1f177fe700 1 -- 172.21.15.52:0/4218323885 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1f18011e50 con 0x7f1f28107a60 2024-03-04T09:21:06.332 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:04.962+0000 7f1f2cea7700 1 -- 172.21.15.52:0/4218323885 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f2819c140 con 0x7f1f28107a60 2024-03-04T09:21:06.332 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:04.962+0000 7f1f177fe700 1 -- 172.21.15.52:0/4218323885 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1f1801a5c0 con 0x7f1f28107a60 2024-03-04T09:21:06.332 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:04.962+0000 7f1f2cea7700 1 -- 172.21.15.52:0/4218323885 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f2819ee30 con 0x7f1f28107a60 2024-03-04T09:21:06.332 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:04.963+0000 7f1f177fe700 1 -- 172.21.15.52:0/4218323885 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 4) v1 ==== 45917+0+0 (secure 0 0 0) 0x7f1f1801a870 con 0x7f1f28107a60 2024-03-04T09:21:06.332 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:04.963+0000 7f1f177fe700 1 --2- 172.21.15.52:0/4218323885 >> [v2:172.21.15.52:6800/2379361333,v1:172.21.15.52:6801/2379361333] conn(0x7f1f10038b50 0x7f1f1003b010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:21:06.332 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:04.963+0000 7f1f2cea7700 1 -- 172.21.15.52:0/4218323885 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f280636f0 con 0x7f1f28107a60 2024-03-04T09:21:06.332 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:04.963+0000 7f1f177fe700 1 -- 172.21.15.52:0/4218323885 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f1f1804cef0 con 0x7f1f28107a60 2024-03-04T09:21:06.333 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:04.963+0000 7f1f25d9b700 1 --2- 172.21.15.52:0/4218323885 >> [v2:172.21.15.52:6800/2379361333,v1:172.21.15.52:6801/2379361333] conn(0x7f1f10038b50 0x7f1f1003b010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:21:06.333 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:04.964+0000 7f1f25d9b700 1 --2- 172.21.15.52:0/4218323885 >> [v2:172.21.15.52:6800/2379361333,v1:172.21.15.52:6801/2379361333] conn(0x7f1f10038b50 0x7f1f1003b010 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f1f1c006fd0 tx=0x7f1f1c006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:21:06.333 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:04.968+0000 7f1f177fe700 1 -- 172.21.15.52:0/4218323885 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f1f18019c70 con 0x7f1f28107a60 2024-03-04T09:21:06.333 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:05.229+0000 7f1f2cea7700 1 -- 172.21.15.52:0/4218323885 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) v1 -- 0x7f1f2819f0a0 con 0x7f1f28107a60 2024-03-04T09:21:06.333 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:05.741+0000 7f1f177fe700 1 -- 172.21.15.52:0/4218323885 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) v1 ==== 86+0+0 (secure 0 0 0) 0x7f1f1804c910 con 0x7f1f28107a60 2024-03-04T09:21:06.333 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:05.743+0000 7f1f177fe700 1 -- 172.21.15.52:0/4218323885 <== mon.0 v2:172.21.15.52:3300/0 8 ==== mgrmap(e 5) v1 ==== 45928+0+0 (secure 0 0 0) 0x7f1f1802d910 con 0x7f1f28107a60 2024-03-04T09:21:06.333 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:05.745+0000 7f1f2cea7700 1 -- 172.21.15.52:0/4218323885 >> [v2:172.21.15.52:6800/2379361333,v1:172.21.15.52:6801/2379361333] conn(0x7f1f10038b50 msgr2=0x7f1f1003b010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:21:06.333 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:05.745+0000 7f1f2cea7700 1 --2- 172.21.15.52:0/4218323885 >> [v2:172.21.15.52:6800/2379361333,v1:172.21.15.52:6801/2379361333] conn(0x7f1f10038b50 0x7f1f1003b010 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f1f1c006fd0 tx=0x7f1f1c006e40 comp rx=0 tx=0).stop 2024-03-04T09:21:06.333 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:05.745+0000 7f1f2cea7700 1 -- 172.21.15.52:0/4218323885 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1f28107a60 msgr2=0x7f1f2819b970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:21:06.333 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:05.745+0000 7f1f2cea7700 1 --2- 172.21.15.52:0/4218323885 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1f28107a60 0x7f1f2819b970 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f1f18011670 tx=0x7f1f18011b60 comp rx=0 tx=0).stop 2024-03-04T09:21:06.333 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:05.745+0000 7f1f2cea7700 1 -- 172.21.15.52:0/4218323885 shutdown_connections 2024-03-04T09:21:06.333 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:05.745+0000 7f1f2cea7700 1 --2- 172.21.15.52:0/4218323885 >> [v2:172.21.15.52:6800/2379361333,v1:172.21.15.52:6801/2379361333] conn(0x7f1f10038b50 0x7f1f1003b010 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:21:06.333 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:05.745+0000 7f1f2cea7700 1 --2- 172.21.15.52:0/4218323885 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1f28107a60 0x7f1f2819b970 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:21:06.333 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:05.745+0000 7f1f2cea7700 1 -- 172.21.15.52:0/4218323885 >> 172.21.15.52:0/4218323885 conn(0x7f1f28103060 msgr2=0x7f1f28103c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:21:06.333 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:05.745+0000 7f1f2cea7700 1 -- 172.21.15.52:0/4218323885 shutdown_connections 2024-03-04T09:21:06.334 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:05.745+0000 7f1f2cea7700 1 -- 172.21.15.52:0/4218323885 wait complete. 2024-03-04T09:21:06.571 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Mar 04 09:21:06 smithi052 ceph-4e059f08-da08-11ee-95c6-87774f69a715-mgr-x[32705]: 2024-03-04T09:21:06.319+0000 7f5077e76200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-04T09:21:06.571 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Mar 04 09:21:06 smithi052 ceph-4e059f08-da08-11ee-95c6-87774f69a715-mgr-x[32705]: 2024-03-04T09:21:06.418+0000 7f5077e76200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-04T09:21:06.834 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:21:06 smithi052 ceph-mon[32473]: from='client.? 172.21.15.52:0/4218323885' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-03-04T09:21:06.834 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:21:06 smithi052 ceph-mon[32473]: mgrmap e5: x(active, since 6s) 2024-03-04T09:21:07.332 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Mar 04 09:21:07 smithi052 ceph-4e059f08-da08-11ee-95c6-87774f69a715-mgr-x[32705]: 2024-03-04T09:21:07.093+0000 7f5077e76200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-04T09:21:07.332 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Mar 04 09:21:07 smithi052 ceph-4e059f08-da08-11ee-95c6-87774f69a715-mgr-x[32705]: 2024-03-04T09:21:07.188+0000 7f5077e76200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-04T09:21:07.332 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Mar 04 09:21:07 smithi052 ceph-4e059f08-da08-11ee-95c6-87774f69a715-mgr-x[32705]: 2024-03-04T09:21:07.335+0000 7f5077e76200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-04T09:21:07.584 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Mar 04 09:21:07 smithi052 ceph-4e059f08-da08-11ee-95c6-87774f69a715-mgr-x[32705]: 2024-03-04T09:21:07.434+0000 7f5077e76200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-04T09:21:07.585 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Mar 04 09:21:07 smithi052 ceph-4e059f08-da08-11ee-95c6-87774f69a715-mgr-x[32705]: 2024-03-04T09:21:07.519+0000 7f5077e76200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-04T09:21:07.637 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout { 2024-03-04T09:21:07.637 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-03-04T09:21:07.637 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "available": true, 2024-03-04T09:21:07.637 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-03-04T09:21:07.637 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-03-04T09:21:07.637 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout } 2024-03-04T09:21:07.637 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:06.760+0000 7f6c20680700 1 Processor -- start 2024-03-04T09:21:07.637 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:06.760+0000 7f6c20680700 1 -- start start 2024-03-04T09:21:07.637 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:06.761+0000 7f6c20680700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6c181068d0 0x7f6c18106cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:21:07.638 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:06.761+0000 7f6c20680700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c181072c0 con 0x7f6c181068d0 2024-03-04T09:21:07.638 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:06.761+0000 7f6c1e41c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6c181068d0 0x7f6c18106cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:21:07.638 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:06.761+0000 7f6c1e41c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6c181068d0 0x7f6c18106cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:60230/0 (socket says 172.21.15.52:60230) 2024-03-04T09:21:07.638 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:06.761+0000 7f6c1e41c700 1 -- 172.21.15.52:0/3135576067 learned_addr learned my addr 172.21.15.52:0/3135576067 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-04T09:21:07.638 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:06.761+0000 7f6c1e41c700 1 -- 172.21.15.52:0/3135576067 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c18107ad0 con 0x7f6c181068d0 2024-03-04T09:21:07.638 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:06.761+0000 7f6c1e41c700 1 --2- 172.21.15.52:0/3135576067 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6c181068d0 0x7f6c18106cf0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f6c0c0098a0 tx=0x7f6c0c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e03f5f8cc03a3c45 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:21:07.638 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:06.761+0000 7f6c1d41a700 1 -- 172.21.15.52:0/3135576067 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6c0c004160 con 0x7f6c181068d0 2024-03-04T09:21:07.638 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:06.761+0000 7f6c1d41a700 1 -- 172.21.15.52:0/3135576067 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6c0c004300 con 0x7f6c181068d0 2024-03-04T09:21:07.638 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:06.761+0000 7f6c1d41a700 1 -- 172.21.15.52:0/3135576067 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6c0c011b50 con 0x7f6c181068d0 2024-03-04T09:21:07.638 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:06.762+0000 7f6c20680700 1 -- 172.21.15.52:0/3135576067 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6c181068d0 msgr2=0x7f6c18106cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:21:07.638 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:06.762+0000 7f6c20680700 1 --2- 172.21.15.52:0/3135576067 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6c181068d0 0x7f6c18106cf0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f6c0c0098a0 tx=0x7f6c0c009bb0 comp rx=0 tx=0).stop 2024-03-04T09:21:07.638 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:06.762+0000 7f6c20680700 1 -- 172.21.15.52:0/3135576067 shutdown_connections 2024-03-04T09:21:07.638 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:06.762+0000 7f6c20680700 1 --2- 172.21.15.52:0/3135576067 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6c181068d0 0x7f6c18106cf0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:21:07.638 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:06.762+0000 7f6c20680700 1 -- 172.21.15.52:0/3135576067 >> 172.21.15.52:0/3135576067 conn(0x7f6c18101f70 msgr2=0x7f6c18104390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:21:07.639 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:06.762+0000 7f6c20680700 1 -- 172.21.15.52:0/3135576067 shutdown_connections 2024-03-04T09:21:07.639 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:06.762+0000 7f6c20680700 1 -- 172.21.15.52:0/3135576067 wait complete. 2024-03-04T09:21:07.639 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:06.763+0000 7f6c20680700 1 Processor -- start 2024-03-04T09:21:07.639 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:06.763+0000 7f6c20680700 1 -- start start 2024-03-04T09:21:07.639 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:06.763+0000 7f6c20680700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6c181068d0 0x7f6c1807aac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:21:07.639 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:06.763+0000 7f6c20680700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c1810ae80 con 0x7f6c181068d0 2024-03-04T09:21:07.639 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:06.763+0000 7f6c1e41c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6c181068d0 0x7f6c1807aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:21:07.639 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:06.763+0000 7f6c1e41c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6c181068d0 0x7f6c1807aac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:60244/0 (socket says 172.21.15.52:60244) 2024-03-04T09:21:07.639 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:06.763+0000 7f6c1e41c700 1 -- 172.21.15.52:0/2899422489 learned_addr learned my addr 172.21.15.52:0/2899422489 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-04T09:21:07.639 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:06.763+0000 7f6c1e41c700 1 -- 172.21.15.52:0/2899422489 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c1819e710 con 0x7f6c181068d0 2024-03-04T09:21:07.639 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:06.764+0000 7f6c1e41c700 1 --2- 172.21.15.52:0/2899422489 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6c181068d0 0x7f6c1807aac0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f6c0c011850 tx=0x7f6c0c011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:21:07.639 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:06.764+0000 7f6c0b7fe700 1 -- 172.21.15.52:0/2899422489 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6c0c003af0 con 0x7f6c181068d0 2024-03-04T09:21:07.639 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:06.764+0000 7f6c0b7fe700 1 -- 172.21.15.52:0/2899422489 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6c0c020070 con 0x7f6c181068d0 2024-03-04T09:21:07.639 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:06.764+0000 7f6c20680700 1 -- 172.21.15.52:0/2899422489 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c1819e8e0 con 0x7f6c181068d0 2024-03-04T09:21:07.639 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:06.764+0000 7f6c0b7fe700 1 -- 172.21.15.52:0/2899422489 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6c0c0195a0 con 0x7f6c181068d0 2024-03-04T09:21:07.640 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:06.764+0000 7f6c20680700 1 -- 172.21.15.52:0/2899422489 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c1819ed00 con 0x7f6c181068d0 2024-03-04T09:21:07.640 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:06.765+0000 7f6c0b7fe700 1 -- 172.21.15.52:0/2899422489 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) v1 ==== 45928+0+0 (secure 0 0 0) 0x7f6c0c019850 con 0x7f6c181068d0 2024-03-04T09:21:07.640 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:06.765+0000 7f6c20680700 1 -- 172.21.15.52:0/2899422489 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c18069170 con 0x7f6c181068d0 2024-03-04T09:21:07.640 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:06.765+0000 7f6c0b7fe700 1 --2- 172.21.15.52:0/2899422489 >> [v2:172.21.15.52:6800/2379361333,v1:172.21.15.52:6801/2379361333] conn(0x7f6c04038bf0 0x7f6c0403b0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:21:07.640 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:06.765+0000 7f6c0b7fe700 1 -- 172.21.15.52:0/2899422489 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f6c0c011e00 con 0x7f6c181068d0 2024-03-04T09:21:07.640 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:06.765+0000 7f6c1dc1b700 1 -- 172.21.15.52:0/2899422489 >> [v2:172.21.15.52:6800/2379361333,v1:172.21.15.52:6801/2379361333] conn(0x7f6c04038bf0 msgr2=0x7f6c0403b0b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.52:6800/2379361333 2024-03-04T09:21:07.640 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:06.765+0000 7f6c1dc1b700 1 --2- 172.21.15.52:0/2899422489 >> [v2:172.21.15.52:6800/2379361333,v1:172.21.15.52:6801/2379361333] conn(0x7f6c04038bf0 0x7f6c0403b0b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T09:21:07.640 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:06.770+0000 7f6c0b7fe700 1 -- 172.21.15.52:0/2899422489 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f6c0c019b40 con 0x7f6c181068d0 2024-03-04T09:21:07.640 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:06.966+0000 7f6c1dc1b700 1 -- 172.21.15.52:0/2899422489 >> [v2:172.21.15.52:6800/2379361333,v1:172.21.15.52:6801/2379361333] conn(0x7f6c04038bf0 msgr2=0x7f6c0403b0b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.52:6800/2379361333 2024-03-04T09:21:07.640 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:06.966+0000 7f6c1dc1b700 1 --2- 172.21.15.52:0/2899422489 >> [v2:172.21.15.52:6800/2379361333,v1:172.21.15.52:6801/2379361333] conn(0x7f6c04038bf0 0x7f6c0403b0b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T09:21:07.640 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:07.030+0000 7f6c20680700 1 -- 172.21.15.52:0/2899422489 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) v1 -- 0x7f6c1819f570 con 0x7f6c181068d0 2024-03-04T09:21:07.640 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:07.031+0000 7f6c0b7fe700 1 -- 172.21.15.52:0/2899422489 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) v1 ==== 56+0+88 (secure 0 0 0) 0x7f6c0c003c90 con 0x7f6c181068d0 2024-03-04T09:21:07.641 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:07.033+0000 7f6c20680700 1 -- 172.21.15.52:0/2899422489 >> [v2:172.21.15.52:6800/2379361333,v1:172.21.15.52:6801/2379361333] conn(0x7f6c04038bf0 msgr2=0x7f6c0403b0b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-04T09:21:07.641 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:07.033+0000 7f6c20680700 1 --2- 172.21.15.52:0/2899422489 >> [v2:172.21.15.52:6800/2379361333,v1:172.21.15.52:6801/2379361333] conn(0x7f6c04038bf0 0x7f6c0403b0b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:21:07.641 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:07.033+0000 7f6c20680700 1 -- 172.21.15.52:0/2899422489 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6c181068d0 msgr2=0x7f6c1807aac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:21:07.641 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:07.033+0000 7f6c20680700 1 --2- 172.21.15.52:0/2899422489 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6c181068d0 0x7f6c1807aac0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f6c0c011850 tx=0x7f6c0c011c10 comp rx=0 tx=0).stop 2024-03-04T09:21:07.641 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:07.033+0000 7f6c20680700 1 -- 172.21.15.52:0/2899422489 shutdown_connections 2024-03-04T09:21:07.641 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:07.033+0000 7f6c20680700 1 --2- 172.21.15.52:0/2899422489 >> [v2:172.21.15.52:6800/2379361333,v1:172.21.15.52:6801/2379361333] conn(0x7f6c04038bf0 0x7f6c0403b0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:21:07.641 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:07.033+0000 7f6c20680700 1 --2- 172.21.15.52:0/2899422489 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6c181068d0 0x7f6c1807aac0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:21:07.641 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:07.033+0000 7f6c20680700 1 -- 172.21.15.52:0/2899422489 >> 172.21.15.52:0/2899422489 conn(0x7f6c18101f70 msgr2=0x7f6c18102b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:21:07.641 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:07.034+0000 7f6c20680700 1 -- 172.21.15.52:0/2899422489 shutdown_connections 2024-03-04T09:21:07.641 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:07.034+0000 7f6c20680700 1 -- 172.21.15.52:0/2899422489 wait complete. 2024-03-04T09:21:07.641 INFO:teuthology.orchestra.run.smithi052.stdout:Waiting for the mgr to restart... 2024-03-04T09:21:07.641 INFO:teuthology.orchestra.run.smithi052.stdout:Waiting for mgr epoch 5... 2024-03-04T09:21:07.905 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:21:07 smithi052 ceph-mon[32473]: from='client.? 172.21.15.52:0/2899422489' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-03-04T09:21:07.905 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Mar 04 09:21:07 smithi052 ceph-4e059f08-da08-11ee-95c6-87774f69a715-mgr-x[32705]: 2024-03-04T09:21:07.793+0000 7f5077e76200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-04T09:21:08.234 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Mar 04 09:21:07 smithi052 ceph-4e059f08-da08-11ee-95c6-87774f69a715-mgr-x[32705]: 2024-03-04T09:21:07.908+0000 7f5077e76200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-04T09:21:08.234 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Mar 04 09:21:08 smithi052 ceph-4e059f08-da08-11ee-95c6-87774f69a715-mgr-x[32705]: 2024-03-04T09:21:07.999+0000 7f5077e76200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-04T09:21:08.234 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Mar 04 09:21:08 smithi052 ceph-4e059f08-da08-11ee-95c6-87774f69a715-mgr-x[32705]: 2024-03-04T09:21:08.099+0000 7f5077e76200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-04T09:21:08.584 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Mar 04 09:21:08 smithi052 ceph-4e059f08-da08-11ee-95c6-87774f69a715-mgr-x[32705]: 2024-03-04T09:21:08.277+0000 7f5077e76200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-04T09:21:09.584 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Mar 04 09:21:09 smithi052 ceph-4e059f08-da08-11ee-95c6-87774f69a715-mgr-x[32705]: 2024-03-04T09:21:09.258+0000 7f5077e76200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-04T09:21:09.585 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Mar 04 09:21:09 smithi052 ceph-4e059f08-da08-11ee-95c6-87774f69a715-mgr-x[32705]: 2024-03-04T09:21:09.392+0000 7f5077e76200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-04T09:21:10.018 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Mar 04 09:21:09 smithi052 ceph-4e059f08-da08-11ee-95c6-87774f69a715-mgr-x[32705]: 2024-03-04T09:21:09.729+0000 7f5077e76200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-04T09:21:10.018 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Mar 04 09:21:09 smithi052 ceph-4e059f08-da08-11ee-95c6-87774f69a715-mgr-x[32705]: 2024-03-04T09:21:09.820+0000 7f5077e76200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-04T09:21:10.334 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Mar 04 09:21:10 smithi052 ceph-4e059f08-da08-11ee-95c6-87774f69a715-mgr-x[32705]: 2024-03-04T09:21:10.021+0000 7f5077e76200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-04T09:21:10.335 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Mar 04 09:21:10 smithi052 ceph-4e059f08-da08-11ee-95c6-87774f69a715-mgr-x[32705]: 2024-03-04T09:21:10.212+0000 7f5077e76200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-04T09:21:10.762 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Mar 04 09:21:10 smithi052 ceph-4e059f08-da08-11ee-95c6-87774f69a715-mgr-x[32705]: 2024-03-04T09:21:10.505+0000 7f5077e76200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-04T09:21:10.762 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Mar 04 09:21:10 smithi052 ceph-4e059f08-da08-11ee-95c6-87774f69a715-mgr-x[32705]: 2024-03-04T09:21:10.765+0000 7f5077e76200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-04T09:21:11.037 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Mar 04 09:21:11 smithi052 ceph-4e059f08-da08-11ee-95c6-87774f69a715-mgr-x[32705]: 2024-03-04T09:21:11.006+0000 7f5077e76200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-04T09:21:12.084 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Mar 04 09:21:11 smithi052 ceph-4e059f08-da08-11ee-95c6-87774f69a715-mgr-x[32705]: 2024-03-04T09:21:11.832+0000 7f5077e76200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-04T09:21:12.743 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Mar 04 09:21:12 smithi052 ceph-4e059f08-da08-11ee-95c6-87774f69a715-mgr-x[32705]: 2024-03-04T09:21:12.411+0000 7f5077e76200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-04T09:21:12.744 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Mar 04 09:21:12 smithi052 ceph-4e059f08-da08-11ee-95c6-87774f69a715-mgr-x[32705]: 2024-03-04T09:21:12.515+0000 7f5077e76200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-04T09:21:12.744 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Mar 04 09:21:12 smithi052 ceph-4e059f08-da08-11ee-95c6-87774f69a715-mgr-x[32705]: 2024-03-04T09:21:12.746+0000 7f5077e76200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-04T09:21:13.060 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:21:12 smithi052 ceph-mon[32473]: Active manager daemon x restarted 2024-03-04T09:21:13.060 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:21:12 smithi052 ceph-mon[32473]: Activating manager daemon x 2024-03-04T09:21:13.060 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:21:12 smithi052 ceph-mon[32473]: osdmap e2: 0 total, 0 up, 0 in 2024-03-04T09:21:13.060 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:21:12 smithi052 ceph-mon[32473]: mgrmap e6: x(active, starting, since 0.00334823s) 2024-03-04T09:21:13.061 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:21:12 smithi052 ceph-mon[32473]: from='mgr.14120 172.21.15.52:0/3140567671' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-04T09:21:13.061 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:21:12 smithi052 ceph-mon[32473]: from='mgr.14120 172.21.15.52:0/3140567671' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-03-04T09:21:13.061 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:21:12 smithi052 ceph-mon[32473]: from='mgr.14120 172.21.15.52:0/3140567671' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-04T09:21:13.061 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:21:12 smithi052 ceph-mon[32473]: from='mgr.14120 172.21.15.52:0/3140567671' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-04T09:21:13.061 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:21:12 smithi052 ceph-mon[32473]: from='mgr.14120 172.21.15.52:0/3140567671' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-04T09:21:13.061 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:21:12 smithi052 ceph-mon[32473]: Manager daemon x is now available 2024-03-04T09:21:13.061 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:21:12 smithi052 ceph-mon[32473]: from='mgr.14120 172.21.15.52:0/3140567671' entity='mgr.x' 2024-03-04T09:21:13.061 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:21:12 smithi052 ceph-mon[32473]: from='mgr.14120 172.21.15.52:0/3140567671' entity='mgr.x' 2024-03-04T09:21:13.061 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:21:12 smithi052 ceph-mon[32473]: from='mgr.14120 172.21.15.52:0/3140567671' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T09:21:13.061 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:21:12 smithi052 ceph-mon[32473]: from='mgr.14120 172.21.15.52:0/3140567671' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T09:21:13.061 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:21:12 smithi052 ceph-mon[32473]: from='mgr.14120 172.21.15.52:0/3140567671' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T09:21:14.076 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:21:13 smithi052 ceph-mon[32473]: Found migration_current of "None". Setting to last migration. 2024-03-04T09:21:14.076 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:21:13 smithi052 ceph-mon[32473]: from='mgr.14120 172.21.15.52:0/3140567671' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-03-04T09:21:14.076 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:21:13 smithi052 ceph-mon[32473]: from='mgr.14120 172.21.15.52:0/3140567671' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-03-04T09:21:14.076 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:21:13 smithi052 ceph-mon[32473]: from='mgr.14120 172.21.15.52:0/3140567671' entity='mgr.x' 2024-03-04T09:21:14.076 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:21:13 smithi052 ceph-mon[32473]: from='mgr.14120 172.21.15.52:0/3140567671' entity='mgr.x' 2024-03-04T09:21:14.076 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:21:13 smithi052 ceph-mon[32473]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-03-04T09:21:14.076 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:21:13 smithi052 ceph-mon[32473]: mgrmap e7: x(active, since 1.00525s) 2024-03-04T09:21:14.076 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:21:13 smithi052 ceph-mon[32473]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-03-04T09:21:14.370 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout { 2024-03-04T09:21:14.370 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-03-04T09:21:14.371 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "initialized": true 2024-03-04T09:21:14.371 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout } 2024-03-04T09:21:14.371 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:08.031+0000 7f25aefe6700 1 Processor -- start 2024-03-04T09:21:14.371 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:08.031+0000 7f25aefe6700 1 -- start start 2024-03-04T09:21:14.371 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:08.031+0000 7f25aefe6700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f25a81068c0 0x7f25a8106ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:21:14.371 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:08.031+0000 7f25aefe6700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25a81072b0 con 0x7f25a81068c0 2024-03-04T09:21:14.371 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:08.031+0000 7f25acd82700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f25a81068c0 0x7f25a8106ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:21:14.371 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:08.031+0000 7f25acd82700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f25a81068c0 0x7f25a8106ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:60252/0 (socket says 172.21.15.52:60252) 2024-03-04T09:21:14.371 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:08.031+0000 7f25acd82700 1 -- 172.21.15.52:0/3756307623 learned_addr learned my addr 172.21.15.52:0/3756307623 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-04T09:21:14.371 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:08.031+0000 7f25acd82700 1 -- 172.21.15.52:0/3756307623 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25a8107ac0 con 0x7f25a81068c0 2024-03-04T09:21:14.371 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:08.032+0000 7f25acd82700 1 --2- 172.21.15.52:0/3756307623 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f25a81068c0 0x7f25a8106ce0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f259c0098a0 tx=0x7f259c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e6c34fa6a64433d server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:21:14.371 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:08.032+0000 7f25a77fe700 1 -- 172.21.15.52:0/3756307623 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f259c004160 con 0x7f25a81068c0 2024-03-04T09:21:14.371 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:08.032+0000 7f25a77fe700 1 -- 172.21.15.52:0/3756307623 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f259c004300 con 0x7f25a81068c0 2024-03-04T09:21:14.372 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:08.032+0000 7f25a77fe700 1 -- 172.21.15.52:0/3756307623 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f259c011b50 con 0x7f25a81068c0 2024-03-04T09:21:14.372 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:08.033+0000 7f25aefe6700 1 -- 172.21.15.52:0/3756307623 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f25a81068c0 msgr2=0x7f25a8106ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:21:14.372 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:08.033+0000 7f25aefe6700 1 --2- 172.21.15.52:0/3756307623 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f25a81068c0 0x7f25a8106ce0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f259c0098a0 tx=0x7f259c009bb0 comp rx=0 tx=0).stop 2024-03-04T09:21:14.372 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:08.033+0000 7f25aefe6700 1 -- 172.21.15.52:0/3756307623 shutdown_connections 2024-03-04T09:21:14.372 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:08.033+0000 7f25aefe6700 1 --2- 172.21.15.52:0/3756307623 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f25a81068c0 0x7f25a8106ce0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:21:14.372 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:08.033+0000 7f25aefe6700 1 -- 172.21.15.52:0/3756307623 >> 172.21.15.52:0/3756307623 conn(0x7f25a8101f40 msgr2=0x7f25a8104360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:21:14.372 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:08.033+0000 7f25aefe6700 1 -- 172.21.15.52:0/3756307623 shutdown_connections 2024-03-04T09:21:14.372 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:08.033+0000 7f25aefe6700 1 -- 172.21.15.52:0/3756307623 wait complete. 2024-03-04T09:21:14.372 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:08.034+0000 7f25aefe6700 1 Processor -- start 2024-03-04T09:21:14.372 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:08.034+0000 7f25aefe6700 1 -- start start 2024-03-04T09:21:14.372 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:08.034+0000 7f25aefe6700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f25a81068c0 0x7f25a807aac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:21:14.372 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:08.034+0000 7f25aefe6700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25a810ae70 con 0x7f25a81068c0 2024-03-04T09:21:14.373 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:08.034+0000 7f25acd82700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f25a81068c0 0x7f25a807aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:21:14.373 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:08.034+0000 7f25acd82700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f25a81068c0 0x7f25a807aac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:60254/0 (socket says 172.21.15.52:60254) 2024-03-04T09:21:14.373 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:08.034+0000 7f25acd82700 1 -- 172.21.15.52:0/3689705732 learned_addr learned my addr 172.21.15.52:0/3689705732 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-04T09:21:14.373 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:08.034+0000 7f25acd82700 1 -- 172.21.15.52:0/3689705732 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25a819e720 con 0x7f25a81068c0 2024-03-04T09:21:14.373 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:08.034+0000 7f25acd82700 1 --2- 172.21.15.52:0/3689705732 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f25a81068c0 0x7f25a807aac0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f259c011850 tx=0x7f259c011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:21:14.373 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:08.035+0000 7f25a5ffb700 1 -- 172.21.15.52:0/3689705732 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f259c003af0 con 0x7f25a81068c0 2024-03-04T09:21:14.373 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:08.035+0000 7f25aefe6700 1 -- 172.21.15.52:0/3689705732 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25a819e8f0 con 0x7f25a81068c0 2024-03-04T09:21:14.373 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:08.035+0000 7f25a5ffb700 1 -- 172.21.15.52:0/3689705732 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f259c020070 con 0x7f25a81068c0 2024-03-04T09:21:14.373 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:08.035+0000 7f25a5ffb700 1 -- 172.21.15.52:0/3689705732 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f259c0195a0 con 0x7f25a81068c0 2024-03-04T09:21:14.373 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:08.035+0000 7f25aefe6700 1 -- 172.21.15.52:0/3689705732 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25a819ed10 con 0x7f25a81068c0 2024-03-04T09:21:14.373 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:08.036+0000 7f25a5ffb700 1 -- 172.21.15.52:0/3689705732 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) v1 ==== 45928+0+0 (secure 0 0 0) 0x7f259c0198a0 con 0x7f25a81068c0 2024-03-04T09:21:14.373 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:08.036+0000 7f25a5ffb700 1 --2- 172.21.15.52:0/3689705732 >> [v2:172.21.15.52:6800/2379361333,v1:172.21.15.52:6801/2379361333] conn(0x7f2598038ba0 0x7f259803b060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:21:14.373 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:08.036+0000 7f25a5ffb700 1 -- 172.21.15.52:0/3689705732 --> [v2:172.21.15.52:6800/2379361333,v1:172.21.15.52:6801/2379361333] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f259803b770 con 0x7f2598038ba0 2024-03-04T09:21:14.374 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:08.036+0000 7f25a5ffb700 1 -- 172.21.15.52:0/3689705732 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f259c011e00 con 0x7f25a81068c0 2024-03-04T09:21:14.374 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:08.036+0000 7f25a7fff700 1 -- 172.21.15.52:0/3689705732 >> [v2:172.21.15.52:6800/2379361333,v1:172.21.15.52:6801/2379361333] conn(0x7f2598038ba0 msgr2=0x7f259803b060 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.52:6800/2379361333 2024-03-04T09:21:14.374 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:08.036+0000 7f25a7fff700 1 --2- 172.21.15.52:0/3689705732 >> [v2:172.21.15.52:6800/2379361333,v1:172.21.15.52:6801/2379361333] conn(0x7f2598038ba0 0x7f259803b060 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T09:21:14.374 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:08.237+0000 7f25a7fff700 1 -- 172.21.15.52:0/3689705732 >> [v2:172.21.15.52:6800/2379361333,v1:172.21.15.52:6801/2379361333] conn(0x7f2598038ba0 msgr2=0x7f259803b060 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.52:6800/2379361333 2024-03-04T09:21:14.374 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:08.237+0000 7f25a7fff700 1 --2- 172.21.15.52:0/3689705732 >> [v2:172.21.15.52:6800/2379361333,v1:172.21.15.52:6801/2379361333] conn(0x7f2598038ba0 0x7f259803b060 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T09:21:14.374 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:08.638+0000 7f25a7fff700 1 -- 172.21.15.52:0/3689705732 >> [v2:172.21.15.52:6800/2379361333,v1:172.21.15.52:6801/2379361333] conn(0x7f2598038ba0 msgr2=0x7f259803b060 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.52:6800/2379361333 2024-03-04T09:21:14.374 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:08.638+0000 7f25a7fff700 1 --2- 172.21.15.52:0/3689705732 >> [v2:172.21.15.52:6800/2379361333,v1:172.21.15.52:6801/2379361333] conn(0x7f2598038ba0 0x7f259803b060 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T09:21:14.374 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:09.439+0000 7f25a7fff700 1 -- 172.21.15.52:0/3689705732 >> [v2:172.21.15.52:6800/2379361333,v1:172.21.15.52:6801/2379361333] conn(0x7f2598038ba0 msgr2=0x7f259803b060 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.52:6800/2379361333 2024-03-04T09:21:14.374 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:09.439+0000 7f25a7fff700 1 --2- 172.21.15.52:0/3689705732 >> [v2:172.21.15.52:6800/2379361333,v1:172.21.15.52:6801/2379361333] conn(0x7f2598038ba0 0x7f259803b060 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T09:21:14.374 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:11.040+0000 7f25a7fff700 1 -- 172.21.15.52:0/3689705732 >> [v2:172.21.15.52:6800/2379361333,v1:172.21.15.52:6801/2379361333] conn(0x7f2598038ba0 msgr2=0x7f259803b060 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.52:6800/2379361333 2024-03-04T09:21:14.374 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:11.040+0000 7f25a7fff700 1 --2- 172.21.15.52:0/3689705732 >> [v2:172.21.15.52:6800/2379361333,v1:172.21.15.52:6801/2379361333] conn(0x7f2598038ba0 0x7f259803b060 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-03-04T09:21:14.374 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:12.762+0000 7f25a5ffb700 1 -- 172.21.15.52:0/3689705732 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mgrmap(e 6) v1 ==== 45695+0+0 (secure 0 0 0) 0x7f259c02bd00 con 0x7f25a81068c0 2024-03-04T09:21:14.374 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:12.762+0000 7f25a5ffb700 1 -- 172.21.15.52:0/3689705732 >> [v2:172.21.15.52:6800/2379361333,v1:172.21.15.52:6801/2379361333] conn(0x7f2598038ba0 msgr2=0x7f259803b060 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-04T09:21:14.374 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:12.762+0000 7f25a5ffb700 1 --2- 172.21.15.52:0/3689705732 >> [v2:172.21.15.52:6800/2379361333,v1:172.21.15.52:6801/2379361333] conn(0x7f2598038ba0 0x7f259803b060 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:21:14.375 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:13.763+0000 7f25a5ffb700 1 -- 172.21.15.52:0/3689705732 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mgrmap(e 7) v1 ==== 45822+0+0 (secure 0 0 0) 0x7f259c04e7b0 con 0x7f25a81068c0 2024-03-04T09:21:14.375 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:13.763+0000 7f25a5ffb700 1 --2- 172.21.15.52:0/3689705732 >> [v2:172.21.15.52:6800/2425730636,v1:172.21.15.52:6801/2425730636] conn(0x7f259803c4d0 0x7f259803e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:21:14.375 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:13.763+0000 7f25a5ffb700 1 -- 172.21.15.52:0/3689705732 --> [v2:172.21.15.52:6800/2425730636,v1:172.21.15.52:6801/2425730636] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f259803b770 con 0x7f259803c4d0 2024-03-04T09:21:14.375 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:13.764+0000 7f25a7fff700 1 --2- 172.21.15.52:0/3689705732 >> [v2:172.21.15.52:6800/2425730636,v1:172.21.15.52:6801/2425730636] conn(0x7f259803c4d0 0x7f259803e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:21:14.375 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:13.764+0000 7f25a7fff700 1 --2- 172.21.15.52:0/3689705732 >> [v2:172.21.15.52:6800/2425730636,v1:172.21.15.52:6801/2425730636] conn(0x7f259803c4d0 0x7f259803e8c0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f2594003a10 tx=0x7f2594009290 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:21:14.375 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:13.765+0000 7f25a5ffb700 1 -- 172.21.15.52:0/3689705732 <== mgr.14120 v2:172.21.15.52:6800/2425730636 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+7774 (secure 0 0 0) 0x7f259803b770 con 0x7f259803c4d0 2024-03-04T09:21:14.375 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:13.772+0000 7f25aefe6700 1 -- 172.21.15.52:0/3689705732 --> [v2:172.21.15.52:6800/2425730636,v1:172.21.15.52:6801/2425730636] -- command(tid 1: {"prefix": "mgr_status"}) v1 -- 0x7f25a8069170 con 0x7f259803c4d0 2024-03-04T09:21:14.375 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:13.772+0000 7f25a5ffb700 1 -- 172.21.15.52:0/3689705732 <== mgr.14120 v2:172.21.15.52:6800/2425730636 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7f25a8069170 con 0x7f259803c4d0 2024-03-04T09:21:14.375 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:13.772+0000 7f25aefe6700 1 -- 172.21.15.52:0/3689705732 >> [v2:172.21.15.52:6800/2425730636,v1:172.21.15.52:6801/2425730636] conn(0x7f259803c4d0 msgr2=0x7f259803e8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:21:14.375 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:13.772+0000 7f25aefe6700 1 --2- 172.21.15.52:0/3689705732 >> [v2:172.21.15.52:6800/2425730636,v1:172.21.15.52:6801/2425730636] conn(0x7f259803c4d0 0x7f259803e8c0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f2594003a10 tx=0x7f2594009290 comp rx=0 tx=0).stop 2024-03-04T09:21:14.376 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:13.772+0000 7f25aefe6700 1 -- 172.21.15.52:0/3689705732 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f25a81068c0 msgr2=0x7f25a807aac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:21:14.376 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:13.772+0000 7f25aefe6700 1 --2- 172.21.15.52:0/3689705732 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f25a81068c0 0x7f25a807aac0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f259c011850 tx=0x7f259c011c10 comp rx=0 tx=0).stop 2024-03-04T09:21:14.376 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:13.772+0000 7f25aefe6700 1 -- 172.21.15.52:0/3689705732 shutdown_connections 2024-03-04T09:21:14.376 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:13.772+0000 7f25aefe6700 1 --2- 172.21.15.52:0/3689705732 >> [v2:172.21.15.52:6800/2425730636,v1:172.21.15.52:6801/2425730636] conn(0x7f259803c4d0 0x7f259803e8c0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:21:14.376 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:13.772+0000 7f25aefe6700 1 --2- 172.21.15.52:0/3689705732 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f25a81068c0 0x7f25a807aac0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:21:14.376 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:13.772+0000 7f25aefe6700 1 --2- 172.21.15.52:0/3689705732 >> [v2:172.21.15.52:6800/2379361333,v1:172.21.15.52:6801/2379361333] conn(0x7f2598038ba0 0x7f259803b060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:21:14.376 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:13.772+0000 7f25aefe6700 1 -- 172.21.15.52:0/3689705732 >> 172.21.15.52:0/3689705732 conn(0x7f25a8101f40 msgr2=0x7f25a8102b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:21:14.376 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:13.772+0000 7f25aefe6700 1 -- 172.21.15.52:0/3689705732 shutdown_connections 2024-03-04T09:21:14.376 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:13.772+0000 7f25aefe6700 1 -- 172.21.15.52:0/3689705732 wait complete. 2024-03-04T09:21:14.376 INFO:teuthology.orchestra.run.smithi052.stdout:mgr epoch 5 is available 2024-03-04T09:21:14.376 INFO:teuthology.orchestra.run.smithi052.stdout:Setting orchestrator backend to cephadm... 2024-03-04T09:21:15.022 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:21:14 smithi052 ceph-mon[32473]: [04/Mar/2024:09:21:14] ENGINE Bus STARTING 2024-03-04T09:21:15.022 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:21:14 smithi052 ceph-mon[32473]: [04/Mar/2024:09:21:14] ENGINE Serving on http://172.21.15.52:8765 2024-03-04T09:21:15.022 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:21:14 smithi052 ceph-mon[32473]: [04/Mar/2024:09:21:14] ENGINE Serving on https://172.21.15.52:7150 2024-03-04T09:21:15.022 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:21:14 smithi052 ceph-mon[32473]: [04/Mar/2024:09:21:14] ENGINE Bus STARTED 2024-03-04T09:21:15.022 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:21:14 smithi052 ceph-mon[32473]: from='mgr.14120 172.21.15.52:0/3140567671' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T09:21:15.609 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:14.794+0000 7f4c501f2700 1 Processor -- start 2024-03-04T09:21:15.609 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:14.795+0000 7f4c501f2700 1 -- start start 2024-03-04T09:21:15.609 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:14.795+0000 7f4c501f2700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4c48107a60 0x7f4c48107e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:21:15.609 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:14.795+0000 7f4c501f2700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c48108450 con 0x7f4c48107a60 2024-03-04T09:21:15.609 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:14.795+0000 7f4c4df8e700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4c48107a60 0x7f4c48107e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:21:15.609 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:14.795+0000 7f4c4df8e700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4c48107a60 0x7f4c48107e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:59998/0 (socket says 172.21.15.52:59998) 2024-03-04T09:21:15.609 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:14.795+0000 7f4c4df8e700 1 -- 172.21.15.52:0/337864730 learned_addr learned my addr 172.21.15.52:0/337864730 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-04T09:21:15.609 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:14.795+0000 7f4c4df8e700 1 -- 172.21.15.52:0/337864730 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c48108c10 con 0x7f4c48107a60 2024-03-04T09:21:15.609 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:14.795+0000 7f4c4df8e700 1 --2- 172.21.15.52:0/337864730 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4c48107a60 0x7f4c48107e80 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f4c440098a0 tx=0x7f4c44009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4dca21b66baf78a5 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:21:15.609 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:14.796+0000 7f4c4cf8c700 1 -- 172.21.15.52:0/337864730 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4c44004160 con 0x7f4c48107a60 2024-03-04T09:21:15.609 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:14.796+0000 7f4c4cf8c700 1 -- 172.21.15.52:0/337864730 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4c44004300 con 0x7f4c48107a60 2024-03-04T09:21:15.609 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:14.796+0000 7f4c4cf8c700 1 -- 172.21.15.52:0/337864730 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4c44011b50 con 0x7f4c48107a60 2024-03-04T09:21:15.610 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:14.796+0000 7f4c501f2700 1 -- 172.21.15.52:0/337864730 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4c48107a60 msgr2=0x7f4c48107e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:21:15.610 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:14.796+0000 7f4c501f2700 1 --2- 172.21.15.52:0/337864730 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4c48107a60 0x7f4c48107e80 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f4c440098a0 tx=0x7f4c44009bb0 comp rx=0 tx=0).stop 2024-03-04T09:21:15.610 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:14.796+0000 7f4c501f2700 1 -- 172.21.15.52:0/337864730 shutdown_connections 2024-03-04T09:21:15.610 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:14.796+0000 7f4c501f2700 1 --2- 172.21.15.52:0/337864730 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4c48107a60 0x7f4c48107e80 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:21:15.610 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:14.796+0000 7f4c501f2700 1 -- 172.21.15.52:0/337864730 >> 172.21.15.52:0/337864730 conn(0x7f4c48103060 msgr2=0x7f4c48105480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:21:15.610 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:14.796+0000 7f4c501f2700 1 -- 172.21.15.52:0/337864730 shutdown_connections 2024-03-04T09:21:15.610 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:14.796+0000 7f4c501f2700 1 -- 172.21.15.52:0/337864730 wait complete. 2024-03-04T09:21:15.610 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:14.797+0000 7f4c501f2700 1 Processor -- start 2024-03-04T09:21:15.610 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:14.797+0000 7f4c501f2700 1 -- start start 2024-03-04T09:21:15.610 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:14.797+0000 7f4c501f2700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4c48107a60 0x7f4c4819b970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:21:15.610 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:14.797+0000 7f4c501f2700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c4810bfc0 con 0x7f4c48107a60 2024-03-04T09:21:15.610 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:14.797+0000 7f4c4df8e700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4c48107a60 0x7f4c4819b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:21:15.610 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:14.797+0000 7f4c4df8e700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4c48107a60 0x7f4c4819b970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:60004/0 (socket says 172.21.15.52:60004) 2024-03-04T09:21:15.610 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:14.797+0000 7f4c4df8e700 1 -- 172.21.15.52:0/2989214130 learned_addr learned my addr 172.21.15.52:0/2989214130 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-04T09:21:15.611 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:14.797+0000 7f4c4df8e700 1 -- 172.21.15.52:0/2989214130 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c4819beb0 con 0x7f4c48107a60 2024-03-04T09:21:15.611 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:14.798+0000 7f4c4df8e700 1 --2- 172.21.15.52:0/2989214130 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4c48107a60 0x7f4c4819b970 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f4c44011670 tx=0x7f4c44011b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:21:15.611 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:14.798+0000 7f4c3effd700 1 -- 172.21.15.52:0/2989214130 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4c440039c0 con 0x7f4c48107a60 2024-03-04T09:21:15.611 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:14.798+0000 7f4c3effd700 1 -- 172.21.15.52:0/2989214130 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4c44011e50 con 0x7f4c48107a60 2024-03-04T09:21:15.611 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:14.798+0000 7f4c3effd700 1 -- 172.21.15.52:0/2989214130 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4c4401a5c0 con 0x7f4c48107a60 2024-03-04T09:21:15.611 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:14.798+0000 7f4c501f2700 1 -- 172.21.15.52:0/2989214130 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c4819c140 con 0x7f4c48107a60 2024-03-04T09:21:15.611 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:14.798+0000 7f4c501f2700 1 -- 172.21.15.52:0/2989214130 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c4819ee30 con 0x7f4c48107a60 2024-03-04T09:21:15.611 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:14.799+0000 7f4c3effd700 1 -- 172.21.15.52:0/2989214130 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 7) v1 ==== 45822+0+0 (secure 0 0 0) 0x7f4c44024430 con 0x7f4c48107a60 2024-03-04T09:21:15.611 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:14.799+0000 7f4c501f2700 1 -- 172.21.15.52:0/2989214130 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c480636f0 con 0x7f4c48107a60 2024-03-04T09:21:15.611 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:14.799+0000 7f4c3effd700 1 --2- 172.21.15.52:0/2989214130 >> [v2:172.21.15.52:6800/2425730636,v1:172.21.15.52:6801/2425730636] conn(0x7f4c34038a40 0x7f4c3403af00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:21:15.611 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:14.799+0000 7f4c3effd700 1 -- 172.21.15.52:0/2989214130 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f4c4404ccc0 con 0x7f4c48107a60 2024-03-04T09:21:15.611 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:14.799+0000 7f4c4d78d700 1 --2- 172.21.15.52:0/2989214130 >> [v2:172.21.15.52:6800/2425730636,v1:172.21.15.52:6801/2425730636] conn(0x7f4c34038a40 0x7f4c3403af00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:21:15.611 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:14.799+0000 7f4c4d78d700 1 --2- 172.21.15.52:0/2989214130 >> [v2:172.21.15.52:6800/2425730636,v1:172.21.15.52:6801/2425730636] conn(0x7f4c34038a40 0x7f4c3403af00 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f4c38006fd0 tx=0x7f4c38006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:21:15.612 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:14.804+0000 7f4c3effd700 1 -- 172.21.15.52:0/2989214130 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f4c4402d910 con 0x7f4c48107a60 2024-03-04T09:21:15.612 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:14.813+0000 7f4c3effd700 1 -- 172.21.15.52:0/2989214130 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mgrmap(e 8) v1 ==== 45928+0+0 (secure 0 0 0) 0x7f4c4402d3f0 con 0x7f4c48107a60 2024-03-04T09:21:15.612 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:15.025+0000 7f4c501f2700 1 -- 172.21.15.52:0/2989214130 --> [v2:172.21.15.52:6800/2425730636,v1:172.21.15.52:6801/2425730636] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) v1 -- 0x7f4c48104320 con 0x7f4c34038a40 2024-03-04T09:21:15.612 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:15.031+0000 7f4c3effd700 1 -- 172.21.15.52:0/2989214130 <== mgr.14120 v2:172.21.15.52:6800/2425730636 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f4c48104320 con 0x7f4c34038a40 2024-03-04T09:21:15.612 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:15.033+0000 7f4c501f2700 1 -- 172.21.15.52:0/2989214130 >> [v2:172.21.15.52:6800/2425730636,v1:172.21.15.52:6801/2425730636] conn(0x7f4c34038a40 msgr2=0x7f4c3403af00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:21:15.612 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:15.033+0000 7f4c501f2700 1 --2- 172.21.15.52:0/2989214130 >> [v2:172.21.15.52:6800/2425730636,v1:172.21.15.52:6801/2425730636] conn(0x7f4c34038a40 0x7f4c3403af00 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f4c38006fd0 tx=0x7f4c38006e40 comp rx=0 tx=0).stop 2024-03-04T09:21:15.612 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:15.033+0000 7f4c501f2700 1 -- 172.21.15.52:0/2989214130 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4c48107a60 msgr2=0x7f4c4819b970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:21:15.612 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:15.033+0000 7f4c501f2700 1 --2- 172.21.15.52:0/2989214130 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4c48107a60 0x7f4c4819b970 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f4c44011670 tx=0x7f4c44011b60 comp rx=0 tx=0).stop 2024-03-04T09:21:15.612 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:15.033+0000 7f4c501f2700 1 -- 172.21.15.52:0/2989214130 shutdown_connections 2024-03-04T09:21:15.612 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:15.033+0000 7f4c501f2700 1 --2- 172.21.15.52:0/2989214130 >> [v2:172.21.15.52:6800/2425730636,v1:172.21.15.52:6801/2425730636] conn(0x7f4c34038a40 0x7f4c3403af00 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:21:15.612 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:15.033+0000 7f4c501f2700 1 --2- 172.21.15.52:0/2989214130 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4c48107a60 0x7f4c4819b970 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:21:15.612 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:15.033+0000 7f4c501f2700 1 -- 172.21.15.52:0/2989214130 >> 172.21.15.52:0/2989214130 conn(0x7f4c48103060 msgr2=0x7f4c48103c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:21:15.612 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:15.034+0000 7f4c501f2700 1 -- 172.21.15.52:0/2989214130 shutdown_connections 2024-03-04T09:21:15.612 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:15.034+0000 7f4c501f2700 1 -- 172.21.15.52:0/2989214130 wait complete. 2024-03-04T09:21:15.944 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:21:15 smithi052 ceph-mon[32473]: mgrmap e8: x(active, since 2s) 2024-03-04T09:21:15.945 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:21:15 smithi052 ceph-mon[32473]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T09:21:15.945 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:21:15 smithi052 ceph-mon[32473]: from='mgr.14120 172.21.15.52:0/3140567671' entity='mgr.x' 2024-03-04T09:21:15.945 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:21:15 smithi052 ceph-mon[32473]: from='mgr.14120 172.21.15.52:0/3140567671' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T09:21:16.815 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout value unchanged 2024-03-04T09:21:16.815 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:16.059+0000 7f9e55baa700 1 Processor -- start 2024-03-04T09:21:16.815 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:16.060+0000 7f9e55baa700 1 -- start start 2024-03-04T09:21:16.815 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:16.060+0000 7f9e55baa700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9e50107a90 0x7f9e50107eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:21:16.815 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:16.060+0000 7f9e55baa700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e50108480 con 0x7f9e50107a90 2024-03-04T09:21:16.815 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:16.060+0000 7f9e4f7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9e50107a90 0x7f9e50107eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:21:16.815 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:16.060+0000 7f9e4f7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9e50107a90 0x7f9e50107eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:60018/0 (socket says 172.21.15.52:60018) 2024-03-04T09:21:16.816 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:16.060+0000 7f9e4f7fe700 1 -- 172.21.15.52:0/600272768 learned_addr learned my addr 172.21.15.52:0/600272768 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-04T09:21:16.816 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:16.060+0000 7f9e4f7fe700 1 -- 172.21.15.52:0/600272768 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e50108c40 con 0x7f9e50107a90 2024-03-04T09:21:16.816 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:16.060+0000 7f9e4f7fe700 1 --2- 172.21.15.52:0/600272768 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9e50107a90 0x7f9e50107eb0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f9e400098a0 tx=0x7f9e40009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4917f6f964506232 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:21:16.816 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:16.060+0000 7f9e4e7fc700 1 -- 172.21.15.52:0/600272768 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9e40004160 con 0x7f9e50107a90 2024-03-04T09:21:16.816 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:16.061+0000 7f9e4e7fc700 1 -- 172.21.15.52:0/600272768 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9e40004300 con 0x7f9e50107a90 2024-03-04T09:21:16.816 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:16.061+0000 7f9e4e7fc700 1 -- 172.21.15.52:0/600272768 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9e40011b50 con 0x7f9e50107a90 2024-03-04T09:21:16.816 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:16.061+0000 7f9e55baa700 1 -- 172.21.15.52:0/600272768 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9e50107a90 msgr2=0x7f9e50107eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:21:16.816 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:16.061+0000 7f9e55baa700 1 --2- 172.21.15.52:0/600272768 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9e50107a90 0x7f9e50107eb0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f9e400098a0 tx=0x7f9e40009bb0 comp rx=0 tx=0).stop 2024-03-04T09:21:16.816 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:16.061+0000 7f9e55baa700 1 -- 172.21.15.52:0/600272768 shutdown_connections 2024-03-04T09:21:16.816 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:16.061+0000 7f9e55baa700 1 --2- 172.21.15.52:0/600272768 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9e50107a90 0x7f9e50107eb0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:21:16.816 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:16.061+0000 7f9e55baa700 1 -- 172.21.15.52:0/600272768 >> 172.21.15.52:0/600272768 conn(0x7f9e50103090 msgr2=0x7f9e501054b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:21:16.816 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:16.061+0000 7f9e55baa700 1 -- 172.21.15.52:0/600272768 shutdown_connections 2024-03-04T09:21:16.817 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:16.061+0000 7f9e55baa700 1 -- 172.21.15.52:0/600272768 wait complete. 2024-03-04T09:21:16.817 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:16.062+0000 7f9e55baa700 1 Processor -- start 2024-03-04T09:21:16.817 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:16.062+0000 7f9e55baa700 1 -- start start 2024-03-04T09:21:16.817 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:16.062+0000 7f9e55baa700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9e50107a90 0x7f9e5019b9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:21:16.817 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:16.062+0000 7f9e55baa700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e5010bff0 con 0x7f9e50107a90 2024-03-04T09:21:16.817 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:16.062+0000 7f9e4f7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9e50107a90 0x7f9e5019b9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:21:16.817 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:16.062+0000 7f9e4f7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9e50107a90 0x7f9e5019b9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:60028/0 (socket says 172.21.15.52:60028) 2024-03-04T09:21:16.817 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:16.062+0000 7f9e4f7fe700 1 -- 172.21.15.52:0/4156378577 learned_addr learned my addr 172.21.15.52:0/4156378577 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-04T09:21:16.817 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:16.063+0000 7f9e4f7fe700 1 -- 172.21.15.52:0/4156378577 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e5019bf30 con 0x7f9e50107a90 2024-03-04T09:21:16.817 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:16.063+0000 7f9e4f7fe700 1 --2- 172.21.15.52:0/4156378577 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9e50107a90 0x7f9e5019b9f0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f9e40009fb0 tx=0x7f9e40011bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:21:16.817 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:16.063+0000 7f9e4cff9700 1 -- 172.21.15.52:0/4156378577 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9e400038b0 con 0x7f9e50107a90 2024-03-04T09:21:16.817 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:16.063+0000 7f9e4cff9700 1 -- 172.21.15.52:0/4156378577 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9e40011e50 con 0x7f9e50107a90 2024-03-04T09:21:16.817 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:16.063+0000 7f9e55baa700 1 -- 172.21.15.52:0/4156378577 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e5019c1c0 con 0x7f9e50107a90 2024-03-04T09:21:16.818 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:16.063+0000 7f9e4cff9700 1 -- 172.21.15.52:0/4156378577 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9e40019550 con 0x7f9e50107a90 2024-03-04T09:21:16.818 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:16.063+0000 7f9e55baa700 1 -- 172.21.15.52:0/4156378577 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e5019eeb0 con 0x7f9e50107a90 2024-03-04T09:21:16.818 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:16.064+0000 7f9e4cff9700 1 -- 172.21.15.52:0/4156378577 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 8) v1 ==== 45928+0+0 (secure 0 0 0) 0x7f9e40019800 con 0x7f9e50107a90 2024-03-04T09:21:16.818 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:16.064+0000 7f9e55baa700 1 -- 172.21.15.52:0/4156378577 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9e50069170 con 0x7f9e50107a90 2024-03-04T09:21:16.818 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:16.064+0000 7f9e4cff9700 1 --2- 172.21.15.52:0/4156378577 >> [v2:172.21.15.52:6800/2425730636,v1:172.21.15.52:6801/2425730636] conn(0x7f9e38038ba0 0x7f9e3803b060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:21:16.818 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:16.064+0000 7f9e4cff9700 1 -- 172.21.15.52:0/4156378577 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f9e4004cdc0 con 0x7f9e50107a90 2024-03-04T09:21:16.818 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:16.064+0000 7f9e4effd700 1 --2- 172.21.15.52:0/4156378577 >> [v2:172.21.15.52:6800/2425730636,v1:172.21.15.52:6801/2425730636] conn(0x7f9e38038ba0 0x7f9e3803b060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:21:16.818 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:16.065+0000 7f9e4effd700 1 --2- 172.21.15.52:0/4156378577 >> [v2:172.21.15.52:6800/2425730636,v1:172.21.15.52:6801/2425730636] conn(0x7f9e38038ba0 0x7f9e3803b060 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f9e44006fd0 tx=0x7f9e44006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:21:16.818 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:16.070+0000 7f9e4cff9700 1 -- 172.21.15.52:0/4156378577 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f9e4002d960 con 0x7f9e50107a90 2024-03-04T09:21:16.818 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:16.275+0000 7f9e55baa700 1 -- 172.21.15.52:0/4156378577 --> [v2:172.21.15.52:6800/2425730636,v1:172.21.15.52:6801/2425730636] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) v1 -- 0x7f9e501043a0 con 0x7f9e38038ba0 2024-03-04T09:21:16.818 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:16.275+0000 7f9e4cff9700 1 -- 172.21.15.52:0/4156378577 <== mgr.14120 v2:172.21.15.52:6800/2425730636 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+16 (secure 0 0 0) 0x7f9e501043a0 con 0x7f9e38038ba0 2024-03-04T09:21:16.818 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:16.278+0000 7f9e55baa700 1 -- 172.21.15.52:0/4156378577 >> [v2:172.21.15.52:6800/2425730636,v1:172.21.15.52:6801/2425730636] conn(0x7f9e38038ba0 msgr2=0x7f9e3803b060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:21:16.818 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:16.278+0000 7f9e55baa700 1 --2- 172.21.15.52:0/4156378577 >> [v2:172.21.15.52:6800/2425730636,v1:172.21.15.52:6801/2425730636] conn(0x7f9e38038ba0 0x7f9e3803b060 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f9e44006fd0 tx=0x7f9e44006e40 comp rx=0 tx=0).stop 2024-03-04T09:21:16.818 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:16.278+0000 7f9e55baa700 1 -- 172.21.15.52:0/4156378577 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9e50107a90 msgr2=0x7f9e5019b9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:21:16.819 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:16.278+0000 7f9e55baa700 1 --2- 172.21.15.52:0/4156378577 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9e50107a90 0x7f9e5019b9f0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f9e40009fb0 tx=0x7f9e40011bf0 comp rx=0 tx=0).stop 2024-03-04T09:21:16.819 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:16.278+0000 7f9e55baa700 1 -- 172.21.15.52:0/4156378577 shutdown_connections 2024-03-04T09:21:16.819 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:16.278+0000 7f9e55baa700 1 --2- 172.21.15.52:0/4156378577 >> [v2:172.21.15.52:6800/2425730636,v1:172.21.15.52:6801/2425730636] conn(0x7f9e38038ba0 0x7f9e3803b060 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:21:16.819 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:16.278+0000 7f9e55baa700 1 --2- 172.21.15.52:0/4156378577 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9e50107a90 0x7f9e5019b9f0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:21:16.819 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:16.278+0000 7f9e55baa700 1 -- 172.21.15.52:0/4156378577 >> 172.21.15.52:0/4156378577 conn(0x7f9e50103090 msgr2=0x7f9e50103c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:21:16.819 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:16.278+0000 7f9e55baa700 1 -- 172.21.15.52:0/4156378577 shutdown_connections 2024-03-04T09:21:16.819 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:16.278+0000 7f9e55baa700 1 -- 172.21.15.52:0/4156378577 wait complete. 2024-03-04T09:21:16.819 INFO:teuthology.orchestra.run.smithi052.stdout:Generating ssh key... 2024-03-04T09:21:17.084 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:21:16 smithi052 ceph-mon[32473]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T09:21:17.835 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Mar 04 09:21:17 smithi052 ceph-4e059f08-da08-11ee-95c6-87774f69a715-mgr-x[32705]: Generating public/private rsa key pair. 2024-03-04T09:21:17.835 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Mar 04 09:21:17 smithi052 ceph-4e059f08-da08-11ee-95c6-87774f69a715-mgr-x[32705]: Your identification has been saved in /tmp/tmp_osk7vjl/key. 2024-03-04T09:21:17.835 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Mar 04 09:21:17 smithi052 ceph-4e059f08-da08-11ee-95c6-87774f69a715-mgr-x[32705]: Your public key has been saved in /tmp/tmp_osk7vjl/key.pub. 2024-03-04T09:21:17.835 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Mar 04 09:21:17 smithi052 ceph-4e059f08-da08-11ee-95c6-87774f69a715-mgr-x[32705]: The key fingerprint is: 2024-03-04T09:21:17.835 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Mar 04 09:21:17 smithi052 ceph-4e059f08-da08-11ee-95c6-87774f69a715-mgr-x[32705]: SHA256:ZgnhLi/5ePxUM4ax9JrOIvkY1c3nwWnz8qhaoSYaAbE ceph-4e059f08-da08-11ee-95c6-87774f69a715 2024-03-04T09:21:17.835 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Mar 04 09:21:17 smithi052 ceph-4e059f08-da08-11ee-95c6-87774f69a715-mgr-x[32705]: The key's randomart image is: 2024-03-04T09:21:17.836 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Mar 04 09:21:17 smithi052 ceph-4e059f08-da08-11ee-95c6-87774f69a715-mgr-x[32705]: +---[RSA 3072]----+ 2024-03-04T09:21:17.836 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Mar 04 09:21:17 smithi052 ceph-4e059f08-da08-11ee-95c6-87774f69a715-mgr-x[32705]: | . . | 2024-03-04T09:21:17.836 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Mar 04 09:21:17 smithi052 ceph-4e059f08-da08-11ee-95c6-87774f69a715-mgr-x[32705]: | o . . | 2024-03-04T09:21:17.836 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Mar 04 09:21:17 smithi052 ceph-4e059f08-da08-11ee-95c6-87774f69a715-mgr-x[32705]: | E o o | 2024-03-04T09:21:17.836 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Mar 04 09:21:17 smithi052 ceph-4e059f08-da08-11ee-95c6-87774f69a715-mgr-x[32705]: | . . + O . . | 2024-03-04T09:21:17.836 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Mar 04 09:21:17 smithi052 ceph-4e059f08-da08-11ee-95c6-87774f69a715-mgr-x[32705]: | o o S X B | 2024-03-04T09:21:17.836 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Mar 04 09:21:17 smithi052 ceph-4e059f08-da08-11ee-95c6-87774f69a715-mgr-x[32705]: | * o * B + | 2024-03-04T09:21:17.836 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Mar 04 09:21:17 smithi052 ceph-4e059f08-da08-11ee-95c6-87774f69a715-mgr-x[32705]: | =oo * . o . | 2024-03-04T09:21:17.836 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Mar 04 09:21:17 smithi052 ceph-4e059f08-da08-11ee-95c6-87774f69a715-mgr-x[32705]: | oO+* . + | 2024-03-04T09:21:17.836 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Mar 04 09:21:17 smithi052 ceph-4e059f08-da08-11ee-95c6-87774f69a715-mgr-x[32705]: | ++oo=.... . | 2024-03-04T09:21:17.836 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Mar 04 09:21:17 smithi052 ceph-4e059f08-da08-11ee-95c6-87774f69a715-mgr-x[32705]: +----[SHA256]-----+ 2024-03-04T09:21:18.089 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:17.208+0000 7f9bb9051700 1 Processor -- start 2024-03-04T09:21:18.089 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:17.208+0000 7f9bb9051700 1 -- start start 2024-03-04T09:21:18.089 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:17.208+0000 7f9bb9051700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9bb4105d30 0x7f9bb4106150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:21:18.089 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:17.208+0000 7f9bb9051700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bb4106720 con 0x7f9bb4105d30 2024-03-04T09:21:18.090 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:17.208+0000 7f9bb2d9d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9bb4105d30 0x7f9bb4106150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:21:18.090 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:17.208+0000 7f9bb2d9d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9bb4105d30 0x7f9bb4106150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:60034/0 (socket says 172.21.15.52:60034) 2024-03-04T09:21:18.090 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:17.208+0000 7f9bb2d9d700 1 -- 172.21.15.52:0/3756293745 learned_addr learned my addr 172.21.15.52:0/3756293745 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-04T09:21:18.090 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:17.208+0000 7f9bb2d9d700 1 -- 172.21.15.52:0/3756293745 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9bb4108cc0 con 0x7f9bb4105d30 2024-03-04T09:21:18.090 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:17.208+0000 7f9bb2d9d700 1 --2- 172.21.15.52:0/3756293745 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9bb4105d30 0x7f9bb4106150 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f9ba40098a0 tx=0x7f9ba4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6574c8989669b5e5 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:21:18.090 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:17.209+0000 7f9bb1d9b700 1 -- 172.21.15.52:0/3756293745 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9ba4004160 con 0x7f9bb4105d30 2024-03-04T09:21:18.090 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:17.209+0000 7f9bb1d9b700 1 -- 172.21.15.52:0/3756293745 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9ba4004300 con 0x7f9bb4105d30 2024-03-04T09:21:18.090 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:17.209+0000 7f9bb1d9b700 1 -- 172.21.15.52:0/3756293745 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9ba4011b50 con 0x7f9bb4105d30 2024-03-04T09:21:18.090 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:17.209+0000 7f9bb9051700 1 -- 172.21.15.52:0/3756293745 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9bb4105d30 msgr2=0x7f9bb4106150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:21:18.090 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:17.209+0000 7f9bb9051700 1 --2- 172.21.15.52:0/3756293745 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9bb4105d30 0x7f9bb4106150 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f9ba40098a0 tx=0x7f9ba4009bb0 comp rx=0 tx=0).stop 2024-03-04T09:21:18.090 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:17.209+0000 7f9bb9051700 1 -- 172.21.15.52:0/3756293745 shutdown_connections 2024-03-04T09:21:18.090 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:17.209+0000 7f9bb9051700 1 --2- 172.21.15.52:0/3756293745 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9bb4105d30 0x7f9bb4106150 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:21:18.090 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:17.209+0000 7f9bb9051700 1 -- 172.21.15.52:0/3756293745 >> 172.21.15.52:0/3756293745 conn(0x7f9bb4101680 msgr2=0x7f9bb4103aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:21:18.090 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:17.209+0000 7f9bb9051700 1 -- 172.21.15.52:0/3756293745 shutdown_connections 2024-03-04T09:21:18.090 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:17.209+0000 7f9bb9051700 1 -- 172.21.15.52:0/3756293745 wait complete. 2024-03-04T09:21:18.091 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:17.210+0000 7f9bb9051700 1 Processor -- start 2024-03-04T09:21:18.091 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:17.210+0000 7f9bb9051700 1 -- start start 2024-03-04T09:21:18.091 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:17.210+0000 7f9bb9051700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9bb4105d30 0x7f9bb419ba00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:21:18.091 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:17.210+0000 7f9bb9051700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bb410c070 con 0x7f9bb4105d30 2024-03-04T09:21:18.091 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:17.211+0000 7f9bb2d9d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9bb4105d30 0x7f9bb419ba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:21:18.091 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:17.211+0000 7f9bb2d9d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9bb4105d30 0x7f9bb419ba00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:60042/0 (socket says 172.21.15.52:60042) 2024-03-04T09:21:18.092 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:17.211+0000 7f9bb2d9d700 1 -- 172.21.15.52:0/2396233449 learned_addr learned my addr 172.21.15.52:0/2396233449 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-04T09:21:18.092 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:17.211+0000 7f9bb2d9d700 1 -- 172.21.15.52:0/2396233449 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9bb419bf40 con 0x7f9bb4105d30 2024-03-04T09:21:18.092 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:17.211+0000 7f9bb2d9d700 1 --2- 172.21.15.52:0/2396233449 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9bb4105d30 0x7f9bb419ba00 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f9ba4009fb0 tx=0x7f9ba4011bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:21:18.092 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:17.211+0000 7f9b9bfff700 1 -- 172.21.15.52:0/2396233449 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9ba40038b0 con 0x7f9bb4105d30 2024-03-04T09:21:18.092 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:17.211+0000 7f9bb9051700 1 -- 172.21.15.52:0/2396233449 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9bb419c1d0 con 0x7f9bb4105d30 2024-03-04T09:21:18.092 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:17.211+0000 7f9b9bfff700 1 -- 172.21.15.52:0/2396233449 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9ba4011e50 con 0x7f9bb4105d30 2024-03-04T09:21:18.093 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:17.211+0000 7f9b9bfff700 1 -- 172.21.15.52:0/2396233449 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9ba4019550 con 0x7f9bb4105d30 2024-03-04T09:21:18.093 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:17.211+0000 7f9bb9051700 1 -- 172.21.15.52:0/2396233449 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9bb419eec0 con 0x7f9bb4105d30 2024-03-04T09:21:18.093 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:17.212+0000 7f9b9bfff700 1 -- 172.21.15.52:0/2396233449 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 8) v1 ==== 45928+0+0 (secure 0 0 0) 0x7f9ba4019800 con 0x7f9bb4105d30 2024-03-04T09:21:18.093 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:17.212+0000 7f9bb9051700 1 -- 172.21.15.52:0/2396233449 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9bb4069170 con 0x7f9bb4105d30 2024-03-04T09:21:18.093 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:17.212+0000 7f9b9bfff700 1 --2- 172.21.15.52:0/2396233449 >> [v2:172.21.15.52:6800/2425730636,v1:172.21.15.52:6801/2425730636] conn(0x7f9b9c038ba0 0x7f9b9c03b060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:21:18.093 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:17.212+0000 7f9b9bfff700 1 -- 172.21.15.52:0/2396233449 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f9ba404cc40 con 0x7f9bb4105d30 2024-03-04T09:21:18.094 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:17.212+0000 7f9bb259c700 1 --2- 172.21.15.52:0/2396233449 >> [v2:172.21.15.52:6800/2425730636,v1:172.21.15.52:6801/2425730636] conn(0x7f9b9c038ba0 0x7f9b9c03b060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:21:18.094 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:17.213+0000 7f9bb259c700 1 --2- 172.21.15.52:0/2396233449 >> [v2:172.21.15.52:6800/2425730636,v1:172.21.15.52:6801/2425730636] conn(0x7f9b9c038ba0 0x7f9b9c03b060 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f9ba8006fd0 tx=0x7f9ba8006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:21:18.094 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:17.217+0000 7f9b9bfff700 1 -- 172.21.15.52:0/2396233449 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f9ba402d960 con 0x7f9bb4105d30 2024-03-04T09:21:18.094 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:17.422+0000 7f9bb9051700 1 -- 172.21.15.52:0/2396233449 --> [v2:172.21.15.52:6800/2425730636,v1:172.21.15.52:6801/2425730636] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) v1 -- 0x7f9bb4102940 con 0x7f9b9c038ba0 2024-03-04T09:21:18.094 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:17.561+0000 7f9b9bfff700 1 -- 172.21.15.52:0/2396233449 <== mgr.14120 v2:172.21.15.52:6800/2425730636 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f9bb4102940 con 0x7f9b9c038ba0 2024-03-04T09:21:18.094 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:17.563+0000 7f9bb9051700 1 -- 172.21.15.52:0/2396233449 >> [v2:172.21.15.52:6800/2425730636,v1:172.21.15.52:6801/2425730636] conn(0x7f9b9c038ba0 msgr2=0x7f9b9c03b060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:21:18.094 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:17.563+0000 7f9bb9051700 1 --2- 172.21.15.52:0/2396233449 >> [v2:172.21.15.52:6800/2425730636,v1:172.21.15.52:6801/2425730636] conn(0x7f9b9c038ba0 0x7f9b9c03b060 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f9ba8006fd0 tx=0x7f9ba8006e40 comp rx=0 tx=0).stop 2024-03-04T09:21:18.094 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:17.563+0000 7f9bb9051700 1 -- 172.21.15.52:0/2396233449 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9bb4105d30 msgr2=0x7f9bb419ba00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:21:18.094 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:17.563+0000 7f9bb9051700 1 --2- 172.21.15.52:0/2396233449 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9bb4105d30 0x7f9bb419ba00 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f9ba4009fb0 tx=0x7f9ba4011bf0 comp rx=0 tx=0).stop 2024-03-04T09:21:18.094 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:17.563+0000 7f9bb9051700 1 -- 172.21.15.52:0/2396233449 shutdown_connections 2024-03-04T09:21:18.094 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:17.563+0000 7f9bb9051700 1 --2- 172.21.15.52:0/2396233449 >> [v2:172.21.15.52:6800/2425730636,v1:172.21.15.52:6801/2425730636] conn(0x7f9b9c038ba0 0x7f9b9c03b060 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:21:18.094 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:17.564+0000 7f9bb9051700 1 --2- 172.21.15.52:0/2396233449 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9bb4105d30 0x7f9bb419ba00 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:21:18.095 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:17.564+0000 7f9bb9051700 1 -- 172.21.15.52:0/2396233449 >> 172.21.15.52:0/2396233449 conn(0x7f9bb4101680 msgr2=0x7f9bb4102230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:21:18.095 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:17.564+0000 7f9bb9051700 1 -- 172.21.15.52:0/2396233449 shutdown_connections 2024-03-04T09:21:18.095 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:17.564+0000 7f9bb9051700 1 -- 172.21.15.52:0/2396233449 wait complete. 2024-03-04T09:21:18.751 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:21:18 smithi052 ceph-mon[32473]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T09:21:18.751 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:21:18 smithi052 ceph-mon[32473]: Generating ssh key... 2024-03-04T09:21:18.751 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:21:18 smithi052 ceph-mon[32473]: from='mgr.14120 172.21.15.52:0/3140567671' entity='mgr.x' 2024-03-04T09:21:18.751 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:21:18 smithi052 ceph-mon[32473]: from='mgr.14120 172.21.15.52:0/3140567671' entity='mgr.x' 2024-03-04T09:21:19.271 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDBw1lK2UDv8zi6usYCcvXwie9Yt7cWtkCfM0J6TUU8MUnE1I4NluKTDKCK6OSPeKlnIWvWxKZaTWli9wX+Gc4O86yUw5R26W3H+JqnMXDUSDgSM1mDtYn39KpkkQQs3dVXpJODW3oZEbqj7bubB/EjiCmDzd7k4mBV18dnDLJNsGI4yFojydHD2VzIMg3Fb3ZvtwQk3o8OOet4hobp1T2Im0GRJIzmLVSoZ9IsgWxaYlvRbt5qmnAN14dZJ/psDCz2noz5JkqklqWl/ln8xS8A4nRzZs0bNbn5kcbBWt+sNiQIep3n8A+E6AW2CpX+OB7lcenUaNpUXbwL5/mF79hBfseOiqA59CAMntwivu5HYhDW1bPGac8UHGGq+P3E9OECF5p5t0q3Zqr8CcWiPU7iqUFZA5kAwiRIVpcnwhrn1h54RNf2AyTShmF8NTk/Y7CIrFxmnww2eHC5q62J4d3rfAyjYpwzvsZGu5Rua/WAx9LyFffhLcjdU1NvX26s1v8= ceph-4e059f08-da08-11ee-95c6-87774f69a715 2024-03-04T09:21:19.271 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:18.525+0000 7f7272bc8700 1 Processor -- start 2024-03-04T09:21:19.271 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:18.525+0000 7f7272bc8700 1 -- start start 2024-03-04T09:21:19.271 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:18.525+0000 7f7272bc8700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f726c07cea0 0x7f726c07b350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:21:19.271 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:18.525+0000 7f7272bc8700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f726c07b920 con 0x7f726c07cea0 2024-03-04T09:21:19.271 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:18.525+0000 7f7270964700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f726c07cea0 0x7f726c07b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:21:19.271 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:18.525+0000 7f7270964700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f726c07cea0 0x7f726c07b350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:60048/0 (socket says 172.21.15.52:60048) 2024-03-04T09:21:19.271 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:18.525+0000 7f7270964700 1 -- 172.21.15.52:0/3117570844 learned_addr learned my addr 172.21.15.52:0/3117570844 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-04T09:21:19.271 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:18.526+0000 7f7270964700 1 -- 172.21.15.52:0/3117570844 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f726c07baa0 con 0x7f726c07cea0 2024-03-04T09:21:19.271 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:18.526+0000 7f7270964700 1 --2- 172.21.15.52:0/3117570844 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f726c07cea0 0x7f726c07b350 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f72600098a0 tx=0x7f7260009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5eddbdbce431f78a server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:21:19.272 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:18.526+0000 7f726b7fe700 1 -- 172.21.15.52:0/3117570844 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7260004160 con 0x7f726c07cea0 2024-03-04T09:21:19.272 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:18.526+0000 7f726b7fe700 1 -- 172.21.15.52:0/3117570844 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7260004300 con 0x7f726c07cea0 2024-03-04T09:21:19.272 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:18.526+0000 7f726b7fe700 1 -- 172.21.15.52:0/3117570844 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7260011b50 con 0x7f726c07cea0 2024-03-04T09:21:19.272 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:18.527+0000 7f7272bc8700 1 -- 172.21.15.52:0/3117570844 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f726c07cea0 msgr2=0x7f726c07b350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:21:19.272 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:18.527+0000 7f7272bc8700 1 --2- 172.21.15.52:0/3117570844 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f726c07cea0 0x7f726c07b350 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f72600098a0 tx=0x7f7260009bb0 comp rx=0 tx=0).stop 2024-03-04T09:21:19.272 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:18.527+0000 7f7272bc8700 1 -- 172.21.15.52:0/3117570844 shutdown_connections 2024-03-04T09:21:19.272 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:18.527+0000 7f7272bc8700 1 --2- 172.21.15.52:0/3117570844 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f726c07cea0 0x7f726c07b350 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:21:19.272 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:18.527+0000 7f7272bc8700 1 -- 172.21.15.52:0/3117570844 >> 172.21.15.52:0/3117570844 conn(0x7f726c102ff0 msgr2=0x7f726c105410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:21:19.272 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:18.527+0000 7f7272bc8700 1 -- 172.21.15.52:0/3117570844 shutdown_connections 2024-03-04T09:21:19.272 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:18.527+0000 7f7272bc8700 1 -- 172.21.15.52:0/3117570844 wait complete. 2024-03-04T09:21:19.273 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:18.527+0000 7f7272bc8700 1 Processor -- start 2024-03-04T09:21:19.273 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:18.527+0000 7f7272bc8700 1 -- start start 2024-03-04T09:21:19.273 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:18.527+0000 7f7272bc8700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f726c07cea0 0x7f726c19db90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:21:19.273 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:18.528+0000 7f7272bc8700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f726c10dfe0 con 0x7f726c07cea0 2024-03-04T09:21:19.273 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:18.528+0000 7f7270964700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f726c07cea0 0x7f726c19db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:21:19.273 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:18.528+0000 7f7270964700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f726c07cea0 0x7f726c19db90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:60052/0 (socket says 172.21.15.52:60052) 2024-03-04T09:21:19.273 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:18.528+0000 7f7270964700 1 -- 172.21.15.52:0/51626329 learned_addr learned my addr 172.21.15.52:0/51626329 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-04T09:21:19.273 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:18.528+0000 7f7270964700 1 -- 172.21.15.52:0/51626329 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f726c19e0d0 con 0x7f726c07cea0 2024-03-04T09:21:19.273 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:18.528+0000 7f7270964700 1 --2- 172.21.15.52:0/51626329 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f726c07cea0 0x7f726c19db90 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f7260011b60 tx=0x7f7260011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:21:19.273 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:18.528+0000 7f7269ffb700 1 -- 172.21.15.52:0/51626329 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7260003970 con 0x7f726c07cea0 2024-03-04T09:21:19.273 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:18.528+0000 7f7269ffb700 1 -- 172.21.15.52:0/51626329 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7260003b10 con 0x7f726c07cea0 2024-03-04T09:21:19.273 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:18.528+0000 7f7272bc8700 1 -- 172.21.15.52:0/51626329 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f726c19e360 con 0x7f726c07cea0 2024-03-04T09:21:19.273 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:18.528+0000 7f7269ffb700 1 -- 172.21.15.52:0/51626329 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7260024610 con 0x7f726c07cea0 2024-03-04T09:21:19.273 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:18.528+0000 7f7272bc8700 1 -- 172.21.15.52:0/51626329 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f726c1a1050 con 0x7f726c07cea0 2024-03-04T09:21:19.274 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:18.529+0000 7f7269ffb700 1 -- 172.21.15.52:0/51626329 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 8) v1 ==== 45928+0+0 (secure 0 0 0) 0x7f72600248c0 con 0x7f726c07cea0 2024-03-04T09:21:19.274 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:18.529+0000 7f7269ffb700 1 --2- 172.21.15.52:0/51626329 >> [v2:172.21.15.52:6800/2425730636,v1:172.21.15.52:6801/2425730636] conn(0x7f725c038bf0 0x7f725c03b0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:21:19.274 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:18.530+0000 7f7269ffb700 1 -- 172.21.15.52:0/51626329 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f7260011e00 con 0x7f726c07cea0 2024-03-04T09:21:19.274 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:18.529+0000 7f7272bc8700 1 -- 172.21.15.52:0/51626329 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f726c069170 con 0x7f726c07cea0 2024-03-04T09:21:19.274 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:18.530+0000 7f726bfff700 1 --2- 172.21.15.52:0/51626329 >> [v2:172.21.15.52:6800/2425730636,v1:172.21.15.52:6801/2425730636] conn(0x7f725c038bf0 0x7f725c03b0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:21:19.274 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:18.530+0000 7f726bfff700 1 --2- 172.21.15.52:0/51626329 >> [v2:172.21.15.52:6800/2425730636,v1:172.21.15.52:6801/2425730636] conn(0x7f725c038bf0 0x7f725c03b0b0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f7258006fd0 tx=0x7f7258006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:21:19.274 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:18.535+0000 7f7269ffb700 1 -- 172.21.15.52:0/51626329 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f726000ba00 con 0x7f726c07cea0 2024-03-04T09:21:19.274 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:18.741+0000 7f7272bc8700 1 -- 172.21.15.52:0/51626329 --> [v2:172.21.15.52:6800/2425730636,v1:172.21.15.52:6801/2425730636] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) v1 -- 0x7f726c104280 con 0x7f725c038bf0 2024-03-04T09:21:19.274 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:18.741+0000 7f7269ffb700 1 -- 172.21.15.52:0/51626329 <== mgr.14120 v2:172.21.15.52:6800/2425730636 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+595 (secure 0 0 0) 0x7f726c104280 con 0x7f725c038bf0 2024-03-04T09:21:19.274 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:18.743+0000 7f7272bc8700 1 -- 172.21.15.52:0/51626329 >> [v2:172.21.15.52:6800/2425730636,v1:172.21.15.52:6801/2425730636] conn(0x7f725c038bf0 msgr2=0x7f725c03b0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:21:19.274 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:18.743+0000 7f7272bc8700 1 --2- 172.21.15.52:0/51626329 >> [v2:172.21.15.52:6800/2425730636,v1:172.21.15.52:6801/2425730636] conn(0x7f725c038bf0 0x7f725c03b0b0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f7258006fd0 tx=0x7f7258006e40 comp rx=0 tx=0).stop 2024-03-04T09:21:19.274 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:18.743+0000 7f7272bc8700 1 -- 172.21.15.52:0/51626329 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f726c07cea0 msgr2=0x7f726c19db90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:21:19.274 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:18.743+0000 7f7272bc8700 1 --2- 172.21.15.52:0/51626329 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f726c07cea0 0x7f726c19db90 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f7260011b60 tx=0x7f7260011c40 comp rx=0 tx=0).stop 2024-03-04T09:21:19.274 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:18.743+0000 7f7272bc8700 1 -- 172.21.15.52:0/51626329 shutdown_connections 2024-03-04T09:21:19.275 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:18.743+0000 7f7272bc8700 1 --2- 172.21.15.52:0/51626329 >> [v2:172.21.15.52:6800/2425730636,v1:172.21.15.52:6801/2425730636] conn(0x7f725c038bf0 0x7f725c03b0b0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:21:19.275 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:18.743+0000 7f7272bc8700 1 --2- 172.21.15.52:0/51626329 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f726c07cea0 0x7f726c19db90 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:21:19.275 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:18.743+0000 7f7272bc8700 1 -- 172.21.15.52:0/51626329 >> 172.21.15.52:0/51626329 conn(0x7f726c102ff0 msgr2=0x7f726c103b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:21:19.275 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:18.744+0000 7f7272bc8700 1 -- 172.21.15.52:0/51626329 shutdown_connections 2024-03-04T09:21:19.275 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:18.744+0000 7f7272bc8700 1 -- 172.21.15.52:0/51626329 wait complete. 2024-03-04T09:21:19.275 INFO:teuthology.orchestra.run.smithi052.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-03-04T09:21:19.275 INFO:teuthology.orchestra.run.smithi052.stdout:Adding key to root@localhost authorized_keys... 2024-03-04T09:21:19.275 INFO:teuthology.orchestra.run.smithi052.stdout:Adding host smithi052... 2024-03-04T09:21:19.834 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:21:19 smithi052 ceph-mon[32473]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T09:21:21.084 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:21:20 smithi052 ceph-mon[32473]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi052", "addr": "172.21.15.52", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T09:21:21.085 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:21:20 smithi052 ceph-mon[32473]: Deploying cephadm binary to smithi052 2024-03-04T09:21:25.104 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout Added host 'smithi052' with addr '172.21.15.52' 2024-03-04T09:21:25.104 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:19.745+0000 7f7860da0700 1 Processor -- start 2024-03-04T09:21:25.104 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:19.745+0000 7f7860da0700 1 -- start start 2024-03-04T09:21:25.105 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:19.746+0000 7f7860da0700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f785c105b00 0x7f785c105f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:21:25.105 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:19.746+0000 7f7860da0700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f785c1064f0 con 0x7f785c105b00 2024-03-04T09:21:25.105 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:19.746+0000 7f785b7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f785c105b00 0x7f785c105f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:21:25.105 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:19.746+0000 7f785b7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f785c105b00 0x7f785c105f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:60060/0 (socket says 172.21.15.52:60060) 2024-03-04T09:21:25.105 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:19.746+0000 7f785b7fe700 1 -- 172.21.15.52:0/2243789414 learned_addr learned my addr 172.21.15.52:0/2243789414 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-04T09:21:25.105 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:19.746+0000 7f785b7fe700 1 -- 172.21.15.52:0/2243789414 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f785c108a90 con 0x7f785c105b00 2024-03-04T09:21:25.105 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:19.746+0000 7f785b7fe700 1 --2- 172.21.15.52:0/2243789414 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f785c105b00 0x7f785c105f20 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f784c0098a0 tx=0x7f784c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0cd68a097ed1207 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:21:25.105 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:19.746+0000 7f785a7fc700 1 -- 172.21.15.52:0/2243789414 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f784c004160 con 0x7f785c105b00 2024-03-04T09:21:25.105 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:19.746+0000 7f785a7fc700 1 -- 172.21.15.52:0/2243789414 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f784c004300 con 0x7f785c105b00 2024-03-04T09:21:25.105 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:19.746+0000 7f785a7fc700 1 -- 172.21.15.52:0/2243789414 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f784c011b50 con 0x7f785c105b00 2024-03-04T09:21:25.105 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:19.747+0000 7f7860da0700 1 -- 172.21.15.52:0/2243789414 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f785c105b00 msgr2=0x7f785c105f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:21:25.105 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:19.747+0000 7f7860da0700 1 --2- 172.21.15.52:0/2243789414 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f785c105b00 0x7f785c105f20 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f784c0098a0 tx=0x7f784c009bb0 comp rx=0 tx=0).stop 2024-03-04T09:21:25.106 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:19.747+0000 7f7860da0700 1 -- 172.21.15.52:0/2243789414 shutdown_connections 2024-03-04T09:21:25.106 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:19.747+0000 7f7860da0700 1 --2- 172.21.15.52:0/2243789414 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f785c105b00 0x7f785c105f20 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:21:25.106 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:19.747+0000 7f7860da0700 1 -- 172.21.15.52:0/2243789414 >> 172.21.15.52:0/2243789414 conn(0x7f785c101470 msgr2=0x7f785c103890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:21:25.106 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:19.747+0000 7f7860da0700 1 -- 172.21.15.52:0/2243789414 shutdown_connections 2024-03-04T09:21:25.106 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:19.747+0000 7f7860da0700 1 -- 172.21.15.52:0/2243789414 wait complete. 2024-03-04T09:21:25.106 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:19.748+0000 7f7860da0700 1 Processor -- start 2024-03-04T09:21:25.106 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:19.748+0000 7f7860da0700 1 -- start start 2024-03-04T09:21:25.106 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:19.748+0000 7f7860da0700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f785c105b00 0x7f785c108310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:21:25.106 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:19.748+0000 7f7860da0700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f785c10be40 con 0x7f785c105b00 2024-03-04T09:21:25.106 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:19.748+0000 7f785b7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f785c105b00 0x7f785c108310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:21:25.106 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:19.748+0000 7f785b7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f785c105b00 0x7f785c108310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:60068/0 (socket says 172.21.15.52:60068) 2024-03-04T09:21:25.107 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:19.748+0000 7f785b7fe700 1 -- 172.21.15.52:0/944767163 learned_addr learned my addr 172.21.15.52:0/944767163 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-04T09:21:25.107 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:19.748+0000 7f785b7fe700 1 -- 172.21.15.52:0/944767163 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f785c108850 con 0x7f785c105b00 2024-03-04T09:21:25.107 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:19.749+0000 7f785b7fe700 1 --2- 172.21.15.52:0/944767163 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f785c105b00 0x7f785c108310 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f784c009fb0 tx=0x7f784c011bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:21:25.107 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:19.749+0000 7f7858ff9700 1 -- 172.21.15.52:0/944767163 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f784c0038b0 con 0x7f785c105b00 2024-03-04T09:21:25.107 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:19.749+0000 7f7858ff9700 1 -- 172.21.15.52:0/944767163 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f784c011e50 con 0x7f785c105b00 2024-03-04T09:21:25.107 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:19.749+0000 7f7860da0700 1 -- 172.21.15.52:0/944767163 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f785c106a40 con 0x7f785c105b00 2024-03-04T09:21:25.107 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:19.749+0000 7f7858ff9700 1 -- 172.21.15.52:0/944767163 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f784c019550 con 0x7f785c105b00 2024-03-04T09:21:25.107 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:19.749+0000 7f7860da0700 1 -- 172.21.15.52:0/944767163 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f785c106f80 con 0x7f785c105b00 2024-03-04T09:21:25.107 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:19.750+0000 7f7858ff9700 1 -- 172.21.15.52:0/944767163 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 8) v1 ==== 45928+0+0 (secure 0 0 0) 0x7f784c019850 con 0x7f785c105b00 2024-03-04T09:21:25.107 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:19.750+0000 7f7860da0700 1 -- 172.21.15.52:0/944767163 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f785c069170 con 0x7f785c105b00 2024-03-04T09:21:25.107 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:19.750+0000 7f7858ff9700 1 --2- 172.21.15.52:0/944767163 >> [v2:172.21.15.52:6800/2425730636,v1:172.21.15.52:6801/2425730636] conn(0x7f7844038b50 0x7f784403b010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:21:25.107 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:19.750+0000 7f7858ff9700 1 -- 172.21.15.52:0/944767163 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f784c04ccb0 con 0x7f785c105b00 2024-03-04T09:21:25.107 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:19.750+0000 7f785affd700 1 --2- 172.21.15.52:0/944767163 >> [v2:172.21.15.52:6800/2425730636,v1:172.21.15.52:6801/2425730636] conn(0x7f7844038b50 0x7f784403b010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:21:25.108 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:19.750+0000 7f785affd700 1 --2- 172.21.15.52:0/944767163 >> [v2:172.21.15.52:6800/2425730636,v1:172.21.15.52:6801/2425730636] conn(0x7f7844038b50 0x7f784403b010 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f7850006fd0 tx=0x7f7850006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:21:25.108 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:19.755+0000 7f7858ff9700 1 -- 172.21.15.52:0/944767163 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f784c023e50 con 0x7f785c105b00 2024-03-04T09:21:25.108 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:19.964+0000 7f7860da0700 1 -- 172.21.15.52:0/944767163 --> [v2:172.21.15.52:6800/2425730636,v1:172.21.15.52:6801/2425730636] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi052", "addr": "172.21.15.52", "target": ["mon-mgr", ""]}) v1 -- 0x7f785c1026e0 con 0x7f7844038b50 2024-03-04T09:21:25.108 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:24.491+0000 7f7858ff9700 1 -- 172.21.15.52:0/944767163 <== mgr.14120 v2:172.21.15.52:6800/2425730636 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+48 (secure 0 0 0) 0x7f785c1026e0 con 0x7f7844038b50 2024-03-04T09:21:25.108 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:24.494+0000 7f7860da0700 1 -- 172.21.15.52:0/944767163 >> [v2:172.21.15.52:6800/2425730636,v1:172.21.15.52:6801/2425730636] conn(0x7f7844038b50 msgr2=0x7f784403b010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:21:25.108 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:24.494+0000 7f7860da0700 1 --2- 172.21.15.52:0/944767163 >> [v2:172.21.15.52:6800/2425730636,v1:172.21.15.52:6801/2425730636] conn(0x7f7844038b50 0x7f784403b010 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f7850006fd0 tx=0x7f7850006e40 comp rx=0 tx=0).stop 2024-03-04T09:21:25.108 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:24.494+0000 7f7860da0700 1 -- 172.21.15.52:0/944767163 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f785c105b00 msgr2=0x7f785c108310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:21:25.108 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:24.494+0000 7f7860da0700 1 --2- 172.21.15.52:0/944767163 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f785c105b00 0x7f785c108310 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f784c009fb0 tx=0x7f784c011bf0 comp rx=0 tx=0).stop 2024-03-04T09:21:25.108 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:24.494+0000 7f7860da0700 1 -- 172.21.15.52:0/944767163 shutdown_connections 2024-03-04T09:21:25.108 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:24.494+0000 7f7860da0700 1 --2- 172.21.15.52:0/944767163 >> [v2:172.21.15.52:6800/2425730636,v1:172.21.15.52:6801/2425730636] conn(0x7f7844038b50 0x7f784403b010 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:21:25.108 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:24.494+0000 7f7860da0700 1 --2- 172.21.15.52:0/944767163 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f785c105b00 0x7f785c108310 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:21:25.108 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:24.494+0000 7f7860da0700 1 -- 172.21.15.52:0/944767163 >> 172.21.15.52:0/944767163 conn(0x7f785c101470 msgr2=0x7f785c101fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:21:25.108 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:24.494+0000 7f7860da0700 1 -- 172.21.15.52:0/944767163 shutdown_connections 2024-03-04T09:21:25.108 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:24.494+0000 7f7860da0700 1 -- 172.21.15.52:0/944767163 wait complete. 2024-03-04T09:21:25.109 INFO:teuthology.orchestra.run.smithi052.stdout:Deploying unmanaged mon service... 2024-03-04T09:21:25.739 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:21:25 smithi052 ceph-mon[32473]: from='mgr.14120 172.21.15.52:0/3140567671' entity='mgr.x' 2024-03-04T09:21:25.739 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:21:25 smithi052 ceph-mon[32473]: Added host smithi052 2024-03-04T09:21:25.739 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:21:25 smithi052 ceph-mon[32473]: from='mgr.14120 172.21.15.52:0/3140567671' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T09:21:26.438 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-03-04T09:21:26.438 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:25.514+0000 7f5fb53b7700 1 Processor -- start 2024-03-04T09:21:26.438 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:25.514+0000 7f5fb53b7700 1 -- start start 2024-03-04T09:21:26.438 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:25.514+0000 7f5fb53b7700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5fb01084b0 0x7f5fb010a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:21:26.438 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:25.514+0000 7f5fb53b7700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fb007ae60 con 0x7f5fb01084b0 2024-03-04T09:21:26.438 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:25.514+0000 7f5faeffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5fb01084b0 0x7f5fb010a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:21:26.438 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:25.514+0000 7f5faeffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5fb01084b0 0x7f5fb010a8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:58412/0 (socket says 172.21.15.52:58412) 2024-03-04T09:21:26.439 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:25.514+0000 7f5faeffd700 1 -- 172.21.15.52:0/805372493 learned_addr learned my addr 172.21.15.52:0/805372493 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-04T09:21:26.439 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:25.514+0000 7f5faeffd700 1 -- 172.21.15.52:0/805372493 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fb010ade0 con 0x7f5fb01084b0 2024-03-04T09:21:26.439 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:25.515+0000 7f5faeffd700 1 --2- 172.21.15.52:0/805372493 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5fb01084b0 0x7f5fb010a8a0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f5fa00098a0 tx=0x7f5fa0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3293607208f4ee21 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:21:26.439 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:25.515+0000 7f5fadffb700 1 -- 172.21.15.52:0/805372493 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5fa0004160 con 0x7f5fb01084b0 2024-03-04T09:21:26.439 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:25.515+0000 7f5fadffb700 1 -- 172.21.15.52:0/805372493 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5fa0004300 con 0x7f5fb01084b0 2024-03-04T09:21:26.439 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:25.515+0000 7f5fadffb700 1 -- 172.21.15.52:0/805372493 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5fa0011b50 con 0x7f5fb01084b0 2024-03-04T09:21:26.439 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:25.515+0000 7f5fb53b7700 1 -- 172.21.15.52:0/805372493 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5fb01084b0 msgr2=0x7f5fb010a8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:21:26.439 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:25.515+0000 7f5fb53b7700 1 --2- 172.21.15.52:0/805372493 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5fb01084b0 0x7f5fb010a8a0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f5fa00098a0 tx=0x7f5fa0009bb0 comp rx=0 tx=0).stop 2024-03-04T09:21:26.439 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:25.515+0000 7f5fb53b7700 1 -- 172.21.15.52:0/805372493 shutdown_connections 2024-03-04T09:21:26.439 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:25.515+0000 7f5fb53b7700 1 --2- 172.21.15.52:0/805372493 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5fb01084b0 0x7f5fb010a8a0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:21:26.439 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:25.515+0000 7f5fb53b7700 1 -- 172.21.15.52:0/805372493 >> 172.21.15.52:0/805372493 conn(0x7f5fb0101f80 msgr2=0x7f5fb01043a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:21:26.439 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:25.515+0000 7f5fb53b7700 1 -- 172.21.15.52:0/805372493 shutdown_connections 2024-03-04T09:21:26.440 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:25.515+0000 7f5fb53b7700 1 -- 172.21.15.52:0/805372493 wait complete. 2024-03-04T09:21:26.440 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:25.516+0000 7f5fb53b7700 1 Processor -- start 2024-03-04T09:21:26.440 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:25.516+0000 7f5fb53b7700 1 -- start start 2024-03-04T09:21:26.440 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:25.516+0000 7f5fb53b7700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5fb01084b0 0x7f5fb019b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:21:26.440 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:25.516+0000 7f5fb53b7700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fb010bdb0 con 0x7f5fb01084b0 2024-03-04T09:21:26.440 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:25.516+0000 7f5faeffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5fb01084b0 0x7f5fb019b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:21:26.440 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:25.517+0000 7f5faeffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5fb01084b0 0x7f5fb019b9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:58424/0 (socket says 172.21.15.52:58424) 2024-03-04T09:21:26.440 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:25.517+0000 7f5faeffd700 1 -- 172.21.15.52:0/2705473444 learned_addr learned my addr 172.21.15.52:0/2705473444 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-04T09:21:26.440 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:25.517+0000 7f5faeffd700 1 -- 172.21.15.52:0/2705473444 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fb019bef0 con 0x7f5fb01084b0 2024-03-04T09:21:26.440 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:25.517+0000 7f5faeffd700 1 --2- 172.21.15.52:0/2705473444 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5fb01084b0 0x7f5fb019b9b0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f5fa0011b60 tx=0x7f5fa0011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:21:26.440 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:25.517+0000 7f5f97fff700 1 -- 172.21.15.52:0/2705473444 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5fa0003970 con 0x7f5fb01084b0 2024-03-04T09:21:26.440 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:25.517+0000 7f5f97fff700 1 -- 172.21.15.52:0/2705473444 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5fa0003b10 con 0x7f5fb01084b0 2024-03-04T09:21:26.441 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:25.517+0000 7f5fb53b7700 1 -- 172.21.15.52:0/2705473444 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5fb019c180 con 0x7f5fb01084b0 2024-03-04T09:21:26.441 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:25.517+0000 7f5f97fff700 1 -- 172.21.15.52:0/2705473444 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5fa0024610 con 0x7f5fb01084b0 2024-03-04T09:21:26.441 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:25.517+0000 7f5fb53b7700 1 -- 172.21.15.52:0/2705473444 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5fb019ee70 con 0x7f5fb01084b0 2024-03-04T09:21:26.441 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:25.518+0000 7f5f97fff700 1 -- 172.21.15.52:0/2705473444 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 8) v1 ==== 45928+0+0 (secure 0 0 0) 0x7f5fa0024910 con 0x7f5fb01084b0 2024-03-04T09:21:26.441 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:25.518+0000 7f5fb53b7700 1 -- 172.21.15.52:0/2705473444 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5fb00636f0 con 0x7f5fb01084b0 2024-03-04T09:21:26.441 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:25.518+0000 7f5f97fff700 1 --2- 172.21.15.52:0/2705473444 >> [v2:172.21.15.52:6800/2425730636,v1:172.21.15.52:6801/2425730636] conn(0x7f5f98038ba0 0x7f5f9803b060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:21:26.441 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:25.518+0000 7f5f97fff700 1 -- 172.21.15.52:0/2705473444 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f5fa0011e00 con 0x7f5fb01084b0 2024-03-04T09:21:26.441 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:25.518+0000 7f5fae7fc700 1 --2- 172.21.15.52:0/2705473444 >> [v2:172.21.15.52:6800/2425730636,v1:172.21.15.52:6801/2425730636] conn(0x7f5f98038ba0 0x7f5f9803b060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:21:26.441 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:25.519+0000 7f5fae7fc700 1 --2- 172.21.15.52:0/2705473444 >> [v2:172.21.15.52:6800/2425730636,v1:172.21.15.52:6801/2425730636] conn(0x7f5f98038ba0 0x7f5f9803b060 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f5fa4006fd0 tx=0x7f5fa4006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:21:26.441 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:25.523+0000 7f5f97fff700 1 -- 172.21.15.52:0/2705473444 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f5fa000ba00 con 0x7f5fb01084b0 2024-03-04T09:21:26.441 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:25.742+0000 7f5fb53b7700 1 -- 172.21.15.52:0/2705473444 --> [v2:172.21.15.52:6800/2425730636,v1:172.21.15.52:6801/2425730636] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}) v1 -- 0x7f5fb01031f0 con 0x7f5f98038ba0 2024-03-04T09:21:26.441 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:25.746+0000 7f5f97fff700 1 -- 172.21.15.52:0/2705473444 <== mgr.14120 v2:172.21.15.52:6800/2425730636 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f5fb01031f0 con 0x7f5f98038ba0 2024-03-04T09:21:26.441 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:25.748+0000 7f5fb53b7700 1 -- 172.21.15.52:0/2705473444 >> [v2:172.21.15.52:6800/2425730636,v1:172.21.15.52:6801/2425730636] conn(0x7f5f98038ba0 msgr2=0x7f5f9803b060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:21:26.442 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:25.748+0000 7f5fb53b7700 1 --2- 172.21.15.52:0/2705473444 >> [v2:172.21.15.52:6800/2425730636,v1:172.21.15.52:6801/2425730636] conn(0x7f5f98038ba0 0x7f5f9803b060 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f5fa4006fd0 tx=0x7f5fa4006e40 comp rx=0 tx=0).stop 2024-03-04T09:21:26.442 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:25.748+0000 7f5fb53b7700 1 -- 172.21.15.52:0/2705473444 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5fb01084b0 msgr2=0x7f5fb019b9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:21:26.442 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:25.748+0000 7f5fb53b7700 1 --2- 172.21.15.52:0/2705473444 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5fb01084b0 0x7f5fb019b9b0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f5fa0011b60 tx=0x7f5fa0011c40 comp rx=0 tx=0).stop 2024-03-04T09:21:26.442 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:25.748+0000 7f5fb53b7700 1 -- 172.21.15.52:0/2705473444 shutdown_connections 2024-03-04T09:21:26.442 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:25.748+0000 7f5fb53b7700 1 --2- 172.21.15.52:0/2705473444 >> [v2:172.21.15.52:6800/2425730636,v1:172.21.15.52:6801/2425730636] conn(0x7f5f98038ba0 0x7f5f9803b060 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:21:26.442 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:25.748+0000 7f5fb53b7700 1 --2- 172.21.15.52:0/2705473444 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5fb01084b0 0x7f5fb019b9b0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:21:26.442 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:25.748+0000 7f5fb53b7700 1 -- 172.21.15.52:0/2705473444 >> 172.21.15.52:0/2705473444 conn(0x7f5fb0101f80 msgr2=0x7f5fb0102ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:21:26.442 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:25.749+0000 7f5fb53b7700 1 -- 172.21.15.52:0/2705473444 shutdown_connections 2024-03-04T09:21:26.442 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:25.749+0000 7f5fb53b7700 1 -- 172.21.15.52:0/2705473444 wait complete. 2024-03-04T09:21:26.442 INFO:teuthology.orchestra.run.smithi052.stdout:Deploying unmanaged mgr service... 2024-03-04T09:21:26.992 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:21:26 smithi052 ceph-mon[32473]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T09:21:26.992 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:21:26 smithi052 ceph-mon[32473]: Saving service mon spec with placement count:5 2024-03-04T09:21:26.992 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:21:26 smithi052 ceph-mon[32473]: from='mgr.14120 172.21.15.52:0/3140567671' entity='mgr.x' 2024-03-04T09:21:28.335 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:21:28 smithi052 ceph-mon[32473]: from='client.14144 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T09:21:28.335 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:21:28 smithi052 ceph-mon[32473]: Saving service mgr spec with placement count:2 2024-03-04T09:21:28.335 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:21:28 smithi052 ceph-mon[32473]: from='mgr.14120 172.21.15.52:0/3140567671' entity='mgr.x' 2024-03-04T09:21:28.335 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:21:28 smithi052 ceph-mon[32473]: from='mgr.14120 172.21.15.52:0/3140567671' entity='mgr.x' 2024-03-04T09:21:28.417 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-03-04T09:21:28.417 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:26.843+0000 7f4eab4c2700 1 Processor -- start 2024-03-04T09:21:28.417 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:26.844+0000 7f4eab4c2700 1 -- start start 2024-03-04T09:21:28.417 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:26.844+0000 7f4eab4c2700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4ea4106890 0x7f4ea4106cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:21:28.417 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:26.844+0000 7f4eab4c2700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ea4107280 con 0x7f4ea4106890 2024-03-04T09:21:28.417 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:26.844+0000 7f4ea925e700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4ea4106890 0x7f4ea4106cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:21:28.417 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:26.844+0000 7f4ea925e700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4ea4106890 0x7f4ea4106cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:58430/0 (socket says 172.21.15.52:58430) 2024-03-04T09:21:28.417 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:26.844+0000 7f4ea925e700 1 -- 172.21.15.52:0/2669683041 learned_addr learned my addr 172.21.15.52:0/2669683041 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-04T09:21:28.418 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:26.844+0000 7f4ea925e700 1 -- 172.21.15.52:0/2669683041 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ea4107a90 con 0x7f4ea4106890 2024-03-04T09:21:28.418 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:26.844+0000 7f4ea925e700 1 --2- 172.21.15.52:0/2669683041 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4ea4106890 0x7f4ea4106cb0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f4ea00098a0 tx=0x7f4ea0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71738f33edb0bda6 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:21:28.418 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:26.844+0000 7f4e9bfff700 1 -- 172.21.15.52:0/2669683041 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4ea0004160 con 0x7f4ea4106890 2024-03-04T09:21:28.418 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:26.844+0000 7f4e9bfff700 1 -- 172.21.15.52:0/2669683041 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4ea0004300 con 0x7f4ea4106890 2024-03-04T09:21:28.418 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:26.844+0000 7f4e9bfff700 1 -- 172.21.15.52:0/2669683041 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4ea0011b50 con 0x7f4ea4106890 2024-03-04T09:21:28.418 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:26.845+0000 7f4eab4c2700 1 -- 172.21.15.52:0/2669683041 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4ea4106890 msgr2=0x7f4ea4106cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:21:28.418 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:26.845+0000 7f4eab4c2700 1 --2- 172.21.15.52:0/2669683041 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4ea4106890 0x7f4ea4106cb0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f4ea00098a0 tx=0x7f4ea0009bb0 comp rx=0 tx=0).stop 2024-03-04T09:21:28.418 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:26.845+0000 7f4eab4c2700 1 -- 172.21.15.52:0/2669683041 shutdown_connections 2024-03-04T09:21:28.418 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:26.845+0000 7f4eab4c2700 1 --2- 172.21.15.52:0/2669683041 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4ea4106890 0x7f4ea4106cb0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:21:28.418 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:26.845+0000 7f4eab4c2700 1 -- 172.21.15.52:0/2669683041 >> 172.21.15.52:0/2669683041 conn(0x7f4ea4101ed0 msgr2=0x7f4ea41042f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:21:28.418 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:26.845+0000 7f4eab4c2700 1 -- 172.21.15.52:0/2669683041 shutdown_connections 2024-03-04T09:21:28.418 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:26.845+0000 7f4eab4c2700 1 -- 172.21.15.52:0/2669683041 wait complete. 2024-03-04T09:21:28.418 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:26.846+0000 7f4eab4c2700 1 Processor -- start 2024-03-04T09:21:28.419 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:26.846+0000 7f4eab4c2700 1 -- start start 2024-03-04T09:21:28.419 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:26.846+0000 7f4eab4c2700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4ea4106890 0x7f4ea419a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:21:28.419 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:26.846+0000 7f4eab4c2700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ea410ae40 con 0x7f4ea4106890 2024-03-04T09:21:28.419 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:26.846+0000 7f4ea925e700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4ea4106890 0x7f4ea419a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:21:28.419 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:26.846+0000 7f4ea925e700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4ea4106890 0x7f4ea419a8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:58446/0 (socket says 172.21.15.52:58446) 2024-03-04T09:21:28.419 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:26.846+0000 7f4ea925e700 1 -- 172.21.15.52:0/2189018719 learned_addr learned my addr 172.21.15.52:0/2189018719 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-04T09:21:28.419 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:26.846+0000 7f4ea925e700 1 -- 172.21.15.52:0/2189018719 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ea419ae10 con 0x7f4ea4106890 2024-03-04T09:21:28.419 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:26.847+0000 7f4ea925e700 1 --2- 172.21.15.52:0/2189018719 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4ea4106890 0x7f4ea419a8d0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f4ea0011850 tx=0x7f4ea0011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:21:28.419 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:26.847+0000 7f4e9a7fc700 1 -- 172.21.15.52:0/2189018719 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4ea0003af0 con 0x7f4ea4106890 2024-03-04T09:21:28.419 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:26.847+0000 7f4e9a7fc700 1 -- 172.21.15.52:0/2189018719 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4ea0020070 con 0x7f4ea4106890 2024-03-04T09:21:28.419 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:26.847+0000 7f4eab4c2700 1 -- 172.21.15.52:0/2189018719 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ea419b0a0 con 0x7f4ea4106890 2024-03-04T09:21:28.419 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:26.847+0000 7f4e9a7fc700 1 -- 172.21.15.52:0/2189018719 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4ea00195a0 con 0x7f4ea4106890 2024-03-04T09:21:28.419 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:26.847+0000 7f4eab4c2700 1 -- 172.21.15.52:0/2189018719 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ea419dd90 con 0x7f4ea4106890 2024-03-04T09:21:28.419 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:26.848+0000 7f4e9a7fc700 1 -- 172.21.15.52:0/2189018719 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 8) v1 ==== 45928+0+0 (secure 0 0 0) 0x7f4ea0019850 con 0x7f4ea4106890 2024-03-04T09:21:28.420 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:26.848+0000 7f4eab4c2700 1 -- 172.21.15.52:0/2189018719 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ea40636f0 con 0x7f4ea4106890 2024-03-04T09:21:28.420 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:26.848+0000 7f4e9a7fc700 1 --2- 172.21.15.52:0/2189018719 >> [v2:172.21.15.52:6800/2425730636,v1:172.21.15.52:6801/2425730636] conn(0x7f4e90038bf0 0x7f4e9003b0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:21:28.420 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:26.848+0000 7f4e9a7fc700 1 -- 172.21.15.52:0/2189018719 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f4ea0011e00 con 0x7f4ea4106890 2024-03-04T09:21:28.420 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:26.848+0000 7f4ea8a5d700 1 --2- 172.21.15.52:0/2189018719 >> [v2:172.21.15.52:6800/2425730636,v1:172.21.15.52:6801/2425730636] conn(0x7f4e90038bf0 0x7f4e9003b0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:21:28.420 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:26.848+0000 7f4ea8a5d700 1 --2- 172.21.15.52:0/2189018719 >> [v2:172.21.15.52:6800/2425730636,v1:172.21.15.52:6801/2425730636] conn(0x7f4e90038bf0 0x7f4e9003b0b0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f4e94006fd0 tx=0x7f4e94006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:21:28.420 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:26.853+0000 7f4e9a7fc700 1 -- 172.21.15.52:0/2189018719 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f4ea0003c90 con 0x7f4ea4106890 2024-03-04T09:21:28.420 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:27.074+0000 7f4eab4c2700 1 -- 172.21.15.52:0/2189018719 --> [v2:172.21.15.52:6800/2425730636,v1:172.21.15.52:6801/2425730636] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}) v1 -- 0x7f4ea4103190 con 0x7f4e90038bf0 2024-03-04T09:21:28.420 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:27.078+0000 7f4e9a7fc700 1 -- 172.21.15.52:0/2189018719 <== mgr.14120 v2:172.21.15.52:6800/2425730636 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f4ea4103190 con 0x7f4e90038bf0 2024-03-04T09:21:28.420 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:27.080+0000 7f4eab4c2700 1 -- 172.21.15.52:0/2189018719 >> [v2:172.21.15.52:6800/2425730636,v1:172.21.15.52:6801/2425730636] conn(0x7f4e90038bf0 msgr2=0x7f4e9003b0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:21:28.420 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:27.080+0000 7f4eab4c2700 1 --2- 172.21.15.52:0/2189018719 >> [v2:172.21.15.52:6800/2425730636,v1:172.21.15.52:6801/2425730636] conn(0x7f4e90038bf0 0x7f4e9003b0b0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f4e94006fd0 tx=0x7f4e94006e40 comp rx=0 tx=0).stop 2024-03-04T09:21:28.420 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:27.080+0000 7f4eab4c2700 1 -- 172.21.15.52:0/2189018719 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4ea4106890 msgr2=0x7f4ea419a8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:21:28.420 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:27.080+0000 7f4eab4c2700 1 --2- 172.21.15.52:0/2189018719 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4ea4106890 0x7f4ea419a8d0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f4ea0011850 tx=0x7f4ea0011c10 comp rx=0 tx=0).stop 2024-03-04T09:21:28.420 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:27.080+0000 7f4eab4c2700 1 -- 172.21.15.52:0/2189018719 shutdown_connections 2024-03-04T09:21:28.421 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:27.080+0000 7f4eab4c2700 1 --2- 172.21.15.52:0/2189018719 >> [v2:172.21.15.52:6800/2425730636,v1:172.21.15.52:6801/2425730636] conn(0x7f4e90038bf0 0x7f4e9003b0b0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:21:28.421 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:27.080+0000 7f4eab4c2700 1 --2- 172.21.15.52:0/2189018719 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4ea4106890 0x7f4ea419a8d0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:21:28.421 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:27.080+0000 7f4eab4c2700 1 -- 172.21.15.52:0/2189018719 >> 172.21.15.52:0/2189018719 conn(0x7f4ea4101ed0 msgr2=0x7f4ea4102a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:21:28.421 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:27.080+0000 7f4eab4c2700 1 -- 172.21.15.52:0/2189018719 shutdown_connections 2024-03-04T09:21:28.421 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:27.080+0000 7f4eab4c2700 1 -- 172.21.15.52:0/2189018719 wait complete. 2024-03-04T09:21:29.582 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:28.828+0000 7ff10ec6c700 1 Processor -- start 2024-03-04T09:21:29.582 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:28.828+0000 7ff10ec6c700 1 -- start start 2024-03-04T09:21:29.582 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:28.828+0000 7ff10ec6c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff108106290 0x7ff1081066b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:21:29.582 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:28.828+0000 7ff10ec6c700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff10807ae60 con 0x7ff108106290 2024-03-04T09:21:29.582 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:28.828+0000 7ff10ca08700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff108106290 0x7ff1081066b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:21:29.582 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:28.828+0000 7ff10ca08700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff108106290 0x7ff1081066b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:58454/0 (socket says 172.21.15.52:58454) 2024-03-04T09:21:29.582 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:28.828+0000 7ff10ca08700 1 -- 172.21.15.52:0/1691572696 learned_addr learned my addr 172.21.15.52:0/1691572696 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-04T09:21:29.582 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:28.828+0000 7ff10ca08700 1 -- 172.21.15.52:0/1691572696 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff108106bf0 con 0x7ff108106290 2024-03-04T09:21:29.583 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:28.828+0000 7ff10ca08700 1 --2- 172.21.15.52:0/1691572696 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff108106290 0x7ff1081066b0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7ff0fc0098a0 tx=0x7ff0fc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4fd5ca8afa7305ab server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:21:29.583 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:28.829+0000 7ff1077fe700 1 -- 172.21.15.52:0/1691572696 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff0fc004160 con 0x7ff108106290 2024-03-04T09:21:29.583 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:28.829+0000 7ff1077fe700 1 -- 172.21.15.52:0/1691572696 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff0fc004300 con 0x7ff108106290 2024-03-04T09:21:29.583 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:28.829+0000 7ff1077fe700 1 -- 172.21.15.52:0/1691572696 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff0fc011b20 con 0x7ff108106290 2024-03-04T09:21:29.583 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:28.829+0000 7ff10ec6c700 1 -- 172.21.15.52:0/1691572696 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff108106290 msgr2=0x7ff1081066b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:21:29.583 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:28.829+0000 7ff10ec6c700 1 --2- 172.21.15.52:0/1691572696 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff108106290 0x7ff1081066b0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7ff0fc0098a0 tx=0x7ff0fc009bb0 comp rx=0 tx=0).stop 2024-03-04T09:21:29.583 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:28.829+0000 7ff10ec6c700 1 -- 172.21.15.52:0/1691572696 shutdown_connections 2024-03-04T09:21:29.583 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:28.829+0000 7ff10ec6c700 1 --2- 172.21.15.52:0/1691572696 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff108106290 0x7ff1081066b0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:21:29.583 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:28.829+0000 7ff10ec6c700 1 -- 172.21.15.52:0/1691572696 >> 172.21.15.52:0/1691572696 conn(0x7ff108101f70 msgr2=0x7ff108104390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:21:29.583 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:28.829+0000 7ff10ec6c700 1 -- 172.21.15.52:0/1691572696 shutdown_connections 2024-03-04T09:21:29.583 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:28.829+0000 7ff10ec6c700 1 -- 172.21.15.52:0/1691572696 wait complete. 2024-03-04T09:21:29.583 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:28.830+0000 7ff10ec6c700 1 Processor -- start 2024-03-04T09:21:29.583 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:28.830+0000 7ff10ec6c700 1 -- start start 2024-03-04T09:21:29.584 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:28.830+0000 7ff10ec6c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff10819dd10 0x7ff10819e130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:21:29.584 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:28.830+0000 7ff10ec6c700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff108107cc0 con 0x7ff10819dd10 2024-03-04T09:21:29.584 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:28.830+0000 7ff10ca08700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff10819dd10 0x7ff10819e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:21:29.584 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:28.830+0000 7ff10ca08700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff10819dd10 0x7ff10819e130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:58470/0 (socket says 172.21.15.52:58470) 2024-03-04T09:21:29.584 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:28.830+0000 7ff10ca08700 1 -- 172.21.15.52:0/952986807 learned_addr learned my addr 172.21.15.52:0/952986807 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-04T09:21:29.584 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:28.830+0000 7ff10ca08700 1 -- 172.21.15.52:0/952986807 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff10819e670 con 0x7ff10819dd10 2024-03-04T09:21:29.584 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:28.831+0000 7ff10ca08700 1 --2- 172.21.15.52:0/952986807 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff10819dd10 0x7ff10819e130 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7ff0fc009fb0 tx=0x7ff0fc011b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:21:29.584 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:28.831+0000 7ff105ffb700 1 -- 172.21.15.52:0/952986807 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff0fc0036a0 con 0x7ff10819dd10 2024-03-04T09:21:29.584 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:28.831+0000 7ff105ffb700 1 -- 172.21.15.52:0/952986807 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff0fc01a460 con 0x7ff10819dd10 2024-03-04T09:21:29.584 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:28.831+0000 7ff10ec6c700 1 -- 172.21.15.52:0/952986807 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff10819e900 con 0x7ff10819dd10 2024-03-04T09:21:29.584 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:28.831+0000 7ff105ffb700 1 -- 172.21.15.52:0/952986807 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff0fc01b440 con 0x7ff10819dd10 2024-03-04T09:21:29.584 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:28.831+0000 7ff10ec6c700 1 -- 172.21.15.52:0/952986807 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1081a13e0 con 0x7ff10819dd10 2024-03-04T09:21:29.584 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:28.832+0000 7ff105ffb700 1 -- 172.21.15.52:0/952986807 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 8) v1 ==== 45928+0+0 (secure 0 0 0) 0x7ff0fc01b6f0 con 0x7ff10819dd10 2024-03-04T09:21:29.585 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:28.832+0000 7ff10ec6c700 1 -- 172.21.15.52:0/952986807 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff108069170 con 0x7ff10819dd10 2024-03-04T09:21:29.585 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:28.832+0000 7ff105ffb700 1 --2- 172.21.15.52:0/952986807 >> [v2:172.21.15.52:6800/2425730636,v1:172.21.15.52:6801/2425730636] conn(0x7ff0f8038ba0 0x7ff0f803b060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:21:29.585 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:28.832+0000 7ff105ffb700 1 -- 172.21.15.52:0/952986807 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7ff0fc04d230 con 0x7ff10819dd10 2024-03-04T09:21:29.585 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:28.832+0000 7ff107fff700 1 --2- 172.21.15.52:0/952986807 >> [v2:172.21.15.52:6800/2425730636,v1:172.21.15.52:6801/2425730636] conn(0x7ff0f8038ba0 0x7ff0f803b060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:21:29.585 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:28.832+0000 7ff107fff700 1 --2- 172.21.15.52:0/952986807 >> [v2:172.21.15.52:6800/2425730636,v1:172.21.15.52:6801/2425730636] conn(0x7ff0f8038ba0 0x7ff0f803b060 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7ff0f4006fd0 tx=0x7ff0f4006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:21:29.585 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:28.837+0000 7ff105ffb700 1 -- 172.21.15.52:0/952986807 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7ff0fc01a680 con 0x7ff10819dd10 2024-03-04T09:21:29.585 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:29.042+0000 7ff10ec6c700 1 -- 172.21.15.52:0/952986807 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) v1 -- 0x7ff10802de00 con 0x7ff10819dd10 2024-03-04T09:21:29.585 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:29.045+0000 7ff105ffb700 1 -- 172.21.15.52:0/952986807 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v7)=0 v7) v1 ==== 142+0+0 (secure 0 0 0) 0x7ff0fc02d7c0 con 0x7ff10819dd10 2024-03-04T09:21:29.585 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:29.048+0000 7ff10ec6c700 1 -- 172.21.15.52:0/952986807 >> [v2:172.21.15.52:6800/2425730636,v1:172.21.15.52:6801/2425730636] conn(0x7ff0f8038ba0 msgr2=0x7ff0f803b060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:21:29.585 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:29.048+0000 7ff10ec6c700 1 --2- 172.21.15.52:0/952986807 >> [v2:172.21.15.52:6800/2425730636,v1:172.21.15.52:6801/2425730636] conn(0x7ff0f8038ba0 0x7ff0f803b060 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7ff0f4006fd0 tx=0x7ff0f4006e40 comp rx=0 tx=0).stop 2024-03-04T09:21:29.585 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:29.048+0000 7ff10ec6c700 1 -- 172.21.15.52:0/952986807 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff10819dd10 msgr2=0x7ff10819e130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:21:29.585 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:29.048+0000 7ff10ec6c700 1 --2- 172.21.15.52:0/952986807 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff10819dd10 0x7ff10819e130 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7ff0fc009fb0 tx=0x7ff0fc011b80 comp rx=0 tx=0).stop 2024-03-04T09:21:29.585 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:29.048+0000 7ff10ec6c700 1 -- 172.21.15.52:0/952986807 shutdown_connections 2024-03-04T09:21:29.585 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:29.048+0000 7ff10ec6c700 1 --2- 172.21.15.52:0/952986807 >> [v2:172.21.15.52:6800/2425730636,v1:172.21.15.52:6801/2425730636] conn(0x7ff0f8038ba0 0x7ff0f803b060 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:21:29.586 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:29.048+0000 7ff10ec6c700 1 --2- 172.21.15.52:0/952986807 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff10819dd10 0x7ff10819e130 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:21:29.586 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:29.048+0000 7ff10ec6c700 1 -- 172.21.15.52:0/952986807 >> 172.21.15.52:0/952986807 conn(0x7ff108101f70 msgr2=0x7ff108194840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:21:29.586 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:29.048+0000 7ff10ec6c700 1 -- 172.21.15.52:0/952986807 shutdown_connections 2024-03-04T09:21:29.586 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:29.048+0000 7ff10ec6c700 1 -- 172.21.15.52:0/952986807 wait complete. 2024-03-04T09:21:30.313 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:21:30 smithi052 ceph-mon[32473]: from='mgr.14120 172.21.15.52:0/3140567671' entity='mgr.x' 2024-03-04T09:21:30.313 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:21:30 smithi052 ceph-mon[32473]: from='client.? 172.21.15.52:0/952986807' entity='client.admin' 2024-03-04T09:21:31.334 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:21:31 smithi052 ceph-mon[32473]: from='client.? 172.21.15.52:0/3723116656' entity='client.admin' 2024-03-04T09:21:31.334 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:21:31 smithi052 ceph-mon[32473]: from='mgr.14120 172.21.15.52:0/3140567671' entity='mgr.x' 2024-03-04T09:21:31.364 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:29.985+0000 7fbe0ebe7700 1 Processor -- start 2024-03-04T09:21:31.364 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:29.985+0000 7fbe0ebe7700 1 -- start start 2024-03-04T09:21:31.364 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:29.986+0000 7fbe0ebe7700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbe081079b0 0x7fbe08107dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:21:31.364 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:29.986+0000 7fbe0ebe7700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe081083a0 con 0x7fbe081079b0 2024-03-04T09:21:31.364 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:29.986+0000 7fbe0c983700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbe081079b0 0x7fbe08107dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:21:31.364 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:29.986+0000 7fbe0c983700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbe081079b0 0x7fbe08107dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:58478/0 (socket says 172.21.15.52:58478) 2024-03-04T09:21:31.364 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:29.986+0000 7fbe0c983700 1 -- 172.21.15.52:0/3064930772 learned_addr learned my addr 172.21.15.52:0/3064930772 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-04T09:21:31.364 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:29.986+0000 7fbe0c983700 1 -- 172.21.15.52:0/3064930772 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe08108bb0 con 0x7fbe081079b0 2024-03-04T09:21:31.364 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:29.986+0000 7fbe0c983700 1 --2- 172.21.15.52:0/3064930772 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbe081079b0 0x7fbe08107dd0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fbdfc0098a0 tx=0x7fbdfc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9e93666f4432a40 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:21:31.364 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:29.987+0000 7fbe077fe700 1 -- 172.21.15.52:0/3064930772 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbdfc004160 con 0x7fbe081079b0 2024-03-04T09:21:31.364 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:29.987+0000 7fbe077fe700 1 -- 172.21.15.52:0/3064930772 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fbdfc004300 con 0x7fbe081079b0 2024-03-04T09:21:31.365 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:29.987+0000 7fbe077fe700 1 -- 172.21.15.52:0/3064930772 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbdfc011b20 con 0x7fbe081079b0 2024-03-04T09:21:31.365 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:29.988+0000 7fbe0ebe7700 1 -- 172.21.15.52:0/3064930772 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbe081079b0 msgr2=0x7fbe08107dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:21:31.365 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:29.988+0000 7fbe0ebe7700 1 --2- 172.21.15.52:0/3064930772 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbe081079b0 0x7fbe08107dd0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fbdfc0098a0 tx=0x7fbdfc009bb0 comp rx=0 tx=0).stop 2024-03-04T09:21:31.365 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:29.988+0000 7fbe0ebe7700 1 -- 172.21.15.52:0/3064930772 shutdown_connections 2024-03-04T09:21:31.365 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:29.988+0000 7fbe0ebe7700 1 --2- 172.21.15.52:0/3064930772 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbe081079b0 0x7fbe08107dd0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:21:31.365 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:29.988+0000 7fbe0ebe7700 1 -- 172.21.15.52:0/3064930772 >> 172.21.15.52:0/3064930772 conn(0x7fbe08102fb0 msgr2=0x7fbe081053d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:21:31.365 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:29.988+0000 7fbe0ebe7700 1 -- 172.21.15.52:0/3064930772 shutdown_connections 2024-03-04T09:21:31.365 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:29.988+0000 7fbe0ebe7700 1 -- 172.21.15.52:0/3064930772 wait complete. 2024-03-04T09:21:31.365 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:29.988+0000 7fbe0ebe7700 1 Processor -- start 2024-03-04T09:21:31.365 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:29.988+0000 7fbe0ebe7700 1 -- start start 2024-03-04T09:21:31.365 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:29.989+0000 7fbe0ebe7700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbe081079b0 0x7fbe0807cb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:21:31.365 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:29.989+0000 7fbe0ebe7700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe0810bf60 con 0x7fbe081079b0 2024-03-04T09:21:31.365 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:29.989+0000 7fbe0c983700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbe081079b0 0x7fbe0807cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:21:31.366 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:29.989+0000 7fbe0c983700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbe081079b0 0x7fbe0807cb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:58484/0 (socket says 172.21.15.52:58484) 2024-03-04T09:21:31.366 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:29.989+0000 7fbe0c983700 1 -- 172.21.15.52:0/3723116656 learned_addr learned my addr 172.21.15.52:0/3723116656 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-04T09:21:31.366 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:29.989+0000 7fbe0c983700 1 -- 172.21.15.52:0/3723116656 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe0807d0c0 con 0x7fbe081079b0 2024-03-04T09:21:31.366 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:29.989+0000 7fbe0c983700 1 --2- 172.21.15.52:0/3723116656 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbe081079b0 0x7fbe0807cb80 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fbdfc0117e0 tx=0x7fbdfc011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:21:31.366 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:29.989+0000 7fbe05ffb700 1 -- 172.21.15.52:0/3723116656 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbdfc003bf0 con 0x7fbe081079b0 2024-03-04T09:21:31.366 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:29.989+0000 7fbe05ffb700 1 -- 172.21.15.52:0/3723116656 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fbdfc01a910 con 0x7fbe081079b0 2024-03-04T09:21:31.366 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:29.989+0000 7fbe0ebe7700 1 -- 172.21.15.52:0/3723116656 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe0807b2b0 con 0x7fbe081079b0 2024-03-04T09:21:31.366 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:29.989+0000 7fbe05ffb700 1 -- 172.21.15.52:0/3723116656 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbdfc0249a0 con 0x7fbe081079b0 2024-03-04T09:21:31.366 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:29.990+0000 7fbe0ebe7700 1 -- 172.21.15.52:0/3723116656 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe0807b770 con 0x7fbe081079b0 2024-03-04T09:21:31.366 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:29.991+0000 7fbe05ffb700 1 -- 172.21.15.52:0/3723116656 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 8) v1 ==== 45928+0+0 (secure 0 0 0) 0x7fbdfc022020 con 0x7fbe081079b0 2024-03-04T09:21:31.366 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:29.991+0000 7fbe0ebe7700 1 -- 172.21.15.52:0/3723116656 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe080636f0 con 0x7fbe081079b0 2024-03-04T09:21:31.366 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:29.991+0000 7fbe05ffb700 1 --2- 172.21.15.52:0/3723116656 >> [v2:172.21.15.52:6800/2425730636,v1:172.21.15.52:6801/2425730636] conn(0x7fbdf8038ba0 0x7fbdf803b060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:21:31.366 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:29.991+0000 7fbe05ffb700 1 -- 172.21.15.52:0/3723116656 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fbdfc04cd00 con 0x7fbe081079b0 2024-03-04T09:21:31.366 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:29.991+0000 7fbe07fff700 1 --2- 172.21.15.52:0/3723116656 >> [v2:172.21.15.52:6800/2425730636,v1:172.21.15.52:6801/2425730636] conn(0x7fbdf8038ba0 0x7fbdf803b060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:21:31.367 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:29.991+0000 7fbe07fff700 1 --2- 172.21.15.52:0/3723116656 >> [v2:172.21.15.52:6800/2425730636,v1:172.21.15.52:6801/2425730636] conn(0x7fbdf8038ba0 0x7fbdf803b060 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fbdf4006fd0 tx=0x7fbdf4006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:21:31.367 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:29.997+0000 7fbe05ffb700 1 -- 172.21.15.52:0/3723116656 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fbdfc011e20 con 0x7fbe081079b0 2024-03-04T09:21:31.367 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:30.207+0000 7fbe0ebe7700 1 -- 172.21.15.52:0/3723116656 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) v1 -- 0x7fbe0807bab0 con 0x7fbe081079b0 2024-03-04T09:21:31.367 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:30.209+0000 7fbe05ffb700 1 -- 172.21.15.52:0/3723116656 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) v1 ==== 130+0+0 (secure 0 0 0) 0x7fbe0807bab0 con 0x7fbe081079b0 2024-03-04T09:21:31.367 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:30.212+0000 7fbe0ebe7700 1 -- 172.21.15.52:0/3723116656 >> [v2:172.21.15.52:6800/2425730636,v1:172.21.15.52:6801/2425730636] conn(0x7fbdf8038ba0 msgr2=0x7fbdf803b060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:21:31.367 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:30.212+0000 7fbe0ebe7700 1 --2- 172.21.15.52:0/3723116656 >> [v2:172.21.15.52:6800/2425730636,v1:172.21.15.52:6801/2425730636] conn(0x7fbdf8038ba0 0x7fbdf803b060 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fbdf4006fd0 tx=0x7fbdf4006e40 comp rx=0 tx=0).stop 2024-03-04T09:21:31.367 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:30.212+0000 7fbe0ebe7700 1 -- 172.21.15.52:0/3723116656 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbe081079b0 msgr2=0x7fbe0807cb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:21:31.367 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:30.212+0000 7fbe0ebe7700 1 --2- 172.21.15.52:0/3723116656 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbe081079b0 0x7fbe0807cb80 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fbdfc0117e0 tx=0x7fbdfc011c10 comp rx=0 tx=0).stop 2024-03-04T09:21:31.367 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:30.212+0000 7fbe0ebe7700 1 -- 172.21.15.52:0/3723116656 shutdown_connections 2024-03-04T09:21:31.367 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:30.212+0000 7fbe0ebe7700 1 --2- 172.21.15.52:0/3723116656 >> [v2:172.21.15.52:6800/2425730636,v1:172.21.15.52:6801/2425730636] conn(0x7fbdf8038ba0 0x7fbdf803b060 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:21:31.367 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:30.212+0000 7fbe0ebe7700 1 --2- 172.21.15.52:0/3723116656 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbe081079b0 0x7fbe0807cb80 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:21:31.367 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:30.212+0000 7fbe0ebe7700 1 -- 172.21.15.52:0/3723116656 >> 172.21.15.52:0/3723116656 conn(0x7fbe08102fb0 msgr2=0x7fbe08103aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:21:31.367 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:30.212+0000 7fbe0ebe7700 1 -- 172.21.15.52:0/3723116656 shutdown_connections 2024-03-04T09:21:31.368 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:30.212+0000 7fbe0ebe7700 1 -- 172.21.15.52:0/3723116656 wait complete. 2024-03-04T09:21:31.368 INFO:teuthology.orchestra.run.smithi052.stdout:Enabling the dashboard module... 2024-03-04T09:21:32.334 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:21:32 smithi052 ceph-mon[32473]: from='client.? 172.21.15.52:0/1644296212' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-03-04T09:21:32.335 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Mar 04 09:21:32 smithi052 ceph-4e059f08-da08-11ee-95c6-87774f69a715-mgr-x[32705]: ignoring --setuser ceph since I am not root 2024-03-04T09:21:32.335 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Mar 04 09:21:32 smithi052 ceph-4e059f08-da08-11ee-95c6-87774f69a715-mgr-x[32705]: ignoring --setgroup ceph since I am not root 2024-03-04T09:21:32.812 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:31.773+0000 7ff87d171700 1 Processor -- start 2024-03-04T09:21:32.812 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:31.773+0000 7ff87d171700 1 -- start start 2024-03-04T09:21:32.812 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:31.773+0000 7ff87d171700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff8781068e0 0x7ff878106d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:21:32.812 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:31.773+0000 7ff87d171700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8781072d0 con 0x7ff8781068e0 2024-03-04T09:21:32.813 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:31.774+0000 7ff876d9d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff8781068e0 0x7ff878106d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:21:32.813 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:31.774+0000 7ff876d9d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff8781068e0 0x7ff878106d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:58492/0 (socket says 172.21.15.52:58492) 2024-03-04T09:21:32.813 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:31.774+0000 7ff876d9d700 1 -- 172.21.15.52:0/1236701492 learned_addr learned my addr 172.21.15.52:0/1236701492 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-04T09:21:32.813 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:31.774+0000 7ff876d9d700 1 -- 172.21.15.52:0/1236701492 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff878107ae0 con 0x7ff8781068e0 2024-03-04T09:21:32.813 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:31.774+0000 7ff876d9d700 1 --2- 172.21.15.52:0/1236701492 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff8781068e0 0x7ff878106d00 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7ff86800d180 tx=0x7ff86800d490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6896e2f6f1c3b3ed server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:21:32.813 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:31.774+0000 7ff875d9b700 1 -- 172.21.15.52:0/1236701492 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff868019070 con 0x7ff8781068e0 2024-03-04T09:21:32.813 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:31.774+0000 7ff875d9b700 1 -- 172.21.15.52:0/1236701492 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff86800de20 con 0x7ff8781068e0 2024-03-04T09:21:32.813 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:31.774+0000 7ff875d9b700 1 -- 172.21.15.52:0/1236701492 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff868009af0 con 0x7ff8781068e0 2024-03-04T09:21:32.813 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:31.775+0000 7ff87d171700 1 -- 172.21.15.52:0/1236701492 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff8781068e0 msgr2=0x7ff878106d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:21:32.813 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:31.775+0000 7ff87d171700 1 --2- 172.21.15.52:0/1236701492 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff8781068e0 0x7ff878106d00 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7ff86800d180 tx=0x7ff86800d490 comp rx=0 tx=0).stop 2024-03-04T09:21:32.813 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:31.775+0000 7ff87d171700 1 -- 172.21.15.52:0/1236701492 shutdown_connections 2024-03-04T09:21:32.813 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:31.775+0000 7ff87d171700 1 --2- 172.21.15.52:0/1236701492 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff8781068e0 0x7ff878106d00 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:21:32.813 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:31.775+0000 7ff87d171700 1 -- 172.21.15.52:0/1236701492 >> 172.21.15.52:0/1236701492 conn(0x7ff878101f80 msgr2=0x7ff8781043a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:21:32.814 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:31.775+0000 7ff87d171700 1 -- 172.21.15.52:0/1236701492 shutdown_connections 2024-03-04T09:21:32.814 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:31.775+0000 7ff87d171700 1 -- 172.21.15.52:0/1236701492 wait complete. 2024-03-04T09:21:32.814 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:31.776+0000 7ff87d171700 1 Processor -- start 2024-03-04T09:21:32.814 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:31.776+0000 7ff87d171700 1 -- start start 2024-03-04T09:21:32.814 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:31.776+0000 7ff87d171700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff8781068e0 0x7ff8781997c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:21:32.814 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:31.776+0000 7ff87d171700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff878079fe0 con 0x7ff8781068e0 2024-03-04T09:21:32.814 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:31.776+0000 7ff876d9d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff8781068e0 0x7ff8781997c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:21:32.814 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:31.776+0000 7ff876d9d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff8781068e0 0x7ff8781997c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:58496/0 (socket says 172.21.15.52:58496) 2024-03-04T09:21:32.814 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:31.776+0000 7ff876d9d700 1 -- 172.21.15.52:0/1644296212 learned_addr learned my addr 172.21.15.52:0/1644296212 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-04T09:21:32.814 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:31.776+0000 7ff876d9d700 1 -- 172.21.15.52:0/1644296212 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff878199d00 con 0x7ff8781068e0 2024-03-04T09:21:32.814 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:31.776+0000 7ff876d9d700 1 --2- 172.21.15.52:0/1644296212 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff8781068e0 0x7ff8781997c0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7ff868003c50 tx=0x7ff868003d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:21:32.814 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:31.777+0000 7ff86ffff700 1 -- 172.21.15.52:0/1644296212 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff868023020 con 0x7ff8781068e0 2024-03-04T09:21:32.814 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:31.777+0000 7ff86ffff700 1 -- 172.21.15.52:0/1644296212 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff86800b200 con 0x7ff8781068e0 2024-03-04T09:21:32.815 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:31.777+0000 7ff87d171700 1 -- 172.21.15.52:0/1644296212 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff878199f90 con 0x7ff8781068e0 2024-03-04T09:21:32.815 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:31.777+0000 7ff86ffff700 1 -- 172.21.15.52:0/1644296212 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff868019040 con 0x7ff8781068e0 2024-03-04T09:21:32.815 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:31.777+0000 7ff87d171700 1 -- 172.21.15.52:0/1644296212 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff87819cc80 con 0x7ff8781068e0 2024-03-04T09:21:32.815 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:31.778+0000 7ff87d171700 1 -- 172.21.15.52:0/1644296212 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff8780636f0 con 0x7ff8781068e0 2024-03-04T09:21:32.815 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:31.778+0000 7ff86ffff700 1 -- 172.21.15.52:0/1644296212 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 8) v1 ==== 45928+0+0 (secure 0 0 0) 0x7ff868026020 con 0x7ff8781068e0 2024-03-04T09:21:32.815 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:31.778+0000 7ff86ffff700 1 --2- 172.21.15.52:0/1644296212 >> [v2:172.21.15.52:6800/2425730636,v1:172.21.15.52:6801/2425730636] conn(0x7ff864038ba0 0x7ff86403b060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:21:32.815 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:31.778+0000 7ff86ffff700 1 -- 172.21.15.52:0/1644296212 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7ff868011050 con 0x7ff8781068e0 2024-03-04T09:21:32.815 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:31.778+0000 7ff87659c700 1 --2- 172.21.15.52:0/1644296212 >> [v2:172.21.15.52:6800/2425730636,v1:172.21.15.52:6801/2425730636] conn(0x7ff864038ba0 0x7ff86403b060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:21:32.815 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:31.778+0000 7ff87659c700 1 --2- 172.21.15.52:0/1644296212 >> [v2:172.21.15.52:6800/2425730636,v1:172.21.15.52:6801/2425730636] conn(0x7ff864038ba0 0x7ff86403b060 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7ff860006fd0 tx=0x7ff860006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:21:32.815 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:31.783+0000 7ff86ffff700 1 -- 172.21.15.52:0/1644296212 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7ff86802d020 con 0x7ff8781068e0 2024-03-04T09:21:32.815 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:32.079+0000 7ff87d171700 1 -- 172.21.15.52:0/1644296212 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) v1 -- 0x7ff87819ce50 con 0x7ff8781068e0 2024-03-04T09:21:32.815 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:32.212+0000 7ff86ffff700 1 -- 172.21.15.52:0/1644296212 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) v1 ==== 88+0+0 (secure 0 0 0) 0x7ff87819ce50 con 0x7ff8781068e0 2024-03-04T09:21:32.816 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:32.213+0000 7ff86ffff700 1 -- 172.21.15.52:0/1644296212 <== mon.0 v2:172.21.15.52:3300/0 8 ==== mgrmap(e 9) v1 ==== 45941+0+0 (secure 0 0 0) 0x7ff868004000 con 0x7ff8781068e0 2024-03-04T09:21:32.816 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:32.215+0000 7ff87d171700 1 -- 172.21.15.52:0/1644296212 >> [v2:172.21.15.52:6800/2425730636,v1:172.21.15.52:6801/2425730636] conn(0x7ff864038ba0 msgr2=0x7ff86403b060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:21:32.816 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:32.215+0000 7ff87d171700 1 --2- 172.21.15.52:0/1644296212 >> [v2:172.21.15.52:6800/2425730636,v1:172.21.15.52:6801/2425730636] conn(0x7ff864038ba0 0x7ff86403b060 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7ff860006fd0 tx=0x7ff860006e40 comp rx=0 tx=0).stop 2024-03-04T09:21:32.816 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:32.215+0000 7ff87d171700 1 -- 172.21.15.52:0/1644296212 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff8781068e0 msgr2=0x7ff8781997c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:21:32.816 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:32.215+0000 7ff87d171700 1 --2- 172.21.15.52:0/1644296212 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff8781068e0 0x7ff8781997c0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7ff868003c50 tx=0x7ff868003d30 comp rx=0 tx=0).stop 2024-03-04T09:21:32.816 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:32.215+0000 7ff87d171700 1 -- 172.21.15.52:0/1644296212 shutdown_connections 2024-03-04T09:21:32.816 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:32.215+0000 7ff87d171700 1 --2- 172.21.15.52:0/1644296212 >> [v2:172.21.15.52:6800/2425730636,v1:172.21.15.52:6801/2425730636] conn(0x7ff864038ba0 0x7ff86403b060 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:21:32.816 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:32.215+0000 7ff87d171700 1 --2- 172.21.15.52:0/1644296212 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff8781068e0 0x7ff8781997c0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:21:32.816 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:32.215+0000 7ff87d171700 1 -- 172.21.15.52:0/1644296212 >> 172.21.15.52:0/1644296212 conn(0x7ff878101f80 msgr2=0x7ff87818fde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:21:32.816 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:32.215+0000 7ff87d171700 1 -- 172.21.15.52:0/1644296212 shutdown_connections 2024-03-04T09:21:32.816 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:32.215+0000 7ff87d171700 1 -- 172.21.15.52:0/1644296212 wait complete. 2024-03-04T09:21:33.062 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Mar 04 09:21:32 smithi052 ceph-4e059f08-da08-11ee-95c6-87774f69a715-mgr-x[32705]: 2024-03-04T09:21:32.857+0000 7f9cb070a200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-04T09:21:33.062 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Mar 04 09:21:32 smithi052 ceph-4e059f08-da08-11ee-95c6-87774f69a715-mgr-x[32705]: 2024-03-04T09:21:32.958+0000 7f9cb070a200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-04T09:21:33.335 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:21:33 smithi052 ceph-mon[32473]: from='client.? 172.21.15.52:0/1644296212' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-03-04T09:21:33.335 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:21:33 smithi052 ceph-mon[32473]: mgrmap e9: x(active, since 19s) 2024-03-04T09:21:33.752 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Mar 04 09:21:33 smithi052 ceph-4e059f08-da08-11ee-95c6-87774f69a715-mgr-x[32705]: 2024-03-04T09:21:33.659+0000 7f9cb070a200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-04T09:21:33.752 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Mar 04 09:21:33 smithi052 ceph-4e059f08-da08-11ee-95c6-87774f69a715-mgr-x[32705]: 2024-03-04T09:21:33.755+0000 7f9cb070a200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-04T09:21:34.084 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Mar 04 09:21:33 smithi052 ceph-4e059f08-da08-11ee-95c6-87774f69a715-mgr-x[32705]: 2024-03-04T09:21:33.903+0000 7f9cb070a200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-04T09:21:34.085 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Mar 04 09:21:34 smithi052 ceph-4e059f08-da08-11ee-95c6-87774f69a715-mgr-x[32705]: 2024-03-04T09:21:34.003+0000 7f9cb070a200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-04T09:21:34.099 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout { 2024-03-04T09:21:34.100 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-03-04T09:21:34.100 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "available": true, 2024-03-04T09:21:34.100 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-03-04T09:21:34.100 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-03-04T09:21:34.100 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout } 2024-03-04T09:21:34.100 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:33.268+0000 7fd55336b700 1 Processor -- start 2024-03-04T09:21:34.100 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:33.269+0000 7fd55336b700 1 -- start start 2024-03-04T09:21:34.100 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:33.269+0000 7fd55336b700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd54c1068d0 0x7fd54c106cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:21:34.100 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:33.269+0000 7fd55336b700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd54c1072c0 con 0x7fd54c1068d0 2024-03-04T09:21:34.100 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:33.269+0000 7fd551107700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd54c1068d0 0x7fd54c106cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:21:34.100 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:33.269+0000 7fd551107700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd54c1068d0 0x7fd54c106cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:58512/0 (socket says 172.21.15.52:58512) 2024-03-04T09:21:34.101 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:33.269+0000 7fd551107700 1 -- 172.21.15.52:0/2202635442 learned_addr learned my addr 172.21.15.52:0/2202635442 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-04T09:21:34.101 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:33.270+0000 7fd551107700 1 -- 172.21.15.52:0/2202635442 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd54c107ad0 con 0x7fd54c1068d0 2024-03-04T09:21:34.101 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:33.270+0000 7fd551107700 1 --2- 172.21.15.52:0/2202635442 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd54c1068d0 0x7fd54c106cf0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fd5480098a0 tx=0x7fd548009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fbadb88aa5aa4d7a server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:21:34.101 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:33.270+0000 7fd543fff700 1 -- 172.21.15.52:0/2202635442 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd548004160 con 0x7fd54c1068d0 2024-03-04T09:21:34.101 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:33.270+0000 7fd543fff700 1 -- 172.21.15.52:0/2202635442 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd548004300 con 0x7fd54c1068d0 2024-03-04T09:21:34.101 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:33.270+0000 7fd543fff700 1 -- 172.21.15.52:0/2202635442 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd548011b20 con 0x7fd54c1068d0 2024-03-04T09:21:34.101 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:33.271+0000 7fd55336b700 1 -- 172.21.15.52:0/2202635442 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd54c1068d0 msgr2=0x7fd54c106cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:21:34.101 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:33.271+0000 7fd55336b700 1 --2- 172.21.15.52:0/2202635442 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd54c1068d0 0x7fd54c106cf0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fd5480098a0 tx=0x7fd548009bb0 comp rx=0 tx=0).stop 2024-03-04T09:21:34.101 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:33.271+0000 7fd55336b700 1 -- 172.21.15.52:0/2202635442 shutdown_connections 2024-03-04T09:21:34.101 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:33.271+0000 7fd55336b700 1 --2- 172.21.15.52:0/2202635442 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd54c1068d0 0x7fd54c106cf0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:21:34.101 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:33.271+0000 7fd55336b700 1 -- 172.21.15.52:0/2202635442 >> 172.21.15.52:0/2202635442 conn(0x7fd54c101f70 msgr2=0x7fd54c104390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:21:34.101 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:33.271+0000 7fd55336b700 1 -- 172.21.15.52:0/2202635442 shutdown_connections 2024-03-04T09:21:34.101 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:33.271+0000 7fd55336b700 1 -- 172.21.15.52:0/2202635442 wait complete. 2024-03-04T09:21:34.102 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:33.272+0000 7fd55336b700 1 Processor -- start 2024-03-04T09:21:34.102 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:33.272+0000 7fd55336b700 1 -- start start 2024-03-04T09:21:34.102 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:33.272+0000 7fd55336b700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd54c1068d0 0x7fd54c10c790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:21:34.102 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:33.272+0000 7fd55336b700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd54c079fe0 con 0x7fd54c1068d0 2024-03-04T09:21:34.102 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:33.272+0000 7fd551107700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd54c1068d0 0x7fd54c10c790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:21:34.102 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:33.272+0000 7fd551107700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd54c1068d0 0x7fd54c10c790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:58514/0 (socket says 172.21.15.52:58514) 2024-03-04T09:21:34.102 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:33.272+0000 7fd551107700 1 -- 172.21.15.52:0/3998980865 learned_addr learned my addr 172.21.15.52:0/3998980865 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-04T09:21:34.102 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:33.272+0000 7fd551107700 1 -- 172.21.15.52:0/3998980865 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd54c10ccd0 con 0x7fd54c1068d0 2024-03-04T09:21:34.102 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:33.272+0000 7fd551107700 1 --2- 172.21.15.52:0/3998980865 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd54c1068d0 0x7fd54c10c790 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fd5480116a0 tx=0x7fd548011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:21:34.102 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:33.273+0000 7fd5427fc700 1 -- 172.21.15.52:0/3998980865 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd548003a40 con 0x7fd54c1068d0 2024-03-04T09:21:34.102 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:33.273+0000 7fd55336b700 1 -- 172.21.15.52:0/3998980865 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd54c10aec0 con 0x7fd54c1068d0 2024-03-04T09:21:34.102 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:33.273+0000 7fd5427fc700 1 -- 172.21.15.52:0/3998980865 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd548003be0 con 0x7fd54c1068d0 2024-03-04T09:21:34.102 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:33.274+0000 7fd5427fc700 1 -- 172.21.15.52:0/3998980865 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd54801a720 con 0x7fd54c1068d0 2024-03-04T09:21:34.102 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:33.274+0000 7fd55336b700 1 -- 172.21.15.52:0/3998980865 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd54c10b380 con 0x7fd54c1068d0 2024-03-04T09:21:34.103 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:33.274+0000 7fd5427fc700 1 -- 172.21.15.52:0/3998980865 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 9) v1 ==== 45941+0+0 (secure 0 0 0) 0x7fd54801aa70 con 0x7fd54c1068d0 2024-03-04T09:21:34.103 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:33.274+0000 7fd5427fc700 1 --2- 172.21.15.52:0/3998980865 >> [v2:172.21.15.52:6800/2425730636,v1:172.21.15.52:6801/2425730636] conn(0x7fd538038c00 0x7fd53803b0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:21:34.103 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:33.274+0000 7fd5427fc700 1 -- 172.21.15.52:0/3998980865 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fd54804e920 con 0x7fd54c1068d0 2024-03-04T09:21:34.103 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:33.274+0000 7fd550906700 1 -- 172.21.15.52:0/3998980865 >> [v2:172.21.15.52:6800/2425730636,v1:172.21.15.52:6801/2425730636] conn(0x7fd538038c00 msgr2=0x7fd53803b0c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.52:6800/2425730636 2024-03-04T09:21:34.103 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:33.274+0000 7fd550906700 1 --2- 172.21.15.52:0/3998980865 >> [v2:172.21.15.52:6800/2425730636,v1:172.21.15.52:6801/2425730636] conn(0x7fd538038c00 0x7fd53803b0c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T09:21:34.103 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:33.275+0000 7fd55336b700 1 -- 172.21.15.52:0/3998980865 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd54c069170 con 0x7fd54c1068d0 2024-03-04T09:21:34.103 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:33.284+0000 7fd5427fc700 1 -- 172.21.15.52:0/3998980865 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fd54804e020 con 0x7fd54c1068d0 2024-03-04T09:21:34.103 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:33.475+0000 7fd550906700 1 -- 172.21.15.52:0/3998980865 >> [v2:172.21.15.52:6800/2425730636,v1:172.21.15.52:6801/2425730636] conn(0x7fd538038c00 msgr2=0x7fd53803b0c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.52:6800/2425730636 2024-03-04T09:21:34.103 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:33.475+0000 7fd550906700 1 --2- 172.21.15.52:0/3998980865 >> [v2:172.21.15.52:6800/2425730636,v1:172.21.15.52:6801/2425730636] conn(0x7fd538038c00 0x7fd53803b0c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T09:21:34.103 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:33.546+0000 7fd55336b700 1 -- 172.21.15.52:0/3998980865 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) v1 -- 0x7fd54c194e60 con 0x7fd54c1068d0 2024-03-04T09:21:34.103 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:33.546+0000 7fd5427fc700 1 -- 172.21.15.52:0/3998980865 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) v1 ==== 56+0+88 (secure 0 0 0) 0x7fd548011e20 con 0x7fd54c1068d0 2024-03-04T09:21:34.103 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:33.548+0000 7fd55336b700 1 -- 172.21.15.52:0/3998980865 >> [v2:172.21.15.52:6800/2425730636,v1:172.21.15.52:6801/2425730636] conn(0x7fd538038c00 msgr2=0x7fd53803b0c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-04T09:21:34.103 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:33.548+0000 7fd55336b700 1 --2- 172.21.15.52:0/3998980865 >> [v2:172.21.15.52:6800/2425730636,v1:172.21.15.52:6801/2425730636] conn(0x7fd538038c00 0x7fd53803b0c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:21:34.104 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:33.548+0000 7fd55336b700 1 -- 172.21.15.52:0/3998980865 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd54c1068d0 msgr2=0x7fd54c10c790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:21:34.104 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:33.548+0000 7fd55336b700 1 --2- 172.21.15.52:0/3998980865 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd54c1068d0 0x7fd54c10c790 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fd5480116a0 tx=0x7fd548011ba0 comp rx=0 tx=0).stop 2024-03-04T09:21:34.104 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:33.548+0000 7fd55336b700 1 -- 172.21.15.52:0/3998980865 shutdown_connections 2024-03-04T09:21:34.104 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:33.548+0000 7fd55336b700 1 --2- 172.21.15.52:0/3998980865 >> [v2:172.21.15.52:6800/2425730636,v1:172.21.15.52:6801/2425730636] conn(0x7fd538038c00 0x7fd53803b0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:21:34.104 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:33.548+0000 7fd55336b700 1 --2- 172.21.15.52:0/3998980865 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd54c1068d0 0x7fd54c10c790 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:21:34.104 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:33.548+0000 7fd55336b700 1 -- 172.21.15.52:0/3998980865 >> 172.21.15.52:0/3998980865 conn(0x7fd54c101f70 msgr2=0x7fd54c1027d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:21:34.104 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:33.548+0000 7fd55336b700 1 -- 172.21.15.52:0/3998980865 shutdown_connections 2024-03-04T09:21:34.104 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:33.548+0000 7fd55336b700 1 -- 172.21.15.52:0/3998980865 wait complete. 2024-03-04T09:21:34.104 INFO:teuthology.orchestra.run.smithi052.stdout:Waiting for the mgr to restart... 2024-03-04T09:21:34.104 INFO:teuthology.orchestra.run.smithi052.stdout:Waiting for mgr epoch 9... 2024-03-04T09:21:34.369 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:21:34 smithi052 ceph-mon[32473]: from='client.? 172.21.15.52:0/3998980865' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-03-04T09:21:34.370 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Mar 04 09:21:34 smithi052 ceph-4e059f08-da08-11ee-95c6-87774f69a715-mgr-x[32705]: 2024-03-04T09:21:34.089+0000 7f9cb070a200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-04T09:21:34.370 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Mar 04 09:21:34 smithi052 ceph-4e059f08-da08-11ee-95c6-87774f69a715-mgr-x[32705]: 2024-03-04T09:21:34.372+0000 7f9cb070a200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-04T09:21:34.674 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Mar 04 09:21:34 smithi052 ceph-4e059f08-da08-11ee-95c6-87774f69a715-mgr-x[32705]: 2024-03-04T09:21:34.488+0000 7f9cb070a200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-04T09:21:34.674 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Mar 04 09:21:34 smithi052 ceph-4e059f08-da08-11ee-95c6-87774f69a715-mgr-x[32705]: 2024-03-04T09:21:34.579+0000 7f9cb070a200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-04T09:21:35.084 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Mar 04 09:21:34 smithi052 ceph-4e059f08-da08-11ee-95c6-87774f69a715-mgr-x[32705]: 2024-03-04T09:21:34.677+0000 7f9cb070a200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-04T09:21:35.085 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Mar 04 09:21:34 smithi052 ceph-4e059f08-da08-11ee-95c6-87774f69a715-mgr-x[32705]: 2024-03-04T09:21:34.856+0000 7f9cb070a200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-04T09:21:36.334 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Mar 04 09:21:35 smithi052 ceph-4e059f08-da08-11ee-95c6-87774f69a715-mgr-x[32705]: 2024-03-04T09:21:35.852+0000 7f9cb070a200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-04T09:21:36.335 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Mar 04 09:21:35 smithi052 ceph-4e059f08-da08-11ee-95c6-87774f69a715-mgr-x[32705]: 2024-03-04T09:21:35.988+0000 7f9cb070a200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-04T09:21:36.638 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Mar 04 09:21:36 smithi052 ceph-4e059f08-da08-11ee-95c6-87774f69a715-mgr-x[32705]: 2024-03-04T09:21:36.345+0000 7f9cb070a200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-04T09:21:36.638 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Mar 04 09:21:36 smithi052 ceph-4e059f08-da08-11ee-95c6-87774f69a715-mgr-x[32705]: 2024-03-04T09:21:36.438+0000 7f9cb070a200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-04T09:21:36.638 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Mar 04 09:21:36 smithi052 ceph-4e059f08-da08-11ee-95c6-87774f69a715-mgr-x[32705]: 2024-03-04T09:21:36.641+0000 7f9cb070a200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-04T09:21:37.084 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Mar 04 09:21:36 smithi052 ceph-4e059f08-da08-11ee-95c6-87774f69a715-mgr-x[32705]: 2024-03-04T09:21:36.833+0000 7f9cb070a200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-04T09:21:37.387 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Mar 04 09:21:37 smithi052 ceph-4e059f08-da08-11ee-95c6-87774f69a715-mgr-x[32705]: 2024-03-04T09:21:37.128+0000 7f9cb070a200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-04T09:21:37.834 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Mar 04 09:21:37 smithi052 ceph-4e059f08-da08-11ee-95c6-87774f69a715-mgr-x[32705]: 2024-03-04T09:21:37.390+0000 7f9cb070a200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-04T09:21:37.835 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Mar 04 09:21:37 smithi052 ceph-4e059f08-da08-11ee-95c6-87774f69a715-mgr-x[32705]: 2024-03-04T09:21:37.635+0000 7f9cb070a200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-04T09:21:38.834 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Mar 04 09:21:38 smithi052 ceph-4e059f08-da08-11ee-95c6-87774f69a715-mgr-x[32705]: 2024-03-04T09:21:38.473+0000 7f9cb070a200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-04T09:21:39.334 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Mar 04 09:21:39 smithi052 ceph-4e059f08-da08-11ee-95c6-87774f69a715-mgr-x[32705]: 2024-03-04T09:21:39.058+0000 7f9cb070a200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-04T09:21:39.335 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Mar 04 09:21:39 smithi052 ceph-4e059f08-da08-11ee-95c6-87774f69a715-mgr-x[32705]: 2024-03-04T09:21:39.161+0000 7f9cb070a200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-04T09:21:39.835 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Mar 04 09:21:39 smithi052 ceph-4e059f08-da08-11ee-95c6-87774f69a715-mgr-x[32705]: 2024-03-04T09:21:39.388+0000 7f9cb070a200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-04T09:21:39.835 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:21:39 smithi052 ceph-mon[32473]: Active manager daemon x restarted 2024-03-04T09:21:39.835 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:21:39 smithi052 ceph-mon[32473]: Activating manager daemon x 2024-03-04T09:21:39.835 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:21:39 smithi052 ceph-mon[32473]: osdmap e3: 0 total, 0 up, 0 in 2024-03-04T09:21:39.835 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:21:39 smithi052 ceph-mon[32473]: mgrmap e10: x(active, starting, since 0.00330268s) 2024-03-04T09:21:39.835 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:21:39 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-04T09:21:39.835 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:21:39 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-03-04T09:21:39.835 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:21:39 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-04T09:21:39.835 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:21:39 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-04T09:21:39.835 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:21:39 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-04T09:21:39.836 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:21:39 smithi052 ceph-mon[32473]: Manager daemon x is now available 2024-03-04T09:21:39.836 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:21:39 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T09:21:39.836 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:21:39 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T09:21:40.732 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:21:40 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-03-04T09:21:40.732 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:21:40 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-03-04T09:21:40.732 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:21:40 smithi052 ceph-mon[32473]: mgrmap e11: x(active, since 1.00582s) 2024-03-04T09:21:40.975 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout { 2024-03-04T09:21:40.975 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-03-04T09:21:40.975 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "initialized": true 2024-03-04T09:21:40.976 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout } 2024-03-04T09:21:40.976 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:34.492+0000 7fb872a2c700 1 Processor -- start 2024-03-04T09:21:40.976 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:34.492+0000 7fb872a2c700 1 -- start start 2024-03-04T09:21:40.976 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:34.492+0000 7fb872a2c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb86c108460 0x7fb86c10a850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:21:40.976 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:34.492+0000 7fb872a2c700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb86c07aea0 con 0x7fb86c108460 2024-03-04T09:21:40.976 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:34.493+0000 7fb86bfff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb86c108460 0x7fb86c10a850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:21:40.976 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:34.493+0000 7fb86bfff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb86c108460 0x7fb86c10a850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:49672/0 (socket says 172.21.15.52:49672) 2024-03-04T09:21:40.976 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:34.493+0000 7fb86bfff700 1 -- 172.21.15.52:0/56071291 learned_addr learned my addr 172.21.15.52:0/56071291 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-04T09:21:40.976 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:34.493+0000 7fb86bfff700 1 -- 172.21.15.52:0/56071291 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb86c10ad90 con 0x7fb86c108460 2024-03-04T09:21:40.976 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:34.493+0000 7fb86bfff700 1 --2- 172.21.15.52:0/56071291 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb86c108460 0x7fb86c10a850 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fb85c0098a0 tx=0x7fb85c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f419903fa45a5a5 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:21:40.976 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:34.493+0000 7fb86affd700 1 -- 172.21.15.52:0/56071291 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb85c004160 con 0x7fb86c108460 2024-03-04T09:21:40.977 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:34.493+0000 7fb86affd700 1 -- 172.21.15.52:0/56071291 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb85c004300 con 0x7fb86c108460 2024-03-04T09:21:40.977 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:34.493+0000 7fb86affd700 1 -- 172.21.15.52:0/56071291 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb85c011b20 con 0x7fb86c108460 2024-03-04T09:21:40.977 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:34.494+0000 7fb872a2c700 1 -- 172.21.15.52:0/56071291 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb86c108460 msgr2=0x7fb86c10a850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:21:40.977 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:34.494+0000 7fb872a2c700 1 --2- 172.21.15.52:0/56071291 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb86c108460 0x7fb86c10a850 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fb85c0098a0 tx=0x7fb85c009bb0 comp rx=0 tx=0).stop 2024-03-04T09:21:40.977 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:34.494+0000 7fb872a2c700 1 -- 172.21.15.52:0/56071291 shutdown_connections 2024-03-04T09:21:40.977 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:34.494+0000 7fb872a2c700 1 --2- 172.21.15.52:0/56071291 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb86c108460 0x7fb86c10a850 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:21:40.977 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:34.494+0000 7fb872a2c700 1 -- 172.21.15.52:0/56071291 >> 172.21.15.52:0/56071291 conn(0x7fb86c101f70 msgr2=0x7fb86c104390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:21:40.977 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:34.494+0000 7fb872a2c700 1 -- 172.21.15.52:0/56071291 shutdown_connections 2024-03-04T09:21:40.977 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:34.494+0000 7fb872a2c700 1 -- 172.21.15.52:0/56071291 wait complete. 2024-03-04T09:21:40.977 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:34.495+0000 7fb872a2c700 1 Processor -- start 2024-03-04T09:21:40.977 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:34.495+0000 7fb872a2c700 1 -- start start 2024-03-04T09:21:40.978 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:34.495+0000 7fb872a2c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb86c108460 0x7fb86c19b8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:21:40.978 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:34.495+0000 7fb872a2c700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb86c10bd90 con 0x7fb86c108460 2024-03-04T09:21:40.978 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:34.495+0000 7fb86bfff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb86c108460 0x7fb86c19b8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:21:40.978 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:34.495+0000 7fb86bfff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb86c108460 0x7fb86c19b8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:49686/0 (socket says 172.21.15.52:49686) 2024-03-04T09:21:40.978 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:34.495+0000 7fb86bfff700 1 -- 172.21.15.52:0/2461748865 learned_addr learned my addr 172.21.15.52:0/2461748865 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-04T09:21:40.978 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:34.496+0000 7fb86bfff700 1 -- 172.21.15.52:0/2461748865 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb86c19be30 con 0x7fb86c108460 2024-03-04T09:21:40.978 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:34.496+0000 7fb86bfff700 1 --2- 172.21.15.52:0/2461748865 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb86c108460 0x7fb86c19b8f0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fb85c011b60 tx=0x7fb85c011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:21:40.978 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:34.496+0000 7fb8697fa700 1 -- 172.21.15.52:0/2461748865 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb85c003b60 con 0x7fb86c108460 2024-03-04T09:21:40.978 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:34.496+0000 7fb8697fa700 1 -- 172.21.15.52:0/2461748865 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb85c01a910 con 0x7fb86c108460 2024-03-04T09:21:40.978 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:34.496+0000 7fb872a2c700 1 -- 172.21.15.52:0/2461748865 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb86c19c120 con 0x7fb86c108460 2024-03-04T09:21:40.978 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:34.496+0000 7fb8697fa700 1 -- 172.21.15.52:0/2461748865 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb85c024930 con 0x7fb86c108460 2024-03-04T09:21:40.978 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:34.496+0000 7fb872a2c700 1 -- 172.21.15.52:0/2461748865 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb86c19edb0 con 0x7fb86c108460 2024-03-04T09:21:40.979 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:34.497+0000 7fb8697fa700 1 -- 172.21.15.52:0/2461748865 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 9) v1 ==== 45941+0+0 (secure 0 0 0) 0x7fb85c022020 con 0x7fb86c108460 2024-03-04T09:21:40.979 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:34.497+0000 7fb8697fa700 1 --2- 172.21.15.52:0/2461748865 >> [v2:172.21.15.52:6800/2425730636,v1:172.21.15.52:6801/2425730636] conn(0x7fb854038c50 0x7fb85403b110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:21:40.979 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:34.497+0000 7fb8697fa700 1 -- 172.21.15.52:0/2461748865 --> [v2:172.21.15.52:6800/2425730636,v1:172.21.15.52:6801/2425730636] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7fb85403b820 con 0x7fb854038c50 2024-03-04T09:21:40.979 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:34.497+0000 7fb8697fa700 1 -- 172.21.15.52:0/2461748865 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fb85c04cbf0 con 0x7fb86c108460 2024-03-04T09:21:40.979 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:34.497+0000 7fb86b7fe700 1 -- 172.21.15.52:0/2461748865 >> [v2:172.21.15.52:6800/2425730636,v1:172.21.15.52:6801/2425730636] conn(0x7fb854038c50 msgr2=0x7fb85403b110 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.52:6800/2425730636 2024-03-04T09:21:40.979 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:34.497+0000 7fb86b7fe700 1 --2- 172.21.15.52:0/2461748865 >> [v2:172.21.15.52:6800/2425730636,v1:172.21.15.52:6801/2425730636] conn(0x7fb854038c50 0x7fb85403b110 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T09:21:40.979 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:34.698+0000 7fb86b7fe700 1 -- 172.21.15.52:0/2461748865 >> [v2:172.21.15.52:6800/2425730636,v1:172.21.15.52:6801/2425730636] conn(0x7fb854038c50 msgr2=0x7fb85403b110 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.52:6800/2425730636 2024-03-04T09:21:40.979 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:34.698+0000 7fb86b7fe700 1 --2- 172.21.15.52:0/2461748865 >> [v2:172.21.15.52:6800/2425730636,v1:172.21.15.52:6801/2425730636] conn(0x7fb854038c50 0x7fb85403b110 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T09:21:40.979 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:35.099+0000 7fb86b7fe700 1 -- 172.21.15.52:0/2461748865 >> [v2:172.21.15.52:6800/2425730636,v1:172.21.15.52:6801/2425730636] conn(0x7fb854038c50 msgr2=0x7fb85403b110 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.52:6800/2425730636 2024-03-04T09:21:40.979 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:35.099+0000 7fb86b7fe700 1 --2- 172.21.15.52:0/2461748865 >> [v2:172.21.15.52:6800/2425730636,v1:172.21.15.52:6801/2425730636] conn(0x7fb854038c50 0x7fb85403b110 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T09:21:40.979 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:35.900+0000 7fb86b7fe700 1 -- 172.21.15.52:0/2461748865 >> [v2:172.21.15.52:6800/2425730636,v1:172.21.15.52:6801/2425730636] conn(0x7fb854038c50 msgr2=0x7fb85403b110 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.52:6800/2425730636 2024-03-04T09:21:40.979 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:35.900+0000 7fb86b7fe700 1 --2- 172.21.15.52:0/2461748865 >> [v2:172.21.15.52:6800/2425730636,v1:172.21.15.52:6801/2425730636] conn(0x7fb854038c50 0x7fb85403b110 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T09:21:40.980 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:37.501+0000 7fb86b7fe700 1 -- 172.21.15.52:0/2461748865 >> [v2:172.21.15.52:6800/2425730636,v1:172.21.15.52:6801/2425730636] conn(0x7fb854038c50 msgr2=0x7fb85403b110 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.52:6800/2425730636 2024-03-04T09:21:40.980 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:37.501+0000 7fb86b7fe700 1 --2- 172.21.15.52:0/2461748865 >> [v2:172.21.15.52:6800/2425730636,v1:172.21.15.52:6801/2425730636] conn(0x7fb854038c50 0x7fb85403b110 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-03-04T09:21:40.980 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:39.403+0000 7fb8697fa700 1 -- 172.21.15.52:0/2461748865 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mgrmap(e 10) v1 ==== 45708+0+0 (secure 0 0 0) 0x7fb85c024ad0 con 0x7fb86c108460 2024-03-04T09:21:40.980 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:39.403+0000 7fb8697fa700 1 -- 172.21.15.52:0/2461748865 >> [v2:172.21.15.52:6800/2425730636,v1:172.21.15.52:6801/2425730636] conn(0x7fb854038c50 msgr2=0x7fb85403b110 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-04T09:21:40.980 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:39.403+0000 7fb8697fa700 1 --2- 172.21.15.52:0/2461748865 >> [v2:172.21.15.52:6800/2425730636,v1:172.21.15.52:6801/2425730636] conn(0x7fb854038c50 0x7fb85403b110 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:21:40.980 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:40.404+0000 7fb8697fa700 1 -- 172.21.15.52:0/2461748865 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mgrmap(e 11) v1 ==== 45835+0+0 (secure 0 0 0) 0x7fb85c04ef30 con 0x7fb86c108460 2024-03-04T09:21:40.980 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:40.404+0000 7fb8697fa700 1 --2- 172.21.15.52:0/2461748865 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7fb85403c740 0x7fb85403eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:21:40.980 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:40.405+0000 7fb8697fa700 1 -- 172.21.15.52:0/2461748865 --> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7fb85403b820 con 0x7fb85403c740 2024-03-04T09:21:40.980 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:40.405+0000 7fb86b7fe700 1 --2- 172.21.15.52:0/2461748865 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7fb85403c740 0x7fb85403eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:21:40.980 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:40.405+0000 7fb86b7fe700 1 --2- 172.21.15.52:0/2461748865 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7fb85403c740 0x7fb85403eb30 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fb860003a10 tx=0x7fb860009270 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:21:40.980 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:40.406+0000 7fb8697fa700 1 -- 172.21.15.52:0/2461748865 <== mgr.14152 v2:172.21.15.52:6800/3036033923 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+7774 (secure 0 0 0) 0x7fb85403b820 con 0x7fb85403c740 2024-03-04T09:21:40.981 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:40.418+0000 7fb872a2c700 1 -- 172.21.15.52:0/2461748865 --> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] -- command(tid 1: {"prefix": "mgr_status"}) v1 -- 0x7fb86c069170 con 0x7fb85403c740 2024-03-04T09:21:40.981 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:40.418+0000 7fb8697fa700 1 -- 172.21.15.52:0/2461748865 <== mgr.14152 v2:172.21.15.52:6800/3036033923 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+52 (secure 0 0 0) 0x7fb86c069170 con 0x7fb85403c740 2024-03-04T09:21:40.981 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:40.418+0000 7fb872a2c700 1 -- 172.21.15.52:0/2461748865 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7fb85403c740 msgr2=0x7fb85403eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:21:40.981 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:40.418+0000 7fb872a2c700 1 --2- 172.21.15.52:0/2461748865 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7fb85403c740 0x7fb85403eb30 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fb860003a10 tx=0x7fb860009270 comp rx=0 tx=0).stop 2024-03-04T09:21:40.981 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:40.418+0000 7fb872a2c700 1 -- 172.21.15.52:0/2461748865 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb86c108460 msgr2=0x7fb86c19b8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:21:40.981 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:40.418+0000 7fb872a2c700 1 --2- 172.21.15.52:0/2461748865 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb86c108460 0x7fb86c19b8f0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fb85c011b60 tx=0x7fb85c011c40 comp rx=0 tx=0).stop 2024-03-04T09:21:40.981 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:40.418+0000 7fb872a2c700 1 -- 172.21.15.52:0/2461748865 shutdown_connections 2024-03-04T09:21:40.981 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:40.418+0000 7fb872a2c700 1 --2- 172.21.15.52:0/2461748865 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7fb85403c740 0x7fb85403eb30 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:21:40.981 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:40.418+0000 7fb872a2c700 1 --2- 172.21.15.52:0/2461748865 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb86c108460 0x7fb86c19b8f0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:21:40.981 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:40.418+0000 7fb872a2c700 1 --2- 172.21.15.52:0/2461748865 >> [v2:172.21.15.52:6800/2425730636,v1:172.21.15.52:6801/2425730636] conn(0x7fb854038c50 0x7fb85403b110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:21:40.981 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:40.418+0000 7fb872a2c700 1 -- 172.21.15.52:0/2461748865 >> 172.21.15.52:0/2461748865 conn(0x7fb86c101f70 msgr2=0x7fb86c1029f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:21:40.981 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:40.419+0000 7fb872a2c700 1 -- 172.21.15.52:0/2461748865 shutdown_connections 2024-03-04T09:21:40.981 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:40.419+0000 7fb872a2c700 1 -- 172.21.15.52:0/2461748865 wait complete. 2024-03-04T09:21:40.982 INFO:teuthology.orchestra.run.smithi052.stdout:mgr epoch 9 is available 2024-03-04T09:21:40.982 INFO:teuthology.orchestra.run.smithi052.stdout:Generating a dashboard self-signed certificate... 2024-03-04T09:21:41.570 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:21:41 smithi052 ceph-mon[32473]: [04/Mar/2024:09:21:40] ENGINE Bus STARTING 2024-03-04T09:21:41.570 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:21:41 smithi052 ceph-mon[32473]: [04/Mar/2024:09:21:40] ENGINE Serving on http://172.21.15.52:8765 2024-03-04T09:21:41.570 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:21:41 smithi052 ceph-mon[32473]: [04/Mar/2024:09:21:41] ENGINE Serving on https://172.21.15.52:7150 2024-03-04T09:21:41.570 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:21:41 smithi052 ceph-mon[32473]: [04/Mar/2024:09:21:41] ENGINE Bus STARTED 2024-03-04T09:21:41.570 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:21:41 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:21:42.466 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-03-04T09:21:42.466 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:41.601+0000 7f66662a1700 1 Processor -- start 2024-03-04T09:21:42.466 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:41.601+0000 7f66662a1700 1 -- start start 2024-03-04T09:21:42.466 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:41.601+0000 7f66662a1700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6660107a60 0x7f6660107e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:21:42.466 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:41.601+0000 7f66662a1700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6660108450 con 0x7f6660107a60 2024-03-04T09:21:42.466 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:41.601+0000 7f665ffff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6660107a60 0x7f6660107e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:21:42.467 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:41.601+0000 7f665ffff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6660107a60 0x7f6660107e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:49740/0 (socket says 172.21.15.52:49740) 2024-03-04T09:21:42.467 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:41.601+0000 7f665ffff700 1 -- 172.21.15.52:0/1700569989 learned_addr learned my addr 172.21.15.52:0/1700569989 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-04T09:21:42.467 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:41.602+0000 7f665ffff700 1 -- 172.21.15.52:0/1700569989 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6660108c10 con 0x7f6660107a60 2024-03-04T09:21:42.467 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:41.602+0000 7f665ffff700 1 --2- 172.21.15.52:0/1700569989 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6660107a60 0x7f6660107e80 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f66500098a0 tx=0x7f6650009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=defd22cd7dfead1f server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:21:42.467 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:41.602+0000 7f665effd700 1 -- 172.21.15.52:0/1700569989 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6650004160 con 0x7f6660107a60 2024-03-04T09:21:42.467 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:41.602+0000 7f665effd700 1 -- 172.21.15.52:0/1700569989 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6650004300 con 0x7f6660107a60 2024-03-04T09:21:42.467 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:41.602+0000 7f665effd700 1 -- 172.21.15.52:0/1700569989 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6650011b20 con 0x7f6660107a60 2024-03-04T09:21:42.467 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:41.602+0000 7f66662a1700 1 -- 172.21.15.52:0/1700569989 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6660107a60 msgr2=0x7f6660107e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:21:42.467 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:41.603+0000 7f66662a1700 1 --2- 172.21.15.52:0/1700569989 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6660107a60 0x7f6660107e80 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f66500098a0 tx=0x7f6650009bb0 comp rx=0 tx=0).stop 2024-03-04T09:21:42.467 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:41.603+0000 7f66662a1700 1 -- 172.21.15.52:0/1700569989 shutdown_connections 2024-03-04T09:21:42.467 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:41.603+0000 7f66662a1700 1 --2- 172.21.15.52:0/1700569989 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6660107a60 0x7f6660107e80 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:21:42.467 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:41.603+0000 7f66662a1700 1 -- 172.21.15.52:0/1700569989 >> 172.21.15.52:0/1700569989 conn(0x7f6660103060 msgr2=0x7f6660105480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:21:42.468 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:41.603+0000 7f66662a1700 1 -- 172.21.15.52:0/1700569989 shutdown_connections 2024-03-04T09:21:42.468 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:41.603+0000 7f66662a1700 1 -- 172.21.15.52:0/1700569989 wait complete. 2024-03-04T09:21:42.468 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:41.603+0000 7f66662a1700 1 Processor -- start 2024-03-04T09:21:42.468 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:41.603+0000 7f66662a1700 1 -- start start 2024-03-04T09:21:42.468 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:41.604+0000 7f66662a1700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6660107a60 0x7f666019b930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:21:42.468 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:41.604+0000 7f66662a1700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f666010bfc0 con 0x7f6660107a60 2024-03-04T09:21:42.468 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:41.604+0000 7f665ffff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6660107a60 0x7f666019b930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:21:42.468 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:41.604+0000 7f665ffff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6660107a60 0x7f666019b930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:49756/0 (socket says 172.21.15.52:49756) 2024-03-04T09:21:42.468 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:41.604+0000 7f665ffff700 1 -- 172.21.15.52:0/4171140106 learned_addr learned my addr 172.21.15.52:0/4171140106 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-04T09:21:42.468 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:41.604+0000 7f665ffff700 1 -- 172.21.15.52:0/4171140106 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f666019be70 con 0x7f6660107a60 2024-03-04T09:21:42.468 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:41.604+0000 7f665ffff700 1 --2- 172.21.15.52:0/4171140106 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6660107a60 0x7f666019b930 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f66500117e0 tx=0x7f6650011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:21:42.469 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:41.604+0000 7f665d7fa700 1 -- 172.21.15.52:0/4171140106 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6650003bf0 con 0x7f6660107a60 2024-03-04T09:21:42.469 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:41.604+0000 7f66662a1700 1 -- 172.21.15.52:0/4171140106 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f666019c100 con 0x7f6660107a60 2024-03-04T09:21:42.469 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:41.604+0000 7f665d7fa700 1 -- 172.21.15.52:0/4171140106 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f665001a910 con 0x7f6660107a60 2024-03-04T09:21:42.469 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:41.604+0000 7f665d7fa700 1 -- 172.21.15.52:0/4171140106 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6650024970 con 0x7f6660107a60 2024-03-04T09:21:42.469 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:41.604+0000 7f66662a1700 1 -- 172.21.15.52:0/4171140106 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f666019c560 con 0x7f6660107a60 2024-03-04T09:21:42.469 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:41.605+0000 7f66662a1700 1 -- 172.21.15.52:0/4171140106 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6660069170 con 0x7f6660107a60 2024-03-04T09:21:42.469 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:41.605+0000 7f665d7fa700 1 -- 172.21.15.52:0/4171140106 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 12) v1 ==== 45941+0+0 (secure 0 0 0) 0x7f6650022020 con 0x7f6660107a60 2024-03-04T09:21:42.469 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:41.606+0000 7f665d7fa700 1 --2- 172.21.15.52:0/4171140106 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f6648038bf0 0x7f664803b0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:21:42.469 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:41.606+0000 7f665d7fa700 1 -- 172.21.15.52:0/4171140106 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f665004d2f0 con 0x7f6660107a60 2024-03-04T09:21:42.469 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:41.606+0000 7f665f7fe700 1 --2- 172.21.15.52:0/4171140106 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f6648038bf0 0x7f664803b0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:21:42.469 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:41.606+0000 7f665f7fe700 1 --2- 172.21.15.52:0/4171140106 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f6648038bf0 0x7f664803b0b0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f6654006fd0 tx=0x7f6654006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:21:42.469 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:41.611+0000 7f665d7fa700 1 -- 172.21.15.52:0/4171140106 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f66500172b0 con 0x7f6660107a60 2024-03-04T09:21:42.470 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:41.820+0000 7f66662a1700 1 -- 172.21.15.52:0/4171140106 --> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) v1 -- 0x7f6660104210 con 0x7f6648038bf0 2024-03-04T09:21:42.470 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:41.896+0000 7f665d7fa700 1 -- 172.21.15.52:0/4171140106 <== mgr.14152 v2:172.21.15.52:6800/3036033923 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+32 (secure 0 0 0) 0x7f6660104210 con 0x7f6648038bf0 2024-03-04T09:21:42.470 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:41.898+0000 7f66662a1700 1 -- 172.21.15.52:0/4171140106 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f6648038bf0 msgr2=0x7f664803b0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:21:42.470 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:41.898+0000 7f66662a1700 1 --2- 172.21.15.52:0/4171140106 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f6648038bf0 0x7f664803b0b0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f6654006fd0 tx=0x7f6654006e40 comp rx=0 tx=0).stop 2024-03-04T09:21:42.470 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:41.898+0000 7f66662a1700 1 -- 172.21.15.52:0/4171140106 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6660107a60 msgr2=0x7f666019b930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:21:42.470 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:41.898+0000 7f66662a1700 1 --2- 172.21.15.52:0/4171140106 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6660107a60 0x7f666019b930 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f66500117e0 tx=0x7f6650011ba0 comp rx=0 tx=0).stop 2024-03-04T09:21:42.470 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:41.898+0000 7f66662a1700 1 -- 172.21.15.52:0/4171140106 shutdown_connections 2024-03-04T09:21:42.470 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:41.898+0000 7f66662a1700 1 --2- 172.21.15.52:0/4171140106 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f6648038bf0 0x7f664803b0b0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:21:42.470 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:41.898+0000 7f66662a1700 1 --2- 172.21.15.52:0/4171140106 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6660107a60 0x7f666019b930 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:21:42.470 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:41.898+0000 7f66662a1700 1 -- 172.21.15.52:0/4171140106 >> 172.21.15.52:0/4171140106 conn(0x7f6660103060 msgr2=0x7f6660103b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:21:42.470 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:41.898+0000 7f66662a1700 1 -- 172.21.15.52:0/4171140106 shutdown_connections 2024-03-04T09:21:42.470 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:41.898+0000 7f66662a1700 1 -- 172.21.15.52:0/4171140106 wait complete. 2024-03-04T09:21:42.471 INFO:teuthology.orchestra.run.smithi052.stdout:Creating initial admin user... 2024-03-04T09:21:42.707 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:21:42 smithi052 ceph-mon[32473]: mgrmap e12: x(active, since 2s) 2024-03-04T09:21:42.707 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:21:42 smithi052 ceph-mon[32473]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T09:21:42.707 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:21:42 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:21:42.707 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:21:42 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:21:43.584 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:21:43 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:21:43.585 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:21:43 smithi052 ceph-mon[32473]: from='client.14166 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T09:21:43.585 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:21:43 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:21:44.579 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$Slnze5K0BIw/GU8ggTYbROIeXhwzslwtGXTkEgtY2o.v/DV8QOgZi", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1709544103, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-03-04T09:21:44.579 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:42.885+0000 7fd00b57b700 1 Processor -- start 2024-03-04T09:21:44.579 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:42.886+0000 7fd00b57b700 1 -- start start 2024-03-04T09:21:44.580 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:42.886+0000 7fd00b57b700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd004108410 0x7fd00410a800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:21:44.580 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:42.886+0000 7fd00b57b700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd00407aec0 con 0x7fd004108410 2024-03-04T09:21:44.580 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:42.886+0000 7fd009317700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd004108410 0x7fd00410a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:21:44.580 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:42.886+0000 7fd009317700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd004108410 0x7fd00410a800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:49764/0 (socket says 172.21.15.52:49764) 2024-03-04T09:21:44.580 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:42.886+0000 7fd009317700 1 -- 172.21.15.52:0/1183733061 learned_addr learned my addr 172.21.15.52:0/1183733061 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-04T09:21:44.580 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:42.886+0000 7fd009317700 1 -- 172.21.15.52:0/1183733061 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd00410ad40 con 0x7fd004108410 2024-03-04T09:21:44.580 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:42.887+0000 7fd009317700 1 --2- 172.21.15.52:0/1183733061 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd004108410 0x7fd00410a800 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fcff80098a0 tx=0x7fcff8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24322dc8a3ef7de2 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:21:44.580 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:42.887+0000 7fcff7fff700 1 -- 172.21.15.52:0/1183733061 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fcff8004160 con 0x7fd004108410 2024-03-04T09:21:44.580 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:42.887+0000 7fcff7fff700 1 -- 172.21.15.52:0/1183733061 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fcff8004300 con 0x7fd004108410 2024-03-04T09:21:44.581 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:42.887+0000 7fcff7fff700 1 -- 172.21.15.52:0/1183733061 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fcff8011b20 con 0x7fd004108410 2024-03-04T09:21:44.581 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:42.888+0000 7fd00b57b700 1 -- 172.21.15.52:0/1183733061 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd004108410 msgr2=0x7fd00410a800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:21:44.581 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:42.888+0000 7fd00b57b700 1 --2- 172.21.15.52:0/1183733061 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd004108410 0x7fd00410a800 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fcff80098a0 tx=0x7fcff8009bb0 comp rx=0 tx=0).stop 2024-03-04T09:21:44.581 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:42.888+0000 7fd00b57b700 1 -- 172.21.15.52:0/1183733061 shutdown_connections 2024-03-04T09:21:44.581 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:42.888+0000 7fd00b57b700 1 --2- 172.21.15.52:0/1183733061 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd004108410 0x7fd00410a800 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:21:44.581 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:42.888+0000 7fd00b57b700 1 -- 172.21.15.52:0/1183733061 >> 172.21.15.52:0/1183733061 conn(0x7fd004101ee0 msgr2=0x7fd004104300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:21:44.581 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:42.888+0000 7fd00b57b700 1 -- 172.21.15.52:0/1183733061 shutdown_connections 2024-03-04T09:21:44.581 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:42.888+0000 7fd00b57b700 1 -- 172.21.15.52:0/1183733061 wait complete. 2024-03-04T09:21:44.581 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:42.888+0000 7fd00b57b700 1 Processor -- start 2024-03-04T09:21:44.581 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:42.888+0000 7fd00b57b700 1 -- start start 2024-03-04T09:21:44.581 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:42.889+0000 7fd00b57b700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd004108410 0x7fd00419b910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:21:44.581 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:42.889+0000 7fd00b57b700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd00410bdb0 con 0x7fd004108410 2024-03-04T09:21:44.581 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:42.889+0000 7fd009317700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd004108410 0x7fd00419b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:21:44.581 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:42.889+0000 7fd009317700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd004108410 0x7fd00419b910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:49774/0 (socket says 172.21.15.52:49774) 2024-03-04T09:21:44.582 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:42.889+0000 7fd009317700 1 -- 172.21.15.52:0/941901968 learned_addr learned my addr 172.21.15.52:0/941901968 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-04T09:21:44.582 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:42.889+0000 7fd009317700 1 -- 172.21.15.52:0/941901968 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd00419be50 con 0x7fd004108410 2024-03-04T09:21:44.582 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:42.889+0000 7fd009317700 1 --2- 172.21.15.52:0/941901968 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd004108410 0x7fd00419b910 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fcff8011b60 tx=0x7fcff8011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:21:44.582 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:42.889+0000 7fcff67fc700 1 -- 172.21.15.52:0/941901968 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fcff8003b60 con 0x7fd004108410 2024-03-04T09:21:44.582 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:42.889+0000 7fcff67fc700 1 -- 172.21.15.52:0/941901968 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fcff801a910 con 0x7fd004108410 2024-03-04T09:21:44.582 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:42.889+0000 7fd00b57b700 1 -- 172.21.15.52:0/941901968 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd00419c0e0 con 0x7fd004108410 2024-03-04T09:21:44.582 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:42.889+0000 7fcff67fc700 1 -- 172.21.15.52:0/941901968 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fcff8024930 con 0x7fd004108410 2024-03-04T09:21:44.582 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:42.889+0000 7fd00b57b700 1 -- 172.21.15.52:0/941901968 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd00419c540 con 0x7fd004108410 2024-03-04T09:21:44.582 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:42.890+0000 7fcff67fc700 1 -- 172.21.15.52:0/941901968 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 12) v1 ==== 45941+0+0 (secure 0 0 0) 0x7fcff8022020 con 0x7fd004108410 2024-03-04T09:21:44.582 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:42.891+0000 7fcff67fc700 1 --2- 172.21.15.52:0/941901968 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7fcff0038bf0 0x7fcff003b0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:21:44.582 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:42.891+0000 7fd00b57b700 1 -- 172.21.15.52:0/941901968 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd004069170 con 0x7fd004108410 2024-03-04T09:21:44.582 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:42.891+0000 7fcff67fc700 1 -- 172.21.15.52:0/941901968 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fcff804cea0 con 0x7fd004108410 2024-03-04T09:21:44.582 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:42.891+0000 7fd008b16700 1 --2- 172.21.15.52:0/941901968 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7fcff0038bf0 0x7fcff003b0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:21:44.582 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:42.891+0000 7fd008b16700 1 --2- 172.21.15.52:0/941901968 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7fcff0038bf0 0x7fcff003b0b0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fd000006fd0 tx=0x7fd000006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:21:44.583 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:42.896+0000 7fcff67fc700 1 -- 172.21.15.52:0/941901968 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fcff8027120 con 0x7fd004108410 2024-03-04T09:21:44.583 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:43.149+0000 7fd00b57b700 1 -- 172.21.15.52:0/941901968 --> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) v1 -- 0x7fd00419ef50 con 0x7fcff0038bf0 2024-03-04T09:21:44.583 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:43.396+0000 7fcff67fc700 1 -- 172.21.15.52:0/941901968 <== mgr.14152 v2:172.21.15.52:6800/3036033923 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+252 (secure 0 0 0) 0x7fd00419ef50 con 0x7fcff0038bf0 2024-03-04T09:21:44.583 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:43.398+0000 7fd00b57b700 1 -- 172.21.15.52:0/941901968 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7fcff0038bf0 msgr2=0x7fcff003b0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:21:44.583 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:43.398+0000 7fd00b57b700 1 --2- 172.21.15.52:0/941901968 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7fcff0038bf0 0x7fcff003b0b0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fd000006fd0 tx=0x7fd000006e40 comp rx=0 tx=0).stop 2024-03-04T09:21:44.583 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:43.398+0000 7fd00b57b700 1 -- 172.21.15.52:0/941901968 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd004108410 msgr2=0x7fd00419b910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:21:44.583 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:43.398+0000 7fd00b57b700 1 --2- 172.21.15.52:0/941901968 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd004108410 0x7fd00419b910 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fcff8011b60 tx=0x7fcff8011c40 comp rx=0 tx=0).stop 2024-03-04T09:21:44.583 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:43.398+0000 7fd00b57b700 1 -- 172.21.15.52:0/941901968 shutdown_connections 2024-03-04T09:21:44.583 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:43.398+0000 7fd00b57b700 1 --2- 172.21.15.52:0/941901968 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7fcff0038bf0 0x7fcff003b0b0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:21:44.583 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:43.398+0000 7fd00b57b700 1 --2- 172.21.15.52:0/941901968 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd004108410 0x7fd00419b910 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:21:44.583 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:43.398+0000 7fd00b57b700 1 -- 172.21.15.52:0/941901968 >> 172.21.15.52:0/941901968 conn(0x7fd004101ee0 msgr2=0x7fd0041029d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:21:44.583 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:43.398+0000 7fd00b57b700 1 -- 172.21.15.52:0/941901968 shutdown_connections 2024-03-04T09:21:44.583 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:43.398+0000 7fd00b57b700 1 -- 172.21.15.52:0/941901968 wait complete. 2024-03-04T09:21:44.583 INFO:teuthology.orchestra.run.smithi052.stdout:Fetching dashboard port number... 2024-03-04T09:21:45.585 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:21:45 smithi052 ceph-mon[32473]: from='client.? 172.21.15.52:0/1153621703' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-03-04T09:21:45.827 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout 8443 2024-03-04T09:21:45.827 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:45.012+0000 7fa5adea0700 1 Processor -- start 2024-03-04T09:21:45.827 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:45.012+0000 7fa5adea0700 1 -- start start 2024-03-04T09:21:45.827 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:45.012+0000 7fa5adea0700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa5a8108480 0x7fa5a810a870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:21:45.828 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:45.012+0000 7fa5adea0700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5a807ae60 con 0x7fa5a8108480 2024-03-04T09:21:45.828 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:45.013+0000 7fa5a77fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa5a8108480 0x7fa5a810a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:21:45.828 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:45.013+0000 7fa5a77fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa5a8108480 0x7fa5a810a870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:56332/0 (socket says 172.21.15.52:56332) 2024-03-04T09:21:45.828 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:45.013+0000 7fa5a77fe700 1 -- 172.21.15.52:0/1444940823 learned_addr learned my addr 172.21.15.52:0/1444940823 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-04T09:21:45.828 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:45.013+0000 7fa5a77fe700 1 -- 172.21.15.52:0/1444940823 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5a810adb0 con 0x7fa5a8108480 2024-03-04T09:21:45.828 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:45.013+0000 7fa5a77fe700 1 --2- 172.21.15.52:0/1444940823 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa5a8108480 0x7fa5a810a870 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fa5980098a0 tx=0x7fa598009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22612365253190c2 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:21:45.828 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:45.013+0000 7fa5a67fc700 1 -- 172.21.15.52:0/1444940823 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa598004160 con 0x7fa5a8108480 2024-03-04T09:21:45.828 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:45.013+0000 7fa5a67fc700 1 -- 172.21.15.52:0/1444940823 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa598004300 con 0x7fa5a8108480 2024-03-04T09:21:45.828 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:45.013+0000 7fa5a67fc700 1 -- 172.21.15.52:0/1444940823 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa598011b20 con 0x7fa5a8108480 2024-03-04T09:21:45.828 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:45.014+0000 7fa5adea0700 1 -- 172.21.15.52:0/1444940823 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa5a8108480 msgr2=0x7fa5a810a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:21:45.828 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:45.014+0000 7fa5adea0700 1 --2- 172.21.15.52:0/1444940823 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa5a8108480 0x7fa5a810a870 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fa5980098a0 tx=0x7fa598009bb0 comp rx=0 tx=0).stop 2024-03-04T09:21:45.828 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:45.014+0000 7fa5adea0700 1 -- 172.21.15.52:0/1444940823 shutdown_connections 2024-03-04T09:21:45.829 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:45.014+0000 7fa5adea0700 1 --2- 172.21.15.52:0/1444940823 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa5a8108480 0x7fa5a810a870 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:21:45.829 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:45.014+0000 7fa5adea0700 1 -- 172.21.15.52:0/1444940823 >> 172.21.15.52:0/1444940823 conn(0x7fa5a8101e90 msgr2=0x7fa5a81042b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:21:45.829 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:45.014+0000 7fa5adea0700 1 -- 172.21.15.52:0/1444940823 shutdown_connections 2024-03-04T09:21:45.829 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:45.014+0000 7fa5adea0700 1 -- 172.21.15.52:0/1444940823 wait complete. 2024-03-04T09:21:45.829 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:45.014+0000 7fa5adea0700 1 Processor -- start 2024-03-04T09:21:45.829 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:45.015+0000 7fa5adea0700 1 -- start start 2024-03-04T09:21:45.829 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:45.015+0000 7fa5adea0700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa5a8108480 0x7fa5a819dac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:21:45.829 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:45.015+0000 7fa5adea0700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5a810bdc0 con 0x7fa5a8108480 2024-03-04T09:21:45.829 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:45.015+0000 7fa5a77fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa5a8108480 0x7fa5a819dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:21:45.829 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:45.015+0000 7fa5a77fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa5a8108480 0x7fa5a819dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:56338/0 (socket says 172.21.15.52:56338) 2024-03-04T09:21:45.830 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:45.015+0000 7fa5a77fe700 1 -- 172.21.15.52:0/1153621703 learned_addr learned my addr 172.21.15.52:0/1153621703 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-04T09:21:45.830 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:45.015+0000 7fa5a77fe700 1 -- 172.21.15.52:0/1153621703 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5a819e000 con 0x7fa5a8108480 2024-03-04T09:21:45.830 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:45.015+0000 7fa5a77fe700 1 --2- 172.21.15.52:0/1153621703 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa5a8108480 0x7fa5a819dac0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fa5980117e0 tx=0x7fa598011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:21:45.830 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:45.015+0000 7fa5a4ff9700 1 -- 172.21.15.52:0/1153621703 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa598003bf0 con 0x7fa5a8108480 2024-03-04T09:21:45.830 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:45.015+0000 7fa5a4ff9700 1 -- 172.21.15.52:0/1153621703 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa59801a910 con 0x7fa5a8108480 2024-03-04T09:21:45.830 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:45.015+0000 7fa5adea0700 1 -- 172.21.15.52:0/1153621703 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5a819e290 con 0x7fa5a8108480 2024-03-04T09:21:45.830 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:45.015+0000 7fa5a4ff9700 1 -- 172.21.15.52:0/1153621703 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa598024970 con 0x7fa5a8108480 2024-03-04T09:21:45.830 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:45.016+0000 7fa5adea0700 1 -- 172.21.15.52:0/1153621703 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5a819e6f0 con 0x7fa5a8108480 2024-03-04T09:21:45.830 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:45.016+0000 7fa5a4ff9700 1 -- 172.21.15.52:0/1153621703 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 12) v1 ==== 45941+0+0 (secure 0 0 0) 0x7fa598022020 con 0x7fa5a8108480 2024-03-04T09:21:45.830 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:45.016+0000 7fa5adea0700 1 -- 172.21.15.52:0/1153621703 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa5a80636f0 con 0x7fa5a8108480 2024-03-04T09:21:45.830 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:45.017+0000 7fa5a4ff9700 1 --2- 172.21.15.52:0/1153621703 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7fa590038c00 0x7fa59003b0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:21:45.830 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:45.017+0000 7fa5a4ff9700 1 -- 172.21.15.52:0/1153621703 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fa59804ce60 con 0x7fa5a8108480 2024-03-04T09:21:45.830 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:45.017+0000 7fa5a6ffd700 1 --2- 172.21.15.52:0/1153621703 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7fa590038c00 0x7fa59003b0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:21:45.830 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:45.017+0000 7fa5a6ffd700 1 --2- 172.21.15.52:0/1153621703 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7fa590038c00 0x7fa59003b0c0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fa59c006fd0 tx=0x7fa59c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:21:45.831 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:45.021+0000 7fa5a4ff9700 1 -- 172.21.15.52:0/1153621703 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fa598011e20 con 0x7fa5a8108480 2024-03-04T09:21:45.831 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:45.227+0000 7fa5adea0700 1 -- 172.21.15.52:0/1153621703 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) v1 -- 0x7fa5a81a10c0 con 0x7fa5a8108480 2024-03-04T09:21:45.831 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:45.227+0000 7fa5a4ff9700 1 -- 172.21.15.52:0/1153621703 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) v1 ==== 112+0+5 (secure 0 0 0) 0x7fa59800ba30 con 0x7fa5a8108480 2024-03-04T09:21:45.831 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:45.229+0000 7fa5adea0700 1 -- 172.21.15.52:0/1153621703 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7fa590038c00 msgr2=0x7fa59003b0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:21:45.831 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:45.229+0000 7fa5adea0700 1 --2- 172.21.15.52:0/1153621703 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7fa590038c00 0x7fa59003b0c0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fa59c006fd0 tx=0x7fa59c006e40 comp rx=0 tx=0).stop 2024-03-04T09:21:45.831 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:45.229+0000 7fa5adea0700 1 -- 172.21.15.52:0/1153621703 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa5a8108480 msgr2=0x7fa5a819dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:21:45.831 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:45.229+0000 7fa5adea0700 1 --2- 172.21.15.52:0/1153621703 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa5a8108480 0x7fa5a819dac0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fa5980117e0 tx=0x7fa598011c10 comp rx=0 tx=0).stop 2024-03-04T09:21:45.831 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:45.229+0000 7fa5adea0700 1 -- 172.21.15.52:0/1153621703 shutdown_connections 2024-03-04T09:21:45.831 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:45.229+0000 7fa5adea0700 1 --2- 172.21.15.52:0/1153621703 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7fa590038c00 0x7fa59003b0c0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:21:45.831 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:45.229+0000 7fa5adea0700 1 --2- 172.21.15.52:0/1153621703 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa5a8108480 0x7fa5a819dac0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:21:45.831 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:45.229+0000 7fa5adea0700 1 -- 172.21.15.52:0/1153621703 >> 172.21.15.52:0/1153621703 conn(0x7fa5a8101e90 msgr2=0x7fa5a8102930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:21:45.831 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:45.230+0000 7fa5adea0700 1 -- 172.21.15.52:0/1153621703 shutdown_connections 2024-03-04T09:21:45.831 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:45.230+0000 7fa5adea0700 1 -- 172.21.15.52:0/1153621703 wait complete. 2024-03-04T09:21:45.835 INFO:teuthology.orchestra.run.smithi052.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-03-04T09:21:45.835 INFO:teuthology.orchestra.run.smithi052.stdout:systemctl: stdout disabled 2024-03-04T09:21:45.842 INFO:teuthology.orchestra.run.smithi052.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-03-04T09:21:45.842 INFO:teuthology.orchestra.run.smithi052.stdout:systemctl: stdout inactive 2024-03-04T09:21:45.843 INFO:teuthology.orchestra.run.smithi052.stdout:firewalld.service is not enabled 2024-03-04T09:21:45.843 INFO:teuthology.orchestra.run.smithi052.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-03-04T09:21:45.845 INFO:teuthology.orchestra.run.smithi052.stdout:Ceph Dashboard is now available at: 2024-03-04T09:21:45.845 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-03-04T09:21:45.845 INFO:teuthology.orchestra.run.smithi052.stdout: URL: https://smithi052.front.sepia.ceph.com:8443/ 2024-03-04T09:21:45.845 INFO:teuthology.orchestra.run.smithi052.stdout: User: admin 2024-03-04T09:21:45.845 INFO:teuthology.orchestra.run.smithi052.stdout: Password: ymx9vu8m6x 2024-03-04T09:21:45.845 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-03-04T09:21:45.845 INFO:teuthology.orchestra.run.smithi052.stdout:Saving cluster configuration to /var/lib/ceph/4e059f08-da08-11ee-95c6-87774f69a715/config directory 2024-03-04T09:21:45.846 INFO:teuthology.orchestra.run.smithi052.stdout:Enabling autotune for osd_memory_target 2024-03-04T09:21:47.076 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:46.303+0000 7f893cd18700 1 Processor -- start 2024-03-04T09:21:47.076 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:46.303+0000 7f893cd18700 1 -- start start 2024-03-04T09:21:47.076 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:46.304+0000 7f893cd18700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f89381079c0 0x7f8938107de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:21:47.076 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:46.304+0000 7f893cd18700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89381083b0 con 0x7f89381079c0 2024-03-04T09:21:47.076 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:46.304+0000 7f893659c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f89381079c0 0x7f8938107de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:21:47.076 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:46.304+0000 7f893659c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f89381079c0 0x7f8938107de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:56348/0 (socket says 172.21.15.52:56348) 2024-03-04T09:21:47.076 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:46.304+0000 7f893659c700 1 -- 172.21.15.52:0/3835514906 learned_addr learned my addr 172.21.15.52:0/3835514906 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-04T09:21:47.076 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:46.304+0000 7f893659c700 1 -- 172.21.15.52:0/3835514906 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8938108bc0 con 0x7f89381079c0 2024-03-04T09:21:47.076 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:46.304+0000 7f893659c700 1 --2- 172.21.15.52:0/3835514906 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f89381079c0 0x7f8938107de0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f89280098a0 tx=0x7f8928009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24d46f2eda4d9804 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:21:47.076 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:46.304+0000 7f893559a700 1 -- 172.21.15.52:0/3835514906 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8928004160 con 0x7f89381079c0 2024-03-04T09:21:47.077 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:46.305+0000 7f893559a700 1 -- 172.21.15.52:0/3835514906 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8928004300 con 0x7f89381079c0 2024-03-04T09:21:47.077 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:46.305+0000 7f893559a700 1 -- 172.21.15.52:0/3835514906 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8928011b20 con 0x7f89381079c0 2024-03-04T09:21:47.077 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:46.305+0000 7f893cd18700 1 -- 172.21.15.52:0/3835514906 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f89381079c0 msgr2=0x7f8938107de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:21:47.077 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:46.305+0000 7f893cd18700 1 --2- 172.21.15.52:0/3835514906 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f89381079c0 0x7f8938107de0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f89280098a0 tx=0x7f8928009bb0 comp rx=0 tx=0).stop 2024-03-04T09:21:47.077 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:46.305+0000 7f893cd18700 1 -- 172.21.15.52:0/3835514906 shutdown_connections 2024-03-04T09:21:47.077 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:46.305+0000 7f893cd18700 1 --2- 172.21.15.52:0/3835514906 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f89381079c0 0x7f8938107de0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:21:47.077 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:46.305+0000 7f893cd18700 1 -- 172.21.15.52:0/3835514906 >> 172.21.15.52:0/3835514906 conn(0x7f8938103000 msgr2=0x7f8938105420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:21:47.077 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:46.305+0000 7f893cd18700 1 -- 172.21.15.52:0/3835514906 shutdown_connections 2024-03-04T09:21:47.077 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:46.305+0000 7f893cd18700 1 -- 172.21.15.52:0/3835514906 wait complete. 2024-03-04T09:21:47.077 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:46.306+0000 7f893cd18700 1 Processor -- start 2024-03-04T09:21:47.077 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:46.306+0000 7f893cd18700 1 -- start start 2024-03-04T09:21:47.077 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:46.306+0000 7f893cd18700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f89381079c0 0x7f893819b990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:21:47.077 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:46.306+0000 7f893cd18700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f893810bf70 con 0x7f89381079c0 2024-03-04T09:21:47.078 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:46.306+0000 7f893659c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f89381079c0 0x7f893819b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:21:47.078 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:46.306+0000 7f893659c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f89381079c0 0x7f893819b990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:56360/0 (socket says 172.21.15.52:56360) 2024-03-04T09:21:47.078 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:46.306+0000 7f893659c700 1 -- 172.21.15.52:0/2812261473 learned_addr learned my addr 172.21.15.52:0/2812261473 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-04T09:21:47.078 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:46.307+0000 7f893659c700 1 -- 172.21.15.52:0/2812261473 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f893819bed0 con 0x7f89381079c0 2024-03-04T09:21:47.078 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:46.307+0000 7f893659c700 1 --2- 172.21.15.52:0/2812261473 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f89381079c0 0x7f893819b990 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f89280117e0 tx=0x7f8928011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:21:47.078 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:46.307+0000 7f89277fe700 1 -- 172.21.15.52:0/2812261473 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8928003bf0 con 0x7f89381079c0 2024-03-04T09:21:47.078 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:46.307+0000 7f89277fe700 1 -- 172.21.15.52:0/2812261473 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f892801a910 con 0x7f89381079c0 2024-03-04T09:21:47.078 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:46.307+0000 7f893cd18700 1 -- 172.21.15.52:0/2812261473 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f893819c160 con 0x7f89381079c0 2024-03-04T09:21:47.078 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:46.307+0000 7f89277fe700 1 -- 172.21.15.52:0/2812261473 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f89280249a0 con 0x7f89381079c0 2024-03-04T09:21:47.078 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:46.307+0000 7f893cd18700 1 -- 172.21.15.52:0/2812261473 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f893819c5c0 con 0x7f89381079c0 2024-03-04T09:21:47.078 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:46.308+0000 7f89277fe700 1 -- 172.21.15.52:0/2812261473 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 13) v1 ==== 45984+0+0 (secure 0 0 0) 0x7f8928022020 con 0x7f89381079c0 2024-03-04T09:21:47.078 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:46.308+0000 7f893cd18700 1 -- 172.21.15.52:0/2812261473 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f89380636f0 con 0x7f89381079c0 2024-03-04T09:21:47.078 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:46.308+0000 7f89277fe700 1 --2- 172.21.15.52:0/2812261473 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f8920038c40 0x7f892003b100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:21:47.078 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:46.308+0000 7f89277fe700 1 -- 172.21.15.52:0/2812261473 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f892804d1c0 con 0x7f89381079c0 2024-03-04T09:21:47.079 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:46.308+0000 7f8935d9b700 1 --2- 172.21.15.52:0/2812261473 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f8920038c40 0x7f892003b100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:21:47.079 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:46.309+0000 7f8935d9b700 1 --2- 172.21.15.52:0/2812261473 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f8920038c40 0x7f892003b100 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f892c006fd0 tx=0x7f892c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:21:47.079 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:46.313+0000 7f89277fe700 1 -- 172.21.15.52:0/2812261473 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f8928011e20 con 0x7f89381079c0 2024-03-04T09:21:47.079 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:46.515+0000 7f893cd18700 1 -- 172.21.15.52:0/2812261473 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command([{prefix=config set, name=osd_memory_target_autotune}] v 0) v1 -- 0x7f893819ef90 con 0x7f89381079c0 2024-03-04T09:21:47.079 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:46.516+0000 7f89277fe700 1 -- 172.21.15.52:0/2812261473 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_memory_target_autotune}]=0 v8)=0 v8) v1 ==== 127+0+0 (secure 0 0 0) 0x7f892800bc40 con 0x7f89381079c0 2024-03-04T09:21:47.079 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:46.518+0000 7f893cd18700 1 -- 172.21.15.52:0/2812261473 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f8920038c40 msgr2=0x7f892003b100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:21:47.079 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:46.518+0000 7f893cd18700 1 --2- 172.21.15.52:0/2812261473 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f8920038c40 0x7f892003b100 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f892c006fd0 tx=0x7f892c006e40 comp rx=0 tx=0).stop 2024-03-04T09:21:47.079 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:46.518+0000 7f893cd18700 1 -- 172.21.15.52:0/2812261473 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f89381079c0 msgr2=0x7f893819b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:21:47.079 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:46.518+0000 7f893cd18700 1 --2- 172.21.15.52:0/2812261473 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f89381079c0 0x7f893819b990 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f89280117e0 tx=0x7f8928011c10 comp rx=0 tx=0).stop 2024-03-04T09:21:47.079 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:46.518+0000 7f893cd18700 1 -- 172.21.15.52:0/2812261473 shutdown_connections 2024-03-04T09:21:47.079 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:46.518+0000 7f893cd18700 1 --2- 172.21.15.52:0/2812261473 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f8920038c40 0x7f892003b100 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:21:47.079 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:46.518+0000 7f893cd18700 1 --2- 172.21.15.52:0/2812261473 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f89381079c0 0x7f893819b990 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:21:47.079 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:46.518+0000 7f893cd18700 1 -- 172.21.15.52:0/2812261473 >> 172.21.15.52:0/2812261473 conn(0x7f8938103000 msgr2=0x7f8938103b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:21:47.080 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:46.518+0000 7f893cd18700 1 -- 172.21.15.52:0/2812261473 shutdown_connections 2024-03-04T09:21:47.080 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:46.518+0000 7f893cd18700 1 -- 172.21.15.52:0/2812261473 wait complete. 2024-03-04T09:21:47.334 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:21:47 smithi052 ceph-mon[32473]: mgrmap e13: x(active, since 6s) 2024-03-04T09:21:48.332 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:47.481+0000 7f1d0901d700 1 Processor -- start 2024-03-04T09:21:48.332 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:47.481+0000 7f1d0901d700 1 -- start start 2024-03-04T09:21:48.332 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:47.481+0000 7f1d0901d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1d041079c0 0x7f1d04107de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:21:48.332 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:47.481+0000 7f1d0901d700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d041083b0 con 0x7f1d041079c0 2024-03-04T09:21:48.332 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:47.481+0000 7f1d02d9d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1d041079c0 0x7f1d04107de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:21:48.333 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:47.481+0000 7f1d02d9d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1d041079c0 0x7f1d04107de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:56370/0 (socket says 172.21.15.52:56370) 2024-03-04T09:21:48.333 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:47.482+0000 7f1d02d9d700 1 -- 172.21.15.52:0/298598727 learned_addr learned my addr 172.21.15.52:0/298598727 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-04T09:21:48.333 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:47.482+0000 7f1d02d9d700 1 -- 172.21.15.52:0/298598727 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d04108bc0 con 0x7f1d041079c0 2024-03-04T09:21:48.333 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:47.482+0000 7f1d02d9d700 1 --2- 172.21.15.52:0/298598727 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1d041079c0 0x7f1d04107de0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f1cf40098a0 tx=0x7f1cf4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6eb170554bc7f301 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:21:48.333 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:47.482+0000 7f1d01d9b700 1 -- 172.21.15.52:0/298598727 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1cf4004160 con 0x7f1d041079c0 2024-03-04T09:21:48.333 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:47.482+0000 7f1d01d9b700 1 -- 172.21.15.52:0/298598727 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1cf4004300 con 0x7f1d041079c0 2024-03-04T09:21:48.333 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:47.482+0000 7f1d01d9b700 1 -- 172.21.15.52:0/298598727 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1cf4011b20 con 0x7f1d041079c0 2024-03-04T09:21:48.333 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:47.483+0000 7f1d0901d700 1 -- 172.21.15.52:0/298598727 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1d041079c0 msgr2=0x7f1d04107de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:21:48.333 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:47.483+0000 7f1d0901d700 1 --2- 172.21.15.52:0/298598727 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1d041079c0 0x7f1d04107de0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f1cf40098a0 tx=0x7f1cf4009bb0 comp rx=0 tx=0).stop 2024-03-04T09:21:48.333 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:47.483+0000 7f1d0901d700 1 -- 172.21.15.52:0/298598727 shutdown_connections 2024-03-04T09:21:48.333 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:47.483+0000 7f1d0901d700 1 --2- 172.21.15.52:0/298598727 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1d041079c0 0x7f1d04107de0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:21:48.333 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:47.483+0000 7f1d0901d700 1 -- 172.21.15.52:0/298598727 >> 172.21.15.52:0/298598727 conn(0x7f1d04103000 msgr2=0x7f1d04105420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:21:48.333 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:47.483+0000 7f1d0901d700 1 -- 172.21.15.52:0/298598727 shutdown_connections 2024-03-04T09:21:48.333 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:47.483+0000 7f1d0901d700 1 -- 172.21.15.52:0/298598727 wait complete. 2024-03-04T09:21:48.334 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:47.483+0000 7f1d0901d700 1 Processor -- start 2024-03-04T09:21:48.334 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:47.484+0000 7f1d0901d700 1 -- start start 2024-03-04T09:21:48.334 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:47.484+0000 7f1d0901d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1d041079c0 0x7f1d0419b990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:21:48.334 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:47.484+0000 7f1d0901d700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d0410bf70 con 0x7f1d041079c0 2024-03-04T09:21:48.334 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:47.484+0000 7f1d02d9d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1d041079c0 0x7f1d0419b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:21:48.334 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:47.484+0000 7f1d02d9d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1d041079c0 0x7f1d0419b990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:56372/0 (socket says 172.21.15.52:56372) 2024-03-04T09:21:48.334 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:47.484+0000 7f1d02d9d700 1 -- 172.21.15.52:0/3315360768 learned_addr learned my addr 172.21.15.52:0/3315360768 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-04T09:21:48.334 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:47.484+0000 7f1d02d9d700 1 -- 172.21.15.52:0/3315360768 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d0419bed0 con 0x7f1d041079c0 2024-03-04T09:21:48.334 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:47.484+0000 7f1d02d9d700 1 --2- 172.21.15.52:0/3315360768 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1d041079c0 0x7f1d0419b990 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f1cf40117e0 tx=0x7f1cf4011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:21:48.334 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:47.485+0000 7f1cfbfff700 1 -- 172.21.15.52:0/3315360768 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1cf4003bf0 con 0x7f1d041079c0 2024-03-04T09:21:48.334 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:47.485+0000 7f1cfbfff700 1 -- 172.21.15.52:0/3315360768 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1cf401a910 con 0x7f1d041079c0 2024-03-04T09:21:48.334 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:47.485+0000 7f1d0901d700 1 -- 172.21.15.52:0/3315360768 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d0419c160 con 0x7f1d041079c0 2024-03-04T09:21:48.334 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:47.485+0000 7f1cfbfff700 1 -- 172.21.15.52:0/3315360768 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1cf40249a0 con 0x7f1d041079c0 2024-03-04T09:21:48.334 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:47.485+0000 7f1d0901d700 1 -- 172.21.15.52:0/3315360768 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d0419c5c0 con 0x7f1d041079c0 2024-03-04T09:21:48.335 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:47.485+0000 7f1cfbfff700 1 -- 172.21.15.52:0/3315360768 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 13) v1 ==== 45984+0+0 (secure 0 0 0) 0x7f1cf4022020 con 0x7f1d041079c0 2024-03-04T09:21:48.335 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:47.486+0000 7f1cfbfff700 1 --2- 172.21.15.52:0/3315360768 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f1cf0038ca0 0x7f1cf003b160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:21:48.335 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:47.486+0000 7f1cfbfff700 1 -- 172.21.15.52:0/3315360768 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f1cf404e200 con 0x7f1d041079c0 2024-03-04T09:21:48.335 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:47.486+0000 7f1d0259c700 1 --2- 172.21.15.52:0/3315360768 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f1cf0038ca0 0x7f1cf003b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:21:48.335 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:47.486+0000 7f1d0901d700 1 -- 172.21.15.52:0/3315360768 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d040636f0 con 0x7f1d041079c0 2024-03-04T09:21:48.335 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:47.486+0000 7f1d0259c700 1 --2- 172.21.15.52:0/3315360768 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f1cf0038ca0 0x7f1cf003b160 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f1cec006fd0 tx=0x7f1cec006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:21:48.335 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:47.491+0000 7f1cfbfff700 1 -- 172.21.15.52:0/3315360768 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f1cf4011e20 con 0x7f1d041079c0 2024-03-04T09:21:48.335 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:47.805+0000 7f1d0901d700 1 -- 172.21.15.52:0/3315360768 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) v1 -- 0x7f1d0419eef0 con 0x7f1d041079c0 2024-03-04T09:21:48.335 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:47.807+0000 7f1cfbfff700 1 -- 172.21.15.52:0/3315360768 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v27)=0 set mgr/dashboard/cluster/status v27) v1 ==== 153+0+0 (secure 0 0 0) 0x7f1cf4004300 con 0x7f1d041079c0 2024-03-04T09:21:48.335 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:47.809+0000 7f1d0901d700 1 -- 172.21.15.52:0/3315360768 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f1cf0038ca0 msgr2=0x7f1cf003b160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:21:48.335 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:47.809+0000 7f1d0901d700 1 --2- 172.21.15.52:0/3315360768 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f1cf0038ca0 0x7f1cf003b160 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f1cec006fd0 tx=0x7f1cec006e40 comp rx=0 tx=0).stop 2024-03-04T09:21:48.335 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:47.809+0000 7f1d0901d700 1 -- 172.21.15.52:0/3315360768 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1d041079c0 msgr2=0x7f1d0419b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:21:48.335 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:47.809+0000 7f1d0901d700 1 --2- 172.21.15.52:0/3315360768 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1d041079c0 0x7f1d0419b990 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f1cf40117e0 tx=0x7f1cf4011c10 comp rx=0 tx=0).stop 2024-03-04T09:21:48.335 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:47.809+0000 7f1d0901d700 1 -- 172.21.15.52:0/3315360768 shutdown_connections 2024-03-04T09:21:48.336 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:47.809+0000 7f1d0901d700 1 --2- 172.21.15.52:0/3315360768 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f1cf0038ca0 0x7f1cf003b160 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:21:48.336 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:47.809+0000 7f1d0901d700 1 --2- 172.21.15.52:0/3315360768 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1d041079c0 0x7f1d0419b990 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:21:48.336 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:47.809+0000 7f1d0901d700 1 -- 172.21.15.52:0/3315360768 >> 172.21.15.52:0/3315360768 conn(0x7f1d04103000 msgr2=0x7f1d04103b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:21:48.336 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:47.810+0000 7f1d0901d700 1 -- 172.21.15.52:0/3315360768 shutdown_connections 2024-03-04T09:21:48.336 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-03-04T09:21:47.810+0000 7f1d0901d700 1 -- 172.21.15.52:0/3315360768 wait complete. 2024-03-04T09:21:48.336 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-03-04T09:21:48.336 INFO:teuthology.orchestra.run.smithi052.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-03-04T09:21:48.336 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-03-04T09:21:48.336 INFO:teuthology.orchestra.run.smithi052.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid 4e059f08-da08-11ee-95c6-87774f69a715 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-03-04T09:21:48.336 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-03-04T09:21:48.336 INFO:teuthology.orchestra.run.smithi052.stdout:Or, if you are only running a single cluster on this host: 2024-03-04T09:21:48.336 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-03-04T09:21:48.336 INFO:teuthology.orchestra.run.smithi052.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-03-04T09:21:48.336 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-03-04T09:21:48.337 INFO:teuthology.orchestra.run.smithi052.stdout:Please consider enabling telemetry to help improve Ceph: 2024-03-04T09:21:48.337 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-03-04T09:21:48.337 INFO:teuthology.orchestra.run.smithi052.stdout: ceph telemetry on 2024-03-04T09:21:48.337 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-03-04T09:21:48.337 INFO:teuthology.orchestra.run.smithi052.stdout:For more information see: 2024-03-04T09:21:48.337 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-03-04T09:21:48.337 INFO:teuthology.orchestra.run.smithi052.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-03-04T09:21:48.337 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-03-04T09:21:48.337 INFO:teuthology.orchestra.run.smithi052.stdout:Bootstrap complete. 2024-03-04T09:21:48.369 INFO:tasks.cephadm:Fetching config... 2024-03-04T09:21:48.370 DEBUG:teuthology.orchestra.run.smithi052:> set -ex 2024-03-04T09:21:48.370 DEBUG:teuthology.orchestra.run.smithi052:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-03-04T09:21:48.385 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-03-04T09:21:48.385 DEBUG:teuthology.orchestra.run.smithi052:> set -ex 2024-03-04T09:21:48.385 DEBUG:teuthology.orchestra.run.smithi052:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-03-04T09:21:48.440 INFO:tasks.cephadm:Fetching mon keyring... 2024-03-04T09:21:48.440 DEBUG:teuthology.orchestra.run.smithi052:> set -ex 2024-03-04T09:21:48.440 DEBUG:teuthology.orchestra.run.smithi052:> sudo dd if=/var/lib/ceph/4e059f08-da08-11ee-95c6-87774f69a715/mon.a/keyring of=/dev/stdout 2024-03-04T09:21:48.505 INFO:tasks.cephadm:Fetching pub ssh key... 2024-03-04T09:21:48.506 DEBUG:teuthology.orchestra.run.smithi052:> set -ex 2024-03-04T09:21:48.506 DEBUG:teuthology.orchestra.run.smithi052:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-03-04T09:21:48.561 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-03-04T09:21:48.561 DEBUG:teuthology.orchestra.run.smithi052:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDBw1lK2UDv8zi6usYCcvXwie9Yt7cWtkCfM0J6TUU8MUnE1I4NluKTDKCK6OSPeKlnIWvWxKZaTWli9wX+Gc4O86yUw5R26W3H+JqnMXDUSDgSM1mDtYn39KpkkQQs3dVXpJODW3oZEbqj7bubB/EjiCmDzd7k4mBV18dnDLJNsGI4yFojydHD2VzIMg3Fb3ZvtwQk3o8OOet4hobp1T2Im0GRJIzmLVSoZ9IsgWxaYlvRbt5qmnAN14dZJ/psDCz2noz5JkqklqWl/ln8xS8A4nRzZs0bNbn5kcbBWt+sNiQIep3n8A+E6AW2CpX+OB7lcenUaNpUXbwL5/mF79hBfseOiqA59CAMntwivu5HYhDW1bPGac8UHGGq+P3E9OECF5p5t0q3Zqr8CcWiPU7iqUFZA5kAwiRIVpcnwhrn1h54RNf2AyTShmF8NTk/Y7CIrFxmnww2eHC5q62J4d3rfAyjYpwzvsZGu5Rua/WAx9LyFffhLcjdU1NvX26s1v8= ceph-4e059f08-da08-11ee-95c6-87774f69a715' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-03-04T09:21:48.645 INFO:teuthology.orchestra.run.smithi052.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDBw1lK2UDv8zi6usYCcvXwie9Yt7cWtkCfM0J6TUU8MUnE1I4NluKTDKCK6OSPeKlnIWvWxKZaTWli9wX+Gc4O86yUw5R26W3H+JqnMXDUSDgSM1mDtYn39KpkkQQs3dVXpJODW3oZEbqj7bubB/EjiCmDzd7k4mBV18dnDLJNsGI4yFojydHD2VzIMg3Fb3ZvtwQk3o8OOet4hobp1T2Im0GRJIzmLVSoZ9IsgWxaYlvRbt5qmnAN14dZJ/psDCz2noz5JkqklqWl/ln8xS8A4nRzZs0bNbn5kcbBWt+sNiQIep3n8A+E6AW2CpX+OB7lcenUaNpUXbwL5/mF79hBfseOiqA59CAMntwivu5HYhDW1bPGac8UHGGq+P3E9OECF5p5t0q3Zqr8CcWiPU7iqUFZA5kAwiRIVpcnwhrn1h54RNf2AyTShmF8NTk/Y7CIrFxmnww2eHC5q62J4d3rfAyjYpwzvsZGu5Rua/WAx9LyFffhLcjdU1NvX26s1v8= ceph-4e059f08-da08-11ee-95c6-87774f69a715 2024-03-04T09:21:48.658 DEBUG:teuthology.orchestra.run.smithi106:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDBw1lK2UDv8zi6usYCcvXwie9Yt7cWtkCfM0J6TUU8MUnE1I4NluKTDKCK6OSPeKlnIWvWxKZaTWli9wX+Gc4O86yUw5R26W3H+JqnMXDUSDgSM1mDtYn39KpkkQQs3dVXpJODW3oZEbqj7bubB/EjiCmDzd7k4mBV18dnDLJNsGI4yFojydHD2VzIMg3Fb3ZvtwQk3o8OOet4hobp1T2Im0GRJIzmLVSoZ9IsgWxaYlvRbt5qmnAN14dZJ/psDCz2noz5JkqklqWl/ln8xS8A4nRzZs0bNbn5kcbBWt+sNiQIep3n8A+E6AW2CpX+OB7lcenUaNpUXbwL5/mF79hBfseOiqA59CAMntwivu5HYhDW1bPGac8UHGGq+P3E9OECF5p5t0q3Zqr8CcWiPU7iqUFZA5kAwiRIVpcnwhrn1h54RNf2AyTShmF8NTk/Y7CIrFxmnww2eHC5q62J4d3rfAyjYpwzvsZGu5Rua/WAx9LyFffhLcjdU1NvX26s1v8= ceph-4e059f08-da08-11ee-95c6-87774f69a715' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-03-04T09:21:48.703 INFO:teuthology.orchestra.run.smithi106.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDBw1lK2UDv8zi6usYCcvXwie9Yt7cWtkCfM0J6TUU8MUnE1I4NluKTDKCK6OSPeKlnIWvWxKZaTWli9wX+Gc4O86yUw5R26W3H+JqnMXDUSDgSM1mDtYn39KpkkQQs3dVXpJODW3oZEbqj7bubB/EjiCmDzd7k4mBV18dnDLJNsGI4yFojydHD2VzIMg3Fb3ZvtwQk3o8OOet4hobp1T2Im0GRJIzmLVSoZ9IsgWxaYlvRbt5qmnAN14dZJ/psDCz2noz5JkqklqWl/ln8xS8A4nRzZs0bNbn5kcbBWt+sNiQIep3n8A+E6AW2CpX+OB7lcenUaNpUXbwL5/mF79hBfseOiqA59CAMntwivu5HYhDW1bPGac8UHGGq+P3E9OECF5p5t0q3Zqr8CcWiPU7iqUFZA5kAwiRIVpcnwhrn1h54RNf2AyTShmF8NTk/Y7CIrFxmnww2eHC5q62J4d3rfAyjYpwzvsZGu5Rua/WAx9LyFffhLcjdU1NvX26s1v8= ceph-4e059f08-da08-11ee-95c6-87774f69a715 2024-03-04T09:21:48.718 DEBUG:teuthology.orchestra.run.smithi191:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDBw1lK2UDv8zi6usYCcvXwie9Yt7cWtkCfM0J6TUU8MUnE1I4NluKTDKCK6OSPeKlnIWvWxKZaTWli9wX+Gc4O86yUw5R26W3H+JqnMXDUSDgSM1mDtYn39KpkkQQs3dVXpJODW3oZEbqj7bubB/EjiCmDzd7k4mBV18dnDLJNsGI4yFojydHD2VzIMg3Fb3ZvtwQk3o8OOet4hobp1T2Im0GRJIzmLVSoZ9IsgWxaYlvRbt5qmnAN14dZJ/psDCz2noz5JkqklqWl/ln8xS8A4nRzZs0bNbn5kcbBWt+sNiQIep3n8A+E6AW2CpX+OB7lcenUaNpUXbwL5/mF79hBfseOiqA59CAMntwivu5HYhDW1bPGac8UHGGq+P3E9OECF5p5t0q3Zqr8CcWiPU7iqUFZA5kAwiRIVpcnwhrn1h54RNf2AyTShmF8NTk/Y7CIrFxmnww2eHC5q62J4d3rfAyjYpwzvsZGu5Rua/WAx9LyFffhLcjdU1NvX26s1v8= ceph-4e059f08-da08-11ee-95c6-87774f69a715' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-03-04T09:21:48.763 INFO:teuthology.orchestra.run.smithi191.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDBw1lK2UDv8zi6usYCcvXwie9Yt7cWtkCfM0J6TUU8MUnE1I4NluKTDKCK6OSPeKlnIWvWxKZaTWli9wX+Gc4O86yUw5R26W3H+JqnMXDUSDgSM1mDtYn39KpkkQQs3dVXpJODW3oZEbqj7bubB/EjiCmDzd7k4mBV18dnDLJNsGI4yFojydHD2VzIMg3Fb3ZvtwQk3o8OOet4hobp1T2Im0GRJIzmLVSoZ9IsgWxaYlvRbt5qmnAN14dZJ/psDCz2noz5JkqklqWl/ln8xS8A4nRzZs0bNbn5kcbBWt+sNiQIep3n8A+E6AW2CpX+OB7lcenUaNpUXbwL5/mF79hBfseOiqA59CAMntwivu5HYhDW1bPGac8UHGGq+P3E9OECF5p5t0q3Zqr8CcWiPU7iqUFZA5kAwiRIVpcnwhrn1h54RNf2AyTShmF8NTk/Y7CIrFxmnww2eHC5q62J4d3rfAyjYpwzvsZGu5Rua/WAx9LyFffhLcjdU1NvX26s1v8= ceph-4e059f08-da08-11ee-95c6-87774f69a715 2024-03-04T09:21:48.778 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4e059f08-da08-11ee-95c6-87774f69a715 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-03-04T09:21:49.019 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/4e059f08-da08-11ee-95c6-87774f69a715/mon.a/config 2024-03-04T09:21:49.085 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:21:48 smithi052 ceph-mon[32473]: from='client.? 172.21.15.52:0/3315360768' entity='client.admin' 2024-03-04T09:21:51.078 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:21:51.077+0000 7f82df176700 1 -- 172.21.15.52:0/1942569292 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f82d8101b10 msgr2=0x7f82d8101f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:21:51.079 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:21:51.077+0000 7f82df176700 1 --2- 172.21.15.52:0/1942569292 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f82d8101b10 0x7f82d8101f30 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f82cc009910 tx=0x7f82cc009c20 comp rx=0 tx=0).stop 2024-03-04T09:21:51.079 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:21:51.077+0000 7f82df176700 1 -- 172.21.15.52:0/1942569292 shutdown_connections 2024-03-04T09:21:51.079 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:21:51.077+0000 7f82df176700 1 --2- 172.21.15.52:0/1942569292 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f82d8101b10 0x7f82d8101f30 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:21:51.079 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:21:51.077+0000 7f82df176700 1 -- 172.21.15.52:0/1942569292 >> 172.21.15.52:0/1942569292 conn(0x7f82d80fd0b0 msgr2=0x7f82d80ff4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:21:51.079 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:21:51.077+0000 7f82df176700 1 -- 172.21.15.52:0/1942569292 shutdown_connections 2024-03-04T09:21:51.080 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:21:51.077+0000 7f82df176700 1 -- 172.21.15.52:0/1942569292 wait complete. 2024-03-04T09:21:51.080 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:21:51.078+0000 7f82df176700 1 Processor -- start 2024-03-04T09:21:51.080 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:21:51.078+0000 7f82df176700 1 -- start start 2024-03-04T09:21:51.080 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:21:51.078+0000 7f82df176700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f82d8101b10 0x7f82d8197220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:21:51.080 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:21:51.078+0000 7f82df176700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82d8107ba0 con 0x7f82d8101b10 2024-03-04T09:21:51.080 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:21:51.078+0000 7f82dcf12700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f82d8101b10 0x7f82d8197220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:21:51.081 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:21:51.078+0000 7f82dcf12700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f82d8101b10 0x7f82d8197220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:56392/0 (socket says 172.21.15.52:56392) 2024-03-04T09:21:51.081 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:21:51.078+0000 7f82dcf12700 1 -- 172.21.15.52:0/2554137761 learned_addr learned my addr 172.21.15.52:0/2554137761 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-04T09:21:51.081 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:21:51.079+0000 7f82dcf12700 1 -- 172.21.15.52:0/2554137761 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82d8197760 con 0x7f82d8101b10 2024-03-04T09:21:51.081 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:21:51.079+0000 7f82dcf12700 1 --2- 172.21.15.52:0/2554137761 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f82d8101b10 0x7f82d8197220 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f82cc006010 tx=0x7f82cc005050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:21:51.081 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:21:51.079+0000 7f82d5ffb700 1 -- 172.21.15.52:0/2554137761 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f82cc027070 con 0x7f82d8101b10 2024-03-04T09:21:51.081 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:21:51.079+0000 7f82d5ffb700 1 -- 172.21.15.52:0/2554137761 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f82cc00be40 con 0x7f82d8101b10 2024-03-04T09:21:51.082 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:21:51.079+0000 7f82d5ffb700 1 -- 172.21.15.52:0/2554137761 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f82cc00f460 con 0x7f82d8101b10 2024-03-04T09:21:51.082 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:21:51.079+0000 7f82df176700 1 -- 172.21.15.52:0/2554137761 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82d81979f0 con 0x7f82d8101b10 2024-03-04T09:21:51.082 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:21:51.079+0000 7f82df176700 1 -- 172.21.15.52:0/2554137761 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82d819a6e0 con 0x7f82d8101b10 2024-03-04T09:21:51.082 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:21:51.080+0000 7f82d5ffb700 1 -- 172.21.15.52:0/2554137761 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 13) v1 ==== 45984+0+0 (secure 0 0 0) 0x7f82cc02a070 con 0x7f82d8101b10 2024-03-04T09:21:51.082 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:21:51.080+0000 7f82df176700 1 -- 172.21.15.52:0/2554137761 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f82d8108320 con 0x7f82d8101b10 2024-03-04T09:21:51.083 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:21:51.080+0000 7f82d5ffb700 1 --2- 172.21.15.52:0/2554137761 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f82c8038ca0 0x7f82c803b160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:21:51.083 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:21:51.080+0000 7f82d5ffb700 1 -- 172.21.15.52:0/2554137761 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f82cc01e030 con 0x7f82d8101b10 2024-03-04T09:21:51.083 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:21:51.080+0000 7f82d7fff700 1 --2- 172.21.15.52:0/2554137761 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f82c8038ca0 0x7f82c803b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:21:51.083 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:21:51.081+0000 7f82d7fff700 1 --2- 172.21.15.52:0/2554137761 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f82c8038ca0 0x7f82c803b160 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f82c4006fd0 tx=0x7f82c4006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:21:51.084 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:21:51.085+0000 7f82d5ffb700 1 -- 172.21.15.52:0/2554137761 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f82cc013070 con 0x7f82d8101b10 2024-03-04T09:21:51.286 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:21:51.289+0000 7f82df176700 1 -- 172.21.15.52:0/2554137761 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) v1 -- 0x7f82d802dac0 con 0x7f82d8101b10 2024-03-04T09:21:51.288 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:21:51.291+0000 7f82d5ffb700 1 -- 172.21.15.52:0/2554137761 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) v1 ==== 125+0+0 (secure 0 0 0) 0x7f82cc031020 con 0x7f82d8101b10 2024-03-04T09:21:51.290 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:21:51.293+0000 7f82df176700 1 -- 172.21.15.52:0/2554137761 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f82c8038ca0 msgr2=0x7f82c803b160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:21:51.291 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:21:51.293+0000 7f82df176700 1 --2- 172.21.15.52:0/2554137761 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f82c8038ca0 0x7f82c803b160 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f82c4006fd0 tx=0x7f82c4006e40 comp rx=0 tx=0).stop 2024-03-04T09:21:51.291 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:21:51.293+0000 7f82df176700 1 -- 172.21.15.52:0/2554137761 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f82d8101b10 msgr2=0x7f82d8197220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:21:51.291 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:21:51.293+0000 7f82df176700 1 --2- 172.21.15.52:0/2554137761 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f82d8101b10 0x7f82d8197220 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f82cc006010 tx=0x7f82cc005050 comp rx=0 tx=0).stop 2024-03-04T09:21:51.291 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:21:51.293+0000 7f82df176700 1 -- 172.21.15.52:0/2554137761 shutdown_connections 2024-03-04T09:21:51.291 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:21:51.293+0000 7f82df176700 1 --2- 172.21.15.52:0/2554137761 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f82c8038ca0 0x7f82c803b160 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:21:51.291 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:21:51.293+0000 7f82df176700 1 --2- 172.21.15.52:0/2554137761 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f82d8101b10 0x7f82d8197220 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:21:51.291 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:21:51.293+0000 7f82df176700 1 -- 172.21.15.52:0/2554137761 >> 172.21.15.52:0/2554137761 conn(0x7f82d80fd0b0 msgr2=0x7f82d80fdba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:21:51.291 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:21:51.294+0000 7f82df176700 1 -- 172.21.15.52:0/2554137761 shutdown_connections 2024-03-04T09:21:51.291 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:21:51.294+0000 7f82df176700 1 -- 172.21.15.52:0/2554137761 wait complete. 2024-03-04T09:21:51.580 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:21:51 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:21:51.580 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:21:51 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:21:51.580 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:21:51 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi052", "name": "osd_memory_target"}]: dispatch 2024-03-04T09:21:51.580 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:21:51 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:21:51.580 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:21:51 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T09:21:51.580 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:21:51 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:21:51.841 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-03-04T09:21:51.842 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4e059f08-da08-11ee-95c6-87774f69a715 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-03-04T09:21:52.081 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/4e059f08-da08-11ee-95c6-87774f69a715/mon.a/config 2024-03-04T09:21:52.584 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:21:52 smithi052 ceph-mon[32473]: from='client.? 172.21.15.52:0/2554137761' entity='client.admin' 2024-03-04T09:21:52.585 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:21:52 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T09:21:52.585 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:21:52 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:21:53.541 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:21:53.542+0000 7fb41fc2f700 1 -- 172.21.15.52:0/2099359303 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb4180ffdf0 msgr2=0x7fb418100210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:21:53.541 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:21:53.542+0000 7fb41fc2f700 1 --2- 172.21.15.52:0/2099359303 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb4180ffdf0 0x7fb418100210 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fb414009910 tx=0x7fb414009c20 comp rx=0 tx=0).stop 2024-03-04T09:21:53.541 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:21:53.542+0000 7fb41fc2f700 1 -- 172.21.15.52:0/2099359303 shutdown_connections 2024-03-04T09:21:53.541 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:21:53.542+0000 7fb41fc2f700 1 --2- 172.21.15.52:0/2099359303 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb4180ffdf0 0x7fb418100210 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:21:53.541 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:21:53.542+0000 7fb41fc2f700 1 -- 172.21.15.52:0/2099359303 >> 172.21.15.52:0/2099359303 conn(0x7fb4180fb770 msgr2=0x7fb4180fdb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:21:53.541 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:21:53.542+0000 7fb41fc2f700 1 -- 172.21.15.52:0/2099359303 shutdown_connections 2024-03-04T09:21:53.541 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:21:53.543+0000 7fb41fc2f700 1 -- 172.21.15.52:0/2099359303 wait complete. 2024-03-04T09:21:53.541 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:21:53.543+0000 7fb41fc2f700 1 Processor -- start 2024-03-04T09:21:53.542 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:21:53.543+0000 7fb41fc2f700 1 -- start start 2024-03-04T09:21:53.542 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:21:53.543+0000 7fb41fc2f700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb4180ffdf0 0x7fb418197260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:21:53.542 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:21:53.543+0000 7fb41fc2f700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb418107bc0 con 0x7fb4180ffdf0 2024-03-04T09:21:53.542 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:21:53.543+0000 7fb41d9cb700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb4180ffdf0 0x7fb418197260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:21:53.542 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:21:53.543+0000 7fb41d9cb700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb4180ffdf0 0x7fb418197260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:56414/0 (socket says 172.21.15.52:56414) 2024-03-04T09:21:53.542 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:21:53.543+0000 7fb41d9cb700 1 -- 172.21.15.52:0/2175021805 learned_addr learned my addr 172.21.15.52:0/2175021805 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-04T09:21:53.542 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:21:53.544+0000 7fb41d9cb700 1 -- 172.21.15.52:0/2175021805 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4181977a0 con 0x7fb4180ffdf0 2024-03-04T09:21:53.542 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:21:53.544+0000 7fb41d9cb700 1 --2- 172.21.15.52:0/2175021805 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb4180ffdf0 0x7fb418197260 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fb414006010 tx=0x7fb414005050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:21:53.542 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:21:53.544+0000 7fb40effd700 1 -- 172.21.15.52:0/2175021805 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb414027070 con 0x7fb4180ffdf0 2024-03-04T09:21:53.542 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:21:53.544+0000 7fb40effd700 1 -- 172.21.15.52:0/2175021805 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb41400be40 con 0x7fb4180ffdf0 2024-03-04T09:21:53.542 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:21:53.544+0000 7fb41fc2f700 1 -- 172.21.15.52:0/2175021805 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb418197a30 con 0x7fb4180ffdf0 2024-03-04T09:21:53.542 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:21:53.544+0000 7fb40effd700 1 -- 172.21.15.52:0/2175021805 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb41400f460 con 0x7fb4180ffdf0 2024-03-04T09:21:53.542 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:21:53.544+0000 7fb41fc2f700 1 -- 172.21.15.52:0/2175021805 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb41819a720 con 0x7fb4180ffdf0 2024-03-04T09:21:53.543 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:21:53.545+0000 7fb40effd700 1 -- 172.21.15.52:0/2175021805 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 13) v1 ==== 45984+0+0 (secure 0 0 0) 0x7fb41402a070 con 0x7fb4180ffdf0 2024-03-04T09:21:53.543 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:21:53.545+0000 7fb41fc2f700 1 -- 172.21.15.52:0/2175021805 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb418108340 con 0x7fb4180ffdf0 2024-03-04T09:21:53.543 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:21:53.545+0000 7fb40effd700 1 --2- 172.21.15.52:0/2175021805 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7fb404038ca0 0x7fb40403b160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:21:53.544 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:21:53.545+0000 7fb40effd700 1 -- 172.21.15.52:0/2175021805 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fb41401e030 con 0x7fb4180ffdf0 2024-03-04T09:21:53.544 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:21:53.545+0000 7fb41d1ca700 1 --2- 172.21.15.52:0/2175021805 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7fb404038ca0 0x7fb40403b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:21:53.544 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:21:53.546+0000 7fb41d1ca700 1 --2- 172.21.15.52:0/2175021805 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7fb404038ca0 0x7fb40403b160 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fb408006fd0 tx=0x7fb408006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:21:53.547 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:21:53.550+0000 7fb40effd700 1 -- 172.21.15.52:0/2175021805 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fb414013070 con 0x7fb4180ffdf0 2024-03-04T09:21:53.755 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:21:53.758+0000 7fb41fc2f700 1 -- 172.21.15.52:0/2175021805 --> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) v1 -- 0x7fb41802dac0 con 0x7fb404038ca0 2024-03-04T09:21:53.758 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:21:53.762+0000 7fb40effd700 1 -- 172.21.15.52:0/2175021805 <== mgr.14152 v2:172.21.15.52:6800/3036033923 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fb41802dac0 con 0x7fb404038ca0 2024-03-04T09:21:53.761 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:21:53.764+0000 7fb41fc2f700 1 -- 172.21.15.52:0/2175021805 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7fb404038ca0 msgr2=0x7fb40403b160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:21:53.761 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:21:53.764+0000 7fb41fc2f700 1 --2- 172.21.15.52:0/2175021805 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7fb404038ca0 0x7fb40403b160 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fb408006fd0 tx=0x7fb408006e40 comp rx=0 tx=0).stop 2024-03-04T09:21:53.761 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:21:53.764+0000 7fb41fc2f700 1 -- 172.21.15.52:0/2175021805 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb4180ffdf0 msgr2=0x7fb418197260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:21:53.761 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:21:53.764+0000 7fb41fc2f700 1 --2- 172.21.15.52:0/2175021805 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb4180ffdf0 0x7fb418197260 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fb414006010 tx=0x7fb414005050 comp rx=0 tx=0).stop 2024-03-04T09:21:53.761 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:21:53.764+0000 7fb41fc2f700 1 -- 172.21.15.52:0/2175021805 shutdown_connections 2024-03-04T09:21:53.761 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:21:53.764+0000 7fb41fc2f700 1 --2- 172.21.15.52:0/2175021805 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7fb404038ca0 0x7fb40403b160 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:21:53.761 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:21:53.764+0000 7fb41fc2f700 1 --2- 172.21.15.52:0/2175021805 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb4180ffdf0 0x7fb418197260 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:21:53.761 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:21:53.764+0000 7fb41fc2f700 1 -- 172.21.15.52:0/2175021805 >> 172.21.15.52:0/2175021805 conn(0x7fb4180fb770 msgr2=0x7fb4180fc210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:21:53.761 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:21:53.764+0000 7fb41fc2f700 1 -- 172.21.15.52:0/2175021805 shutdown_connections 2024-03-04T09:21:53.761 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:21:53.764+0000 7fb41fc2f700 1 -- 172.21.15.52:0/2175021805 wait complete. 2024-03-04T09:21:54.501 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi106 2024-03-04T09:21:54.501 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2024-03-04T09:21:54.501 DEBUG:teuthology.orchestra.run.smithi106:> dd of=/etc/ceph/ceph.conf 2024-03-04T09:21:54.520 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2024-03-04T09:21:54.520 DEBUG:teuthology.orchestra.run.smithi106:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-03-04T09:21:54.577 INFO:tasks.cephadm:Adding host smithi106 to orchestrator... 2024-03-04T09:21:54.577 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4e059f08-da08-11ee-95c6-87774f69a715 -- ceph orch host add smithi106 2024-03-04T09:21:54.814 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/4e059f08-da08-11ee-95c6-87774f69a715/mon.a/config 2024-03-04T09:21:55.015 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:21:54 smithi052 ceph-mon[32473]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T09:21:55.015 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:21:54 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:21:55.015 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:21:54 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T09:21:55.015 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:21:54 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T09:21:55.015 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:21:54 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T09:21:55.015 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:21:54 smithi052 ceph-mon[32473]: Updating smithi052:/etc/ceph/ceph.conf 2024-03-04T09:21:55.016 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:21:54 smithi052 ceph-mon[32473]: Updating smithi052:/var/lib/ceph/4e059f08-da08-11ee-95c6-87774f69a715/config/ceph.conf 2024-03-04T09:21:55.016 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:21:54 smithi052 ceph-mon[32473]: Updating smithi052:/etc/ceph/ceph.client.admin.keyring 2024-03-04T09:21:55.016 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:21:54 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:21:55.016 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:21:54 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:21:55.016 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:21:54 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:21:55.949 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:21:55 smithi052 ceph-mon[32473]: Updating smithi052:/var/lib/ceph/4e059f08-da08-11ee-95c6-87774f69a715/config/ceph.client.admin.keyring 2024-03-04T09:21:56.161 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:21:56.164+0000 7efc1cbb5700 1 -- 172.21.15.52:0/2587916182 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efc18106980 msgr2=0x7efc18106d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:21:56.161 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:21:56.164+0000 7efc1cbb5700 1 --2- 172.21.15.52:0/2587916182 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efc18106980 0x7efc18106d60 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7efc08009910 tx=0x7efc08009c20 comp rx=0 tx=0).stop 2024-03-04T09:21:56.161 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:21:56.164+0000 7efc1cbb5700 1 -- 172.21.15.52:0/2587916182 shutdown_connections 2024-03-04T09:21:56.161 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:21:56.164+0000 7efc1cbb5700 1 --2- 172.21.15.52:0/2587916182 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efc18106980 0x7efc18106d60 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:21:56.161 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:21:56.164+0000 7efc1cbb5700 1 -- 172.21.15.52:0/2587916182 >> 172.21.15.52:0/2587916182 conn(0x7efc180fc470 msgr2=0x7efc180fe890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:21:56.162 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:21:56.164+0000 7efc1cbb5700 1 -- 172.21.15.52:0/2587916182 shutdown_connections 2024-03-04T09:21:56.162 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:21:56.164+0000 7efc1cbb5700 1 -- 172.21.15.52:0/2587916182 wait complete. 2024-03-04T09:21:56.164 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:21:56.165+0000 7efc1cbb5700 1 Processor -- start 2024-03-04T09:21:56.164 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:21:56.165+0000 7efc1cbb5700 1 -- start start 2024-03-04T09:21:56.164 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:21:56.165+0000 7efc1cbb5700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efc18106980 0x7efc18198440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:21:56.164 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:21:56.165+0000 7efc1cbb5700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc1810afc0 con 0x7efc18106980 2024-03-04T09:21:56.164 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:21:56.165+0000 7efc1659c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efc18106980 0x7efc18198440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:21:56.164 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:21:56.165+0000 7efc1659c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efc18106980 0x7efc18198440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:41706/0 (socket says 172.21.15.52:41706) 2024-03-04T09:21:56.164 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:21:56.165+0000 7efc1659c700 1 -- 172.21.15.52:0/167394041 learned_addr learned my addr 172.21.15.52:0/167394041 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-04T09:21:56.164 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:21:56.165+0000 7efc1659c700 1 -- 172.21.15.52:0/167394041 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc18198980 con 0x7efc18106980 2024-03-04T09:21:56.164 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:21:56.166+0000 7efc1659c700 1 --2- 172.21.15.52:0/167394041 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efc18106980 0x7efc18198440 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7efc08006010 tx=0x7efc08005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:21:56.165 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:21:56.166+0000 7efc077fe700 1 -- 172.21.15.52:0/167394041 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7efc08026070 con 0x7efc18106980 2024-03-04T09:21:56.165 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:21:56.166+0000 7efc077fe700 1 -- 172.21.15.52:0/167394041 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7efc0800bbd0 con 0x7efc18106980 2024-03-04T09:21:56.165 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:21:56.166+0000 7efc077fe700 1 -- 172.21.15.52:0/167394041 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7efc08003910 con 0x7efc18106980 2024-03-04T09:21:56.165 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:21:56.166+0000 7efc1cbb5700 1 -- 172.21.15.52:0/167394041 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc18198c70 con 0x7efc18106980 2024-03-04T09:21:56.165 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:21:56.166+0000 7efc1cbb5700 1 -- 172.21.15.52:0/167394041 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc18199130 con 0x7efc18106980 2024-03-04T09:21:56.165 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:21:56.167+0000 7efc077fe700 1 -- 172.21.15.52:0/167394041 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 13) v1 ==== 45984+0+0 (secure 0 0 0) 0x7efc08029070 con 0x7efc18106980 2024-03-04T09:21:56.165 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:21:56.167+0000 7efc077fe700 1 --2- 172.21.15.52:0/167394041 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7efc00038ca0 0x7efc0003b160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:21:56.165 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:21:56.167+0000 7efc077fe700 1 -- 172.21.15.52:0/167394041 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7efc0801e070 con 0x7efc18106980 2024-03-04T09:21:56.165 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:21:56.167+0000 7efc1cbb5700 1 -- 172.21.15.52:0/167394041 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efc181020c0 con 0x7efc18106980 2024-03-04T09:21:56.165 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:21:56.167+0000 7efc15d9b700 1 --2- 172.21.15.52:0/167394041 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7efc00038ca0 0x7efc0003b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:21:56.166 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:21:56.167+0000 7efc15d9b700 1 --2- 172.21.15.52:0/167394041 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7efc00038ca0 0x7efc0003b160 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7efc0c006fd0 tx=0x7efc0c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:21:56.168 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:21:56.172+0000 7efc077fe700 1 -- 172.21.15.52:0/167394041 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7efc08030070 con 0x7efc18106980 2024-03-04T09:21:56.376 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:21:56.379+0000 7efc1cbb5700 1 -- 172.21.15.52:0/167394041 --> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi106", "target": ["mon-mgr", ""]}) v1 -- 0x7efc180636f0 con 0x7efc00038ca0 2024-03-04T09:21:57.834 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:21:57 smithi052 ceph-mon[32473]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi106", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T09:21:57.835 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:21:57 smithi052 ceph-mon[32473]: Deploying cephadm binary to smithi106 2024-03-04T09:22:00.639 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:22:00.642+0000 7efc077fe700 1 -- 172.21.15.52:0/167394041 <== mgr.14152 v2:172.21.15.52:6800/3036033923 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7efc180636f0 con 0x7efc00038ca0 2024-03-04T09:22:00.639 INFO:teuthology.orchestra.run.smithi052.stdout:Added host 'smithi106' with addr '172.21.15.106' 2024-03-04T09:22:00.641 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:22:00.644+0000 7efc1cbb5700 1 -- 172.21.15.52:0/167394041 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7efc00038ca0 msgr2=0x7efc0003b160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:22:00.641 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:22:00.644+0000 7efc1cbb5700 1 --2- 172.21.15.52:0/167394041 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7efc00038ca0 0x7efc0003b160 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7efc0c006fd0 tx=0x7efc0c006e40 comp rx=0 tx=0).stop 2024-03-04T09:22:00.641 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:22:00.644+0000 7efc1cbb5700 1 -- 172.21.15.52:0/167394041 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efc18106980 msgr2=0x7efc18198440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:22:00.641 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:22:00.644+0000 7efc1cbb5700 1 --2- 172.21.15.52:0/167394041 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efc18106980 0x7efc18198440 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7efc08006010 tx=0x7efc08005030 comp rx=0 tx=0).stop 2024-03-04T09:22:00.642 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:22:00.644+0000 7efc1cbb5700 1 -- 172.21.15.52:0/167394041 shutdown_connections 2024-03-04T09:22:00.642 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:22:00.644+0000 7efc1cbb5700 1 --2- 172.21.15.52:0/167394041 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7efc00038ca0 0x7efc0003b160 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:22:00.642 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:22:00.644+0000 7efc1cbb5700 1 --2- 172.21.15.52:0/167394041 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efc18106980 0x7efc18198440 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:22:00.642 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:22:00.644+0000 7efc1cbb5700 1 -- 172.21.15.52:0/167394041 >> 172.21.15.52:0/167394041 conn(0x7efc180fc470 msgr2=0x7efc18108500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:22:00.642 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:22:00.644+0000 7efc1cbb5700 1 -- 172.21.15.52:0/167394041 shutdown_connections 2024-03-04T09:22:00.642 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:22:00.644+0000 7efc1cbb5700 1 -- 172.21.15.52:0/167394041 wait complete. 2024-03-04T09:22:00.835 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:22:00 smithi052 ceph-mon[32473]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T09:22:01.188 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4e059f08-da08-11ee-95c6-87774f69a715 -- ceph orch host ls --format=json 2024-03-04T09:22:01.426 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/4e059f08-da08-11ee-95c6-87774f69a715/mon.a/config 2024-03-04T09:22:01.830 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:22:01 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:22:01.830 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:22:01 smithi052 ceph-mon[32473]: Added host smithi106 2024-03-04T09:22:01.830 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:22:01 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T09:22:01.830 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:22:01 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:22:02.818 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:22:02.819+0000 7fb05927b700 1 -- 172.21.15.52:0/1915513445 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb054102c60 msgr2=0x7fb054103040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:22:02.818 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:22:02.819+0000 7fb05927b700 1 --2- 172.21.15.52:0/1915513445 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb054102c60 0x7fb054103040 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fb044009910 tx=0x7fb044009c20 comp rx=0 tx=0).stop 2024-03-04T09:22:02.818 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:22:02.819+0000 7fb05927b700 1 -- 172.21.15.52:0/1915513445 shutdown_connections 2024-03-04T09:22:02.818 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:22:02.819+0000 7fb05927b700 1 --2- 172.21.15.52:0/1915513445 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb054102c60 0x7fb054103040 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:22:02.818 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:22:02.819+0000 7fb05927b700 1 -- 172.21.15.52:0/1915513445 >> 172.21.15.52:0/1915513445 conn(0x7fb0540fc490 msgr2=0x7fb0540fe8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:22:02.818 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:22:02.820+0000 7fb05927b700 1 -- 172.21.15.52:0/1915513445 shutdown_connections 2024-03-04T09:22:02.818 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:22:02.820+0000 7fb05927b700 1 -- 172.21.15.52:0/1915513445 wait complete. 2024-03-04T09:22:02.818 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:22:02.820+0000 7fb05927b700 1 Processor -- start 2024-03-04T09:22:02.818 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:22:02.820+0000 7fb05927b700 1 -- start start 2024-03-04T09:22:02.819 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:22:02.821+0000 7fb05927b700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb054102c60 0x7fb054197610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:22:02.819 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:22:02.821+0000 7fb05927b700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb05410c150 con 0x7fb054102c60 2024-03-04T09:22:02.819 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:22:02.821+0000 7fb052ffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb054102c60 0x7fb054197610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:22:02.819 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:22:02.821+0000 7fb052ffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb054102c60 0x7fb054197610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:41720/0 (socket says 172.21.15.52:41720) 2024-03-04T09:22:02.819 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:22:02.821+0000 7fb052ffd700 1 -- 172.21.15.52:0/525083650 learned_addr learned my addr 172.21.15.52:0/525083650 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-04T09:22:02.819 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:22:02.821+0000 7fb052ffd700 1 -- 172.21.15.52:0/525083650 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb054197b50 con 0x7fb054102c60 2024-03-04T09:22:02.819 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:22:02.821+0000 7fb052ffd700 1 --2- 172.21.15.52:0/525083650 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb054102c60 0x7fb054197610 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fb044006010 tx=0x7fb044005050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:22:02.820 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:22:02.821+0000 7fb03bfff700 1 -- 172.21.15.52:0/525083650 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb044026070 con 0x7fb054102c60 2024-03-04T09:22:02.820 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:22:02.821+0000 7fb05927b700 1 -- 172.21.15.52:0/525083650 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb054197de0 con 0x7fb054102c60 2024-03-04T09:22:02.820 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:22:02.821+0000 7fb03bfff700 1 -- 172.21.15.52:0/525083650 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb04400bc30 con 0x7fb054102c60 2024-03-04T09:22:02.820 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:22:02.821+0000 7fb03bfff700 1 -- 172.21.15.52:0/525083650 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb044003ab0 con 0x7fb054102c60 2024-03-04T09:22:02.821 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:22:02.821+0000 7fb05927b700 1 -- 172.21.15.52:0/525083650 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb054198240 con 0x7fb054102c60 2024-03-04T09:22:02.821 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:22:02.822+0000 7fb05927b700 1 -- 172.21.15.52:0/525083650 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb05406a7e0 con 0x7fb054102c60 2024-03-04T09:22:02.821 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:22:02.822+0000 7fb03bfff700 1 -- 172.21.15.52:0/525083650 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 13) v1 ==== 45984+0+0 (secure 0 0 0) 0x7fb044029070 con 0x7fb054102c60 2024-03-04T09:22:02.821 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:22:02.823+0000 7fb03bfff700 1 --2- 172.21.15.52:0/525083650 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7fb03c038c40 0x7fb03c03b100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:22:02.821 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:22:02.823+0000 7fb03bfff700 1 -- 172.21.15.52:0/525083650 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fb04401e070 con 0x7fb054102c60 2024-03-04T09:22:02.821 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:22:02.823+0000 7fb0527fc700 1 --2- 172.21.15.52:0/525083650 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7fb03c038c40 0x7fb03c03b100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:22:02.821 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:22:02.823+0000 7fb0527fc700 1 --2- 172.21.15.52:0/525083650 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7fb03c038c40 0x7fb03c03b100 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fb048006fd0 tx=0x7fb048006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:22:02.824 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:22:02.827+0000 7fb03bfff700 1 -- 172.21.15.52:0/525083650 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fb044013070 con 0x7fb054102c60 2024-03-04T09:22:03.016 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:22:02 smithi052 ceph-mon[32473]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T09:22:03.017 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:22:02 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:22:03.038 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:22:03.041+0000 7fb05927b700 1 -- 172.21.15.52:0/525083650 --> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fb054198970 con 0x7fb03c038c40 2024-03-04T09:22:03.039 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:22:03.042+0000 7fb03bfff700 1 -- 172.21.15.52:0/525083650 <== mgr.14152 v2:172.21.15.52:6800/3036033923 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+160 (secure 0 0 0) 0x7fb054198970 con 0x7fb03c038c40 2024-03-04T09:22:03.039 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-03-04T09:22:03.039 INFO:teuthology.orchestra.run.smithi052.stdout:[{"addr": "172.21.15.52", "hostname": "smithi052", "labels": [], "status": ""}, {"addr": "172.21.15.106", "hostname": "smithi106", "labels": [], "status": ""}] 2024-03-04T09:22:03.042 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:22:03.045+0000 7fb05927b700 1 -- 172.21.15.52:0/525083650 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7fb03c038c40 msgr2=0x7fb03c03b100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:22:03.042 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:22:03.045+0000 7fb05927b700 1 --2- 172.21.15.52:0/525083650 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7fb03c038c40 0x7fb03c03b100 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fb048006fd0 tx=0x7fb048006e40 comp rx=0 tx=0).stop 2024-03-04T09:22:03.042 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:22:03.045+0000 7fb05927b700 1 -- 172.21.15.52:0/525083650 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb054102c60 msgr2=0x7fb054197610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:22:03.042 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:22:03.045+0000 7fb05927b700 1 --2- 172.21.15.52:0/525083650 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb054102c60 0x7fb054197610 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fb044006010 tx=0x7fb044005050 comp rx=0 tx=0).stop 2024-03-04T09:22:03.042 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:22:03.045+0000 7fb05927b700 1 -- 172.21.15.52:0/525083650 shutdown_connections 2024-03-04T09:22:03.042 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:22:03.045+0000 7fb05927b700 1 --2- 172.21.15.52:0/525083650 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7fb03c038c40 0x7fb03c03b100 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:22:03.042 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:22:03.045+0000 7fb05927b700 1 --2- 172.21.15.52:0/525083650 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb054102c60 0x7fb054197610 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:22:03.042 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:22:03.045+0000 7fb05927b700 1 -- 172.21.15.52:0/525083650 >> 172.21.15.52:0/525083650 conn(0x7fb0540fc490 msgr2=0x7fb054103a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:22:03.042 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:22:03.045+0000 7fb05927b700 1 -- 172.21.15.52:0/525083650 shutdown_connections 2024-03-04T09:22:03.042 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:22:03.045+0000 7fb05927b700 1 -- 172.21.15.52:0/525083650 wait complete. 2024-03-04T09:22:03.566 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi191 2024-03-04T09:22:03.566 DEBUG:teuthology.orchestra.run.smithi191:> set -ex 2024-03-04T09:22:03.566 DEBUG:teuthology.orchestra.run.smithi191:> dd of=/etc/ceph/ceph.conf 2024-03-04T09:22:03.585 DEBUG:teuthology.orchestra.run.smithi191:> set -ex 2024-03-04T09:22:03.586 DEBUG:teuthology.orchestra.run.smithi191:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-03-04T09:22:03.643 INFO:tasks.cephadm:Adding host smithi191 to orchestrator... 2024-03-04T09:22:03.644 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4e059f08-da08-11ee-95c6-87774f69a715 -- ceph orch host add smithi191 2024-03-04T09:22:03.883 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/4e059f08-da08-11ee-95c6-87774f69a715/mon.a/config 2024-03-04T09:22:04.294 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:22:04 smithi052 ceph-mon[32473]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-04T09:22:04.294 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:22:04 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:22:05.348 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:22:05.350+0000 7f51df17a700 1 -- 172.21.15.52:0/1432845137 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f51d81069a0 msgr2=0x7f51d8106d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:22:05.348 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:22:05.350+0000 7f51df17a700 1 --2- 172.21.15.52:0/1432845137 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f51d81069a0 0x7f51d8106d80 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f51d000b0a0 tx=0x7f51d000b3b0 comp rx=0 tx=0).stop 2024-03-04T09:22:05.348 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:22:05.350+0000 7f51df17a700 1 -- 172.21.15.52:0/1432845137 shutdown_connections 2024-03-04T09:22:05.348 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:22:05.350+0000 7f51df17a700 1 --2- 172.21.15.52:0/1432845137 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f51d81069a0 0x7f51d8106d80 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:22:05.348 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:22:05.350+0000 7f51df17a700 1 -- 172.21.15.52:0/1432845137 >> 172.21.15.52:0/1432845137 conn(0x7f51d80fc490 msgr2=0x7f51d80fe8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:22:05.348 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:22:05.350+0000 7f51df17a700 1 -- 172.21.15.52:0/1432845137 shutdown_connections 2024-03-04T09:22:05.348 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:22:05.350+0000 7f51df17a700 1 -- 172.21.15.52:0/1432845137 wait complete. 2024-03-04T09:22:05.349 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:22:05.350+0000 7f51df17a700 1 Processor -- start 2024-03-04T09:22:05.349 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:22:05.350+0000 7f51df17a700 1 -- start start 2024-03-04T09:22:05.349 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:22:05.351+0000 7f51df17a700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f51d81069a0 0x7f51d81983d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:22:05.349 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:22:05.351+0000 7f51df17a700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51d810afe0 con 0x7f51d81069a0 2024-03-04T09:22:05.349 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:22:05.351+0000 7f51dcf16700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f51d81069a0 0x7f51d81983d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:22:05.349 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:22:05.351+0000 7f51dcf16700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f51d81069a0 0x7f51d81983d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:35150/0 (socket says 172.21.15.52:35150) 2024-03-04T09:22:05.349 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:22:05.351+0000 7f51dcf16700 1 -- 172.21.15.52:0/1905378246 learned_addr learned my addr 172.21.15.52:0/1905378246 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-04T09:22:05.349 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:22:05.351+0000 7f51dcf16700 1 -- 172.21.15.52:0/1905378246 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51d8198910 con 0x7f51d81069a0 2024-03-04T09:22:05.349 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:22:05.351+0000 7f51dcf16700 1 --2- 172.21.15.52:0/1905378246 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f51d81069a0 0x7f51d81983d0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f51d0015040 tx=0x7f51d0003d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:22:05.349 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:22:05.351+0000 7f51d5ffb700 1 -- 172.21.15.52:0/1905378246 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f51d0017040 con 0x7f51d81069a0 2024-03-04T09:22:05.349 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:22:05.351+0000 7f51d5ffb700 1 -- 172.21.15.52:0/1905378246 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f51d0022070 con 0x7f51d81069a0 2024-03-04T09:22:05.349 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:22:05.351+0000 7f51df17a700 1 -- 172.21.15.52:0/1905378246 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51d8198ba0 con 0x7f51d81069a0 2024-03-04T09:22:05.349 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:22:05.351+0000 7f51d5ffb700 1 -- 172.21.15.52:0/1905378246 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f51d001d5d0 con 0x7f51d81069a0 2024-03-04T09:22:05.350 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:22:05.351+0000 7f51df17a700 1 -- 172.21.15.52:0/1905378246 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51d8199060 con 0x7f51d81069a0 2024-03-04T09:22:05.351 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:22:05.352+0000 7f51d5ffb700 1 -- 172.21.15.52:0/1905378246 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 13) v1 ==== 45984+0+0 (secure 0 0 0) 0x7f51d0004000 con 0x7f51d81069a0 2024-03-04T09:22:05.351 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:22:05.352+0000 7f51df17a700 1 -- 172.21.15.52:0/1905378246 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f51d81020e0 con 0x7f51d81069a0 2024-03-04T09:22:05.351 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:22:05.352+0000 7f51d5ffb700 1 --2- 172.21.15.52:0/1905378246 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f51c0038c40 0x7f51c003b100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:22:05.351 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:22:05.352+0000 7f51d5ffb700 1 -- 172.21.15.52:0/1905378246 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f51d000f050 con 0x7f51d81069a0 2024-03-04T09:22:05.351 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:22:05.353+0000 7f51d7fff700 1 --2- 172.21.15.52:0/1905378246 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f51c0038c40 0x7f51c003b100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:22:05.351 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:22:05.353+0000 7f51d7fff700 1 --2- 172.21.15.52:0/1905378246 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f51c0038c40 0x7f51c003b100 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f51cc006fd0 tx=0x7f51cc006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:22:05.354 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:22:05.357+0000 7f51d5ffb700 1 -- 172.21.15.52:0/1905378246 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f51d0007e20 con 0x7f51d81069a0 2024-03-04T09:22:05.566 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:22:05.569+0000 7f51df17a700 1 -- 172.21.15.52:0/1905378246 --> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi191", "target": ["mon-mgr", ""]}) v1 -- 0x7f51d80636f0 con 0x7f51c0038c40 2024-03-04T09:22:05.566 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:22:05 smithi052 ceph-mon[32473]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T09:22:07.584 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:22:07 smithi052 ceph-mon[32473]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T09:22:07.585 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:22:07 smithi052 ceph-mon[32473]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi191", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T09:22:07.585 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:22:07 smithi052 ceph-mon[32473]: Deploying cephadm binary to smithi191 2024-03-04T09:22:07.585 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:22:07 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:22:07.585 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:22:07 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:22:07.585 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:22:07 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:22:07.585 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:22:07 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:22:07.585 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:22:07 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi106", "name": "osd_memory_target"}]: dispatch 2024-03-04T09:22:07.585 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:22:07 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T09:22:07.585 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:22:07 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T09:22:08.584 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:22:08 smithi052 ceph-mon[32473]: Updating smithi106:/etc/ceph/ceph.conf 2024-03-04T09:22:09.584 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:22:09 smithi052 ceph-mon[32473]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T09:22:09.837 INFO:teuthology.orchestra.run.smithi052.stdout:Added host 'smithi191' with addr '172.21.15.191' 2024-03-04T09:22:09.837 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:22:09.840+0000 7f51d5ffb700 1 -- 172.21.15.52:0/1905378246 <== mgr.14152 v2:172.21.15.52:6800/3036033923 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7f51d80636f0 con 0x7f51c0038c40 2024-03-04T09:22:09.839 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:22:09.842+0000 7f51df17a700 1 -- 172.21.15.52:0/1905378246 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f51c0038c40 msgr2=0x7f51c003b100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:22:09.839 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:22:09.842+0000 7f51df17a700 1 --2- 172.21.15.52:0/1905378246 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f51c0038c40 0x7f51c003b100 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f51cc006fd0 tx=0x7f51cc006e40 comp rx=0 tx=0).stop 2024-03-04T09:22:09.839 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:22:09.842+0000 7f51df17a700 1 -- 172.21.15.52:0/1905378246 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f51d81069a0 msgr2=0x7f51d81983d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:22:09.840 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:22:09.842+0000 7f51df17a700 1 --2- 172.21.15.52:0/1905378246 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f51d81069a0 0x7f51d81983d0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f51d0015040 tx=0x7f51d0003d50 comp rx=0 tx=0).stop 2024-03-04T09:22:09.840 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:22:09.842+0000 7f51df17a700 1 -- 172.21.15.52:0/1905378246 shutdown_connections 2024-03-04T09:22:09.840 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:22:09.842+0000 7f51df17a700 1 --2- 172.21.15.52:0/1905378246 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f51c0038c40 0x7f51c003b100 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:22:09.840 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:22:09.842+0000 7f51df17a700 1 --2- 172.21.15.52:0/1905378246 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f51d81069a0 0x7f51d81983d0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:22:09.840 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:22:09.842+0000 7f51df17a700 1 -- 172.21.15.52:0/1905378246 >> 172.21.15.52:0/1905378246 conn(0x7f51d80fc490 msgr2=0x7f51d80fcf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:22:09.840 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:22:09.842+0000 7f51df17a700 1 -- 172.21.15.52:0/1905378246 shutdown_connections 2024-03-04T09:22:09.840 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:22:09.842+0000 7f51df17a700 1 -- 172.21.15.52:0/1905378246 wait complete. 2024-03-04T09:22:10.403 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:22:10 smithi052 ceph-mon[32473]: Updating smithi106:/var/lib/ceph/4e059f08-da08-11ee-95c6-87774f69a715/config/ceph.conf 2024-03-04T09:22:10.403 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:22:10 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:22:10.403 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:22:10 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:22:10.403 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:22:10 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:22:10.403 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:22:10 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:22:10.404 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:22:10 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T09:22:10.433 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4e059f08-da08-11ee-95c6-87774f69a715 -- ceph orch host ls --format=json 2024-03-04T09:22:10.672 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/4e059f08-da08-11ee-95c6-87774f69a715/mon.a/config 2024-03-04T09:22:11.485 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:22:11 smithi052 ceph-mon[32473]: Updating smithi106:/etc/ceph/ceph.client.admin.keyring 2024-03-04T09:22:11.485 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:22:11 smithi052 ceph-mon[32473]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T09:22:11.485 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:22:11 smithi052 ceph-mon[32473]: Updating smithi106:/var/lib/ceph/4e059f08-da08-11ee-95c6-87774f69a715/config/ceph.client.admin.keyring 2024-03-04T09:22:11.486 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:22:11 smithi052 ceph-mon[32473]: Added host smithi191 2024-03-04T09:22:11.486 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:22:11 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:22:11.486 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:22:11 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:22:12.226 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:22:12.228+0000 7f8d5dd55700 1 -- 172.21.15.52:0/2818203480 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8d58107b70 msgr2=0x7f8d58107f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:22:12.227 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:22:12.228+0000 7f8d5dd55700 1 --2- 172.21.15.52:0/2818203480 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8d58107b70 0x7f8d58107f50 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f8d4800b0a0 tx=0x7f8d4800b3b0 comp rx=0 tx=0).stop 2024-03-04T09:22:12.227 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:22:12.228+0000 7f8d5dd55700 1 -- 172.21.15.52:0/2818203480 shutdown_connections 2024-03-04T09:22:12.227 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:22:12.228+0000 7f8d5dd55700 1 --2- 172.21.15.52:0/2818203480 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8d58107b70 0x7f8d58107f50 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:22:12.227 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:22:12.228+0000 7f8d5dd55700 1 -- 172.21.15.52:0/2818203480 >> 172.21.15.52:0/2818203480 conn(0x7f8d580fd5f0 msgr2=0x7f8d580ffa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:22:12.227 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:22:12.228+0000 7f8d5dd55700 1 -- 172.21.15.52:0/2818203480 shutdown_connections 2024-03-04T09:22:12.227 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:22:12.228+0000 7f8d5dd55700 1 -- 172.21.15.52:0/2818203480 wait complete. 2024-03-04T09:22:12.227 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:22:12.228+0000 7f8d5dd55700 1 Processor -- start 2024-03-04T09:22:12.227 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:22:12.228+0000 7f8d5dd55700 1 -- start start 2024-03-04T09:22:12.227 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:22:12.229+0000 7f8d5dd55700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8d58107b70 0x7f8d58197510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:22:12.228 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:22:12.229+0000 7f8d5dd55700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d5810c120 con 0x7f8d58107b70 2024-03-04T09:22:12.228 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:22:12.229+0000 7f8d577fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8d58107b70 0x7f8d58197510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:22:12.228 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:22:12.229+0000 7f8d577fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8d58107b70 0x7f8d58197510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:35172/0 (socket says 172.21.15.52:35172) 2024-03-04T09:22:12.228 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:22:12.229+0000 7f8d577fe700 1 -- 172.21.15.52:0/2809677564 learned_addr learned my addr 172.21.15.52:0/2809677564 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-04T09:22:12.228 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:22:12.229+0000 7f8d577fe700 1 -- 172.21.15.52:0/2809677564 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d58197a50 con 0x7f8d58107b70 2024-03-04T09:22:12.228 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:22:12.229+0000 7f8d577fe700 1 --2- 172.21.15.52:0/2809677564 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8d58107b70 0x7f8d58197510 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f8d48015040 tx=0x7f8d48003d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:22:12.228 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:22:12.229+0000 7f8d54ff9700 1 -- 172.21.15.52:0/2809677564 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8d48017040 con 0x7f8d58107b70 2024-03-04T09:22:12.228 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:22:12.229+0000 7f8d54ff9700 1 -- 172.21.15.52:0/2809677564 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8d48022070 con 0x7f8d58107b70 2024-03-04T09:22:12.228 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:22:12.229+0000 7f8d5dd55700 1 -- 172.21.15.52:0/2809677564 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d58197ce0 con 0x7f8d58107b70 2024-03-04T09:22:12.229 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:22:12.229+0000 7f8d54ff9700 1 -- 172.21.15.52:0/2809677564 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8d4801d5d0 con 0x7f8d58107b70 2024-03-04T09:22:12.229 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:22:12.229+0000 7f8d5dd55700 1 -- 172.21.15.52:0/2809677564 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d5819bde0 con 0x7f8d58107b70 2024-03-04T09:22:12.230 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:22:12.230+0000 7f8d54ff9700 1 -- 172.21.15.52:0/2809677564 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 13) v1 ==== 45984+0+0 (secure 0 0 0) 0x7f8d4801d8d0 con 0x7f8d58107b70 2024-03-04T09:22:12.230 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:22:12.230+0000 7f8d5dd55700 1 -- 172.21.15.52:0/2809677564 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d581031e0 con 0x7f8d58107b70 2024-03-04T09:22:12.230 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:22:12.230+0000 7f8d54ff9700 1 --2- 172.21.15.52:0/2809677564 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f8d44038c40 0x7f8d4403b100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:22:12.230 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:22:12.231+0000 7f8d54ff9700 1 -- 172.21.15.52:0/2809677564 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f8d4800f050 con 0x7f8d58107b70 2024-03-04T09:22:12.230 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:22:12.231+0000 7f8d56ffd700 1 --2- 172.21.15.52:0/2809677564 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f8d44038c40 0x7f8d4403b100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:22:12.230 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:22:12.231+0000 7f8d56ffd700 1 --2- 172.21.15.52:0/2809677564 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f8d44038c40 0x7f8d4403b100 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f8d40006fd0 tx=0x7f8d40006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:22:12.232 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:22:12.235+0000 7f8d54ff9700 1 -- 172.21.15.52:0/2809677564 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f8d48007e20 con 0x7f8d58107b70 2024-03-04T09:22:12.443 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:22:12.446+0000 7f8d5dd55700 1 -- 172.21.15.52:0/2809677564 --> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f8d58198870 con 0x7f8d44038c40 2024-03-04T09:22:12.444 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:22:12.447+0000 7f8d54ff9700 1 -- 172.21.15.52:0/2809677564 <== mgr.14152 v2:172.21.15.52:6800/3036033923 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+240 (secure 0 0 0) 0x7f8d58198870 con 0x7f8d44038c40 2024-03-04T09:22:12.444 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-03-04T09:22:12.444 INFO:teuthology.orchestra.run.smithi052.stdout:[{"addr": "172.21.15.52", "hostname": "smithi052", "labels": [], "status": ""}, {"addr": "172.21.15.106", "hostname": "smithi106", "labels": [], "status": ""}, {"addr": "172.21.15.191", "hostname": "smithi191", "labels": [], "status": ""}] 2024-03-04T09:22:12.446 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:22:12.449+0000 7f8d5dd55700 1 -- 172.21.15.52:0/2809677564 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f8d44038c40 msgr2=0x7f8d4403b100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:22:12.446 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:22:12.449+0000 7f8d5dd55700 1 --2- 172.21.15.52:0/2809677564 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f8d44038c40 0x7f8d4403b100 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f8d40006fd0 tx=0x7f8d40006e40 comp rx=0 tx=0).stop 2024-03-04T09:22:12.447 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:22:12.449+0000 7f8d5dd55700 1 -- 172.21.15.52:0/2809677564 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8d58107b70 msgr2=0x7f8d58197510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:22:12.447 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:22:12.449+0000 7f8d5dd55700 1 --2- 172.21.15.52:0/2809677564 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8d58107b70 0x7f8d58197510 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f8d48015040 tx=0x7f8d48003d50 comp rx=0 tx=0).stop 2024-03-04T09:22:12.447 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:22:12.449+0000 7f8d5dd55700 1 -- 172.21.15.52:0/2809677564 shutdown_connections 2024-03-04T09:22:12.447 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:22:12.449+0000 7f8d5dd55700 1 --2- 172.21.15.52:0/2809677564 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f8d44038c40 0x7f8d4403b100 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:22:12.447 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:22:12.449+0000 7f8d5dd55700 1 --2- 172.21.15.52:0/2809677564 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8d58107b70 0x7f8d58197510 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:22:12.447 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:22:12.449+0000 7f8d5dd55700 1 -- 172.21.15.52:0/2809677564 >> 172.21.15.52:0/2809677564 conn(0x7f8d580fd5f0 msgr2=0x7f8d58109600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:22:12.447 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:22:12.450+0000 7f8d5dd55700 1 -- 172.21.15.52:0/2809677564 shutdown_connections 2024-03-04T09:22:12.447 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:22:12.450+0000 7f8d5dd55700 1 -- 172.21.15.52:0/2809677564 wait complete. 2024-03-04T09:22:13.052 INFO:tasks.cephadm:Setting crush tunables to default 2024-03-04T09:22:13.052 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4e059f08-da08-11ee-95c6-87774f69a715 -- ceph osd crush tunables default 2024-03-04T09:22:13.293 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/4e059f08-da08-11ee-95c6-87774f69a715/mon.a/config 2024-03-04T09:22:13.297 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:22:13 smithi052 ceph-mon[32473]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T09:22:13.297 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:22:13 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:22:14.450 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:22:14 smithi052 ceph-mon[32473]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-04T09:22:14.664 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:22:14.666+0000 7fb58e328700 1 -- 172.21.15.52:0/1768658901 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb588107aa0 msgr2=0x7fb588107e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:22:14.664 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:22:14.666+0000 7fb58e328700 1 --2- 172.21.15.52:0/1768658901 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb588107aa0 0x7fb588107e80 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fb578009910 tx=0x7fb578009c20 comp rx=0 tx=0).stop 2024-03-04T09:22:14.664 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:22:14.666+0000 7fb58e328700 1 -- 172.21.15.52:0/1768658901 shutdown_connections 2024-03-04T09:22:14.664 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:22:14.666+0000 7fb58e328700 1 --2- 172.21.15.52:0/1768658901 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb588107aa0 0x7fb588107e80 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:22:14.664 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:22:14.666+0000 7fb58e328700 1 -- 172.21.15.52:0/1768658901 >> 172.21.15.52:0/1768658901 conn(0x7fb5880fd5d0 msgr2=0x7fb5880ff9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:22:14.664 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:22:14.666+0000 7fb58e328700 1 -- 172.21.15.52:0/1768658901 shutdown_connections 2024-03-04T09:22:14.664 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:22:14.666+0000 7fb58e328700 1 -- 172.21.15.52:0/1768658901 wait complete. 2024-03-04T09:22:14.664 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:22:14.666+0000 7fb58e328700 1 Processor -- start 2024-03-04T09:22:14.665 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:22:14.666+0000 7fb58e328700 1 -- start start 2024-03-04T09:22:14.665 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:22:14.667+0000 7fb58e328700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb588107aa0 0x7fb5880743c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:22:14.665 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:22:14.667+0000 7fb58e328700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb58810c0e0 con 0x7fb588107aa0 2024-03-04T09:22:14.665 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:22:14.667+0000 7fb587fff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb588107aa0 0x7fb5880743c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:22:14.665 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:22:14.667+0000 7fb587fff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb588107aa0 0x7fb5880743c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:39242/0 (socket says 172.21.15.52:39242) 2024-03-04T09:22:14.665 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:22:14.667+0000 7fb587fff700 1 -- 172.21.15.52:0/252953357 learned_addr learned my addr 172.21.15.52:0/252953357 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-04T09:22:14.665 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:22:14.667+0000 7fb587fff700 1 -- 172.21.15.52:0/252953357 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5880762b0 con 0x7fb588107aa0 2024-03-04T09:22:14.666 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:22:14.667+0000 7fb587fff700 1 --2- 172.21.15.52:0/252953357 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb588107aa0 0x7fb5880743c0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fb578005850 tx=0x7fb578005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:22:14.667 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:22:14.667+0000 7fb5857fa700 1 -- 172.21.15.52:0/252953357 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb578026070 con 0x7fb588107aa0 2024-03-04T09:22:14.667 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:22:14.668+0000 7fb5857fa700 1 -- 172.21.15.52:0/252953357 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb57800bc30 con 0x7fb588107aa0 2024-03-04T09:22:14.667 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:22:14.668+0000 7fb58e328700 1 -- 172.21.15.52:0/252953357 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb588074960 con 0x7fb588107aa0 2024-03-04T09:22:14.667 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:22:14.668+0000 7fb5857fa700 1 -- 172.21.15.52:0/252953357 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb578003a20 con 0x7fb588107aa0 2024-03-04T09:22:14.667 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:22:14.668+0000 7fb58e328700 1 -- 172.21.15.52:0/252953357 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb588074e20 con 0x7fb588107aa0 2024-03-04T09:22:14.667 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:22:14.668+0000 7fb5857fa700 1 -- 172.21.15.52:0/252953357 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 13) v1 ==== 45984+0+0 (secure 0 0 0) 0x7fb578029070 con 0x7fb588107aa0 2024-03-04T09:22:14.667 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:22:14.668+0000 7fb58e328700 1 -- 172.21.15.52:0/252953357 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb5881031e0 con 0x7fb588107aa0 2024-03-04T09:22:14.667 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:22:14.669+0000 7fb5857fa700 1 --2- 172.21.15.52:0/252953357 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7fb574038ca0 0x7fb57403b160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:22:14.667 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:22:14.669+0000 7fb5857fa700 1 -- 172.21.15.52:0/252953357 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fb57801e070 con 0x7fb588107aa0 2024-03-04T09:22:14.667 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:22:14.669+0000 7fb5877fe700 1 --2- 172.21.15.52:0/252953357 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7fb574038ca0 0x7fb57403b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:22:14.667 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:22:14.669+0000 7fb5877fe700 1 --2- 172.21.15.52:0/252953357 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7fb574038ca0 0x7fb57403b160 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fb570006fd0 tx=0x7fb570006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:22:14.670 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:22:14.674+0000 7fb5857fa700 1 -- 172.21.15.52:0/252953357 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fb578013070 con 0x7fb588107aa0 2024-03-04T09:22:14.878 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:22:14.881+0000 7fb58e328700 1 -- 172.21.15.52:0/252953357 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7fb5880636f0 con 0x7fb588107aa0 2024-03-04T09:22:15.206 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:22:15.209+0000 7fb5857fa700 1 -- 172.21.15.52:0/252953357 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) v1 ==== 124+0+0 (secure 0 0 0) 0x7fb578030090 con 0x7fb588107aa0 2024-03-04T09:22:15.209 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:22:15.212+0000 7fb58e328700 1 -- 172.21.15.52:0/252953357 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7fb574038ca0 msgr2=0x7fb57403b160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:22:15.209 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:22:15.212+0000 7fb58e328700 1 --2- 172.21.15.52:0/252953357 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7fb574038ca0 0x7fb57403b160 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fb570006fd0 tx=0x7fb570006e40 comp rx=0 tx=0).stop 2024-03-04T09:22:15.209 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:22:15.212+0000 7fb58e328700 1 -- 172.21.15.52:0/252953357 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb588107aa0 msgr2=0x7fb5880743c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:22:15.209 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:22:15.212+0000 7fb58e328700 1 --2- 172.21.15.52:0/252953357 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb588107aa0 0x7fb5880743c0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fb578005850 tx=0x7fb578005030 comp rx=0 tx=0).stop 2024-03-04T09:22:15.209 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:22:15.212+0000 7fb58e328700 1 -- 172.21.15.52:0/252953357 shutdown_connections 2024-03-04T09:22:15.209 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:22:15.212+0000 7fb58e328700 1 --2- 172.21.15.52:0/252953357 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7fb574038ca0 0x7fb57403b160 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:22:15.209 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:22:15.212+0000 7fb58e328700 1 --2- 172.21.15.52:0/252953357 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb588107aa0 0x7fb5880743c0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:22:15.210 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:22:15.212+0000 7fb58e328700 1 -- 172.21.15.52:0/252953357 >> 172.21.15.52:0/252953357 conn(0x7fb5880fd5d0 msgr2=0x7fb588109610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:22:15.210 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:22:15.212+0000 7fb58e328700 1 -- 172.21.15.52:0/252953357 shutdown_connections 2024-03-04T09:22:15.210 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:22:15.212+0000 7fb58e328700 1 -- 172.21.15.52:0/252953357 wait complete. 2024-03-04T09:22:15.210 INFO:teuthology.orchestra.run.smithi052.stderr:adjusted tunables profile to default 2024-03-04T09:22:15.528 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:22:15 smithi052 ceph-mon[32473]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T09:22:15.528 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:22:15 smithi052 ceph-mon[32473]: from='client.? 172.21.15.52:0/252953357' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-03-04T09:22:15.800 INFO:tasks.cephadm:Adding mon.a on smithi052 2024-03-04T09:22:15.801 INFO:tasks.cephadm:Adding mon.b on smithi106 2024-03-04T09:22:15.801 INFO:tasks.cephadm:Adding mon.c on smithi191 2024-03-04T09:22:15.801 DEBUG:teuthology.orchestra.run.smithi191:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4e059f08-da08-11ee-95c6-87774f69a715 -- ceph orch apply mon '3;smithi052:172.21.15.52=a;smithi106:172.21.15.106=b;smithi191:172.21.15.191=c' 2024-03-04T09:22:16.036 INFO:teuthology.orchestra.run.smithi191.stderr:Inferring config /var/lib/ceph/4e059f08-da08-11ee-95c6-87774f69a715/config/ceph.conf 2024-03-04T09:22:16.584 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:22:16 smithi052 ceph-mon[32473]: from='client.? 172.21.15.52:0/252953357' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-03-04T09:22:16.585 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:22:16 smithi052 ceph-mon[32473]: osdmap e4: 0 total, 0 up, 0 in 2024-03-04T09:22:16.585 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:22:16 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:22:16.585 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:22:16 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:22:16.585 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:22:16 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:22:16.585 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:22:16 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:22:16.585 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:22:16 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi191", "name": "osd_memory_target"}]: dispatch 2024-03-04T09:22:16.585 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:22:16 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T09:22:16.585 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:22:16 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T09:22:17.585 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:22:17 smithi052 ceph-mon[32473]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T09:22:17.585 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:22:17 smithi052 ceph-mon[32473]: Updating smithi191:/etc/ceph/ceph.conf 2024-03-04T09:22:17.585 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:22:17 smithi052 ceph-mon[32473]: Updating smithi191:/var/lib/ceph/4e059f08-da08-11ee-95c6-87774f69a715/config/ceph.conf 2024-03-04T09:22:17.585 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:22:17 smithi052 ceph-mon[32473]: Updating smithi191:/etc/ceph/ceph.client.admin.keyring 2024-03-04T09:22:17.585 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:22:17 smithi052 ceph-mon[32473]: Updating smithi191:/var/lib/ceph/4e059f08-da08-11ee-95c6-87774f69a715/config/ceph.client.admin.keyring 2024-03-04T09:22:17.585 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:22:17 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:22:17.585 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:22:17 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:22:17.585 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:22:17 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:22:17.638 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:22:17.641+0000 7fb7e6765700 1 -- 172.21.15.191:0/3389217661 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb7e01069a0 msgr2=0x7fb7e0106d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:22:17.639 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:22:17.641+0000 7fb7e6765700 1 --2- 172.21.15.191:0/3389217661 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb7e01069a0 0x7fb7e0106d80 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fb7d0009910 tx=0x7fb7d0009c20 comp rx=0 tx=0).stop 2024-03-04T09:22:17.639 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:22:17.641+0000 7fb7e6765700 1 -- 172.21.15.191:0/3389217661 shutdown_connections 2024-03-04T09:22:17.639 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:22:17.641+0000 7fb7e6765700 1 --2- 172.21.15.191:0/3389217661 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb7e01069a0 0x7fb7e0106d80 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:22:17.639 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:22:17.641+0000 7fb7e6765700 1 -- 172.21.15.191:0/3389217661 >> 172.21.15.191:0/3389217661 conn(0x7fb7e00fc4b0 msgr2=0x7fb7e00fe8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:22:17.639 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:22:17.641+0000 7fb7e6765700 1 -- 172.21.15.191:0/3389217661 shutdown_connections 2024-03-04T09:22:17.639 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:22:17.641+0000 7fb7e6765700 1 -- 172.21.15.191:0/3389217661 wait complete. 2024-03-04T09:22:17.641 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:22:17.642+0000 7fb7e6765700 1 Processor -- start 2024-03-04T09:22:17.641 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:22:17.642+0000 7fb7e6765700 1 -- start start 2024-03-04T09:22:17.641 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:22:17.642+0000 7fb7e6765700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb7e01069a0 0x7fb7e0197570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:22:17.641 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:22:17.642+0000 7fb7e6765700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7e010afe0 con 0x7fb7e01069a0 2024-03-04T09:22:17.641 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:22:17.643+0000 7fb7dffff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb7e01069a0 0x7fb7e0197570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:22:17.641 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:22:17.643+0000 7fb7dffff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb7e01069a0 0x7fb7e0197570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.191:57008/0 (socket says 172.21.15.191:57008) 2024-03-04T09:22:17.641 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:22:17.643+0000 7fb7dffff700 1 -- 172.21.15.191:0/848088692 learned_addr learned my addr 172.21.15.191:0/848088692 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-03-04T09:22:17.641 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:22:17.643+0000 7fb7dffff700 1 -- 172.21.15.191:0/848088692 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7e0197ab0 con 0x7fb7e01069a0 2024-03-04T09:22:17.641 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:22:17.643+0000 7fb7dffff700 1 --2- 172.21.15.191:0/848088692 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb7e01069a0 0x7fb7e0197570 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fb7d0006010 tx=0x7fb7d0005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:22:17.641 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:22:17.643+0000 7fb7dd7fa700 1 -- 172.21.15.191:0/848088692 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb7d0026070 con 0x7fb7e01069a0 2024-03-04T09:22:17.641 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:22:17.643+0000 7fb7e6765700 1 -- 172.21.15.191:0/848088692 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7e0197da0 con 0x7fb7e01069a0 2024-03-04T09:22:17.642 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:22:17.643+0000 7fb7dd7fa700 1 -- 172.21.15.191:0/848088692 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb7d000bbd0 con 0x7fb7e01069a0 2024-03-04T09:22:17.642 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:22:17.643+0000 7fb7dd7fa700 1 -- 172.21.15.191:0/848088692 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb7d0003910 con 0x7fb7e01069a0 2024-03-04T09:22:17.642 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:22:17.643+0000 7fb7e6765700 1 -- 172.21.15.191:0/848088692 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7e019be40 con 0x7fb7e01069a0 2024-03-04T09:22:17.643 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:22:17.644+0000 7fb7dd7fa700 1 -- 172.21.15.191:0/848088692 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 13) v1 ==== 45984+0+0 (secure 0 0 0) 0x7fb7d0029070 con 0x7fb7e01069a0 2024-03-04T09:22:17.643 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:22:17.644+0000 7fb7e6765700 1 -- 172.21.15.191:0/848088692 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb7e01020e0 con 0x7fb7e01069a0 2024-03-04T09:22:17.643 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:22:17.644+0000 7fb7dd7fa700 1 --2- 172.21.15.191:0/848088692 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7fb7c8038c40 0x7fb7c803b100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:22:17.643 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:22:17.644+0000 7fb7dd7fa700 1 -- 172.21.15.191:0/848088692 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fb7d001e070 con 0x7fb7e01069a0 2024-03-04T09:22:17.643 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:22:17.645+0000 7fb7df7fe700 1 --2- 172.21.15.191:0/848088692 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7fb7c8038c40 0x7fb7c803b100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:22:17.643 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:22:17.645+0000 7fb7df7fe700 1 --2- 172.21.15.191:0/848088692 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7fb7c8038c40 0x7fb7c803b100 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fb7d4006fd0 tx=0x7fb7d4006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:22:17.647 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:22:17.650+0000 7fb7dd7fa700 1 -- 172.21.15.191:0/848088692 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fb7d0013070 con 0x7fb7e01069a0 2024-03-04T09:22:17.853 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:22:17.855+0000 7fb7e6765700 1 -- 172.21.15.191:0/848088692 --> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi052:172.21.15.52=a;smithi106:172.21.15.106=b;smithi191:172.21.15.191=c", "target": ["mon-mgr", ""]}) v1 -- 0x7fb7e01988d0 con 0x7fb7c8038c40 2024-03-04T09:22:17.859 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:22:17.862+0000 7fb7dd7fa700 1 -- 172.21.15.191:0/848088692 <== mgr.14152 v2:172.21.15.52:6800/3036033923 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7fb7e01988d0 con 0x7fb7c8038c40 2024-03-04T09:22:17.859 INFO:teuthology.orchestra.run.smithi191.stdout:Scheduled mon update... 2024-03-04T09:22:17.861 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:22:17.864+0000 7fb7e6765700 1 -- 172.21.15.191:0/848088692 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7fb7c8038c40 msgr2=0x7fb7c803b100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:22:17.861 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:22:17.864+0000 7fb7e6765700 1 --2- 172.21.15.191:0/848088692 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7fb7c8038c40 0x7fb7c803b100 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fb7d4006fd0 tx=0x7fb7d4006e40 comp rx=0 tx=0).stop 2024-03-04T09:22:17.861 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:22:17.864+0000 7fb7e6765700 1 -- 172.21.15.191:0/848088692 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb7e01069a0 msgr2=0x7fb7e0197570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:22:17.862 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:22:17.864+0000 7fb7e6765700 1 --2- 172.21.15.191:0/848088692 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb7e01069a0 0x7fb7e0197570 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fb7d0006010 tx=0x7fb7d0005030 comp rx=0 tx=0).stop 2024-03-04T09:22:17.862 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:22:17.864+0000 7fb7e6765700 1 -- 172.21.15.191:0/848088692 shutdown_connections 2024-03-04T09:22:17.862 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:22:17.864+0000 7fb7e6765700 1 --2- 172.21.15.191:0/848088692 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7fb7c8038c40 0x7fb7c803b100 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:22:17.862 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:22:17.864+0000 7fb7e6765700 1 --2- 172.21.15.191:0/848088692 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb7e01069a0 0x7fb7e0197570 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:22:17.862 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:22:17.864+0000 7fb7e6765700 1 -- 172.21.15.191:0/848088692 >> 172.21.15.191:0/848088692 conn(0x7fb7e00fc4b0 msgr2=0x7fb7e01084e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:22:17.862 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:22:17.864+0000 7fb7e6765700 1 -- 172.21.15.191:0/848088692 shutdown_connections 2024-03-04T09:22:17.862 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:22:17.864+0000 7fb7e6765700 1 -- 172.21.15.191:0/848088692 wait complete. 2024-03-04T09:22:18.443 DEBUG:teuthology.orchestra.run.smithi106:mon.b> sudo journalctl -f -n 0 -u ceph-4e059f08-da08-11ee-95c6-87774f69a715@mon.b.service 2024-03-04T09:22:18.446 DEBUG:teuthology.orchestra.run.smithi191:mon.c> sudo journalctl -f -n 0 -u ceph-4e059f08-da08-11ee-95c6-87774f69a715@mon.c.service 2024-03-04T09:22:18.448 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-03-04T09:22:18.448 DEBUG:teuthology.orchestra.run.smithi191:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4e059f08-da08-11ee-95c6-87774f69a715 -- ceph mon dump -f json 2024-03-04T09:22:19.334 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:22:18 smithi052 ceph-mon[32473]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T09:22:19.335 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:22:18 smithi052 ceph-mon[32473]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi052:172.21.15.52=a;smithi106:172.21.15.106=b;smithi191:172.21.15.191=c", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T09:22:19.335 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:22:18 smithi052 ceph-mon[32473]: Saving service mon spec with placement smithi052:172.21.15.52=a;smithi106:172.21.15.106=b;smithi191:172.21.15.191=c;count:3 2024-03-04T09:22:19.335 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:22:18 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:22:19.335 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:22:18 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T09:22:19.335 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:22:18 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T09:22:19.335 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:22:18 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T09:22:19.335 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:22:18 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:22:19.335 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:22:18 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-04T09:22:19.335 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:22:18 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T09:22:19.335 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:22:18 smithi052 ceph-mon[32473]: Deploying daemon mon.c on smithi191 2024-03-04T09:22:20.050 INFO:teuthology.orchestra.run.smithi191.stderr:Inferring config /var/lib/ceph/4e059f08-da08-11ee-95c6-87774f69a715/config/ceph.conf 2024-03-04T09:22:21.335 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:22:20 smithi052 ceph-mon[32473]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T09:22:21.949 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:22:21.948+0000 7fc885781700 1 -- 172.21.15.191:0/2744275103 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc880107aa0 msgr2=0x7fc880107e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:22:21.949 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:22:21.948+0000 7fc885781700 1 --2- 172.21.15.191:0/2744275103 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc880107aa0 0x7fc880107e80 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fc870009910 tx=0x7fc870009c20 comp rx=0 tx=0).stop 2024-03-04T09:22:21.950 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:22:21.948+0000 7fc885781700 1 -- 172.21.15.191:0/2744275103 shutdown_connections 2024-03-04T09:22:21.950 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:22:21.948+0000 7fc885781700 1 --2- 172.21.15.191:0/2744275103 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc880107aa0 0x7fc880107e80 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:22:21.950 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:22:21.948+0000 7fc885781700 1 -- 172.21.15.191:0/2744275103 >> 172.21.15.191:0/2744275103 conn(0x7fc8800fd5d0 msgr2=0x7fc8800ff9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:22:21.950 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:22:21.948+0000 7fc885781700 1 -- 172.21.15.191:0/2744275103 shutdown_connections 2024-03-04T09:22:21.950 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:22:21.948+0000 7fc885781700 1 -- 172.21.15.191:0/2744275103 wait complete. 2024-03-04T09:22:21.950 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:22:21.948+0000 7fc885781700 1 Processor -- start 2024-03-04T09:22:21.950 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:22:21.949+0000 7fc885781700 1 -- start start 2024-03-04T09:22:21.950 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:22:21.949+0000 7fc885781700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc880107aa0 0x7fc880197650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:22:21.950 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:22:21.949+0000 7fc885781700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc88010c0e0 con 0x7fc880107aa0 2024-03-04T09:22:21.950 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:22:21.949+0000 7fc87effd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc880107aa0 0x7fc880197650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:22:21.950 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:22:21.949+0000 7fc87effd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc880107aa0 0x7fc880197650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.191:57018/0 (socket says 172.21.15.191:57018) 2024-03-04T09:22:21.950 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:22:21.949+0000 7fc87effd700 1 -- 172.21.15.191:0/3947224788 learned_addr learned my addr 172.21.15.191:0/3947224788 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-03-04T09:22:21.950 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:22:21.949+0000 7fc87effd700 1 -- 172.21.15.191:0/3947224788 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc880197b90 con 0x7fc880107aa0 2024-03-04T09:22:21.951 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:22:21.950+0000 7fc87effd700 1 --2- 172.21.15.191:0/3947224788 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc880107aa0 0x7fc880197650 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fc870006010 tx=0x7fc870005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:22:21.951 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:22:21.950+0000 7fc867fff700 1 -- 172.21.15.191:0/3947224788 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc870026070 con 0x7fc880107aa0 2024-03-04T09:22:21.951 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:22:21.950+0000 7fc885781700 1 -- 172.21.15.191:0/3947224788 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc880197e80 con 0x7fc880107aa0 2024-03-04T09:22:21.951 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:22:21.950+0000 7fc867fff700 1 -- 172.21.15.191:0/3947224788 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc87000bbd0 con 0x7fc880107aa0 2024-03-04T09:22:21.951 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:22:21.950+0000 7fc867fff700 1 -- 172.21.15.191:0/3947224788 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc870003910 con 0x7fc880107aa0 2024-03-04T09:22:21.951 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:22:21.950+0000 7fc885781700 1 -- 172.21.15.191:0/3947224788 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc88019bf20 con 0x7fc880107aa0 2024-03-04T09:22:21.951 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:22:21.951+0000 7fc867fff700 1 -- 172.21.15.191:0/3947224788 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 13) v1 ==== 45984+0+0 (secure 0 0 0) 0x7fc870029070 con 0x7fc880107aa0 2024-03-04T09:22:21.951 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:22:21.951+0000 7fc885781700 1 -- 172.21.15.191:0/3947224788 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc8801031e0 con 0x7fc880107aa0 2024-03-04T09:22:21.951 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:22:21.951+0000 7fc867fff700 1 --2- 172.21.15.191:0/3947224788 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7fc868038c40 0x7fc86803b100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:22:21.951 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:22:21.951+0000 7fc867fff700 1 -- 172.21.15.191:0/3947224788 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fc87001e070 con 0x7fc880107aa0 2024-03-04T09:22:21.951 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:22:21.951+0000 7fc87e7fc700 1 --2- 172.21.15.191:0/3947224788 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7fc868038c40 0x7fc86803b100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:22:21.951 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:22:21.952+0000 7fc87e7fc700 1 --2- 172.21.15.191:0/3947224788 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7fc868038c40 0x7fc86803b100 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fc874006fd0 tx=0x7fc874006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:22:21.953 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:22:21.957+0000 7fc867fff700 1 -- 172.21.15.191:0/3947224788 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fc870013070 con 0x7fc880107aa0 2024-03-04T09:22:22.251 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:22:22.255+0000 7fc885781700 1 -- 172.21.15.191:0/3947224788 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fc8801989b0 con 0x7fc880107aa0 2024-03-04T09:22:22.252 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:22:22.256+0000 7fc867fff700 1 -- 172.21.15.191:0/3947224788 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+745 (secure 0 0 0) 0x7fc870030090 con 0x7fc880107aa0 2024-03-04T09:22:22.252 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-03-04T09:22:22.253 INFO:teuthology.orchestra.run.smithi191.stdout:{"epoch":1,"fsid":"4e059f08-da08-11ee-95c6-87774f69a715","modified":"2024-03-04T09:20:42.665781Z","created":"2024-03-04T09:20:42.665781Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:3300","nonce":0},{"type":"v1","addr":"172.21.15.52:6789","nonce":0}]},"addr":"172.21.15.52:6789/0","public_addr":"172.21.15.52:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-03-04T09:22:22.258 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:22:22.261+0000 7fc885781700 1 -- 172.21.15.191:0/3947224788 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7fc868038c40 msgr2=0x7fc86803b100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:22:22.258 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:22:22.261+0000 7fc885781700 1 --2- 172.21.15.191:0/3947224788 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7fc868038c40 0x7fc86803b100 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fc874006fd0 tx=0x7fc874006e40 comp rx=0 tx=0).stop 2024-03-04T09:22:22.258 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:22:22.261+0000 7fc885781700 1 -- 172.21.15.191:0/3947224788 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc880107aa0 msgr2=0x7fc880197650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:22:22.259 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:22:22.261+0000 7fc885781700 1 --2- 172.21.15.191:0/3947224788 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc880107aa0 0x7fc880197650 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fc870006010 tx=0x7fc870005030 comp rx=0 tx=0).stop 2024-03-04T09:22:22.259 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:22:22.262+0000 7fc885781700 1 -- 172.21.15.191:0/3947224788 shutdown_connections 2024-03-04T09:22:22.259 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:22:22.262+0000 7fc885781700 1 --2- 172.21.15.191:0/3947224788 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7fc868038c40 0x7fc86803b100 secure :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fc874006fd0 tx=0x7fc874006e40 comp rx=0 tx=0).stop 2024-03-04T09:22:22.259 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:22:22.262+0000 7fc885781700 1 --2- 172.21.15.191:0/3947224788 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc880107aa0 0x7fc880197650 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:22:22.259 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:22:22.262+0000 7fc885781700 1 -- 172.21.15.191:0/3947224788 >> 172.21.15.191:0/3947224788 conn(0x7fc8800fd5d0 msgr2=0x7fc880109650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:22:22.259 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:22:22.262+0000 7fc885781700 1 -- 172.21.15.191:0/3947224788 shutdown_connections 2024-03-04T09:22:22.259 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:22:22.263+0000 7fc885781700 1 -- 172.21.15.191:0/3947224788 wait complete. 2024-03-04T09:22:22.261 INFO:teuthology.orchestra.run.smithi191.stderr:dumped monmap epoch 1 2024-03-04T09:22:23.334 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:22:22 smithi052 ceph-mon[32473]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T09:22:23.335 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:22:22 smithi052 ceph-mon[32473]: from='client.? 172.21.15.191:0/3947224788' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-04T09:22:23.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:22:23 smithi191 systemd[1]: Started Ceph mon.c for 4e059f08-da08-11ee-95c6-87774f69a715. 2024-03-04T09:22:23.863 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-03-04T09:22:23.864 DEBUG:teuthology.orchestra.run.smithi191:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4e059f08-da08-11ee-95c6-87774f69a715 -- ceph mon dump -f json 2024-03-04T09:22:24.098 INFO:teuthology.orchestra.run.smithi191.stderr:Inferring config /var/lib/ceph/4e059f08-da08-11ee-95c6-87774f69a715/mon.c/config 2024-03-04T09:22:28.463 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:22:28.466+0000 7fba30d23700 1 -- 172.21.15.191:0/795488539 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fba140035c0 msgr2=0x7fba14005a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:22:28.463 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:22:28.466+0000 7fba30d23700 1 --2- 172.21.15.191:0/795488539 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fba140035c0 0x7fba14005a50 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fba2c1009c0 tx=0x7fba1c00feb0 comp rx=0 tx=0).stop 2024-03-04T09:22:28.463 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:22:28.467+0000 7fba30d23700 1 -- 172.21.15.191:0/795488539 shutdown_connections 2024-03-04T09:22:28.464 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:22:28.467+0000 7fba30d23700 1 --2- 172.21.15.191:0/795488539 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fba140035c0 0x7fba14005a50 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:22:28.464 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:22:28.467+0000 7fba30d23700 1 -- 172.21.15.191:0/795488539 >> 172.21.15.191:0/795488539 conn(0x7fba2c0fc580 msgr2=0x7fba2c0fe9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:22:28.464 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:22:28.467+0000 7fba30d23700 1 -- 172.21.15.191:0/795488539 shutdown_connections 2024-03-04T09:22:28.464 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:22:28.467+0000 7fba30d23700 1 -- 172.21.15.191:0/795488539 wait complete. 2024-03-04T09:22:28.466 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:22:28.468+0000 7fba30d23700 1 Processor -- start 2024-03-04T09:22:28.466 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:22:28.468+0000 7fba30d23700 1 -- start start 2024-03-04T09:22:28.466 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:22:28.468+0000 7fba30d23700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fba140035c0 0x7fba2c19f640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:22:28.466 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:22:28.468+0000 7fba30d23700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fba2c19fb80 0x7fba2c199710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:22:28.466 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:22:28.468+0000 7fba30d23700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba2c108b30 con 0x7fba140035c0 2024-03-04T09:22:28.466 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:22:28.468+0000 7fba30d23700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba2c1089b0 con 0x7fba2c19fb80 2024-03-04T09:22:28.466 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:22:28.468+0000 7fba29d9b700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fba2c19fb80 0x7fba2c199710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:22:28.466 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:22:28.468+0000 7fba2a59c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fba140035c0 0x7fba2c19f640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:22:28.466 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:22:28.468+0000 7fba2a59c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fba140035c0 0x7fba2c19f640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.191:51074/0 (socket says 172.21.15.191:51074) 2024-03-04T09:22:28.466 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:22:28.468+0000 7fba29d9b700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fba2c19fb80 0x7fba2c199710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.191:48508/0 (socket says 172.21.15.191:48508) 2024-03-04T09:22:28.467 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:22:28.468+0000 7fba2a59c700 1 -- 172.21.15.191:0/2271921604 learned_addr learned my addr 172.21.15.191:0/2271921604 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-03-04T09:22:28.467 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:22:28.469+0000 7fba29d9b700 1 -- 172.21.15.191:0/2271921604 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fba2c19fb80 msgr2=0x7fba2c199710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-03-04T09:22:28.467 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:22:28.469+0000 7fba29d9b700 1 -- 172.21.15.191:0/2271921604 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fba2c19fb80 msgr2=0x7fba2c199710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-03-04T09:22:28.467 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:22:28.469+0000 7fba29d9b700 1 --2- 172.21.15.191:0/2271921604 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fba2c19fb80 0x7fba2c199710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-04T09:22:28.467 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:22:28.469+0000 7fba29d9b700 1 --2- 172.21.15.191:0/2271921604 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fba2c19fb80 0x7fba2c199710 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T09:22:28.467 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:22:28.469+0000 7fba2a59c700 1 -- 172.21.15.191:0/2271921604 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fba2c19fb80 msgr2=0x7fba2c199710 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-04T09:22:28.467 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:22:28.469+0000 7fba2a59c700 1 --2- 172.21.15.191:0/2271921604 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fba2c19fb80 0x7fba2c199710 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:22:28.467 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:22:28.469+0000 7fba2a59c700 1 -- 172.21.15.191:0/2271921604 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba2c1a0100 con 0x7fba140035c0 2024-03-04T09:22:28.467 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:22:28.469+0000 7fba2a59c700 1 --2- 172.21.15.191:0/2271921604 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fba140035c0 0x7fba2c19f640 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fba1c001eb0 tx=0x7fba1c0057d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:22:28.467 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:22:28.469+0000 7fba1b7fe700 1 -- 172.21.15.191:0/2271921604 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 309+0+0 (secure 0 0 0) 0x7fba1c004ac0 con 0x7fba140035c0 2024-03-04T09:22:28.467 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:22:28.469+0000 7fba1b7fe700 1 -- 172.21.15.191:0/2271921604 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fba1c013e40 con 0x7fba140035c0 2024-03-04T09:22:28.467 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:22:28.469+0000 7fba30d23700 1 -- 172.21.15.191:0/2271921604 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba2c199cb0 con 0x7fba140035c0 2024-03-04T09:22:28.467 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:22:28.469+0000 7fba1b7fe700 1 -- 172.21.15.191:0/2271921604 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 309+0+0 (secure 0 0 0) 0x7fba1c011aa0 con 0x7fba140035c0 2024-03-04T09:22:28.467 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:22:28.469+0000 7fba30d23700 1 -- 172.21.15.191:0/2271921604 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba2c19a240 con 0x7fba140035c0 2024-03-04T09:22:28.468 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:22:28.470+0000 7fba1b7fe700 1 -- 172.21.15.191:0/2271921604 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 13) v1 ==== 45984+0+0 (secure 0 0 0) 0x7fba1c02c070 con 0x7fba140035c0 2024-03-04T09:22:28.468 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:22:28.471+0000 7fba1b7fe700 1 --2- 172.21.15.191:0/2271921604 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7fba0c038a30 0x7fba0c03aef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:22:28.468 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:22:28.471+0000 7fba1b7fe700 1 -- 172.21.15.191:0/2271921604 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fba1c02a070 con 0x7fba140035c0 2024-03-04T09:22:28.468 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:22:28.471+0000 7fba30d23700 1 -- 172.21.15.191:0/2271921604 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fba2c1021d0 con 0x7fba140035c0 2024-03-04T09:22:28.469 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:22:28.471+0000 7fba29d9b700 1 --2- 172.21.15.191:0/2271921604 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7fba0c038a30 0x7fba0c03aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:22:28.469 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:22:28.472+0000 7fba29d9b700 1 --2- 172.21.15.191:0/2271921604 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7fba0c038a30 0x7fba0c03aef0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fba20009850 tx=0x7fba20008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:22:28.472 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:22:28.476+0000 7fba1b7fe700 1 -- 172.21.15.191:0/2271921604 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fba1c023360 con 0x7fba140035c0 2024-03-04T09:22:28.748 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:22:28.752+0000 7fba30d23700 1 -- 172.21.15.191:0/2271921604 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fba2c19ace0 con 0x7fba140035c0 2024-03-04T09:22:28.749 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:22:28 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-04T09:22:28.749 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:22:28 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-04T09:22:28.749 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:22:28 smithi191 ceph-mon[36816]: mon.a calling monitor election 2024-03-04T09:22:28.749 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:22:28 smithi191 ceph-mon[36816]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T09:22:28.749 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:22:28 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-04T09:22:28.749 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:22:28 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-04T09:22:28.749 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:22:28 smithi191 ceph-mon[36816]: mon.c calling monitor election 2024-03-04T09:22:28.749 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:22:28 smithi191 ceph-mon[36816]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T09:22:28.749 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:22:28 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-04T09:22:28.750 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:22:28 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-04T09:22:28.750 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:22:28 smithi191 ceph-mon[36816]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T09:22:28.750 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:22:28 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-04T09:22:28.750 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:22:28 smithi191 ceph-mon[36816]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-03-04T09:22:28.750 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:22:28 smithi191 ceph-mon[36816]: monmap e2: 2 mons at {a=[v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0],c=[v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-04T09:22:28.750 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:22:28 smithi191 ceph-mon[36816]: fsmap 2024-03-04T09:22:28.750 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:22:28 smithi191 ceph-mon[36816]: osdmap e4: 0 total, 0 up, 0 in 2024-03-04T09:22:28.750 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:22:28 smithi191 ceph-mon[36816]: mgrmap e13: x(active, since 48s) 2024-03-04T09:22:28.750 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:22:28 smithi191 ceph-mon[36816]: overall HEALTH_OK 2024-03-04T09:22:28.750 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:22:28 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:22:28.750 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:22:28 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:22:28.751 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:22:28.753+0000 7fba1b7fe700 1 -- 172.21.15.191:0/2271921604 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) v1 ==== 95+0+1015 (secure 0 0 0) 0x7fba1c011d50 con 0x7fba140035c0 2024-03-04T09:22:28.751 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-03-04T09:22:28.751 INFO:teuthology.orchestra.run.smithi191.stdout:{"epoch":2,"fsid":"4e059f08-da08-11ee-95c6-87774f69a715","modified":"2024-03-04T09:22:23.336118Z","created":"2024-03-04T09:20:42.665781Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:3300","nonce":0},{"type":"v1","addr":"172.21.15.52:6789","nonce":0}]},"addr":"172.21.15.52:6789/0","public_addr":"172.21.15.52:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:3300","nonce":0},{"type":"v1","addr":"172.21.15.191:6789","nonce":0}]},"addr":"172.21.15.191:6789/0","public_addr":"172.21.15.191:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-03-04T09:22:28.752 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:22:28.755+0000 7fba30d23700 1 -- 172.21.15.191:0/2271921604 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7fba0c038a30 msgr2=0x7fba0c03aef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:22:28.752 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:22:28.755+0000 7fba30d23700 1 --2- 172.21.15.191:0/2271921604 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7fba0c038a30 0x7fba0c03aef0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fba20009850 tx=0x7fba20008040 comp rx=0 tx=0).stop 2024-03-04T09:22:28.752 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:22:28.755+0000 7fba30d23700 1 -- 172.21.15.191:0/2271921604 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fba140035c0 msgr2=0x7fba2c19f640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:22:28.752 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:22:28.755+0000 7fba30d23700 1 --2- 172.21.15.191:0/2271921604 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fba140035c0 0x7fba2c19f640 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fba1c001eb0 tx=0x7fba1c0057d0 comp rx=0 tx=0).stop 2024-03-04T09:22:28.752 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:22:28.755+0000 7fba30d23700 1 -- 172.21.15.191:0/2271921604 shutdown_connections 2024-03-04T09:22:28.752 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:22:28.755+0000 7fba30d23700 1 --2- 172.21.15.191:0/2271921604 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7fba0c038a30 0x7fba0c03aef0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:22:28.752 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:22:28.756+0000 7fba30d23700 1 --2- 172.21.15.191:0/2271921604 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fba140035c0 0x7fba2c19f640 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:22:28.752 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:22:28.756+0000 7fba30d23700 1 --2- 172.21.15.191:0/2271921604 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fba2c19fb80 0x7fba2c199710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:22:28.753 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:22:28.756+0000 7fba30d23700 1 -- 172.21.15.191:0/2271921604 >> 172.21.15.191:0/2271921604 conn(0x7fba2c0fc580 msgr2=0x7fba2c06a6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:22:28.753 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:22:28.756+0000 7fba30d23700 1 -- 172.21.15.191:0/2271921604 shutdown_connections 2024-03-04T09:22:28.753 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:22:28.756+0000 7fba30d23700 1 -- 172.21.15.191:0/2271921604 wait complete. 2024-03-04T09:22:28.753 INFO:teuthology.orchestra.run.smithi191.stderr:dumped monmap epoch 2 2024-03-04T09:22:28.835 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:22:28 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-04T09:22:28.835 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:22:28 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-04T09:22:28.835 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:22:28 smithi052 ceph-mon[32473]: mon.a calling monitor election 2024-03-04T09:22:28.835 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:22:28 smithi052 ceph-mon[32473]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T09:22:28.835 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:22:28 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-04T09:22:28.835 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:22:28 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-04T09:22:28.835 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:22:28 smithi052 ceph-mon[32473]: mon.c calling monitor election 2024-03-04T09:22:28.836 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:22:28 smithi052 ceph-mon[32473]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T09:22:28.836 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:22:28 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-04T09:22:28.836 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:22:28 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-04T09:22:28.836 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:22:28 smithi052 ceph-mon[32473]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T09:22:28.836 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:22:28 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-04T09:22:28.836 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:22:28 smithi052 ceph-mon[32473]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-03-04T09:22:28.836 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:22:28 smithi052 ceph-mon[32473]: monmap e2: 2 mons at {a=[v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0],c=[v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-04T09:22:28.836 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:22:28 smithi052 ceph-mon[32473]: fsmap 2024-03-04T09:22:28.836 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:22:28 smithi052 ceph-mon[32473]: osdmap e4: 0 total, 0 up, 0 in 2024-03-04T09:22:28.836 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:22:28 smithi052 ceph-mon[32473]: mgrmap e13: x(active, since 48s) 2024-03-04T09:22:28.836 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:22:28 smithi052 ceph-mon[32473]: overall HEALTH_OK 2024-03-04T09:22:28.836 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:22:28 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:22:28.836 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:22:28 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:22:29.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:22:29 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:22:29.587 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:22:29 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-04T09:22:29.587 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:22:29 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T09:22:29.587 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:22:29 smithi191 ceph-mon[36816]: Deploying daemon mon.b on smithi106 2024-03-04T09:22:29.587 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:22:29 smithi191 ceph-mon[36816]: from='client.? 172.21.15.191:0/2271921604' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-04T09:22:29.587 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:22:29 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-04T09:22:29.834 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:22:29 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:22:29.835 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:22:29 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-04T09:22:29.835 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:22:29 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T09:22:29.835 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:22:29 smithi052 ceph-mon[32473]: Deploying daemon mon.b on smithi106 2024-03-04T09:22:29.835 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:22:29 smithi052 ceph-mon[32473]: from='client.? 172.21.15.191:0/2271921604' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-04T09:22:29.835 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:22:29 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-04T09:22:30.320 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-03-04T09:22:30.320 DEBUG:teuthology.orchestra.run.smithi191:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4e059f08-da08-11ee-95c6-87774f69a715 -- ceph mon dump -f json 2024-03-04T09:22:30.559 INFO:teuthology.orchestra.run.smithi191.stderr:Inferring config /var/lib/ceph/4e059f08-da08-11ee-95c6-87774f69a715/mon.c/config 2024-03-04T09:22:30.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:22:30 smithi191 ceph-mon[36816]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T09:22:30.834 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:22:30 smithi052 ceph-mon[32473]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T09:22:32.014 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:22:32.016+0000 7fb63f805700 1 -- 172.21.15.191:0/7661238 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb638101e00 msgr2=0x7fb6381021e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:22:32.014 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:22:32.016+0000 7fb63f805700 1 --2- 172.21.15.191:0/7661238 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb638101e00 0x7fb6381021e0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fb628009910 tx=0x7fb628009c20 comp rx=0 tx=0).stop 2024-03-04T09:22:32.014 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:22:32.017+0000 7fb63f805700 1 -- 172.21.15.191:0/7661238 shutdown_connections 2024-03-04T09:22:32.014 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:22:32.017+0000 7fb63f805700 1 --2- 172.21.15.191:0/7661238 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb638101e00 0x7fb6381021e0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:22:32.014 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:22:32.017+0000 7fb63f805700 1 -- 172.21.15.191:0/7661238 >> 172.21.15.191:0/7661238 conn(0x7fb6380fd690 msgr2=0x7fb6380ffab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:22:32.015 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:22:32.017+0000 7fb63f805700 1 -- 172.21.15.191:0/7661238 shutdown_connections 2024-03-04T09:22:32.015 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:22:32.017+0000 7fb63f805700 1 -- 172.21.15.191:0/7661238 wait complete. 2024-03-04T09:22:32.015 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:22:32.017+0000 7fb63f805700 1 Processor -- start 2024-03-04T09:22:32.015 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:22:32.017+0000 7fb63f805700 1 -- start start 2024-03-04T09:22:32.015 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:22:32.017+0000 7fb63f805700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb638101e00 0x7fb6380743c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:22:32.015 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:22:32.017+0000 7fb63f805700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb638074900 0x7fb638074d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:22:32.015 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:22:32.017+0000 7fb63f805700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb63810c1d0 con 0x7fb638101e00 2024-03-04T09:22:32.015 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:22:32.017+0000 7fb63f805700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb63810c050 con 0x7fb638074900 2024-03-04T09:22:32.015 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:22:32.018+0000 7fb63cda0700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb638074900 0x7fb638074d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:22:32.015 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:22:32.018+0000 7fb63cda0700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb638074900 0x7fb638074d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.191:48514/0 (socket says 172.21.15.191:48514) 2024-03-04T09:22:32.015 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:22:32.018+0000 7fb63cda0700 1 -- 172.21.15.191:0/3498096939 learned_addr learned my addr 172.21.15.191:0/3498096939 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-03-04T09:22:32.015 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:22:32.018+0000 7fb63cda0700 1 -- 172.21.15.191:0/3498096939 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb638074900 msgr2=0x7fb638074d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-03-04T09:22:32.016 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:22:32.018+0000 7fb63cda0700 1 -- 172.21.15.191:0/3498096939 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb638074900 msgr2=0x7fb638074d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-03-04T09:22:32.016 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:22:32.018+0000 7fb63cda0700 1 --2- 172.21.15.191:0/3498096939 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb638074900 0x7fb638074d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-04T09:22:32.016 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:22:32.018+0000 7fb63d5a1700 1 --2- 172.21.15.191:0/3498096939 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb638101e00 0x7fb6380743c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:22:32.016 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:22:32.018+0000 7fb63cda0700 1 --2- 172.21.15.191:0/3498096939 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb638074900 0x7fb638074d80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T09:22:32.016 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:22:32.018+0000 7fb63d5a1700 1 -- 172.21.15.191:0/3498096939 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb638074900 msgr2=0x7fb638074d80 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-04T09:22:32.016 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:22:32.018+0000 7fb63d5a1700 1 --2- 172.21.15.191:0/3498096939 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb638074900 0x7fb638074d80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:22:32.016 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:22:32.018+0000 7fb63d5a1700 1 -- 172.21.15.191:0/3498096939 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb638076240 con 0x7fb638101e00 2024-03-04T09:22:32.017 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:22:32.018+0000 7fb63d5a1700 1 --2- 172.21.15.191:0/3498096939 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb638101e00 0x7fb6380743c0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fb628014000 tx=0x7fb628005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:22:32.017 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:22:32.018+0000 7fb62e7fc700 1 -- 172.21.15.191:0/3498096939 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 309+0+0 (secure 0 0 0) 0x7fb628005dc0 con 0x7fb638101e00 2024-03-04T09:22:32.017 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:22:32.019+0000 7fb62e7fc700 1 -- 172.21.15.191:0/3498096939 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb62800be40 con 0x7fb638101e00 2024-03-04T09:22:32.017 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:22:32.019+0000 7fb63f805700 1 -- 172.21.15.191:0/3498096939 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6381a3ac0 con 0x7fb638101e00 2024-03-04T09:22:32.018 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:22:32.019+0000 7fb62e7fc700 1 -- 172.21.15.191:0/3498096939 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 309+0+0 (secure 0 0 0) 0x7fb62800f460 con 0x7fb638101e00 2024-03-04T09:22:32.018 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:22:32.019+0000 7fb63f805700 1 -- 172.21.15.191:0/3498096939 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6381a4020 con 0x7fb638101e00 2024-03-04T09:22:32.018 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:22:32.019+0000 7fb63f805700 1 -- 172.21.15.191:0/3498096939 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb6381032e0 con 0x7fb638101e00 2024-03-04T09:22:32.018 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:22:32.019+0000 7fb62e7fc700 1 -- 172.21.15.191:0/3498096939 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 13) v1 ==== 45984+0+0 (secure 0 0 0) 0x7fb62802a070 con 0x7fb638101e00 2024-03-04T09:22:32.018 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:22:32.020+0000 7fb62e7fc700 1 --2- 172.21.15.191:0/3498096939 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7fb6240389e0 0x7fb62403aea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:22:32.018 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:22:32.020+0000 7fb62e7fc700 1 -- 172.21.15.191:0/3498096939 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fb62801e030 con 0x7fb638101e00 2024-03-04T09:22:32.018 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:22:32.020+0000 7fb63cda0700 1 --2- 172.21.15.191:0/3498096939 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7fb6240389e0 0x7fb62403aea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:22:32.018 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:22:32.020+0000 7fb63cda0700 1 --2- 172.21.15.191:0/3498096939 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7fb6240389e0 0x7fb62403aea0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fb634009180 tx=0x7fb634008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:22:32.021 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:22:32.025+0000 7fb62e7fc700 1 -- 172.21.15.191:0/3498096939 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fb628013070 con 0x7fb638101e00 2024-03-04T09:22:32.270 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:22:32.274+0000 7fb62e7fc700 1 -- 172.21.15.191:0/3498096939 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6280047d0 con 0x7fb638101e00 2024-03-04T09:22:32.296 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:22:32.300+0000 7fb63f805700 1 -- 172.21.15.191:0/3498096939 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fb6381a42f0 con 0x7fb638101e00 2024-03-04T09:22:32.381 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:22:32 smithi106 systemd[1]: Started Ceph mon.b for 4e059f08-da08-11ee-95c6-87774f69a715. 2024-03-04T09:22:37.302 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:22:37.306+0000 7fb62e7fc700 1 -- 172.21.15.191:0/3498096939 <== mon.0 v2:172.21.15.52:3300/0 8 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 3 v3) v1 ==== 95+0+1283 (secure 0 0 0) 0x7fb628031020 con 0x7fb638101e00 2024-03-04T09:22:37.303 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-03-04T09:22:37.303 INFO:teuthology.orchestra.run.smithi191.stdout:{"epoch":3,"fsid":"4e059f08-da08-11ee-95c6-87774f69a715","modified":"2024-03-04T09:22:32.273634Z","created":"2024-03-04T09:20:42.665781Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:3300","nonce":0},{"type":"v1","addr":"172.21.15.52:6789","nonce":0}]},"addr":"172.21.15.52:6789/0","public_addr":"172.21.15.52:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:3300","nonce":0},{"type":"v1","addr":"172.21.15.191:6789","nonce":0}]},"addr":"172.21.15.191:6789/0","public_addr":"172.21.15.191:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:3300","nonce":0},{"type":"v1","addr":"172.21.15.106:6789","nonce":0}]},"addr":"172.21.15.106:6789/0","public_addr":"172.21.15.106:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-03-04T09:22:37.305 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:22:37.309+0000 7fb63f805700 1 -- 172.21.15.191:0/3498096939 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7fb6240389e0 msgr2=0x7fb62403aea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:22:37.306 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:22:37.309+0000 7fb63f805700 1 --2- 172.21.15.191:0/3498096939 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7fb6240389e0 0x7fb62403aea0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fb634009180 tx=0x7fb634008040 comp rx=0 tx=0).stop 2024-03-04T09:22:37.306 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:22:37.309+0000 7fb63f805700 1 -- 172.21.15.191:0/3498096939 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb638101e00 msgr2=0x7fb6380743c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:22:37.306 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:22:37.309+0000 7fb63f805700 1 --2- 172.21.15.191:0/3498096939 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb638101e00 0x7fb6380743c0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fb628014000 tx=0x7fb628005030 comp rx=0 tx=0).stop 2024-03-04T09:22:37.306 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:22:37.309+0000 7fb63f805700 1 -- 172.21.15.191:0/3498096939 shutdown_connections 2024-03-04T09:22:37.306 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:22:37.309+0000 7fb63f805700 1 --2- 172.21.15.191:0/3498096939 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7fb6240389e0 0x7fb62403aea0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:22:37.306 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:22:37.309+0000 7fb63f805700 1 --2- 172.21.15.191:0/3498096939 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb638101e00 0x7fb6380743c0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:22:37.306 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:22:37.309+0000 7fb63f805700 1 --2- 172.21.15.191:0/3498096939 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb638074900 0x7fb638074d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:22:37.306 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:22:37.309+0000 7fb63f805700 1 -- 172.21.15.191:0/3498096939 >> 172.21.15.191:0/3498096939 conn(0x7fb6380fd690 msgr2=0x7fb6381037d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:22:37.306 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:22:37.309+0000 7fb63f805700 1 -- 172.21.15.191:0/3498096939 shutdown_connections 2024-03-04T09:22:37.306 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:22:37.309+0000 7fb63f805700 1 -- 172.21.15.191:0/3498096939 wait complete. 2024-03-04T09:22:37.307 INFO:teuthology.orchestra.run.smithi191.stderr:dumped monmap epoch 3 2024-03-04T09:22:37.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:22:37 smithi191 ceph-mon[36816]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T09:22:37.587 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:22:37 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-04T09:22:37.587 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:22:37 smithi191 ceph-mon[36816]: mon.c calling monitor election 2024-03-04T09:22:37.587 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:22:37 smithi191 ceph-mon[36816]: mon.a calling monitor election 2024-03-04T09:22:37.587 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:22:37 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-04T09:22:37.587 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:22:37 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-04T09:22:37.587 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:22:37 smithi191 ceph-mon[36816]: from='client.? 172.21.15.191:0/3498096939' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-04T09:22:37.587 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:22:37 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-04T09:22:37.587 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:22:37 smithi191 ceph-mon[36816]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T09:22:37.587 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:22:37 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-04T09:22:37.587 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:22:37 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-04T09:22:37.587 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:22:37 smithi191 ceph-mon[36816]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T09:22:37.587 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:22:37 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-04T09:22:37.588 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:22:37 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-04T09:22:37.588 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:22:37 smithi191 ceph-mon[36816]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-03-04T09:22:37.588 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:22:37 smithi191 ceph-mon[36816]: monmap e3: 3 mons at {a=[v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0],b=[v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0],c=[v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-04T09:22:37.588 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:22:37 smithi191 ceph-mon[36816]: fsmap 2024-03-04T09:22:37.588 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:22:37 smithi191 ceph-mon[36816]: osdmap e4: 0 total, 0 up, 0 in 2024-03-04T09:22:37.588 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:22:37 smithi191 ceph-mon[36816]: mgrmap e13: x(active, since 57s) 2024-03-04T09:22:37.588 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:22:37 smithi191 ceph-mon[36816]: Health check failed: 1/3 mons down, quorum a,c (MON_DOWN) 2024-03-04T09:22:37.588 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:22:37 smithi191 ceph-mon[36816]: Health detail: HEALTH_WARN 1/3 mons down, quorum a,c 2024-03-04T09:22:37.588 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:22:37 smithi191 ceph-mon[36816]: [WRN] MON_DOWN: 1/3 mons down, quorum a,c 2024-03-04T09:22:37.588 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:22:37 smithi191 ceph-mon[36816]: mon.b (rank 2) addr [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] is down (out of quorum) 2024-03-04T09:22:37.588 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:22:37 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:22:37.588 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:22:37 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:22:37.588 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:22:37 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:22:37.588 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:22:37 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:22:37.835 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:22:37 smithi052 ceph-mon[32473]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T09:22:37.835 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:22:37 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-04T09:22:37.835 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:22:37 smithi052 ceph-mon[32473]: mon.c calling monitor election 2024-03-04T09:22:37.835 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:22:37 smithi052 ceph-mon[32473]: mon.a calling monitor election 2024-03-04T09:22:37.835 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:22:37 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-04T09:22:37.835 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:22:37 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-04T09:22:37.836 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:22:37 smithi052 ceph-mon[32473]: from='client.? 172.21.15.191:0/3498096939' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-04T09:22:37.836 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:22:37 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-04T09:22:37.836 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:22:37 smithi052 ceph-mon[32473]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T09:22:37.836 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:22:37 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-04T09:22:37.836 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:22:37 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-04T09:22:37.836 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:22:37 smithi052 ceph-mon[32473]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T09:22:37.836 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:22:37 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-04T09:22:37.836 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:22:37 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-04T09:22:37.836 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:22:37 smithi052 ceph-mon[32473]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-03-04T09:22:37.836 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:22:37 smithi052 ceph-mon[32473]: monmap e3: 3 mons at {a=[v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0],b=[v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0],c=[v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-04T09:22:37.836 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:22:37 smithi052 ceph-mon[32473]: fsmap 2024-03-04T09:22:37.836 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:22:37 smithi052 ceph-mon[32473]: osdmap e4: 0 total, 0 up, 0 in 2024-03-04T09:22:37.836 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:22:37 smithi052 ceph-mon[32473]: mgrmap e13: x(active, since 57s) 2024-03-04T09:22:37.837 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:22:37 smithi052 ceph-mon[32473]: Health check failed: 1/3 mons down, quorum a,c (MON_DOWN) 2024-03-04T09:22:37.837 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:22:37 smithi052 ceph-mon[32473]: Health detail: HEALTH_WARN 1/3 mons down, quorum a,c 2024-03-04T09:22:37.837 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:22:37 smithi052 ceph-mon[32473]: [WRN] MON_DOWN: 1/3 mons down, quorum a,c 2024-03-04T09:22:37.837 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:22:37 smithi052 ceph-mon[32473]: mon.b (rank 2) addr [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] is down (out of quorum) 2024-03-04T09:22:37.837 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:22:37 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:22:37.837 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:22:37 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:22:37.837 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:22:37 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:22:37.837 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:22:37 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:22:37.879 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-03-04T09:22:37.879 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4e059f08-da08-11ee-95c6-87774f69a715 -- ceph config generate-minimal-conf 2024-03-04T09:22:38.131 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/4e059f08-da08-11ee-95c6-87774f69a715/mon.a/config 2024-03-04T09:22:38.383 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:22:38 smithi052 ceph-mon[32473]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T09:22:38.383 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:22:38 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T09:22:38.383 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:22:38 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-04T09:22:38.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:22:38 smithi191 ceph-mon[36816]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T09:22:38.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:22:38 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T09:22:38.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:22:38 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-04T09:22:39.631 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:22:39 smithi106 ceph-mon[36484]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T09:22:39.632 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:22:39 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-04T09:22:39.632 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:22:39 smithi106 ceph-mon[36484]: mon.c calling monitor election 2024-03-04T09:22:39.632 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:22:39 smithi106 ceph-mon[36484]: mon.a calling monitor election 2024-03-04T09:22:39.632 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:22:39 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-04T09:22:39.632 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:22:39 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-04T09:22:39.632 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:22:39 smithi106 ceph-mon[36484]: from='client.? 172.21.15.191:0/3498096939' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-04T09:22:39.632 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:22:39 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-04T09:22:39.632 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:22:39 smithi106 ceph-mon[36484]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T09:22:39.632 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:22:39 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-04T09:22:39.632 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:22:39 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-04T09:22:39.632 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:22:39 smithi106 ceph-mon[36484]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T09:22:39.633 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:22:39 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-04T09:22:39.633 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:22:39 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-04T09:22:39.633 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:22:39 smithi106 ceph-mon[36484]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-03-04T09:22:39.633 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:22:39 smithi106 ceph-mon[36484]: monmap e3: 3 mons at {a=[v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0],b=[v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0],c=[v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-04T09:22:39.633 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:22:39 smithi106 ceph-mon[36484]: fsmap 2024-03-04T09:22:39.633 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:22:39 smithi106 ceph-mon[36484]: osdmap e4: 0 total, 0 up, 0 in 2024-03-04T09:22:39.633 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:22:39 smithi106 ceph-mon[36484]: mgrmap e13: x(active, since 57s) 2024-03-04T09:22:39.633 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:22:39 smithi106 ceph-mon[36484]: Health check failed: 1/3 mons down, quorum a,c (MON_DOWN) 2024-03-04T09:22:39.633 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:22:39 smithi106 ceph-mon[36484]: Health detail: HEALTH_WARN 1/3 mons down, quorum a,c 2024-03-04T09:22:39.633 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:22:39 smithi106 ceph-mon[36484]: [WRN] MON_DOWN: 1/3 mons down, quorum a,c 2024-03-04T09:22:39.633 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:22:39 smithi106 ceph-mon[36484]: mon.b (rank 2) addr [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] is down (out of quorum) 2024-03-04T09:22:39.633 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:22:39 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:22:39.633 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:22:39 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:22:39.634 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:22:39 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:22:39.634 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:22:39 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:22:39.634 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:22:39 smithi106 ceph-mon[36484]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T09:22:39.634 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:22:39 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T09:22:39.634 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:22:39 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-04T09:22:39.703 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:22:39.704+0000 7fb4ae9c2700 1 -- 172.21.15.52:0/1249469741 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb4a8107a90 msgr2=0x7fb4a8107e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:22:39.703 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:22:39.704+0000 7fb4ae9c2700 1 --2- 172.21.15.52:0/1249469741 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb4a8107a90 0x7fb4a8107e70 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fb498009910 tx=0x7fb498009c20 comp rx=0 tx=0).stop 2024-03-04T09:22:39.703 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:22:39.704+0000 7fb4ae9c2700 1 -- 172.21.15.52:0/1249469741 shutdown_connections 2024-03-04T09:22:39.703 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:22:39.704+0000 7fb4ae9c2700 1 --2- 172.21.15.52:0/1249469741 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb4a8107a90 0x7fb4a8107e70 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:22:39.704 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:22:39.704+0000 7fb4ae9c2700 1 -- 172.21.15.52:0/1249469741 >> 172.21.15.52:0/1249469741 conn(0x7fb4a80fd580 msgr2=0x7fb4a80ff9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:22:39.704 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:22:39.704+0000 7fb4ae9c2700 1 -- 172.21.15.52:0/1249469741 shutdown_connections 2024-03-04T09:22:39.704 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:22:39.704+0000 7fb4ae9c2700 1 -- 172.21.15.52:0/1249469741 wait complete. 2024-03-04T09:22:39.704 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:22:39.705+0000 7fb4ae9c2700 1 Processor -- start 2024-03-04T09:22:39.704 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:22:39.705+0000 7fb4ae9c2700 1 -- start start 2024-03-04T09:22:39.704 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:22:39.705+0000 7fb4ae9c2700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb4a8107a90 0x7fb4a80743c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:22:39.704 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:22:39.705+0000 7fb4ae9c2700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb4a8074900 0x7fb4a8074d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:22:39.704 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:22:39.705+0000 7fb4ae9c2700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb4a81a39e0 0x7fb4a81a5dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:22:39.704 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:22:39.705+0000 7fb4ae9c2700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4a810c080 con 0x7fb4a8074900 2024-03-04T09:22:39.704 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:22:39.705+0000 7fb4ae9c2700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4a810bf00 con 0x7fb4a81a39e0 2024-03-04T09:22:39.704 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:22:39.705+0000 7fb4ae9c2700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4a810c200 con 0x7fb4a8107a90 2024-03-04T09:22:39.704 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:22:39.705+0000 7fb4a77fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb4a8074900 0x7fb4a8074d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:22:39.705 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:22:39.705+0000 7fb4a77fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb4a8074900 0x7fb4a8074d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:36766/0 (socket says 172.21.15.52:36766) 2024-03-04T09:22:39.705 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:22:39.705+0000 7fb4a77fe700 1 -- 172.21.15.52:0/3575982259 learned_addr learned my addr 172.21.15.52:0/3575982259 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-04T09:22:39.705 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:22:39.705+0000 7fb4a7fff700 1 --2- 172.21.15.52:0/3575982259 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb4a8107a90 0x7fb4a80743c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:22:39.705 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:22:39.705+0000 7fb4acf5f700 1 --2- 172.21.15.52:0/3575982259 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb4a81a39e0 0x7fb4a81a5dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:22:39.705 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:22:39.706+0000 7fb4a77fe700 1 -- 172.21.15.52:0/3575982259 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb4a8107a90 msgr2=0x7fb4a80743c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:22:39.705 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:22:39.706+0000 7fb4a77fe700 1 --2- 172.21.15.52:0/3575982259 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb4a8107a90 0x7fb4a80743c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:22:39.705 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:22:39.706+0000 7fb4a77fe700 1 -- 172.21.15.52:0/3575982259 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb4a81a39e0 msgr2=0x7fb4a81a5dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:22:39.705 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:22:39.706+0000 7fb4a77fe700 1 --2- 172.21.15.52:0/3575982259 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb4a81a39e0 0x7fb4a81a5dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:22:39.705 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:22:39.706+0000 7fb4a77fe700 1 -- 172.21.15.52:0/3575982259 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4a81a6500 con 0x7fb4a8074900 2024-03-04T09:22:39.705 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:22:39.706+0000 7fb4a77fe700 1 --2- 172.21.15.52:0/3575982259 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb4a8074900 0x7fb4a8074d80 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fb49000b780 tx=0x7fb49000ba90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:22:39.705 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:22:39.706+0000 7fb4a57fa700 1 -- 172.21.15.52:0/3575982259 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb490016070 con 0x7fb4a8074900 2024-03-04T09:22:39.705 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:22:39.706+0000 7fb4a57fa700 1 -- 172.21.15.52:0/3575982259 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb4900036a0 con 0x7fb4a8074900 2024-03-04T09:22:39.705 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:22:39.706+0000 7fb4a57fa700 1 -- 172.21.15.52:0/3575982259 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb490011af0 con 0x7fb4a8074900 2024-03-04T09:22:39.706 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:22:39.706+0000 7fb4ae9c2700 1 -- 172.21.15.52:0/3575982259 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4a81a67f0 con 0x7fb4a8074900 2024-03-04T09:22:39.706 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:22:39.706+0000 7fb4ae9c2700 1 -- 172.21.15.52:0/3575982259 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4a81a6ce0 con 0x7fb4a8074900 2024-03-04T09:22:39.706 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:22:39.707+0000 7fb4a57fa700 1 -- 172.21.15.52:0/3575982259 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 13) v1 ==== 45984+0+0 (secure 0 0 0) 0x7fb490019070 con 0x7fb4a8074900 2024-03-04T09:22:39.707 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:22:39.707+0000 7fb4ae9c2700 1 -- 172.21.15.52:0/3575982259 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb4a80636f0 con 0x7fb4a8074900 2024-03-04T09:22:39.707 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:22:39.707+0000 7fb4a57fa700 1 --2- 172.21.15.52:0/3575982259 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7fb4940390d0 0x7fb49403b590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:22:39.707 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:22:39.708+0000 7fb4a57fa700 1 -- 172.21.15.52:0/3575982259 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fb49004a7f0 con 0x7fb4a8074900 2024-03-04T09:22:39.707 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:22:39.708+0000 7fb4a7fff700 1 --2- 172.21.15.52:0/3575982259 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7fb4940390d0 0x7fb49403b590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:22:39.707 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:22:39.708+0000 7fb4a7fff700 1 --2- 172.21.15.52:0/3575982259 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7fb4940390d0 0x7fb49403b590 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fb498009750 tx=0x7fb49800b540 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:22:39.709 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:22:39.712+0000 7fb4a57fa700 1 -- 172.21.15.52:0/3575982259 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fb490018e10 con 0x7fb4a8074900 2024-03-04T09:22:39.915 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:22:39.918+0000 7fb4ae9c2700 1 -- 172.21.15.52:0/3575982259 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) v1 -- 0x7fb4a81a70e0 con 0x7fb4a8074900 2024-03-04T09:22:39.915 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:22:39.918+0000 7fb4a57fa700 1 -- 172.21.15.52:0/3575982259 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v9) v1 ==== 76+0+275 (secure 0 0 0) 0x7fb4900038c0 con 0x7fb4a8074900 2024-03-04T09:22:39.916 INFO:teuthology.orchestra.run.smithi052.stdout:# minimal ceph.conf for 4e059f08-da08-11ee-95c6-87774f69a715 2024-03-04T09:22:39.916 INFO:teuthology.orchestra.run.smithi052.stdout:[global] 2024-03-04T09:22:39.916 INFO:teuthology.orchestra.run.smithi052.stdout: fsid = 4e059f08-da08-11ee-95c6-87774f69a715 2024-03-04T09:22:39.916 INFO:teuthology.orchestra.run.smithi052.stdout: mon_host = [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] 2024-03-04T09:22:39.918 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:22:39.920+0000 7fb4ae9c2700 1 -- 172.21.15.52:0/3575982259 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7fb4940390d0 msgr2=0x7fb49403b590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:22:39.918 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:22:39.920+0000 7fb4ae9c2700 1 --2- 172.21.15.52:0/3575982259 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7fb4940390d0 0x7fb49403b590 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fb498009750 tx=0x7fb49800b540 comp rx=0 tx=0).stop 2024-03-04T09:22:39.918 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:22:39.920+0000 7fb4ae9c2700 1 -- 172.21.15.52:0/3575982259 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb4a8074900 msgr2=0x7fb4a8074d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:22:39.918 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:22:39.920+0000 7fb4ae9c2700 1 --2- 172.21.15.52:0/3575982259 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb4a8074900 0x7fb4a8074d80 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fb49000b780 tx=0x7fb49000ba90 comp rx=0 tx=0).stop 2024-03-04T09:22:39.918 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:22:39.921+0000 7fb4ae9c2700 1 -- 172.21.15.52:0/3575982259 shutdown_connections 2024-03-04T09:22:39.918 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:22:39.921+0000 7fb4ae9c2700 1 --2- 172.21.15.52:0/3575982259 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7fb4940390d0 0x7fb49403b590 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:22:39.918 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:22:39.921+0000 7fb4ae9c2700 1 --2- 172.21.15.52:0/3575982259 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb4a8107a90 0x7fb4a80743c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:22:39.918 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:22:39.921+0000 7fb4ae9c2700 1 --2- 172.21.15.52:0/3575982259 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb4a81a39e0 0x7fb4a81a5dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:22:39.918 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:22:39.921+0000 7fb4ae9c2700 1 --2- 172.21.15.52:0/3575982259 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb4a8074900 0x7fb4a8074d80 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:22:39.918 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:22:39.921+0000 7fb4ae9c2700 1 -- 172.21.15.52:0/3575982259 >> 172.21.15.52:0/3575982259 conn(0x7fb4a80fd580 msgr2=0x7fb4a8103590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:22:39.919 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:22:39.921+0000 7fb4ae9c2700 1 -- 172.21.15.52:0/3575982259 shutdown_connections 2024-03-04T09:22:39.919 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:22:39.921+0000 7fb4ae9c2700 1 -- 172.21.15.52:0/3575982259 wait complete. 2024-03-04T09:22:40.536 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:22:40 smithi052 ceph-mon[32473]: mon.b calling monitor election 2024-03-04T09:22:40.536 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:22:40 smithi052 ceph-mon[32473]: Updating smithi052:/etc/ceph/ceph.conf 2024-03-04T09:22:40.536 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:22:40 smithi052 ceph-mon[32473]: Updating smithi106:/etc/ceph/ceph.conf 2024-03-04T09:22:40.536 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:22:40 smithi052 ceph-mon[32473]: Updating smithi191:/etc/ceph/ceph.conf 2024-03-04T09:22:40.536 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:22:40 smithi052 ceph-mon[32473]: Updating smithi052:/var/lib/ceph/4e059f08-da08-11ee-95c6-87774f69a715/config/ceph.conf 2024-03-04T09:22:40.536 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:22:40 smithi052 ceph-mon[32473]: Updating smithi106:/var/lib/ceph/4e059f08-da08-11ee-95c6-87774f69a715/config/ceph.conf 2024-03-04T09:22:40.536 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:22:40 smithi052 ceph-mon[32473]: Updating smithi191:/var/lib/ceph/4e059f08-da08-11ee-95c6-87774f69a715/config/ceph.conf 2024-03-04T09:22:40.536 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:22:40 smithi052 ceph-mon[32473]: mon.b calling monitor election 2024-03-04T09:22:40.536 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:22:40 smithi052 ceph-mon[32473]: mon.c calling monitor election 2024-03-04T09:22:40.536 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:22:40 smithi052 ceph-mon[32473]: mon.a calling monitor election 2024-03-04T09:22:40.537 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:22:40 smithi052 ceph-mon[32473]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-03-04T09:22:40.537 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:22:40 smithi052 ceph-mon[32473]: monmap e3: 3 mons at {a=[v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0],b=[v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0],c=[v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-04T09:22:40.537 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:22:40 smithi052 ceph-mon[32473]: fsmap 2024-03-04T09:22:40.537 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:22:40 smithi052 ceph-mon[32473]: osdmap e4: 0 total, 0 up, 0 in 2024-03-04T09:22:40.537 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:22:40 smithi052 ceph-mon[32473]: mgrmap e13: x(active, since 59s) 2024-03-04T09:22:40.537 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:22:40 smithi052 ceph-mon[32473]: Health check cleared: MON_DOWN (was: 1/3 mons down, quorum a,c) 2024-03-04T09:22:40.537 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:22:40 smithi052 ceph-mon[32473]: Cluster is now healthy 2024-03-04T09:22:40.537 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:22:40 smithi052 ceph-mon[32473]: overall HEALTH_OK 2024-03-04T09:22:40.537 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:22:40 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:22:40.537 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:22:40 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:22:40.537 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:22:40 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:22:40.537 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:22:40 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:22:40.537 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:22:40 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:22:40.537 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:22:40 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:22:40.538 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:22:40 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:22:40.538 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:22:40 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:22:40.538 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:22:40 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:22:40.538 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:22:40 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-04T09:22:40.538 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:22:40 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-04T09:22:40.538 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:22:40 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T09:22:40.538 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:22:40 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:22:40.538 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:22:40 smithi052 ceph-mon[32473]: from='client.? 172.21.15.52:0/3575982259' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T09:22:40.563 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-03-04T09:22:40.563 DEBUG:teuthology.orchestra.run.smithi052:> set -ex 2024-03-04T09:22:40.563 DEBUG:teuthology.orchestra.run.smithi052:> sudo dd of=/etc/ceph/ceph.conf 2024-03-04T09:22:40.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:22:40 smithi191 ceph-mon[36816]: mon.b calling monitor election 2024-03-04T09:22:40.587 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:22:40 smithi191 ceph-mon[36816]: Updating smithi052:/etc/ceph/ceph.conf 2024-03-04T09:22:40.587 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:22:40 smithi191 ceph-mon[36816]: Updating smithi106:/etc/ceph/ceph.conf 2024-03-04T09:22:40.587 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:22:40 smithi191 ceph-mon[36816]: Updating smithi191:/etc/ceph/ceph.conf 2024-03-04T09:22:40.587 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:22:40 smithi191 ceph-mon[36816]: Updating smithi052:/var/lib/ceph/4e059f08-da08-11ee-95c6-87774f69a715/config/ceph.conf 2024-03-04T09:22:40.587 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:22:40 smithi191 ceph-mon[36816]: Updating smithi106:/var/lib/ceph/4e059f08-da08-11ee-95c6-87774f69a715/config/ceph.conf 2024-03-04T09:22:40.587 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:22:40 smithi191 ceph-mon[36816]: Updating smithi191:/var/lib/ceph/4e059f08-da08-11ee-95c6-87774f69a715/config/ceph.conf 2024-03-04T09:22:40.587 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:22:40 smithi191 ceph-mon[36816]: mon.b calling monitor election 2024-03-04T09:22:40.587 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:22:40 smithi191 ceph-mon[36816]: mon.c calling monitor election 2024-03-04T09:22:40.587 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:22:40 smithi191 ceph-mon[36816]: mon.a calling monitor election 2024-03-04T09:22:40.587 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:22:40 smithi191 ceph-mon[36816]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-03-04T09:22:40.587 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:22:40 smithi191 ceph-mon[36816]: monmap e3: 3 mons at {a=[v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0],b=[v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0],c=[v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-04T09:22:40.587 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:22:40 smithi191 ceph-mon[36816]: fsmap 2024-03-04T09:22:40.588 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:22:40 smithi191 ceph-mon[36816]: osdmap e4: 0 total, 0 up, 0 in 2024-03-04T09:22:40.588 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:22:40 smithi191 ceph-mon[36816]: mgrmap e13: x(active, since 59s) 2024-03-04T09:22:40.588 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:22:40 smithi191 ceph-mon[36816]: Health check cleared: MON_DOWN (was: 1/3 mons down, quorum a,c) 2024-03-04T09:22:40.588 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:22:40 smithi191 ceph-mon[36816]: Cluster is now healthy 2024-03-04T09:22:40.588 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:22:40 smithi191 ceph-mon[36816]: overall HEALTH_OK 2024-03-04T09:22:40.588 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:22:40 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:22:40.588 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:22:40 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:22:40.588 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:22:40 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:22:40.588 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:22:40 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:22:40.588 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:22:40 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:22:40.588 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:22:40 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:22:40.588 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:22:40 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:22:40.588 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:22:40 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:22:40.588 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:22:40 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:22:40.589 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:22:40 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-04T09:22:40.589 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:22:40 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-04T09:22:40.589 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:22:40 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T09:22:40.589 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:22:40 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:22:40.589 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:22:40 smithi191 ceph-mon[36816]: from='client.? 172.21.15.52:0/3575982259' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T09:22:40.594 DEBUG:teuthology.orchestra.run.smithi052:> set -ex 2024-03-04T09:22:40.594 DEBUG:teuthology.orchestra.run.smithi052:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-03-04T09:22:40.631 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:22:40 smithi106 ceph-mon[36484]: mon.b calling monitor election 2024-03-04T09:22:40.631 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:22:40 smithi106 ceph-mon[36484]: Updating smithi052:/etc/ceph/ceph.conf 2024-03-04T09:22:40.632 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:22:40 smithi106 ceph-mon[36484]: Updating smithi106:/etc/ceph/ceph.conf 2024-03-04T09:22:40.632 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:22:40 smithi106 ceph-mon[36484]: Updating smithi191:/etc/ceph/ceph.conf 2024-03-04T09:22:40.632 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:22:40 smithi106 ceph-mon[36484]: Updating smithi052:/var/lib/ceph/4e059f08-da08-11ee-95c6-87774f69a715/config/ceph.conf 2024-03-04T09:22:40.632 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:22:40 smithi106 ceph-mon[36484]: Updating smithi106:/var/lib/ceph/4e059f08-da08-11ee-95c6-87774f69a715/config/ceph.conf 2024-03-04T09:22:40.632 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:22:40 smithi106 ceph-mon[36484]: Updating smithi191:/var/lib/ceph/4e059f08-da08-11ee-95c6-87774f69a715/config/ceph.conf 2024-03-04T09:22:40.632 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:22:40 smithi106 ceph-mon[36484]: mon.b calling monitor election 2024-03-04T09:22:40.632 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:22:40 smithi106 ceph-mon[36484]: mon.c calling monitor election 2024-03-04T09:22:40.632 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:22:40 smithi106 ceph-mon[36484]: mon.a calling monitor election 2024-03-04T09:22:40.632 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:22:40 smithi106 ceph-mon[36484]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-03-04T09:22:40.632 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:22:40 smithi106 ceph-mon[36484]: monmap e3: 3 mons at {a=[v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0],b=[v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0],c=[v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-04T09:22:40.632 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:22:40 smithi106 ceph-mon[36484]: fsmap 2024-03-04T09:22:40.632 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:22:40 smithi106 ceph-mon[36484]: osdmap e4: 0 total, 0 up, 0 in 2024-03-04T09:22:40.633 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:22:40 smithi106 ceph-mon[36484]: mgrmap e13: x(active, since 59s) 2024-03-04T09:22:40.633 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:22:40 smithi106 ceph-mon[36484]: Health check cleared: MON_DOWN (was: 1/3 mons down, quorum a,c) 2024-03-04T09:22:40.633 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:22:40 smithi106 ceph-mon[36484]: Cluster is now healthy 2024-03-04T09:22:40.633 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:22:40 smithi106 ceph-mon[36484]: overall HEALTH_OK 2024-03-04T09:22:40.633 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:22:40 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:22:40.633 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:22:40 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:22:40.633 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:22:40 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:22:40.633 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:22:40 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:22:40.633 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:22:40 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:22:40.633 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:22:40 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:22:40.633 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:22:40 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:22:40.633 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:22:40 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:22:40.633 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:22:40 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:22:40.633 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:22:40 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-04T09:22:40.633 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:22:40 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-04T09:22:40.634 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:22:40 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T09:22:40.634 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:22:40 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:22:40.634 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:22:40 smithi106 ceph-mon[36484]: from='client.? 172.21.15.52:0/3575982259' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T09:22:40.661 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2024-03-04T09:22:40.661 DEBUG:teuthology.orchestra.run.smithi106:> sudo dd of=/etc/ceph/ceph.conf 2024-03-04T09:22:40.690 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2024-03-04T09:22:40.690 DEBUG:teuthology.orchestra.run.smithi106:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-03-04T09:22:40.757 DEBUG:teuthology.orchestra.run.smithi191:> set -ex 2024-03-04T09:22:40.757 DEBUG:teuthology.orchestra.run.smithi191:> sudo dd of=/etc/ceph/ceph.conf 2024-03-04T09:22:40.790 DEBUG:teuthology.orchestra.run.smithi191:> set -ex 2024-03-04T09:22:40.791 DEBUG:teuthology.orchestra.run.smithi191:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-03-04T09:22:40.860 INFO:tasks.cephadm:Adding mgr.x on smithi052 2024-03-04T09:22:40.861 INFO:tasks.cephadm:Adding mgr.y on smithi106 2024-03-04T09:22:40.861 INFO:tasks.cephadm:Adding mgr.z on smithi191 2024-03-04T09:22:40.861 DEBUG:teuthology.orchestra.run.smithi191:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4e059f08-da08-11ee-95c6-87774f69a715 -- ceph orch apply mgr '3;smithi052=x;smithi106=y;smithi191=z' 2024-03-04T09:22:41.135 INFO:teuthology.orchestra.run.smithi191.stderr:Inferring config /var/lib/ceph/4e059f08-da08-11ee-95c6-87774f69a715/mon.c/config 2024-03-04T09:22:41.567 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:22:41 smithi191 ceph-mon[36816]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T09:22:41.568 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:22:41 smithi191 ceph-mon[36816]: Reconfiguring mon.a (unknown last config time)... 2024-03-04T09:22:41.568 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:22:41 smithi191 ceph-mon[36816]: Reconfiguring daemon mon.a on smithi052 2024-03-04T09:22:41.568 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:22:41 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-04T09:22:41.584 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:22:41 smithi052 ceph-mon[32473]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T09:22:41.584 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:22:41 smithi052 ceph-mon[32473]: Reconfiguring mon.a (unknown last config time)... 2024-03-04T09:22:41.584 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:22:41 smithi052 ceph-mon[32473]: Reconfiguring daemon mon.a on smithi052 2024-03-04T09:22:41.584 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:22:41 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-04T09:22:41.631 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:22:41 smithi106 ceph-mon[36484]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T09:22:41.631 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:22:41 smithi106 ceph-mon[36484]: Reconfiguring mon.a (unknown last config time)... 2024-03-04T09:22:41.631 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:22:41 smithi106 ceph-mon[36484]: Reconfiguring daemon mon.a on smithi052 2024-03-04T09:22:41.631 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:22:41 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-04T09:22:42.740 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:22:42.741+0000 7f70dc659700 1 -- 172.21.15.191:0/1917889306 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f70d4103310 msgr2=0x7f70d4103730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:22:42.740 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:22:42.741+0000 7f70dc659700 1 --2- 172.21.15.191:0/1917889306 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f70d4103310 0x7f70d4103730 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f70c4009910 tx=0x7f70c4009c20 comp rx=0 tx=0).stop 2024-03-04T09:22:42.740 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:22:42.741+0000 7f70dc659700 1 -- 172.21.15.191:0/1917889306 shutdown_connections 2024-03-04T09:22:42.740 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:22:42.741+0000 7f70dc659700 1 --2- 172.21.15.191:0/1917889306 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f70d4103310 0x7f70d4103730 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:22:42.740 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:22:42.741+0000 7f70dc659700 1 -- 172.21.15.191:0/1917889306 >> 172.21.15.191:0/1917889306 conn(0x7f70d40fd1b0 msgr2=0x7f70d40ff610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:22:42.740 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:22:42.741+0000 7f70dc659700 1 -- 172.21.15.191:0/1917889306 shutdown_connections 2024-03-04T09:22:42.741 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:22:42.741+0000 7f70dc659700 1 -- 172.21.15.191:0/1917889306 wait complete. 2024-03-04T09:22:42.741 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:22:42.742+0000 7f70dc659700 1 Processor -- start 2024-03-04T09:22:42.741 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:22:42.742+0000 7f70dc659700 1 -- start start 2024-03-04T09:22:42.741 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:22:42.742+0000 7f70dc659700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f70d4103310 0x7f70d41975e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:22:42.741 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:22:42.742+0000 7f70dc659700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f70d4197b20 0x7f70d419cb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:22:42.741 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:22:42.742+0000 7f70dc659700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f70d4197fa0 0x7f70d419f120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:22:42.741 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:22:42.742+0000 7f70dc659700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70d4107dd0 con 0x7f70d4103310 2024-03-04T09:22:42.741 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:22:42.742+0000 7f70dc659700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70d4107c50 con 0x7f70d4197b20 2024-03-04T09:22:42.741 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:22:42.742+0000 7f70dc659700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70d4107f50 con 0x7f70d4197fa0 2024-03-04T09:22:42.741 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:22:42.742+0000 7f70dabf6700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f70d4197fa0 0x7f70d419f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:22:42.741 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:22:42.742+0000 7f70dabf6700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f70d4197fa0 0x7f70d419f120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.191:43434/0 (socket says 172.21.15.191:43434) 2024-03-04T09:22:42.741 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:22:42.742+0000 7f70dabf6700 1 -- 172.21.15.191:0/3616389347 learned_addr learned my addr 172.21.15.191:0/3616389347 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-03-04T09:22:42.742 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:22:42.742+0000 7f70da3f5700 1 --2- 172.21.15.191:0/3616389347 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f70d4103310 0x7f70d41975e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:22:42.742 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:22:42.742+0000 7f70d9bf4700 1 --2- 172.21.15.191:0/3616389347 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f70d4197b20 0x7f70d419cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:22:42.742 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:22:42.742+0000 7f70dabf6700 1 -- 172.21.15.191:0/3616389347 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f70d4197b20 msgr2=0x7f70d419cb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:22:42.742 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:22:42.742+0000 7f70dabf6700 1 --2- 172.21.15.191:0/3616389347 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f70d4197b20 0x7f70d419cb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:22:42.742 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:22:42.742+0000 7f70dabf6700 1 -- 172.21.15.191:0/3616389347 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f70d4103310 msgr2=0x7f70d41975e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:22:42.742 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:22:42.742+0000 7f70dabf6700 1 --2- 172.21.15.191:0/3616389347 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f70d4103310 0x7f70d41975e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:22:42.742 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:22:42.742+0000 7f70dabf6700 1 -- 172.21.15.191:0/3616389347 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70d419f9f0 con 0x7f70d4197fa0 2024-03-04T09:22:42.742 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:22:42.742+0000 7f70da3f5700 1 --2- 172.21.15.191:0/3616389347 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f70d4103310 0x7f70d41975e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T09:22:42.742 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:22:42.743+0000 7f70dabf6700 1 --2- 172.21.15.191:0/3616389347 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f70d4197fa0 0x7f70d419f120 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f70cc00ba50 tx=0x7f70cc00bd60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:22:42.742 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:22:42.743+0000 7f70cb7fe700 1 -- 172.21.15.191:0/3616389347 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70cc019070 con 0x7f70d4197fa0 2024-03-04T09:22:42.742 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:22:42.743+0000 7f70cb7fe700 1 -- 172.21.15.191:0/3616389347 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f70cc014440 con 0x7f70d4197fa0 2024-03-04T09:22:42.742 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:22:42.743+0000 7f70dc659700 1 -- 172.21.15.191:0/3616389347 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70d419fce0 con 0x7f70d4197fa0 2024-03-04T09:22:42.742 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:22:42.743+0000 7f70cb7fe700 1 -- 172.21.15.191:0/3616389347 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70cc0075b0 con 0x7f70d4197fa0 2024-03-04T09:22:42.742 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:22:42.743+0000 7f70dc659700 1 -- 172.21.15.191:0/3616389347 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70d41a0170 con 0x7f70d4197fa0 2024-03-04T09:22:42.743 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:22:42.744+0000 7f70cb7fe700 1 -- 172.21.15.191:0/3616389347 <== mon.1 v2:172.21.15.191:3300/0 4 ==== mgrmap(e 13) v1 ==== 45984+0+0 (secure 0 0 0) 0x7f70cc01c070 con 0x7f70d4197fa0 2024-03-04T09:22:42.744 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:22:42.744+0000 7f70dc659700 1 -- 172.21.15.191:0/3616389347 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f70d40636f0 con 0x7f70d4197fa0 2024-03-04T09:22:42.744 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:22:42.744+0000 7f70cb7fe700 1 --2- 172.21.15.191:0/3616389347 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f70c00390d0 0x7f70c003b590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:22:42.744 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:22:42.744+0000 7f70cb7fe700 1 -- 172.21.15.191:0/3616389347 <== mon.1 v2:172.21.15.191:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f70cc04b180 con 0x7f70d4197fa0 2024-03-04T09:22:42.744 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:22:42.744+0000 7f70da3f5700 1 --2- 172.21.15.191:0/3616389347 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f70c00390d0 0x7f70c003b590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:22:42.744 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:22:42.745+0000 7f70da3f5700 1 --2- 172.21.15.191:0/3616389347 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f70c00390d0 0x7f70c003b590 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f70c4005330 tx=0x7f70c401a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:22:42.746 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:22:42.749+0000 7f70cb7fe700 1 -- 172.21.15.191:0/3616389347 <== mon.1 v2:172.21.15.191:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f70cc014980 con 0x7f70d4197fa0 2024-03-04T09:22:42.835 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:22:42 smithi052 ceph-mon[32473]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T09:22:42.835 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:22:42 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:22:42.835 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:22:42 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:22:42.835 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:22:42 smithi052 ceph-mon[32473]: Reconfiguring mon.b (monmap changed)... 2024-03-04T09:22:42.835 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:22:42 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-04T09:22:42.835 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:22:42 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-04T09:22:42.835 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:22:42 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T09:22:42.835 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:22:42 smithi052 ceph-mon[32473]: Reconfiguring daemon mon.b on smithi106 2024-03-04T09:22:42.836 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:22:42 smithi191 ceph-mon[36816]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T09:22:42.836 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:22:42 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:22:42.836 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:22:42 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:22:42.836 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:22:42 smithi191 ceph-mon[36816]: Reconfiguring mon.b (monmap changed)... 2024-03-04T09:22:42.836 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:22:42 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-04T09:22:42.836 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:22:42 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-04T09:22:42.836 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:22:42 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T09:22:42.837 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:22:42 smithi191 ceph-mon[36816]: Reconfiguring daemon mon.b on smithi106 2024-03-04T09:22:42.881 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:22:42 smithi106 ceph-mon[36484]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T09:22:42.881 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:22:42 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:22:42.881 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:22:42 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:22:42.881 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:22:42 smithi106 ceph-mon[36484]: Reconfiguring mon.b (monmap changed)... 2024-03-04T09:22:42.882 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:22:42 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-04T09:22:42.882 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:22:42 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-04T09:22:42.882 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:22:42 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T09:22:42.882 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:22:42 smithi106 ceph-mon[36484]: Reconfiguring daemon mon.b on smithi106 2024-03-04T09:22:42.952 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:22:42.955+0000 7f70dc659700 1 -- 172.21.15.191:0/3616389347 --> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi052=x;smithi106=y;smithi191=z", "target": ["mon-mgr", ""]}) v1 -- 0x7f70d4062650 con 0x7f70c00390d0 2024-03-04T09:22:42.959 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:22:42.962+0000 7f70cb7fe700 1 -- 172.21.15.191:0/3616389347 <== mgr.14152 v2:172.21.15.52:6800/3036033923 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f70d4062650 con 0x7f70c00390d0 2024-03-04T09:22:42.959 INFO:teuthology.orchestra.run.smithi191.stdout:Scheduled mgr update... 2024-03-04T09:22:42.961 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:22:42.964+0000 7f70dc659700 1 -- 172.21.15.191:0/3616389347 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f70c00390d0 msgr2=0x7f70c003b590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:22:42.961 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:22:42.964+0000 7f70dc659700 1 --2- 172.21.15.191:0/3616389347 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f70c00390d0 0x7f70c003b590 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f70c4005330 tx=0x7f70c401a040 comp rx=0 tx=0).stop 2024-03-04T09:22:42.961 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:22:42.964+0000 7f70dc659700 1 -- 172.21.15.191:0/3616389347 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f70d4197fa0 msgr2=0x7f70d419f120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:22:42.961 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:22:42.964+0000 7f70dc659700 1 --2- 172.21.15.191:0/3616389347 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f70d4197fa0 0x7f70d419f120 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f70cc00ba50 tx=0x7f70cc00bd60 comp rx=0 tx=0).stop 2024-03-04T09:22:42.961 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:22:42.964+0000 7f70dc659700 1 -- 172.21.15.191:0/3616389347 shutdown_connections 2024-03-04T09:22:42.962 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:22:42.964+0000 7f70dc659700 1 --2- 172.21.15.191:0/3616389347 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f70c00390d0 0x7f70c003b590 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:22:42.962 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:22:42.964+0000 7f70dc659700 1 --2- 172.21.15.191:0/3616389347 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f70d4197fa0 0x7f70d419f120 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:22:42.962 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:22:42.964+0000 7f70dc659700 1 --2- 172.21.15.191:0/3616389347 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f70d4103310 0x7f70d41975e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:22:42.962 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:22:42.964+0000 7f70dc659700 1 --2- 172.21.15.191:0/3616389347 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f70d4197b20 0x7f70d419cb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:22:42.962 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:22:42.964+0000 7f70dc659700 1 -- 172.21.15.191:0/3616389347 >> 172.21.15.191:0/3616389347 conn(0x7f70d40fd1b0 msgr2=0x7f70d40fdef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:22:42.962 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:22:42.964+0000 7f70dc659700 1 -- 172.21.15.191:0/3616389347 shutdown_connections 2024-03-04T09:22:42.962 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:22:42.964+0000 7f70dc659700 1 -- 172.21.15.191:0/3616389347 wait complete. 2024-03-04T09:22:43.595 DEBUG:teuthology.orchestra.run.smithi106:mgr.y> sudo journalctl -f -n 0 -u ceph-4e059f08-da08-11ee-95c6-87774f69a715@mgr.y.service 2024-03-04T09:22:43.598 DEBUG:teuthology.orchestra.run.smithi191:mgr.z> sudo journalctl -f -n 0 -u ceph-4e059f08-da08-11ee-95c6-87774f69a715@mgr.z.service 2024-03-04T09:22:43.601 INFO:tasks.cephadm:Deploying OSDs... 2024-03-04T09:22:43.601 DEBUG:teuthology.orchestra.run.smithi052:> set -ex 2024-03-04T09:22:43.601 DEBUG:teuthology.orchestra.run.smithi052:> dd if=/scratch_devs of=/dev/stdout 2024-03-04T09:22:43.619 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-03-04T09:22:43.620 DEBUG:teuthology.orchestra.run.smithi052:> stat /dev/vg_nvme/lv_1 2024-03-04T09:22:43.677 INFO:teuthology.orchestra.run.smithi052.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-03-04T09:22:43.677 INFO:teuthology.orchestra.run.smithi052.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-04T09:22:43.677 INFO:teuthology.orchestra.run.smithi052.stdout:Device: 5h/5d Inode: 665 Links: 1 2024-03-04T09:22:43.677 INFO:teuthology.orchestra.run.smithi052.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-04T09:22:43.677 INFO:teuthology.orchestra.run.smithi052.stdout:Context: system_u:object_r:device_t:s0 2024-03-04T09:22:43.677 INFO:teuthology.orchestra.run.smithi052.stdout:Access: 2024-03-04 09:21:49.320257266 +0000 2024-03-04T09:22:43.677 INFO:teuthology.orchestra.run.smithi052.stdout:Modify: 2024-03-04 09:16:45.582157825 +0000 2024-03-04T09:22:43.677 INFO:teuthology.orchestra.run.smithi052.stdout:Change: 2024-03-04 09:16:45.582157825 +0000 2024-03-04T09:22:43.677 INFO:teuthology.orchestra.run.smithi052.stdout: Birth: 2024-03-04 09:16:45.582157825 +0000 2024-03-04T09:22:43.678 DEBUG:teuthology.orchestra.run.smithi052:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-03-04T09:22:43.747 INFO:teuthology.orchestra.run.smithi052.stderr:1+0 records in 2024-03-04T09:22:43.747 INFO:teuthology.orchestra.run.smithi052.stderr:1+0 records out 2024-03-04T09:22:43.747 INFO:teuthology.orchestra.run.smithi052.stderr:512 bytes copied, 0.000117356 s, 4.4 MB/s 2024-03-04T09:22:43.748 DEBUG:teuthology.orchestra.run.smithi052:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-03-04T09:22:43.806 DEBUG:teuthology.orchestra.run.smithi052:> stat /dev/vg_nvme/lv_2 2024-03-04T09:22:43.863 INFO:teuthology.orchestra.run.smithi052.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-03-04T09:22:43.863 INFO:teuthology.orchestra.run.smithi052.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-04T09:22:43.863 INFO:teuthology.orchestra.run.smithi052.stdout:Device: 5h/5d Inode: 686 Links: 1 2024-03-04T09:22:43.863 INFO:teuthology.orchestra.run.smithi052.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-04T09:22:43.863 INFO:teuthology.orchestra.run.smithi052.stdout:Context: system_u:object_r:device_t:s0 2024-03-04T09:22:43.863 INFO:teuthology.orchestra.run.smithi052.stdout:Access: 2024-03-04 09:21:49.360256739 +0000 2024-03-04T09:22:43.863 INFO:teuthology.orchestra.run.smithi052.stdout:Modify: 2024-03-04 09:16:45.904153533 +0000 2024-03-04T09:22:43.863 INFO:teuthology.orchestra.run.smithi052.stdout:Change: 2024-03-04 09:16:45.904153533 +0000 2024-03-04T09:22:43.863 INFO:teuthology.orchestra.run.smithi052.stdout: Birth: 2024-03-04 09:16:45.904153533 +0000 2024-03-04T09:22:43.864 DEBUG:teuthology.orchestra.run.smithi052:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-03-04T09:22:43.929 INFO:teuthology.orchestra.run.smithi052.stderr:1+0 records in 2024-03-04T09:22:43.929 INFO:teuthology.orchestra.run.smithi052.stderr:1+0 records out 2024-03-04T09:22:43.929 INFO:teuthology.orchestra.run.smithi052.stderr:512 bytes copied, 0.000110361 s, 4.6 MB/s 2024-03-04T09:22:43.931 DEBUG:teuthology.orchestra.run.smithi052:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-03-04T09:22:43.987 DEBUG:teuthology.orchestra.run.smithi052:> stat /dev/vg_nvme/lv_3 2024-03-04T09:22:44.042 INFO:teuthology.orchestra.run.smithi052.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-03-04T09:22:44.043 INFO:teuthology.orchestra.run.smithi052.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-04T09:22:44.043 INFO:teuthology.orchestra.run.smithi052.stdout:Device: 5h/5d Inode: 706 Links: 1 2024-03-04T09:22:44.043 INFO:teuthology.orchestra.run.smithi052.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-04T09:22:44.043 INFO:teuthology.orchestra.run.smithi052.stdout:Context: system_u:object_r:device_t:s0 2024-03-04T09:22:44.043 INFO:teuthology.orchestra.run.smithi052.stdout:Access: 2024-03-04 09:21:49.398256237 +0000 2024-03-04T09:22:44.043 INFO:teuthology.orchestra.run.smithi052.stdout:Modify: 2024-03-04 09:16:46.236149108 +0000 2024-03-04T09:22:44.043 INFO:teuthology.orchestra.run.smithi052.stdout:Change: 2024-03-04 09:16:46.236149108 +0000 2024-03-04T09:22:44.043 INFO:teuthology.orchestra.run.smithi052.stdout: Birth: 2024-03-04 09:16:46.236149108 +0000 2024-03-04T09:22:44.043 DEBUG:teuthology.orchestra.run.smithi052:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-03-04T09:22:44.085 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:22:43 smithi052 ceph-mon[32473]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi052=x;smithi106=y;smithi191=z", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T09:22:44.085 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:22:43 smithi052 ceph-mon[32473]: Saving service mgr spec with placement smithi052=x;smithi106=y;smithi191=z;count:3 2024-03-04T09:22:44.085 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:22:43 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:22:44.085 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:22:43 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:22:44.085 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:22:43 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:22:44.085 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:22:43 smithi052 ceph-mon[32473]: Reconfiguring mon.c (monmap changed)... 2024-03-04T09:22:44.085 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:22:43 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-04T09:22:44.085 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:22:43 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-04T09:22:44.086 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:22:43 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T09:22:44.086 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:22:43 smithi052 ceph-mon[32473]: Reconfiguring daemon mon.c on smithi191 2024-03-04T09:22:44.086 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:22:43 smithi191 ceph-mon[36816]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi052=x;smithi106=y;smithi191=z", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T09:22:44.086 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:22:43 smithi191 ceph-mon[36816]: Saving service mgr spec with placement smithi052=x;smithi106=y;smithi191=z;count:3 2024-03-04T09:22:44.086 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:22:43 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:22:44.086 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:22:43 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:22:44.086 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:22:43 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:22:44.086 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:22:43 smithi191 ceph-mon[36816]: Reconfiguring mon.c (monmap changed)... 2024-03-04T09:22:44.086 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:22:43 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-04T09:22:44.087 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:22:43 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-04T09:22:44.087 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:22:43 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T09:22:44.087 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:22:43 smithi191 ceph-mon[36816]: Reconfiguring daemon mon.c on smithi191 2024-03-04T09:22:44.109 INFO:teuthology.orchestra.run.smithi052.stderr:1+0 records in 2024-03-04T09:22:44.109 INFO:teuthology.orchestra.run.smithi052.stderr:1+0 records out 2024-03-04T09:22:44.109 INFO:teuthology.orchestra.run.smithi052.stderr:512 bytes copied, 0.00010894 s, 4.7 MB/s 2024-03-04T09:22:44.110 DEBUG:teuthology.orchestra.run.smithi052:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-03-04T09:22:44.166 DEBUG:teuthology.orchestra.run.smithi052:> stat /dev/vg_nvme/lv_4 2024-03-04T09:22:44.221 INFO:teuthology.orchestra.run.smithi052.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-03-04T09:22:44.222 INFO:teuthology.orchestra.run.smithi052.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-04T09:22:44.222 INFO:teuthology.orchestra.run.smithi052.stdout:Device: 5h/5d Inode: 725 Links: 1 2024-03-04T09:22:44.222 INFO:teuthology.orchestra.run.smithi052.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-04T09:22:44.222 INFO:teuthology.orchestra.run.smithi052.stdout:Context: system_u:object_r:device_t:s0 2024-03-04T09:22:44.222 INFO:teuthology.orchestra.run.smithi052.stdout:Access: 2024-03-04 09:21:49.432255789 +0000 2024-03-04T09:22:44.222 INFO:teuthology.orchestra.run.smithi052.stdout:Modify: 2024-03-04 09:16:46.583144482 +0000 2024-03-04T09:22:44.222 INFO:teuthology.orchestra.run.smithi052.stdout:Change: 2024-03-04 09:16:46.583144482 +0000 2024-03-04T09:22:44.222 INFO:teuthology.orchestra.run.smithi052.stdout: Birth: 2024-03-04 09:16:46.583144482 +0000 2024-03-04T09:22:44.222 DEBUG:teuthology.orchestra.run.smithi052:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-03-04T09:22:44.287 INFO:teuthology.orchestra.run.smithi052.stderr:1+0 records in 2024-03-04T09:22:44.287 INFO:teuthology.orchestra.run.smithi052.stderr:1+0 records out 2024-03-04T09:22:44.287 INFO:teuthology.orchestra.run.smithi052.stderr:512 bytes copied, 0.000101668 s, 5.0 MB/s 2024-03-04T09:22:44.289 DEBUG:teuthology.orchestra.run.smithi052:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-03-04T09:22:44.343 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2024-03-04T09:22:44.344 DEBUG:teuthology.orchestra.run.smithi106:> dd if=/scratch_devs of=/dev/stdout 2024-03-04T09:22:44.362 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-03-04T09:22:44.362 DEBUG:teuthology.orchestra.run.smithi106:> stat /dev/vg_nvme/lv_1 2024-03-04T09:22:44.381 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:22:43 smithi106 ceph-mon[36484]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi052=x;smithi106=y;smithi191=z", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T09:22:44.381 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:22:43 smithi106 ceph-mon[36484]: Saving service mgr spec with placement smithi052=x;smithi106=y;smithi191=z;count:3 2024-03-04T09:22:44.381 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:22:43 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:22:44.381 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:22:43 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:22:44.382 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:22:43 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:22:44.382 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:22:43 smithi106 ceph-mon[36484]: Reconfiguring mon.c (monmap changed)... 2024-03-04T09:22:44.382 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:22:43 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-04T09:22:44.382 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:22:43 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-04T09:22:44.382 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:22:43 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T09:22:44.382 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:22:43 smithi106 ceph-mon[36484]: Reconfiguring daemon mon.c on smithi191 2024-03-04T09:22:44.399 INFO:teuthology.orchestra.run.smithi106.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-03-04T09:22:44.399 INFO:teuthology.orchestra.run.smithi106.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-04T09:22:44.399 INFO:teuthology.orchestra.run.smithi106.stdout:Device: 5h/5d Inode: 661 Links: 1 2024-03-04T09:22:44.399 INFO:teuthology.orchestra.run.smithi106.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-04T09:22:44.399 INFO:teuthology.orchestra.run.smithi106.stdout:Context: system_u:object_r:device_t:s0 2024-03-04T09:22:44.399 INFO:teuthology.orchestra.run.smithi106.stdout:Access: 2024-03-04 09:22:31.758812248 +0000 2024-03-04T09:22:44.399 INFO:teuthology.orchestra.run.smithi106.stdout:Modify: 2024-03-04 09:17:12.907371433 +0000 2024-03-04T09:22:44.399 INFO:teuthology.orchestra.run.smithi106.stdout:Change: 2024-03-04 09:17:12.907371433 +0000 2024-03-04T09:22:44.399 INFO:teuthology.orchestra.run.smithi106.stdout: Birth: 2024-03-04 09:17:12.907371433 +0000 2024-03-04T09:22:44.400 DEBUG:teuthology.orchestra.run.smithi106:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-03-04T09:22:44.468 INFO:teuthology.orchestra.run.smithi106.stderr:1+0 records in 2024-03-04T09:22:44.468 INFO:teuthology.orchestra.run.smithi106.stderr:1+0 records out 2024-03-04T09:22:44.468 INFO:teuthology.orchestra.run.smithi106.stderr:512 bytes copied, 0.000291003 s, 1.8 MB/s 2024-03-04T09:22:44.470 DEBUG:teuthology.orchestra.run.smithi106:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-03-04T09:22:44.527 DEBUG:teuthology.orchestra.run.smithi106:> stat /dev/vg_nvme/lv_2 2024-03-04T09:22:44.584 INFO:teuthology.orchestra.run.smithi106.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-03-04T09:22:44.585 INFO:teuthology.orchestra.run.smithi106.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-04T09:22:44.585 INFO:teuthology.orchestra.run.smithi106.stdout:Device: 5h/5d Inode: 682 Links: 1 2024-03-04T09:22:44.585 INFO:teuthology.orchestra.run.smithi106.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-04T09:22:44.585 INFO:teuthology.orchestra.run.smithi106.stdout:Context: system_u:object_r:device_t:s0 2024-03-04T09:22:44.585 INFO:teuthology.orchestra.run.smithi106.stdout:Access: 2024-03-04 09:22:31.758812248 +0000 2024-03-04T09:22:44.585 INFO:teuthology.orchestra.run.smithi106.stdout:Modify: 2024-03-04 09:17:13.246363250 +0000 2024-03-04T09:22:44.585 INFO:teuthology.orchestra.run.smithi106.stdout:Change: 2024-03-04 09:17:13.246363250 +0000 2024-03-04T09:22:44.585 INFO:teuthology.orchestra.run.smithi106.stdout: Birth: 2024-03-04 09:17:13.246363250 +0000 2024-03-04T09:22:44.585 DEBUG:teuthology.orchestra.run.smithi106:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-03-04T09:22:44.653 INFO:teuthology.orchestra.run.smithi106.stderr:1+0 records in 2024-03-04T09:22:44.653 INFO:teuthology.orchestra.run.smithi106.stderr:1+0 records out 2024-03-04T09:22:44.653 INFO:teuthology.orchestra.run.smithi106.stderr:512 bytes copied, 0.000351372 s, 1.5 MB/s 2024-03-04T09:22:44.654 DEBUG:teuthology.orchestra.run.smithi106:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-03-04T09:22:44.710 DEBUG:teuthology.orchestra.run.smithi106:> stat /dev/vg_nvme/lv_3 2024-03-04T09:22:44.766 INFO:teuthology.orchestra.run.smithi106.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-03-04T09:22:44.766 INFO:teuthology.orchestra.run.smithi106.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-04T09:22:44.766 INFO:teuthology.orchestra.run.smithi106.stdout:Device: 5h/5d Inode: 699 Links: 1 2024-03-04T09:22:44.766 INFO:teuthology.orchestra.run.smithi106.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-04T09:22:44.766 INFO:teuthology.orchestra.run.smithi106.stdout:Context: system_u:object_r:device_t:s0 2024-03-04T09:22:44.767 INFO:teuthology.orchestra.run.smithi106.stdout:Access: 2024-03-04 09:22:31.758812248 +0000 2024-03-04T09:22:44.767 INFO:teuthology.orchestra.run.smithi106.stdout:Modify: 2024-03-04 09:17:13.604354609 +0000 2024-03-04T09:22:44.767 INFO:teuthology.orchestra.run.smithi106.stdout:Change: 2024-03-04 09:17:13.604354609 +0000 2024-03-04T09:22:44.767 INFO:teuthology.orchestra.run.smithi106.stdout: Birth: 2024-03-04 09:17:13.604354609 +0000 2024-03-04T09:22:44.767 DEBUG:teuthology.orchestra.run.smithi106:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-03-04T09:22:44.833 INFO:teuthology.orchestra.run.smithi106.stderr:1+0 records in 2024-03-04T09:22:44.833 INFO:teuthology.orchestra.run.smithi106.stderr:1+0 records out 2024-03-04T09:22:44.833 INFO:teuthology.orchestra.run.smithi106.stderr:512 bytes copied, 0.00020849 s, 2.5 MB/s 2024-03-04T09:22:44.834 DEBUG:teuthology.orchestra.run.smithi106:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-03-04T09:22:44.891 DEBUG:teuthology.orchestra.run.smithi106:> stat /dev/vg_nvme/lv_4 2024-03-04T09:22:44.946 INFO:teuthology.orchestra.run.smithi106.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-03-04T09:22:44.947 INFO:teuthology.orchestra.run.smithi106.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-04T09:22:44.947 INFO:teuthology.orchestra.run.smithi106.stdout:Device: 5h/5d Inode: 717 Links: 1 2024-03-04T09:22:44.947 INFO:teuthology.orchestra.run.smithi106.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-04T09:22:44.947 INFO:teuthology.orchestra.run.smithi106.stdout:Context: system_u:object_r:device_t:s0 2024-03-04T09:22:44.947 INFO:teuthology.orchestra.run.smithi106.stdout:Access: 2024-03-04 09:22:31.759812225 +0000 2024-03-04T09:22:44.947 INFO:teuthology.orchestra.run.smithi106.stdout:Modify: 2024-03-04 09:17:13.938346546 +0000 2024-03-04T09:22:44.947 INFO:teuthology.orchestra.run.smithi106.stdout:Change: 2024-03-04 09:17:13.938346546 +0000 2024-03-04T09:22:44.947 INFO:teuthology.orchestra.run.smithi106.stdout: Birth: 2024-03-04 09:17:13.938346546 +0000 2024-03-04T09:22:44.947 DEBUG:teuthology.orchestra.run.smithi106:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-03-04T09:22:45.013 INFO:teuthology.orchestra.run.smithi106.stderr:1+0 records in 2024-03-04T09:22:45.013 INFO:teuthology.orchestra.run.smithi106.stderr:1+0 records out 2024-03-04T09:22:45.014 INFO:teuthology.orchestra.run.smithi106.stderr:512 bytes copied, 0.000197383 s, 2.6 MB/s 2024-03-04T09:22:45.014 DEBUG:teuthology.orchestra.run.smithi106:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-03-04T09:22:45.071 DEBUG:teuthology.orchestra.run.smithi191:> set -ex 2024-03-04T09:22:45.071 DEBUG:teuthology.orchestra.run.smithi191:> dd if=/scratch_devs of=/dev/stdout 2024-03-04T09:22:45.090 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-03-04T09:22:45.090 DEBUG:teuthology.orchestra.run.smithi191:> stat /dev/vg_nvme/lv_1 2024-03-04T09:22:45.131 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:22:45 smithi106 ceph-mon[36484]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T09:22:45.131 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:22:45 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:22:45.132 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:22:45 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:22:45.132 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:22:45 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T09:22:45.147 INFO:teuthology.orchestra.run.smithi191.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-03-04T09:22:45.147 INFO:teuthology.orchestra.run.smithi191.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-04T09:22:45.147 INFO:teuthology.orchestra.run.smithi191.stdout:Device: 5h/5d Inode: 661 Links: 1 2024-03-04T09:22:45.147 INFO:teuthology.orchestra.run.smithi191.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-04T09:22:45.147 INFO:teuthology.orchestra.run.smithi191.stdout:Context: system_u:object_r:device_t:s0 2024-03-04T09:22:45.147 INFO:teuthology.orchestra.run.smithi191.stdout:Access: 2024-03-04 09:22:22.340302433 +0000 2024-03-04T09:22:45.147 INFO:teuthology.orchestra.run.smithi191.stdout:Modify: 2024-03-04 09:16:54.123423972 +0000 2024-03-04T09:22:45.147 INFO:teuthology.orchestra.run.smithi191.stdout:Change: 2024-03-04 09:16:54.123423972 +0000 2024-03-04T09:22:45.147 INFO:teuthology.orchestra.run.smithi191.stdout: Birth: 2024-03-04 09:16:54.123423972 +0000 2024-03-04T09:22:45.148 DEBUG:teuthology.orchestra.run.smithi191:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-03-04T09:22:45.215 INFO:teuthology.orchestra.run.smithi191.stderr:1+0 records in 2024-03-04T09:22:45.216 INFO:teuthology.orchestra.run.smithi191.stderr:1+0 records out 2024-03-04T09:22:45.216 INFO:teuthology.orchestra.run.smithi191.stderr:512 bytes copied, 0.000288099 s, 1.8 MB/s 2024-03-04T09:22:45.217 DEBUG:teuthology.orchestra.run.smithi191:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-03-04T09:22:45.274 DEBUG:teuthology.orchestra.run.smithi191:> stat /dev/vg_nvme/lv_2 2024-03-04T09:22:45.329 INFO:teuthology.orchestra.run.smithi191.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-03-04T09:22:45.329 INFO:teuthology.orchestra.run.smithi191.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-04T09:22:45.329 INFO:teuthology.orchestra.run.smithi191.stdout:Device: 5h/5d Inode: 682 Links: 1 2024-03-04T09:22:45.329 INFO:teuthology.orchestra.run.smithi191.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-04T09:22:45.329 INFO:teuthology.orchestra.run.smithi191.stdout:Context: system_u:object_r:device_t:s0 2024-03-04T09:22:45.329 INFO:teuthology.orchestra.run.smithi191.stdout:Access: 2024-03-04 09:22:22.340302433 +0000 2024-03-04T09:22:45.329 INFO:teuthology.orchestra.run.smithi191.stdout:Modify: 2024-03-04 09:16:54.458417905 +0000 2024-03-04T09:22:45.329 INFO:teuthology.orchestra.run.smithi191.stdout:Change: 2024-03-04 09:16:54.458417905 +0000 2024-03-04T09:22:45.329 INFO:teuthology.orchestra.run.smithi191.stdout: Birth: 2024-03-04 09:16:54.458417905 +0000 2024-03-04T09:22:45.330 DEBUG:teuthology.orchestra.run.smithi191:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-03-04T09:22:45.334 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:22:45 smithi052 ceph-mon[32473]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T09:22:45.335 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:22:45 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:22:45.335 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:22:45 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:22:45.335 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:22:45 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T09:22:45.336 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:22:45 smithi191 ceph-mon[36816]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T09:22:45.336 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:22:45 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:22:45.336 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:22:45 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:22:45.336 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:22:45 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T09:22:45.362 INFO:teuthology.orchestra.run.smithi191.stderr:1+0 records in 2024-03-04T09:22:45.362 INFO:teuthology.orchestra.run.smithi191.stderr:1+0 records out 2024-03-04T09:22:45.362 INFO:teuthology.orchestra.run.smithi191.stderr:512 bytes copied, 0.000221167 s, 2.3 MB/s 2024-03-04T09:22:45.363 DEBUG:teuthology.orchestra.run.smithi191:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-03-04T09:22:45.419 DEBUG:teuthology.orchestra.run.smithi191:> stat /dev/vg_nvme/lv_3 2024-03-04T09:22:45.473 INFO:teuthology.orchestra.run.smithi191.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-03-04T09:22:45.474 INFO:teuthology.orchestra.run.smithi191.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-04T09:22:45.474 INFO:teuthology.orchestra.run.smithi191.stdout:Device: 5h/5d Inode: 700 Links: 1 2024-03-04T09:22:45.474 INFO:teuthology.orchestra.run.smithi191.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-04T09:22:45.474 INFO:teuthology.orchestra.run.smithi191.stdout:Context: system_u:object_r:device_t:s0 2024-03-04T09:22:45.474 INFO:teuthology.orchestra.run.smithi191.stdout:Access: 2024-03-04 09:22:22.341302416 +0000 2024-03-04T09:22:45.474 INFO:teuthology.orchestra.run.smithi191.stdout:Modify: 2024-03-04 09:16:54.803411657 +0000 2024-03-04T09:22:45.474 INFO:teuthology.orchestra.run.smithi191.stdout:Change: 2024-03-04 09:16:54.803411657 +0000 2024-03-04T09:22:45.474 INFO:teuthology.orchestra.run.smithi191.stdout: Birth: 2024-03-04 09:16:54.803411657 +0000 2024-03-04T09:22:45.474 DEBUG:teuthology.orchestra.run.smithi191:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-03-04T09:22:45.540 INFO:teuthology.orchestra.run.smithi191.stderr:1+0 records in 2024-03-04T09:22:45.540 INFO:teuthology.orchestra.run.smithi191.stderr:1+0 records out 2024-03-04T09:22:45.540 INFO:teuthology.orchestra.run.smithi191.stderr:512 bytes copied, 0.000201866 s, 2.5 MB/s 2024-03-04T09:22:45.541 DEBUG:teuthology.orchestra.run.smithi191:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-03-04T09:22:45.596 DEBUG:teuthology.orchestra.run.smithi191:> stat /dev/vg_nvme/lv_4 2024-03-04T09:22:45.652 INFO:teuthology.orchestra.run.smithi191.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-03-04T09:22:45.652 INFO:teuthology.orchestra.run.smithi191.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-04T09:22:45.653 INFO:teuthology.orchestra.run.smithi191.stdout:Device: 5h/5d Inode: 718 Links: 1 2024-03-04T09:22:45.653 INFO:teuthology.orchestra.run.smithi191.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-04T09:22:45.653 INFO:teuthology.orchestra.run.smithi191.stdout:Context: system_u:object_r:device_t:s0 2024-03-04T09:22:45.653 INFO:teuthology.orchestra.run.smithi191.stdout:Access: 2024-03-04 09:22:22.341302416 +0000 2024-03-04T09:22:45.653 INFO:teuthology.orchestra.run.smithi191.stdout:Modify: 2024-03-04 09:16:55.160405192 +0000 2024-03-04T09:22:45.653 INFO:teuthology.orchestra.run.smithi191.stdout:Change: 2024-03-04 09:16:55.160405192 +0000 2024-03-04T09:22:45.653 INFO:teuthology.orchestra.run.smithi191.stdout: Birth: 2024-03-04 09:16:55.160405192 +0000 2024-03-04T09:22:45.653 DEBUG:teuthology.orchestra.run.smithi191:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-03-04T09:22:45.719 INFO:teuthology.orchestra.run.smithi191.stderr:1+0 records in 2024-03-04T09:22:45.719 INFO:teuthology.orchestra.run.smithi191.stderr:1+0 records out 2024-03-04T09:22:45.719 INFO:teuthology.orchestra.run.smithi191.stderr:512 bytes copied, 0.000185731 s, 2.8 MB/s 2024-03-04T09:22:45.720 DEBUG:teuthology.orchestra.run.smithi191:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-03-04T09:22:45.775 INFO:tasks.cephadm:Deploying osd.0 on smithi052 with /dev/vg_nvme/lv_4... 2024-03-04T09:22:45.776 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4e059f08-da08-11ee-95c6-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-03-04T09:22:46.020 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/4e059f08-da08-11ee-95c6-87774f69a715/mon.a/config 2024-03-04T09:22:46.085 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:22:46 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T09:22:46.085 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:22:46 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T09:22:46.085 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:22:46 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:22:46.085 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:22:46 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-04T09:22:46.085 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:22:46 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-03-04T09:22:46.085 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:22:46 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-04T09:22:46.085 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:22:46 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T09:22:46.085 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:22:46 smithi052 ceph-mon[32473]: Deploying daemon mgr.z on smithi191 2024-03-04T09:22:46.336 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:22:46 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T09:22:46.336 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:22:46 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T09:22:46.337 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:22:46 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:22:46.337 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:22:46 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-04T09:22:46.337 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:22:46 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-03-04T09:22:46.337 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:22:46 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-04T09:22:46.337 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:22:46 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T09:22:46.337 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:22:46 smithi191 ceph-mon[36816]: Deploying daemon mgr.z on smithi191 2024-03-04T09:22:46.381 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:22:46 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T09:22:46.381 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:22:46 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T09:22:46.381 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:22:46 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:22:46.381 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:22:46 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-04T09:22:46.381 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:22:46 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-03-04T09:22:46.381 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:22:46 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-04T09:22:46.381 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:22:46 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T09:22:46.381 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:22:46 smithi106 ceph-mon[36484]: Deploying daemon mgr.z on smithi191 2024-03-04T09:22:47.290 INFO:journalctl@ceph.mgr.z.smithi191.stdout:Mar 04 09:22:47 smithi191 systemd[1]: Starting Ceph mgr.z for 4e059f08-da08-11ee-95c6-87774f69a715... 2024-03-04T09:22:47.443 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:22:47 smithi052 ceph-mon[32473]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T09:22:47.547 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:22:47 smithi191 ceph-mon[36816]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T09:22:47.547 INFO:journalctl@ceph.mgr.z.smithi191.stdout:Mar 04 09:22:47 smithi191 podman[38652]: 2024-03-04 09:22:47.339596495 +0000 UTC m=+0.113379951 container create f5a6a77297b8bdcfba5bc2d61a06446a1ad68d6593dc6fccacc4b3095948d61b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5, name=ceph-4e059f08-da08-11ee-95c6-87774f69a715-mgr-z, ceph=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.build-date=20240222, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=wip-lusov-qdb-1bc250c, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8) 2024-03-04T09:22:47.547 INFO:journalctl@ceph.mgr.z.smithi191.stdout:Mar 04 09:22:47 smithi191 podman[38652]: 2024-03-04 09:22:47.243785244 +0000 UTC m=+0.017568660 image pull f88ffb32aa6a7d5dfdc954d9f096f0c9cc47981d05ec8d117ba859a22d8185ba quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5 2024-03-04T09:22:47.547 INFO:journalctl@ceph.mgr.z.smithi191.stdout:Mar 04 09:22:47 smithi191 podman[38652]: 2024-03-04 09:22:47.550930699 +0000 UTC m=+0.324714123 container init f5a6a77297b8bdcfba5bc2d61a06446a1ad68d6593dc6fccacc4b3095948d61b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5, name=ceph-4e059f08-da08-11ee-95c6-87774f69a715-mgr-z, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, org.label-schema.build-date=20240222, ceph=True, GIT_BRANCH=HEAD, RELEASE=wip-lusov-qdb-1bc250c) 2024-03-04T09:22:47.631 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:22:47 smithi106 ceph-mon[36484]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T09:22:47.812 INFO:teuthology.orchestra.run.smithi052.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-03-04T09:22:47.812 INFO:teuthology.orchestra.run.smithi052.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-03-04T09:22:47.812 INFO:teuthology.orchestra.run.smithi052.stderr: stderr: 10+0 records in 2024-03-04T09:22:47.812 INFO:teuthology.orchestra.run.smithi052.stderr:10+0 records out 2024-03-04T09:22:47.812 INFO:teuthology.orchestra.run.smithi052.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.00820058 s, 1.3 GB/s 2024-03-04T09:22:47.812 INFO:teuthology.orchestra.run.smithi052.stderr:--> Zapping successful for: 2024-03-04T09:22:47.836 INFO:journalctl@ceph.mgr.z.smithi191.stdout:Mar 04 09:22:47 smithi191 podman[38652]: 2024-03-04 09:22:47.553468152 +0000 UTC m=+0.327251566 container start f5a6a77297b8bdcfba5bc2d61a06446a1ad68d6593dc6fccacc4b3095948d61b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5, name=ceph-4e059f08-da08-11ee-95c6-87774f69a715-mgr-z, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240222, CEPH_POINT_RELEASE=, ceph=True, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=wip-lusov-qdb-1bc250c) 2024-03-04T09:22:47.836 INFO:journalctl@ceph.mgr.z.smithi191.stdout:Mar 04 09:22:47 smithi191 ceph-mgr[38671]: -- 172.21.15.191:0/3360795234 <== mon.1 v2:172.21.15.191:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55fa074c85a0 con 0x55fa0749c800 2024-03-04T09:22:47.836 INFO:journalctl@ceph.mgr.z.smithi191.stdout:Mar 04 09:22:47 smithi191 bash[38652]: f5a6a77297b8bdcfba5bc2d61a06446a1ad68d6593dc6fccacc4b3095948d61b 2024-03-04T09:22:47.836 INFO:journalctl@ceph.mgr.z.smithi191.stdout:Mar 04 09:22:47 smithi191 systemd[1]: Started Ceph mgr.z for 4e059f08-da08-11ee-95c6-87774f69a715. 2024-03-04T09:22:48.425 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4e059f08-da08-11ee-95c6-87774f69a715 -- ceph orch daemon add osd smithi052:vg_nvme/lv_4 2024-03-04T09:22:48.586 INFO:journalctl@ceph.mgr.z.smithi191.stdout:Mar 04 09:22:48 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-mgr-z[38667]: 2024-03-04T09:22:48.104+0000 7f8698586200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-04T09:22:48.586 INFO:journalctl@ceph.mgr.z.smithi191.stdout:Mar 04 09:22:48 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-mgr-z[38667]: 2024-03-04T09:22:48.197+0000 7f8698586200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-04T09:22:48.662 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/4e059f08-da08-11ee-95c6-87774f69a715/mon.a/config 2024-03-04T09:22:48.881 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:22:48 smithi106 ceph-mon[36484]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T09:22:48.881 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:22:48 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:22:48.881 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:22:48 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:22:48.881 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:22:48 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:22:48.881 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:22:48 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-04T09:22:48.881 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:22:48 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-03-04T09:22:48.882 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:22:48 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-04T09:22:48.882 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:22:48 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T09:22:48.882 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:22:48 smithi106 ceph-mon[36484]: Deploying daemon mgr.y on smithi106 2024-03-04T09:22:48.991 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:22:48 smithi052 ceph-mon[32473]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T09:22:48.991 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:22:48 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:22:48.991 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:22:48 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:22:48.991 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:22:48 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:22:48.991 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:22:48 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-04T09:22:48.992 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:22:48 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-03-04T09:22:48.992 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:22:48 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-04T09:22:48.992 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:22:48 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T09:22:48.992 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:22:48 smithi052 ceph-mon[32473]: Deploying daemon mgr.y on smithi106 2024-03-04T09:22:49.068 INFO:journalctl@ceph.mgr.z.smithi191.stdout:Mar 04 09:22:48 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-mgr-z[38667]: 2024-03-04T09:22:48.837+0000 7f8698586200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-04T09:22:49.069 INFO:journalctl@ceph.mgr.z.smithi191.stdout:Mar 04 09:22:48 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-mgr-z[38667]: 2024-03-04T09:22:48.930+0000 7f8698586200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-04T09:22:49.069 INFO:journalctl@ceph.mgr.z.smithi191.stdout:Mar 04 09:22:49 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-mgr-z[38667]: 2024-03-04T09:22:49.071+0000 7f8698586200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-04T09:22:49.069 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:22:48 smithi191 ceph-mon[36816]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T09:22:49.069 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:22:48 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:22:49.069 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:22:48 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:22:49.069 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:22:48 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:22:49.069 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:22:48 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-04T09:22:49.069 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:22:48 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-03-04T09:22:49.069 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:22:48 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-04T09:22:49.069 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:22:48 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T09:22:49.070 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:22:48 smithi191 ceph-mon[36816]: Deploying daemon mgr.y on smithi106 2024-03-04T09:22:49.336 INFO:journalctl@ceph.mgr.z.smithi191.stdout:Mar 04 09:22:49 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-mgr-z[38667]: 2024-03-04T09:22:49.167+0000 7f8698586200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-04T09:22:49.336 INFO:journalctl@ceph.mgr.z.smithi191.stdout:Mar 04 09:22:49 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-mgr-z[38667]: 2024-03-04T09:22:49.249+0000 7f8698586200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-04T09:22:49.802 INFO:journalctl@ceph.mgr.z.smithi191.stdout:Mar 04 09:22:49 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-mgr-z[38667]: 2024-03-04T09:22:49.511+0000 7f8698586200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-04T09:22:49.803 INFO:journalctl@ceph.mgr.z.smithi191.stdout:Mar 04 09:22:49 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-mgr-z[38667]: 2024-03-04T09:22:49.619+0000 7f8698586200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-04T09:22:49.803 INFO:journalctl@ceph.mgr.z.smithi191.stdout:Mar 04 09:22:49 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-mgr-z[38667]: 2024-03-04T09:22:49.706+0000 7f8698586200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-04T09:22:49.803 INFO:journalctl@ceph.mgr.z.smithi191.stdout:Mar 04 09:22:49 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-mgr-z[38667]: 2024-03-04T09:22:49.804+0000 7f8698586200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-04T09:22:49.881 INFO:journalctl@ceph.mgr.y.smithi106.stdout:Mar 04 09:22:49 smithi106 systemd[1]: Starting Ceph mgr.y for 4e059f08-da08-11ee-95c6-87774f69a715... 2024-03-04T09:22:50.086 INFO:journalctl@ceph.mgr.z.smithi191.stdout:Mar 04 09:22:49 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-mgr-z[38667]: 2024-03-04T09:22:49.989+0000 7f8698586200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-04T09:22:50.240 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:22:50.242+0000 7f3a0d1cc700 1 -- 172.21.15.52:0/1870221505 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3a08102240 msgr2=0x7f3a081026a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:22:50.240 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:22:50.242+0000 7f3a0d1cc700 1 --2- 172.21.15.52:0/1870221505 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3a08102240 0x7f3a081026a0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f39fc0099c0 tx=0x7f39fc009cd0 comp rx=0 tx=0).stop 2024-03-04T09:22:50.240 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:22:50.242+0000 7f3a0d1cc700 1 -- 172.21.15.52:0/1870221505 shutdown_connections 2024-03-04T09:22:50.240 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:22:50.242+0000 7f3a0d1cc700 1 --2- 172.21.15.52:0/1870221505 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3a08102be0 0x7f3a08109490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:22:50.240 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:22:50.243+0000 7f3a0d1cc700 1 --2- 172.21.15.52:0/1870221505 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3a08101040 0x7f3a08101460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:22:50.241 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:22:50.243+0000 7f3a0d1cc700 1 --2- 172.21.15.52:0/1870221505 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3a08102240 0x7f3a081026a0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:22:50.241 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:22:50.243+0000 7f3a0d1cc700 1 -- 172.21.15.52:0/1870221505 >> 172.21.15.52:0/1870221505 conn(0x7f3a080fc5c0 msgr2=0x7f3a080fea20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:22:50.241 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:22:50.243+0000 7f3a0d1cc700 1 -- 172.21.15.52:0/1870221505 shutdown_connections 2024-03-04T09:22:50.241 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:22:50.243+0000 7f3a0d1cc700 1 -- 172.21.15.52:0/1870221505 wait complete. 2024-03-04T09:22:50.241 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:22:50.243+0000 7f3a0d1cc700 1 Processor -- start 2024-03-04T09:22:50.241 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:22:50.243+0000 7f3a0d1cc700 1 -- start start 2024-03-04T09:22:50.243 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:22:50.244+0000 7f3a0d1cc700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3a08101040 0x7f3a081982b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:22:50.243 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:22:50.244+0000 7f3a0d1cc700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3a08102240 0x7f3a081987f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:22:50.243 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:22:50.244+0000 7f3a0d1cc700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3a08102be0 0x7f3a0819f870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:22:50.243 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:22:50.244+0000 7f3a0d1cc700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a0810cc60 con 0x7f3a08101040 2024-03-04T09:22:50.243 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:22:50.244+0000 7f3a0d1cc700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a0810cae0 con 0x7f3a08102be0 2024-03-04T09:22:50.243 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:22:50.244+0000 7f3a0d1cc700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a0810cde0 con 0x7f3a08102240 2024-03-04T09:22:50.244 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:22:50.244+0000 7f3a06d9d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3a08101040 0x7f3a081982b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:22:50.244 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:22:50.244+0000 7f3a06d9d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3a08101040 0x7f3a081982b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:45280/0 (socket says 172.21.15.52:45280) 2024-03-04T09:22:50.244 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:22:50.244+0000 7f3a06d9d700 1 -- 172.21.15.52:0/1132851562 learned_addr learned my addr 172.21.15.52:0/1132851562 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-04T09:22:50.244 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:22:50.244+0000 7f3a0759e700 1 --2- 172.21.15.52:0/1132851562 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3a08102be0 0x7f3a0819f870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:22:50.244 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:22:50.244+0000 7f3a0659c700 1 --2- 172.21.15.52:0/1132851562 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3a08102240 0x7f3a081987f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:22:50.244 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:22:50.244+0000 7f3a06d9d700 1 -- 172.21.15.52:0/1132851562 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3a08102240 msgr2=0x7f3a081987f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:22:50.244 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:22:50.244+0000 7f3a06d9d700 1 --2- 172.21.15.52:0/1132851562 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3a08102240 0x7f3a081987f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:22:50.244 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:22:50.244+0000 7f3a06d9d700 1 -- 172.21.15.52:0/1132851562 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3a08102be0 msgr2=0x7f3a0819f870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:22:50.244 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:22:50.244+0000 7f3a06d9d700 1 --2- 172.21.15.52:0/1132851562 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3a08102be0 0x7f3a0819f870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:22:50.244 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:22:50.244+0000 7f3a06d9d700 1 -- 172.21.15.52:0/1132851562 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a0819ff10 con 0x7f3a08101040 2024-03-04T09:22:50.244 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:22:50.244+0000 7f3a0759e700 1 --2- 172.21.15.52:0/1132851562 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3a08102be0 0x7f3a0819f870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-04T09:22:50.244 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:22:50.244+0000 7f3a06d9d700 1 --2- 172.21.15.52:0/1132851562 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3a08101040 0x7f3a081982b0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f39f800e9a0 tx=0x7f39f800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:22:50.245 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:22:50.244+0000 7f39effff700 1 -- 172.21.15.52:0/1132851562 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39f8004220 con 0x7f3a08101040 2024-03-04T09:22:50.245 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:22:50.244+0000 7f39effff700 1 -- 172.21.15.52:0/1132851562 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f39f80043c0 con 0x7f3a08101040 2024-03-04T09:22:50.245 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:22:50.244+0000 7f39effff700 1 -- 172.21.15.52:0/1132851562 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39f8003df0 con 0x7f3a08101040 2024-03-04T09:22:50.245 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:22:50.244+0000 7f3a0d1cc700 1 -- 172.21.15.52:0/1132851562 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a081a0200 con 0x7f3a08101040 2024-03-04T09:22:50.245 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:22:50.245+0000 7f3a0d1cc700 1 -- 172.21.15.52:0/1132851562 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a081a0790 con 0x7f3a08101040 2024-03-04T09:22:50.245 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:22:50.245+0000 7f39effff700 1 -- 172.21.15.52:0/1132851562 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 13) v1 ==== 45984+0+0 (secure 0 0 0) 0x7f39f8018430 con 0x7f3a08101040 2024-03-04T09:22:50.245 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:22:50.245+0000 7f3a0d1cc700 1 -- 172.21.15.52:0/1132851562 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a08068d80 con 0x7f3a08101040 2024-03-04T09:22:50.245 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:22:50.246+0000 7f39effff700 1 --2- 172.21.15.52:0/1132851562 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f39f0038e80 0x7f39f003b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:22:50.245 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:22:50.246+0000 7f39effff700 1 -- 172.21.15.52:0/1132851562 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f39f804bec0 con 0x7f3a08101040 2024-03-04T09:22:50.245 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:22:50.246+0000 7f3a0659c700 1 --2- 172.21.15.52:0/1132851562 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f39f0038e80 0x7f39f003b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:22:50.245 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:22:50.246+0000 7f3a0659c700 1 --2- 172.21.15.52:0/1132851562 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f39f0038e80 0x7f39f003b340 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f39fc000c00 tx=0x7f39fc005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:22:50.247 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:22:50.251+0000 7f39effff700 1 -- 172.21.15.52:0/1132851562 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f39f800ce10 con 0x7f3a08101040 2024-03-04T09:22:50.258 INFO:journalctl@ceph.mgr.y.smithi106.stdout:Mar 04 09:22:49 smithi106 podman[37850]: 2024-03-04 09:22:49.993412042 +0000 UTC m=+0.114093694 container create d19f286d79b1d9e97c75aabccbdf8a9c1c8dbe4a179c0ad2dae8c8767e248a6c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5, name=ceph-4e059f08-da08-11ee-95c6-87774f69a715-mgr-y, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240222, RELEASE=wip-lusov-qdb-1bc250c, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD) 2024-03-04T09:22:50.258 INFO:journalctl@ceph.mgr.y.smithi106.stdout:Mar 04 09:22:49 smithi106 podman[37850]: 2024-03-04 09:22:49.897201843 +0000 UTC m=+0.017883502 image pull f88ffb32aa6a7d5dfdc954d9f096f0c9cc47981d05ec8d117ba859a22d8185ba quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5 2024-03-04T09:22:50.258 INFO:journalctl@ceph.mgr.y.smithi106.stdout:Mar 04 09:22:50 smithi106 podman[37850]: 2024-03-04 09:22:50.195595584 +0000 UTC m=+0.316277237 container init d19f286d79b1d9e97c75aabccbdf8a9c1c8dbe4a179c0ad2dae8c8767e248a6c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5, name=ceph-4e059f08-da08-11ee-95c6-87774f69a715-mgr-y, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, org.label-schema.build-date=20240222, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, ceph=True, RELEASE=wip-lusov-qdb-1bc250c) 2024-03-04T09:22:50.258 INFO:journalctl@ceph.mgr.y.smithi106.stdout:Mar 04 09:22:50 smithi106 podman[37850]: 2024-03-04 09:22:50.198835823 +0000 UTC m=+0.319517477 container start d19f286d79b1d9e97c75aabccbdf8a9c1c8dbe4a179c0ad2dae8c8767e248a6c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5, name=ceph-4e059f08-da08-11ee-95c6-87774f69a715-mgr-y, GIT_BRANCH=HEAD, org.label-schema.build-date=20240222, GIT_CLEAN=True, ceph=True, RELEASE=wip-lusov-qdb-1bc250c, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, io.buildah.version=1.33.5) 2024-03-04T09:22:50.258 INFO:journalctl@ceph.mgr.y.smithi106.stdout:Mar 04 09:22:50 smithi106 ceph-mgr[37869]: -- 172.21.15.106:0/1008478050 <== mon.2 v2:172.21.15.106:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x564ea5f605a0 con 0x564ea5f34800 2024-03-04T09:22:50.258 INFO:journalctl@ceph.mgr.y.smithi106.stdout:Mar 04 09:22:50 smithi106 bash[37850]: d19f286d79b1d9e97c75aabccbdf8a9c1c8dbe4a179c0ad2dae8c8767e248a6c 2024-03-04T09:22:50.464 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:22:50.467+0000 7f3a0d1cc700 1 -- 172.21.15.52:0/1132851562 --> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi052:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7f3a0806aed0 con 0x7f39f0038e80 2024-03-04T09:22:50.631 INFO:journalctl@ceph.mgr.y.smithi106.stdout:Mar 04 09:22:50 smithi106 systemd[1]: Started Ceph mgr.y for 4e059f08-da08-11ee-95c6-87774f69a715. 2024-03-04T09:22:50.835 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:22:50 smithi052 ceph-mon[32473]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T09:22:50.835 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:22:50 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:22:50.835 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:22:50 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:22:50.836 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:22:50 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:22:50.836 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:22:50 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:22:50.836 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:22:50 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T09:22:50.836 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:22:50 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-04T09:22:50.836 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:22:50 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-04T09:22:50.836 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:22:50 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T09:22:51.009 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:22:50 smithi191 ceph-mon[36816]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T09:22:51.009 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:22:50 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:22:51.009 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:22:50 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:22:51.009 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:22:50 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:22:51.009 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:22:50 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:22:51.009 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:22:50 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T09:22:51.010 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:22:50 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-04T09:22:51.010 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:22:50 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-04T09:22:51.010 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:22:50 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T09:22:51.131 INFO:journalctl@ceph.mgr.y.smithi106.stdout:Mar 04 09:22:50 smithi106 ceph-4e059f08-da08-11ee-95c6-87774f69a715-mgr-y[37865]: 2024-03-04T09:22:50.756+0000 7fbd35b2d200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-04T09:22:51.131 INFO:journalctl@ceph.mgr.y.smithi106.stdout:Mar 04 09:22:50 smithi106 ceph-4e059f08-da08-11ee-95c6-87774f69a715-mgr-y[37865]: 2024-03-04T09:22:50.852+0000 7fbd35b2d200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-04T09:22:51.131 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:22:50 smithi106 ceph-mon[36484]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T09:22:51.132 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:22:50 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:22:51.132 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:22:50 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:22:51.132 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:22:50 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:22:51.132 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:22:50 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:22:51.132 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:22:50 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T09:22:51.132 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:22:50 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-04T09:22:51.132 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:22:50 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-04T09:22:51.132 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:22:50 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T09:22:51.283 INFO:journalctl@ceph.mgr.z.smithi191.stdout:Mar 04 09:22:51 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-mgr-z[38667]: 2024-03-04T09:22:51.044+0000 7f8698586200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-04T09:22:51.283 INFO:journalctl@ceph.mgr.z.smithi191.stdout:Mar 04 09:22:51 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-mgr-z[38667]: 2024-03-04T09:22:51.177+0000 7f8698586200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-04T09:22:51.586 INFO:journalctl@ceph.mgr.z.smithi191.stdout:Mar 04 09:22:51 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-mgr-z[38667]: 2024-03-04T09:22:51.523+0000 7f8698586200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-04T09:22:51.864 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:22:51 smithi106 ceph-mon[36484]: from='client.14226 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi052:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T09:22:51.864 INFO:journalctl@ceph.mgr.y.smithi106.stdout:Mar 04 09:22:51 smithi106 ceph-4e059f08-da08-11ee-95c6-87774f69a715-mgr-y[37865]: 2024-03-04T09:22:51.530+0000 7fbd35b2d200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-04T09:22:51.864 INFO:journalctl@ceph.mgr.y.smithi106.stdout:Mar 04 09:22:51 smithi106 ceph-4e059f08-da08-11ee-95c6-87774f69a715-mgr-y[37865]: 2024-03-04T09:22:51.623+0000 7fbd35b2d200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-04T09:22:51.864 INFO:journalctl@ceph.mgr.y.smithi106.stdout:Mar 04 09:22:51 smithi106 ceph-4e059f08-da08-11ee-95c6-87774f69a715-mgr-y[37865]: 2024-03-04T09:22:51.768+0000 7fbd35b2d200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-04T09:22:51.864 INFO:journalctl@ceph.mgr.y.smithi106.stdout:Mar 04 09:22:51 smithi106 ceph-4e059f08-da08-11ee-95c6-87774f69a715-mgr-y[37865]: 2024-03-04T09:22:51.866+0000 7fbd35b2d200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-04T09:22:51.988 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:22:51 smithi052 ceph-mon[32473]: from='client.14226 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi052:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T09:22:52.003 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:22:51 smithi191 ceph-mon[36816]: from='client.14226 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi052:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T09:22:52.004 INFO:journalctl@ceph.mgr.z.smithi191.stdout:Mar 04 09:22:51 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-mgr-z[38667]: 2024-03-04T09:22:51.612+0000 7f8698586200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-04T09:22:52.004 INFO:journalctl@ceph.mgr.z.smithi191.stdout:Mar 04 09:22:51 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-mgr-z[38667]: 2024-03-04T09:22:51.819+0000 7f8698586200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-04T09:22:52.004 INFO:journalctl@ceph.mgr.z.smithi191.stdout:Mar 04 09:22:52 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-mgr-z[38667]: 2024-03-04T09:22:52.006+0000 7f8698586200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-04T09:22:52.131 INFO:journalctl@ceph.mgr.y.smithi106.stdout:Mar 04 09:22:51 smithi106 ceph-4e059f08-da08-11ee-95c6-87774f69a715-mgr-y[37865]: 2024-03-04T09:22:51.951+0000 7fbd35b2d200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-04T09:22:52.492 INFO:journalctl@ceph.mgr.y.smithi106.stdout:Mar 04 09:22:52 smithi106 ceph-4e059f08-da08-11ee-95c6-87774f69a715-mgr-y[37865]: 2024-03-04T09:22:52.210+0000 7fbd35b2d200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-04T09:22:52.492 INFO:journalctl@ceph.mgr.y.smithi106.stdout:Mar 04 09:22:52 smithi106 ceph-4e059f08-da08-11ee-95c6-87774f69a715-mgr-y[37865]: 2024-03-04T09:22:52.319+0000 7fbd35b2d200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-04T09:22:52.492 INFO:journalctl@ceph.mgr.y.smithi106.stdout:Mar 04 09:22:52 smithi106 ceph-4e059f08-da08-11ee-95c6-87774f69a715-mgr-y[37865]: 2024-03-04T09:22:52.403+0000 7fbd35b2d200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-04T09:22:52.492 INFO:journalctl@ceph.mgr.y.smithi106.stdout:Mar 04 09:22:52 smithi106 ceph-4e059f08-da08-11ee-95c6-87774f69a715-mgr-y[37865]: 2024-03-04T09:22:52.495+0000 7fbd35b2d200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-04T09:22:52.544 INFO:journalctl@ceph.mgr.z.smithi191.stdout:Mar 04 09:22:52 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-mgr-z[38667]: 2024-03-04T09:22:52.287+0000 7f8698586200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-04T09:22:52.822 INFO:journalctl@ceph.mgr.y.smithi106.stdout:Mar 04 09:22:52 smithi106 ceph-4e059f08-da08-11ee-95c6-87774f69a715-mgr-y[37865]: 2024-03-04T09:22:52.668+0000 7fbd35b2d200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-04T09:22:52.823 INFO:journalctl@ceph.mgr.z.smithi191.stdout:Mar 04 09:22:52 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-mgr-z[38667]: 2024-03-04T09:22:52.547+0000 7f8698586200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-04T09:22:52.823 INFO:journalctl@ceph.mgr.z.smithi191.stdout:Mar 04 09:22:52 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-mgr-z[38667]: 2024-03-04T09:22:52.780+0000 7f8698586200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-04T09:22:53.085 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:22:52 smithi052 ceph-mon[32473]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T09:22:53.085 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:22:52 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:22:53.085 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:22:52 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:22:53.085 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:22:52 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:22:53.085 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:22:52 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:22:53.085 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:22:52 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T09:22:53.085 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:22:52 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T09:22:53.086 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:22:52 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:22:53.086 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:22:52 smithi052 ceph-mon[32473]: Reconfiguring mgr.x (unknown last config time)... 2024-03-04T09:22:53.086 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:22:52 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-04T09:22:53.086 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:22:52 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-04T09:22:53.086 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:22:52 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T09:22:53.086 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:22:52 smithi052 ceph-mon[32473]: Reconfiguring daemon mgr.x on smithi052 2024-03-04T09:22:53.086 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:22:52 smithi191 ceph-mon[36816]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T09:22:53.086 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:22:52 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:22:53.086 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:22:52 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:22:53.087 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:22:52 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:22:53.087 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:22:52 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:22:53.087 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:22:52 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T09:22:53.087 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:22:52 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T09:22:53.087 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:22:52 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:22:53.087 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:22:52 smithi191 ceph-mon[36816]: Reconfiguring mgr.x (unknown last config time)... 2024-03-04T09:22:53.087 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:22:52 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-04T09:22:53.087 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:22:52 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-04T09:22:53.087 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:22:52 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T09:22:53.087 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:22:52 smithi191 ceph-mon[36816]: Reconfiguring daemon mgr.x on smithi052 2024-03-04T09:22:53.131 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:22:52 smithi106 ceph-mon[36484]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T09:22:53.131 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:22:52 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:22:53.131 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:22:52 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:22:53.131 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:22:52 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:22:53.131 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:22:52 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:22:53.131 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:22:52 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T09:22:53.131 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:22:52 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T09:22:53.131 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:22:52 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:22:53.131 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:22:52 smithi106 ceph-mon[36484]: Reconfiguring mgr.x (unknown last config time)... 2024-03-04T09:22:53.132 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:22:52 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-04T09:22:53.132 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:22:52 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-04T09:22:53.132 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:22:52 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T09:22:53.132 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:22:52 smithi106 ceph-mon[36484]: Reconfiguring daemon mgr.x on smithi052 2024-03-04T09:22:53.823 INFO:journalctl@ceph.mgr.z.smithi191.stdout:Mar 04 09:22:53 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-mgr-z[38667]: 2024-03-04T09:22:53.575+0000 7f8698586200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-04T09:22:53.881 INFO:journalctl@ceph.mgr.y.smithi106.stdout:Mar 04 09:22:53 smithi106 ceph-4e059f08-da08-11ee-95c6-87774f69a715-mgr-y[37865]: 2024-03-04T09:22:53.627+0000 7fbd35b2d200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-04T09:22:53.881 INFO:journalctl@ceph.mgr.y.smithi106.stdout:Mar 04 09:22:53 smithi106 ceph-4e059f08-da08-11ee-95c6-87774f69a715-mgr-y[37865]: 2024-03-04T09:22:53.756+0000 7fbd35b2d200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-04T09:22:53.881 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:22:53 smithi106 ceph-mon[36484]: from='client.? 172.21.15.52:0/3337005636' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "583172be-602f-4bf4-bfcb-42663cf5b277"}]: dispatch 2024-03-04T09:22:53.882 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:22:53 smithi106 ceph-mon[36484]: from='client.? 172.21.15.52:0/3337005636' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "583172be-602f-4bf4-bfcb-42663cf5b277"}]': finished 2024-03-04T09:22:53.882 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:22:53 smithi106 ceph-mon[36484]: osdmap e5: 1 total, 0 up, 1 in 2024-03-04T09:22:53.882 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:22:53 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-04T09:22:53.882 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:22:53 smithi106 ceph-mon[36484]: from='client.? 172.21.15.52:0/4214997513' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-04T09:22:54.085 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:22:53 smithi052 ceph-mon[32473]: from='client.? 172.21.15.52:0/3337005636' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "583172be-602f-4bf4-bfcb-42663cf5b277"}]: dispatch 2024-03-04T09:22:54.085 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:22:53 smithi052 ceph-mon[32473]: from='client.? 172.21.15.52:0/3337005636' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "583172be-602f-4bf4-bfcb-42663cf5b277"}]': finished 2024-03-04T09:22:54.085 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:22:53 smithi052 ceph-mon[32473]: osdmap e5: 1 total, 0 up, 1 in 2024-03-04T09:22:54.085 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:22:53 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-04T09:22:54.085 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:22:53 smithi052 ceph-mon[32473]: from='client.? 172.21.15.52:0/4214997513' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-04T09:22:54.086 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:22:53 smithi191 ceph-mon[36816]: from='client.? 172.21.15.52:0/3337005636' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "583172be-602f-4bf4-bfcb-42663cf5b277"}]: dispatch 2024-03-04T09:22:54.086 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:22:53 smithi191 ceph-mon[36816]: from='client.? 172.21.15.52:0/3337005636' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "583172be-602f-4bf4-bfcb-42663cf5b277"}]': finished 2024-03-04T09:22:54.086 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:22:53 smithi191 ceph-mon[36816]: osdmap e5: 1 total, 0 up, 1 in 2024-03-04T09:22:54.086 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:22:53 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-04T09:22:54.086 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:22:53 smithi191 ceph-mon[36816]: from='client.? 172.21.15.52:0/4214997513' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-04T09:22:54.365 INFO:journalctl@ceph.mgr.y.smithi106.stdout:Mar 04 09:22:54 smithi106 ceph-4e059f08-da08-11ee-95c6-87774f69a715-mgr-y[37865]: 2024-03-04T09:22:54.087+0000 7fbd35b2d200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-04T09:22:54.365 INFO:journalctl@ceph.mgr.y.smithi106.stdout:Mar 04 09:22:54 smithi106 ceph-4e059f08-da08-11ee-95c6-87774f69a715-mgr-y[37865]: 2024-03-04T09:22:54.175+0000 7fbd35b2d200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-04T09:22:54.365 INFO:journalctl@ceph.mgr.y.smithi106.stdout:Mar 04 09:22:54 smithi106 ceph-4e059f08-da08-11ee-95c6-87774f69a715-mgr-y[37865]: 2024-03-04T09:22:54.368+0000 7fbd35b2d200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-04T09:22:54.449 INFO:journalctl@ceph.mgr.z.smithi191.stdout:Mar 04 09:22:54 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-mgr-z[38667]: 2024-03-04T09:22:54.141+0000 7f8698586200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-04T09:22:54.449 INFO:journalctl@ceph.mgr.z.smithi191.stdout:Mar 04 09:22:54 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-mgr-z[38667]: 2024-03-04T09:22:54.237+0000 7f8698586200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-04T09:22:54.449 INFO:journalctl@ceph.mgr.z.smithi191.stdout:Mar 04 09:22:54 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-mgr-z[38667]: 2024-03-04T09:22:54.452+0000 7f8698586200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-04T09:22:54.631 INFO:journalctl@ceph.mgr.y.smithi106.stdout:Mar 04 09:22:54 smithi106 ceph-4e059f08-da08-11ee-95c6-87774f69a715-mgr-y[37865]: 2024-03-04T09:22:54.551+0000 7fbd35b2d200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-04T09:22:55.084 INFO:journalctl@ceph.mgr.y.smithi106.stdout:Mar 04 09:22:54 smithi106 ceph-4e059f08-da08-11ee-95c6-87774f69a715-mgr-y[37865]: 2024-03-04T09:22:54.833+0000 7fbd35b2d200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-04T09:22:55.084 INFO:journalctl@ceph.mgr.y.smithi106.stdout:Mar 04 09:22:55 smithi106 ceph-4e059f08-da08-11ee-95c6-87774f69a715-mgr-y[37865]: 2024-03-04T09:22:55.087+0000 7fbd35b2d200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-04T09:22:55.112 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:22:55.114+0000 7f39effff700 1 -- 172.21.15.52:0/1132851562 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mgrmap(e 14) v1 ==== 91342+0+0 (secure 0 0 0) 0x7f39f8017cd0 con 0x7f3a08101040 2024-03-04T09:22:55.381 INFO:journalctl@ceph.mgr.y.smithi106.stdout:Mar 04 09:22:55 smithi106 ceph-4e059f08-da08-11ee-95c6-87774f69a715-mgr-y[37865]: 2024-03-04T09:22:55.323+0000 7fbd35b2d200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-04T09:22:55.381 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:22:55 smithi106 ceph-mon[36484]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T09:22:55.381 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:22:55 smithi106 ceph-mon[36484]: Standby manager daemon z started 2024-03-04T09:22:55.382 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:22:55 smithi106 ceph-mon[36484]: from='mgr.? 172.21.15.191:0/2280732575' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-03-04T09:22:55.382 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:22:55 smithi106 ceph-mon[36484]: from='mgr.? 172.21.15.191:0/2280732575' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-04T09:22:55.382 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:22:55 smithi106 ceph-mon[36484]: from='mgr.? 172.21.15.191:0/2280732575' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-03-04T09:22:55.382 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:22:55 smithi106 ceph-mon[36484]: from='mgr.? 172.21.15.191:0/2280732575' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-04T09:22:55.382 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:22:55 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:22:55.584 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:22:55 smithi052 ceph-mon[32473]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T09:22:55.585 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:22:55 smithi052 ceph-mon[32473]: Standby manager daemon z started 2024-03-04T09:22:55.585 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:22:55 smithi052 ceph-mon[32473]: from='mgr.? 172.21.15.191:0/2280732575' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-03-04T09:22:55.585 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:22:55 smithi052 ceph-mon[32473]: from='mgr.? 172.21.15.191:0/2280732575' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-04T09:22:55.585 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:22:55 smithi052 ceph-mon[32473]: from='mgr.? 172.21.15.191:0/2280732575' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-03-04T09:22:55.585 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:22:55 smithi052 ceph-mon[32473]: from='mgr.? 172.21.15.191:0/2280732575' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-04T09:22:55.585 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:22:55 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:22:55.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:22:55 smithi191 ceph-mon[36816]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T09:22:55.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:22:55 smithi191 ceph-mon[36816]: Standby manager daemon z started 2024-03-04T09:22:55.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:22:55 smithi191 ceph-mon[36816]: from='mgr.? 172.21.15.191:0/2280732575' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-03-04T09:22:55.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:22:55 smithi191 ceph-mon[36816]: from='mgr.? 172.21.15.191:0/2280732575' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-04T09:22:55.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:22:55 smithi191 ceph-mon[36816]: from='mgr.? 172.21.15.191:0/2280732575' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-03-04T09:22:55.587 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:22:55 smithi191 ceph-mon[36816]: from='mgr.? 172.21.15.191:0/2280732575' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-04T09:22:55.587 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:22:55 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:22:56.381 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:22:56 smithi106 ceph-mon[36484]: mgrmap e14: x(active, since 75s), standbys: z 2024-03-04T09:22:56.381 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:22:56 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-03-04T09:22:56.381 INFO:journalctl@ceph.mgr.y.smithi106.stdout:Mar 04 09:22:56 smithi106 ceph-4e059f08-da08-11ee-95c6-87774f69a715-mgr-y[37865]: 2024-03-04T09:22:56.117+0000 7fbd35b2d200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-04T09:22:56.584 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:22:56 smithi052 ceph-mon[32473]: mgrmap e14: x(active, since 75s), standbys: z 2024-03-04T09:22:56.585 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:22:56 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-03-04T09:22:56.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:22:56 smithi191 ceph-mon[36816]: mgrmap e14: x(active, since 75s), standbys: z 2024-03-04T09:22:56.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:22:56 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-03-04T09:22:56.977 INFO:journalctl@ceph.mgr.y.smithi106.stdout:Mar 04 09:22:56 smithi106 ceph-4e059f08-da08-11ee-95c6-87774f69a715-mgr-y[37865]: 2024-03-04T09:22:56.674+0000 7fbd35b2d200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-04T09:22:56.977 INFO:journalctl@ceph.mgr.y.smithi106.stdout:Mar 04 09:22:56 smithi106 ceph-4e059f08-da08-11ee-95c6-87774f69a715-mgr-y[37865]: 2024-03-04T09:22:56.769+0000 7fbd35b2d200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-04T09:22:56.977 INFO:journalctl@ceph.mgr.y.smithi106.stdout:Mar 04 09:22:56 smithi106 ceph-4e059f08-da08-11ee-95c6-87774f69a715-mgr-y[37865]: 2024-03-04T09:22:56.980+0000 7fbd35b2d200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-04T09:22:57.132 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:22:57.134+0000 7f39effff700 1 -- 172.21.15.52:0/1132851562 <== mon.0 v2:172.21.15.52:3300/0 8 ==== mgrmap(e 15) v1 ==== 136700+0+0 (secure 0 0 0) 0x7f39f800f3c0 con 0x7f3a08101040 2024-03-04T09:22:57.334 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:22:57 smithi052 ceph-mon[32473]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T09:22:57.335 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:22:57 smithi052 ceph-mon[32473]: Standby manager daemon y started 2024-03-04T09:22:57.335 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:22:57 smithi052 ceph-mon[32473]: from='mgr.? 172.21.15.106:0/3500511876' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-03-04T09:22:57.335 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:22:57 smithi052 ceph-mon[32473]: from='mgr.? 172.21.15.106:0/3500511876' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-04T09:22:57.335 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:22:57 smithi052 ceph-mon[32473]: from='mgr.? 172.21.15.106:0/3500511876' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-03-04T09:22:57.335 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:22:57 smithi052 ceph-mon[32473]: from='mgr.? 172.21.15.106:0/3500511876' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-04T09:22:57.381 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:22:57 smithi106 ceph-mon[36484]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T09:22:57.381 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:22:57 smithi106 ceph-mon[36484]: Standby manager daemon y started 2024-03-04T09:22:57.381 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:22:57 smithi106 ceph-mon[36484]: from='mgr.? 172.21.15.106:0/3500511876' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-03-04T09:22:57.381 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:22:57 smithi106 ceph-mon[36484]: from='mgr.? 172.21.15.106:0/3500511876' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-04T09:22:57.381 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:22:57 smithi106 ceph-mon[36484]: from='mgr.? 172.21.15.106:0/3500511876' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-03-04T09:22:57.381 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:22:57 smithi106 ceph-mon[36484]: from='mgr.? 172.21.15.106:0/3500511876' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-04T09:22:57.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:22:57 smithi191 ceph-mon[36816]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T09:22:57.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:22:57 smithi191 ceph-mon[36816]: Standby manager daemon y started 2024-03-04T09:22:57.587 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:22:57 smithi191 ceph-mon[36816]: from='mgr.? 172.21.15.106:0/3500511876' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-03-04T09:22:57.587 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:22:57 smithi191 ceph-mon[36816]: from='mgr.? 172.21.15.106:0/3500511876' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-04T09:22:57.587 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:22:57 smithi191 ceph-mon[36816]: from='mgr.? 172.21.15.106:0/3500511876' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-03-04T09:22:57.587 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:22:57 smithi191 ceph-mon[36816]: from='mgr.? 172.21.15.106:0/3500511876' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-04T09:22:58.462 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:22:58 smithi052 ceph-mon[32473]: mgrmap e15: x(active, since 77s), standbys: z, y 2024-03-04T09:22:58.462 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:22:58 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-03-04T09:22:58.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:22:58 smithi191 ceph-mon[36816]: mgrmap e15: x(active, since 77s), standbys: z, y 2024-03-04T09:22:58.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:22:58 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-03-04T09:22:58.631 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:22:58 smithi106 ceph-mon[36484]: mgrmap e15: x(active, since 77s), standbys: z, y 2024-03-04T09:22:58.631 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:22:58 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-03-04T09:22:59.533 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:22:59 smithi106 ceph-mon[36484]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T09:22:59.533 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:22:59 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:22:59.533 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:22:59 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:22:59.533 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:22:59 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T09:22:59.533 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:22:59 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T09:22:59.533 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:22:59 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T09:22:59.533 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:22:59 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:22:59.543 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:22:59 smithi052 ceph-mon[32473]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T09:22:59.543 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:22:59 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:22:59.543 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:22:59 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:22:59.543 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:22:59 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T09:22:59.543 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:22:59 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T09:22:59.543 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:22:59 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T09:22:59.543 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:22:59 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:22:59.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:22:59 smithi191 ceph-mon[36816]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T09:22:59.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:22:59 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:22:59.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:22:59 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:22:59.587 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:22:59 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T09:22:59.587 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:22:59 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T09:22:59.587 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:22:59 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T09:22:59.587 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:22:59 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:23:01.336 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:23:01 smithi191 ceph-mon[36816]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T09:23:01.336 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:23:01 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-03-04T09:23:01.336 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:23:01 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T09:23:01.449 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:23:01 smithi052 ceph-mon[32473]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T09:23:01.450 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:23:01 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-03-04T09:23:01.450 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:23:01 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T09:23:01.631 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:23:01 smithi106 ceph-mon[36484]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T09:23:01.631 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:23:01 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-03-04T09:23:01.631 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:23:01 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T09:23:02.398 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:23:02 smithi052 ceph-mon[32473]: Deploying daemon osd.0 on smithi052 2024-03-04T09:23:02.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:23:02 smithi191 ceph-mon[36816]: Deploying daemon osd.0 on smithi052 2024-03-04T09:23:02.631 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:23:02 smithi106 ceph-mon[36484]: Deploying daemon osd.0 on smithi052 2024-03-04T09:23:03.334 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:23:03 smithi052 ceph-mon[32473]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T09:23:03.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:23:03 smithi191 ceph-mon[36816]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T09:23:03.631 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:23:03 smithi106 ceph-mon[36484]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T09:23:05.254 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:23:05 smithi052 ceph-mon[32473]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T09:23:05.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:23:05 smithi191 ceph-mon[36816]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T09:23:05.631 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:23:05 smithi106 ceph-mon[36484]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T09:23:07.264 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:23:07 smithi052 ceph-mon[32473]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T09:23:07.264 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:23:07 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:23:07.264 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:23:07 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:23:07.265 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:23:07 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T09:23:07.336 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:23:07 smithi191 ceph-mon[36816]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T09:23:07.336 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:23:07 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:23:07.336 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:23:07 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:23:07.336 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:23:07 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T09:23:07.381 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:23:07 smithi106 ceph-mon[36484]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T09:23:07.381 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:23:07 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:23:07.381 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:23:07 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:23:07.381 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:23:07 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T09:23:09.309 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:23:09 smithi052 ceph-mon[32473]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T09:23:09.309 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:23:09 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:23:09.336 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:23:09 smithi191 ceph-mon[36816]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T09:23:09.336 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:23:09 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:23:09.375 INFO:teuthology.orchestra.run.smithi052.stdout:Created osd(s) 0 on host 'smithi052' 2024-03-04T09:23:09.375 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:23:09.377+0000 7f39effff700 1 -- 172.21.15.52:0/1132851562 <== mgr.14152 v2:172.21.15.52:6800/3036033923 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f3a0806aed0 con 0x7f39f0038e80 2024-03-04T09:23:09.377 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:23:09.379+0000 7f3a0d1cc700 1 -- 172.21.15.52:0/1132851562 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f39f0038e80 msgr2=0x7f39f003b340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:23:09.377 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:23:09.379+0000 7f3a0d1cc700 1 --2- 172.21.15.52:0/1132851562 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f39f0038e80 0x7f39f003b340 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f39fc000c00 tx=0x7f39fc005fb0 comp rx=0 tx=0).stop 2024-03-04T09:23:09.377 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:23:09.380+0000 7f3a0d1cc700 1 -- 172.21.15.52:0/1132851562 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3a08101040 msgr2=0x7f3a081982b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:23:09.377 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:23:09.380+0000 7f3a0d1cc700 1 --2- 172.21.15.52:0/1132851562 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3a08101040 0x7f3a081982b0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f39f800e9a0 tx=0x7f39f800ecb0 comp rx=0 tx=0).stop 2024-03-04T09:23:09.377 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:23:09.380+0000 7f3a0d1cc700 1 -- 172.21.15.52:0/1132851562 shutdown_connections 2024-03-04T09:23:09.377 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:23:09.380+0000 7f3a0d1cc700 1 --2- 172.21.15.52:0/1132851562 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f39f0038e80 0x7f39f003b340 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:23:09.377 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:23:09.380+0000 7f3a0d1cc700 1 --2- 172.21.15.52:0/1132851562 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3a08102be0 0x7f3a0819f870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:23:09.377 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:23:09.380+0000 7f3a0d1cc700 1 --2- 172.21.15.52:0/1132851562 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3a08101040 0x7f3a081982b0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:23:09.377 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:23:09.380+0000 7f3a0d1cc700 1 --2- 172.21.15.52:0/1132851562 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3a08102240 0x7f3a081987f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:23:09.377 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:23:09.380+0000 7f3a0d1cc700 1 -- 172.21.15.52:0/1132851562 >> 172.21.15.52:0/1132851562 conn(0x7f3a080fc5c0 msgr2=0x7f3a0806a7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:23:09.377 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:23:09.380+0000 7f3a0d1cc700 1 -- 172.21.15.52:0/1132851562 shutdown_connections 2024-03-04T09:23:09.377 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:23:09.380+0000 7f3a0d1cc700 1 -- 172.21.15.52:0/1132851562 wait complete. 2024-03-04T09:23:09.381 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:23:09 smithi106 ceph-mon[36484]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T09:23:09.381 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:23:09 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:23:09.973 DEBUG:teuthology.orchestra.run.smithi052:osd.0> sudo journalctl -f -n 0 -u ceph-4e059f08-da08-11ee-95c6-87774f69a715@osd.0.service 2024-03-04T09:23:09.975 INFO:tasks.cephadm:Deploying osd.1 on smithi106 with /dev/vg_nvme/lv_4... 2024-03-04T09:23:09.976 DEBUG:teuthology.orchestra.run.smithi106:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4e059f08-da08-11ee-95c6-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-03-04T09:23:10.235 INFO:teuthology.orchestra.run.smithi106.stderr:Inferring config /var/lib/ceph/4e059f08-da08-11ee-95c6-87774f69a715/mon.b/config 2024-03-04T09:23:10.239 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:23:10 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:23:10.239 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:23:10 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T09:23:10.239 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:23:10 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T09:23:10.239 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:23:10 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:23:10.239 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:23:10 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:23:10.239 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:23:10 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:23:10.239 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:23:10 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T09:23:10.239 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:23:10 smithi052 ceph-mon[32473]: from='osd.0 [v2:172.21.15.52:6802/1371221397,v1:172.21.15.52:6803/1371221397]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-03-04T09:23:10.336 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:23:10 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:23:10.336 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:23:10 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T09:23:10.336 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:23:10 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T09:23:10.337 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:23:10 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:23:10.337 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:23:10 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:23:10.337 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:23:10 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:23:10.337 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:23:10 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T09:23:10.337 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:23:10 smithi191 ceph-mon[36816]: from='osd.0 [v2:172.21.15.52:6802/1371221397,v1:172.21.15.52:6803/1371221397]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-03-04T09:23:10.356 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:23:10 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:23:10.356 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:23:10 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T09:23:10.356 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:23:10 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T09:23:10.356 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:23:10 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:23:10.356 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:23:10 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:23:10.357 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:23:10 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:23:10.357 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:23:10 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T09:23:10.357 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:23:10 smithi106 ceph-mon[36484]: from='osd.0 [v2:172.21.15.52:6802/1371221397,v1:172.21.15.52:6803/1371221397]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-03-04T09:23:11.275 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:23:11 smithi106 ceph-mon[36484]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T09:23:11.276 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:23:11 smithi106 ceph-mon[36484]: from='osd.0 [v2:172.21.15.52:6802/1371221397,v1:172.21.15.52:6803/1371221397]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-03-04T09:23:11.276 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:23:11 smithi106 ceph-mon[36484]: osdmap e6: 1 total, 0 up, 1 in 2024-03-04T09:23:11.276 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:23:11 smithi106 ceph-mon[36484]: from='osd.0 [v2:172.21.15.52:6802/1371221397,v1:172.21.15.52:6803/1371221397]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi052", "root=default"]}]: dispatch 2024-03-04T09:23:11.276 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:23:11 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-04T09:23:11.292 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:23:11 smithi052 ceph-mon[32473]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T09:23:11.292 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:23:11 smithi052 ceph-mon[32473]: from='osd.0 [v2:172.21.15.52:6802/1371221397,v1:172.21.15.52:6803/1371221397]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-03-04T09:23:11.292 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:23:11 smithi052 ceph-mon[32473]: osdmap e6: 1 total, 0 up, 1 in 2024-03-04T09:23:11.293 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:23:11 smithi052 ceph-mon[32473]: from='osd.0 [v2:172.21.15.52:6802/1371221397,v1:172.21.15.52:6803/1371221397]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi052", "root=default"]}]: dispatch 2024-03-04T09:23:11.293 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:23:11 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-04T09:23:11.336 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:23:11 smithi191 ceph-mon[36816]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T09:23:11.337 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:23:11 smithi191 ceph-mon[36816]: from='osd.0 [v2:172.21.15.52:6802/1371221397,v1:172.21.15.52:6803/1371221397]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-03-04T09:23:11.337 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:23:11 smithi191 ceph-mon[36816]: osdmap e6: 1 total, 0 up, 1 in 2024-03-04T09:23:11.337 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:23:11 smithi191 ceph-mon[36816]: from='osd.0 [v2:172.21.15.52:6802/1371221397,v1:172.21.15.52:6803/1371221397]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi052", "root=default"]}]: dispatch 2024-03-04T09:23:11.337 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:23:11 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-04T09:23:11.543 INFO:journalctl@ceph.osd.0.smithi052.stdout:Mar 04 09:23:11 smithi052 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-0[44607]: 2024-03-04T09:23:11.362+0000 7fab28250700 -1 osd.0 0 waiting for initial osdmap 2024-03-04T09:23:11.543 INFO:journalctl@ceph.osd.0.smithi052.stdout:Mar 04 09:23:11 smithi052 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-0[44607]: 2024-03-04T09:23:11.365+0000 7fab24043700 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-04T09:23:12.092 INFO:teuthology.orchestra.run.smithi106.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-03-04T09:23:12.092 INFO:teuthology.orchestra.run.smithi106.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-03-04T09:23:12.092 INFO:teuthology.orchestra.run.smithi106.stderr: stderr: 10+0 records in 2024-03-04T09:23:12.093 INFO:teuthology.orchestra.run.smithi106.stderr:10+0 records out 2024-03-04T09:23:12.093 INFO:teuthology.orchestra.run.smithi106.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0128911 s, 813 MB/s 2024-03-04T09:23:12.093 INFO:teuthology.orchestra.run.smithi106.stderr:--> Zapping successful for: 2024-03-04T09:23:12.568 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:23:12 smithi052 ceph-mon[32473]: from='osd.0 [v2:172.21.15.52:6802/1371221397,v1:172.21.15.52:6803/1371221397]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi052", "root=default"]}]': finished 2024-03-04T09:23:12.568 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:23:12 smithi052 ceph-mon[32473]: osdmap e7: 1 total, 0 up, 1 in 2024-03-04T09:23:12.568 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:23:12 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-04T09:23:12.568 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:23:12 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-04T09:23:12.568 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:23:12 smithi052 ceph-mon[32473]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T09:23:12.631 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:23:12 smithi106 ceph-mon[36484]: from='osd.0 [v2:172.21.15.52:6802/1371221397,v1:172.21.15.52:6803/1371221397]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi052", "root=default"]}]': finished 2024-03-04T09:23:12.631 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:23:12 smithi106 ceph-mon[36484]: osdmap e7: 1 total, 0 up, 1 in 2024-03-04T09:23:12.631 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:23:12 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-04T09:23:12.632 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:23:12 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-04T09:23:12.632 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:23:12 smithi106 ceph-mon[36484]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T09:23:12.697 DEBUG:teuthology.orchestra.run.smithi106:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4e059f08-da08-11ee-95c6-87774f69a715 -- ceph orch daemon add osd smithi106:vg_nvme/lv_4 2024-03-04T09:23:12.836 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:23:12 smithi191 ceph-mon[36816]: from='osd.0 [v2:172.21.15.52:6802/1371221397,v1:172.21.15.52:6803/1371221397]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi052", "root=default"]}]': finished 2024-03-04T09:23:12.836 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:23:12 smithi191 ceph-mon[36816]: osdmap e7: 1 total, 0 up, 1 in 2024-03-04T09:23:12.836 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:23:12 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-04T09:23:12.837 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:23:12 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-04T09:23:12.837 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:23:12 smithi191 ceph-mon[36816]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T09:23:12.937 INFO:teuthology.orchestra.run.smithi106.stderr:Inferring config /var/lib/ceph/4e059f08-da08-11ee-95c6-87774f69a715/mon.b/config 2024-03-04T09:23:13.631 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:23:13 smithi106 ceph-mon[36484]: purged_snaps scrub starts 2024-03-04T09:23:13.631 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:23:13 smithi106 ceph-mon[36484]: purged_snaps scrub ok 2024-03-04T09:23:13.632 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:23:13 smithi106 ceph-mon[36484]: osd.0 [v2:172.21.15.52:6802/1371221397,v1:172.21.15.52:6803/1371221397] boot 2024-03-04T09:23:13.632 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:23:13 smithi106 ceph-mon[36484]: osdmap e8: 1 total, 1 up, 1 in 2024-03-04T09:23:13.632 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:23:13 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-04T09:23:13.632 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:23:13 smithi106 ceph-mon[36484]: Detected new or changed devices on smithi052 2024-03-04T09:23:13.632 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:23:13 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:23:13.632 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:23:13 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:23:13.632 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:23:13 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-04T09:23:13.632 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:23:13 smithi106 ceph-mon[36484]: Adjusting osd_memory_target on smithi052 to 17093M 2024-03-04T09:23:13.632 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:23:13 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:23:13.632 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:23:13 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T09:23:13.632 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:23:13 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T09:23:13.632 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:23:13 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:23:13.835 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:23:13 smithi052 ceph-mon[32473]: purged_snaps scrub starts 2024-03-04T09:23:13.835 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:23:13 smithi052 ceph-mon[32473]: purged_snaps scrub ok 2024-03-04T09:23:13.835 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:23:13 smithi052 ceph-mon[32473]: osd.0 [v2:172.21.15.52:6802/1371221397,v1:172.21.15.52:6803/1371221397] boot 2024-03-04T09:23:13.835 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:23:13 smithi052 ceph-mon[32473]: osdmap e8: 1 total, 1 up, 1 in 2024-03-04T09:23:13.835 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:23:13 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-04T09:23:13.835 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:23:13 smithi052 ceph-mon[32473]: Detected new or changed devices on smithi052 2024-03-04T09:23:13.835 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:23:13 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:23:13.835 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:23:13 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:23:13.835 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:23:13 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-04T09:23:13.835 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:23:13 smithi052 ceph-mon[32473]: Adjusting osd_memory_target on smithi052 to 17093M 2024-03-04T09:23:13.836 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:23:13 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:23:13.836 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:23:13 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T09:23:13.836 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:23:13 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T09:23:13.836 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:23:13 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:23:13.836 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:23:13 smithi191 ceph-mon[36816]: purged_snaps scrub starts 2024-03-04T09:23:13.836 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:23:13 smithi191 ceph-mon[36816]: purged_snaps scrub ok 2024-03-04T09:23:13.836 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:23:13 smithi191 ceph-mon[36816]: osd.0 [v2:172.21.15.52:6802/1371221397,v1:172.21.15.52:6803/1371221397] boot 2024-03-04T09:23:13.836 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:23:13 smithi191 ceph-mon[36816]: osdmap e8: 1 total, 1 up, 1 in 2024-03-04T09:23:13.837 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:23:13 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-04T09:23:13.837 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:23:13 smithi191 ceph-mon[36816]: Detected new or changed devices on smithi052 2024-03-04T09:23:13.837 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:23:13 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:23:13.837 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:23:13 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:23:13.837 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:23:13 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-04T09:23:13.837 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:23:13 smithi191 ceph-mon[36816]: Adjusting osd_memory_target on smithi052 to 17093M 2024-03-04T09:23:13.837 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:23:13 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:23:13.837 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:23:13 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T09:23:13.837 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:23:13 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T09:23:13.837 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:23:13 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:23:14.604 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-04T09:23:14.604+0000 7fd9d24f4700 1 -- 172.21.15.106:0/1584241751 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd9cc103c90 msgr2=0x7fd9cc10a540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:23:14.604 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-04T09:23:14.604+0000 7fd9d24f4700 1 --2- 172.21.15.106:0/1584241751 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd9cc103c90 0x7fd9cc10a540 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fd9c400b0d0 tx=0x7fd9c400b3e0 comp rx=0 tx=0).stop 2024-03-04T09:23:14.604 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-04T09:23:14.604+0000 7fd9d24f4700 1 -- 172.21.15.106:0/1584241751 shutdown_connections 2024-03-04T09:23:14.605 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-04T09:23:14.604+0000 7fd9d24f4700 1 --2- 172.21.15.106:0/1584241751 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd9cc103c90 0x7fd9cc10a540 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:23:14.605 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-04T09:23:14.604+0000 7fd9d24f4700 1 --2- 172.21.15.106:0/1584241751 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd9cc1020f0 0x7fd9cc102510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:23:14.605 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-04T09:23:14.604+0000 7fd9d24f4700 1 --2- 172.21.15.106:0/1584241751 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd9cc1032f0 0x7fd9cc103750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:23:14.605 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-04T09:23:14.604+0000 7fd9d24f4700 1 -- 172.21.15.106:0/1584241751 >> 172.21.15.106:0/1584241751 conn(0x7fd9cc0fd6d0 msgr2=0x7fd9cc0ffaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:23:14.605 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-04T09:23:14.605+0000 7fd9d24f4700 1 -- 172.21.15.106:0/1584241751 shutdown_connections 2024-03-04T09:23:14.605 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-04T09:23:14.605+0000 7fd9d24f4700 1 -- 172.21.15.106:0/1584241751 wait complete. 2024-03-04T09:23:14.605 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-04T09:23:14.605+0000 7fd9d24f4700 1 Processor -- start 2024-03-04T09:23:14.605 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-04T09:23:14.605+0000 7fd9d24f4700 1 -- start start 2024-03-04T09:23:14.605 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-04T09:23:14.605+0000 7fd9d24f4700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd9cc1020f0 0x7fd9cc075e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:23:14.605 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-04T09:23:14.605+0000 7fd9d24f4700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd9cc1032f0 0x7fd9cc0744b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:23:14.605 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-04T09:23:14.605+0000 7fd9d24f4700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd9cc103c90 0x7fd9cc0749f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:23:14.605 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-04T09:23:14.605+0000 7fd9d24f4700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9cc10dd30 con 0x7fd9cc103c90 2024-03-04T09:23:14.606 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-04T09:23:14.605+0000 7fd9d24f4700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9cc10dbb0 con 0x7fd9cc1032f0 2024-03-04T09:23:14.606 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-04T09:23:14.605+0000 7fd9d24f4700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9cc10deb0 con 0x7fd9cc1020f0 2024-03-04T09:23:14.606 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-04T09:23:14.606+0000 7fd9cb7fe700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd9cc1032f0 0x7fd9cc0744b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:23:14.606 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-04T09:23:14.606+0000 7fd9cb7fe700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd9cc1032f0 0x7fd9cc0744b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:46728/0 (socket says 172.21.15.106:46728) 2024-03-04T09:23:14.606 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-04T09:23:14.606+0000 7fd9cb7fe700 1 -- 172.21.15.106:0/2502992429 learned_addr learned my addr 172.21.15.106:0/2502992429 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-03-04T09:23:14.606 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-04T09:23:14.606+0000 7fd9cbfff700 1 --2- 172.21.15.106:0/2502992429 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd9cc1020f0 0x7fd9cc075e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:23:14.606 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-04T09:23:14.606+0000 7fd9d0a91700 1 --2- 172.21.15.106:0/2502992429 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd9cc103c90 0x7fd9cc0749f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:23:14.606 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-04T09:23:14.606+0000 7fd9cb7fe700 1 -- 172.21.15.106:0/2502992429 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd9cc1020f0 msgr2=0x7fd9cc075e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:23:14.606 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-04T09:23:14.606+0000 7fd9cb7fe700 1 --2- 172.21.15.106:0/2502992429 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd9cc1020f0 0x7fd9cc075e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:23:14.606 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-04T09:23:14.606+0000 7fd9cb7fe700 1 -- 172.21.15.106:0/2502992429 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd9cc103c90 msgr2=0x7fd9cc0749f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:23:14.606 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-04T09:23:14.606+0000 7fd9cb7fe700 1 --2- 172.21.15.106:0/2502992429 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd9cc103c90 0x7fd9cc0749f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:23:14.606 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-04T09:23:14.606+0000 7fd9cb7fe700 1 -- 172.21.15.106:0/2502992429 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9cc0752a0 con 0x7fd9cc1032f0 2024-03-04T09:23:14.606 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-04T09:23:14.606+0000 7fd9cbfff700 1 --2- 172.21.15.106:0/2502992429 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd9cc1020f0 0x7fd9cc075e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T09:23:14.607 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-04T09:23:14.606+0000 7fd9cb7fe700 1 --2- 172.21.15.106:0/2502992429 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd9cc1032f0 0x7fd9cc0744b0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fd9c000b530 tx=0x7fd9c000b8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:23:14.607 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-04T09:23:14.606+0000 7fd9c97fa700 1 -- 172.21.15.106:0/2502992429 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9c0015020 con 0x7fd9cc1032f0 2024-03-04T09:23:14.607 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-04T09:23:14.606+0000 7fd9c97fa700 1 -- 172.21.15.106:0/2502992429 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd9c0011d10 con 0x7fd9cc1032f0 2024-03-04T09:23:14.607 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-04T09:23:14.606+0000 7fd9d24f4700 1 -- 172.21.15.106:0/2502992429 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9cc1a2450 con 0x7fd9cc1032f0 2024-03-04T09:23:14.607 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-04T09:23:14.606+0000 7fd9c97fa700 1 -- 172.21.15.106:0/2502992429 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9c0010e50 con 0x7fd9cc1032f0 2024-03-04T09:23:14.607 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-04T09:23:14.606+0000 7fd9d24f4700 1 -- 172.21.15.106:0/2502992429 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9cc1a29e0 con 0x7fd9cc1032f0 2024-03-04T09:23:14.607 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-04T09:23:14.607+0000 7fd9d24f4700 1 -- 172.21.15.106:0/2502992429 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd9cc068d80 con 0x7fd9cc1032f0 2024-03-04T09:23:14.608 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-04T09:23:14.608+0000 7fd9c97fa700 1 -- 172.21.15.106:0/2502992429 <== mon.2 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 15) v1 ==== 136700+0+0 (secure 0 0 0) 0x7fd9c0018020 con 0x7fd9cc1032f0 2024-03-04T09:23:14.608 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-04T09:23:14.609+0000 7fd9c97fa700 1 --2- 172.21.15.106:0/2502992429 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7fd9b40a1fa0 0x7fd9b40a4460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:23:14.608 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-04T09:23:14.609+0000 7fd9c97fa700 1 -- 172.21.15.106:0/2502992429 <== mon.2 v2:172.21.15.106:3300/0 5 ==== osd_map(9..9 src has 1..9) v4 ==== 1767+0+0 (secure 0 0 0) 0x7fd9c00cc0d0 con 0x7fd9cc1032f0 2024-03-04T09:23:14.608 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-04T09:23:14.609+0000 7fd9cbfff700 1 --2- 172.21.15.106:0/2502992429 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7fd9b40a1fa0 0x7fd9b40a4460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:23:14.608 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-04T09:23:14.610+0000 7fd9cbfff700 1 --2- 172.21.15.106:0/2502992429 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7fd9b40a1fa0 0x7fd9b40a4460 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fd9cc103150 tx=0x7fd9bc005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:23:14.610 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-04T09:23:14.613+0000 7fd9c97fa700 1 -- 172.21.15.106:0/2502992429 <== mon.2 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fd9c001f020 con 0x7fd9cc1032f0 2024-03-04T09:23:14.814 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-04T09:23:14.817+0000 7fd9d24f4700 1 -- 172.21.15.106:0/2502992429 --> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi106:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7fd9cc062500 con 0x7fd9b40a1fa0 2024-03-04T09:23:15.131 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:23:14 smithi106 ceph-mon[36484]: pgmap v45: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-04T09:23:15.131 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:23:14 smithi106 ceph-mon[36484]: osdmap e9: 1 total, 1 up, 1 in 2024-03-04T09:23:15.131 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:23:14 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-04T09:23:15.131 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:23:14 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-04T09:23:15.131 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:23:14 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T09:23:15.336 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:23:14 smithi191 ceph-mon[36816]: pgmap v45: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-04T09:23:15.337 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:23:14 smithi191 ceph-mon[36816]: osdmap e9: 1 total, 1 up, 1 in 2024-03-04T09:23:15.337 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:23:14 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-04T09:23:15.337 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:23:14 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-04T09:23:15.337 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:23:14 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T09:23:15.338 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:23:14 smithi052 ceph-mon[32473]: pgmap v45: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-04T09:23:15.338 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:23:14 smithi052 ceph-mon[32473]: osdmap e9: 1 total, 1 up, 1 in 2024-03-04T09:23:15.338 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:23:14 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-04T09:23:15.338 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:23:14 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-04T09:23:15.338 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:23:14 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T09:23:16.063 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:23:15 smithi106 ceph-mon[36484]: from='client.24143 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi106:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T09:23:16.334 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:23:15 smithi052 ceph-mon[32473]: from='client.24143 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi106:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T09:23:16.336 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:23:15 smithi191 ceph-mon[36816]: from='client.24143 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi106:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T09:23:17.131 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:23:16 smithi106 ceph-mon[36484]: pgmap v47: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-04T09:23:17.335 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:23:16 smithi052 ceph-mon[32473]: pgmap v47: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-04T09:23:17.336 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:23:16 smithi191 ceph-mon[36816]: pgmap v47: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-04T09:23:18.335 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:23:17 smithi052 ceph-mon[32473]: from='client.? 172.21.15.106:0/1502173376' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5ae0081c-a2f6-46eb-ace0-0368208377b3"}]: dispatch 2024-03-04T09:23:18.335 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:23:17 smithi052 ceph-mon[32473]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5ae0081c-a2f6-46eb-ace0-0368208377b3"}]: dispatch 2024-03-04T09:23:18.335 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:23:17 smithi052 ceph-mon[32473]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5ae0081c-a2f6-46eb-ace0-0368208377b3"}]': finished 2024-03-04T09:23:18.335 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:23:17 smithi052 ceph-mon[32473]: osdmap e10: 2 total, 1 up, 2 in 2024-03-04T09:23:18.335 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:23:17 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-04T09:23:18.336 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:23:17 smithi191 ceph-mon[36816]: from='client.? 172.21.15.106:0/1502173376' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5ae0081c-a2f6-46eb-ace0-0368208377b3"}]: dispatch 2024-03-04T09:23:18.336 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:23:17 smithi191 ceph-mon[36816]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5ae0081c-a2f6-46eb-ace0-0368208377b3"}]: dispatch 2024-03-04T09:23:18.336 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:23:17 smithi191 ceph-mon[36816]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5ae0081c-a2f6-46eb-ace0-0368208377b3"}]': finished 2024-03-04T09:23:18.336 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:23:17 smithi191 ceph-mon[36816]: osdmap e10: 2 total, 1 up, 2 in 2024-03-04T09:23:18.336 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:23:17 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-04T09:23:18.381 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:23:17 smithi106 ceph-mon[36484]: from='client.? 172.21.15.106:0/1502173376' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5ae0081c-a2f6-46eb-ace0-0368208377b3"}]: dispatch 2024-03-04T09:23:18.381 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:23:17 smithi106 ceph-mon[36484]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5ae0081c-a2f6-46eb-ace0-0368208377b3"}]: dispatch 2024-03-04T09:23:18.381 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:23:17 smithi106 ceph-mon[36484]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5ae0081c-a2f6-46eb-ace0-0368208377b3"}]': finished 2024-03-04T09:23:18.381 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:23:17 smithi106 ceph-mon[36484]: osdmap e10: 2 total, 1 up, 2 in 2024-03-04T09:23:18.381 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:23:17 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-04T09:23:19.237 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:23:18 smithi191 ceph-mon[36816]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-04T09:23:19.238 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:23:18 smithi191 ceph-mon[36816]: from='client.? 172.21.15.106:0/1487459048' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-04T09:23:19.334 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:23:18 smithi052 ceph-mon[32473]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-04T09:23:19.334 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:23:18 smithi052 ceph-mon[32473]: from='client.? 172.21.15.106:0/1487459048' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-04T09:23:19.381 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:23:18 smithi106 ceph-mon[36484]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-04T09:23:19.381 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:23:18 smithi106 ceph-mon[36484]: from='client.? 172.21.15.106:0/1487459048' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-04T09:23:21.272 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:23:20 smithi106 ceph-mon[36484]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-04T09:23:21.335 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:23:20 smithi052 ceph-mon[32473]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-04T09:23:21.336 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:23:20 smithi191 ceph-mon[36816]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-04T09:23:23.131 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:23:22 smithi106 ceph-mon[36484]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-04T09:23:23.334 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:23:22 smithi052 ceph-mon[32473]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-04T09:23:23.336 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:23:22 smithi191 ceph-mon[36816]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-04T09:23:25.179 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:23:24 smithi106 ceph-mon[36484]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-04T09:23:25.179 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:23:24 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-03-04T09:23:25.179 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:23:24 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T09:23:25.334 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:23:24 smithi052 ceph-mon[32473]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-04T09:23:25.335 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:23:24 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-03-04T09:23:25.335 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:23:24 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T09:23:25.336 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:23:24 smithi191 ceph-mon[36816]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-04T09:23:25.336 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:23:24 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-03-04T09:23:25.336 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:23:24 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T09:23:26.210 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:23:25 smithi106 ceph-mon[36484]: Deploying daemon osd.1 on smithi106 2024-03-04T09:23:26.334 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:23:25 smithi052 ceph-mon[32473]: Deploying daemon osd.1 on smithi106 2024-03-04T09:23:26.336 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:23:25 smithi191 ceph-mon[36816]: Deploying daemon osd.1 on smithi106 2024-03-04T09:23:27.316 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:23:26 smithi106 ceph-mon[36484]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-04T09:23:27.334 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:23:26 smithi052 ceph-mon[32473]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-04T09:23:27.336 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:23:26 smithi191 ceph-mon[36816]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-04T09:23:29.131 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:23:28 smithi106 ceph-mon[36484]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-04T09:23:29.334 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:23:28 smithi052 ceph-mon[32473]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-04T09:23:29.336 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:23:28 smithi191 ceph-mon[36816]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-04T09:23:30.835 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:23:30 smithi052 ceph-mon[32473]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-04T09:23:30.835 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:23:30 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:23:30.835 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:23:30 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:23:30.835 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:23:30 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T09:23:30.836 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:23:30 smithi191 ceph-mon[36816]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-04T09:23:30.836 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:23:30 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:23:30.836 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:23:30 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:23:30.836 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:23:30 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T09:23:30.881 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:23:30 smithi106 ceph-mon[36484]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-04T09:23:30.881 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:23:30 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:23:30.881 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:23:30 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:23:30.881 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:23:30 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T09:23:33.082 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:23:32 smithi106 ceph-mon[36484]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-04T09:23:33.083 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:23:32 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:23:33.083 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:23:32 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:23:33.083 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:23:32 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T09:23:33.083 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:23:32 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T09:23:33.083 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:23:32 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:23:33.084 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:23:32 smithi052 ceph-mon[32473]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-04T09:23:33.085 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:23:32 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:23:33.085 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:23:32 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:23:33.085 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:23:32 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T09:23:33.085 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:23:32 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T09:23:33.085 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:23:32 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:23:33.086 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:23:32 smithi191 ceph-mon[36816]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-04T09:23:33.086 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:23:32 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:23:33.086 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:23:32 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:23:33.086 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:23:32 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T09:23:33.086 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:23:32 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T09:23:33.086 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:23:32 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:23:33.136 INFO:teuthology.orchestra.run.smithi106.stdout:Created osd(s) 1 on host 'smithi106' 2024-03-04T09:23:33.136 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-04T09:23:33.139+0000 7fd9c97fa700 1 -- 172.21.15.106:0/2502992429 <== mgr.14152 v2:172.21.15.52:6800/3036033923 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fd9cc062500 con 0x7fd9b40a1fa0 2024-03-04T09:23:33.138 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-04T09:23:33.141+0000 7fd9d24f4700 1 -- 172.21.15.106:0/2502992429 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7fd9b40a1fa0 msgr2=0x7fd9b40a4460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:23:33.139 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-04T09:23:33.141+0000 7fd9d24f4700 1 --2- 172.21.15.106:0/2502992429 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7fd9b40a1fa0 0x7fd9b40a4460 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fd9cc103150 tx=0x7fd9bc005f90 comp rx=0 tx=0).stop 2024-03-04T09:23:33.139 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-04T09:23:33.141+0000 7fd9d24f4700 1 -- 172.21.15.106:0/2502992429 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd9cc1032f0 msgr2=0x7fd9cc0744b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:23:33.139 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-04T09:23:33.141+0000 7fd9d24f4700 1 --2- 172.21.15.106:0/2502992429 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd9cc1032f0 0x7fd9cc0744b0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fd9c000b530 tx=0x7fd9c000b8f0 comp rx=0 tx=0).stop 2024-03-04T09:23:33.139 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-04T09:23:33.141+0000 7fd9d24f4700 1 -- 172.21.15.106:0/2502992429 shutdown_connections 2024-03-04T09:23:33.139 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-04T09:23:33.141+0000 7fd9d24f4700 1 --2- 172.21.15.106:0/2502992429 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7fd9b40a1fa0 0x7fd9b40a4460 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:23:33.139 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-04T09:23:33.141+0000 7fd9d24f4700 1 --2- 172.21.15.106:0/2502992429 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd9cc1020f0 0x7fd9cc075e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:23:33.139 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-04T09:23:33.141+0000 7fd9d24f4700 1 --2- 172.21.15.106:0/2502992429 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd9cc103c90 0x7fd9cc0749f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:23:33.139 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-04T09:23:33.141+0000 7fd9d24f4700 1 --2- 172.21.15.106:0/2502992429 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd9cc1032f0 0x7fd9cc0744b0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:23:33.139 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-04T09:23:33.141+0000 7fd9d24f4700 1 -- 172.21.15.106:0/2502992429 >> 172.21.15.106:0/2502992429 conn(0x7fd9cc0fd6d0 msgr2=0x7fd9cc1065f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:23:33.139 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-04T09:23:33.141+0000 7fd9d24f4700 1 -- 172.21.15.106:0/2502992429 shutdown_connections 2024-03-04T09:23:33.139 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-04T09:23:33.141+0000 7fd9d24f4700 1 -- 172.21.15.106:0/2502992429 wait complete. 2024-03-04T09:23:33.787 DEBUG:teuthology.orchestra.run.smithi106:osd.1> sudo journalctl -f -n 0 -u ceph-4e059f08-da08-11ee-95c6-87774f69a715@osd.1.service 2024-03-04T09:23:33.790 INFO:tasks.cephadm:Deploying osd.2 on smithi191 with /dev/vg_nvme/lv_4... 2024-03-04T09:23:33.790 DEBUG:teuthology.orchestra.run.smithi191:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4e059f08-da08-11ee-95c6-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-03-04T09:23:34.034 INFO:teuthology.orchestra.run.smithi191.stderr:Inferring config /var/lib/ceph/4e059f08-da08-11ee-95c6-87774f69a715/mon.c/config 2024-03-04T09:23:34.364 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:23:34 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:23:34.365 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:23:34 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:23:34.365 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:23:34 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T09:23:34.365 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:23:34 smithi191 ceph-mon[36816]: from='osd.1 [v2:172.21.15.106:6800/2540865537,v1:172.21.15.106:6801/2540865537]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-03-04T09:23:34.365 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:23:34 smithi191 ceph-mon[36816]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-03-04T09:23:34.381 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:23:34 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:23:34.381 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:23:34 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:23:34.381 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:23:34 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T09:23:34.382 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:23:34 smithi106 ceph-mon[36484]: from='osd.1 [v2:172.21.15.106:6800/2540865537,v1:172.21.15.106:6801/2540865537]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-03-04T09:23:34.382 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:23:34 smithi106 ceph-mon[36484]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-03-04T09:23:34.584 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:23:34 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:23:34.585 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:23:34 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:23:34.585 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:23:34 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T09:23:34.585 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:23:34 smithi052 ceph-mon[32473]: from='osd.1 [v2:172.21.15.106:6800/2540865537,v1:172.21.15.106:6801/2540865537]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-03-04T09:23:34.585 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:23:34 smithi052 ceph-mon[32473]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-03-04T09:23:35.281 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:23:35 smithi106 ceph-mon[36484]: pgmap v57: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-04T09:23:35.281 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:23:35 smithi106 ceph-mon[36484]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-03-04T09:23:35.281 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:23:35 smithi106 ceph-mon[36484]: from='osd.1 [v2:172.21.15.106:6800/2540865537,v1:172.21.15.106:6801/2540865537]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi106", "root=default"]}]: dispatch 2024-03-04T09:23:35.281 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:23:35 smithi106 ceph-mon[36484]: osdmap e11: 2 total, 1 up, 2 in 2024-03-04T09:23:35.281 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:23:35 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-04T09:23:35.282 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:23:35 smithi106 ceph-mon[36484]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi106", "root=default"]}]: dispatch 2024-03-04T09:23:35.282 INFO:journalctl@ceph.osd.1.smithi106.stdout:Mar 04 09:23:35 smithi106 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-1[41081]: 2024-03-04T09:23:35.131+0000 7f14a9807700 -1 osd.1 0 waiting for initial osdmap 2024-03-04T09:23:35.282 INFO:journalctl@ceph.osd.1.smithi106.stdout:Mar 04 09:23:35 smithi106 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-1[41081]: 2024-03-04T09:23:35.134+0000 7f14a55fa700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-04T09:23:35.320 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:23:35 smithi191 ceph-mon[36816]: pgmap v57: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-04T09:23:35.320 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:23:35 smithi191 ceph-mon[36816]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-03-04T09:23:35.320 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:23:35 smithi191 ceph-mon[36816]: from='osd.1 [v2:172.21.15.106:6800/2540865537,v1:172.21.15.106:6801/2540865537]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi106", "root=default"]}]: dispatch 2024-03-04T09:23:35.320 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:23:35 smithi191 ceph-mon[36816]: osdmap e11: 2 total, 1 up, 2 in 2024-03-04T09:23:35.320 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:23:35 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-04T09:23:35.320 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:23:35 smithi191 ceph-mon[36816]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi106", "root=default"]}]: dispatch 2024-03-04T09:23:35.585 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:23:35 smithi052 ceph-mon[32473]: pgmap v57: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-04T09:23:35.585 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:23:35 smithi052 ceph-mon[32473]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-03-04T09:23:35.585 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:23:35 smithi052 ceph-mon[32473]: from='osd.1 [v2:172.21.15.106:6800/2540865537,v1:172.21.15.106:6801/2540865537]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi106", "root=default"]}]: dispatch 2024-03-04T09:23:35.585 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:23:35 smithi052 ceph-mon[32473]: osdmap e11: 2 total, 1 up, 2 in 2024-03-04T09:23:35.585 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:23:35 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-04T09:23:35.585 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:23:35 smithi052 ceph-mon[32473]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi106", "root=default"]}]: dispatch 2024-03-04T09:23:35.906 INFO:teuthology.orchestra.run.smithi191.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-03-04T09:23:35.921 INFO:teuthology.orchestra.run.smithi191.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-03-04T09:23:35.921 INFO:teuthology.orchestra.run.smithi191.stderr: stderr: 10+0 records in 2024-03-04T09:23:35.922 INFO:teuthology.orchestra.run.smithi191.stderr:10+0 records out 2024-03-04T09:23:35.922 INFO:teuthology.orchestra.run.smithi191.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0122424 s, 857 MB/s 2024-03-04T09:23:35.922 INFO:teuthology.orchestra.run.smithi191.stderr:--> Zapping successful for: 2024-03-04T09:23:36.226 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:23:36 smithi191 ceph-mon[36816]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi106", "root=default"]}]': finished 2024-03-04T09:23:36.226 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:23:36 smithi191 ceph-mon[36816]: osdmap e12: 2 total, 1 up, 2 in 2024-03-04T09:23:36.226 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:23:36 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-04T09:23:36.226 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:23:36 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-04T09:23:36.381 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:23:36 smithi106 ceph-mon[36484]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi106", "root=default"]}]': finished 2024-03-04T09:23:36.381 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:23:36 smithi106 ceph-mon[36484]: osdmap e12: 2 total, 1 up, 2 in 2024-03-04T09:23:36.382 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:23:36 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-04T09:23:36.382 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:23:36 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-04T09:23:36.529 DEBUG:teuthology.orchestra.run.smithi191:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4e059f08-da08-11ee-95c6-87774f69a715 -- ceph orch daemon add osd smithi191:vg_nvme/lv_4 2024-03-04T09:23:36.584 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:23:36 smithi052 ceph-mon[32473]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi106", "root=default"]}]': finished 2024-03-04T09:23:36.585 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:23:36 smithi052 ceph-mon[32473]: osdmap e12: 2 total, 1 up, 2 in 2024-03-04T09:23:36.585 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:23:36 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-04T09:23:36.585 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:23:36 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-04T09:23:36.777 INFO:teuthology.orchestra.run.smithi191.stderr:Inferring config /var/lib/ceph/4e059f08-da08-11ee-95c6-87774f69a715/mon.c/config 2024-03-04T09:23:37.486 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:23:37 smithi052 ceph-mon[32473]: purged_snaps scrub starts 2024-03-04T09:23:37.486 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:23:37 smithi052 ceph-mon[32473]: purged_snaps scrub ok 2024-03-04T09:23:37.486 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:23:37 smithi052 ceph-mon[32473]: pgmap v60: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-04T09:23:37.486 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:23:37 smithi052 ceph-mon[32473]: osd.1 [v2:172.21.15.106:6800/2540865537,v1:172.21.15.106:6801/2540865537] boot 2024-03-04T09:23:37.486 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:23:37 smithi052 ceph-mon[32473]: osdmap e13: 2 total, 2 up, 2 in 2024-03-04T09:23:37.487 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:23:37 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-04T09:23:37.487 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:23:37 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:23:37.487 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:23:37 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:23:37.487 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:23:37 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-04T09:23:37.487 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:23:37 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:23:37.487 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:23:37 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T09:23:37.487 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:23:37 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T09:23:37.487 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:23:37 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:23:37.574 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:23:37 smithi191 ceph-mon[36816]: purged_snaps scrub starts 2024-03-04T09:23:37.574 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:23:37 smithi191 ceph-mon[36816]: purged_snaps scrub ok 2024-03-04T09:23:37.574 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:23:37 smithi191 ceph-mon[36816]: pgmap v60: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-04T09:23:37.575 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:23:37 smithi191 ceph-mon[36816]: osd.1 [v2:172.21.15.106:6800/2540865537,v1:172.21.15.106:6801/2540865537] boot 2024-03-04T09:23:37.575 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:23:37 smithi191 ceph-mon[36816]: osdmap e13: 2 total, 2 up, 2 in 2024-03-04T09:23:37.575 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:23:37 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-04T09:23:37.575 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:23:37 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:23:37.575 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:23:37 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:23:37.575 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:23:37 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-04T09:23:37.575 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:23:37 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:23:37.575 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:23:37 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T09:23:37.575 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:23:37 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T09:23:37.575 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:23:37 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:23:37.631 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:23:37 smithi106 ceph-mon[36484]: purged_snaps scrub starts 2024-03-04T09:23:37.631 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:23:37 smithi106 ceph-mon[36484]: purged_snaps scrub ok 2024-03-04T09:23:37.631 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:23:37 smithi106 ceph-mon[36484]: pgmap v60: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-04T09:23:37.631 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:23:37 smithi106 ceph-mon[36484]: osd.1 [v2:172.21.15.106:6800/2540865537,v1:172.21.15.106:6801/2540865537] boot 2024-03-04T09:23:37.632 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:23:37 smithi106 ceph-mon[36484]: osdmap e13: 2 total, 2 up, 2 in 2024-03-04T09:23:37.632 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:23:37 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-04T09:23:37.632 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:23:37 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:23:37.632 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:23:37 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:23:37.632 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:23:37 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-04T09:23:37.632 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:23:37 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:23:37.632 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:23:37 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T09:23:37.632 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:23:37 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T09:23:37.632 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:23:37 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:23:38.333 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:23:38.334+0000 7f18b5a10700 1 -- 172.21.15.191:0/667611306 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f18b01020f0 msgr2=0x7f18b0102510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:23:38.334 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:23:38.334+0000 7f18b5a10700 1 --2- 172.21.15.191:0/667611306 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f18b01020f0 0x7f18b0102510 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f18a0009990 tx=0x7f18a0009ca0 comp rx=0 tx=0).stop 2024-03-04T09:23:38.334 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:23:38.334+0000 7f18b5a10700 1 -- 172.21.15.191:0/667611306 shutdown_connections 2024-03-04T09:23:38.334 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:23:38.334+0000 7f18b5a10700 1 --2- 172.21.15.191:0/667611306 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f18b01020f0 0x7f18b0102510 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:23:38.334 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:23:38.334+0000 7f18b5a10700 1 --2- 172.21.15.191:0/667611306 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f18b0103c90 0x7f18b010a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:23:38.334 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:23:38.334+0000 7f18b5a10700 1 --2- 172.21.15.191:0/667611306 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f18b01032f0 0x7f18b0103750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:23:38.334 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:23:38.334+0000 7f18b5a10700 1 -- 172.21.15.191:0/667611306 >> 172.21.15.191:0/667611306 conn(0x7f18b00fd6d0 msgr2=0x7f18b00ffaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:23:38.334 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:23:38.334+0000 7f18b5a10700 1 -- 172.21.15.191:0/667611306 shutdown_connections 2024-03-04T09:23:38.334 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:23:38.334+0000 7f18b5a10700 1 -- 172.21.15.191:0/667611306 wait complete. 2024-03-04T09:23:38.334 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:23:38.334+0000 7f18b5a10700 1 Processor -- start 2024-03-04T09:23:38.335 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:23:38.334+0000 7f18b5a10700 1 -- start start 2024-03-04T09:23:38.335 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:23:38.335+0000 7f18b5a10700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f18b01020f0 0x7f18b0198360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:23:38.335 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:23:38.335+0000 7f18b5a10700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f18b01032f0 0x7f18b01988a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:23:38.335 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:23:38.335+0000 7f18b5a10700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f18b0103c90 0x7f18b019f920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:23:38.335 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:23:38.335+0000 7f18b5a10700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18b010dd80 con 0x7f18b01032f0 2024-03-04T09:23:38.335 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:23:38.335+0000 7f18b5a10700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18b010dc00 con 0x7f18b0103c90 2024-03-04T09:23:38.335 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:23:38.335+0000 7f18b5a10700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18b010df00 con 0x7f18b01020f0 2024-03-04T09:23:38.335 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:23:38.335+0000 7f18aeffd700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f18b01020f0 0x7f18b0198360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:23:38.335 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:23:38.335+0000 7f18aeffd700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f18b01020f0 0x7f18b0198360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.191:51142/0 (socket says 172.21.15.191:51142) 2024-03-04T09:23:38.335 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:23:38.335+0000 7f18aeffd700 1 -- 172.21.15.191:0/3500362094 learned_addr learned my addr 172.21.15.191:0/3500362094 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-03-04T09:23:38.335 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:23:38.335+0000 7f18af7fe700 1 --2- 172.21.15.191:0/3500362094 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f18b0103c90 0x7f18b019f920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:23:38.336 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:23:38.335+0000 7f18a67fc700 1 --2- 172.21.15.191:0/3500362094 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f18b01032f0 0x7f18b01988a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:23:38.336 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:23:38.335+0000 7f18aeffd700 1 -- 172.21.15.191:0/3500362094 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f18b0103c90 msgr2=0x7f18b019f920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:23:38.336 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:23:38.335+0000 7f18aeffd700 1 --2- 172.21.15.191:0/3500362094 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f18b0103c90 0x7f18b019f920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:23:38.336 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:23:38.335+0000 7f18aeffd700 1 -- 172.21.15.191:0/3500362094 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f18b01032f0 msgr2=0x7f18b01988a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:23:38.336 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:23:38.335+0000 7f18aeffd700 1 --2- 172.21.15.191:0/3500362094 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f18b01032f0 0x7f18b01988a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:23:38.336 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:23:38.335+0000 7f18aeffd700 1 -- 172.21.15.191:0/3500362094 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18b019ffc0 con 0x7f18b01020f0 2024-03-04T09:23:38.336 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:23:38.335+0000 7f18a67fc700 1 --2- 172.21.15.191:0/3500362094 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f18b01032f0 0x7f18b01988a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T09:23:38.336 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:23:38.335+0000 7f18aeffd700 1 --2- 172.21.15.191:0/3500362094 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f18b01020f0 0x7f18b0198360 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f18a0005f50 tx=0x7f18a00057c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:23:38.336 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:23:38.335+0000 7f18b4a0e700 1 -- 172.21.15.191:0/3500362094 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18a0027070 con 0x7f18b01020f0 2024-03-04T09:23:38.336 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:23:38.336+0000 7f18b5a10700 1 -- 172.21.15.191:0/3500362094 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18b01a0250 con 0x7f18b01020f0 2024-03-04T09:23:38.336 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:23:38.336+0000 7f18b4a0e700 1 -- 172.21.15.191:0/3500362094 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f18a0004150 con 0x7f18b01020f0 2024-03-04T09:23:38.336 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:23:38.336+0000 7f18b4a0e700 1 -- 172.21.15.191:0/3500362094 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18a000f460 con 0x7f18b01020f0 2024-03-04T09:23:38.337 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:23:38.336+0000 7f18b5a10700 1 -- 172.21.15.191:0/3500362094 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18b01a0780 con 0x7f18b01020f0 2024-03-04T09:23:38.337 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:23:38.336+0000 7f18b5a10700 1 -- 172.21.15.191:0/3500362094 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f18b0068d80 con 0x7f18b01020f0 2024-03-04T09:23:38.337 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:23:38.338+0000 7f18b4a0e700 1 -- 172.21.15.191:0/3500362094 <== mon.1 v2:172.21.15.191:3300/0 4 ==== mgrmap(e 15) v1 ==== 136700+0+0 (secure 0 0 0) 0x7f18a002a070 con 0x7f18b01020f0 2024-03-04T09:23:38.338 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:23:38.338+0000 7f18b4a0e700 1 --2- 172.21.15.191:0/3500362094 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f189c0a1d80 0x7f189c0a4240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:23:38.338 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:23:38.339+0000 7f18a67fc700 1 --2- 172.21.15.191:0/3500362094 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f189c0a1d80 0x7f189c0a4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:23:38.338 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:23:38.339+0000 7f18b4a0e700 1 -- 172.21.15.191:0/3500362094 <== mon.1 v2:172.21.15.191:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 2309+0+0 (secure 0 0 0) 0x7f18a00cdd50 con 0x7f18b01020f0 2024-03-04T09:23:38.338 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:23:38.339+0000 7f18a67fc700 1 --2- 172.21.15.191:0/3500362094 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f189c0a1d80 0x7f189c0a4240 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f18980099d0 tx=0x7f1898008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:23:38.339 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:23:38.342+0000 7f18b4a0e700 1 -- 172.21.15.191:0/3500362094 <== mon.1 v2:172.21.15.191:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f18a0031020 con 0x7f18b01020f0 2024-03-04T09:23:38.541 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:23:38.543+0000 7f18b5a10700 1 -- 172.21.15.191:0/3500362094 --> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi191:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7f18b0062500 con 0x7f189c0a1d80 2024-03-04T09:23:38.541 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:23:38 smithi191 ceph-mon[36816]: Detected new or changed devices on smithi106 2024-03-04T09:23:38.541 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:23:38 smithi191 ceph-mon[36816]: Adjusting osd_memory_target on smithi106 to 17080M 2024-03-04T09:23:38.541 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:23:38 smithi191 ceph-mon[36816]: osdmap e14: 2 total, 2 up, 2 in 2024-03-04T09:23:38.584 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:23:38 smithi052 ceph-mon[32473]: Detected new or changed devices on smithi106 2024-03-04T09:23:38.585 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:23:38 smithi052 ceph-mon[32473]: Adjusting osd_memory_target on smithi106 to 17080M 2024-03-04T09:23:38.585 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:23:38 smithi052 ceph-mon[32473]: osdmap e14: 2 total, 2 up, 2 in 2024-03-04T09:23:38.631 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:23:38 smithi106 ceph-mon[36484]: Detected new or changed devices on smithi106 2024-03-04T09:23:38.631 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:23:38 smithi106 ceph-mon[36484]: Adjusting osd_memory_target on smithi106 to 17080M 2024-03-04T09:23:38.631 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:23:38 smithi106 ceph-mon[36484]: osdmap e14: 2 total, 2 up, 2 in 2024-03-04T09:23:39.422 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:23:39 smithi191 ceph-mon[36816]: pgmap v63: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-04T09:23:39.423 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:23:39 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-04T09:23:39.423 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:23:39 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-04T09:23:39.423 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:23:39 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T09:23:39.585 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:23:39 smithi052 ceph-mon[32473]: pgmap v63: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-04T09:23:39.585 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:23:39 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-04T09:23:39.585 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:23:39 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-04T09:23:39.585 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:23:39 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T09:23:39.631 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:23:39 smithi106 ceph-mon[36484]: pgmap v63: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-04T09:23:39.631 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:23:39 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-04T09:23:39.631 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:23:39 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-04T09:23:39.631 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:23:39 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T09:23:40.336 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:23:40 smithi191 ceph-mon[36816]: from='client.24157 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi191:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T09:23:40.584 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:23:40 smithi052 ceph-mon[32473]: from='client.24157 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi191:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T09:23:40.631 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:23:40 smithi106 ceph-mon[36484]: from='client.24157 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi191:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T09:23:41.381 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:23:41 smithi106 ceph-mon[36484]: pgmap v64: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-04T09:23:41.510 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:23:41 smithi191 ceph-mon[36816]: pgmap v64: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-04T09:23:41.584 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:23:41 smithi052 ceph-mon[32473]: pgmap v64: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-04T09:23:42.584 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:23:42 smithi052 ceph-mon[32473]: from='client.? 172.21.15.191:0/2692570730' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1948bac7-2bb2-4218-8b93-4e4743a0de04"}]: dispatch 2024-03-04T09:23:42.585 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:23:42 smithi052 ceph-mon[32473]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1948bac7-2bb2-4218-8b93-4e4743a0de04"}]: dispatch 2024-03-04T09:23:42.585 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:23:42 smithi052 ceph-mon[32473]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1948bac7-2bb2-4218-8b93-4e4743a0de04"}]': finished 2024-03-04T09:23:42.585 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:23:42 smithi052 ceph-mon[32473]: osdmap e15: 3 total, 2 up, 3 in 2024-03-04T09:23:42.585 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:23:42 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-04T09:23:42.585 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:23:42 smithi052 ceph-mon[32473]: from='client.? 172.21.15.191:0/3150595227' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-04T09:23:42.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:23:42 smithi191 ceph-mon[36816]: from='client.? 172.21.15.191:0/2692570730' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1948bac7-2bb2-4218-8b93-4e4743a0de04"}]: dispatch 2024-03-04T09:23:42.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:23:42 smithi191 ceph-mon[36816]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1948bac7-2bb2-4218-8b93-4e4743a0de04"}]: dispatch 2024-03-04T09:23:42.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:23:42 smithi191 ceph-mon[36816]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1948bac7-2bb2-4218-8b93-4e4743a0de04"}]': finished 2024-03-04T09:23:42.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:23:42 smithi191 ceph-mon[36816]: osdmap e15: 3 total, 2 up, 3 in 2024-03-04T09:23:42.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:23:42 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-04T09:23:42.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:23:42 smithi191 ceph-mon[36816]: from='client.? 172.21.15.191:0/3150595227' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-04T09:23:42.631 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:23:42 smithi106 ceph-mon[36484]: from='client.? 172.21.15.191:0/2692570730' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1948bac7-2bb2-4218-8b93-4e4743a0de04"}]: dispatch 2024-03-04T09:23:42.631 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:23:42 smithi106 ceph-mon[36484]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1948bac7-2bb2-4218-8b93-4e4743a0de04"}]: dispatch 2024-03-04T09:23:42.632 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:23:42 smithi106 ceph-mon[36484]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1948bac7-2bb2-4218-8b93-4e4743a0de04"}]': finished 2024-03-04T09:23:42.632 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:23:42 smithi106 ceph-mon[36484]: osdmap e15: 3 total, 2 up, 3 in 2024-03-04T09:23:42.632 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:23:42 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-04T09:23:42.632 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:23:42 smithi106 ceph-mon[36484]: from='client.? 172.21.15.191:0/3150595227' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-04T09:23:43.585 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:23:43 smithi052 ceph-mon[32473]: pgmap v66: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-04T09:23:43.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:23:43 smithi191 ceph-mon[36816]: pgmap v66: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-04T09:23:43.631 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:23:43 smithi106 ceph-mon[36484]: pgmap v66: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-04T09:23:45.386 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:23:45 smithi191 ceph-mon[36816]: pgmap v67: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-04T09:23:45.584 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:23:45 smithi052 ceph-mon[32473]: pgmap v67: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-04T09:23:45.631 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:23:45 smithi106 ceph-mon[36484]: pgmap v67: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-04T09:23:47.217 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:23:47 smithi191 ceph-mon[36816]: pgmap v68: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-04T09:23:47.584 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:23:47 smithi052 ceph-mon[32473]: pgmap v68: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-04T09:23:47.631 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:23:47 smithi106 ceph-mon[36484]: pgmap v68: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-04T09:23:49.585 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:23:49 smithi052 ceph-mon[32473]: pgmap v69: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-04T09:23:49.585 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:23:49 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-03-04T09:23:49.585 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:23:49 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T09:23:49.585 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:23:49 smithi052 ceph-mon[32473]: Deploying daemon osd.2 on smithi191 2024-03-04T09:23:49.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:23:49 smithi191 ceph-mon[36816]: pgmap v69: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-04T09:23:49.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:23:49 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-03-04T09:23:49.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:23:49 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T09:23:49.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:23:49 smithi191 ceph-mon[36816]: Deploying daemon osd.2 on smithi191 2024-03-04T09:23:49.631 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:23:49 smithi106 ceph-mon[36484]: pgmap v69: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-04T09:23:49.631 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:23:49 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-03-04T09:23:49.631 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:23:49 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T09:23:49.631 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:23:49 smithi106 ceph-mon[36484]: Deploying daemon osd.2 on smithi191 2024-03-04T09:23:51.397 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:23:51 smithi191 ceph-mon[36816]: pgmap v70: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-04T09:23:51.585 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:23:51 smithi052 ceph-mon[32473]: pgmap v70: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-04T09:23:51.631 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:23:51 smithi106 ceph-mon[36484]: pgmap v70: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-04T09:23:53.584 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:23:53 smithi052 ceph-mon[32473]: pgmap v71: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-04T09:23:53.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:23:53 smithi191 ceph-mon[36816]: pgmap v71: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-04T09:23:53.631 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:23:53 smithi106 ceph-mon[36484]: pgmap v71: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-04T09:23:54.835 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:23:54 smithi052 ceph-mon[32473]: pgmap v72: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-04T09:23:54.835 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:23:54 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:23:54.835 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:23:54 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:23:54.835 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:23:54 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T09:23:54.836 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:23:54 smithi191 ceph-mon[36816]: pgmap v72: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-04T09:23:54.836 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:23:54 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:23:54.836 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:23:54 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:23:54.836 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:23:54 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T09:23:54.881 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:23:54 smithi106 ceph-mon[36484]: pgmap v72: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-04T09:23:54.881 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:23:54 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:23:54.881 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:23:54 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:23:54.881 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:23:54 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T09:23:57.209 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:23:57.212+0000 7f18b4a0e700 1 -- 172.21.15.191:0/3500362094 <== mgr.14152 v2:172.21.15.52:6800/3036033923 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f18b0062500 con 0x7f189c0a1d80 2024-03-04T09:23:57.210 INFO:teuthology.orchestra.run.smithi191.stdout:Created osd(s) 2 on host 'smithi191' 2024-03-04T09:23:57.212 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:23:57.214+0000 7f18b5a10700 1 -- 172.21.15.191:0/3500362094 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f189c0a1d80 msgr2=0x7f189c0a4240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:23:57.212 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:23:57.214+0000 7f18b5a10700 1 --2- 172.21.15.191:0/3500362094 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f189c0a1d80 0x7f189c0a4240 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f18980099d0 tx=0x7f1898008040 comp rx=0 tx=0).stop 2024-03-04T09:23:57.212 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:23:57.214+0000 7f18b5a10700 1 -- 172.21.15.191:0/3500362094 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f18b01020f0 msgr2=0x7f18b0198360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:23:57.212 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:23:57.214+0000 7f18b5a10700 1 --2- 172.21.15.191:0/3500362094 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f18b01020f0 0x7f18b0198360 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f18a0005f50 tx=0x7f18a00057c0 comp rx=0 tx=0).stop 2024-03-04T09:23:57.212 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:23:57.214+0000 7f18b5a10700 1 -- 172.21.15.191:0/3500362094 shutdown_connections 2024-03-04T09:23:57.212 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:23:57.214+0000 7f18b5a10700 1 --2- 172.21.15.191:0/3500362094 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f189c0a1d80 0x7f189c0a4240 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:23:57.212 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:23:57.214+0000 7f18b5a10700 1 --2- 172.21.15.191:0/3500362094 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f18b01020f0 0x7f18b0198360 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:23:57.212 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:23:57.214+0000 7f18b5a10700 1 --2- 172.21.15.191:0/3500362094 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f18b0103c90 0x7f18b019f920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:23:57.212 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:23:57.214+0000 7f18b5a10700 1 --2- 172.21.15.191:0/3500362094 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f18b01032f0 0x7f18b01988a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:23:57.212 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:23:57.214+0000 7f18b5a10700 1 -- 172.21.15.191:0/3500362094 >> 172.21.15.191:0/3500362094 conn(0x7f18b00fd6d0 msgr2=0x7f18b01065f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:23:57.213 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:23:57.215+0000 7f18b5a10700 1 -- 172.21.15.191:0/3500362094 shutdown_connections 2024-03-04T09:23:57.213 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:23:57.215+0000 7f18b5a10700 1 -- 172.21.15.191:0/3500362094 wait complete. 2024-03-04T09:23:57.508 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:23:57 smithi191 ceph-mon[36816]: pgmap v73: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-04T09:23:57.508 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:23:57 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:23:57.508 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:23:57 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:23:57.508 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:23:57 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T09:23:57.508 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:23:57 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T09:23:57.508 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:23:57 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:23:57.509 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:23:57 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:23:57.509 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:23:57 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:23:57.509 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:23:57 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T09:23:57.584 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:23:57 smithi052 ceph-mon[32473]: pgmap v73: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-04T09:23:57.585 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:23:57 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:23:57.585 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:23:57 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:23:57.585 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:23:57 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T09:23:57.585 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:23:57 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T09:23:57.585 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:23:57 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:23:57.585 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:23:57 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:23:57.585 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:23:57 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:23:57.585 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:23:57 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T09:23:57.631 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:23:57 smithi106 ceph-mon[36484]: pgmap v73: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-04T09:23:57.631 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:23:57 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:23:57.631 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:23:57 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:23:57.631 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:23:57 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T09:23:57.631 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:23:57 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T09:23:57.631 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:23:57 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:23:57.632 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:23:57 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:23:57.632 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:23:57 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:23:57.632 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:23:57 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T09:23:57.816 DEBUG:teuthology.orchestra.run.smithi191:osd.2> sudo journalctl -f -n 0 -u ceph-4e059f08-da08-11ee-95c6-87774f69a715@osd.2.service 2024-03-04T09:23:57.818 INFO:tasks.cephadm:Deploying osd.3 on smithi052 with /dev/vg_nvme/lv_3... 2024-03-04T09:23:57.818 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4e059f08-da08-11ee-95c6-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-03-04T09:23:58.069 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/4e059f08-da08-11ee-95c6-87774f69a715/mon.a/config 2024-03-04T09:23:58.336 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:23:58 smithi191 ceph-mon[36816]: from='osd.2 [v2:172.21.15.191:6800/1377083164,v1:172.21.15.191:6801/1377083164]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-03-04T09:23:58.336 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:23:58 smithi191 ceph-mon[36816]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-03-04T09:23:58.445 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:23:58 smithi052 ceph-mon[32473]: from='osd.2 [v2:172.21.15.191:6800/1377083164,v1:172.21.15.191:6801/1377083164]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-03-04T09:23:58.445 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:23:58 smithi052 ceph-mon[32473]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-03-04T09:23:58.631 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:23:58 smithi106 ceph-mon[36484]: from='osd.2 [v2:172.21.15.191:6800/1377083164,v1:172.21.15.191:6801/1377083164]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-03-04T09:23:58.631 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:23:58 smithi106 ceph-mon[36484]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-03-04T09:23:59.502 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:23:59 smithi052 ceph-mon[32473]: pgmap v74: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-04T09:23:59.502 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:23:59 smithi052 ceph-mon[32473]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-03-04T09:23:59.502 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:23:59 smithi052 ceph-mon[32473]: osdmap e16: 3 total, 2 up, 3 in 2024-03-04T09:23:59.502 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:23:59 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-04T09:23:59.502 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:23:59 smithi052 ceph-mon[32473]: from='osd.2 [v2:172.21.15.191:6800/1377083164,v1:172.21.15.191:6801/1377083164]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi191", "root=default"]}]: dispatch 2024-03-04T09:23:59.503 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:23:59 smithi052 ceph-mon[32473]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi191", "root=default"]}]: dispatch 2024-03-04T09:23:59.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:23:59 smithi191 ceph-mon[36816]: pgmap v74: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-04T09:23:59.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:23:59 smithi191 ceph-mon[36816]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-03-04T09:23:59.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:23:59 smithi191 ceph-mon[36816]: osdmap e16: 3 total, 2 up, 3 in 2024-03-04T09:23:59.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:23:59 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-04T09:23:59.587 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:23:59 smithi191 ceph-mon[36816]: from='osd.2 [v2:172.21.15.191:6800/1377083164,v1:172.21.15.191:6801/1377083164]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi191", "root=default"]}]: dispatch 2024-03-04T09:23:59.587 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:23:59 smithi191 ceph-mon[36816]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi191", "root=default"]}]: dispatch 2024-03-04T09:23:59.587 INFO:journalctl@ceph.osd.2.smithi191.stdout:Mar 04 09:23:59 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-2[41970]: 2024-03-04T09:23:59.324+0000 7f4818641700 -1 osd.2 0 waiting for initial osdmap 2024-03-04T09:23:59.587 INFO:journalctl@ceph.osd.2.smithi191.stdout:Mar 04 09:23:59 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-2[41970]: 2024-03-04T09:23:59.329+0000 7f4814434700 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-04T09:23:59.631 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:23:59 smithi106 ceph-mon[36484]: pgmap v74: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-04T09:23:59.631 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:23:59 smithi106 ceph-mon[36484]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-03-04T09:23:59.631 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:23:59 smithi106 ceph-mon[36484]: osdmap e16: 3 total, 2 up, 3 in 2024-03-04T09:23:59.631 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:23:59 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-04T09:23:59.631 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:23:59 smithi106 ceph-mon[36484]: from='osd.2 [v2:172.21.15.191:6800/1377083164,v1:172.21.15.191:6801/1377083164]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi191", "root=default"]}]: dispatch 2024-03-04T09:23:59.631 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:23:59 smithi106 ceph-mon[36484]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi191", "root=default"]}]: dispatch 2024-03-04T09:24:00.080 INFO:teuthology.orchestra.run.smithi052.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-03-04T09:24:00.080 INFO:teuthology.orchestra.run.smithi052.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-03-04T09:24:00.081 INFO:teuthology.orchestra.run.smithi052.stderr: stderr: 10+0 records in 2024-03-04T09:24:00.081 INFO:teuthology.orchestra.run.smithi052.stderr:10+0 records out 2024-03-04T09:24:00.081 INFO:teuthology.orchestra.run.smithi052.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.00845256 s, 1.2 GB/s 2024-03-04T09:24:00.081 INFO:teuthology.orchestra.run.smithi052.stderr:--> Zapping successful for: 2024-03-04T09:24:00.393 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:24:00 smithi052 ceph-mon[32473]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi191", "root=default"]}]': finished 2024-03-04T09:24:00.393 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:24:00 smithi052 ceph-mon[32473]: osdmap e17: 3 total, 2 up, 3 in 2024-03-04T09:24:00.393 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:24:00 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-04T09:24:00.393 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:24:00 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-04T09:24:00.394 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:24:00 smithi052 ceph-mon[32473]: pgmap v77: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-04T09:24:00.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:24:00 smithi191 ceph-mon[36816]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi191", "root=default"]}]': finished 2024-03-04T09:24:00.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:24:00 smithi191 ceph-mon[36816]: osdmap e17: 3 total, 2 up, 3 in 2024-03-04T09:24:00.587 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:24:00 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-04T09:24:00.587 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:24:00 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-04T09:24:00.587 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:24:00 smithi191 ceph-mon[36816]: pgmap v77: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-04T09:24:00.631 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:24:00 smithi106 ceph-mon[36484]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi191", "root=default"]}]': finished 2024-03-04T09:24:00.631 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:24:00 smithi106 ceph-mon[36484]: osdmap e17: 3 total, 2 up, 3 in 2024-03-04T09:24:00.631 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:24:00 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-04T09:24:00.631 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:24:00 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-04T09:24:00.631 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:24:00 smithi106 ceph-mon[36484]: pgmap v77: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-04T09:24:00.726 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4e059f08-da08-11ee-95c6-87774f69a715 -- ceph orch daemon add osd smithi052:vg_nvme/lv_3 2024-03-04T09:24:00.969 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/4e059f08-da08-11ee-95c6-87774f69a715/mon.a/config 2024-03-04T09:24:01.335 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:24:01 smithi052 ceph-mon[32473]: purged_snaps scrub starts 2024-03-04T09:24:01.335 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:24:01 smithi052 ceph-mon[32473]: purged_snaps scrub ok 2024-03-04T09:24:01.335 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:24:01 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-04T09:24:01.335 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:24:01 smithi052 ceph-mon[32473]: osd.2 [v2:172.21.15.191:6800/1377083164,v1:172.21.15.191:6801/1377083164] boot 2024-03-04T09:24:01.335 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:24:01 smithi052 ceph-mon[32473]: osdmap e18: 3 total, 3 up, 3 in 2024-03-04T09:24:01.335 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:24:01 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-04T09:24:01.336 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:24:01 smithi052 ceph-mon[32473]: Detected new or changed devices on smithi191 2024-03-04T09:24:01.336 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:24:01 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:24:01.336 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:24:01 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:24:01.336 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:24:01 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-04T09:24:01.336 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:24:01 smithi052 ceph-mon[32473]: Adjusting osd_memory_target on smithi191 to 17080M 2024-03-04T09:24:01.336 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:24:01 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:24:01.336 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:24:01 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T09:24:01.336 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:24:01 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T09:24:01.336 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:24:01 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:24:01.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:24:01 smithi191 ceph-mon[36816]: purged_snaps scrub starts 2024-03-04T09:24:01.587 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:24:01 smithi191 ceph-mon[36816]: purged_snaps scrub ok 2024-03-04T09:24:01.587 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:24:01 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-04T09:24:01.587 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:24:01 smithi191 ceph-mon[36816]: osd.2 [v2:172.21.15.191:6800/1377083164,v1:172.21.15.191:6801/1377083164] boot 2024-03-04T09:24:01.587 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:24:01 smithi191 ceph-mon[36816]: osdmap e18: 3 total, 3 up, 3 in 2024-03-04T09:24:01.587 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:24:01 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-04T09:24:01.587 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:24:01 smithi191 ceph-mon[36816]: Detected new or changed devices on smithi191 2024-03-04T09:24:01.587 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:24:01 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:24:01.587 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:24:01 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:24:01.587 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:24:01 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-04T09:24:01.587 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:24:01 smithi191 ceph-mon[36816]: Adjusting osd_memory_target on smithi191 to 17080M 2024-03-04T09:24:01.588 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:24:01 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:24:01.588 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:24:01 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T09:24:01.588 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:24:01 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T09:24:01.588 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:24:01 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:24:01.631 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:24:01 smithi106 ceph-mon[36484]: purged_snaps scrub starts 2024-03-04T09:24:01.631 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:24:01 smithi106 ceph-mon[36484]: purged_snaps scrub ok 2024-03-04T09:24:01.632 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:24:01 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-04T09:24:01.632 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:24:01 smithi106 ceph-mon[36484]: osd.2 [v2:172.21.15.191:6800/1377083164,v1:172.21.15.191:6801/1377083164] boot 2024-03-04T09:24:01.632 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:24:01 smithi106 ceph-mon[36484]: osdmap e18: 3 total, 3 up, 3 in 2024-03-04T09:24:01.632 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:24:01 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-04T09:24:01.632 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:24:01 smithi106 ceph-mon[36484]: Detected new or changed devices on smithi191 2024-03-04T09:24:01.632 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:24:01 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:24:01.632 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:24:01 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:24:01.632 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:24:01 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-04T09:24:01.632 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:24:01 smithi106 ceph-mon[36484]: Adjusting osd_memory_target on smithi191 to 17080M 2024-03-04T09:24:01.632 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:24:01 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:24:01.632 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:24:01 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T09:24:01.633 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:24:01 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T09:24:01.633 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:24:01 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:24:02.575 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:24:02.577+0000 7f6dd96df700 1 -- 172.21.15.52:0/1838652174 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6dd41020f0 msgr2=0x7f6dd4102510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:24:02.575 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:24:02.577+0000 7f6dd96df700 1 --2- 172.21.15.52:0/1838652174 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6dd41020f0 0x7f6dd4102510 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f6dc40099c0 tx=0x7f6dc4009cd0 comp rx=0 tx=0).stop 2024-03-04T09:24:02.575 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:24:02.577+0000 7f6dd96df700 1 -- 172.21.15.52:0/1838652174 shutdown_connections 2024-03-04T09:24:02.575 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:24:02.577+0000 7f6dd96df700 1 --2- 172.21.15.52:0/1838652174 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6dd4103c90 0x7f6dd410a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:24:02.575 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:24:02.577+0000 7f6dd96df700 1 --2- 172.21.15.52:0/1838652174 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6dd41020f0 0x7f6dd4102510 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:24:02.575 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:24:02.577+0000 7f6dd96df700 1 --2- 172.21.15.52:0/1838652174 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6dd41032f0 0x7f6dd4103750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:24:02.575 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:24:02.577+0000 7f6dd96df700 1 -- 172.21.15.52:0/1838652174 >> 172.21.15.52:0/1838652174 conn(0x7f6dd40fd6d0 msgr2=0x7f6dd40ffaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:24:02.575 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:24:02.578+0000 7f6dd96df700 1 -- 172.21.15.52:0/1838652174 shutdown_connections 2024-03-04T09:24:02.575 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:24:02.578+0000 7f6dd96df700 1 -- 172.21.15.52:0/1838652174 wait complete. 2024-03-04T09:24:02.576 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:24:02.578+0000 7f6dd96df700 1 Processor -- start 2024-03-04T09:24:02.576 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:24:02.578+0000 7f6dd96df700 1 -- start start 2024-03-04T09:24:02.577 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:24:02.578+0000 7f6dd96df700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6dd41020f0 0x7f6dd4075e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:24:02.577 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:24:02.578+0000 7f6dd96df700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6dd41032f0 0x7f6dd40744b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:24:02.577 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:24:02.578+0000 7f6dd96df700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6dd4103c90 0x7f6dd40749f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:24:02.577 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:24:02.578+0000 7f6dd96df700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6dd410de00 con 0x7f6dd41032f0 2024-03-04T09:24:02.577 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:24:02.578+0000 7f6dd96df700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6dd410dc80 con 0x7f6dd4103c90 2024-03-04T09:24:02.577 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:24:02.578+0000 7f6dd96df700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6dd410df80 con 0x7f6dd41020f0 2024-03-04T09:24:02.578 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:24:02.579+0000 7f6dd27fc700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6dd41032f0 0x7f6dd40744b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:24:02.578 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:24:02.579+0000 7f6dd27fc700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6dd41032f0 0x7f6dd40744b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53318/0 (socket says 172.21.15.52:53318) 2024-03-04T09:24:02.578 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:24:02.579+0000 7f6dd27fc700 1 -- 172.21.15.52:0/60579992 learned_addr learned my addr 172.21.15.52:0/60579992 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-04T09:24:02.578 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:24:02.579+0000 7f6dd37fe700 1 --2- 172.21.15.52:0/60579992 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6dd4103c90 0x7f6dd40749f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:24:02.578 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:24:02.579+0000 7f6dd2ffd700 1 --2- 172.21.15.52:0/60579992 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6dd41020f0 0x7f6dd4075e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:24:02.578 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:24:02.579+0000 7f6dd27fc700 1 -- 172.21.15.52:0/60579992 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6dd41020f0 msgr2=0x7f6dd4075e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:24:02.578 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:24:02.579+0000 7f6dd27fc700 1 --2- 172.21.15.52:0/60579992 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6dd41020f0 0x7f6dd4075e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:24:02.578 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:24:02.579+0000 7f6dd27fc700 1 -- 172.21.15.52:0/60579992 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6dd4103c90 msgr2=0x7f6dd40749f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:24:02.578 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:24:02.579+0000 7f6dd27fc700 1 --2- 172.21.15.52:0/60579992 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6dd4103c90 0x7f6dd40749f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:24:02.578 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:24:02.579+0000 7f6dd27fc700 1 -- 172.21.15.52:0/60579992 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6dd40752a0 con 0x7f6dd41032f0 2024-03-04T09:24:02.578 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:24:02.579+0000 7f6dd2ffd700 1 --2- 172.21.15.52:0/60579992 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6dd41020f0 0x7f6dd4075e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-04T09:24:02.578 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:24:02.579+0000 7f6dd27fc700 1 --2- 172.21.15.52:0/60579992 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6dd41032f0 0x7f6dd40744b0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f6dc800b530 tx=0x7f6dc800b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:24:02.578 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:24:02.579+0000 7f6dbbfff700 1 -- 172.21.15.52:0/60579992 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dc8015020 con 0x7f6dd41032f0 2024-03-04T09:24:02.578 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:24:02.579+0000 7f6dd96df700 1 -- 172.21.15.52:0/60579992 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6dd41a2690 con 0x7f6dd41032f0 2024-03-04T09:24:02.578 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:24:02.579+0000 7f6dbbfff700 1 -- 172.21.15.52:0/60579992 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6dc8011d10 con 0x7f6dd41032f0 2024-03-04T09:24:02.579 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:24:02.579+0000 7f6dbbfff700 1 -- 172.21.15.52:0/60579992 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dc8010e50 con 0x7f6dd41032f0 2024-03-04T09:24:02.579 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:24:02.579+0000 7f6dd96df700 1 -- 172.21.15.52:0/60579992 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6dd41a2c20 con 0x7f6dd41032f0 2024-03-04T09:24:02.579 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:24:02.580+0000 7f6dd96df700 1 -- 172.21.15.52:0/60579992 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6dd4068d80 con 0x7f6dd41032f0 2024-03-04T09:24:02.579 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:24:02 smithi052 ceph-mon[32473]: pgmap v79: 0 pgs: ; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-03-04T09:24:02.579 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:24:02 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-03-04T09:24:02.579 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:24:02 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-03-04T09:24:02.579 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:24:02 smithi052 ceph-mon[32473]: osdmap e19: 3 total, 3 up, 3 in 2024-03-04T09:24:02.579 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:24:02 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-03-04T09:24:02.580 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:24:02.582+0000 7f6dbbfff700 1 -- 172.21.15.52:0/60579992 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 15) v1 ==== 136700+0+0 (secure 0 0 0) 0x7f6dc8018020 con 0x7f6dd41032f0 2024-03-04T09:24:02.580 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:24:02.583+0000 7f6dbbfff700 1 --2- 172.21.15.52:0/60579992 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f6dbc0a1f20 0x7f6dbc0a43e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:24:02.580 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:24:02.583+0000 7f6dd2ffd700 1 --2- 172.21.15.52:0/60579992 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f6dbc0a1f20 0x7f6dbc0a43e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:24:02.580 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:24:02.583+0000 7f6dbbfff700 1 -- 172.21.15.52:0/60579992 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 3226+0+0 (secure 0 0 0) 0x7f6dc80cc6f0 con 0x7f6dd41032f0 2024-03-04T09:24:02.580 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:24:02.583+0000 7f6dd2ffd700 1 --2- 172.21.15.52:0/60579992 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f6dbc0a1f20 0x7f6dbc0a43e0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f6dc4009fd0 tx=0x7f6dc4000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:24:02.583 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:24:02.587+0000 7f6dbbfff700 1 -- 172.21.15.52:0/60579992 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f6dc801f020 con 0x7f6dd41032f0 2024-03-04T09:24:02.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:24:02 smithi191 ceph-mon[36816]: pgmap v79: 0 pgs: ; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-03-04T09:24:02.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:24:02 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-03-04T09:24:02.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:24:02 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-03-04T09:24:02.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:24:02 smithi191 ceph-mon[36816]: osdmap e19: 3 total, 3 up, 3 in 2024-03-04T09:24:02.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:24:02 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-03-04T09:24:02.631 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:24:02 smithi106 ceph-mon[36484]: pgmap v79: 0 pgs: ; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-03-04T09:24:02.631 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:24:02 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-03-04T09:24:02.631 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:24:02 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-03-04T09:24:02.631 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:24:02 smithi106 ceph-mon[36484]: osdmap e19: 3 total, 3 up, 3 in 2024-03-04T09:24:02.631 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:24:02 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-03-04T09:24:02.802 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:24:02.805+0000 7f6dd96df700 1 -- 172.21.15.52:0/60579992 --> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi052:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7f6dd4062500 con 0x7f6dbc0a1f20 2024-03-04T09:24:03.368 INFO:journalctl@ceph.osd.0.smithi052.stdout:Mar 04 09:24:03 smithi052 sudo[49346]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-03-04T09:24:03.368 INFO:journalctl@ceph.osd.0.smithi052.stdout:Mar 04 09:24:03 smithi052 sudo[49346]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-04T09:24:03.368 INFO:journalctl@ceph.osd.0.smithi052.stdout:Mar 04 09:24:03 smithi052 sudo[49346]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-04T09:24:03.368 INFO:journalctl@ceph.osd.0.smithi052.stdout:Mar 04 09:24:03 smithi052 sudo[49346]: pam_unix(sudo:session): session closed for user root 2024-03-04T09:24:03.664 INFO:journalctl@ceph.osd.0.smithi052.stdout:Mar 04 09:24:03 smithi052 sudo[49403]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-03-04T09:24:03.664 INFO:journalctl@ceph.osd.0.smithi052.stdout:Mar 04 09:24:03 smithi052 sudo[49403]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-04T09:24:03.664 INFO:journalctl@ceph.osd.0.smithi052.stdout:Mar 04 09:24:03 smithi052 sudo[49403]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-04T09:24:03.664 INFO:journalctl@ceph.osd.0.smithi052.stdout:Mar 04 09:24:03 smithi052 sudo[49403]: pam_unix(sudo:session): session closed for user root 2024-03-04T09:24:03.893 INFO:journalctl@ceph.osd.1.smithi106.stdout:Mar 04 09:24:03 smithi106 sudo[44973]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-03-04T09:24:03.893 INFO:journalctl@ceph.osd.1.smithi106.stdout:Mar 04 09:24:03 smithi106 sudo[44973]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-04T09:24:03.894 INFO:journalctl@ceph.osd.1.smithi106.stdout:Mar 04 09:24:03 smithi106 sudo[44973]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-04T09:24:03.894 INFO:journalctl@ceph.osd.1.smithi106.stdout:Mar 04 09:24:03 smithi106 sudo[44973]: pam_unix(sudo:session): session closed for user root 2024-03-04T09:24:03.894 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:24:03 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-03-04T09:24:03.894 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:24:03 smithi106 ceph-mon[36484]: osdmap e20: 3 total, 3 up, 3 in 2024-03-04T09:24:03.894 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:24:03 smithi106 ceph-mon[36484]: from='client.14286 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi052:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T09:24:03.894 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:24:03 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-04T09:24:03.894 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:24:03 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-04T09:24:03.894 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:24:03 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T09:24:03.894 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:24:03 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:24:04.084 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:24:03 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-03-04T09:24:04.085 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:24:03 smithi052 ceph-mon[32473]: osdmap e20: 3 total, 3 up, 3 in 2024-03-04T09:24:04.085 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:24:03 smithi052 ceph-mon[32473]: from='client.14286 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi052:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T09:24:04.085 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:24:03 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-04T09:24:04.085 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:24:03 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-04T09:24:04.085 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:24:03 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T09:24:04.085 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:24:03 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:24:04.086 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:24:03 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-03-04T09:24:04.086 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:24:03 smithi191 ceph-mon[36816]: osdmap e20: 3 total, 3 up, 3 in 2024-03-04T09:24:04.086 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:24:03 smithi191 ceph-mon[36816]: from='client.14286 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi052:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T09:24:04.086 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:24:03 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-04T09:24:04.086 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:24:03 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-04T09:24:04.086 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:24:03 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T09:24:04.086 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:24:03 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:24:04.381 INFO:journalctl@ceph.osd.1.smithi106.stdout:Mar 04 09:24:03 smithi106 sudo[44976]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-03-04T09:24:04.382 INFO:journalctl@ceph.osd.1.smithi106.stdout:Mar 04 09:24:03 smithi106 sudo[44976]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-04T09:24:04.382 INFO:journalctl@ceph.osd.1.smithi106.stdout:Mar 04 09:24:03 smithi106 sudo[44976]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-04T09:24:04.382 INFO:journalctl@ceph.osd.1.smithi106.stdout:Mar 04 09:24:03 smithi106 sudo[44976]: pam_unix(sudo:session): session closed for user root 2024-03-04T09:24:04.417 INFO:journalctl@ceph.osd.2.smithi191.stdout:Mar 04 09:24:04 smithi191 sudo[46053]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-03-04T09:24:04.417 INFO:journalctl@ceph.osd.2.smithi191.stdout:Mar 04 09:24:04 smithi191 sudo[46053]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-04T09:24:04.418 INFO:journalctl@ceph.osd.2.smithi191.stdout:Mar 04 09:24:04 smithi191 sudo[46053]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-04T09:24:04.418 INFO:journalctl@ceph.osd.2.smithi191.stdout:Mar 04 09:24:04 smithi191 sudo[46053]: pam_unix(sudo:session): session closed for user root 2024-03-04T09:24:04.836 INFO:journalctl@ceph.osd.2.smithi191.stdout:Mar 04 09:24:04 smithi191 sudo[46056]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-03-04T09:24:04.836 INFO:journalctl@ceph.osd.2.smithi191.stdout:Mar 04 09:24:04 smithi191 sudo[46056]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-04T09:24:04.836 INFO:journalctl@ceph.osd.2.smithi191.stdout:Mar 04 09:24:04 smithi191 sudo[46056]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-04T09:24:04.836 INFO:journalctl@ceph.osd.2.smithi191.stdout:Mar 04 09:24:04 smithi191 sudo[46056]: pam_unix(sudo:session): session closed for user root 2024-03-04T09:24:04.930 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:24:04 smithi052 sudo[49453]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-03-04T09:24:04.930 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:24:04 smithi052 sudo[49453]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-04T09:24:04.930 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:24:04 smithi052 sudo[49453]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-04T09:24:04.930 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:24:04 smithi052 sudo[49453]: pam_unix(sudo:session): session closed for user root 2024-03-04T09:24:04.988 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:24:04.990+0000 7f6dbbfff700 1 -- 172.21.15.52:0/60579992 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mgrmap(e 16) v1 ==== 136756+0+0 (secure 0 0 0) 0x7f6dc8011830 con 0x7f6dd41032f0 2024-03-04T09:24:05.231 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:24:04 smithi191 ceph-mon[36816]: pgmap v82: 1 pgs: 1 creating+peering; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-03-04T09:24:05.231 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:24:04 smithi191 ceph-mon[36816]: osdmap e21: 3 total, 3 up, 3 in 2024-03-04T09:24:05.231 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:24:04 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:24:05.231 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:24:04 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:24:05.231 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:24:04 smithi191 ceph-mon[36816]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-04T09:24:05.232 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:24:05 smithi191 sudo[46059]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-03-04T09:24:05.232 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:24:05 smithi191 sudo[46059]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-04T09:24:05.232 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:24:05 smithi191 sudo[46059]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-04T09:24:05.232 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:24:05 smithi191 sudo[46059]: pam_unix(sudo:session): session closed for user root 2024-03-04T09:24:05.335 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:24:04 smithi052 sudo[49461]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-03-04T09:24:05.335 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:24:04 smithi052 sudo[49461]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-04T09:24:05.335 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:24:04 smithi052 sudo[49461]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-04T09:24:05.335 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:24:04 smithi052 sudo[49461]: pam_unix(sudo:session): session closed for user root 2024-03-04T09:24:05.335 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:24:04 smithi052 ceph-mon[32473]: pgmap v82: 1 pgs: 1 creating+peering; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-03-04T09:24:05.335 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:24:04 smithi052 ceph-mon[32473]: osdmap e21: 3 total, 3 up, 3 in 2024-03-04T09:24:05.335 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:24:04 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:24:05.335 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:24:04 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:24:05.335 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:24:04 smithi052 ceph-mon[32473]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-04T09:24:05.381 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:24:04 smithi106 ceph-mon[36484]: pgmap v82: 1 pgs: 1 creating+peering; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-03-04T09:24:05.381 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:24:04 smithi106 ceph-mon[36484]: osdmap e21: 3 total, 3 up, 3 in 2024-03-04T09:24:05.381 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:24:04 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:24:05.381 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:24:04 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:24:05.382 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:24:04 smithi106 ceph-mon[36484]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-04T09:24:05.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:24:05 smithi191 sudo[46062]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-03-04T09:24:05.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:24:05 smithi191 sudo[46062]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-04T09:24:05.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:24:05 smithi191 sudo[46062]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-04T09:24:05.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:24:05 smithi191 sudo[46062]: pam_unix(sudo:session): session closed for user root 2024-03-04T09:24:05.961 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:24:05 smithi106 sudo[44979]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-03-04T09:24:05.961 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:24:05 smithi106 sudo[44979]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-04T09:24:05.961 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:24:05 smithi106 sudo[44979]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-04T09:24:05.961 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:24:05 smithi106 sudo[44979]: pam_unix(sudo:session): session closed for user root 2024-03-04T09:24:05.961 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:24:05 smithi106 sudo[44982]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-03-04T09:24:06.335 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:24:05 smithi052 ceph-mon[32473]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-04T09:24:06.335 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:24:05 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-04T09:24:06.335 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:24:05 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-04T09:24:06.335 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:24:05 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-04T09:24:06.335 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:24:05 smithi052 ceph-mon[32473]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-04T09:24:06.335 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:24:05 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-04T09:24:06.335 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:24:05 smithi052 ceph-mon[32473]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-03-04T09:24:06.336 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:24:05 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-04T09:24:06.336 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:24:05 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-04T09:24:06.336 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:24:05 smithi052 ceph-mon[32473]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-04T09:24:06.336 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:24:05 smithi052 ceph-mon[32473]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-04T09:24:06.336 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:24:05 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-04T09:24:06.336 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:24:05 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-04T09:24:06.336 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:24:05 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-04T09:24:06.336 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:24:05 smithi052 ceph-mon[32473]: from='client.? 172.21.15.52:0/3474088507' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "16c9e9fe-7167-4992-bf12-1a7398e0498a"}]: dispatch 2024-03-04T09:24:06.336 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:24:05 smithi052 ceph-mon[32473]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-04T09:24:06.336 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:24:05 smithi052 ceph-mon[32473]: from='client.? 172.21.15.52:0/3474088507' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "16c9e9fe-7167-4992-bf12-1a7398e0498a"}]': finished 2024-03-04T09:24:06.336 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:24:05 smithi052 ceph-mon[32473]: osdmap e22: 4 total, 3 up, 4 in 2024-03-04T09:24:06.336 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:24:05 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-04T09:24:06.337 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:24:05 smithi191 ceph-mon[36816]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-04T09:24:06.337 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:24:05 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-04T09:24:06.337 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:24:05 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-04T09:24:06.337 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:24:05 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-04T09:24:06.337 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:24:05 smithi191 ceph-mon[36816]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-04T09:24:06.337 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:24:05 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-04T09:24:06.337 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:24:05 smithi191 ceph-mon[36816]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-03-04T09:24:06.337 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:24:05 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-04T09:24:06.337 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:24:05 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-04T09:24:06.337 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:24:05 smithi191 ceph-mon[36816]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-04T09:24:06.338 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:24:05 smithi191 ceph-mon[36816]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-04T09:24:06.338 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:24:05 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-04T09:24:06.338 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:24:05 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-04T09:24:06.338 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:24:05 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-04T09:24:06.338 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:24:05 smithi191 ceph-mon[36816]: from='client.? 172.21.15.52:0/3474088507' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "16c9e9fe-7167-4992-bf12-1a7398e0498a"}]: dispatch 2024-03-04T09:24:06.338 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:24:05 smithi191 ceph-mon[36816]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-04T09:24:06.338 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:24:05 smithi191 ceph-mon[36816]: from='client.? 172.21.15.52:0/3474088507' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "16c9e9fe-7167-4992-bf12-1a7398e0498a"}]': finished 2024-03-04T09:24:06.338 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:24:05 smithi191 ceph-mon[36816]: osdmap e22: 4 total, 3 up, 4 in 2024-03-04T09:24:06.338 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:24:05 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-04T09:24:06.381 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:24:05 smithi106 sudo[44982]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-04T09:24:06.381 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:24:05 smithi106 sudo[44982]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-04T09:24:06.381 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:24:05 smithi106 sudo[44982]: pam_unix(sudo:session): session closed for user root 2024-03-04T09:24:06.381 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:24:05 smithi106 ceph-mon[36484]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-04T09:24:06.382 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:24:05 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-04T09:24:06.382 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:24:05 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-04T09:24:06.382 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:24:05 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-04T09:24:06.382 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:24:05 smithi106 ceph-mon[36484]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-04T09:24:06.382 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:24:05 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-04T09:24:06.382 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:24:05 smithi106 ceph-mon[36484]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-03-04T09:24:06.382 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:24:05 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-04T09:24:06.382 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:24:05 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-04T09:24:06.382 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:24:05 smithi106 ceph-mon[36484]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-04T09:24:06.382 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:24:05 smithi106 ceph-mon[36484]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-04T09:24:06.382 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:24:05 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-04T09:24:06.382 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:24:05 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-04T09:24:06.382 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:24:05 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-04T09:24:06.383 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:24:05 smithi106 ceph-mon[36484]: from='client.? 172.21.15.52:0/3474088507' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "16c9e9fe-7167-4992-bf12-1a7398e0498a"}]: dispatch 2024-03-04T09:24:06.383 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:24:05 smithi106 ceph-mon[36484]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-04T09:24:06.383 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:24:05 smithi106 ceph-mon[36484]: from='client.? 172.21.15.52:0/3474088507' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "16c9e9fe-7167-4992-bf12-1a7398e0498a"}]': finished 2024-03-04T09:24:06.383 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:24:05 smithi106 ceph-mon[36484]: osdmap e22: 4 total, 3 up, 4 in 2024-03-04T09:24:06.383 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:24:05 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-04T09:24:07.335 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:24:06 smithi052 ceph-mon[32473]: pgmap v84: 1 pgs: 1 creating+peering; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-03-04T09:24:07.335 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:24:06 smithi052 ceph-mon[32473]: from='client.? 172.21.15.52:0/1157619875' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-04T09:24:07.336 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:24:06 smithi191 ceph-mon[36816]: pgmap v84: 1 pgs: 1 creating+peering; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-03-04T09:24:07.336 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:24:06 smithi191 ceph-mon[36816]: from='client.? 172.21.15.52:0/1157619875' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-04T09:24:07.381 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:24:06 smithi106 ceph-mon[36484]: pgmap v84: 1 pgs: 1 creating+peering; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-03-04T09:24:07.381 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:24:06 smithi106 ceph-mon[36484]: from='client.? 172.21.15.52:0/1157619875' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-04T09:24:09.335 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:24:08 smithi052 ceph-mon[32473]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-03-04T09:24:09.336 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:24:08 smithi191 ceph-mon[36816]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-03-04T09:24:09.381 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:24:08 smithi106 ceph-mon[36484]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-03-04T09:24:11.131 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:24:10 smithi106 ceph-mon[36484]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-03-04T09:24:11.230 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:24:10 smithi052 ceph-mon[32473]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-03-04T09:24:11.336 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:24:10 smithi191 ceph-mon[36816]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-03-04T09:24:13.217 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:24:12 smithi052 ceph-mon[32473]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-03-04T09:24:13.336 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:24:12 smithi191 ceph-mon[36816]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-03-04T09:24:13.381 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:24:12 smithi106 ceph-mon[36484]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-03-04T09:24:14.177 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:24:13 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-03-04T09:24:14.177 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:24:13 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T09:24:14.178 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:24:13 smithi052 ceph-mon[32473]: Deploying daemon osd.3 on smithi052 2024-03-04T09:24:14.336 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:24:13 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-03-04T09:24:14.336 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:24:13 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T09:24:14.336 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:24:13 smithi191 ceph-mon[36816]: Deploying daemon osd.3 on smithi052 2024-03-04T09:24:14.381 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:24:14 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-03-04T09:24:14.381 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:24:14 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T09:24:14.381 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:24:14 smithi106 ceph-mon[36484]: Deploying daemon osd.3 on smithi052 2024-03-04T09:24:15.098 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:24:15 smithi052 ceph-mon[32473]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-03-04T09:24:15.336 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:24:15 smithi191 ceph-mon[36816]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-03-04T09:24:15.381 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:24:15 smithi106 ceph-mon[36484]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-03-04T09:24:17.085 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:24:17 smithi052 ceph-mon[32473]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-03-04T09:24:17.336 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:24:17 smithi191 ceph-mon[36816]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-03-04T09:24:17.381 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:24:17 smithi106 ceph-mon[36484]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-03-04T09:24:19.334 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:24:19 smithi052 ceph-mon[32473]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-03-04T09:24:19.335 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:24:19 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:24:19.335 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:24:19 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:24:19.335 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:24:19 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T09:24:19.336 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:24:19 smithi191 ceph-mon[36816]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-03-04T09:24:19.336 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:24:19 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:24:19.336 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:24:19 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:24:19.336 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:24:19 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T09:24:19.381 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:24:19 smithi106 ceph-mon[36484]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-03-04T09:24:19.381 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:24:19 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:24:19.381 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:24:19 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:24:19.381 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:24:19 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T09:24:21.085 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:24:21 smithi052 ceph-mon[32473]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-03-04T09:24:21.336 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:24:21 smithi191 ceph-mon[36816]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-03-04T09:24:21.381 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:24:21 smithi106 ceph-mon[36484]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-03-04T09:24:22.110 INFO:teuthology.orchestra.run.smithi052.stdout:Created osd(s) 3 on host 'smithi052' 2024-03-04T09:24:22.111 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:24:22.112+0000 7f6dbbfff700 1 -- 172.21.15.52:0/60579992 <== mgr.14152 v2:172.21.15.52:6800/3036033923 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f6dd4062500 con 0x7f6dbc0a1f20 2024-03-04T09:24:22.112 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:24:22.115+0000 7f6dd96df700 1 -- 172.21.15.52:0/60579992 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f6dbc0a1f20 msgr2=0x7f6dbc0a43e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:24:22.112 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:24:22.115+0000 7f6dd96df700 1 --2- 172.21.15.52:0/60579992 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f6dbc0a1f20 0x7f6dbc0a43e0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f6dc4009fd0 tx=0x7f6dc4000bc0 comp rx=0 tx=0).stop 2024-03-04T09:24:22.112 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:24:22.115+0000 7f6dd96df700 1 -- 172.21.15.52:0/60579992 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6dd41032f0 msgr2=0x7f6dd40744b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:24:22.112 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:24:22.115+0000 7f6dd96df700 1 --2- 172.21.15.52:0/60579992 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6dd41032f0 0x7f6dd40744b0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f6dc800b530 tx=0x7f6dc800b8f0 comp rx=0 tx=0).stop 2024-03-04T09:24:22.112 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:24:22.115+0000 7f6dd96df700 1 -- 172.21.15.52:0/60579992 shutdown_connections 2024-03-04T09:24:22.112 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:24:22.115+0000 7f6dd96df700 1 --2- 172.21.15.52:0/60579992 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f6dbc0a1f20 0x7f6dbc0a43e0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:24:22.112 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:24:22.115+0000 7f6dd96df700 1 --2- 172.21.15.52:0/60579992 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6dd41020f0 0x7f6dd4075e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:24:22.112 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:24:22.115+0000 7f6dd96df700 1 --2- 172.21.15.52:0/60579992 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6dd4103c90 0x7f6dd40749f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:24:22.112 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:24:22.115+0000 7f6dd96df700 1 --2- 172.21.15.52:0/60579992 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6dd41032f0 0x7f6dd40744b0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:24:22.113 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:24:22.115+0000 7f6dd96df700 1 -- 172.21.15.52:0/60579992 >> 172.21.15.52:0/60579992 conn(0x7f6dd40fd6d0 msgr2=0x7f6dd4106730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:24:22.113 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:24:22.115+0000 7f6dd96df700 1 -- 172.21.15.52:0/60579992 shutdown_connections 2024-03-04T09:24:22.113 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:24:22.115+0000 7f6dd96df700 1 -- 172.21.15.52:0/60579992 wait complete. 2024-03-04T09:24:22.687 DEBUG:teuthology.orchestra.run.smithi052:osd.3> sudo journalctl -f -n 0 -u ceph-4e059f08-da08-11ee-95c6-87774f69a715@osd.3.service 2024-03-04T09:24:22.689 INFO:tasks.cephadm:Deploying osd.4 on smithi106 with /dev/vg_nvme/lv_3... 2024-03-04T09:24:22.689 DEBUG:teuthology.orchestra.run.smithi106:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4e059f08-da08-11ee-95c6-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-03-04T09:24:22.932 INFO:teuthology.orchestra.run.smithi106.stderr:Inferring config /var/lib/ceph/4e059f08-da08-11ee-95c6-87774f69a715/mon.b/config 2024-03-04T09:24:23.260 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:24:23 smithi052 ceph-mon[32473]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-03-04T09:24:23.260 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:24:23 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:24:23.260 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:24:23 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:24:23.260 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:24:23 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:24:23.260 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:24:23 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:24:23.260 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:24:23 smithi052 ceph-mon[32473]: from='osd.3 [v2:172.21.15.52:6810/2388571810,v1:172.21.15.52:6811/2388571810]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-03-04T09:24:23.310 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:24:23 smithi106 ceph-mon[36484]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-03-04T09:24:23.310 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:24:23 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:24:23.310 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:24:23 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:24:23.310 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:24:23 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:24:23.310 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:24:23 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:24:23.310 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:24:23 smithi106 ceph-mon[36484]: from='osd.3 [v2:172.21.15.52:6810/2388571810,v1:172.21.15.52:6811/2388571810]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-03-04T09:24:23.336 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:24:23 smithi191 ceph-mon[36816]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-03-04T09:24:23.336 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:24:23 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:24:23.336 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:24:23 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:24:23.337 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:24:23 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:24:23.337 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:24:23 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:24:23.337 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:24:23 smithi191 ceph-mon[36816]: from='osd.3 [v2:172.21.15.52:6810/2388571810,v1:172.21.15.52:6811/2388571810]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-03-04T09:24:24.585 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:24:24 smithi052 ceph-mon[32473]: from='osd.3 [v2:172.21.15.52:6810/2388571810,v1:172.21.15.52:6811/2388571810]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-03-04T09:24:24.585 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:24:24 smithi052 ceph-mon[32473]: osdmap e23: 4 total, 3 up, 4 in 2024-03-04T09:24:24.585 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:24:24 smithi052 ceph-mon[32473]: from='osd.3 [v2:172.21.15.52:6810/2388571810,v1:172.21.15.52:6811/2388571810]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi052", "root=default"]}]: dispatch 2024-03-04T09:24:24.585 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:24:24 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-04T09:24:24.585 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:24:24 smithi052 ceph-mon[32473]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-03-04T09:24:24.631 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:24:24 smithi106 ceph-mon[36484]: from='osd.3 [v2:172.21.15.52:6810/2388571810,v1:172.21.15.52:6811/2388571810]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-03-04T09:24:24.631 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:24:24 smithi106 ceph-mon[36484]: osdmap e23: 4 total, 3 up, 4 in 2024-03-04T09:24:24.631 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:24:24 smithi106 ceph-mon[36484]: from='osd.3 [v2:172.21.15.52:6810/2388571810,v1:172.21.15.52:6811/2388571810]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi052", "root=default"]}]: dispatch 2024-03-04T09:24:24.631 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:24:24 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-04T09:24:24.631 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:24:24 smithi106 ceph-mon[36484]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-03-04T09:24:24.836 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:24:24 smithi191 ceph-mon[36816]: from='osd.3 [v2:172.21.15.52:6810/2388571810,v1:172.21.15.52:6811/2388571810]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-03-04T09:24:24.836 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:24:24 smithi191 ceph-mon[36816]: osdmap e23: 4 total, 3 up, 4 in 2024-03-04T09:24:24.836 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:24:24 smithi191 ceph-mon[36816]: from='osd.3 [v2:172.21.15.52:6810/2388571810,v1:172.21.15.52:6811/2388571810]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi052", "root=default"]}]: dispatch 2024-03-04T09:24:24.837 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:24:24 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-04T09:24:24.837 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:24:24 smithi191 ceph-mon[36816]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-03-04T09:24:24.915 INFO:teuthology.orchestra.run.smithi106.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-03-04T09:24:24.915 INFO:teuthology.orchestra.run.smithi106.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-03-04T09:24:24.915 INFO:teuthology.orchestra.run.smithi106.stderr: stderr: 10+0 records in 2024-03-04T09:24:24.915 INFO:teuthology.orchestra.run.smithi106.stderr:10+0 records out 2024-03-04T09:24:24.915 INFO:teuthology.orchestra.run.smithi106.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0112877 s, 929 MB/s 2024-03-04T09:24:24.915 INFO:teuthology.orchestra.run.smithi106.stderr:--> Zapping successful for: 2024-03-04T09:24:25.335 INFO:journalctl@ceph.osd.3.smithi052.stdout:Mar 04 09:24:24 smithi052 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-3[51542]: 2024-03-04T09:24:24.937+0000 7f159b207700 -1 osd.3 0 waiting for initial osdmap 2024-03-04T09:24:25.335 INFO:journalctl@ceph.osd.3.smithi052.stdout:Mar 04 09:24:24 smithi052 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-3[51542]: 2024-03-04T09:24:24.944+0000 7f1592ff2700 -1 osd.3 24 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-04T09:24:25.528 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:24:25 smithi106 ceph-mon[36484]: from='osd.3 [v2:172.21.15.52:6810/2388571810,v1:172.21.15.52:6811/2388571810]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi052", "root=default"]}]': finished 2024-03-04T09:24:25.528 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:24:25 smithi106 ceph-mon[36484]: osdmap e24: 4 total, 3 up, 4 in 2024-03-04T09:24:25.528 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:24:25 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-04T09:24:25.529 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:24:25 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-04T09:24:25.529 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:24:25 smithi106 ceph-mon[36484]: from='osd.3 [v2:172.21.15.52:6810/2388571810,v1:172.21.15.52:6811/2388571810]' entity='osd.3' 2024-03-04T09:24:25.556 DEBUG:teuthology.orchestra.run.smithi106:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4e059f08-da08-11ee-95c6-87774f69a715 -- ceph orch daemon add osd smithi106:vg_nvme/lv_3 2024-03-04T09:24:25.706 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:24:25 smithi052 ceph-mon[32473]: from='osd.3 [v2:172.21.15.52:6810/2388571810,v1:172.21.15.52:6811/2388571810]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi052", "root=default"]}]': finished 2024-03-04T09:24:25.706 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:24:25 smithi052 ceph-mon[32473]: osdmap e24: 4 total, 3 up, 4 in 2024-03-04T09:24:25.706 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:24:25 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-04T09:24:25.706 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:24:25 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-04T09:24:25.706 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:24:25 smithi052 ceph-mon[32473]: from='osd.3 [v2:172.21.15.52:6810/2388571810,v1:172.21.15.52:6811/2388571810]' entity='osd.3' 2024-03-04T09:24:25.796 INFO:teuthology.orchestra.run.smithi106.stderr:Inferring config /var/lib/ceph/4e059f08-da08-11ee-95c6-87774f69a715/mon.b/config 2024-03-04T09:24:25.836 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:24:25 smithi191 ceph-mon[36816]: from='osd.3 [v2:172.21.15.52:6810/2388571810,v1:172.21.15.52:6811/2388571810]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi052", "root=default"]}]': finished 2024-03-04T09:24:25.836 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:24:25 smithi191 ceph-mon[36816]: osdmap e24: 4 total, 3 up, 4 in 2024-03-04T09:24:25.836 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:24:25 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-04T09:24:25.836 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:24:25 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-04T09:24:25.836 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:24:25 smithi191 ceph-mon[36816]: from='osd.3 [v2:172.21.15.52:6810/2388571810,v1:172.21.15.52:6811/2388571810]' entity='osd.3' 2024-03-04T09:24:26.627 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:24:26 smithi106 ceph-mon[36484]: purged_snaps scrub starts 2024-03-04T09:24:26.627 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:24:26 smithi106 ceph-mon[36484]: purged_snaps scrub ok 2024-03-04T09:24:26.627 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:24:26 smithi106 ceph-mon[36484]: osd.3 [v2:172.21.15.52:6810/2388571810,v1:172.21.15.52:6811/2388571810] boot 2024-03-04T09:24:26.627 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:24:26 smithi106 ceph-mon[36484]: osdmap e25: 4 total, 4 up, 4 in 2024-03-04T09:24:26.627 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:24:26 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-04T09:24:26.627 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:24:26 smithi106 ceph-mon[36484]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 508 MiB used, 357 GiB / 358 GiB avail 2024-03-04T09:24:26.627 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:24:26 smithi106 ceph-mon[36484]: osdmap e26: 4 total, 4 up, 4 in 2024-03-04T09:24:26.627 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:24:26 smithi106 ceph-mon[36484]: Detected new or changed devices on smithi052 2024-03-04T09:24:26.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:24:26 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:24:26.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:24:26 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:24:26.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:24:26 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-04T09:24:26.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:24:26 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-04T09:24:26.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:24:26 smithi106 ceph-mon[36484]: Adjusting osd_memory_target on smithi052 to 8546M 2024-03-04T09:24:26.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:24:26 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:24:26.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:24:26 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T09:24:26.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:24:26 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T09:24:26.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:24:26 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:24:26.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:24:26 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T09:24:26.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:24:26 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T09:24:26.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:24:26 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T09:24:26.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:24:26 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:24:26.835 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:24:26 smithi052 ceph-mon[32473]: purged_snaps scrub starts 2024-03-04T09:24:26.835 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:24:26 smithi052 ceph-mon[32473]: purged_snaps scrub ok 2024-03-04T09:24:26.835 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:24:26 smithi052 ceph-mon[32473]: osd.3 [v2:172.21.15.52:6810/2388571810,v1:172.21.15.52:6811/2388571810] boot 2024-03-04T09:24:26.835 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:24:26 smithi052 ceph-mon[32473]: osdmap e25: 4 total, 4 up, 4 in 2024-03-04T09:24:26.835 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:24:26 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-04T09:24:26.835 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:24:26 smithi052 ceph-mon[32473]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 508 MiB used, 357 GiB / 358 GiB avail 2024-03-04T09:24:26.835 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:24:26 smithi052 ceph-mon[32473]: osdmap e26: 4 total, 4 up, 4 in 2024-03-04T09:24:26.835 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:24:26 smithi052 ceph-mon[32473]: Detected new or changed devices on smithi052 2024-03-04T09:24:26.835 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:24:26 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:24:26.836 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:24:26 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:24:26.836 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:24:26 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-04T09:24:26.836 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:24:26 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-04T09:24:26.836 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:24:26 smithi052 ceph-mon[32473]: Adjusting osd_memory_target on smithi052 to 8546M 2024-03-04T09:24:26.836 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:24:26 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:24:26.836 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:24:26 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T09:24:26.836 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:24:26 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T09:24:26.836 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:24:26 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:24:26.836 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:24:26 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T09:24:26.836 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:24:26 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T09:24:26.836 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:24:26 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T09:24:26.836 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:24:26 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:24:26.837 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:24:26 smithi191 ceph-mon[36816]: purged_snaps scrub starts 2024-03-04T09:24:26.837 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:24:26 smithi191 ceph-mon[36816]: purged_snaps scrub ok 2024-03-04T09:24:26.837 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:24:26 smithi191 ceph-mon[36816]: osd.3 [v2:172.21.15.52:6810/2388571810,v1:172.21.15.52:6811/2388571810] boot 2024-03-04T09:24:26.837 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:24:26 smithi191 ceph-mon[36816]: osdmap e25: 4 total, 4 up, 4 in 2024-03-04T09:24:26.837 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:24:26 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-04T09:24:26.837 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:24:26 smithi191 ceph-mon[36816]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 508 MiB used, 357 GiB / 358 GiB avail 2024-03-04T09:24:26.837 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:24:26 smithi191 ceph-mon[36816]: osdmap e26: 4 total, 4 up, 4 in 2024-03-04T09:24:26.837 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:24:26 smithi191 ceph-mon[36816]: Detected new or changed devices on smithi052 2024-03-04T09:24:26.837 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:24:26 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:24:26.837 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:24:26 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:24:26.837 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:24:26 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-04T09:24:26.838 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:24:26 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-04T09:24:26.838 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:24:26 smithi191 ceph-mon[36816]: Adjusting osd_memory_target on smithi052 to 8546M 2024-03-04T09:24:26.838 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:24:26 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:24:26.838 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:24:26 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T09:24:26.838 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:24:26 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T09:24:26.838 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:24:26 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:24:26.838 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:24:26 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T09:24:26.838 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:24:26 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T09:24:26.838 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:24:26 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T09:24:26.838 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:24:26 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:24:27.496 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-04T09:24:27.498+0000 7ff4b08e0700 1 -- 172.21.15.106:0/631692045 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff4ac100400 msgr2=0x7ff4ac100820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:24:27.496 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-04T09:24:27.498+0000 7ff4b08e0700 1 --2- 172.21.15.106:0/631692045 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff4ac100400 0x7ff4ac100820 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7ff4a0009990 tx=0x7ff4a0009ca0 comp rx=0 tx=0).stop 2024-03-04T09:24:27.496 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-04T09:24:27.498+0000 7ff4b08e0700 1 -- 172.21.15.106:0/631692045 shutdown_connections 2024-03-04T09:24:27.496 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-04T09:24:27.498+0000 7ff4b08e0700 1 --2- 172.21.15.106:0/631692045 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff4ac103d80 0x7ff4ac10a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:24:27.496 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-04T09:24:27.498+0000 7ff4b08e0700 1 --2- 172.21.15.106:0/631692045 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff4ac100400 0x7ff4ac100820 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:24:27.496 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-04T09:24:27.498+0000 7ff4b08e0700 1 --2- 172.21.15.106:0/631692045 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff4ac1033e0 0x7ff4ac103840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:24:27.496 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-04T09:24:27.498+0000 7ff4b08e0700 1 -- 172.21.15.106:0/631692045 >> 172.21.15.106:0/631692045 conn(0x7ff4ac0fbd60 msgr2=0x7ff4ac0fe1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:24:27.496 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-04T09:24:27.498+0000 7ff4b08e0700 1 -- 172.21.15.106:0/631692045 shutdown_connections 2024-03-04T09:24:27.497 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-04T09:24:27.498+0000 7ff4b08e0700 1 -- 172.21.15.106:0/631692045 wait complete. 2024-03-04T09:24:27.497 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-04T09:24:27.499+0000 7ff4b08e0700 1 Processor -- start 2024-03-04T09:24:27.497 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-04T09:24:27.499+0000 7ff4b08e0700 1 -- start start 2024-03-04T09:24:27.497 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-04T09:24:27.499+0000 7ff4b08e0700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff4ac100400 0x7ff4ac198310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:24:27.497 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-04T09:24:27.499+0000 7ff4b08e0700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff4ac1033e0 0x7ff4ac198850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:24:27.497 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-04T09:24:27.499+0000 7ff4b08e0700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff4ac103d80 0x7ff4ac19f8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:24:27.497 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-04T09:24:27.499+0000 7ff4b08e0700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4ac10dcf0 con 0x7ff4ac1033e0 2024-03-04T09:24:27.497 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-04T09:24:27.499+0000 7ff4b08e0700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4ac10db70 con 0x7ff4ac103d80 2024-03-04T09:24:27.497 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-04T09:24:27.499+0000 7ff4b08e0700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4ac10de70 con 0x7ff4ac100400 2024-03-04T09:24:27.497 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-04T09:24:27.500+0000 7ff4aad9d700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff4ac103d80 0x7ff4ac19f8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:24:27.497 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-04T09:24:27.500+0000 7ff4aa59c700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff4ac100400 0x7ff4ac198310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:24:27.497 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-04T09:24:27.500+0000 7ff4aad9d700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff4ac103d80 0x7ff4ac19f8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:36258/0 (socket says 172.21.15.106:36258) 2024-03-04T09:24:27.497 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-04T09:24:27.500+0000 7ff4aa59c700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff4ac100400 0x7ff4ac198310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.106:60682/0 (socket says 172.21.15.106:60682) 2024-03-04T09:24:27.497 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-04T09:24:27.500+0000 7ff4aad9d700 1 -- 172.21.15.106:0/1458256903 learned_addr learned my addr 172.21.15.106:0/1458256903 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-03-04T09:24:27.498 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-04T09:24:27.500+0000 7ff4a9d9b700 1 --2- 172.21.15.106:0/1458256903 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff4ac1033e0 0x7ff4ac198850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:24:27.499 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-04T09:24:27.500+0000 7ff4aad9d700 1 -- 172.21.15.106:0/1458256903 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff4ac100400 msgr2=0x7ff4ac198310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:24:27.499 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-04T09:24:27.500+0000 7ff4aad9d700 1 --2- 172.21.15.106:0/1458256903 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff4ac100400 0x7ff4ac198310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:24:27.499 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-04T09:24:27.500+0000 7ff4aad9d700 1 -- 172.21.15.106:0/1458256903 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff4ac1033e0 msgr2=0x7ff4ac198850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:24:27.499 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-04T09:24:27.500+0000 7ff4aad9d700 1 --2- 172.21.15.106:0/1458256903 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff4ac1033e0 0x7ff4ac198850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:24:27.499 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-04T09:24:27.500+0000 7ff4aad9d700 1 -- 172.21.15.106:0/1458256903 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4ac19ff70 con 0x7ff4ac103d80 2024-03-04T09:24:27.499 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-04T09:24:27.500+0000 7ff4a9d9b700 1 --2- 172.21.15.106:0/1458256903 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff4ac1033e0 0x7ff4ac198850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T09:24:27.499 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-04T09:24:27.500+0000 7ff4aa59c700 1 --2- 172.21.15.106:0/1458256903 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff4ac100400 0x7ff4ac198310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-04T09:24:27.499 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-04T09:24:27.500+0000 7ff4aad9d700 1 --2- 172.21.15.106:0/1458256903 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff4ac103d80 0x7ff4ac19f8d0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7ff4a400ada0 tx=0x7ff4a400c330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:24:27.500 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-04T09:24:27.500+0000 7ff49b7fe700 1 -- 172.21.15.106:0/1458256903 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4a4017020 con 0x7ff4ac103d80 2024-03-04T09:24:27.500 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-04T09:24:27.500+0000 7ff49b7fe700 1 -- 172.21.15.106:0/1458256903 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff4a400a0d0 con 0x7ff4ac103d80 2024-03-04T09:24:27.500 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-04T09:24:27.500+0000 7ff4b08e0700 1 -- 172.21.15.106:0/1458256903 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4ac1a0260 con 0x7ff4ac103d80 2024-03-04T09:24:27.500 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-04T09:24:27.500+0000 7ff49b7fe700 1 -- 172.21.15.106:0/1458256903 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4a4005480 con 0x7ff4ac103d80 2024-03-04T09:24:27.500 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-04T09:24:27.500+0000 7ff4b08e0700 1 -- 172.21.15.106:0/1458256903 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4ac1a0750 con 0x7ff4ac103d80 2024-03-04T09:24:27.500 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-04T09:24:27.502+0000 7ff4b08e0700 1 -- 172.21.15.106:0/1458256903 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff4ac068d80 con 0x7ff4ac103d80 2024-03-04T09:24:27.500 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-04T09:24:27.502+0000 7ff49b7fe700 1 -- 172.21.15.106:0/1458256903 <== mon.2 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 16) v1 ==== 136756+0+0 (secure 0 0 0) 0x7ff4a4005620 con 0x7ff4ac103d80 2024-03-04T09:24:27.500 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-04T09:24:27.503+0000 7ff49b7fe700 1 --2- 172.21.15.106:0/1458256903 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7ff4940a1e10 0x7ff4940a42d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:24:27.500 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-04T09:24:27.503+0000 7ff49b7fe700 1 -- 172.21.15.106:0/1458256903 <== mon.2 v2:172.21.15.106:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 3698+0+0 (secure 0 0 0) 0x7ff4a40cc330 con 0x7ff4ac103d80 2024-03-04T09:24:27.501 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-04T09:24:27.503+0000 7ff4aa59c700 1 --2- 172.21.15.106:0/1458256903 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7ff4940a1e10 0x7ff4940a42d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:24:27.501 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-04T09:24:27.504+0000 7ff4aa59c700 1 --2- 172.21.15.106:0/1458256903 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7ff4940a1e10 0x7ff4940a42d0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7ff4a000b5c0 tx=0x7ff4a0003960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:24:27.503 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-04T09:24:27.507+0000 7ff49b7fe700 1 -- 172.21.15.106:0/1458256903 <== mon.2 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7ff4a4021020 con 0x7ff4ac103d80 2024-03-04T09:24:27.705 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-04T09:24:27.708+0000 7ff4b08e0700 1 -- 172.21.15.106:0/1458256903 --> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi106:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7ff4ac062500 con 0x7ff4940a1e10 2024-03-04T09:24:28.085 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:24:27 smithi052 ceph-mon[32473]: osdmap e27: 4 total, 4 up, 4 in 2024-03-04T09:24:28.085 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:24:27 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-04T09:24:28.085 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:24:27 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-04T09:24:28.085 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:24:27 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T09:24:28.086 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:24:27 smithi191 ceph-mon[36816]: osdmap e27: 4 total, 4 up, 4 in 2024-03-04T09:24:28.086 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:24:27 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-04T09:24:28.086 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:24:27 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-04T09:24:28.086 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:24:27 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T09:24:28.131 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:24:27 smithi106 ceph-mon[36484]: osdmap e27: 4 total, 4 up, 4 in 2024-03-04T09:24:28.131 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:24:27 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-04T09:24:28.131 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:24:27 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-04T09:24:28.131 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:24:27 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T09:24:29.085 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:24:28 smithi052 ceph-mon[32473]: pgmap v101: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 508 MiB used, 357 GiB / 358 GiB avail; 2/6 objects degraded (33.333%) 2024-03-04T09:24:29.085 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:24:28 smithi052 ceph-mon[32473]: from='client.24203 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi106:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T09:24:29.085 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:24:28 smithi052 ceph-mon[32473]: Health check failed: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded (PG_DEGRADED) 2024-03-04T09:24:29.085 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:24:28 smithi052 ceph-mon[32473]: osdmap e28: 4 total, 4 up, 4 in 2024-03-04T09:24:29.086 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:24:28 smithi191 ceph-mon[36816]: pgmap v101: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 508 MiB used, 357 GiB / 358 GiB avail; 2/6 objects degraded (33.333%) 2024-03-04T09:24:29.086 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:24:28 smithi191 ceph-mon[36816]: from='client.24203 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi106:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T09:24:29.086 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:24:28 smithi191 ceph-mon[36816]: Health check failed: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded (PG_DEGRADED) 2024-03-04T09:24:29.086 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:24:28 smithi191 ceph-mon[36816]: osdmap e28: 4 total, 4 up, 4 in 2024-03-04T09:24:29.131 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:24:28 smithi106 ceph-mon[36484]: pgmap v101: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 508 MiB used, 357 GiB / 358 GiB avail; 2/6 objects degraded (33.333%) 2024-03-04T09:24:29.131 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:24:28 smithi106 ceph-mon[36484]: from='client.24203 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi106:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T09:24:29.131 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:24:28 smithi106 ceph-mon[36484]: Health check failed: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded (PG_DEGRADED) 2024-03-04T09:24:29.131 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:24:28 smithi106 ceph-mon[36484]: osdmap e28: 4 total, 4 up, 4 in 2024-03-04T09:24:30.084 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:24:29 smithi052 ceph-mon[32473]: osdmap e29: 4 total, 4 up, 4 in 2024-03-04T09:24:30.086 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:24:29 smithi191 ceph-mon[36816]: osdmap e29: 4 total, 4 up, 4 in 2024-03-04T09:24:30.131 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:24:29 smithi106 ceph-mon[36484]: osdmap e29: 4 total, 4 up, 4 in 2024-03-04T09:24:31.085 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:24:30 smithi052 ceph-mon[32473]: pgmap v104: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 508 MiB used, 357 GiB / 358 GiB avail; 2/6 objects degraded (33.333%) 2024-03-04T09:24:31.085 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:24:30 smithi052 ceph-mon[32473]: from='client.? 172.21.15.106:0/2844010395' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c8ef48c7-7c78-44d3-a9e5-99ec750abd3d"}]: dispatch 2024-03-04T09:24:31.085 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:24:30 smithi052 ceph-mon[32473]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c8ef48c7-7c78-44d3-a9e5-99ec750abd3d"}]: dispatch 2024-03-04T09:24:31.085 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:24:30 smithi052 ceph-mon[32473]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c8ef48c7-7c78-44d3-a9e5-99ec750abd3d"}]': finished 2024-03-04T09:24:31.085 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:24:30 smithi052 ceph-mon[32473]: osdmap e30: 5 total, 4 up, 5 in 2024-03-04T09:24:31.085 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:24:30 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-04T09:24:31.086 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:24:30 smithi191 ceph-mon[36816]: pgmap v104: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 508 MiB used, 357 GiB / 358 GiB avail; 2/6 objects degraded (33.333%) 2024-03-04T09:24:31.086 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:24:30 smithi191 ceph-mon[36816]: from='client.? 172.21.15.106:0/2844010395' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c8ef48c7-7c78-44d3-a9e5-99ec750abd3d"}]: dispatch 2024-03-04T09:24:31.086 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:24:30 smithi191 ceph-mon[36816]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c8ef48c7-7c78-44d3-a9e5-99ec750abd3d"}]: dispatch 2024-03-04T09:24:31.086 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:24:30 smithi191 ceph-mon[36816]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c8ef48c7-7c78-44d3-a9e5-99ec750abd3d"}]': finished 2024-03-04T09:24:31.086 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:24:30 smithi191 ceph-mon[36816]: osdmap e30: 5 total, 4 up, 5 in 2024-03-04T09:24:31.086 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:24:30 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-04T09:24:31.131 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:24:30 smithi106 ceph-mon[36484]: pgmap v104: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 508 MiB used, 357 GiB / 358 GiB avail; 2/6 objects degraded (33.333%) 2024-03-04T09:24:31.131 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:24:30 smithi106 ceph-mon[36484]: from='client.? 172.21.15.106:0/2844010395' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c8ef48c7-7c78-44d3-a9e5-99ec750abd3d"}]: dispatch 2024-03-04T09:24:31.131 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:24:30 smithi106 ceph-mon[36484]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c8ef48c7-7c78-44d3-a9e5-99ec750abd3d"}]: dispatch 2024-03-04T09:24:31.131 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:24:30 smithi106 ceph-mon[36484]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c8ef48c7-7c78-44d3-a9e5-99ec750abd3d"}]': finished 2024-03-04T09:24:31.131 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:24:30 smithi106 ceph-mon[36484]: osdmap e30: 5 total, 4 up, 5 in 2024-03-04T09:24:31.131 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:24:30 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-04T09:24:32.085 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:24:31 smithi052 ceph-mon[32473]: from='client.? 172.21.15.106:0/3316911637' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-04T09:24:32.086 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:24:31 smithi191 ceph-mon[36816]: from='client.? 172.21.15.106:0/3316911637' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-04T09:24:32.131 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:24:31 smithi106 ceph-mon[36484]: from='client.? 172.21.15.106:0/3316911637' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-04T09:24:33.085 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:24:32 smithi052 ceph-mon[32473]: pgmap v106: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 117 MiB used, 357 GiB / 358 GiB avail; 2/6 objects degraded (33.333%) 2024-03-04T09:24:33.086 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:24:32 smithi191 ceph-mon[36816]: pgmap v106: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 117 MiB used, 357 GiB / 358 GiB avail; 2/6 objects degraded (33.333%) 2024-03-04T09:24:33.131 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:24:32 smithi106 ceph-mon[36484]: pgmap v106: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 117 MiB used, 357 GiB / 358 GiB avail; 2/6 objects degraded (33.333%) 2024-03-04T09:24:34.086 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:24:33 smithi191 ceph-mon[36816]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded) 2024-03-04T09:24:34.086 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:24:33 smithi191 ceph-mon[36816]: Cluster is now healthy 2024-03-04T09:24:34.131 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:24:33 smithi106 ceph-mon[36484]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded) 2024-03-04T09:24:34.131 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:24:33 smithi106 ceph-mon[36484]: Cluster is now healthy 2024-03-04T09:24:34.335 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:24:33 smithi052 ceph-mon[32473]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded) 2024-03-04T09:24:34.335 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:24:33 smithi052 ceph-mon[32473]: Cluster is now healthy 2024-03-04T09:24:35.131 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:24:34 smithi106 ceph-mon[36484]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-03-04T09:24:35.334 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:24:34 smithi052 ceph-mon[32473]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-03-04T09:24:35.336 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:24:34 smithi191 ceph-mon[36816]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-03-04T09:24:36.881 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:24:36 smithi106 ceph-mon[36484]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 75 KiB/s, 0 objects/s recovering 2024-03-04T09:24:37.137 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:24:36 smithi052 ceph-mon[32473]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 75 KiB/s, 0 objects/s recovering 2024-03-04T09:24:37.336 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:24:36 smithi191 ceph-mon[36816]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 75 KiB/s, 0 objects/s recovering 2024-03-04T09:24:38.131 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:24:37 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-03-04T09:24:38.131 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:24:37 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T09:24:38.334 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:24:37 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-03-04T09:24:38.335 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:24:37 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T09:24:38.336 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:24:37 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-03-04T09:24:38.336 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:24:37 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T09:24:39.131 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:24:38 smithi106 ceph-mon[36484]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 67 KiB/s, 0 objects/s recovering 2024-03-04T09:24:39.131 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:24:38 smithi106 ceph-mon[36484]: Deploying daemon osd.4 on smithi106 2024-03-04T09:24:39.335 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:24:38 smithi052 ceph-mon[32473]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 67 KiB/s, 0 objects/s recovering 2024-03-04T09:24:39.335 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:24:38 smithi052 ceph-mon[32473]: Deploying daemon osd.4 on smithi106 2024-03-04T09:24:39.336 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:24:38 smithi191 ceph-mon[36816]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 67 KiB/s, 0 objects/s recovering 2024-03-04T09:24:39.336 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:24:38 smithi191 ceph-mon[36816]: Deploying daemon osd.4 on smithi106 2024-03-04T09:24:41.123 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:24:40 smithi106 ceph-mon[36484]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-03-04T09:24:41.334 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:24:40 smithi052 ceph-mon[32473]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-03-04T09:24:41.336 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:24:40 smithi191 ceph-mon[36816]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-03-04T09:24:42.881 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:24:42 smithi106 ceph-mon[36484]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 53 KiB/s, 0 objects/s recovering 2024-03-04T09:24:43.334 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:24:42 smithi052 ceph-mon[32473]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 53 KiB/s, 0 objects/s recovering 2024-03-04T09:24:43.336 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:24:42 smithi191 ceph-mon[36816]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 53 KiB/s, 0 objects/s recovering 2024-03-04T09:24:44.585 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:24:44 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:24:44.585 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:24:44 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:24:44.585 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:24:44 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T09:24:44.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:24:44 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:24:44.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:24:44 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:24:44.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:24:44 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T09:24:44.631 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:24:44 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:24:44.631 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:24:44 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:24:44.631 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:24:44 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T09:24:45.500 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:24:45 smithi106 ceph-mon[36484]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-03-04T09:24:45.584 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:24:45 smithi052 ceph-mon[32473]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-03-04T09:24:45.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:24:45 smithi191 ceph-mon[36816]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-03-04T09:24:47.122 INFO:teuthology.orchestra.run.smithi106.stdout:Created osd(s) 4 on host 'smithi106' 2024-03-04T09:24:47.122 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-04T09:24:47.124+0000 7ff49b7fe700 1 -- 172.21.15.106:0/1458256903 <== mgr.14152 v2:172.21.15.52:6800/3036033923 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7ff4ac062500 con 0x7ff4940a1e10 2024-03-04T09:24:47.124 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-04T09:24:47.126+0000 7ff4b08e0700 1 -- 172.21.15.106:0/1458256903 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7ff4940a1e10 msgr2=0x7ff4940a42d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:24:47.124 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-04T09:24:47.126+0000 7ff4b08e0700 1 --2- 172.21.15.106:0/1458256903 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7ff4940a1e10 0x7ff4940a42d0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7ff4a000b5c0 tx=0x7ff4a0003960 comp rx=0 tx=0).stop 2024-03-04T09:24:47.124 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-04T09:24:47.126+0000 7ff4b08e0700 1 -- 172.21.15.106:0/1458256903 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff4ac103d80 msgr2=0x7ff4ac19f8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:24:47.124 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-04T09:24:47.126+0000 7ff4b08e0700 1 --2- 172.21.15.106:0/1458256903 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff4ac103d80 0x7ff4ac19f8d0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7ff4a400ada0 tx=0x7ff4a400c330 comp rx=0 tx=0).stop 2024-03-04T09:24:47.124 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-04T09:24:47.126+0000 7ff4b08e0700 1 -- 172.21.15.106:0/1458256903 shutdown_connections 2024-03-04T09:24:47.124 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-04T09:24:47.126+0000 7ff4b08e0700 1 --2- 172.21.15.106:0/1458256903 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7ff4940a1e10 0x7ff4940a42d0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:24:47.125 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-04T09:24:47.126+0000 7ff4b08e0700 1 --2- 172.21.15.106:0/1458256903 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff4ac100400 0x7ff4ac198310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:24:47.125 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-04T09:24:47.126+0000 7ff4b08e0700 1 --2- 172.21.15.106:0/1458256903 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff4ac103d80 0x7ff4ac19f8d0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:24:47.125 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-04T09:24:47.126+0000 7ff4b08e0700 1 --2- 172.21.15.106:0/1458256903 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff4ac1033e0 0x7ff4ac198850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:24:47.125 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-04T09:24:47.126+0000 7ff4b08e0700 1 -- 172.21.15.106:0/1458256903 >> 172.21.15.106:0/1458256903 conn(0x7ff4ac0fbd60 msgr2=0x7ff4ac106680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:24:47.125 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-04T09:24:47.127+0000 7ff4b08e0700 1 -- 172.21.15.106:0/1458256903 shutdown_connections 2024-03-04T09:24:47.125 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-04T09:24:47.127+0000 7ff4b08e0700 1 -- 172.21.15.106:0/1458256903 wait complete. 2024-03-04T09:24:47.381 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:24:47 smithi106 ceph-mon[36484]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-03-04T09:24:47.381 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:24:47 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:24:47.381 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:24:47 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:24:47.381 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:24:47 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T09:24:47.382 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:24:47 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T09:24:47.382 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:24:47 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:24:47.382 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:24:47 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:24:47.382 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:24:47 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:24:47.382 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:24:47 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T09:24:47.382 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:24:47 smithi106 ceph-mon[36484]: from='osd.4 [v2:172.21.15.106:6808/3465868015,v1:172.21.15.106:6809/3465868015]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-04T09:24:47.382 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:24:47 smithi106 ceph-mon[36484]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-04T09:24:47.585 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:24:47 smithi052 ceph-mon[32473]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-03-04T09:24:47.585 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:24:47 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:24:47.585 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:24:47 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:24:47.585 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:24:47 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T09:24:47.585 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:24:47 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T09:24:47.585 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:24:47 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:24:47.585 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:24:47 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:24:47.585 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:24:47 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:24:47.585 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:24:47 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T09:24:47.585 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:24:47 smithi052 ceph-mon[32473]: from='osd.4 [v2:172.21.15.106:6808/3465868015,v1:172.21.15.106:6809/3465868015]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-04T09:24:47.585 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:24:47 smithi052 ceph-mon[32473]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-04T09:24:47.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:24:47 smithi191 ceph-mon[36816]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-03-04T09:24:47.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:24:47 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:24:47.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:24:47 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:24:47.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:24:47 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T09:24:47.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:24:47 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T09:24:47.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:24:47 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:24:47.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:24:47 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:24:47.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:24:47 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:24:47.587 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:24:47 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T09:24:47.587 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:24:47 smithi191 ceph-mon[36816]: from='osd.4 [v2:172.21.15.106:6808/3465868015,v1:172.21.15.106:6809/3465868015]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-04T09:24:47.587 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:24:47 smithi191 ceph-mon[36816]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-04T09:24:47.843 DEBUG:teuthology.orchestra.run.smithi106:osd.4> sudo journalctl -f -n 0 -u ceph-4e059f08-da08-11ee-95c6-87774f69a715@osd.4.service 2024-03-04T09:24:47.849 INFO:tasks.cephadm:Deploying osd.5 on smithi191 with /dev/vg_nvme/lv_3... 2024-03-04T09:24:47.849 DEBUG:teuthology.orchestra.run.smithi191:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4e059f08-da08-11ee-95c6-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-03-04T09:24:48.095 INFO:teuthology.orchestra.run.smithi191.stderr:Inferring config /var/lib/ceph/4e059f08-da08-11ee-95c6-87774f69a715/mon.c/config 2024-03-04T09:24:49.335 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:24:49 smithi052 ceph-mon[32473]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-03-04T09:24:49.335 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:24:49 smithi052 ceph-mon[32473]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-03-04T09:24:49.335 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:24:49 smithi052 ceph-mon[32473]: osdmap e31: 5 total, 4 up, 5 in 2024-03-04T09:24:49.335 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:24:49 smithi052 ceph-mon[32473]: from='osd.4 [v2:172.21.15.106:6808/3465868015,v1:172.21.15.106:6809/3465868015]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi106", "root=default"]}]: dispatch 2024-03-04T09:24:49.335 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:24:49 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-04T09:24:49.335 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:24:49 smithi052 ceph-mon[32473]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi106", "root=default"]}]: dispatch 2024-03-04T09:24:49.336 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:24:49 smithi191 ceph-mon[36816]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-03-04T09:24:49.337 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:24:49 smithi191 ceph-mon[36816]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-03-04T09:24:49.337 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:24:49 smithi191 ceph-mon[36816]: osdmap e31: 5 total, 4 up, 5 in 2024-03-04T09:24:49.337 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:24:49 smithi191 ceph-mon[36816]: from='osd.4 [v2:172.21.15.106:6808/3465868015,v1:172.21.15.106:6809/3465868015]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi106", "root=default"]}]: dispatch 2024-03-04T09:24:49.337 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:24:49 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-04T09:24:49.337 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:24:49 smithi191 ceph-mon[36816]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi106", "root=default"]}]: dispatch 2024-03-04T09:24:49.381 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:24:49 smithi106 ceph-mon[36484]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-03-04T09:24:49.381 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:24:49 smithi106 ceph-mon[36484]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-03-04T09:24:49.381 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:24:49 smithi106 ceph-mon[36484]: osdmap e31: 5 total, 4 up, 5 in 2024-03-04T09:24:49.381 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:24:49 smithi106 ceph-mon[36484]: from='osd.4 [v2:172.21.15.106:6808/3465868015,v1:172.21.15.106:6809/3465868015]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi106", "root=default"]}]: dispatch 2024-03-04T09:24:49.382 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:24:49 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-04T09:24:49.382 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:24:49 smithi106 ceph-mon[36484]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi106", "root=default"]}]: dispatch 2024-03-04T09:24:49.382 INFO:journalctl@ceph.osd.4.smithi106.stdout:Mar 04 09:24:49 smithi106 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-4[47787]: 2024-03-04T09:24:49.089+0000 7f6e5450a700 -1 osd.4 0 waiting for initial osdmap 2024-03-04T09:24:49.382 INFO:journalctl@ceph.osd.4.smithi106.stdout:Mar 04 09:24:49 smithi106 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-4[47787]: 2024-03-04T09:24:49.096+0000 7f6e502fd700 -1 osd.4 32 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-04T09:24:50.109 INFO:teuthology.orchestra.run.smithi191.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-03-04T09:24:50.109 INFO:teuthology.orchestra.run.smithi191.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-03-04T09:24:50.109 INFO:teuthology.orchestra.run.smithi191.stderr: stderr: 10+0 records in 2024-03-04T09:24:50.109 INFO:teuthology.orchestra.run.smithi191.stderr:10+0 records out 2024-03-04T09:24:50.110 INFO:teuthology.orchestra.run.smithi191.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0139759 s, 750 MB/s 2024-03-04T09:24:50.110 INFO:teuthology.orchestra.run.smithi191.stderr: stderr: 2024-03-04T09:24:50.110 INFO:teuthology.orchestra.run.smithi191.stderr:--> Zapping successful for: 2024-03-04T09:24:50.381 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:24:50 smithi106 ceph-mon[36484]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi106", "root=default"]}]': finished 2024-03-04T09:24:50.381 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:24:50 smithi106 ceph-mon[36484]: osdmap e32: 5 total, 4 up, 5 in 2024-03-04T09:24:50.381 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:24:50 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-04T09:24:50.381 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:24:50 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-04T09:24:50.405 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:24:50 smithi191 ceph-mon[36816]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi106", "root=default"]}]': finished 2024-03-04T09:24:50.406 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:24:50 smithi191 ceph-mon[36816]: osdmap e32: 5 total, 4 up, 5 in 2024-03-04T09:24:50.406 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:24:50 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-04T09:24:50.406 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:24:50 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-04T09:24:50.584 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:24:50 smithi052 ceph-mon[32473]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi106", "root=default"]}]': finished 2024-03-04T09:24:50.585 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:24:50 smithi052 ceph-mon[32473]: osdmap e32: 5 total, 4 up, 5 in 2024-03-04T09:24:50.585 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:24:50 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-04T09:24:50.585 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:24:50 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-04T09:24:50.777 DEBUG:teuthology.orchestra.run.smithi191:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4e059f08-da08-11ee-95c6-87774f69a715 -- ceph orch daemon add osd smithi191:vg_nvme/lv_3 2024-03-04T09:24:51.016 INFO:teuthology.orchestra.run.smithi191.stderr:Inferring config /var/lib/ceph/4e059f08-da08-11ee-95c6-87774f69a715/mon.c/config 2024-03-04T09:24:51.131 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:24:51 smithi106 ceph-mon[36484]: purged_snaps scrub starts 2024-03-04T09:24:51.131 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:24:51 smithi106 ceph-mon[36484]: purged_snaps scrub ok 2024-03-04T09:24:51.131 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:24:51 smithi106 ceph-mon[36484]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-03-04T09:24:51.131 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:24:51 smithi106 ceph-mon[36484]: osd.4 [v2:172.21.15.106:6808/3465868015,v1:172.21.15.106:6809/3465868015] boot 2024-03-04T09:24:51.131 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:24:51 smithi106 ceph-mon[36484]: osdmap e33: 5 total, 5 up, 5 in 2024-03-04T09:24:51.131 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:24:51 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-04T09:24:51.131 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:24:51 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:24:51.131 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:24:51 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:24:51.132 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:24:51 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-04T09:24:51.132 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:24:51 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-04T09:24:51.132 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:24:51 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:24:51.132 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:24:51 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T09:24:51.132 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:24:51 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T09:24:51.132 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:24:51 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:24:51.368 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:24:51 smithi191 ceph-mon[36816]: purged_snaps scrub starts 2024-03-04T09:24:51.368 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:24:51 smithi191 ceph-mon[36816]: purged_snaps scrub ok 2024-03-04T09:24:51.369 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:24:51 smithi191 ceph-mon[36816]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-03-04T09:24:51.369 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:24:51 smithi191 ceph-mon[36816]: osd.4 [v2:172.21.15.106:6808/3465868015,v1:172.21.15.106:6809/3465868015] boot 2024-03-04T09:24:51.369 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:24:51 smithi191 ceph-mon[36816]: osdmap e33: 5 total, 5 up, 5 in 2024-03-04T09:24:51.369 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:24:51 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-04T09:24:51.369 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:24:51 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:24:51.369 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:24:51 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:24:51.369 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:24:51 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-04T09:24:51.369 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:24:51 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-04T09:24:51.369 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:24:51 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:24:51.369 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:24:51 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T09:24:51.369 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:24:51 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T09:24:51.369 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:24:51 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:24:51.585 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:24:51 smithi052 ceph-mon[32473]: purged_snaps scrub starts 2024-03-04T09:24:51.585 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:24:51 smithi052 ceph-mon[32473]: purged_snaps scrub ok 2024-03-04T09:24:51.585 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:24:51 smithi052 ceph-mon[32473]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-03-04T09:24:51.585 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:24:51 smithi052 ceph-mon[32473]: osd.4 [v2:172.21.15.106:6808/3465868015,v1:172.21.15.106:6809/3465868015] boot 2024-03-04T09:24:51.585 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:24:51 smithi052 ceph-mon[32473]: osdmap e33: 5 total, 5 up, 5 in 2024-03-04T09:24:51.585 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:24:51 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-04T09:24:51.585 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:24:51 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:24:51.585 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:24:51 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:24:51.585 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:24:51 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-04T09:24:51.586 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:24:51 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-04T09:24:51.586 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:24:51 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:24:51.586 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:24:51 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T09:24:51.586 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:24:51 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T09:24:51.586 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:24:51 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:24:52.357 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:24:52 smithi191 ceph-mon[36816]: Detected new or changed devices on smithi106 2024-03-04T09:24:52.357 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:24:52 smithi191 ceph-mon[36816]: Adjusting osd_memory_target on smithi106 to 8540M 2024-03-04T09:24:52.357 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:24:52 smithi191 ceph-mon[36816]: osdmap e34: 5 total, 5 up, 5 in 2024-03-04T09:24:52.381 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:24:52 smithi106 ceph-mon[36484]: Detected new or changed devices on smithi106 2024-03-04T09:24:52.381 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:24:52 smithi106 ceph-mon[36484]: Adjusting osd_memory_target on smithi106 to 8540M 2024-03-04T09:24:52.381 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:24:52 smithi106 ceph-mon[36484]: osdmap e34: 5 total, 5 up, 5 in 2024-03-04T09:24:52.564 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:24:52.567+0000 7fbda9721700 1 -- 172.21.15.191:0/889544154 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbda4102260 msgr2=0x7fbda41026c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:24:52.564 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:24:52.567+0000 7fbda9721700 1 --2- 172.21.15.191:0/889544154 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbda4102260 0x7fbda41026c0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fbd98009990 tx=0x7fbd98009ca0 comp rx=0 tx=0).stop 2024-03-04T09:24:52.564 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:24:52.567+0000 7fbda9721700 1 -- 172.21.15.191:0/889544154 shutdown_connections 2024-03-04T09:24:52.564 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:24:52.567+0000 7fbda9721700 1 --2- 172.21.15.191:0/889544154 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbda4102c00 0x7fbda41094b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:24:52.564 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:24:52.567+0000 7fbda9721700 1 --2- 172.21.15.191:0/889544154 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbda4101060 0x7fbda4101480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:24:52.565 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:24:52.567+0000 7fbda9721700 1 --2- 172.21.15.191:0/889544154 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbda4102260 0x7fbda41026c0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:24:52.565 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:24:52.567+0000 7fbda9721700 1 -- 172.21.15.191:0/889544154 >> 172.21.15.191:0/889544154 conn(0x7fbda40fc620 msgr2=0x7fbda40fea40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:24:52.565 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:24:52.567+0000 7fbda9721700 1 -- 172.21.15.191:0/889544154 shutdown_connections 2024-03-04T09:24:52.565 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:24:52.567+0000 7fbda9721700 1 -- 172.21.15.191:0/889544154 wait complete. 2024-03-04T09:24:52.567 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:24:52.567+0000 7fbda9721700 1 Processor -- start 2024-03-04T09:24:52.567 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:24:52.568+0000 7fbda9721700 1 -- start start 2024-03-04T09:24:52.567 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:24:52.568+0000 7fbda9721700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbda4101060 0x7fbda406b380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:24:52.567 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:24:52.568+0000 7fbda9721700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbda4102260 0x7fbda419cf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:24:52.567 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:24:52.568+0000 7fbda9721700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbda4102c00 0x7fbda419f620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:24:52.567 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:24:52.568+0000 7fbda9721700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbda410cbb0 con 0x7fbda4102260 2024-03-04T09:24:52.567 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:24:52.568+0000 7fbda9721700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbda410ca30 con 0x7fbda4102c00 2024-03-04T09:24:52.567 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:24:52.568+0000 7fbda9721700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbda410cd30 con 0x7fbda4101060 2024-03-04T09:24:52.567 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:24:52.568+0000 7fbda2ffd700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbda4101060 0x7fbda406b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:24:52.568 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:24:52.568+0000 7fbda2ffd700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbda4101060 0x7fbda406b380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.191:49452/0 (socket says 172.21.15.191:49452) 2024-03-04T09:24:52.568 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:24:52.568+0000 7fbda2ffd700 1 -- 172.21.15.191:0/2865329314 learned_addr learned my addr 172.21.15.191:0/2865329314 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-03-04T09:24:52.568 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:24:52.568+0000 7fbda37fe700 1 --2- 172.21.15.191:0/2865329314 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbda4102c00 0x7fbda419f620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:24:52.568 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:24:52.568+0000 7fbda27fc700 1 --2- 172.21.15.191:0/2865329314 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbda4102260 0x7fbda419cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:24:52.568 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:24:52.568+0000 7fbda2ffd700 1 -- 172.21.15.191:0/2865329314 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbda4102c00 msgr2=0x7fbda419f620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:24:52.568 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:24:52.568+0000 7fbda2ffd700 1 --2- 172.21.15.191:0/2865329314 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbda4102c00 0x7fbda419f620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:24:52.568 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:24:52.568+0000 7fbda2ffd700 1 -- 172.21.15.191:0/2865329314 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbda4102260 msgr2=0x7fbda419cf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:24:52.568 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:24:52.568+0000 7fbda2ffd700 1 --2- 172.21.15.191:0/2865329314 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbda4102260 0x7fbda419cf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:24:52.568 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:24:52.568+0000 7fbda2ffd700 1 -- 172.21.15.191:0/2865329314 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbda419fef0 con 0x7fbda4101060 2024-03-04T09:24:52.568 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:24:52.568+0000 7fbda37fe700 1 --2- 172.21.15.191:0/2865329314 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbda4102c00 0x7fbda419f620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T09:24:52.568 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:24:52.568+0000 7fbda27fc700 1 --2- 172.21.15.191:0/2865329314 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbda4102260 0x7fbda419cf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-04T09:24:52.568 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:24:52.568+0000 7fbda2ffd700 1 --2- 172.21.15.191:0/2865329314 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbda4101060 0x7fbda406b380 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fbd9c00ada0 tx=0x7fbd9c00c330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:24:52.569 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:24:52.569+0000 7fbd8bfff700 1 -- 172.21.15.191:0/2865329314 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd9c017020 con 0x7fbda4101060 2024-03-04T09:24:52.569 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:24:52.569+0000 7fbda9721700 1 -- 172.21.15.191:0/2865329314 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbda41a01e0 con 0x7fbda4101060 2024-03-04T09:24:52.569 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:24:52.569+0000 7fbd8bfff700 1 -- 172.21.15.191:0/2865329314 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fbd9c00c980 con 0x7fbda4101060 2024-03-04T09:24:52.569 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:24:52.569+0000 7fbd8bfff700 1 -- 172.21.15.191:0/2865329314 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd9c0044c0 con 0x7fbda4101060 2024-03-04T09:24:52.569 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:24:52.569+0000 7fbda9721700 1 -- 172.21.15.191:0/2865329314 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbda41a0720 con 0x7fbda4101060 2024-03-04T09:24:52.569 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:24:52.570+0000 7fbda9721700 1 -- 172.21.15.191:0/2865329314 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbda4068d80 con 0x7fbda4101060 2024-03-04T09:24:52.569 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:24:52.571+0000 7fbd8bfff700 1 -- 172.21.15.191:0/2865329314 <== mon.1 v2:172.21.15.191:3300/0 4 ==== mgrmap(e 16) v1 ==== 136756+0+0 (secure 0 0 0) 0x7fbd9c00cb30 con 0x7fbda4101060 2024-03-04T09:24:52.570 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:24:52.571+0000 7fbd8bfff700 1 --2- 172.21.15.191:0/2865329314 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7fbd8c0a1e10 0x7fbd8c0a42d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:24:52.570 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:24:52.572+0000 7fbd8bfff700 1 -- 172.21.15.191:0/2865329314 <== mon.1 v2:172.21.15.191:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 4101+0+0 (secure 0 0 0) 0x7fbd9c0cc4d0 con 0x7fbda4101060 2024-03-04T09:24:52.570 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:24:52.572+0000 7fbda27fc700 1 --2- 172.21.15.191:0/2865329314 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7fbd8c0a1e10 0x7fbd8c0a42d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:24:52.570 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:24:52.572+0000 7fbda27fc700 1 --2- 172.21.15.191:0/2865329314 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7fbd8c0a1e10 0x7fbd8c0a42d0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fbd980039c0 tx=0x7fbd98005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:24:52.573 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:24:52.576+0000 7fbd8bfff700 1 -- 172.21.15.191:0/2865329314 <== mon.1 v2:172.21.15.191:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fbd9c021020 con 0x7fbda4101060 2024-03-04T09:24:52.585 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:24:52 smithi052 ceph-mon[32473]: Detected new or changed devices on smithi106 2024-03-04T09:24:52.585 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:24:52 smithi052 ceph-mon[32473]: Adjusting osd_memory_target on smithi106 to 8540M 2024-03-04T09:24:52.585 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:24:52 smithi052 ceph-mon[32473]: osdmap e34: 5 total, 5 up, 5 in 2024-03-04T09:24:52.775 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:24:52.778+0000 7fbda9721700 1 -- 172.21.15.191:0/2865329314 --> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi191:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7fbda4062500 con 0x7fbd8c0a1e10 2024-03-04T09:24:53.377 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:24:53 smithi191 ceph-mon[36816]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-03-04T09:24:53.377 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:24:53 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-04T09:24:53.377 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:24:53 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-04T09:24:53.377 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:24:53 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T09:24:53.381 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:24:53 smithi106 ceph-mon[36484]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-03-04T09:24:53.381 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:24:53 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-04T09:24:53.381 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:24:53 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-04T09:24:53.381 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:24:53 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T09:24:53.585 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:24:53 smithi052 ceph-mon[32473]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-03-04T09:24:53.585 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:24:53 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-04T09:24:53.585 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:24:53 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-04T09:24:53.585 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:24:53 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T09:24:54.336 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:24:54 smithi191 ceph-mon[36816]: from='client.24220 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi191:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T09:24:54.381 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:24:54 smithi106 ceph-mon[36484]: from='client.24220 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi191:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T09:24:54.585 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:24:54 smithi052 ceph-mon[32473]: from='client.24220 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi191:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T09:24:55.381 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:24:55 smithi106 ceph-mon[36484]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-03-04T09:24:55.585 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:24:55 smithi052 ceph-mon[32473]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-03-04T09:24:55.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:24:55 smithi191 ceph-mon[36816]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-03-04T09:24:56.336 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:24:56 smithi191 ceph-mon[36816]: from='client.? 172.21.15.191:0/3537569597' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9e41824a-c5a2-4f88-be50-2ba1dceb3220"}]: dispatch 2024-03-04T09:24:56.336 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:24:56 smithi191 ceph-mon[36816]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9e41824a-c5a2-4f88-be50-2ba1dceb3220"}]: dispatch 2024-03-04T09:24:56.336 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:24:56 smithi191 ceph-mon[36816]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9e41824a-c5a2-4f88-be50-2ba1dceb3220"}]': finished 2024-03-04T09:24:56.337 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:24:56 smithi191 ceph-mon[36816]: osdmap e35: 6 total, 5 up, 6 in 2024-03-04T09:24:56.337 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:24:56 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-04T09:24:56.381 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:24:56 smithi106 ceph-mon[36484]: from='client.? 172.21.15.191:0/3537569597' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9e41824a-c5a2-4f88-be50-2ba1dceb3220"}]: dispatch 2024-03-04T09:24:56.381 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:24:56 smithi106 ceph-mon[36484]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9e41824a-c5a2-4f88-be50-2ba1dceb3220"}]: dispatch 2024-03-04T09:24:56.381 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:24:56 smithi106 ceph-mon[36484]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9e41824a-c5a2-4f88-be50-2ba1dceb3220"}]': finished 2024-03-04T09:24:56.381 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:24:56 smithi106 ceph-mon[36484]: osdmap e35: 6 total, 5 up, 6 in 2024-03-04T09:24:56.381 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:24:56 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-04T09:24:56.584 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:24:56 smithi052 ceph-mon[32473]: from='client.? 172.21.15.191:0/3537569597' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9e41824a-c5a2-4f88-be50-2ba1dceb3220"}]: dispatch 2024-03-04T09:24:56.585 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:24:56 smithi052 ceph-mon[32473]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9e41824a-c5a2-4f88-be50-2ba1dceb3220"}]: dispatch 2024-03-04T09:24:56.585 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:24:56 smithi052 ceph-mon[32473]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9e41824a-c5a2-4f88-be50-2ba1dceb3220"}]': finished 2024-03-04T09:24:56.585 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:24:56 smithi052 ceph-mon[32473]: osdmap e35: 6 total, 5 up, 6 in 2024-03-04T09:24:56.585 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:24:56 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-04T09:24:57.381 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:24:57 smithi106 ceph-mon[36484]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-03-04T09:24:57.381 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:24:57 smithi106 ceph-mon[36484]: from='client.? 172.21.15.191:0/3224959437' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-04T09:24:57.584 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:24:57 smithi052 ceph-mon[32473]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-03-04T09:24:57.585 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:24:57 smithi052 ceph-mon[32473]: from='client.? 172.21.15.191:0/3224959437' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-04T09:24:57.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:24:57 smithi191 ceph-mon[36816]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-03-04T09:24:57.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:24:57 smithi191 ceph-mon[36816]: from='client.? 172.21.15.191:0/3224959437' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-04T09:24:59.381 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:24:59 smithi106 ceph-mon[36484]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-03-04T09:24:59.533 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:24:59 smithi191 ceph-mon[36816]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-03-04T09:24:59.584 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:24:59 smithi052 ceph-mon[32473]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-03-04T09:25:01.381 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:25:01 smithi106 ceph-mon[36484]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-03-04T09:25:01.429 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:25:01 smithi191 ceph-mon[36816]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-03-04T09:25:01.584 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:25:01 smithi052 ceph-mon[32473]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-03-04T09:25:03.381 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:25:03 smithi106 ceph-mon[36484]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-03-04T09:25:03.381 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:25:03 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-03-04T09:25:03.381 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:25:03 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T09:25:03.411 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:25:03 smithi191 ceph-mon[36816]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-03-04T09:25:03.412 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:25:03 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-03-04T09:25:03.412 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:25:03 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T09:25:03.585 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:25:03 smithi052 ceph-mon[32473]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-03-04T09:25:03.585 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:25:03 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-03-04T09:25:03.585 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:25:03 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T09:25:04.381 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:25:04 smithi106 ceph-mon[36484]: Deploying daemon osd.5 on smithi191 2024-03-04T09:25:04.421 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:25:04 smithi191 ceph-mon[36816]: Deploying daemon osd.5 on smithi191 2024-03-04T09:25:04.584 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:25:04 smithi052 ceph-mon[32473]: Deploying daemon osd.5 on smithi191 2024-03-04T09:25:05.381 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:25:05 smithi106 ceph-mon[36484]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-03-04T09:25:05.571 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:25:05 smithi191 ceph-mon[36816]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-03-04T09:25:05.584 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:25:05 smithi052 ceph-mon[32473]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-03-04T09:25:07.387 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:25:07 smithi052 ceph-mon[32473]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-03-04T09:25:07.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:25:07 smithi191 ceph-mon[36816]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-03-04T09:25:07.631 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:25:07 smithi106 ceph-mon[36484]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-03-04T09:25:09.170 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:25:09 smithi191 ceph-mon[36816]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-03-04T09:25:09.170 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:25:09 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:25:09.170 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:25:09 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:25:09.170 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:25:09 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T09:25:09.585 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:25:09 smithi052 ceph-mon[32473]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-03-04T09:25:09.585 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:25:09 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:25:09.585 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:25:09 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:25:09.585 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:25:09 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T09:25:09.631 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:25:09 smithi106 ceph-mon[36484]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-03-04T09:25:09.631 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:25:09 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:25:09.631 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:25:09 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:25:09.631 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:25:09 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T09:25:11.337 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:25:11 smithi191 ceph-mon[36816]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-03-04T09:25:11.584 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:25:11 smithi052 ceph-mon[32473]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-03-04T09:25:11.631 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:25:11 smithi106 ceph-mon[36484]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-03-04T09:25:12.346 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:25:12.348+0000 7fbd8bfff700 1 -- 172.21.15.191:0/2865329314 <== mgr.14152 v2:172.21.15.52:6800/3036033923 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fbda4062500 con 0x7fbd8c0a1e10 2024-03-04T09:25:12.347 INFO:teuthology.orchestra.run.smithi191.stdout:Created osd(s) 5 on host 'smithi191' 2024-03-04T09:25:12.348 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:25:12.351+0000 7fbda9721700 1 -- 172.21.15.191:0/2865329314 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7fbd8c0a1e10 msgr2=0x7fbd8c0a42d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:25:12.348 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:25:12.351+0000 7fbda9721700 1 --2- 172.21.15.191:0/2865329314 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7fbd8c0a1e10 0x7fbd8c0a42d0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fbd980039c0 tx=0x7fbd98005fb0 comp rx=0 tx=0).stop 2024-03-04T09:25:12.349 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:25:12.351+0000 7fbda9721700 1 -- 172.21.15.191:0/2865329314 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbda4101060 msgr2=0x7fbda406b380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:25:12.349 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:25:12.351+0000 7fbda9721700 1 --2- 172.21.15.191:0/2865329314 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbda4101060 0x7fbda406b380 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fbd9c00ada0 tx=0x7fbd9c00c330 comp rx=0 tx=0).stop 2024-03-04T09:25:12.349 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:25:12.351+0000 7fbda9721700 1 -- 172.21.15.191:0/2865329314 shutdown_connections 2024-03-04T09:25:12.349 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:25:12.351+0000 7fbda9721700 1 --2- 172.21.15.191:0/2865329314 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7fbd8c0a1e10 0x7fbd8c0a42d0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:25:12.349 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:25:12.351+0000 7fbda9721700 1 --2- 172.21.15.191:0/2865329314 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbda4101060 0x7fbda406b380 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:25:12.349 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:25:12.351+0000 7fbda9721700 1 --2- 172.21.15.191:0/2865329314 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbda4102c00 0x7fbda419f620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:25:12.349 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:25:12.351+0000 7fbda9721700 1 --2- 172.21.15.191:0/2865329314 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbda4102260 0x7fbda419cf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:25:12.349 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:25:12.351+0000 7fbda9721700 1 -- 172.21.15.191:0/2865329314 >> 172.21.15.191:0/2865329314 conn(0x7fbda40fc620 msgr2=0x7fbda4105560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:25:12.349 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:25:12.351+0000 7fbda9721700 1 -- 172.21.15.191:0/2865329314 shutdown_connections 2024-03-04T09:25:12.349 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:25:12.351+0000 7fbda9721700 1 -- 172.21.15.191:0/2865329314 wait complete. 2024-03-04T09:25:12.835 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:25:12 smithi052 ceph-mon[32473]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-03-04T09:25:12.835 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:25:12 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:25:12.835 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:25:12 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:25:12.835 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:25:12 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T09:25:12.835 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:25:12 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T09:25:12.835 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:25:12 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:25:12.835 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:25:12 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:25:12.835 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:25:12 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T09:25:12.835 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:25:12 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:25:12.835 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:25:12 smithi052 ceph-mon[32473]: from='osd.5 [v2:172.21.15.191:6808/1485092643,v1:172.21.15.191:6809/1485092643]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-04T09:25:12.836 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:25:12 smithi052 ceph-mon[32473]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-04T09:25:12.836 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:25:12 smithi191 ceph-mon[36816]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-03-04T09:25:12.836 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:25:12 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:25:12.836 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:25:12 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:25:12.836 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:25:12 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T09:25:12.836 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:25:12 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T09:25:12.836 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:25:12 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:25:12.836 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:25:12 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:25:12.837 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:25:12 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T09:25:12.837 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:25:12 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:25:12.837 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:25:12 smithi191 ceph-mon[36816]: from='osd.5 [v2:172.21.15.191:6808/1485092643,v1:172.21.15.191:6809/1485092643]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-04T09:25:12.837 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:25:12 smithi191 ceph-mon[36816]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-04T09:25:12.881 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:25:12 smithi106 ceph-mon[36484]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-03-04T09:25:12.881 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:25:12 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:25:12.881 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:25:12 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:25:12.881 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:25:12 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T09:25:12.881 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:25:12 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T09:25:12.882 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:25:12 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:25:12.882 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:25:12 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:25:12.882 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:25:12 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T09:25:12.882 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:25:12 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:25:12.882 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:25:12 smithi106 ceph-mon[36484]: from='osd.5 [v2:172.21.15.191:6808/1485092643,v1:172.21.15.191:6809/1485092643]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-04T09:25:12.882 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:25:12 smithi106 ceph-mon[36484]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-04T09:25:13.014 DEBUG:teuthology.orchestra.run.smithi191:osd.5> sudo journalctl -f -n 0 -u ceph-4e059f08-da08-11ee-95c6-87774f69a715@osd.5.service 2024-03-04T09:25:13.016 INFO:tasks.cephadm:Deploying osd.6 on smithi052 with /dev/vg_nvme/lv_2... 2024-03-04T09:25:13.016 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4e059f08-da08-11ee-95c6-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-03-04T09:25:13.267 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/4e059f08-da08-11ee-95c6-87774f69a715/mon.a/config 2024-03-04T09:25:14.482 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:25:14 smithi191 ceph-mon[36816]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-03-04T09:25:14.482 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:25:14 smithi191 ceph-mon[36816]: osdmap e36: 6 total, 5 up, 6 in 2024-03-04T09:25:14.482 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:25:14 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-04T09:25:14.483 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:25:14 smithi191 ceph-mon[36816]: from='osd.5 [v2:172.21.15.191:6808/1485092643,v1:172.21.15.191:6809/1485092643]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi191", "root=default"]}]: dispatch 2024-03-04T09:25:14.483 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:25:14 smithi191 ceph-mon[36816]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi191", "root=default"]}]: dispatch 2024-03-04T09:25:14.584 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:25:14 smithi052 ceph-mon[32473]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-03-04T09:25:14.585 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:25:14 smithi052 ceph-mon[32473]: osdmap e36: 6 total, 5 up, 6 in 2024-03-04T09:25:14.585 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:25:14 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-04T09:25:14.585 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:25:14 smithi052 ceph-mon[32473]: from='osd.5 [v2:172.21.15.191:6808/1485092643,v1:172.21.15.191:6809/1485092643]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi191", "root=default"]}]: dispatch 2024-03-04T09:25:14.585 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:25:14 smithi052 ceph-mon[32473]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi191", "root=default"]}]: dispatch 2024-03-04T09:25:14.631 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:25:14 smithi106 ceph-mon[36484]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-03-04T09:25:14.631 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:25:14 smithi106 ceph-mon[36484]: osdmap e36: 6 total, 5 up, 6 in 2024-03-04T09:25:14.631 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:25:14 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-04T09:25:14.631 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:25:14 smithi106 ceph-mon[36484]: from='osd.5 [v2:172.21.15.191:6808/1485092643,v1:172.21.15.191:6809/1485092643]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi191", "root=default"]}]: dispatch 2024-03-04T09:25:14.632 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:25:14 smithi106 ceph-mon[36484]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi191", "root=default"]}]: dispatch 2024-03-04T09:25:15.310 INFO:journalctl@ceph.osd.5.smithi191.stdout:Mar 04 09:25:15 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-5[48947]: 2024-03-04T09:25:15.020+0000 7ff1e4ede700 -1 osd.5 0 waiting for initial osdmap 2024-03-04T09:25:15.310 INFO:journalctl@ceph.osd.5.smithi191.stdout:Mar 04 09:25:15 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-5[48947]: 2024-03-04T09:25:15.028+0000 7ff1dccc9700 -1 osd.5 37 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-04T09:25:15.414 INFO:teuthology.orchestra.run.smithi052.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-03-04T09:25:15.414 INFO:teuthology.orchestra.run.smithi052.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-03-04T09:25:15.414 INFO:teuthology.orchestra.run.smithi052.stderr: stderr: 10+0 records in 2024-03-04T09:25:15.414 INFO:teuthology.orchestra.run.smithi052.stderr:10+0 records out 2024-03-04T09:25:15.414 INFO:teuthology.orchestra.run.smithi052.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0087363 s, 1.2 GB/s 2024-03-04T09:25:15.415 INFO:teuthology.orchestra.run.smithi052.stderr:--> Zapping successful for: 2024-03-04T09:25:15.585 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:25:15 smithi052 ceph-mon[32473]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-03-04T09:25:15.585 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:25:15 smithi052 ceph-mon[32473]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi191", "root=default"]}]': finished 2024-03-04T09:25:15.585 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:25:15 smithi052 ceph-mon[32473]: osdmap e37: 6 total, 5 up, 6 in 2024-03-04T09:25:15.585 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:25:15 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-04T09:25:15.585 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:25:15 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-04T09:25:15.585 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:25:15 smithi052 ceph-mon[32473]: from='osd.5 ' entity='osd.5' 2024-03-04T09:25:15.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:25:15 smithi191 ceph-mon[36816]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-03-04T09:25:15.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:25:15 smithi191 ceph-mon[36816]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi191", "root=default"]}]': finished 2024-03-04T09:25:15.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:25:15 smithi191 ceph-mon[36816]: osdmap e37: 6 total, 5 up, 6 in 2024-03-04T09:25:15.587 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:25:15 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-04T09:25:15.587 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:25:15 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-04T09:25:15.587 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:25:15 smithi191 ceph-mon[36816]: from='osd.5 ' entity='osd.5' 2024-03-04T09:25:15.631 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:25:15 smithi106 ceph-mon[36484]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-03-04T09:25:15.631 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:25:15 smithi106 ceph-mon[36484]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi191", "root=default"]}]': finished 2024-03-04T09:25:15.631 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:25:15 smithi106 ceph-mon[36484]: osdmap e37: 6 total, 5 up, 6 in 2024-03-04T09:25:15.631 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:25:15 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-04T09:25:15.631 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:25:15 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-04T09:25:15.631 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:25:15 smithi106 ceph-mon[36484]: from='osd.5 ' entity='osd.5' 2024-03-04T09:25:16.204 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4e059f08-da08-11ee-95c6-87774f69a715 -- ceph orch daemon add osd smithi052:vg_nvme/lv_2 2024-03-04T09:25:16.444 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/4e059f08-da08-11ee-95c6-87774f69a715/mon.a/config 2024-03-04T09:25:16.566 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:25:16 smithi052 ceph-mon[32473]: purged_snaps scrub starts 2024-03-04T09:25:16.566 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:25:16 smithi052 ceph-mon[32473]: purged_snaps scrub ok 2024-03-04T09:25:16.566 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:25:16 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-04T09:25:16.566 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:25:16 smithi052 ceph-mon[32473]: osd.5 [v2:172.21.15.191:6808/1485092643,v1:172.21.15.191:6809/1485092643] boot 2024-03-04T09:25:16.566 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:25:16 smithi052 ceph-mon[32473]: osdmap e38: 6 total, 6 up, 6 in 2024-03-04T09:25:16.566 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:25:16 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-04T09:25:16.566 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:25:16 smithi052 ceph-mon[32473]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-03-04T09:25:16.566 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:25:16 smithi052 ceph-mon[32473]: osdmap e39: 6 total, 6 up, 6 in 2024-03-04T09:25:16.566 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:25:16 smithi052 ceph-mon[32473]: Detected new or changed devices on smithi191 2024-03-04T09:25:16.566 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:25:16 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:25:16.566 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:25:16 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:25:16.566 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:25:16 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-04T09:25:16.567 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:25:16 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-04T09:25:16.567 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:25:16 smithi052 ceph-mon[32473]: Adjusting osd_memory_target on smithi191 to 8540M 2024-03-04T09:25:16.567 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:25:16 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:25:16.567 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:25:16 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T09:25:16.567 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:25:16 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T09:25:16.567 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:25:16 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:25:16.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:25:16 smithi191 ceph-mon[36816]: purged_snaps scrub starts 2024-03-04T09:25:16.587 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:25:16 smithi191 ceph-mon[36816]: purged_snaps scrub ok 2024-03-04T09:25:16.587 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:25:16 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-04T09:25:16.587 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:25:16 smithi191 ceph-mon[36816]: osd.5 [v2:172.21.15.191:6808/1485092643,v1:172.21.15.191:6809/1485092643] boot 2024-03-04T09:25:16.587 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:25:16 smithi191 ceph-mon[36816]: osdmap e38: 6 total, 6 up, 6 in 2024-03-04T09:25:16.587 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:25:16 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-04T09:25:16.587 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:25:16 smithi191 ceph-mon[36816]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-03-04T09:25:16.587 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:25:16 smithi191 ceph-mon[36816]: osdmap e39: 6 total, 6 up, 6 in 2024-03-04T09:25:16.587 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:25:16 smithi191 ceph-mon[36816]: Detected new or changed devices on smithi191 2024-03-04T09:25:16.587 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:25:16 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:25:16.587 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:25:16 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:25:16.587 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:25:16 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-04T09:25:16.587 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:25:16 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-04T09:25:16.588 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:25:16 smithi191 ceph-mon[36816]: Adjusting osd_memory_target on smithi191 to 8540M 2024-03-04T09:25:16.588 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:25:16 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:25:16.588 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:25:16 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T09:25:16.588 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:25:16 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T09:25:16.588 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:25:16 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:25:16.631 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:25:16 smithi106 ceph-mon[36484]: purged_snaps scrub starts 2024-03-04T09:25:16.631 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:25:16 smithi106 ceph-mon[36484]: purged_snaps scrub ok 2024-03-04T09:25:16.631 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:25:16 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-04T09:25:16.632 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:25:16 smithi106 ceph-mon[36484]: osd.5 [v2:172.21.15.191:6808/1485092643,v1:172.21.15.191:6809/1485092643] boot 2024-03-04T09:25:16.632 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:25:16 smithi106 ceph-mon[36484]: osdmap e38: 6 total, 6 up, 6 in 2024-03-04T09:25:16.632 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:25:16 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-04T09:25:16.632 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:25:16 smithi106 ceph-mon[36484]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-03-04T09:25:16.632 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:25:16 smithi106 ceph-mon[36484]: osdmap e39: 6 total, 6 up, 6 in 2024-03-04T09:25:16.632 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:25:16 smithi106 ceph-mon[36484]: Detected new or changed devices on smithi191 2024-03-04T09:25:16.632 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:25:16 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:25:16.632 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:25:16 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:25:16.632 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:25:16 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-04T09:25:16.632 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:25:16 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-04T09:25:16.632 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:25:16 smithi106 ceph-mon[36484]: Adjusting osd_memory_target on smithi191 to 8540M 2024-03-04T09:25:16.633 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:25:16 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:25:16.633 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:25:16 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T09:25:16.633 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:25:16 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T09:25:16.633 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:25:16 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:25:17.946 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:25:17.947+0000 7f89b55c1700 1 -- 172.21.15.52:0/2374722290 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f89b0102bd0 msgr2=0x7f89b0109480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:25:17.946 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:25:17.947+0000 7f89b55c1700 1 --2- 172.21.15.52:0/2374722290 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f89b0102bd0 0x7f89b0109480 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f89a800b0d0 tx=0x7f89a800b3e0 comp rx=0 tx=0).stop 2024-03-04T09:25:17.946 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:25:17.947+0000 7f89b55c1700 1 -- 172.21.15.52:0/2374722290 shutdown_connections 2024-03-04T09:25:17.946 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:25:17.947+0000 7f89b55c1700 1 --2- 172.21.15.52:0/2374722290 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f89b0102bd0 0x7f89b0109480 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:25:17.946 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:25:17.947+0000 7f89b55c1700 1 --2- 172.21.15.52:0/2374722290 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f89b0101030 0x7f89b0101450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:25:17.946 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:25:17.947+0000 7f89b55c1700 1 --2- 172.21.15.52:0/2374722290 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f89b0102230 0x7f89b0102690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:25:17.946 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:25:17.947+0000 7f89b55c1700 1 -- 172.21.15.52:0/2374722290 >> 172.21.15.52:0/2374722290 conn(0x7f89b00fc5b0 msgr2=0x7f89b00fea10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:25:17.947 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:25:17.947+0000 7f89b55c1700 1 -- 172.21.15.52:0/2374722290 shutdown_connections 2024-03-04T09:25:17.947 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:25:17.947+0000 7f89b55c1700 1 -- 172.21.15.52:0/2374722290 wait complete. 2024-03-04T09:25:17.947 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:25:17.948+0000 7f89b55c1700 1 Processor -- start 2024-03-04T09:25:17.947 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:25:17.948+0000 7f89b55c1700 1 -- start start 2024-03-04T09:25:17.947 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:25:17.948+0000 7f89b55c1700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f89b0101030 0x7f89b0198440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:25:17.947 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:25:17.948+0000 7f89b55c1700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f89b0102230 0x7f89b0198980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:25:17.947 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:25:17.948+0000 7f89b55c1700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f89b0102bd0 0x7f89b019fa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:25:17.947 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:25:17.948+0000 7f89b55c1700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89b010cd10 con 0x7f89b0101030 2024-03-04T09:25:17.947 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:25:17.948+0000 7f89b55c1700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89b010cb90 con 0x7f89b0102230 2024-03-04T09:25:17.947 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:25:17.948+0000 7f89b55c1700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89b010ce90 con 0x7f89b0102bd0 2024-03-04T09:25:17.947 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:25:17.948+0000 7f89aeffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f89b0101030 0x7f89b0198440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:25:17.947 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:25:17.948+0000 7f89aeffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f89b0101030 0x7f89b0198440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:38622/0 (socket says 172.21.15.52:38622) 2024-03-04T09:25:17.947 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:25:17.948+0000 7f89aeffd700 1 -- 172.21.15.52:0/1032301031 learned_addr learned my addr 172.21.15.52:0/1032301031 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-04T09:25:17.948 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:25:17.948+0000 7f89ae7fc700 1 --2- 172.21.15.52:0/1032301031 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f89b0102230 0x7f89b0198980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:25:17.948 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:25:17.948+0000 7f89af7fe700 1 --2- 172.21.15.52:0/1032301031 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f89b0102bd0 0x7f89b019fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:25:17.948 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:25:17.948+0000 7f89aeffd700 1 -- 172.21.15.52:0/1032301031 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f89b0102bd0 msgr2=0x7f89b019fa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:25:17.948 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:25:17.948+0000 7f89aeffd700 1 --2- 172.21.15.52:0/1032301031 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f89b0102bd0 0x7f89b019fa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:25:17.948 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:25:17.948+0000 7f89aeffd700 1 -- 172.21.15.52:0/1032301031 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f89b0102230 msgr2=0x7f89b0198980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:25:17.948 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:25:17.948+0000 7f89aeffd700 1 --2- 172.21.15.52:0/1032301031 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f89b0102230 0x7f89b0198980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:25:17.948 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:25:17.948+0000 7f89aeffd700 1 -- 172.21.15.52:0/1032301031 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89b01a00a0 con 0x7f89b0101030 2024-03-04T09:25:17.948 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:25:17.948+0000 7f89af7fe700 1 --2- 172.21.15.52:0/1032301031 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f89b0102bd0 0x7f89b019fa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T09:25:17.948 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:25:17.949+0000 7f89aeffd700 1 --2- 172.21.15.52:0/1032301031 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f89b0101030 0x7f89b0198440 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f89a400b530 tx=0x7f89a400b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:25:17.948 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:25:17.949+0000 7f8997fff700 1 -- 172.21.15.52:0/1032301031 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89a4015020 con 0x7f89b0101030 2024-03-04T09:25:17.948 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:25:17.949+0000 7f89b55c1700 1 -- 172.21.15.52:0/1032301031 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89b01a0390 con 0x7f89b0101030 2024-03-04T09:25:17.948 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:25:17.949+0000 7f8997fff700 1 -- 172.21.15.52:0/1032301031 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f89a4011d10 con 0x7f89b0101030 2024-03-04T09:25:17.949 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:25:17.949+0000 7f8997fff700 1 -- 172.21.15.52:0/1032301031 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89a400f2e0 con 0x7f89b0101030 2024-03-04T09:25:17.949 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:25:17.949+0000 7f89b55c1700 1 -- 172.21.15.52:0/1032301031 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89b01a0950 con 0x7f89b0101030 2024-03-04T09:25:17.950 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:25:17.950+0000 7f89b55c1700 1 -- 172.21.15.52:0/1032301031 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f89b0068d80 con 0x7f89b0101030 2024-03-04T09:25:17.950 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:25:17.952+0000 7f8997fff700 1 -- 172.21.15.52:0/1032301031 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 16) v1 ==== 136756+0+0 (secure 0 0 0) 0x7f89a4018020 con 0x7f89b0101030 2024-03-04T09:25:17.950 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:25:17.952+0000 7f8997fff700 1 --2- 172.21.15.52:0/1032301031 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f89980a1e10 0x7f89980a42d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:25:17.950 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:25:17.952+0000 7f89ae7fc700 1 --2- 172.21.15.52:0/1032301031 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f89980a1e10 0x7f89980a42d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:25:17.950 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:25:17.952+0000 7f8997fff700 1 -- 172.21.15.52:0/1032301031 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4533+0+0 (secure 0 0 0) 0x7f89a40cca70 con 0x7f89b0101030 2024-03-04T09:25:17.950 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:25:17.953+0000 7f89ae7fc700 1 --2- 172.21.15.52:0/1032301031 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f89980a1e10 0x7f89980a42d0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f89a0009a10 tx=0x7f89a0009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:25:17.953 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:25:17.956+0000 7f8997fff700 1 -- 172.21.15.52:0/1032301031 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f89a401f020 con 0x7f89b0101030 2024-03-04T09:25:18.084 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:25:17 smithi052 ceph-mon[32473]: osdmap e40: 6 total, 6 up, 6 in 2024-03-04T09:25:18.086 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:25:17 smithi191 ceph-mon[36816]: osdmap e40: 6 total, 6 up, 6 in 2024-03-04T09:25:18.131 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:25:17 smithi106 ceph-mon[36484]: osdmap e40: 6 total, 6 up, 6 in 2024-03-04T09:25:18.168 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:25:18.171+0000 7f89b55c1700 1 -- 172.21.15.52:0/1032301031 --> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi052:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7f89b00fe740 con 0x7f89980a1e10 2024-03-04T09:25:19.084 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:25:18 smithi052 ceph-mon[32473]: pgmap v139: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-03-04T09:25:19.085 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:25:18 smithi052 ceph-mon[32473]: osdmap e41: 6 total, 6 up, 6 in 2024-03-04T09:25:19.085 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:25:18 smithi052 ceph-mon[32473]: from='client.14352 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi052:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T09:25:19.085 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:25:18 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-04T09:25:19.085 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:25:18 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-04T09:25:19.085 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:25:18 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T09:25:19.086 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:25:18 smithi191 ceph-mon[36816]: pgmap v139: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-03-04T09:25:19.086 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:25:18 smithi191 ceph-mon[36816]: osdmap e41: 6 total, 6 up, 6 in 2024-03-04T09:25:19.086 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:25:18 smithi191 ceph-mon[36816]: from='client.14352 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi052:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T09:25:19.086 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:25:18 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-04T09:25:19.087 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:25:18 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-04T09:25:19.087 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:25:18 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T09:25:19.131 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:25:18 smithi106 ceph-mon[36484]: pgmap v139: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-03-04T09:25:19.131 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:25:18 smithi106 ceph-mon[36484]: osdmap e41: 6 total, 6 up, 6 in 2024-03-04T09:25:19.131 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:25:18 smithi106 ceph-mon[36484]: from='client.14352 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi052:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T09:25:19.131 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:25:18 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-04T09:25:19.131 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:25:18 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-04T09:25:19.131 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:25:18 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T09:25:20.084 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:25:19 smithi052 ceph-mon[32473]: osdmap e42: 6 total, 6 up, 6 in 2024-03-04T09:25:20.086 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:25:19 smithi191 ceph-mon[36816]: osdmap e42: 6 total, 6 up, 6 in 2024-03-04T09:25:20.131 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:25:19 smithi106 ceph-mon[36484]: osdmap e42: 6 total, 6 up, 6 in 2024-03-04T09:25:21.085 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:25:20 smithi052 ceph-mon[32473]: pgmap v142: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-03-04T09:25:21.086 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:25:20 smithi191 ceph-mon[36816]: pgmap v142: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-03-04T09:25:21.131 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:25:20 smithi106 ceph-mon[36484]: pgmap v142: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-03-04T09:25:22.085 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:25:21 smithi052 ceph-mon[32473]: from='client.? 172.21.15.52:0/3249440119' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b35dccee-ec16-452e-ac69-3ac2fb7b27fa"}]: dispatch 2024-03-04T09:25:22.085 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:25:21 smithi052 ceph-mon[32473]: from='client.? 172.21.15.52:0/3249440119' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b35dccee-ec16-452e-ac69-3ac2fb7b27fa"}]': finished 2024-03-04T09:25:22.085 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:25:21 smithi052 ceph-mon[32473]: osdmap e43: 7 total, 6 up, 7 in 2024-03-04T09:25:22.085 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:25:21 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-04T09:25:22.085 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:25:21 smithi052 ceph-mon[32473]: from='client.? 172.21.15.52:0/1456382879' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-04T09:25:22.086 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:25:21 smithi191 ceph-mon[36816]: from='client.? 172.21.15.52:0/3249440119' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b35dccee-ec16-452e-ac69-3ac2fb7b27fa"}]: dispatch 2024-03-04T09:25:22.086 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:25:21 smithi191 ceph-mon[36816]: from='client.? 172.21.15.52:0/3249440119' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b35dccee-ec16-452e-ac69-3ac2fb7b27fa"}]': finished 2024-03-04T09:25:22.086 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:25:21 smithi191 ceph-mon[36816]: osdmap e43: 7 total, 6 up, 7 in 2024-03-04T09:25:22.086 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:25:21 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-04T09:25:22.086 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:25:21 smithi191 ceph-mon[36816]: from='client.? 172.21.15.52:0/1456382879' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-04T09:25:22.131 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:25:21 smithi106 ceph-mon[36484]: from='client.? 172.21.15.52:0/3249440119' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b35dccee-ec16-452e-ac69-3ac2fb7b27fa"}]: dispatch 2024-03-04T09:25:22.131 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:25:21 smithi106 ceph-mon[36484]: from='client.? 172.21.15.52:0/3249440119' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b35dccee-ec16-452e-ac69-3ac2fb7b27fa"}]': finished 2024-03-04T09:25:22.131 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:25:21 smithi106 ceph-mon[36484]: osdmap e43: 7 total, 6 up, 7 in 2024-03-04T09:25:22.131 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:25:21 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-04T09:25:22.131 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:25:21 smithi106 ceph-mon[36484]: from='client.? 172.21.15.52:0/1456382879' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-04T09:25:23.334 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:25:22 smithi052 ceph-mon[32473]: pgmap v144: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 123 KiB/s, 0 objects/s recovering 2024-03-04T09:25:23.336 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:25:22 smithi191 ceph-mon[36816]: pgmap v144: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 123 KiB/s, 0 objects/s recovering 2024-03-04T09:25:23.381 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:25:22 smithi106 ceph-mon[36484]: pgmap v144: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 123 KiB/s, 0 objects/s recovering 2024-03-04T09:25:25.214 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:25:24 smithi052 ceph-mon[32473]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-03-04T09:25:25.336 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:25:24 smithi191 ceph-mon[36816]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-03-04T09:25:25.381 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:25:24 smithi106 ceph-mon[36484]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-03-04T09:25:27.085 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:25:26 smithi052 ceph-mon[32473]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-03-04T09:25:27.336 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:25:26 smithi191 ceph-mon[36816]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-03-04T09:25:27.381 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:25:26 smithi106 ceph-mon[36484]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-03-04T09:25:29.166 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:25:28 smithi052 ceph-mon[32473]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-03-04T09:25:29.166 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:25:28 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-03-04T09:25:29.166 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:25:28 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T09:25:29.166 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:25:28 smithi052 ceph-mon[32473]: Deploying daemon osd.6 on smithi052 2024-03-04T09:25:29.336 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:25:28 smithi191 ceph-mon[36816]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-03-04T09:25:29.336 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:25:28 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-03-04T09:25:29.336 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:25:28 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T09:25:29.336 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:25:28 smithi191 ceph-mon[36816]: Deploying daemon osd.6 on smithi052 2024-03-04T09:25:29.381 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:25:28 smithi106 ceph-mon[36484]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-03-04T09:25:29.381 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:25:28 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-03-04T09:25:29.381 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:25:28 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T09:25:29.381 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:25:28 smithi106 ceph-mon[36484]: Deploying daemon osd.6 on smithi052 2024-03-04T09:25:31.242 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:25:30 smithi052 ceph-mon[32473]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-03-04T09:25:31.336 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:25:30 smithi191 ceph-mon[36816]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-03-04T09:25:31.381 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:25:30 smithi106 ceph-mon[36484]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-03-04T09:25:33.169 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:25:32 smithi052 ceph-mon[32473]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-03-04T09:25:33.336 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:25:32 smithi191 ceph-mon[36816]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-03-04T09:25:33.381 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:25:32 smithi106 ceph-mon[36484]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-03-04T09:25:35.085 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:25:34 smithi052 ceph-mon[32473]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-03-04T09:25:35.085 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:25:34 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:25:35.085 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:25:34 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:25:35.085 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:25:34 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T09:25:35.086 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:25:34 smithi191 ceph-mon[36816]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-03-04T09:25:35.086 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:25:34 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:25:35.086 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:25:34 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:25:35.086 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:25:34 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T09:25:35.131 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:25:34 smithi106 ceph-mon[36484]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-03-04T09:25:35.131 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:25:34 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:25:35.131 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:25:34 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:25:35.131 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:25:34 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T09:25:37.085 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:25:36 smithi052 ceph-mon[32473]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-03-04T09:25:37.085 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:25:36 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:25:37.085 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:25:36 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:25:37.085 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:25:36 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T09:25:37.085 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:25:36 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T09:25:37.085 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:25:36 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:25:37.086 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:25:36 smithi191 ceph-mon[36816]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-03-04T09:25:37.086 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:25:36 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:25:37.086 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:25:36 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:25:37.086 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:25:36 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T09:25:37.086 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:25:36 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T09:25:37.086 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:25:36 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:25:37.131 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:25:36 smithi106 ceph-mon[36484]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-03-04T09:25:37.131 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:25:36 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:25:37.131 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:25:36 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:25:37.131 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:25:36 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T09:25:37.131 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:25:36 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T09:25:37.131 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:25:36 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:25:37.731 INFO:teuthology.orchestra.run.smithi052.stdout:Created osd(s) 6 on host 'smithi052' 2024-03-04T09:25:37.732 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:25:37.734+0000 7f8997fff700 1 -- 172.21.15.52:0/1032301031 <== mgr.14152 v2:172.21.15.52:6800/3036033923 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f89b00fe740 con 0x7f89980a1e10 2024-03-04T09:25:37.733 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:25:37.736+0000 7f89b55c1700 1 -- 172.21.15.52:0/1032301031 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f89980a1e10 msgr2=0x7f89980a42d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:25:37.733 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:25:37.736+0000 7f89b55c1700 1 --2- 172.21.15.52:0/1032301031 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f89980a1e10 0x7f89980a42d0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f89a0009a10 tx=0x7f89a0009190 comp rx=0 tx=0).stop 2024-03-04T09:25:37.733 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:25:37.736+0000 7f89b55c1700 1 -- 172.21.15.52:0/1032301031 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f89b0101030 msgr2=0x7f89b0198440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:25:37.733 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:25:37.736+0000 7f89b55c1700 1 --2- 172.21.15.52:0/1032301031 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f89b0101030 0x7f89b0198440 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f89a400b530 tx=0x7f89a400b840 comp rx=0 tx=0).stop 2024-03-04T09:25:37.734 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:25:37.736+0000 7f89b55c1700 1 -- 172.21.15.52:0/1032301031 shutdown_connections 2024-03-04T09:25:37.734 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:25:37.736+0000 7f89b55c1700 1 --2- 172.21.15.52:0/1032301031 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f89980a1e10 0x7f89980a42d0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:25:37.734 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:25:37.736+0000 7f89b55c1700 1 --2- 172.21.15.52:0/1032301031 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f89b0102bd0 0x7f89b019fa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:25:37.734 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:25:37.736+0000 7f89b55c1700 1 --2- 172.21.15.52:0/1032301031 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f89b0101030 0x7f89b0198440 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:25:37.734 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:25:37.736+0000 7f89b55c1700 1 --2- 172.21.15.52:0/1032301031 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f89b0102230 0x7f89b0198980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:25:37.734 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:25:37.736+0000 7f89b55c1700 1 -- 172.21.15.52:0/1032301031 >> 172.21.15.52:0/1032301031 conn(0x7f89b00fc5b0 msgr2=0x7f89b00fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:25:37.734 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:25:37.737+0000 7f89b55c1700 1 -- 172.21.15.52:0/1032301031 shutdown_connections 2024-03-04T09:25:37.734 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:25:37.737+0000 7f89b55c1700 1 -- 172.21.15.52:0/1032301031 wait complete. 2024-03-04T09:25:38.046 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:25:37 smithi052 ceph-mon[32473]: from='osd.6 [v2:172.21.15.52:6818/1989797856,v1:172.21.15.52:6819/1989797856]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-03-04T09:25:38.046 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:25:37 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:25:38.047 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:25:37 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:25:38.047 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:25:37 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T09:25:38.086 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:25:37 smithi191 ceph-mon[36816]: from='osd.6 [v2:172.21.15.52:6818/1989797856,v1:172.21.15.52:6819/1989797856]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-03-04T09:25:38.086 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:25:37 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:25:38.086 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:25:37 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:25:38.086 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:25:37 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T09:25:38.131 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:25:37 smithi106 ceph-mon[36484]: from='osd.6 [v2:172.21.15.52:6818/1989797856,v1:172.21.15.52:6819/1989797856]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-03-04T09:25:38.131 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:25:37 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:25:38.131 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:25:37 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:25:38.131 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:25:37 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T09:25:38.465 DEBUG:teuthology.orchestra.run.smithi052:osd.6> sudo journalctl -f -n 0 -u ceph-4e059f08-da08-11ee-95c6-87774f69a715@osd.6.service 2024-03-04T09:25:38.467 INFO:tasks.cephadm:Deploying osd.7 on smithi106 with /dev/vg_nvme/lv_2... 2024-03-04T09:25:38.467 DEBUG:teuthology.orchestra.run.smithi106:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4e059f08-da08-11ee-95c6-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-03-04T09:25:38.709 INFO:teuthology.orchestra.run.smithi106.stderr:Inferring config /var/lib/ceph/4e059f08-da08-11ee-95c6-87774f69a715/mon.b/config 2024-03-04T09:25:39.061 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:25:38 smithi052 ceph-mon[32473]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-03-04T09:25:39.061 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:25:38 smithi052 ceph-mon[32473]: from='osd.6 [v2:172.21.15.52:6818/1989797856,v1:172.21.15.52:6819/1989797856]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-03-04T09:25:39.061 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:25:38 smithi052 ceph-mon[32473]: osdmap e44: 7 total, 6 up, 7 in 2024-03-04T09:25:39.061 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:25:38 smithi052 ceph-mon[32473]: from='osd.6 [v2:172.21.15.52:6818/1989797856,v1:172.21.15.52:6819/1989797856]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi052", "root=default"]}]: dispatch 2024-03-04T09:25:39.061 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:25:38 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-04T09:25:39.061 INFO:journalctl@ceph.osd.6.smithi052.stdout:Mar 04 09:25:38 smithi052 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-6[58363]: 2024-03-04T09:25:38.798+0000 7fa331260700 -1 osd.6 0 waiting for initial osdmap 2024-03-04T09:25:39.061 INFO:journalctl@ceph.osd.6.smithi052.stdout:Mar 04 09:25:38 smithi052 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-6[58363]: 2024-03-04T09:25:38.807+0000 7fa32d053700 -1 osd.6 45 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-04T09:25:39.086 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:25:38 smithi191 ceph-mon[36816]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-03-04T09:25:39.086 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:25:38 smithi191 ceph-mon[36816]: from='osd.6 [v2:172.21.15.52:6818/1989797856,v1:172.21.15.52:6819/1989797856]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-03-04T09:25:39.087 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:25:38 smithi191 ceph-mon[36816]: osdmap e44: 7 total, 6 up, 7 in 2024-03-04T09:25:39.087 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:25:38 smithi191 ceph-mon[36816]: from='osd.6 [v2:172.21.15.52:6818/1989797856,v1:172.21.15.52:6819/1989797856]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi052", "root=default"]}]: dispatch 2024-03-04T09:25:39.087 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:25:38 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-04T09:25:39.118 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:25:38 smithi106 ceph-mon[36484]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-03-04T09:25:39.118 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:25:38 smithi106 ceph-mon[36484]: from='osd.6 [v2:172.21.15.52:6818/1989797856,v1:172.21.15.52:6819/1989797856]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-03-04T09:25:39.118 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:25:38 smithi106 ceph-mon[36484]: osdmap e44: 7 total, 6 up, 7 in 2024-03-04T09:25:39.118 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:25:38 smithi106 ceph-mon[36484]: from='osd.6 [v2:172.21.15.52:6818/1989797856,v1:172.21.15.52:6819/1989797856]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi052", "root=default"]}]: dispatch 2024-03-04T09:25:39.118 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:25:38 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-04T09:25:39.983 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:25:39 smithi052 ceph-mon[32473]: from='osd.6 [v2:172.21.15.52:6818/1989797856,v1:172.21.15.52:6819/1989797856]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi052", "root=default"]}]': finished 2024-03-04T09:25:39.983 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:25:39 smithi052 ceph-mon[32473]: osdmap e45: 7 total, 6 up, 7 in 2024-03-04T09:25:39.983 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:25:39 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-04T09:25:39.983 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:25:39 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-04T09:25:39.983 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:25:39 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-04T09:25:40.086 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:25:39 smithi191 ceph-mon[36816]: from='osd.6 [v2:172.21.15.52:6818/1989797856,v1:172.21.15.52:6819/1989797856]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi052", "root=default"]}]': finished 2024-03-04T09:25:40.086 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:25:39 smithi191 ceph-mon[36816]: osdmap e45: 7 total, 6 up, 7 in 2024-03-04T09:25:40.086 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:25:39 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-04T09:25:40.087 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:25:39 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-04T09:25:40.087 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:25:39 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-04T09:25:40.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:25:39 smithi106 ceph-mon[36484]: from='osd.6 [v2:172.21.15.52:6818/1989797856,v1:172.21.15.52:6819/1989797856]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi052", "root=default"]}]': finished 2024-03-04T09:25:40.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:25:39 smithi106 ceph-mon[36484]: osdmap e45: 7 total, 6 up, 7 in 2024-03-04T09:25:40.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:25:39 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-04T09:25:40.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:25:39 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-04T09:25:40.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:25:39 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-04T09:25:40.962 INFO:teuthology.orchestra.run.smithi106.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-03-04T09:25:40.962 INFO:teuthology.orchestra.run.smithi106.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-03-04T09:25:40.962 INFO:teuthology.orchestra.run.smithi106.stderr: stderr: 10+0 records in 2024-03-04T09:25:40.962 INFO:teuthology.orchestra.run.smithi106.stderr:10+0 records out 2024-03-04T09:25:40.962 INFO:teuthology.orchestra.run.smithi106.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0112639 s, 931 MB/s 2024-03-04T09:25:40.962 INFO:teuthology.orchestra.run.smithi106.stderr:--> Zapping successful for: 2024-03-04T09:25:40.962 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:25:40 smithi106 ceph-mon[36484]: purged_snaps scrub starts 2024-03-04T09:25:40.962 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:25:40 smithi106 ceph-mon[36484]: purged_snaps scrub ok 2024-03-04T09:25:40.962 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:25:40 smithi106 ceph-mon[36484]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-03-04T09:25:40.962 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:25:40 smithi106 ceph-mon[36484]: osd.6 [v2:172.21.15.52:6818/1989797856,v1:172.21.15.52:6819/1989797856] boot 2024-03-04T09:25:40.962 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:25:40 smithi106 ceph-mon[36484]: osdmap e46: 7 total, 7 up, 7 in 2024-03-04T09:25:40.962 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:25:40 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-04T09:25:41.085 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:25:40 smithi052 ceph-mon[32473]: purged_snaps scrub starts 2024-03-04T09:25:41.085 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:25:40 smithi052 ceph-mon[32473]: purged_snaps scrub ok 2024-03-04T09:25:41.085 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:25:40 smithi052 ceph-mon[32473]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-03-04T09:25:41.085 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:25:40 smithi052 ceph-mon[32473]: osd.6 [v2:172.21.15.52:6818/1989797856,v1:172.21.15.52:6819/1989797856] boot 2024-03-04T09:25:41.085 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:25:40 smithi052 ceph-mon[32473]: osdmap e46: 7 total, 7 up, 7 in 2024-03-04T09:25:41.085 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:25:40 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-04T09:25:41.086 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:25:40 smithi191 ceph-mon[36816]: purged_snaps scrub starts 2024-03-04T09:25:41.086 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:25:40 smithi191 ceph-mon[36816]: purged_snaps scrub ok 2024-03-04T09:25:41.086 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:25:40 smithi191 ceph-mon[36816]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-03-04T09:25:41.086 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:25:40 smithi191 ceph-mon[36816]: osd.6 [v2:172.21.15.52:6818/1989797856,v1:172.21.15.52:6819/1989797856] boot 2024-03-04T09:25:41.086 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:25:40 smithi191 ceph-mon[36816]: osdmap e46: 7 total, 7 up, 7 in 2024-03-04T09:25:41.086 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:25:40 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-04T09:25:41.616 DEBUG:teuthology.orchestra.run.smithi106:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4e059f08-da08-11ee-95c6-87774f69a715 -- ceph orch daemon add osd smithi106:vg_nvme/lv_2 2024-03-04T09:25:41.854 INFO:teuthology.orchestra.run.smithi106.stderr:Inferring config /var/lib/ceph/4e059f08-da08-11ee-95c6-87774f69a715/mon.b/config 2024-03-04T09:25:41.881 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:25:41 smithi106 ceph-mon[36484]: osdmap e47: 7 total, 7 up, 7 in 2024-03-04T09:25:41.881 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:25:41 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:25:42.084 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:25:41 smithi052 ceph-mon[32473]: osdmap e47: 7 total, 7 up, 7 in 2024-03-04T09:25:42.085 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:25:41 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:25:42.086 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:25:41 smithi191 ceph-mon[36816]: osdmap e47: 7 total, 7 up, 7 in 2024-03-04T09:25:42.086 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:25:41 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:25:43.028 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:25:42 smithi106 ceph-mon[36484]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 201 MiB used, 626 GiB / 626 GiB avail 2024-03-04T09:25:43.028 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:25:42 smithi106 ceph-mon[36484]: Detected new or changed devices on smithi052 2024-03-04T09:25:43.028 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:25:42 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:25:43.029 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:25:42 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-04T09:25:43.029 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:25:42 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-04T09:25:43.029 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:25:42 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-04T09:25:43.029 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:25:42 smithi106 ceph-mon[36484]: Adjusting osd_memory_target on smithi052 to 5697M 2024-03-04T09:25:43.029 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:25:42 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:25:43.029 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:25:42 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T09:25:43.029 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:25:42 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T09:25:43.029 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:25:42 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:25:43.085 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:25:42 smithi052 ceph-mon[32473]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 201 MiB used, 626 GiB / 626 GiB avail 2024-03-04T09:25:43.085 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:25:42 smithi052 ceph-mon[32473]: Detected new or changed devices on smithi052 2024-03-04T09:25:43.085 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:25:42 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:25:43.085 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:25:42 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-04T09:25:43.085 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:25:42 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-04T09:25:43.085 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:25:42 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-04T09:25:43.085 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:25:42 smithi052 ceph-mon[32473]: Adjusting osd_memory_target on smithi052 to 5697M 2024-03-04T09:25:43.085 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:25:42 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:25:43.085 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:25:42 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T09:25:43.086 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:25:42 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T09:25:43.086 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:25:42 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:25:43.086 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:25:42 smithi191 ceph-mon[36816]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 201 MiB used, 626 GiB / 626 GiB avail 2024-03-04T09:25:43.086 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:25:42 smithi191 ceph-mon[36816]: Detected new or changed devices on smithi052 2024-03-04T09:25:43.086 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:25:42 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:25:43.086 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:25:42 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-04T09:25:43.086 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:25:42 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-04T09:25:43.087 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:25:42 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-04T09:25:43.087 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:25:42 smithi191 ceph-mon[36816]: Adjusting osd_memory_target on smithi052 to 5697M 2024-03-04T09:25:43.087 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:25:42 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:25:43.087 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:25:42 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T09:25:43.087 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:25:42 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T09:25:43.087 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:25:42 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:25:43.604 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-04T09:25:43.604+0000 7f2b4dbda700 1 -- 172.21.15.106:0/3525567315 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2b48105510 msgr2=0x7f2b48107900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:25:43.604 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-04T09:25:43.604+0000 7f2b4dbda700 1 --2- 172.21.15.106:0/3525567315 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2b48105510 0x7f2b48107900 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f2b3c009990 tx=0x7f2b3c009ca0 comp rx=0 tx=0).stop 2024-03-04T09:25:43.604 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-04T09:25:43.604+0000 7f2b4dbda700 1 -- 172.21.15.106:0/3525567315 shutdown_connections 2024-03-04T09:25:43.604 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-04T09:25:43.604+0000 7f2b4dbda700 1 --2- 172.21.15.106:0/3525567315 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2b48102be0 0x7f2b48104fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:25:43.604 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-04T09:25:43.604+0000 7f2b4dbda700 1 --2- 172.21.15.106:0/3525567315 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2b48107e40 0x7f2b4810a250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:25:43.604 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-04T09:25:43.604+0000 7f2b4dbda700 1 --2- 172.21.15.106:0/3525567315 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2b48105510 0x7f2b48107900 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:25:43.604 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-04T09:25:43.604+0000 7f2b4dbda700 1 -- 172.21.15.106:0/3525567315 >> 172.21.15.106:0/3525567315 conn(0x7f2b480fc590 msgr2=0x7f2b480fe9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:25:43.605 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-04T09:25:43.604+0000 7f2b4dbda700 1 -- 172.21.15.106:0/3525567315 shutdown_connections 2024-03-04T09:25:43.605 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-04T09:25:43.604+0000 7f2b4dbda700 1 -- 172.21.15.106:0/3525567315 wait complete. 2024-03-04T09:25:43.605 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-04T09:25:43.604+0000 7f2b4dbda700 1 Processor -- start 2024-03-04T09:25:43.605 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-04T09:25:43.605+0000 7f2b4dbda700 1 -- start start 2024-03-04T09:25:43.605 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-04T09:25:43.605+0000 7f2b4dbda700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2b48102be0 0x7f2b48198350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:25:43.605 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-04T09:25:43.605+0000 7f2b4dbda700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2b48105510 0x7f2b48198890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:25:43.605 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-04T09:25:43.605+0000 7f2b4dbda700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2b48107e40 0x7f2b4819f910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:25:43.605 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-04T09:25:43.605+0000 7f2b4dbda700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b4810dd00 con 0x7f2b48105510 2024-03-04T09:25:43.605 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-04T09:25:43.605+0000 7f2b4dbda700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b4810db80 con 0x7f2b48107e40 2024-03-04T09:25:43.605 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-04T09:25:43.605+0000 7f2b4dbda700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b4810de80 con 0x7f2b48102be0 2024-03-04T09:25:43.605 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-04T09:25:43.605+0000 7f2b47fff700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2b48107e40 0x7f2b4819f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:25:43.605 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-04T09:25:43.605+0000 7f2b47fff700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2b48107e40 0x7f2b4819f910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:35382/0 (socket says 172.21.15.106:35382) 2024-03-04T09:25:43.605 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-04T09:25:43.605+0000 7f2b477fe700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2b48102be0 0x7f2b48198350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:25:43.606 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-04T09:25:43.605+0000 7f2b47fff700 1 -- 172.21.15.106:0/3659392103 learned_addr learned my addr 172.21.15.106:0/3659392103 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-03-04T09:25:43.606 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-04T09:25:43.605+0000 7f2b46ffd700 1 --2- 172.21.15.106:0/3659392103 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2b48105510 0x7f2b48198890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:25:43.606 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-04T09:25:43.605+0000 7f2b47fff700 1 -- 172.21.15.106:0/3659392103 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2b48102be0 msgr2=0x7f2b48198350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:25:43.606 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-04T09:25:43.605+0000 7f2b47fff700 1 --2- 172.21.15.106:0/3659392103 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2b48102be0 0x7f2b48198350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:25:43.606 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-04T09:25:43.605+0000 7f2b47fff700 1 -- 172.21.15.106:0/3659392103 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2b48105510 msgr2=0x7f2b48198890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:25:43.606 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-04T09:25:43.605+0000 7f2b47fff700 1 --2- 172.21.15.106:0/3659392103 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2b48105510 0x7f2b48198890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:25:43.606 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-04T09:25:43.605+0000 7f2b47fff700 1 -- 172.21.15.106:0/3659392103 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b4819ffb0 con 0x7f2b48107e40 2024-03-04T09:25:43.606 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-04T09:25:43.605+0000 7f2b46ffd700 1 --2- 172.21.15.106:0/3659392103 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2b48105510 0x7f2b48198890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T09:25:43.606 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-04T09:25:43.605+0000 7f2b47fff700 1 --2- 172.21.15.106:0/3659392103 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2b48107e40 0x7f2b4819f910 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f2b4000e970 tx=0x7f2b4000ec80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:25:43.606 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-04T09:25:43.606+0000 7f2b44ff9700 1 -- 172.21.15.106:0/3659392103 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b4000cbd0 con 0x7f2b48107e40 2024-03-04T09:25:43.606 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-04T09:25:43.606+0000 7f2b44ff9700 1 -- 172.21.15.106:0/3659392103 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2b4000ee60 con 0x7f2b48107e40 2024-03-04T09:25:43.606 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-04T09:25:43.606+0000 7f2b4dbda700 1 -- 172.21.15.106:0/3659392103 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b481a02a0 con 0x7f2b48107e40 2024-03-04T09:25:43.607 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-04T09:25:43.606+0000 7f2b44ff9700 1 -- 172.21.15.106:0/3659392103 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b40005630 con 0x7f2b48107e40 2024-03-04T09:25:43.607 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-04T09:25:43.606+0000 7f2b4dbda700 1 -- 172.21.15.106:0/3659392103 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b481a0790 con 0x7f2b48107e40 2024-03-04T09:25:43.607 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-04T09:25:43.607+0000 7f2b4dbda700 1 -- 172.21.15.106:0/3659392103 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2b48068d80 con 0x7f2b48107e40 2024-03-04T09:25:43.608 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-04T09:25:43.609+0000 7f2b44ff9700 1 -- 172.21.15.106:0/3659392103 <== mon.2 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 16) v1 ==== 136756+0+0 (secure 0 0 0) 0x7f2b40004750 con 0x7f2b48107e40 2024-03-04T09:25:43.608 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-04T09:25:43.609+0000 7f2b44ff9700 1 --2- 172.21.15.106:0/3659392103 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f2b300a1e10 0x7f2b300a42d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:25:43.608 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-04T09:25:43.609+0000 7f2b44ff9700 1 -- 172.21.15.106:0/3659392103 <== mon.2 v2:172.21.15.106:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 4965+0+0 (secure 0 0 0) 0x7f2b400cd4f0 con 0x7f2b48107e40 2024-03-04T09:25:43.608 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-04T09:25:43.609+0000 7f2b477fe700 1 --2- 172.21.15.106:0/3659392103 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f2b300a1e10 0x7f2b300a42d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:25:43.608 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-04T09:25:43.610+0000 7f2b477fe700 1 --2- 172.21.15.106:0/3659392103 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f2b300a1e10 0x7f2b300a42d0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f2b3800f540 tx=0x7f2b3800b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:25:43.610 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-04T09:25:43.613+0000 7f2b44ff9700 1 -- 172.21.15.106:0/3659392103 <== mon.2 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f2b40053280 con 0x7f2b48107e40 2024-03-04T09:25:43.814 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-04T09:25:43.817+0000 7f2b4dbda700 1 -- 172.21.15.106:0/3659392103 --> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi106:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7f2b48062500 con 0x7f2b300a1e10 2024-03-04T09:25:45.085 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:25:44 smithi052 ceph-mon[32473]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 201 MiB used, 626 GiB / 626 GiB avail 2024-03-04T09:25:45.085 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:25:44 smithi052 ceph-mon[32473]: from='client.24260 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi106:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T09:25:45.085 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:25:44 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-04T09:25:45.085 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:25:44 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-04T09:25:45.085 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:25:44 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T09:25:45.086 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:25:44 smithi191 ceph-mon[36816]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 201 MiB used, 626 GiB / 626 GiB avail 2024-03-04T09:25:45.086 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:25:44 smithi191 ceph-mon[36816]: from='client.24260 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi106:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T09:25:45.087 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:25:44 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-04T09:25:45.087 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:25:44 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-04T09:25:45.087 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:25:44 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T09:25:45.131 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:25:44 smithi106 ceph-mon[36484]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 201 MiB used, 626 GiB / 626 GiB avail 2024-03-04T09:25:45.131 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:25:44 smithi106 ceph-mon[36484]: from='client.24260 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi106:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T09:25:45.131 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:25:44 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-04T09:25:45.131 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:25:44 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-04T09:25:45.131 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:25:44 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T09:25:47.085 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:25:46 smithi052 ceph-mon[32473]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-03-04T09:25:47.085 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:25:46 smithi052 ceph-mon[32473]: from='client.? 172.21.15.106:0/769549043' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "71303e3d-14ec-4cbd-b19c-67645a1de3c4"}]: dispatch 2024-03-04T09:25:47.085 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:25:46 smithi052 ceph-mon[32473]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "71303e3d-14ec-4cbd-b19c-67645a1de3c4"}]: dispatch 2024-03-04T09:25:47.085 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:25:46 smithi052 ceph-mon[32473]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "71303e3d-14ec-4cbd-b19c-67645a1de3c4"}]': finished 2024-03-04T09:25:47.085 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:25:46 smithi052 ceph-mon[32473]: osdmap e48: 8 total, 7 up, 8 in 2024-03-04T09:25:47.085 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:25:46 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-04T09:25:47.086 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:25:46 smithi191 ceph-mon[36816]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-03-04T09:25:47.086 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:25:46 smithi191 ceph-mon[36816]: from='client.? 172.21.15.106:0/769549043' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "71303e3d-14ec-4cbd-b19c-67645a1de3c4"}]: dispatch 2024-03-04T09:25:47.086 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:25:46 smithi191 ceph-mon[36816]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "71303e3d-14ec-4cbd-b19c-67645a1de3c4"}]: dispatch 2024-03-04T09:25:47.086 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:25:46 smithi191 ceph-mon[36816]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "71303e3d-14ec-4cbd-b19c-67645a1de3c4"}]': finished 2024-03-04T09:25:47.086 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:25:46 smithi191 ceph-mon[36816]: osdmap e48: 8 total, 7 up, 8 in 2024-03-04T09:25:47.086 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:25:46 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-04T09:25:47.131 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:25:46 smithi106 ceph-mon[36484]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-03-04T09:25:47.131 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:25:46 smithi106 ceph-mon[36484]: from='client.? 172.21.15.106:0/769549043' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "71303e3d-14ec-4cbd-b19c-67645a1de3c4"}]: dispatch 2024-03-04T09:25:47.131 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:25:46 smithi106 ceph-mon[36484]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "71303e3d-14ec-4cbd-b19c-67645a1de3c4"}]: dispatch 2024-03-04T09:25:47.131 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:25:46 smithi106 ceph-mon[36484]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "71303e3d-14ec-4cbd-b19c-67645a1de3c4"}]': finished 2024-03-04T09:25:47.131 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:25:46 smithi106 ceph-mon[36484]: osdmap e48: 8 total, 7 up, 8 in 2024-03-04T09:25:47.131 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:25:46 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-04T09:25:48.084 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:25:47 smithi052 ceph-mon[32473]: from='client.? 172.21.15.106:0/1225814248' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-04T09:25:48.086 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:25:47 smithi191 ceph-mon[36816]: from='client.? 172.21.15.106:0/1225814248' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-04T09:25:48.131 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:25:47 smithi106 ceph-mon[36484]: from='client.? 172.21.15.106:0/1225814248' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-04T09:25:49.085 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:25:48 smithi052 ceph-mon[32473]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-03-04T09:25:49.086 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:25:48 smithi191 ceph-mon[36816]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-03-04T09:25:49.131 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:25:48 smithi106 ceph-mon[36484]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-03-04T09:25:51.085 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:25:50 smithi052 ceph-mon[32473]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-03-04T09:25:51.086 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:25:50 smithi191 ceph-mon[36816]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-03-04T09:25:51.131 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:25:50 smithi106 ceph-mon[36484]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-03-04T09:25:53.085 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:25:52 smithi052 ceph-mon[32473]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-03-04T09:25:53.086 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:25:52 smithi191 ceph-mon[36816]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-03-04T09:25:53.131 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:25:52 smithi106 ceph-mon[36484]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-03-04T09:25:55.086 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:25:54 smithi191 ceph-mon[36816]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-03-04T09:25:55.087 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:25:54 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-03-04T09:25:55.087 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:25:54 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T09:25:55.087 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:25:54 smithi191 ceph-mon[36816]: Deploying daemon osd.7 on smithi106 2024-03-04T09:25:55.131 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:25:54 smithi106 ceph-mon[36484]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-03-04T09:25:55.131 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:25:54 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-03-04T09:25:55.131 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:25:54 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T09:25:55.131 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:25:54 smithi106 ceph-mon[36484]: Deploying daemon osd.7 on smithi106 2024-03-04T09:25:55.336 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:25:54 smithi052 ceph-mon[32473]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-03-04T09:25:55.336 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:25:54 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-03-04T09:25:55.336 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:25:54 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T09:25:55.336 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:25:54 smithi052 ceph-mon[32473]: Deploying daemon osd.7 on smithi106 2024-03-04T09:25:56.861 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:25:56 smithi106 ceph-mon[36484]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-03-04T09:25:57.334 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:25:56 smithi052 ceph-mon[32473]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-03-04T09:25:57.336 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:25:56 smithi191 ceph-mon[36816]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-03-04T09:25:58.945 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:25:58 smithi106 ceph-mon[36484]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-03-04T09:25:59.334 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:25:58 smithi052 ceph-mon[32473]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-03-04T09:25:59.336 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:25:58 smithi191 ceph-mon[36816]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-03-04T09:26:00.808 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:26:00 smithi106 ceph-mon[36484]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-03-04T09:26:00.808 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:26:00 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:26:00.808 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:26:00 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:26:00.808 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:26:00 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T09:26:00.834 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:26:00 smithi052 ceph-mon[32473]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-03-04T09:26:00.834 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:26:00 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:26:00.834 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:26:00 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:26:00.835 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:26:00 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T09:26:00.836 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:26:00 smithi191 ceph-mon[36816]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-03-04T09:26:00.836 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:26:00 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:26:00.836 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:26:00 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:26:00.836 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:26:00 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T09:26:02.631 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:26:02 smithi106 ceph-mon[36484]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-03-04T09:26:02.834 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:26:02 smithi052 ceph-mon[32473]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-03-04T09:26:02.836 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:26:02 smithi191 ceph-mon[36816]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-03-04T09:26:03.584 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-04T09:26:03.587+0000 7f2b44ff9700 1 -- 172.21.15.106:0/3659392103 <== mgr.14152 v2:172.21.15.52:6800/3036033923 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f2b48062500 con 0x7f2b300a1e10 2024-03-04T09:26:03.585 INFO:teuthology.orchestra.run.smithi106.stdout:Created osd(s) 7 on host 'smithi106' 2024-03-04T09:26:03.587 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-04T09:26:03.589+0000 7f2b4dbda700 1 -- 172.21.15.106:0/3659392103 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f2b300a1e10 msgr2=0x7f2b300a42d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:26:03.587 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-04T09:26:03.589+0000 7f2b4dbda700 1 --2- 172.21.15.106:0/3659392103 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f2b300a1e10 0x7f2b300a42d0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f2b3800f540 tx=0x7f2b3800b040 comp rx=0 tx=0).stop 2024-03-04T09:26:03.587 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-04T09:26:03.589+0000 7f2b4dbda700 1 -- 172.21.15.106:0/3659392103 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2b48107e40 msgr2=0x7f2b4819f910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:26:03.587 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-04T09:26:03.589+0000 7f2b4dbda700 1 --2- 172.21.15.106:0/3659392103 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2b48107e40 0x7f2b4819f910 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f2b4000e970 tx=0x7f2b4000ec80 comp rx=0 tx=0).stop 2024-03-04T09:26:03.587 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-04T09:26:03.589+0000 7f2b4dbda700 1 -- 172.21.15.106:0/3659392103 shutdown_connections 2024-03-04T09:26:03.587 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-04T09:26:03.589+0000 7f2b4dbda700 1 --2- 172.21.15.106:0/3659392103 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f2b300a1e10 0x7f2b300a42d0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:26:03.588 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-04T09:26:03.589+0000 7f2b4dbda700 1 --2- 172.21.15.106:0/3659392103 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2b48102be0 0x7f2b48198350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:26:03.588 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-04T09:26:03.589+0000 7f2b4dbda700 1 --2- 172.21.15.106:0/3659392103 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2b48107e40 0x7f2b4819f910 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:26:03.588 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-04T09:26:03.589+0000 7f2b4dbda700 1 --2- 172.21.15.106:0/3659392103 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2b48105510 0x7f2b48198890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:26:03.588 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-04T09:26:03.589+0000 7f2b4dbda700 1 -- 172.21.15.106:0/3659392103 >> 172.21.15.106:0/3659392103 conn(0x7f2b480fc590 msgr2=0x7f2b48103660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:26:03.588 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-04T09:26:03.590+0000 7f2b4dbda700 1 -- 172.21.15.106:0/3659392103 shutdown_connections 2024-03-04T09:26:03.588 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-04T09:26:03.590+0000 7f2b4dbda700 1 -- 172.21.15.106:0/3659392103 wait complete. 2024-03-04T09:26:03.881 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:26:03 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:26:03.881 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:26:03 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:26:03.881 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:26:03 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T09:26:03.881 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:26:03 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T09:26:03.881 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:26:03 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:26:03.881 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:26:03 smithi106 ceph-mon[36484]: from='osd.7 [v2:172.21.15.106:6816/1450870329,v1:172.21.15.106:6817/1450870329]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-03-04T09:26:03.881 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:26:03 smithi106 ceph-mon[36484]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-03-04T09:26:03.881 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:26:03 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:26:03.882 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:26:03 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:26:03.882 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:26:03 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T09:26:04.084 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:26:03 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:26:04.085 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:26:03 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:26:04.085 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:26:03 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T09:26:04.085 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:26:03 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T09:26:04.085 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:26:03 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:26:04.085 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:26:03 smithi052 ceph-mon[32473]: from='osd.7 [v2:172.21.15.106:6816/1450870329,v1:172.21.15.106:6817/1450870329]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-03-04T09:26:04.085 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:26:03 smithi052 ceph-mon[32473]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-03-04T09:26:04.085 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:26:03 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:26:04.085 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:26:03 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:26:04.085 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:26:03 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T09:26:04.086 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:26:03 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:26:04.086 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:26:03 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:26:04.086 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:26:03 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T09:26:04.086 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:26:03 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T09:26:04.086 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:26:03 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:26:04.086 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:26:03 smithi191 ceph-mon[36816]: from='osd.7 [v2:172.21.15.106:6816/1450870329,v1:172.21.15.106:6817/1450870329]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-03-04T09:26:04.086 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:26:03 smithi191 ceph-mon[36816]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-03-04T09:26:04.086 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:26:03 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:26:04.086 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:26:03 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:26:04.087 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:26:03 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T09:26:04.316 DEBUG:teuthology.orchestra.run.smithi106:osd.7> sudo journalctl -f -n 0 -u ceph-4e059f08-da08-11ee-95c6-87774f69a715@osd.7.service 2024-03-04T09:26:04.319 INFO:tasks.cephadm:Deploying osd.8 on smithi191 with /dev/vg_nvme/lv_2... 2024-03-04T09:26:04.319 DEBUG:teuthology.orchestra.run.smithi191:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4e059f08-da08-11ee-95c6-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-03-04T09:26:04.565 INFO:teuthology.orchestra.run.smithi191.stderr:Inferring config /var/lib/ceph/4e059f08-da08-11ee-95c6-87774f69a715/mon.c/config 2024-03-04T09:26:04.793 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:26:04 smithi106 ceph-mon[36484]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-03-04T09:26:04.793 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:26:04 smithi106 ceph-mon[36484]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-03-04T09:26:04.793 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:26:04 smithi106 ceph-mon[36484]: osdmap e49: 8 total, 7 up, 8 in 2024-03-04T09:26:04.793 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:26:04 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-04T09:26:04.793 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:26:04 smithi106 ceph-mon[36484]: from='osd.7 [v2:172.21.15.106:6816/1450870329,v1:172.21.15.106:6817/1450870329]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi106", "root=default"]}]: dispatch 2024-03-04T09:26:04.793 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:26:04 smithi106 ceph-mon[36484]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi106", "root=default"]}]: dispatch 2024-03-04T09:26:04.793 INFO:journalctl@ceph.osd.7.smithi106.stdout:Mar 04 09:26:04 smithi106 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-7[54346]: 2024-03-04T09:26:04.712+0000 7f3be465c700 -1 osd.7 0 waiting for initial osdmap 2024-03-04T09:26:04.793 INFO:journalctl@ceph.osd.7.smithi106.stdout:Mar 04 09:26:04 smithi106 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-7[54346]: 2024-03-04T09:26:04.722+0000 7f3be044f700 -1 osd.7 50 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-04T09:26:05.019 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:26:04 smithi191 ceph-mon[36816]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-03-04T09:26:05.019 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:26:04 smithi191 ceph-mon[36816]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-03-04T09:26:05.020 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:26:04 smithi191 ceph-mon[36816]: osdmap e49: 8 total, 7 up, 8 in 2024-03-04T09:26:05.020 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:26:04 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-04T09:26:05.020 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:26:04 smithi191 ceph-mon[36816]: from='osd.7 [v2:172.21.15.106:6816/1450870329,v1:172.21.15.106:6817/1450870329]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi106", "root=default"]}]: dispatch 2024-03-04T09:26:05.020 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:26:04 smithi191 ceph-mon[36816]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi106", "root=default"]}]: dispatch 2024-03-04T09:26:05.085 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:26:04 smithi052 ceph-mon[32473]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-03-04T09:26:05.085 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:26:04 smithi052 ceph-mon[32473]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-03-04T09:26:05.085 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:26:04 smithi052 ceph-mon[32473]: osdmap e49: 8 total, 7 up, 8 in 2024-03-04T09:26:05.085 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:26:04 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-04T09:26:05.085 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:26:04 smithi052 ceph-mon[32473]: from='osd.7 [v2:172.21.15.106:6816/1450870329,v1:172.21.15.106:6817/1450870329]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi106", "root=default"]}]: dispatch 2024-03-04T09:26:05.085 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:26:04 smithi052 ceph-mon[32473]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi106", "root=default"]}]: dispatch 2024-03-04T09:26:05.881 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:26:05 smithi106 ceph-mon[36484]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi106", "root=default"]}]': finished 2024-03-04T09:26:05.881 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:26:05 smithi106 ceph-mon[36484]: osdmap e50: 8 total, 7 up, 8 in 2024-03-04T09:26:05.881 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:26:05 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-04T09:26:05.881 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:26:05 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-04T09:26:06.077 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:26:05 smithi191 ceph-mon[36816]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi106", "root=default"]}]': finished 2024-03-04T09:26:06.077 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:26:05 smithi191 ceph-mon[36816]: osdmap e50: 8 total, 7 up, 8 in 2024-03-04T09:26:06.077 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:26:05 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-04T09:26:06.077 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:26:05 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-04T09:26:06.085 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:26:05 smithi052 ceph-mon[32473]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi106", "root=default"]}]': finished 2024-03-04T09:26:06.085 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:26:05 smithi052 ceph-mon[32473]: osdmap e50: 8 total, 7 up, 8 in 2024-03-04T09:26:06.085 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:26:05 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-04T09:26:06.085 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:26:05 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-04T09:26:06.722 INFO:teuthology.orchestra.run.smithi191.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-03-04T09:26:06.722 INFO:teuthology.orchestra.run.smithi191.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-03-04T09:26:06.722 INFO:teuthology.orchestra.run.smithi191.stderr: stderr: 10+0 records in 2024-03-04T09:26:06.722 INFO:teuthology.orchestra.run.smithi191.stderr:10+0 records out 2024-03-04T09:26:06.722 INFO:teuthology.orchestra.run.smithi191.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0127964 s, 819 MB/s 2024-03-04T09:26:06.722 INFO:teuthology.orchestra.run.smithi191.stderr:--> Zapping successful for: 2024-03-04T09:26:07.021 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:26:06 smithi106 ceph-mon[36484]: purged_snaps scrub starts 2024-03-04T09:26:07.021 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:26:06 smithi106 ceph-mon[36484]: purged_snaps scrub ok 2024-03-04T09:26:07.021 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:26:06 smithi106 ceph-mon[36484]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-03-04T09:26:07.021 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:26:06 smithi106 ceph-mon[36484]: osd.7 [v2:172.21.15.106:6816/1450870329,v1:172.21.15.106:6817/1450870329] boot 2024-03-04T09:26:07.021 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:26:06 smithi106 ceph-mon[36484]: osdmap e51: 8 total, 8 up, 8 in 2024-03-04T09:26:07.021 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:26:06 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-04T09:26:07.022 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:26:06 smithi106 ceph-mon[36484]: osdmap e52: 8 total, 8 up, 8 in 2024-03-04T09:26:07.074 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:26:06 smithi191 ceph-mon[36816]: purged_snaps scrub starts 2024-03-04T09:26:07.074 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:26:06 smithi191 ceph-mon[36816]: purged_snaps scrub ok 2024-03-04T09:26:07.074 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:26:06 smithi191 ceph-mon[36816]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-03-04T09:26:07.074 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:26:06 smithi191 ceph-mon[36816]: osd.7 [v2:172.21.15.106:6816/1450870329,v1:172.21.15.106:6817/1450870329] boot 2024-03-04T09:26:07.075 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:26:06 smithi191 ceph-mon[36816]: osdmap e51: 8 total, 8 up, 8 in 2024-03-04T09:26:07.075 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:26:06 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-04T09:26:07.075 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:26:06 smithi191 ceph-mon[36816]: osdmap e52: 8 total, 8 up, 8 in 2024-03-04T09:26:07.084 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:26:06 smithi052 ceph-mon[32473]: purged_snaps scrub starts 2024-03-04T09:26:07.084 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:26:06 smithi052 ceph-mon[32473]: purged_snaps scrub ok 2024-03-04T09:26:07.085 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:26:06 smithi052 ceph-mon[32473]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-03-04T09:26:07.085 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:26:06 smithi052 ceph-mon[32473]: osd.7 [v2:172.21.15.106:6816/1450870329,v1:172.21.15.106:6817/1450870329] boot 2024-03-04T09:26:07.085 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:26:06 smithi052 ceph-mon[32473]: osdmap e51: 8 total, 8 up, 8 in 2024-03-04T09:26:07.085 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:26:06 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-04T09:26:07.085 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:26:06 smithi052 ceph-mon[32473]: osdmap e52: 8 total, 8 up, 8 in 2024-03-04T09:26:07.344 DEBUG:teuthology.orchestra.run.smithi191:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4e059f08-da08-11ee-95c6-87774f69a715 -- ceph orch daemon add osd smithi191:vg_nvme/lv_2 2024-03-04T09:26:07.593 INFO:teuthology.orchestra.run.smithi191.stderr:Inferring config /var/lib/ceph/4e059f08-da08-11ee-95c6-87774f69a715/mon.c/config 2024-03-04T09:26:07.986 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:26:07 smithi191 ceph-mon[36816]: osdmap e53: 8 total, 8 up, 8 in 2024-03-04T09:26:07.987 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:26:07 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:26:07.987 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:26:07 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:26:07.987 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:26:07 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-04T09:26:07.987 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:26:07 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-04T09:26:07.987 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:26:07 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-04T09:26:07.987 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:26:07 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:26:07.987 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:26:07 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T09:26:07.987 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:26:07 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T09:26:07.987 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:26:07 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:26:08.084 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:26:07 smithi052 ceph-mon[32473]: osdmap e53: 8 total, 8 up, 8 in 2024-03-04T09:26:08.085 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:26:07 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:26:08.085 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:26:07 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:26:08.085 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:26:07 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-04T09:26:08.085 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:26:07 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-04T09:26:08.085 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:26:07 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-04T09:26:08.085 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:26:07 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:26:08.085 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:26:07 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T09:26:08.085 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:26:07 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T09:26:08.085 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:26:07 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:26:08.131 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:26:07 smithi106 ceph-mon[36484]: osdmap e53: 8 total, 8 up, 8 in 2024-03-04T09:26:08.131 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:26:07 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:26:08.131 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:26:07 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:26:08.131 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:26:07 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-04T09:26:08.131 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:26:07 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-04T09:26:08.131 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:26:07 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-04T09:26:08.131 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:26:07 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:26:08.131 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:26:07 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T09:26:08.131 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:26:07 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T09:26:08.131 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:26:07 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:26:09.043 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:26:08 smithi191 ceph-mon[36816]: Detected new or changed devices on smithi106 2024-03-04T09:26:09.043 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:26:08 smithi191 ceph-mon[36816]: Adjusting osd_memory_target on smithi106 to 5693M 2024-03-04T09:26:09.043 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:26:08 smithi191 ceph-mon[36816]: pgmap v177: 1 pgs: 1 remapped+peering; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-03-04T09:26:09.043 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:26:08 smithi191 ceph-mon[36816]: osdmap e54: 8 total, 8 up, 8 in 2024-03-04T09:26:09.084 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:26:08 smithi052 ceph-mon[32473]: Detected new or changed devices on smithi106 2024-03-04T09:26:09.084 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:26:08 smithi052 ceph-mon[32473]: Adjusting osd_memory_target on smithi106 to 5693M 2024-03-04T09:26:09.085 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:26:08 smithi052 ceph-mon[32473]: pgmap v177: 1 pgs: 1 remapped+peering; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-03-04T09:26:09.085 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:26:08 smithi052 ceph-mon[32473]: osdmap e54: 8 total, 8 up, 8 in 2024-03-04T09:26:09.131 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:26:08 smithi106 ceph-mon[36484]: Detected new or changed devices on smithi106 2024-03-04T09:26:09.131 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:26:08 smithi106 ceph-mon[36484]: Adjusting osd_memory_target on smithi106 to 5693M 2024-03-04T09:26:09.131 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:26:08 smithi106 ceph-mon[36484]: pgmap v177: 1 pgs: 1 remapped+peering; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-03-04T09:26:09.131 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:26:08 smithi106 ceph-mon[36484]: osdmap e54: 8 total, 8 up, 8 in 2024-03-04T09:26:09.274 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:26:09.274+0000 7f8d99fdf700 1 -- 172.21.15.191:0/2806900399 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8d94103320 msgr2=0x7f8d94103780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:26:09.274 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:26:09.274+0000 7f8d99fdf700 1 --2- 172.21.15.191:0/2806900399 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8d94103320 0x7f8d94103780 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f8d84009990 tx=0x7f8d84009ca0 comp rx=0 tx=0).stop 2024-03-04T09:26:09.274 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:26:09.274+0000 7f8d99fdf700 1 -- 172.21.15.191:0/2806900399 shutdown_connections 2024-03-04T09:26:09.274 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:26:09.274+0000 7f8d99fdf700 1 --2- 172.21.15.191:0/2806900399 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8d94103cc0 0x7f8d9410a570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:26:09.274 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:26:09.274+0000 7f8d99fdf700 1 --2- 172.21.15.191:0/2806900399 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8d94102120 0x7f8d94102540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:26:09.274 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:26:09.274+0000 7f8d99fdf700 1 --2- 172.21.15.191:0/2806900399 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8d94103320 0x7f8d94103780 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:26:09.274 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:26:09.274+0000 7f8d99fdf700 1 -- 172.21.15.191:0/2806900399 >> 172.21.15.191:0/2806900399 conn(0x7f8d940fd6c0 msgr2=0x7f8d940ffb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:26:09.274 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:26:09.274+0000 7f8d99fdf700 1 -- 172.21.15.191:0/2806900399 shutdown_connections 2024-03-04T09:26:09.274 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:26:09.274+0000 7f8d99fdf700 1 -- 172.21.15.191:0/2806900399 wait complete. 2024-03-04T09:26:09.274 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:26:09.275+0000 7f8d99fdf700 1 Processor -- start 2024-03-04T09:26:09.275 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:26:09.275+0000 7f8d99fdf700 1 -- start start 2024-03-04T09:26:09.275 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:26:09.275+0000 7f8d99fdf700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8d94102120 0x7f8d94198310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:26:09.275 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:26:09.275+0000 7f8d99fdf700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8d94103320 0x7f8d94198850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:26:09.275 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:26:09.275+0000 7f8d99fdf700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8d94103cc0 0x7f8d9419f8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:26:09.275 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:26:09.275+0000 7f8d99fdf700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d9410dd00 con 0x7f8d94103320 2024-03-04T09:26:09.275 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:26:09.275+0000 7f8d99fdf700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d9410db80 con 0x7f8d94103cc0 2024-03-04T09:26:09.275 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:26:09.275+0000 7f8d99fdf700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d9410de80 con 0x7f8d94102120 2024-03-04T09:26:09.275 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:26:09.275+0000 7f8d937fe700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8d94102120 0x7f8d94198310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:26:09.275 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:26:09.276+0000 7f8d937fe700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8d94102120 0x7f8d94198310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.191:41050/0 (socket says 172.21.15.191:41050) 2024-03-04T09:26:09.275 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:26:09.276+0000 7f8d937fe700 1 -- 172.21.15.191:0/1669212869 learned_addr learned my addr 172.21.15.191:0/1669212869 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-03-04T09:26:09.275 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:26:09.276+0000 7f8d92ffd700 1 --2- 172.21.15.191:0/1669212869 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8d94103320 0x7f8d94198850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:26:09.275 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:26:09.276+0000 7f8d93fff700 1 --2- 172.21.15.191:0/1669212869 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8d94103cc0 0x7f8d9419f8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:26:09.275 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:26:09.276+0000 7f8d937fe700 1 -- 172.21.15.191:0/1669212869 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8d94103cc0 msgr2=0x7f8d9419f8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:26:09.275 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:26:09.276+0000 7f8d937fe700 1 --2- 172.21.15.191:0/1669212869 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8d94103cc0 0x7f8d9419f8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:26:09.276 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:26:09.276+0000 7f8d937fe700 1 -- 172.21.15.191:0/1669212869 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8d94103320 msgr2=0x7f8d94198850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:26:09.276 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:26:09.276+0000 7f8d937fe700 1 --2- 172.21.15.191:0/1669212869 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8d94103320 0x7f8d94198850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:26:09.276 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:26:09.276+0000 7f8d937fe700 1 -- 172.21.15.191:0/1669212869 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d9419ff70 con 0x7f8d94102120 2024-03-04T09:26:09.276 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:26:09.276+0000 7f8d93fff700 1 --2- 172.21.15.191:0/1669212869 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8d94103cc0 0x7f8d9419f8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-04T09:26:09.276 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:26:09.276+0000 7f8d937fe700 1 --2- 172.21.15.191:0/1669212869 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8d94102120 0x7f8d94198310 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f8d8800e980 tx=0x7f8d8800ec90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:26:09.276 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:26:09.276+0000 7f8d90ff9700 1 -- 172.21.15.191:0/1669212869 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d8800cbd0 con 0x7f8d94102120 2024-03-04T09:26:09.276 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:26:09.276+0000 7f8d90ff9700 1 -- 172.21.15.191:0/1669212869 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8d88004510 con 0x7f8d94102120 2024-03-04T09:26:09.276 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:26:09.276+0000 7f8d99fdf700 1 -- 172.21.15.191:0/1669212869 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d941a0260 con 0x7f8d94102120 2024-03-04T09:26:09.276 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:26:09.276+0000 7f8d90ff9700 1 -- 172.21.15.191:0/1669212869 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d88010430 con 0x7f8d94102120 2024-03-04T09:26:09.276 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:26:09.276+0000 7f8d99fdf700 1 -- 172.21.15.191:0/1669212869 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d941a0750 con 0x7f8d94102120 2024-03-04T09:26:09.277 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:26:09.277+0000 7f8d99fdf700 1 -- 172.21.15.191:0/1669212869 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d94068d80 con 0x7f8d94102120 2024-03-04T09:26:09.277 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:26:09.278+0000 7f8d90ff9700 1 -- 172.21.15.191:0/1669212869 <== mon.1 v2:172.21.15.191:3300/0 4 ==== mgrmap(e 16) v1 ==== 136756+0+0 (secure 0 0 0) 0x7f8d88003e90 con 0x7f8d94102120 2024-03-04T09:26:09.277 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:26:09.279+0000 7f8d90ff9700 1 --2- 172.21.15.191:0/1669212869 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f8d7c0a1e10 0x7f8d7c0a42d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:26:09.277 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:26:09.279+0000 7f8d90ff9700 1 -- 172.21.15.191:0/1669212869 <== mon.1 v2:172.21.15.191:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 5397+0+0 (secure 0 0 0) 0x7f8d880cd670 con 0x7f8d94102120 2024-03-04T09:26:09.277 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:26:09.279+0000 7f8d92ffd700 1 --2- 172.21.15.191:0/1669212869 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f8d7c0a1e10 0x7f8d7c0a42d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:26:09.277 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:26:09.279+0000 7f8d92ffd700 1 --2- 172.21.15.191:0/1669212869 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f8d7c0a1e10 0x7f8d7c0a42d0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f8d8400b2e0 tx=0x7f8d84005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:26:09.279 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:26:09.282+0000 7f8d90ff9700 1 -- 172.21.15.191:0/1669212869 <== mon.1 v2:172.21.15.191:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f8d8809b600 con 0x7f8d94102120 2024-03-04T09:26:09.485 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:26:09.488+0000 7f8d99fdf700 1 -- 172.21.15.191:0/1669212869 --> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi191:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7f8d94062500 con 0x7f8d7c0a1e10 2024-03-04T09:26:10.085 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:26:09 smithi052 ceph-mon[32473]: osdmap e55: 8 total, 8 up, 8 in 2024-03-04T09:26:10.085 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:26:09 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-04T09:26:10.085 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:26:09 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-04T09:26:10.085 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:26:09 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T09:26:10.086 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:26:09 smithi191 ceph-mon[36816]: osdmap e55: 8 total, 8 up, 8 in 2024-03-04T09:26:10.086 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:26:09 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-04T09:26:10.086 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:26:09 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-04T09:26:10.086 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:26:09 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T09:26:10.131 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:26:09 smithi106 ceph-mon[36484]: osdmap e55: 8 total, 8 up, 8 in 2024-03-04T09:26:10.131 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:26:09 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-04T09:26:10.131 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:26:09 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-04T09:26:10.131 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:26:09 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T09:26:11.084 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:26:10 smithi052 ceph-mon[32473]: pgmap v180: 1 pgs: 1 remapped+peering; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-03-04T09:26:11.085 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:26:10 smithi052 ceph-mon[32473]: from='client.24277 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi191:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T09:26:11.086 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:26:10 smithi191 ceph-mon[36816]: pgmap v180: 1 pgs: 1 remapped+peering; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-03-04T09:26:11.086 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:26:10 smithi191 ceph-mon[36816]: from='client.24277 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi191:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T09:26:11.131 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:26:10 smithi106 ceph-mon[36484]: pgmap v180: 1 pgs: 1 remapped+peering; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-03-04T09:26:11.131 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:26:10 smithi106 ceph-mon[36484]: from='client.24277 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi191:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T09:26:13.085 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:26:12 smithi052 ceph-mon[32473]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 230 MiB used, 715 GiB / 715 GiB avail 2024-03-04T09:26:13.085 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:26:12 smithi052 ceph-mon[32473]: from='client.? 172.21.15.191:0/4101503203' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6f5786c4-4402-435a-bdad-adff46fe9473"}]: dispatch 2024-03-04T09:26:13.085 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:26:12 smithi052 ceph-mon[32473]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6f5786c4-4402-435a-bdad-adff46fe9473"}]: dispatch 2024-03-04T09:26:13.085 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:26:12 smithi052 ceph-mon[32473]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6f5786c4-4402-435a-bdad-adff46fe9473"}]': finished 2024-03-04T09:26:13.085 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:26:12 smithi052 ceph-mon[32473]: osdmap e56: 9 total, 8 up, 9 in 2024-03-04T09:26:13.085 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:26:12 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-04T09:26:13.086 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:26:12 smithi191 ceph-mon[36816]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 230 MiB used, 715 GiB / 715 GiB avail 2024-03-04T09:26:13.086 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:26:12 smithi191 ceph-mon[36816]: from='client.? 172.21.15.191:0/4101503203' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6f5786c4-4402-435a-bdad-adff46fe9473"}]: dispatch 2024-03-04T09:26:13.086 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:26:12 smithi191 ceph-mon[36816]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6f5786c4-4402-435a-bdad-adff46fe9473"}]: dispatch 2024-03-04T09:26:13.086 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:26:12 smithi191 ceph-mon[36816]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6f5786c4-4402-435a-bdad-adff46fe9473"}]': finished 2024-03-04T09:26:13.086 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:26:12 smithi191 ceph-mon[36816]: osdmap e56: 9 total, 8 up, 9 in 2024-03-04T09:26:13.087 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:26:12 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-04T09:26:13.131 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:26:12 smithi106 ceph-mon[36484]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 230 MiB used, 715 GiB / 715 GiB avail 2024-03-04T09:26:13.131 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:26:12 smithi106 ceph-mon[36484]: from='client.? 172.21.15.191:0/4101503203' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6f5786c4-4402-435a-bdad-adff46fe9473"}]: dispatch 2024-03-04T09:26:13.131 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:26:12 smithi106 ceph-mon[36484]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6f5786c4-4402-435a-bdad-adff46fe9473"}]: dispatch 2024-03-04T09:26:13.131 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:26:12 smithi106 ceph-mon[36484]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6f5786c4-4402-435a-bdad-adff46fe9473"}]': finished 2024-03-04T09:26:13.131 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:26:12 smithi106 ceph-mon[36484]: osdmap e56: 9 total, 8 up, 9 in 2024-03-04T09:26:13.131 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:26:12 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-04T09:26:14.084 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:26:13 smithi052 ceph-mon[32473]: from='client.? 172.21.15.191:0/2322236084' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-04T09:26:14.086 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:26:13 smithi191 ceph-mon[36816]: from='client.? 172.21.15.191:0/2322236084' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-04T09:26:14.131 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:26:13 smithi106 ceph-mon[36484]: from='client.? 172.21.15.191:0/2322236084' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-04T09:26:15.085 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:26:14 smithi052 ceph-mon[32473]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-03-04T09:26:15.086 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:26:14 smithi191 ceph-mon[36816]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-03-04T09:26:15.131 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:26:14 smithi106 ceph-mon[36484]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-03-04T09:26:16.836 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:26:16 smithi191 ceph-mon[36816]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-03-04T09:26:17.084 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:26:16 smithi052 ceph-mon[32473]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-03-04T09:26:17.131 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:26:16 smithi106 ceph-mon[36484]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-03-04T09:26:19.084 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:26:18 smithi052 ceph-mon[32473]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-03-04T09:26:19.086 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:26:18 smithi191 ceph-mon[36816]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-03-04T09:26:19.131 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:26:18 smithi106 ceph-mon[36484]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-03-04T09:26:19.836 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:26:19 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-03-04T09:26:19.836 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:26:19 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T09:26:20.084 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:26:19 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-03-04T09:26:20.085 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:26:19 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T09:26:20.131 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:26:19 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-03-04T09:26:20.131 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:26:19 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T09:26:21.085 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:26:20 smithi052 ceph-mon[32473]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-03-04T09:26:21.085 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:26:20 smithi052 ceph-mon[32473]: Deploying daemon osd.8 on smithi191 2024-03-04T09:26:21.086 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:26:20 smithi191 ceph-mon[36816]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-03-04T09:26:21.086 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:26:20 smithi191 ceph-mon[36816]: Deploying daemon osd.8 on smithi191 2024-03-04T09:26:21.131 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:26:20 smithi106 ceph-mon[36484]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-03-04T09:26:21.131 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:26:20 smithi106 ceph-mon[36484]: Deploying daemon osd.8 on smithi191 2024-03-04T09:26:23.084 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:26:22 smithi052 ceph-mon[32473]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-03-04T09:26:23.086 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:26:22 smithi191 ceph-mon[36816]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-03-04T09:26:23.133 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:26:22 smithi106 ceph-mon[36484]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-03-04T09:26:25.084 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:26:24 smithi052 ceph-mon[32473]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-03-04T09:26:25.086 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:26:24 smithi191 ceph-mon[36816]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-03-04T09:26:25.131 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:26:24 smithi106 ceph-mon[36484]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-03-04T09:26:26.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:26:26 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:26:26.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:26:26 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:26:26.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:26:26 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T09:26:26.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:26:26 smithi191 ceph-mon[36816]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-03-04T09:26:26.631 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:26:26 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:26:26.631 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:26:26 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:26:26.631 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:26:26 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T09:26:26.631 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:26:26 smithi106 ceph-mon[36484]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-03-04T09:26:26.834 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:26:26 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:26:26.835 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:26:26 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:26:26.835 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:26:26 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T09:26:26.835 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:26:26 smithi052 ceph-mon[32473]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-03-04T09:26:28.551 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:26:28.555+0000 7f8d90ff9700 1 -- 172.21.15.191:0/1669212869 <== mgr.14152 v2:172.21.15.52:6800/3036033923 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f8d94062500 con 0x7f8d7c0a1e10 2024-03-04T09:26:28.552 INFO:teuthology.orchestra.run.smithi191.stdout:Created osd(s) 8 on host 'smithi191' 2024-03-04T09:26:28.554 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:26:28.558+0000 7f8d99fdf700 1 -- 172.21.15.191:0/1669212869 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f8d7c0a1e10 msgr2=0x7f8d7c0a42d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:26:28.554 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:26:28.558+0000 7f8d99fdf700 1 --2- 172.21.15.191:0/1669212869 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f8d7c0a1e10 0x7f8d7c0a42d0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f8d8400b2e0 tx=0x7f8d84005fd0 comp rx=0 tx=0).stop 2024-03-04T09:26:28.554 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:26:28.558+0000 7f8d99fdf700 1 -- 172.21.15.191:0/1669212869 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8d94102120 msgr2=0x7f8d94198310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:26:28.554 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:26:28.558+0000 7f8d99fdf700 1 --2- 172.21.15.191:0/1669212869 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8d94102120 0x7f8d94198310 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f8d8800e980 tx=0x7f8d8800ec90 comp rx=0 tx=0).stop 2024-03-04T09:26:28.554 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:26:28.558+0000 7f8d99fdf700 1 -- 172.21.15.191:0/1669212869 shutdown_connections 2024-03-04T09:26:28.554 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:26:28.558+0000 7f8d99fdf700 1 --2- 172.21.15.191:0/1669212869 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f8d7c0a1e10 0x7f8d7c0a42d0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:26:28.555 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:26:28.558+0000 7f8d99fdf700 1 --2- 172.21.15.191:0/1669212869 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8d94102120 0x7f8d94198310 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:26:28.555 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:26:28.558+0000 7f8d99fdf700 1 --2- 172.21.15.191:0/1669212869 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8d94103cc0 0x7f8d9419f8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:26:28.555 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:26:28.558+0000 7f8d99fdf700 1 --2- 172.21.15.191:0/1669212869 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8d94103320 0x7f8d94198850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:26:28.555 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:26:28.558+0000 7f8d99fdf700 1 -- 172.21.15.191:0/1669212869 >> 172.21.15.191:0/1669212869 conn(0x7f8d940fd6c0 msgr2=0x7f8d941065c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:26:28.555 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:26:28.558+0000 7f8d99fdf700 1 -- 172.21.15.191:0/1669212869 shutdown_connections 2024-03-04T09:26:28.555 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:26:28.558+0000 7f8d99fdf700 1 -- 172.21.15.191:0/1669212869 wait complete. 2024-03-04T09:26:28.835 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:26:28 smithi052 ceph-mon[32473]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-03-04T09:26:28.836 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:26:28 smithi191 ceph-mon[36816]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-03-04T09:26:28.881 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:26:28 smithi106 ceph-mon[36484]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-03-04T09:26:29.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:26:29 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:26:29.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:26:29 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:26:29.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:26:29 smithi191 ceph-mon[36816]: from='osd.8 [v2:172.21.15.191:6816/1114119072,v1:172.21.15.191:6817/1114119072]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-03-04T09:26:29.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:26:29 smithi191 ceph-mon[36816]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-03-04T09:26:29.659 DEBUG:teuthology.orchestra.run.smithi191:osd.8> sudo journalctl -f -n 0 -u ceph-4e059f08-da08-11ee-95c6-87774f69a715@osd.8.service 2024-03-04T09:26:29.661 INFO:tasks.cephadm:Deploying osd.9 on smithi052 with /dev/vg_nvme/lv_1... 2024-03-04T09:26:29.662 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4e059f08-da08-11ee-95c6-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-03-04T09:26:29.782 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:26:29 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:26:29.783 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:26:29 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:26:29.783 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:26:29 smithi106 ceph-mon[36484]: from='osd.8 [v2:172.21.15.191:6816/1114119072,v1:172.21.15.191:6817/1114119072]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-03-04T09:26:29.783 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:26:29 smithi106 ceph-mon[36484]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-03-04T09:26:29.834 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:26:29 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:26:29.835 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:26:29 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:26:29.835 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:26:29 smithi052 ceph-mon[32473]: from='osd.8 [v2:172.21.15.191:6816/1114119072,v1:172.21.15.191:6817/1114119072]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-03-04T09:26:29.835 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:26:29 smithi052 ceph-mon[32473]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-03-04T09:26:29.916 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/4e059f08-da08-11ee-95c6-87774f69a715/mon.a/config 2024-03-04T09:26:30.585 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:26:30 smithi052 ceph-mon[32473]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-03-04T09:26:30.585 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:26:30 smithi052 ceph-mon[32473]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-03-04T09:26:30.585 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:26:30 smithi052 ceph-mon[32473]: from='osd.8 [v2:172.21.15.191:6816/1114119072,v1:172.21.15.191:6817/1114119072]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi191", "root=default"]}]: dispatch 2024-03-04T09:26:30.585 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:26:30 smithi052 ceph-mon[32473]: osdmap e57: 9 total, 8 up, 9 in 2024-03-04T09:26:30.585 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:26:30 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-04T09:26:30.585 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:26:30 smithi052 ceph-mon[32473]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi191", "root=default"]}]: dispatch 2024-03-04T09:26:30.585 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:26:30 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:26:30.585 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:26:30 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:26:30.686 INFO:journalctl@ceph.osd.8.smithi191.stdout:Mar 04 09:26:30 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-8[55524]: 2024-03-04T09:26:30.548+0000 7f437098f700 -1 osd.8 0 waiting for initial osdmap 2024-03-04T09:26:30.686 INFO:journalctl@ceph.osd.8.smithi191.stdout:Mar 04 09:26:30 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-8[55524]: 2024-03-04T09:26:30.557+0000 7f436c782700 -1 osd.8 58 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-04T09:26:30.686 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:26:30 smithi191 ceph-mon[36816]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-03-04T09:26:30.686 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:26:30 smithi191 ceph-mon[36816]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-03-04T09:26:30.686 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:26:30 smithi191 ceph-mon[36816]: from='osd.8 [v2:172.21.15.191:6816/1114119072,v1:172.21.15.191:6817/1114119072]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi191", "root=default"]}]: dispatch 2024-03-04T09:26:30.686 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:26:30 smithi191 ceph-mon[36816]: osdmap e57: 9 total, 8 up, 9 in 2024-03-04T09:26:30.686 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:26:30 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-04T09:26:30.686 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:26:30 smithi191 ceph-mon[36816]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi191", "root=default"]}]: dispatch 2024-03-04T09:26:30.686 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:26:30 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:26:30.687 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:26:30 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:26:30.796 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:26:30 smithi106 ceph-mon[36484]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-03-04T09:26:30.796 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:26:30 smithi106 ceph-mon[36484]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-03-04T09:26:30.796 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:26:30 smithi106 ceph-mon[36484]: from='osd.8 [v2:172.21.15.191:6816/1114119072,v1:172.21.15.191:6817/1114119072]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi191", "root=default"]}]: dispatch 2024-03-04T09:26:30.796 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:26:30 smithi106 ceph-mon[36484]: osdmap e57: 9 total, 8 up, 9 in 2024-03-04T09:26:30.796 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:26:30 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-04T09:26:30.796 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:26:30 smithi106 ceph-mon[36484]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi191", "root=default"]}]: dispatch 2024-03-04T09:26:30.797 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:26:30 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:26:30.797 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:26:30 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:26:31.622 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:26:31 smithi052 ceph-mon[32473]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi191", "root=default"]}]': finished 2024-03-04T09:26:31.622 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:26:31 smithi052 ceph-mon[32473]: osdmap e58: 9 total, 8 up, 9 in 2024-03-04T09:26:31.622 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:26:31 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-04T09:26:31.622 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:26:31 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-04T09:26:31.623 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:26:31 smithi052 ceph-mon[32473]: osd.8 [v2:172.21.15.191:6816/1114119072,v1:172.21.15.191:6817/1114119072] boot 2024-03-04T09:26:31.623 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:26:31 smithi052 ceph-mon[32473]: osdmap e59: 9 total, 9 up, 9 in 2024-03-04T09:26:31.623 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:26:31 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-04T09:26:31.810 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:26:31 smithi191 ceph-mon[36816]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi191", "root=default"]}]': finished 2024-03-04T09:26:31.810 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:26:31 smithi191 ceph-mon[36816]: osdmap e58: 9 total, 8 up, 9 in 2024-03-04T09:26:31.810 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:26:31 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-04T09:26:31.810 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:26:31 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-04T09:26:31.810 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:26:31 smithi191 ceph-mon[36816]: osd.8 [v2:172.21.15.191:6816/1114119072,v1:172.21.15.191:6817/1114119072] boot 2024-03-04T09:26:31.810 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:26:31 smithi191 ceph-mon[36816]: osdmap e59: 9 total, 9 up, 9 in 2024-03-04T09:26:31.811 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:26:31 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-04T09:26:31.881 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:26:31 smithi106 ceph-mon[36484]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi191", "root=default"]}]': finished 2024-03-04T09:26:31.881 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:26:31 smithi106 ceph-mon[36484]: osdmap e58: 9 total, 8 up, 9 in 2024-03-04T09:26:31.882 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:26:31 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-04T09:26:31.882 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:26:31 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-04T09:26:31.882 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:26:31 smithi106 ceph-mon[36484]: osd.8 [v2:172.21.15.191:6816/1114119072,v1:172.21.15.191:6817/1114119072] boot 2024-03-04T09:26:31.882 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:26:31 smithi106 ceph-mon[36484]: osdmap e59: 9 total, 9 up, 9 in 2024-03-04T09:26:31.882 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:26:31 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-04T09:26:32.070 INFO:teuthology.orchestra.run.smithi052.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-03-04T09:26:32.070 INFO:teuthology.orchestra.run.smithi052.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-03-04T09:26:32.070 INFO:teuthology.orchestra.run.smithi052.stderr: stderr: 10+0 records in 2024-03-04T09:26:32.070 INFO:teuthology.orchestra.run.smithi052.stderr:10+0 records out 2024-03-04T09:26:32.070 INFO:teuthology.orchestra.run.smithi052.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.00849961 s, 1.2 GB/s 2024-03-04T09:26:32.070 INFO:teuthology.orchestra.run.smithi052.stderr:--> Zapping successful for: 2024-03-04T09:26:32.846 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4e059f08-da08-11ee-95c6-87774f69a715 -- ceph orch daemon add osd smithi052:vg_nvme/lv_1 2024-03-04T09:26:33.007 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:26:32 smithi191 ceph-mon[36816]: purged_snaps scrub starts 2024-03-04T09:26:33.007 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:26:32 smithi191 ceph-mon[36816]: purged_snaps scrub ok 2024-03-04T09:26:33.007 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:26:32 smithi191 ceph-mon[36816]: pgmap v195: 1 pgs: 1 peering; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-03-04T09:26:33.007 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:26:32 smithi191 ceph-mon[36816]: osdmap e60: 9 total, 9 up, 9 in 2024-03-04T09:26:33.084 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:26:32 smithi052 ceph-mon[32473]: purged_snaps scrub starts 2024-03-04T09:26:33.084 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:26:32 smithi052 ceph-mon[32473]: purged_snaps scrub ok 2024-03-04T09:26:33.085 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:26:32 smithi052 ceph-mon[32473]: pgmap v195: 1 pgs: 1 peering; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-03-04T09:26:33.085 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:26:32 smithi052 ceph-mon[32473]: osdmap e60: 9 total, 9 up, 9 in 2024-03-04T09:26:33.092 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/4e059f08-da08-11ee-95c6-87774f69a715/mon.a/config 2024-03-04T09:26:33.131 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:26:32 smithi106 ceph-mon[36484]: purged_snaps scrub starts 2024-03-04T09:26:33.131 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:26:32 smithi106 ceph-mon[36484]: purged_snaps scrub ok 2024-03-04T09:26:33.131 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:26:32 smithi106 ceph-mon[36484]: pgmap v195: 1 pgs: 1 peering; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-03-04T09:26:33.131 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:26:32 smithi106 ceph-mon[36484]: osdmap e60: 9 total, 9 up, 9 in 2024-03-04T09:26:34.085 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:26:33 smithi052 ceph-mon[32473]: osdmap e61: 9 total, 9 up, 9 in 2024-03-04T09:26:34.085 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:26:33 smithi052 ceph-mon[32473]: Detected new or changed devices on smithi191 2024-03-04T09:26:34.085 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:26:33 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:26:34.085 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:26:33 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:26:34.085 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:26:33 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-04T09:26:34.085 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:26:33 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-04T09:26:34.085 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:26:33 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-04T09:26:34.085 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:26:33 smithi052 ceph-mon[32473]: Adjusting osd_memory_target on smithi191 to 5693M 2024-03-04T09:26:34.085 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:26:33 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:26:34.085 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:26:33 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T09:26:34.085 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:26:33 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T09:26:34.085 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:26:33 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:26:34.086 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:26:33 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T09:26:34.086 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:26:33 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T09:26:34.086 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:26:33 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T09:26:34.086 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:26:33 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:26:34.086 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:26:33 smithi191 ceph-mon[36816]: osdmap e61: 9 total, 9 up, 9 in 2024-03-04T09:26:34.086 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:26:33 smithi191 ceph-mon[36816]: Detected new or changed devices on smithi191 2024-03-04T09:26:34.086 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:26:33 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:26:34.086 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:26:33 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:26:34.086 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:26:33 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-04T09:26:34.087 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:26:33 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-04T09:26:34.087 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:26:33 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-04T09:26:34.087 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:26:33 smithi191 ceph-mon[36816]: Adjusting osd_memory_target on smithi191 to 5693M 2024-03-04T09:26:34.087 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:26:33 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:26:34.087 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:26:33 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T09:26:34.087 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:26:33 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T09:26:34.087 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:26:33 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:26:34.087 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:26:33 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T09:26:34.087 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:26:33 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T09:26:34.087 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:26:33 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T09:26:34.087 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:26:33 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:26:34.131 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:26:33 smithi106 ceph-mon[36484]: osdmap e61: 9 total, 9 up, 9 in 2024-03-04T09:26:34.131 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:26:33 smithi106 ceph-mon[36484]: Detected new or changed devices on smithi191 2024-03-04T09:26:34.132 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:26:33 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:26:34.132 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:26:33 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:26:34.132 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:26:33 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-04T09:26:34.132 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:26:33 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-04T09:26:34.132 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:26:33 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-04T09:26:34.132 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:26:33 smithi106 ceph-mon[36484]: Adjusting osd_memory_target on smithi191 to 5693M 2024-03-04T09:26:34.132 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:26:33 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:26:34.132 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:26:33 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T09:26:34.132 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:26:33 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T09:26:34.132 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:26:33 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:26:34.132 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:26:33 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T09:26:34.132 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:26:33 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T09:26:34.132 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:26:33 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T09:26:34.132 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:26:33 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:26:34.937 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:26:34.938+0000 7f3424807700 1 -- 172.21.15.52:0/2088799595 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3420103c90 msgr2=0x7f342010a540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:26:34.937 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:26:34.938+0000 7f3424807700 1 --2- 172.21.15.52:0/2088799595 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3420103c90 0x7f342010a540 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f341800b0d0 tx=0x7f341800b3e0 comp rx=0 tx=0).stop 2024-03-04T09:26:34.937 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:26:34.938+0000 7f3424807700 1 -- 172.21.15.52:0/2088799595 shutdown_connections 2024-03-04T09:26:34.937 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:26:34.938+0000 7f3424807700 1 --2- 172.21.15.52:0/2088799595 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f34201020f0 0x7f3420102510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:26:34.937 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:26:34.938+0000 7f3424807700 1 --2- 172.21.15.52:0/2088799595 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3420103c90 0x7f342010a540 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:26:34.937 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:26:34.938+0000 7f3424807700 1 --2- 172.21.15.52:0/2088799595 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f34201032f0 0x7f3420103750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:26:34.937 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:26:34.938+0000 7f3424807700 1 -- 172.21.15.52:0/2088799595 >> 172.21.15.52:0/2088799595 conn(0x7f34200fd6d0 msgr2=0x7f34200ffaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:26:34.937 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:26:34.938+0000 7f3424807700 1 -- 172.21.15.52:0/2088799595 shutdown_connections 2024-03-04T09:26:34.938 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:26:34.938+0000 7f3424807700 1 -- 172.21.15.52:0/2088799595 wait complete. 2024-03-04T09:26:34.938 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:26:34.939+0000 7f3424807700 1 Processor -- start 2024-03-04T09:26:34.938 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:26:34.939+0000 7f3424807700 1 -- start start 2024-03-04T09:26:34.938 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:26:34.939+0000 7f3424807700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f34201020f0 0x7f3420198290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:26:34.938 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:26:34.939+0000 7f3424807700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f34201032f0 0x7f34201987d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:26:34.938 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:26:34.939+0000 7f3424807700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3420103c90 0x7f342019f850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:26:34.938 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:26:34.939+0000 7f3424807700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f342010dd60 con 0x7f34201032f0 2024-03-04T09:26:34.938 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:26:34.939+0000 7f3424807700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f342010dbe0 con 0x7f34201020f0 2024-03-04T09:26:34.938 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:26:34.939+0000 7f3424807700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f342010dee0 con 0x7f3420103c90 2024-03-04T09:26:34.938 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:26:34.939+0000 7f341dd9b700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f34201032f0 0x7f34201987d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:26:34.938 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:26:34.939+0000 7f341dd9b700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f34201032f0 0x7f34201987d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:54120/0 (socket says 172.21.15.52:54120) 2024-03-04T09:26:34.938 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:26:34.939+0000 7f341dd9b700 1 -- 172.21.15.52:0/1533333282 learned_addr learned my addr 172.21.15.52:0/1533333282 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-04T09:26:34.938 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:26:34.939+0000 7f341ed9d700 1 --2- 172.21.15.52:0/1533333282 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3420103c90 0x7f342019f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:26:34.938 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:26:34.939+0000 7f341e59c700 1 --2- 172.21.15.52:0/1533333282 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f34201020f0 0x7f3420198290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:26:34.939 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:26:34.940+0000 7f341dd9b700 1 -- 172.21.15.52:0/1533333282 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3420103c90 msgr2=0x7f342019f850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:26:34.939 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:26:34.940+0000 7f341dd9b700 1 --2- 172.21.15.52:0/1533333282 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3420103c90 0x7f342019f850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:26:34.939 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:26:34.940+0000 7f341dd9b700 1 -- 172.21.15.52:0/1533333282 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f34201020f0 msgr2=0x7f3420198290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:26:34.939 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:26:34.940+0000 7f341dd9b700 1 --2- 172.21.15.52:0/1533333282 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f34201020f0 0x7f3420198290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:26:34.939 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:26:34.940+0000 7f341dd9b700 1 -- 172.21.15.52:0/1533333282 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f342019fef0 con 0x7f34201032f0 2024-03-04T09:26:34.939 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:26:34.940+0000 7f341dd9b700 1 --2- 172.21.15.52:0/1533333282 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f34201032f0 0x7f34201987d0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f341400c9e0 tx=0x7f341400ccf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:26:34.939 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:26:34.940+0000 7f340f7fe700 1 -- 172.21.15.52:0/1533333282 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3414007ac0 con 0x7f34201032f0 2024-03-04T09:26:34.939 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:26:34.940+0000 7f340f7fe700 1 -- 172.21.15.52:0/1533333282 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3414007c60 con 0x7f34201032f0 2024-03-04T09:26:34.939 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:26:34.940+0000 7f3424807700 1 -- 172.21.15.52:0/1533333282 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34201a01e0 con 0x7f34201032f0 2024-03-04T09:26:34.939 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:26:34.940+0000 7f340f7fe700 1 -- 172.21.15.52:0/1533333282 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3414018660 con 0x7f34201032f0 2024-03-04T09:26:34.939 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:26:34.940+0000 7f3424807700 1 -- 172.21.15.52:0/1533333282 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34201a07a0 con 0x7f34201032f0 2024-03-04T09:26:34.940 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:26:34.941+0000 7f3424807700 1 -- 172.21.15.52:0/1533333282 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3420068d80 con 0x7f34201032f0 2024-03-04T09:26:34.940 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:26:34.942+0000 7f340f7fe700 1 -- 172.21.15.52:0/1533333282 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 16) v1 ==== 136756+0+0 (secure 0 0 0) 0x7f341400f450 con 0x7f34201032f0 2024-03-04T09:26:34.940 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:26:34.943+0000 7f340f7fe700 1 --2- 172.21.15.52:0/1533333282 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f34080a1e10 0x7f34080a42d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:26:34.941 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:26:34.943+0000 7f341e59c700 1 --2- 172.21.15.52:0/1533333282 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f34080a1e10 0x7f34080a42d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:26:34.941 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:26:34.943+0000 7f340f7fe700 1 -- 172.21.15.52:0/1533333282 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 5829+0+0 (secure 0 0 0) 0x7f34140cd3b0 con 0x7f34201032f0 2024-03-04T09:26:34.941 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:26:34.943+0000 7f341e59c700 1 --2- 172.21.15.52:0/1533333282 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f34080a1e10 0x7f34080a42d0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f3420103150 tx=0x7f3410009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:26:34.944 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:26:34.947+0000 7f340f7fe700 1 -- 172.21.15.52:0/1533333282 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f3414014030 con 0x7f34201032f0 2024-03-04T09:26:35.158 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:26:35.161+0000 7f3424807700 1 -- 172.21.15.52:0/1533333282 --> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi052:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7f3420062500 con 0x7f34080a1e10 2024-03-04T09:26:35.334 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:26:35 smithi052 ceph-mon[32473]: pgmap v198: 1 pgs: 1 peering; 577 KiB data, 256 MiB used, 804 GiB / 805 GiB avail 2024-03-04T09:26:35.334 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:26:35 smithi052 ceph-mon[32473]: osdmap e62: 9 total, 9 up, 9 in 2024-03-04T09:26:35.381 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:26:35 smithi106 ceph-mon[36484]: pgmap v198: 1 pgs: 1 peering; 577 KiB data, 256 MiB used, 804 GiB / 805 GiB avail 2024-03-04T09:26:35.381 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:26:35 smithi106 ceph-mon[36484]: osdmap e62: 9 total, 9 up, 9 in 2024-03-04T09:26:35.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:26:35 smithi191 ceph-mon[36816]: pgmap v198: 1 pgs: 1 peering; 577 KiB data, 256 MiB used, 804 GiB / 805 GiB avail 2024-03-04T09:26:35.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:26:35 smithi191 ceph-mon[36816]: osdmap e62: 9 total, 9 up, 9 in 2024-03-04T09:26:36.381 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:26:36 smithi106 ceph-mon[36484]: osdmap e63: 9 total, 9 up, 9 in 2024-03-04T09:26:36.381 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:26:36 smithi106 ceph-mon[36484]: from='client.14415 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi052:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T09:26:36.382 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:26:36 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-04T09:26:36.382 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:26:36 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-04T09:26:36.382 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:26:36 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T09:26:36.585 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:26:36 smithi052 ceph-mon[32473]: osdmap e63: 9 total, 9 up, 9 in 2024-03-04T09:26:36.585 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:26:36 smithi052 ceph-mon[32473]: from='client.14415 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi052:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T09:26:36.585 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:26:36 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-04T09:26:36.585 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:26:36 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-04T09:26:36.585 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:26:36 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T09:26:36.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:26:36 smithi191 ceph-mon[36816]: osdmap e63: 9 total, 9 up, 9 in 2024-03-04T09:26:36.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:26:36 smithi191 ceph-mon[36816]: from='client.14415 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi052:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T09:26:36.587 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:26:36 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-04T09:26:36.587 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:26:36 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-04T09:26:36.587 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:26:36 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T09:26:37.377 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:26:37 smithi052 ceph-mon[32473]: pgmap v201: 1 pgs: 1 peering; 577 KiB data, 256 MiB used, 804 GiB / 805 GiB avail 2024-03-04T09:26:37.381 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:26:37 smithi106 ceph-mon[36484]: pgmap v201: 1 pgs: 1 peering; 577 KiB data, 256 MiB used, 804 GiB / 805 GiB avail 2024-03-04T09:26:37.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:26:37 smithi191 ceph-mon[36816]: pgmap v201: 1 pgs: 1 peering; 577 KiB data, 256 MiB used, 804 GiB / 805 GiB avail 2024-03-04T09:26:38.371 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:26:38 smithi052 ceph-mon[32473]: from='client.? 172.21.15.52:0/1472877343' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "062bbaa5-1227-4684-b859-450f5594a90c"}]: dispatch 2024-03-04T09:26:38.371 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:26:38 smithi052 ceph-mon[32473]: from='client.? 172.21.15.52:0/1472877343' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "062bbaa5-1227-4684-b859-450f5594a90c"}]': finished 2024-03-04T09:26:38.371 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:26:38 smithi052 ceph-mon[32473]: osdmap e64: 10 total, 9 up, 10 in 2024-03-04T09:26:38.371 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:26:38 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-04T09:26:38.381 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:26:38 smithi106 ceph-mon[36484]: from='client.? 172.21.15.52:0/1472877343' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "062bbaa5-1227-4684-b859-450f5594a90c"}]: dispatch 2024-03-04T09:26:38.381 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:26:38 smithi106 ceph-mon[36484]: from='client.? 172.21.15.52:0/1472877343' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "062bbaa5-1227-4684-b859-450f5594a90c"}]': finished 2024-03-04T09:26:38.381 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:26:38 smithi106 ceph-mon[36484]: osdmap e64: 10 total, 9 up, 10 in 2024-03-04T09:26:38.381 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:26:38 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-04T09:26:38.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:26:38 smithi191 ceph-mon[36816]: from='client.? 172.21.15.52:0/1472877343' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "062bbaa5-1227-4684-b859-450f5594a90c"}]: dispatch 2024-03-04T09:26:38.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:26:38 smithi191 ceph-mon[36816]: from='client.? 172.21.15.52:0/1472877343' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "062bbaa5-1227-4684-b859-450f5594a90c"}]': finished 2024-03-04T09:26:38.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:26:38 smithi191 ceph-mon[36816]: osdmap e64: 10 total, 9 up, 10 in 2024-03-04T09:26:38.587 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:26:38 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-04T09:26:39.381 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:26:39 smithi106 ceph-mon[36484]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail; 102 KiB/s, 0 objects/s recovering 2024-03-04T09:26:39.381 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:26:39 smithi106 ceph-mon[36484]: from='client.? 172.21.15.52:0/450140574' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-04T09:26:39.584 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:26:39 smithi052 ceph-mon[32473]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail; 102 KiB/s, 0 objects/s recovering 2024-03-04T09:26:39.585 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:26:39 smithi052 ceph-mon[32473]: from='client.? 172.21.15.52:0/450140574' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-04T09:26:39.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:26:39 smithi191 ceph-mon[36816]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail; 102 KiB/s, 0 objects/s recovering 2024-03-04T09:26:39.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:26:39 smithi191 ceph-mon[36816]: from='client.? 172.21.15.52:0/450140574' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-04T09:26:41.381 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:26:41 smithi106 ceph-mon[36484]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-03-04T09:26:41.584 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:26:41 smithi052 ceph-mon[32473]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-03-04T09:26:41.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:26:41 smithi191 ceph-mon[36816]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-03-04T09:26:43.381 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:26:43 smithi106 ceph-mon[36484]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail; 78 KiB/s, 0 objects/s recovering 2024-03-04T09:26:43.405 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:26:43 smithi052 ceph-mon[32473]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail; 78 KiB/s, 0 objects/s recovering 2024-03-04T09:26:43.587 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:26:43 smithi191 ceph-mon[36816]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail; 78 KiB/s, 0 objects/s recovering 2024-03-04T09:26:45.335 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:26:45 smithi052 ceph-mon[32473]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail; 69 KiB/s, 0 objects/s recovering 2024-03-04T09:26:45.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:26:45 smithi191 ceph-mon[36816]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail; 69 KiB/s, 0 objects/s recovering 2024-03-04T09:26:45.631 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:26:45 smithi106 ceph-mon[36484]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail; 69 KiB/s, 0 objects/s recovering 2024-03-04T09:26:46.413 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:26:46 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-03-04T09:26:46.413 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:26:46 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T09:26:46.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:26:46 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-03-04T09:26:46.587 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:26:46 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T09:26:46.631 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:26:46 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-03-04T09:26:46.631 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:26:46 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T09:26:47.447 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:26:47 smithi052 ceph-mon[32473]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-03-04T09:26:47.448 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:26:47 smithi052 ceph-mon[32473]: Deploying daemon osd.9 on smithi052 2024-03-04T09:26:47.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:26:47 smithi191 ceph-mon[36816]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-03-04T09:26:47.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:26:47 smithi191 ceph-mon[36816]: Deploying daemon osd.9 on smithi052 2024-03-04T09:26:47.631 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:26:47 smithi106 ceph-mon[36484]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-03-04T09:26:47.631 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:26:47 smithi106 ceph-mon[36484]: Deploying daemon osd.9 on smithi052 2024-03-04T09:26:49.271 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:26:49 smithi052 ceph-mon[32473]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail 2024-03-04T09:26:49.587 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:26:49 smithi191 ceph-mon[36816]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail 2024-03-04T09:26:49.631 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:26:49 smithi106 ceph-mon[36484]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail 2024-03-04T09:26:51.335 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:26:51 smithi052 ceph-mon[32473]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail 2024-03-04T09:26:51.404 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:26:51 smithi191 ceph-mon[36816]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail 2024-03-04T09:26:51.631 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:26:51 smithi106 ceph-mon[36484]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail 2024-03-04T09:26:52.836 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:26:52 smithi191 ceph-mon[36816]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail 2024-03-04T09:26:52.836 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:26:52 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:26:52.836 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:26:52 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:26:52.837 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:26:52 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T09:26:53.058 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:26:52 smithi052 ceph-mon[32473]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail 2024-03-04T09:26:53.058 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:26:52 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:26:53.058 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:26:52 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:26:53.058 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:26:52 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T09:26:53.131 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:26:52 smithi106 ceph-mon[36484]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail 2024-03-04T09:26:53.131 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:26:52 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:26:53.131 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:26:52 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:26:53.131 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:26:52 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T09:26:54.835 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:26:54 smithi052 ceph-mon[32473]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail 2024-03-04T09:26:55.086 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:26:54 smithi191 ceph-mon[36816]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail 2024-03-04T09:26:55.131 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:26:54 smithi106 ceph-mon[36484]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail 2024-03-04T09:26:55.669 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:26:55.671+0000 7f340f7fe700 1 -- 172.21.15.52:0/1533333282 <== mgr.14152 v2:172.21.15.52:6800/3036033923 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f3420062500 con 0x7f34080a1e10 2024-03-04T09:26:55.669 INFO:teuthology.orchestra.run.smithi052.stdout:Created osd(s) 9 on host 'smithi052' 2024-03-04T09:26:55.671 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:26:55.674+0000 7f3424807700 1 -- 172.21.15.52:0/1533333282 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f34080a1e10 msgr2=0x7f34080a42d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:26:55.671 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:26:55.674+0000 7f3424807700 1 --2- 172.21.15.52:0/1533333282 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f34080a1e10 0x7f34080a42d0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f3420103150 tx=0x7f3410009190 comp rx=0 tx=0).stop 2024-03-04T09:26:55.671 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:26:55.674+0000 7f3424807700 1 -- 172.21.15.52:0/1533333282 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f34201032f0 msgr2=0x7f34201987d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:26:55.671 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:26:55.674+0000 7f3424807700 1 --2- 172.21.15.52:0/1533333282 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f34201032f0 0x7f34201987d0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f341400c9e0 tx=0x7f341400ccf0 comp rx=0 tx=0).stop 2024-03-04T09:26:55.671 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:26:55.674+0000 7f3424807700 1 -- 172.21.15.52:0/1533333282 shutdown_connections 2024-03-04T09:26:55.671 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:26:55.674+0000 7f3424807700 1 --2- 172.21.15.52:0/1533333282 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f34080a1e10 0x7f34080a42d0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:26:55.671 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:26:55.674+0000 7f3424807700 1 --2- 172.21.15.52:0/1533333282 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3420103c90 0x7f342019f850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:26:55.671 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:26:55.674+0000 7f3424807700 1 --2- 172.21.15.52:0/1533333282 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f34201020f0 0x7f3420198290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:26:55.671 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:26:55.674+0000 7f3424807700 1 --2- 172.21.15.52:0/1533333282 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f34201032f0 0x7f34201987d0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:26:55.671 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:26:55.674+0000 7f3424807700 1 -- 172.21.15.52:0/1533333282 >> 172.21.15.52:0/1533333282 conn(0x7f34200fd6d0 msgr2=0x7f3420106590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:26:55.671 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:26:55.674+0000 7f3424807700 1 -- 172.21.15.52:0/1533333282 shutdown_connections 2024-03-04T09:26:55.672 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:26:55.674+0000 7f3424807700 1 -- 172.21.15.52:0/1533333282 wait complete. 2024-03-04T09:26:55.851 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:26:55 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:26:55.851 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:26:55 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:26:55.851 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:26:55 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T09:26:55.851 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:26:55 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T09:26:55.851 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:26:55 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:26:55.852 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:26:55 smithi052 ceph-mon[32473]: from='osd.9 [v2:172.21.15.52:6826/3485821914,v1:172.21.15.52:6827/3485821914]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-03-04T09:26:55.852 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:26:55 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:26:55.852 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:26:55 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:26:55.852 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:26:55 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T09:26:56.132 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:26:55 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:26:56.132 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:26:55 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:26:56.133 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:26:55 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T09:26:56.133 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:26:55 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T09:26:56.133 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:26:55 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:26:56.133 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:26:55 smithi106 ceph-mon[36484]: from='osd.9 [v2:172.21.15.52:6826/3485821914,v1:172.21.15.52:6827/3485821914]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-03-04T09:26:56.133 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:26:55 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:26:56.133 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:26:55 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:26:56.133 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:26:55 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T09:26:56.299 DEBUG:teuthology.orchestra.run.smithi052:osd.9> sudo journalctl -f -n 0 -u ceph-4e059f08-da08-11ee-95c6-87774f69a715@osd.9.service 2024-03-04T09:26:56.302 INFO:tasks.cephadm:Deploying osd.10 on smithi106 with /dev/vg_nvme/lv_1... 2024-03-04T09:26:56.302 DEBUG:teuthology.orchestra.run.smithi106:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4e059f08-da08-11ee-95c6-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-03-04T09:26:56.336 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:26:55 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:26:56.337 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:26:55 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:26:56.337 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:26:55 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T09:26:56.337 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:26:55 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T09:26:56.337 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:26:55 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:26:56.337 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:26:55 smithi191 ceph-mon[36816]: from='osd.9 [v2:172.21.15.52:6826/3485821914,v1:172.21.15.52:6827/3485821914]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-03-04T09:26:56.337 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:26:55 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:26:56.337 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:26:55 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:26:56.337 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:26:55 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T09:26:56.547 INFO:teuthology.orchestra.run.smithi106.stderr:Inferring config /var/lib/ceph/4e059f08-da08-11ee-95c6-87774f69a715/mon.b/config 2024-03-04T09:26:56.949 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:26:56 smithi106 ceph-mon[36484]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail 2024-03-04T09:26:56.949 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:26:56 smithi106 ceph-mon[36484]: from='osd.9 [v2:172.21.15.52:6826/3485821914,v1:172.21.15.52:6827/3485821914]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-03-04T09:26:56.949 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:26:56 smithi106 ceph-mon[36484]: osdmap e65: 10 total, 9 up, 10 in 2024-03-04T09:26:56.950 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:26:56 smithi106 ceph-mon[36484]: from='osd.9 [v2:172.21.15.52:6826/3485821914,v1:172.21.15.52:6827/3485821914]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi052", "root=default"]}]: dispatch 2024-03-04T09:26:56.950 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:26:56 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-04T09:26:57.085 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:26:56 smithi052 ceph-mon[32473]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail 2024-03-04T09:26:57.085 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:26:56 smithi052 ceph-mon[32473]: from='osd.9 [v2:172.21.15.52:6826/3485821914,v1:172.21.15.52:6827/3485821914]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-03-04T09:26:57.085 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:26:56 smithi052 ceph-mon[32473]: osdmap e65: 10 total, 9 up, 10 in 2024-03-04T09:26:57.085 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:26:56 smithi052 ceph-mon[32473]: from='osd.9 [v2:172.21.15.52:6826/3485821914,v1:172.21.15.52:6827/3485821914]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi052", "root=default"]}]: dispatch 2024-03-04T09:26:57.085 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:26:56 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-04T09:26:57.336 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:26:56 smithi191 ceph-mon[36816]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail 2024-03-04T09:26:57.336 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:26:56 smithi191 ceph-mon[36816]: from='osd.9 [v2:172.21.15.52:6826/3485821914,v1:172.21.15.52:6827/3485821914]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-03-04T09:26:57.336 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:26:56 smithi191 ceph-mon[36816]: osdmap e65: 10 total, 9 up, 10 in 2024-03-04T09:26:57.337 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:26:56 smithi191 ceph-mon[36816]: from='osd.9 [v2:172.21.15.52:6826/3485821914,v1:172.21.15.52:6827/3485821914]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi052", "root=default"]}]: dispatch 2024-03-04T09:26:57.337 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:26:56 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-04T09:26:57.714 INFO:journalctl@ceph.osd.9.smithi052.stdout:Mar 04 09:26:57 smithi052 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-9[65396]: 2024-03-04T09:26:57.470+0000 7fd1f88a0700 -1 osd.9 0 waiting for initial osdmap 2024-03-04T09:26:57.714 INFO:journalctl@ceph.osd.9.smithi052.stdout:Mar 04 09:26:57 smithi052 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-9[65396]: 2024-03-04T09:26:57.484+0000 7fd1f068b700 -1 osd.9 66 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-04T09:26:58.057 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:26:57 smithi106 ceph-mon[36484]: from='osd.9 [v2:172.21.15.52:6826/3485821914,v1:172.21.15.52:6827/3485821914]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi052", "root=default"]}]': finished 2024-03-04T09:26:58.057 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:26:57 smithi106 ceph-mon[36484]: osdmap e66: 10 total, 9 up, 10 in 2024-03-04T09:26:58.057 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:26:57 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-04T09:26:58.057 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:26:57 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-04T09:26:58.058 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:26:57 smithi106 ceph-mon[36484]: from='osd.9 [v2:172.21.15.52:6826/3485821914,v1:172.21.15.52:6827/3485821914]' entity='osd.9' 2024-03-04T09:26:58.058 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:26:57 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-04T09:26:58.085 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:26:57 smithi052 ceph-mon[32473]: from='osd.9 [v2:172.21.15.52:6826/3485821914,v1:172.21.15.52:6827/3485821914]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi052", "root=default"]}]': finished 2024-03-04T09:26:58.085 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:26:57 smithi052 ceph-mon[32473]: osdmap e66: 10 total, 9 up, 10 in 2024-03-04T09:26:58.085 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:26:57 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-04T09:26:58.085 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:26:57 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-04T09:26:58.085 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:26:57 smithi052 ceph-mon[32473]: from='osd.9 [v2:172.21.15.52:6826/3485821914,v1:172.21.15.52:6827/3485821914]' entity='osd.9' 2024-03-04T09:26:58.085 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:26:57 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-04T09:26:58.336 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:26:57 smithi191 ceph-mon[36816]: from='osd.9 [v2:172.21.15.52:6826/3485821914,v1:172.21.15.52:6827/3485821914]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi052", "root=default"]}]': finished 2024-03-04T09:26:58.336 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:26:57 smithi191 ceph-mon[36816]: osdmap e66: 10 total, 9 up, 10 in 2024-03-04T09:26:58.337 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:26:57 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-04T09:26:58.337 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:26:57 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-04T09:26:58.337 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:26:57 smithi191 ceph-mon[36816]: from='osd.9 [v2:172.21.15.52:6826/3485821914,v1:172.21.15.52:6827/3485821914]' entity='osd.9' 2024-03-04T09:26:58.337 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:26:57 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-04T09:26:58.637 INFO:teuthology.orchestra.run.smithi106.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-03-04T09:26:58.638 INFO:teuthology.orchestra.run.smithi106.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-03-04T09:26:58.638 INFO:teuthology.orchestra.run.smithi106.stderr: stderr: 10+0 records in 2024-03-04T09:26:58.638 INFO:teuthology.orchestra.run.smithi106.stderr:10+0 records out 2024-03-04T09:26:58.638 INFO:teuthology.orchestra.run.smithi106.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0114135 s, 919 MB/s 2024-03-04T09:26:58.638 INFO:teuthology.orchestra.run.smithi106.stderr:--> Zapping successful for: 2024-03-04T09:26:58.887 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:26:58 smithi106 ceph-mon[36484]: purged_snaps scrub starts 2024-03-04T09:26:58.888 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:26:58 smithi106 ceph-mon[36484]: purged_snaps scrub ok 2024-03-04T09:26:58.888 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:26:58 smithi106 ceph-mon[36484]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail 2024-03-04T09:26:58.888 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:26:58 smithi106 ceph-mon[36484]: osd.9 [v2:172.21.15.52:6826/3485821914,v1:172.21.15.52:6827/3485821914] boot 2024-03-04T09:26:58.888 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:26:58 smithi106 ceph-mon[36484]: osdmap e67: 10 total, 10 up, 10 in 2024-03-04T09:26:59.304 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:26:58 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-04T09:26:59.335 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:26:58 smithi052 ceph-mon[32473]: purged_snaps scrub starts 2024-03-04T09:26:59.335 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:26:58 smithi052 ceph-mon[32473]: purged_snaps scrub ok 2024-03-04T09:26:59.335 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:26:58 smithi052 ceph-mon[32473]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail 2024-03-04T09:26:59.335 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:26:58 smithi052 ceph-mon[32473]: osd.9 [v2:172.21.15.52:6826/3485821914,v1:172.21.15.52:6827/3485821914] boot 2024-03-04T09:26:59.335 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:26:58 smithi052 ceph-mon[32473]: osdmap e67: 10 total, 10 up, 10 in 2024-03-04T09:26:59.335 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:26:58 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-04T09:26:59.336 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:26:58 smithi191 ceph-mon[36816]: purged_snaps scrub starts 2024-03-04T09:26:59.336 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:26:58 smithi191 ceph-mon[36816]: purged_snaps scrub ok 2024-03-04T09:26:59.336 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:26:58 smithi191 ceph-mon[36816]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail 2024-03-04T09:26:59.336 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:26:58 smithi191 ceph-mon[36816]: osd.9 [v2:172.21.15.52:6826/3485821914,v1:172.21.15.52:6827/3485821914] boot 2024-03-04T09:26:59.337 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:26:58 smithi191 ceph-mon[36816]: osdmap e67: 10 total, 10 up, 10 in 2024-03-04T09:26:59.337 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:26:58 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-04T09:26:59.339 DEBUG:teuthology.orchestra.run.smithi106:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4e059f08-da08-11ee-95c6-87774f69a715 -- ceph orch daemon add osd smithi106:vg_nvme/lv_1 2024-03-04T09:26:59.583 INFO:teuthology.orchestra.run.smithi106.stderr:Inferring config /var/lib/ceph/4e059f08-da08-11ee-95c6-87774f69a715/mon.b/config 2024-03-04T09:27:00.720 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:27:00 smithi106 ceph-mon[36484]: Detected new or changed devices on smithi052 2024-03-04T09:27:00.720 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:27:00 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:27:00.720 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:27:00 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:27:00.721 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:27:00 smithi106 ceph-mon[36484]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-03-04T09:27:00.721 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:27:00 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-04T09:27:00.721 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:27:00 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-04T09:27:00.721 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:27:00 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-04T09:27:00.721 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:27:00 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-04T09:27:00.721 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:27:00 smithi106 ceph-mon[36484]: Adjusting osd_memory_target on smithi052 to 4273M 2024-03-04T09:27:00.721 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:27:00 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:27:00.721 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:27:00 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T09:27:00.721 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:27:00 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T09:27:00.721 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:27:00 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:27:00.721 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:27:00 smithi106 ceph-mon[36484]: osdmap e68: 10 total, 10 up, 10 in 2024-03-04T09:27:00.835 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:27:00 smithi052 ceph-mon[32473]: Detected new or changed devices on smithi052 2024-03-04T09:27:00.835 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:27:00 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:27:00.835 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:27:00 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:27:00.835 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:27:00 smithi052 ceph-mon[32473]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-03-04T09:27:00.835 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:27:00 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-04T09:27:00.835 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:27:00 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-04T09:27:00.835 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:27:00 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-04T09:27:00.835 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:27:00 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-04T09:27:00.835 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:27:00 smithi052 ceph-mon[32473]: Adjusting osd_memory_target on smithi052 to 4273M 2024-03-04T09:27:00.835 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:27:00 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:27:00.836 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:27:00 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T09:27:00.836 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:27:00 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T09:27:00.836 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:27:00 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:27:00.836 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:27:00 smithi052 ceph-mon[32473]: osdmap e68: 10 total, 10 up, 10 in 2024-03-04T09:27:00.836 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:27:00 smithi191 ceph-mon[36816]: Detected new or changed devices on smithi052 2024-03-04T09:27:00.836 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:27:00 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:27:00.836 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:27:00 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:27:00.836 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:27:00 smithi191 ceph-mon[36816]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-03-04T09:27:00.836 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:27:00 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-04T09:27:00.836 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:27:00 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-04T09:27:00.837 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:27:00 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-04T09:27:00.837 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:27:00 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-04T09:27:00.837 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:27:00 smithi191 ceph-mon[36816]: Adjusting osd_memory_target on smithi052 to 4273M 2024-03-04T09:27:00.837 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:27:00 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:27:00.837 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:27:00 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T09:27:00.837 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:27:00 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T09:27:00.837 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:27:00 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:27:00.837 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:27:00 smithi191 ceph-mon[36816]: osdmap e68: 10 total, 10 up, 10 in 2024-03-04T09:27:01.275 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-04T09:27:01.276+0000 7f8e6a9de700 1 -- 172.21.15.106:0/1941471389 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8e64100420 msgr2=0x7f8e64100840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:27:01.275 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-04T09:27:01.276+0000 7f8e6a9de700 1 --2- 172.21.15.106:0/1941471389 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8e64100420 0x7f8e64100840 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f8e540099c0 tx=0x7f8e54009cd0 comp rx=0 tx=0).stop 2024-03-04T09:27:01.275 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-04T09:27:01.276+0000 7f8e6a9de700 1 -- 172.21.15.106:0/1941471389 shutdown_connections 2024-03-04T09:27:01.275 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-04T09:27:01.276+0000 7f8e6a9de700 1 --2- 172.21.15.106:0/1941471389 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8e64103e40 0x7f8e6410a6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:27:01.275 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-04T09:27:01.276+0000 7f8e6a9de700 1 --2- 172.21.15.106:0/1941471389 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8e64100420 0x7f8e64100840 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:27:01.276 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-04T09:27:01.276+0000 7f8e6a9de700 1 --2- 172.21.15.106:0/1941471389 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8e641034a0 0x7f8e64103900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:27:01.276 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-04T09:27:01.276+0000 7f8e6a9de700 1 -- 172.21.15.106:0/1941471389 >> 172.21.15.106:0/1941471389 conn(0x7f8e640fbd60 msgr2=0x7f8e640fe1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:27:01.276 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-04T09:27:01.276+0000 7f8e6a9de700 1 -- 172.21.15.106:0/1941471389 shutdown_connections 2024-03-04T09:27:01.276 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-04T09:27:01.276+0000 7f8e6a9de700 1 -- 172.21.15.106:0/1941471389 wait complete. 2024-03-04T09:27:01.276 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-04T09:27:01.276+0000 7f8e6a9de700 1 Processor -- start 2024-03-04T09:27:01.276 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-04T09:27:01.276+0000 7f8e6a9de700 1 -- start start 2024-03-04T09:27:01.276 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-04T09:27:01.277+0000 7f8e6a9de700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8e64100420 0x7f8e64198390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:27:01.276 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-04T09:27:01.277+0000 7f8e6a9de700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8e641034a0 0x7f8e641988d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:27:01.276 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-04T09:27:01.277+0000 7f8e6a9de700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8e64103e40 0x7f8e6419f950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:27:01.276 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-04T09:27:01.277+0000 7f8e6a9de700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e6410dd80 con 0x7f8e64103e40 2024-03-04T09:27:01.277 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-04T09:27:01.277+0000 7f8e6a9de700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e6410dc00 con 0x7f8e64100420 2024-03-04T09:27:01.277 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-04T09:27:01.277+0000 7f8e6a9de700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e6410df00 con 0x7f8e641034a0 2024-03-04T09:27:01.277 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-04T09:27:01.277+0000 7f8e63fff700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8e64100420 0x7f8e64198390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:27:01.277 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-04T09:27:01.277+0000 7f8e63fff700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8e64100420 0x7f8e64198390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:47534/0 (socket says 172.21.15.106:47534) 2024-03-04T09:27:01.277 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-04T09:27:01.277+0000 7f8e63fff700 1 -- 172.21.15.106:0/227902023 learned_addr learned my addr 172.21.15.106:0/227902023 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-03-04T09:27:01.277 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-04T09:27:01.277+0000 7f8e68f7b700 1 --2- 172.21.15.106:0/227902023 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8e64103e40 0x7f8e6419f950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:27:01.277 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-04T09:27:01.277+0000 7f8e637fe700 1 --2- 172.21.15.106:0/227902023 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8e641034a0 0x7f8e641988d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:27:01.277 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-04T09:27:01.277+0000 7f8e63fff700 1 -- 172.21.15.106:0/227902023 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8e641034a0 msgr2=0x7f8e641988d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:27:01.277 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-04T09:27:01.277+0000 7f8e63fff700 1 --2- 172.21.15.106:0/227902023 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8e641034a0 0x7f8e641988d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:27:01.277 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-04T09:27:01.277+0000 7f8e63fff700 1 -- 172.21.15.106:0/227902023 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8e64103e40 msgr2=0x7f8e6419f950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:27:01.277 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-04T09:27:01.277+0000 7f8e63fff700 1 --2- 172.21.15.106:0/227902023 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8e64103e40 0x7f8e6419f950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:27:01.277 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-04T09:27:01.277+0000 7f8e63fff700 1 -- 172.21.15.106:0/227902023 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e6419fff0 con 0x7f8e64100420 2024-03-04T09:27:01.277 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-04T09:27:01.277+0000 7f8e63fff700 1 --2- 172.21.15.106:0/227902023 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8e64100420 0x7f8e64198390 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f8e5400c010 tx=0x7f8e54005e80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:27:01.277 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-04T09:27:01.277+0000 7f8e617fa700 1 -- 172.21.15.106:0/227902023 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e54027070 con 0x7f8e64100420 2024-03-04T09:27:01.278 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-04T09:27:01.277+0000 7f8e617fa700 1 -- 172.21.15.106:0/227902023 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8e540041d0 con 0x7f8e64100420 2024-03-04T09:27:01.278 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-04T09:27:01.278+0000 7f8e6a9de700 1 -- 172.21.15.106:0/227902023 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e641a0280 con 0x7f8e64100420 2024-03-04T09:27:01.278 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-04T09:27:01.278+0000 7f8e617fa700 1 -- 172.21.15.106:0/227902023 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e5400f460 con 0x7f8e64100420 2024-03-04T09:27:01.278 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-04T09:27:01.278+0000 7f8e6a9de700 1 -- 172.21.15.106:0/227902023 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e641a0830 con 0x7f8e64100420 2024-03-04T09:27:01.279 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-04T09:27:01.279+0000 7f8e6a9de700 1 -- 172.21.15.106:0/227902023 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8e64068d80 con 0x7f8e64100420 2024-03-04T09:27:01.279 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-04T09:27:01.280+0000 7f8e617fa700 1 -- 172.21.15.106:0/227902023 <== mon.2 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 16) v1 ==== 136756+0+0 (secure 0 0 0) 0x7f8e5402a070 con 0x7f8e64100420 2024-03-04T09:27:01.279 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-04T09:27:01.280+0000 7f8e617fa700 1 --2- 172.21.15.106:0/227902023 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f8e4c0a1e90 0x7f8e4c0a4350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:27:01.279 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-04T09:27:01.280+0000 7f8e617fa700 1 -- 172.21.15.106:0/227902023 <== mon.2 v2:172.21.15.106:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6261+0+0 (secure 0 0 0) 0x7f8e540ce5a0 con 0x7f8e64100420 2024-03-04T09:27:01.279 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-04T09:27:01.280+0000 7f8e637fe700 1 --2- 172.21.15.106:0/227902023 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f8e4c0a1e90 0x7f8e4c0a4350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:27:01.279 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-04T09:27:01.281+0000 7f8e637fe700 1 --2- 172.21.15.106:0/227902023 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f8e4c0a1e90 0x7f8e4c0a4350 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f8e58006fd0 tx=0x7f8e58008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:27:01.281 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-04T09:27:01.284+0000 7f8e617fa700 1 -- 172.21.15.106:0/227902023 <== mon.2 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f8e5405e280 con 0x7f8e64100420 2024-03-04T09:27:01.485 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-04T09:27:01.488+0000 7f8e6a9de700 1 -- 172.21.15.106:0/227902023 --> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi106:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7f8e64062500 con 0x7f8e4c0a1e90 2024-03-04T09:27:01.834 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:27:01 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-04T09:27:01.835 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:27:01 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-04T09:27:01.835 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:27:01 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T09:27:01.836 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:27:01 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-04T09:27:01.836 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:27:01 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-04T09:27:01.836 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:27:01 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T09:27:01.881 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:27:01 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-04T09:27:01.881 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:27:01 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-04T09:27:01.881 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:27:01 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T09:27:02.834 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:27:02 smithi052 ceph-mon[32473]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-03-04T09:27:02.835 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:27:02 smithi052 ceph-mon[32473]: from='client.24326 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi106:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T09:27:02.836 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:27:02 smithi191 ceph-mon[36816]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-03-04T09:27:02.836 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:27:02 smithi191 ceph-mon[36816]: from='client.24326 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi106:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T09:27:02.881 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:27:02 smithi106 ceph-mon[36484]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-03-04T09:27:02.881 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:27:02 smithi106 ceph-mon[36484]: from='client.24326 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi106:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T09:27:04.835 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:27:04 smithi052 ceph-mon[32473]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 894 GiB / 894 GiB avail 2024-03-04T09:27:04.835 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:27:04 smithi052 ceph-mon[32473]: from='client.? 172.21.15.106:0/3812288069' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1a285a5a-f793-48da-a8eb-5326d9528cd8"}]: dispatch 2024-03-04T09:27:04.835 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:27:04 smithi052 ceph-mon[32473]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1a285a5a-f793-48da-a8eb-5326d9528cd8"}]: dispatch 2024-03-04T09:27:04.835 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:27:04 smithi052 ceph-mon[32473]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1a285a5a-f793-48da-a8eb-5326d9528cd8"}]': finished 2024-03-04T09:27:04.835 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:27:04 smithi052 ceph-mon[32473]: osdmap e69: 11 total, 10 up, 11 in 2024-03-04T09:27:04.835 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:27:04 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-04T09:27:04.836 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:27:04 smithi191 ceph-mon[36816]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 894 GiB / 894 GiB avail 2024-03-04T09:27:04.836 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:27:04 smithi191 ceph-mon[36816]: from='client.? 172.21.15.106:0/3812288069' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1a285a5a-f793-48da-a8eb-5326d9528cd8"}]: dispatch 2024-03-04T09:27:04.836 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:27:04 smithi191 ceph-mon[36816]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1a285a5a-f793-48da-a8eb-5326d9528cd8"}]: dispatch 2024-03-04T09:27:04.836 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:27:04 smithi191 ceph-mon[36816]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1a285a5a-f793-48da-a8eb-5326d9528cd8"}]': finished 2024-03-04T09:27:04.836 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:27:04 smithi191 ceph-mon[36816]: osdmap e69: 11 total, 10 up, 11 in 2024-03-04T09:27:04.836 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:27:04 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-04T09:27:04.881 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:27:04 smithi106 ceph-mon[36484]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 894 GiB / 894 GiB avail 2024-03-04T09:27:04.881 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:27:04 smithi106 ceph-mon[36484]: from='client.? 172.21.15.106:0/3812288069' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1a285a5a-f793-48da-a8eb-5326d9528cd8"}]: dispatch 2024-03-04T09:27:04.881 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:27:04 smithi106 ceph-mon[36484]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1a285a5a-f793-48da-a8eb-5326d9528cd8"}]: dispatch 2024-03-04T09:27:04.881 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:27:04 smithi106 ceph-mon[36484]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1a285a5a-f793-48da-a8eb-5326d9528cd8"}]': finished 2024-03-04T09:27:04.881 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:27:04 smithi106 ceph-mon[36484]: osdmap e69: 11 total, 10 up, 11 in 2024-03-04T09:27:04.882 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:27:04 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-04T09:27:05.834 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:27:05 smithi052 ceph-mon[32473]: from='client.? 172.21.15.106:0/1844113565' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-04T09:27:05.836 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:27:05 smithi191 ceph-mon[36816]: from='client.? 172.21.15.106:0/1844113565' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-04T09:27:05.881 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:27:05 smithi106 ceph-mon[36484]: from='client.? 172.21.15.106:0/1844113565' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-04T09:27:06.835 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:27:06 smithi052 ceph-mon[32473]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 894 GiB / 894 GiB avail 2024-03-04T09:27:06.836 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:27:06 smithi191 ceph-mon[36816]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 894 GiB / 894 GiB avail 2024-03-04T09:27:06.881 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:27:06 smithi106 ceph-mon[36484]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 894 GiB / 894 GiB avail 2024-03-04T09:27:08.835 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:27:08 smithi052 ceph-mon[32473]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 894 GiB / 894 GiB avail 2024-03-04T09:27:08.836 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:27:08 smithi191 ceph-mon[36816]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 894 GiB / 894 GiB avail 2024-03-04T09:27:08.881 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:27:08 smithi106 ceph-mon[36484]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 894 GiB / 894 GiB avail 2024-03-04T09:27:10.835 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:27:10 smithi052 ceph-mon[32473]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 894 GiB / 894 GiB avail 2024-03-04T09:27:10.836 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:27:10 smithi191 ceph-mon[36816]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 894 GiB / 894 GiB avail 2024-03-04T09:27:10.881 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:27:10 smithi106 ceph-mon[36484]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 894 GiB / 894 GiB avail 2024-03-04T09:27:12.631 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:27:12 smithi106 ceph-mon[36484]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 894 GiB / 894 GiB avail 2024-03-04T09:27:12.631 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:27:12 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-03-04T09:27:12.631 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:27:12 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T09:27:12.632 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:27:12 smithi106 ceph-mon[36484]: Deploying daemon osd.10 on smithi106 2024-03-04T09:27:12.835 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:27:12 smithi052 ceph-mon[32473]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 894 GiB / 894 GiB avail 2024-03-04T09:27:12.835 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:27:12 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-03-04T09:27:12.835 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:27:12 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T09:27:12.835 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:27:12 smithi052 ceph-mon[32473]: Deploying daemon osd.10 on smithi106 2024-03-04T09:27:12.836 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:27:12 smithi191 ceph-mon[36816]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 894 GiB / 894 GiB avail 2024-03-04T09:27:12.836 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:27:12 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-03-04T09:27:12.836 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:27:12 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T09:27:12.836 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:27:12 smithi191 ceph-mon[36816]: Deploying daemon osd.10 on smithi106 2024-03-04T09:27:14.759 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:27:14 smithi106 ceph-mon[36484]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 894 GiB / 894 GiB avail 2024-03-04T09:27:14.834 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:27:14 smithi052 ceph-mon[32473]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 894 GiB / 894 GiB avail 2024-03-04T09:27:14.836 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:27:14 smithi191 ceph-mon[36816]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 894 GiB / 894 GiB avail 2024-03-04T09:27:16.835 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:27:16 smithi052 ceph-mon[32473]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 894 GiB / 894 GiB avail 2024-03-04T09:27:16.836 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:27:16 smithi191 ceph-mon[36816]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 894 GiB / 894 GiB avail 2024-03-04T09:27:16.869 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:27:16 smithi106 ceph-mon[36484]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 894 GiB / 894 GiB avail 2024-03-04T09:27:18.834 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:27:18 smithi052 ceph-mon[32473]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 894 GiB / 894 GiB avail 2024-03-04T09:27:18.835 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:27:18 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:27:18.835 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:27:18 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:27:18.835 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:27:18 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T09:27:18.836 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:27:18 smithi191 ceph-mon[36816]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 894 GiB / 894 GiB avail 2024-03-04T09:27:18.836 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:27:18 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:27:18.836 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:27:18 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:27:18.836 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:27:18 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T09:27:18.881 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:27:18 smithi106 ceph-mon[36484]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 894 GiB / 894 GiB avail 2024-03-04T09:27:18.881 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:27:18 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:27:18.881 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:27:18 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:27:18.881 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:27:18 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T09:27:20.632 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:27:20 smithi106 ceph-mon[36484]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 894 GiB / 894 GiB avail 2024-03-04T09:27:20.834 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:27:20 smithi052 ceph-mon[32473]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 894 GiB / 894 GiB avail 2024-03-04T09:27:20.836 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:27:20 smithi191 ceph-mon[36816]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 894 GiB / 894 GiB avail 2024-03-04T09:27:21.151 INFO:teuthology.orchestra.run.smithi106.stdout:Created osd(s) 10 on host 'smithi106' 2024-03-04T09:27:21.151 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-04T09:27:21.154+0000 7f8e617fa700 1 -- 172.21.15.106:0/227902023 <== mgr.14152 v2:172.21.15.52:6800/3036033923 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+38 (secure 0 0 0) 0x7f8e64062500 con 0x7f8e4c0a1e90 2024-03-04T09:27:21.153 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-04T09:27:21.157+0000 7f8e6a9de700 1 -- 172.21.15.106:0/227902023 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f8e4c0a1e90 msgr2=0x7f8e4c0a4350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:27:21.154 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-04T09:27:21.157+0000 7f8e6a9de700 1 --2- 172.21.15.106:0/227902023 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f8e4c0a1e90 0x7f8e4c0a4350 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f8e58006fd0 tx=0x7f8e58008040 comp rx=0 tx=0).stop 2024-03-04T09:27:21.154 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-04T09:27:21.157+0000 7f8e6a9de700 1 -- 172.21.15.106:0/227902023 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8e64100420 msgr2=0x7f8e64198390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:27:21.154 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-04T09:27:21.157+0000 7f8e6a9de700 1 --2- 172.21.15.106:0/227902023 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8e64100420 0x7f8e64198390 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f8e5400c010 tx=0x7f8e54005e80 comp rx=0 tx=0).stop 2024-03-04T09:27:21.154 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-04T09:27:21.157+0000 7f8e6a9de700 1 -- 172.21.15.106:0/227902023 shutdown_connections 2024-03-04T09:27:21.154 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-04T09:27:21.157+0000 7f8e6a9de700 1 --2- 172.21.15.106:0/227902023 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f8e4c0a1e90 0x7f8e4c0a4350 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:27:21.154 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-04T09:27:21.157+0000 7f8e6a9de700 1 --2- 172.21.15.106:0/227902023 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8e64103e40 0x7f8e6419f950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:27:21.154 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-04T09:27:21.157+0000 7f8e6a9de700 1 --2- 172.21.15.106:0/227902023 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8e64100420 0x7f8e64198390 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:27:21.154 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-04T09:27:21.157+0000 7f8e6a9de700 1 --2- 172.21.15.106:0/227902023 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8e641034a0 0x7f8e641988d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:27:21.154 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-04T09:27:21.157+0000 7f8e6a9de700 1 -- 172.21.15.106:0/227902023 >> 172.21.15.106:0/227902023 conn(0x7f8e640fbd60 msgr2=0x7f8e641067a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:27:21.154 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-04T09:27:21.157+0000 7f8e6a9de700 1 -- 172.21.15.106:0/227902023 shutdown_connections 2024-03-04T09:27:21.154 INFO:teuthology.orchestra.run.smithi106.stderr:2024-03-04T09:27:21.157+0000 7f8e6a9de700 1 -- 172.21.15.106:0/227902023 wait complete. 2024-03-04T09:27:21.851 DEBUG:teuthology.orchestra.run.smithi106:osd.10> sudo journalctl -f -n 0 -u ceph-4e059f08-da08-11ee-95c6-87774f69a715@osd.10.service 2024-03-04T09:27:21.854 INFO:tasks.cephadm:Deploying osd.11 on smithi191 with /dev/vg_nvme/lv_1... 2024-03-04T09:27:21.854 DEBUG:teuthology.orchestra.run.smithi191:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4e059f08-da08-11ee-95c6-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-03-04T09:27:22.102 INFO:teuthology.orchestra.run.smithi191.stderr:Inferring config /var/lib/ceph/4e059f08-da08-11ee-95c6-87774f69a715/mon.c/config 2024-03-04T09:27:22.224 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:27:22 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:27:22.225 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:27:22 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:27:22.225 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:27:22 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:27:22.225 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:27:22 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:27:22.225 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:27:22 smithi191 ceph-mon[36816]: from='osd.10 [v2:172.21.15.106:6824/3815586576,v1:172.21.15.106:6825/3815586576]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-03-04T09:27:22.225 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:27:22 smithi191 ceph-mon[36816]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-03-04T09:27:22.381 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:27:22 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:27:22.381 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:27:22 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:27:22.381 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:27:22 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:27:22.381 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:27:22 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:27:22.381 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:27:22 smithi106 ceph-mon[36484]: from='osd.10 [v2:172.21.15.106:6824/3815586576,v1:172.21.15.106:6825/3815586576]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-03-04T09:27:22.381 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:27:22 smithi106 ceph-mon[36484]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-03-04T09:27:22.584 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:27:22 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:27:22.585 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:27:22 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:27:22.585 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:27:22 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:27:22.585 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:27:22 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:27:22.585 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:27:22 smithi052 ceph-mon[32473]: from='osd.10 [v2:172.21.15.106:6824/3815586576,v1:172.21.15.106:6825/3815586576]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-03-04T09:27:22.585 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:27:22 smithi052 ceph-mon[32473]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-03-04T09:27:23.381 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:27:23 smithi106 ceph-mon[36484]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 894 GiB / 894 GiB avail 2024-03-04T09:27:23.381 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:27:23 smithi106 ceph-mon[36484]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-03-04T09:27:23.381 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:27:23 smithi106 ceph-mon[36484]: from='osd.10 [v2:172.21.15.106:6824/3815586576,v1:172.21.15.106:6825/3815586576]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi106", "root=default"]}]: dispatch 2024-03-04T09:27:23.382 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:27:23 smithi106 ceph-mon[36484]: osdmap e70: 11 total, 10 up, 11 in 2024-03-04T09:27:23.382 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:27:23 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-04T09:27:23.382 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:27:23 smithi106 ceph-mon[36484]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi106", "root=default"]}]: dispatch 2024-03-04T09:27:23.417 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:27:23 smithi191 ceph-mon[36816]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 894 GiB / 894 GiB avail 2024-03-04T09:27:23.418 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:27:23 smithi191 ceph-mon[36816]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-03-04T09:27:23.418 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:27:23 smithi191 ceph-mon[36816]: from='osd.10 [v2:172.21.15.106:6824/3815586576,v1:172.21.15.106:6825/3815586576]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi106", "root=default"]}]: dispatch 2024-03-04T09:27:23.418 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:27:23 smithi191 ceph-mon[36816]: osdmap e70: 11 total, 10 up, 11 in 2024-03-04T09:27:23.418 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:27:23 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-04T09:27:23.418 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:27:23 smithi191 ceph-mon[36816]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi106", "root=default"]}]: dispatch 2024-03-04T09:27:23.585 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:27:23 smithi052 ceph-mon[32473]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 894 GiB / 894 GiB avail 2024-03-04T09:27:23.585 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:27:23 smithi052 ceph-mon[32473]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-03-04T09:27:23.585 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:27:23 smithi052 ceph-mon[32473]: from='osd.10 [v2:172.21.15.106:6824/3815586576,v1:172.21.15.106:6825/3815586576]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi106", "root=default"]}]: dispatch 2024-03-04T09:27:23.585 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:27:23 smithi052 ceph-mon[32473]: osdmap e70: 11 total, 10 up, 11 in 2024-03-04T09:27:23.585 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:27:23 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-04T09:27:23.585 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:27:23 smithi052 ceph-mon[32473]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi106", "root=default"]}]: dispatch 2024-03-04T09:27:24.217 INFO:teuthology.orchestra.run.smithi191.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-03-04T09:27:24.218 INFO:teuthology.orchestra.run.smithi191.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-03-04T09:27:24.218 INFO:teuthology.orchestra.run.smithi191.stderr: stderr: 10+0 records in 2024-03-04T09:27:24.218 INFO:teuthology.orchestra.run.smithi191.stderr:10+0 records out 2024-03-04T09:27:24.218 INFO:teuthology.orchestra.run.smithi191.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0112197 s, 935 MB/s 2024-03-04T09:27:24.218 INFO:teuthology.orchestra.run.smithi191.stderr:--> Zapping successful for: 2024-03-04T09:27:24.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:27:24 smithi191 ceph-mon[36816]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi106", "root=default"]}]': finished 2024-03-04T09:27:24.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:27:24 smithi191 ceph-mon[36816]: osdmap e71: 11 total, 10 up, 11 in 2024-03-04T09:27:24.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:27:24 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-04T09:27:24.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:27:24 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-04T09:27:24.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:27:24 smithi191 ceph-mon[36816]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 894 GiB / 894 GiB avail 2024-03-04T09:27:24.587 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:27:24 smithi191 ceph-mon[36816]: from='osd.10 ' entity='osd.10' 2024-03-04T09:27:24.631 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:27:24 smithi106 ceph-mon[36484]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi106", "root=default"]}]': finished 2024-03-04T09:27:24.631 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:27:24 smithi106 ceph-mon[36484]: osdmap e71: 11 total, 10 up, 11 in 2024-03-04T09:27:24.631 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:27:24 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-04T09:27:24.631 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:27:24 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-04T09:27:24.631 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:27:24 smithi106 ceph-mon[36484]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 894 GiB / 894 GiB avail 2024-03-04T09:27:24.631 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:27:24 smithi106 ceph-mon[36484]: from='osd.10 ' entity='osd.10' 2024-03-04T09:27:24.631 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 04 09:27:24 smithi106 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-10[60969]: 2024-03-04T09:27:24.227+0000 7f10f774b700 -1 osd.10 0 waiting for initial osdmap 2024-03-04T09:27:24.632 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 04 09:27:24 smithi106 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-10[60969]: 2024-03-04T09:27:24.240+0000 7f10ef536700 -1 osd.10 71 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-04T09:27:24.835 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:27:24 smithi052 ceph-mon[32473]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi106", "root=default"]}]': finished 2024-03-04T09:27:24.835 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:27:24 smithi052 ceph-mon[32473]: osdmap e71: 11 total, 10 up, 11 in 2024-03-04T09:27:24.835 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:27:24 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-04T09:27:24.835 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:27:24 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-04T09:27:24.835 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:27:24 smithi052 ceph-mon[32473]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 894 GiB / 894 GiB avail 2024-03-04T09:27:24.835 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:27:24 smithi052 ceph-mon[32473]: from='osd.10 ' entity='osd.10' 2024-03-04T09:27:25.040 DEBUG:teuthology.orchestra.run.smithi191:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4e059f08-da08-11ee-95c6-87774f69a715 -- ceph orch daemon add osd smithi191:vg_nvme/lv_1 2024-03-04T09:27:25.279 INFO:teuthology.orchestra.run.smithi191.stderr:Inferring config /var/lib/ceph/4e059f08-da08-11ee-95c6-87774f69a715/mon.c/config 2024-03-04T09:27:25.666 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:27:25 smithi191 ceph-mon[36816]: purged_snaps scrub starts 2024-03-04T09:27:25.666 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:27:25 smithi191 ceph-mon[36816]: purged_snaps scrub ok 2024-03-04T09:27:25.666 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:27:25 smithi191 ceph-mon[36816]: osd.10 [v2:172.21.15.106:6824/3815586576,v1:172.21.15.106:6825/3815586576] boot 2024-03-04T09:27:25.666 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:27:25 smithi191 ceph-mon[36816]: osdmap e72: 11 total, 11 up, 11 in 2024-03-04T09:27:25.667 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:27:25 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-04T09:27:25.667 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:27:25 smithi191 ceph-mon[36816]: Detected new or changed devices on smithi106 2024-03-04T09:27:25.667 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:27:25 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:27:25.667 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:27:25 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:27:25.667 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:27:25 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-04T09:27:25.667 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:27:25 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-04T09:27:25.667 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:27:25 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-04T09:27:25.667 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:27:25 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-04T09:27:25.667 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:27:25 smithi191 ceph-mon[36816]: Adjusting osd_memory_target on smithi106 to 4270M 2024-03-04T09:27:25.667 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:27:25 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:27:25.667 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:27:25 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T09:27:25.667 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:27:25 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T09:27:25.668 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:27:25 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:27:25.668 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:27:25 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T09:27:25.668 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:27:25 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T09:27:25.668 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:27:25 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T09:27:25.668 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:27:25 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:27:25.835 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:27:25 smithi052 ceph-mon[32473]: purged_snaps scrub starts 2024-03-04T09:27:25.835 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:27:25 smithi052 ceph-mon[32473]: purged_snaps scrub ok 2024-03-04T09:27:25.835 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:27:25 smithi052 ceph-mon[32473]: osd.10 [v2:172.21.15.106:6824/3815586576,v1:172.21.15.106:6825/3815586576] boot 2024-03-04T09:27:25.835 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:27:25 smithi052 ceph-mon[32473]: osdmap e72: 11 total, 11 up, 11 in 2024-03-04T09:27:25.835 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:27:25 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-04T09:27:25.835 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:27:25 smithi052 ceph-mon[32473]: Detected new or changed devices on smithi106 2024-03-04T09:27:25.835 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:27:25 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:27:25.835 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:27:25 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:27:25.835 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:27:25 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-04T09:27:25.836 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:27:25 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-04T09:27:25.836 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:27:25 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-04T09:27:25.836 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:27:25 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-04T09:27:25.836 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:27:25 smithi052 ceph-mon[32473]: Adjusting osd_memory_target on smithi106 to 4270M 2024-03-04T09:27:25.836 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:27:25 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:27:25.836 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:27:25 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T09:27:25.836 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:27:25 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T09:27:25.836 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:27:25 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:27:25.836 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:27:25 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T09:27:25.836 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:27:25 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T09:27:25.836 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:27:25 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T09:27:25.836 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:27:25 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:27:25.881 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:27:25 smithi106 ceph-mon[36484]: purged_snaps scrub starts 2024-03-04T09:27:25.881 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:27:25 smithi106 ceph-mon[36484]: purged_snaps scrub ok 2024-03-04T09:27:25.881 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:27:25 smithi106 ceph-mon[36484]: osd.10 [v2:172.21.15.106:6824/3815586576,v1:172.21.15.106:6825/3815586576] boot 2024-03-04T09:27:25.881 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:27:25 smithi106 ceph-mon[36484]: osdmap e72: 11 total, 11 up, 11 in 2024-03-04T09:27:25.881 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:27:25 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-04T09:27:25.881 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:27:25 smithi106 ceph-mon[36484]: Detected new or changed devices on smithi106 2024-03-04T09:27:25.882 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:27:25 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:27:25.882 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:27:25 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:27:25.882 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:27:25 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-04T09:27:25.882 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:27:25 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-04T09:27:25.882 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:27:25 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-04T09:27:25.882 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:27:25 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-04T09:27:25.882 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:27:25 smithi106 ceph-mon[36484]: Adjusting osd_memory_target on smithi106 to 4270M 2024-03-04T09:27:25.882 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:27:25 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:27:25.882 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:27:25 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T09:27:25.882 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:27:25 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T09:27:25.882 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:27:25 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:27:25.882 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:27:25 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T09:27:25.882 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:27:25 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T09:27:25.882 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:27:25 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T09:27:25.883 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:27:25 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:27:26.731 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:27:26 smithi191 ceph-mon[36816]: osdmap e73: 11 total, 11 up, 11 in 2024-03-04T09:27:26.732 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:27:26 smithi191 ceph-mon[36816]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail 2024-03-04T09:27:26.834 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:27:26 smithi052 ceph-mon[32473]: osdmap e73: 11 total, 11 up, 11 in 2024-03-04T09:27:26.835 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:27:26 smithi052 ceph-mon[32473]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail 2024-03-04T09:27:26.881 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:27:26 smithi106 ceph-mon[36484]: osdmap e73: 11 total, 11 up, 11 in 2024-03-04T09:27:26.881 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:27:26 smithi106 ceph-mon[36484]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail 2024-03-04T09:27:26.972 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:27:26.975+0000 7fe573586700 1 -- 172.21.15.191:0/190824966 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe56c102b90 msgr2=0x7fe56c109440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:27:26.972 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:27:26.975+0000 7fe573586700 1 --2- 172.21.15.191:0/190824966 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe56c102b90 0x7fe56c109440 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fe55c009990 tx=0x7fe55c009ca0 comp rx=0 tx=0).stop 2024-03-04T09:27:26.972 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:27:26.975+0000 7fe573586700 1 -- 172.21.15.191:0/190824966 shutdown_connections 2024-03-04T09:27:26.972 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:27:26.975+0000 7fe573586700 1 --2- 172.21.15.191:0/190824966 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe56c102b90 0x7fe56c109440 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:27:26.972 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:27:26.975+0000 7fe573586700 1 --2- 172.21.15.191:0/190824966 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe56c100ff0 0x7fe56c101410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:27:26.972 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:27:26.975+0000 7fe573586700 1 --2- 172.21.15.191:0/190824966 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe56c1021f0 0x7fe56c102650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:27:26.973 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:27:26.975+0000 7fe573586700 1 -- 172.21.15.191:0/190824966 >> 172.21.15.191:0/190824966 conn(0x7fe56c0fc5b0 msgr2=0x7fe56c0fe9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:27:26.973 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:27:26.975+0000 7fe573586700 1 -- 172.21.15.191:0/190824966 shutdown_connections 2024-03-04T09:27:26.973 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:27:26.975+0000 7fe573586700 1 -- 172.21.15.191:0/190824966 wait complete. 2024-03-04T09:27:26.975 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:27:26.976+0000 7fe573586700 1 Processor -- start 2024-03-04T09:27:26.975 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:27:26.976+0000 7fe573586700 1 -- start start 2024-03-04T09:27:26.975 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:27:26.976+0000 7fe573586700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe56c100ff0 0x7fe56c197120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:27:26.975 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:27:26.976+0000 7fe573586700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe56c1021f0 0x7fe56c197660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:27:26.975 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:27:26.976+0000 7fe573586700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe56c102b90 0x7fe56c19e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:27:26.975 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:27:26.976+0000 7fe573586700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe56c10cbc0 con 0x7fe56c100ff0 2024-03-04T09:27:26.976 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:27:26.976+0000 7fe573586700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe56c10ca40 con 0x7fe56c102b90 2024-03-04T09:27:26.976 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:27:26.976+0000 7fe573586700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe56c10cd40 con 0x7fe56c1021f0 2024-03-04T09:27:26.976 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:27:26.977+0000 7fe570b21700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe56c1021f0 0x7fe56c197660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:27:26.976 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:27:26.977+0000 7fe571322700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe56c100ff0 0x7fe56c197120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:27:26.976 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:27:26.977+0000 7fe570b21700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe56c1021f0 0x7fe56c197660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.191:38002/0 (socket says 172.21.15.191:38002) 2024-03-04T09:27:26.976 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:27:26.977+0000 7fe570b21700 1 -- 172.21.15.191:0/299057830 learned_addr learned my addr 172.21.15.191:0/299057830 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-03-04T09:27:26.976 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:27:26.977+0000 7fe571b23700 1 --2- 172.21.15.191:0/299057830 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe56c102b90 0x7fe56c19e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:27:26.976 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:27:26.977+0000 7fe570b21700 1 -- 172.21.15.191:0/299057830 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe56c102b90 msgr2=0x7fe56c19e6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:27:26.977 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:27:26.977+0000 7fe570b21700 1 --2- 172.21.15.191:0/299057830 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe56c102b90 0x7fe56c19e6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:27:26.977 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:27:26.977+0000 7fe570b21700 1 -- 172.21.15.191:0/299057830 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe56c100ff0 msgr2=0x7fe56c197120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:27:26.977 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:27:26.977+0000 7fe570b21700 1 --2- 172.21.15.191:0/299057830 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe56c100ff0 0x7fe56c197120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:27:26.977 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:27:26.977+0000 7fe570b21700 1 -- 172.21.15.191:0/299057830 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe56c19ed80 con 0x7fe56c1021f0 2024-03-04T09:27:26.977 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:27:26.977+0000 7fe570b21700 1 --2- 172.21.15.191:0/299057830 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe56c1021f0 0x7fe56c197660 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fe56000c950 tx=0x7fe56000cc60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:27:26.977 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:27:26.977+0000 7fe56a7fc700 1 -- 172.21.15.191:0/299057830 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5600078d0 con 0x7fe56c1021f0 2024-03-04T09:27:26.977 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:27:26.977+0000 7fe56a7fc700 1 -- 172.21.15.191:0/299057830 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe56000f450 con 0x7fe56c1021f0 2024-03-04T09:27:26.977 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:27:26.977+0000 7fe573586700 1 -- 172.21.15.191:0/299057830 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe56c19f070 con 0x7fe56c1021f0 2024-03-04T09:27:26.977 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:27:26.977+0000 7fe56a7fc700 1 -- 172.21.15.191:0/299057830 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe560018660 con 0x7fe56c1021f0 2024-03-04T09:27:26.977 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:27:26.977+0000 7fe573586700 1 -- 172.21.15.191:0/299057830 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe56c19f600 con 0x7fe56c1021f0 2024-03-04T09:27:26.977 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:27:26.978+0000 7fe573586700 1 -- 172.21.15.191:0/299057830 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe56c068d80 con 0x7fe56c1021f0 2024-03-04T09:27:26.978 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:27:26.979+0000 7fe56a7fc700 1 -- 172.21.15.191:0/299057830 <== mon.1 v2:172.21.15.191:3300/0 4 ==== mgrmap(e 16) v1 ==== 136756+0+0 (secure 0 0 0) 0x7fe56000f600 con 0x7fe56c1021f0 2024-03-04T09:27:26.978 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:27:26.980+0000 7fe56a7fc700 1 --2- 172.21.15.191:0/299057830 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7fe5540a1e90 0x7fe5540a4350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:27:26.978 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:27:26.980+0000 7fe56a7fc700 1 -- 172.21.15.191:0/299057830 <== mon.1 v2:172.21.15.191:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6723+0+0 (secure 0 0 0) 0x7fe5600cd6d0 con 0x7fe56c1021f0 2024-03-04T09:27:26.978 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:27:26.980+0000 7fe571322700 1 --2- 172.21.15.191:0/299057830 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7fe5540a1e90 0x7fe5540a4350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:27:26.978 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:27:26.981+0000 7fe571322700 1 --2- 172.21.15.191:0/299057830 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7fe5540a1e90 0x7fe5540a4350 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fe56c102050 tx=0x7fe5640091c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:27:26.980 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:27:26.984+0000 7fe56a7fc700 1 -- 172.21.15.191:0/299057830 <== mon.1 v2:172.21.15.191:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fe560014030 con 0x7fe56c1021f0 2024-03-04T09:27:27.184 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:27:27.187+0000 7fe573586700 1 -- 172.21.15.191:0/299057830 --> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi191:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7fe56c062500 con 0x7fe5540a1e90 2024-03-04T09:27:27.794 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:27:27 smithi191 ceph-mon[36816]: osdmap e74: 11 total, 11 up, 11 in 2024-03-04T09:27:27.794 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:27:27 smithi191 ceph-mon[36816]: from='client.24346 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi191:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T09:27:27.794 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:27:27 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-04T09:27:27.795 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:27:27 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-04T09:27:27.795 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:27:27 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T09:27:27.835 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:27:27 smithi052 ceph-mon[32473]: osdmap e74: 11 total, 11 up, 11 in 2024-03-04T09:27:27.835 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:27:27 smithi052 ceph-mon[32473]: from='client.24346 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi191:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T09:27:27.835 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:27:27 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-04T09:27:27.835 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:27:27 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-04T09:27:27.835 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:27:27 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T09:27:27.881 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:27:27 smithi106 ceph-mon[36484]: osdmap e74: 11 total, 11 up, 11 in 2024-03-04T09:27:27.881 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:27:27 smithi106 ceph-mon[36484]: from='client.24346 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi191:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T09:27:27.881 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:27:27 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-04T09:27:27.882 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:27:27 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-04T09:27:27.882 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:27:27 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T09:27:28.835 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:27:28 smithi052 ceph-mon[32473]: pgmap v238: 1 pgs: 1 remapped+peering; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail 2024-03-04T09:27:28.835 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:27:28 smithi052 ceph-mon[32473]: osdmap e75: 11 total, 11 up, 11 in 2024-03-04T09:27:28.836 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:27:28 smithi191 ceph-mon[36816]: pgmap v238: 1 pgs: 1 remapped+peering; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail 2024-03-04T09:27:28.836 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:27:28 smithi191 ceph-mon[36816]: osdmap e75: 11 total, 11 up, 11 in 2024-03-04T09:27:28.881 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:27:28 smithi106 ceph-mon[36484]: pgmap v238: 1 pgs: 1 remapped+peering; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail 2024-03-04T09:27:28.881 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:27:28 smithi106 ceph-mon[36484]: osdmap e75: 11 total, 11 up, 11 in 2024-03-04T09:27:29.782 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:27:29 smithi106 ceph-mon[36484]: osdmap e76: 11 total, 11 up, 11 in 2024-03-04T09:27:29.834 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:27:29 smithi052 ceph-mon[32473]: osdmap e76: 11 total, 11 up, 11 in 2024-03-04T09:27:29.836 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:27:29 smithi191 ceph-mon[36816]: osdmap e76: 11 total, 11 up, 11 in 2024-03-04T09:27:30.835 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:27:30 smithi052 ceph-mon[32473]: pgmap v241: 1 pgs: 1 remapped+peering; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail 2024-03-04T09:27:30.835 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:27:30 smithi052 ceph-mon[32473]: from='client.? 172.21.15.191:0/680513597' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1f8b9a43-57fa-41f4-9969-3b803a74bf45"}]: dispatch 2024-03-04T09:27:30.835 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:27:30 smithi052 ceph-mon[32473]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1f8b9a43-57fa-41f4-9969-3b803a74bf45"}]: dispatch 2024-03-04T09:27:30.835 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:27:30 smithi052 ceph-mon[32473]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1f8b9a43-57fa-41f4-9969-3b803a74bf45"}]': finished 2024-03-04T09:27:30.835 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:27:30 smithi052 ceph-mon[32473]: osdmap e77: 12 total, 11 up, 12 in 2024-03-04T09:27:30.836 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:27:30 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-04T09:27:30.837 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:27:30 smithi191 ceph-mon[36816]: pgmap v241: 1 pgs: 1 remapped+peering; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail 2024-03-04T09:27:30.837 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:27:30 smithi191 ceph-mon[36816]: from='client.? 172.21.15.191:0/680513597' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1f8b9a43-57fa-41f4-9969-3b803a74bf45"}]: dispatch 2024-03-04T09:27:30.837 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:27:30 smithi191 ceph-mon[36816]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1f8b9a43-57fa-41f4-9969-3b803a74bf45"}]: dispatch 2024-03-04T09:27:30.837 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:27:30 smithi191 ceph-mon[36816]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1f8b9a43-57fa-41f4-9969-3b803a74bf45"}]': finished 2024-03-04T09:27:30.837 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:27:30 smithi191 ceph-mon[36816]: osdmap e77: 12 total, 11 up, 12 in 2024-03-04T09:27:30.837 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:27:30 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-04T09:27:30.881 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:27:30 smithi106 ceph-mon[36484]: pgmap v241: 1 pgs: 1 remapped+peering; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail 2024-03-04T09:27:30.881 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:27:30 smithi106 ceph-mon[36484]: from='client.? 172.21.15.191:0/680513597' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1f8b9a43-57fa-41f4-9969-3b803a74bf45"}]: dispatch 2024-03-04T09:27:30.881 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:27:30 smithi106 ceph-mon[36484]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1f8b9a43-57fa-41f4-9969-3b803a74bf45"}]: dispatch 2024-03-04T09:27:30.882 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:27:30 smithi106 ceph-mon[36484]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1f8b9a43-57fa-41f4-9969-3b803a74bf45"}]': finished 2024-03-04T09:27:30.882 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:27:30 smithi106 ceph-mon[36484]: osdmap e77: 12 total, 11 up, 12 in 2024-03-04T09:27:30.882 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:27:30 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-04T09:27:31.834 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:27:31 smithi052 ceph-mon[32473]: from='client.? 172.21.15.191:0/4033238129' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-04T09:27:31.836 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:27:31 smithi191 ceph-mon[36816]: from='client.? 172.21.15.191:0/4033238129' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-04T09:27:31.881 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:27:31 smithi106 ceph-mon[36484]: from='client.? 172.21.15.191:0/4033238129' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-04T09:27:32.835 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:27:32 smithi052 ceph-mon[32473]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail; 115 KiB/s, 0 objects/s recovering 2024-03-04T09:27:32.836 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:27:32 smithi191 ceph-mon[36816]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail; 115 KiB/s, 0 objects/s recovering 2024-03-04T09:27:32.881 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:27:32 smithi106 ceph-mon[36484]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail; 115 KiB/s, 0 objects/s recovering 2024-03-04T09:27:34.834 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:27:34 smithi052 ceph-mon[32473]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-03-04T09:27:34.836 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:27:34 smithi191 ceph-mon[36816]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-03-04T09:27:34.881 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:27:34 smithi106 ceph-mon[36484]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-03-04T09:27:36.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:27:36 smithi191 ceph-mon[36816]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-03-04T09:27:36.834 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:27:36 smithi052 ceph-mon[32473]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-03-04T09:27:36.881 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:27:36 smithi106 ceph-mon[36484]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-03-04T09:27:37.835 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:27:37 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-03-04T09:27:37.835 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:27:37 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T09:27:37.836 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:27:37 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-03-04T09:27:37.836 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:27:37 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T09:27:37.881 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:27:37 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-03-04T09:27:37.881 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:27:37 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T09:27:38.835 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:27:38 smithi052 ceph-mon[32473]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail; 64 KiB/s, 0 objects/s recovering 2024-03-04T09:27:38.835 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:27:38 smithi052 ceph-mon[32473]: Deploying daemon osd.11 on smithi191 2024-03-04T09:27:38.836 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:27:38 smithi191 ceph-mon[36816]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail; 64 KiB/s, 0 objects/s recovering 2024-03-04T09:27:38.836 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:27:38 smithi191 ceph-mon[36816]: Deploying daemon osd.11 on smithi191 2024-03-04T09:27:38.881 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:27:38 smithi106 ceph-mon[36484]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail; 64 KiB/s, 0 objects/s recovering 2024-03-04T09:27:38.881 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:27:38 smithi106 ceph-mon[36484]: Deploying daemon osd.11 on smithi191 2024-03-04T09:27:40.834 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:27:40 smithi052 ceph-mon[32473]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-03-04T09:27:40.836 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:27:40 smithi191 ceph-mon[36816]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-03-04T09:27:40.881 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:27:40 smithi106 ceph-mon[36484]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-03-04T09:27:42.834 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:27:42 smithi052 ceph-mon[32473]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail; 50 KiB/s, 0 objects/s recovering 2024-03-04T09:27:42.836 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:27:42 smithi191 ceph-mon[36816]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail; 50 KiB/s, 0 objects/s recovering 2024-03-04T09:27:42.881 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:27:42 smithi106 ceph-mon[36484]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail; 50 KiB/s, 0 objects/s recovering 2024-03-04T09:27:44.584 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:27:44 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:27:44.585 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:27:44 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:27:44.585 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:27:44 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T09:27:44.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:27:44 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:27:44.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:27:44 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:27:44.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:27:44 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T09:27:44.631 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:27:44 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:27:44.631 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:27:44 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:27:44.631 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:27:44 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T09:27:45.431 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:27:45 smithi191 ceph-mon[36816]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail 2024-03-04T09:27:45.585 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:27:45 smithi052 ceph-mon[32473]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail 2024-03-04T09:27:45.631 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:27:45 smithi106 ceph-mon[36484]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail 2024-03-04T09:27:47.155 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:27:47.157+0000 7fe56a7fc700 1 -- 172.21.15.191:0/299057830 <== mgr.14152 v2:172.21.15.52:6800/3036033923 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+38 (secure 0 0 0) 0x7fe56c062500 con 0x7fe5540a1e90 2024-03-04T09:27:47.155 INFO:teuthology.orchestra.run.smithi191.stdout:Created osd(s) 11 on host 'smithi191' 2024-03-04T09:27:47.156 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:27:47.159+0000 7fe573586700 1 -- 172.21.15.191:0/299057830 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7fe5540a1e90 msgr2=0x7fe5540a4350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:27:47.156 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:27:47.159+0000 7fe573586700 1 --2- 172.21.15.191:0/299057830 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7fe5540a1e90 0x7fe5540a4350 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fe56c102050 tx=0x7fe5640091c0 comp rx=0 tx=0).stop 2024-03-04T09:27:47.156 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:27:47.159+0000 7fe573586700 1 -- 172.21.15.191:0/299057830 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe56c1021f0 msgr2=0x7fe56c197660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:27:47.157 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:27:47.159+0000 7fe573586700 1 --2- 172.21.15.191:0/299057830 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe56c1021f0 0x7fe56c197660 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fe56000c950 tx=0x7fe56000cc60 comp rx=0 tx=0).stop 2024-03-04T09:27:47.157 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:27:47.159+0000 7fe573586700 1 -- 172.21.15.191:0/299057830 shutdown_connections 2024-03-04T09:27:47.157 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:27:47.159+0000 7fe573586700 1 --2- 172.21.15.191:0/299057830 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7fe5540a1e90 0x7fe5540a4350 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:27:47.157 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:27:47.159+0000 7fe573586700 1 --2- 172.21.15.191:0/299057830 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe56c102b90 0x7fe56c19e6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:27:47.157 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:27:47.159+0000 7fe573586700 1 --2- 172.21.15.191:0/299057830 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe56c100ff0 0x7fe56c197120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:27:47.157 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:27:47.159+0000 7fe573586700 1 --2- 172.21.15.191:0/299057830 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe56c1021f0 0x7fe56c197660 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:27:47.157 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:27:47.159+0000 7fe573586700 1 -- 172.21.15.191:0/299057830 >> 172.21.15.191:0/299057830 conn(0x7fe56c0fc5b0 msgr2=0x7fe56c105490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:27:47.157 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:27:47.159+0000 7fe573586700 1 -- 172.21.15.191:0/299057830 shutdown_connections 2024-03-04T09:27:47.157 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:27:47.159+0000 7fe573586700 1 -- 172.21.15.191:0/299057830 wait complete. 2024-03-04T09:27:47.336 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:27:47 smithi191 ceph-mon[36816]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail 2024-03-04T09:27:47.336 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:27:47 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:27:47.336 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:27:47 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:27:47.336 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:27:47 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T09:27:47.336 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:27:47 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T09:27:47.337 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:27:47 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:27:47.337 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:27:47 smithi191 ceph-mon[36816]: from='osd.11 [v2:172.21.15.191:6824/2009632843,v1:172.21.15.191:6825/2009632843]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-03-04T09:27:47.337 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:27:47 smithi191 ceph-mon[36816]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-03-04T09:27:47.337 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:27:47 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:27:47.337 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:27:47 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:27:47.337 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:27:47 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T09:27:47.585 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:27:47 smithi052 ceph-mon[32473]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail 2024-03-04T09:27:47.585 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:27:47 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:27:47.585 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:27:47 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:27:47.585 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:27:47 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T09:27:47.585 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:27:47 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T09:27:47.585 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:27:47 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:27:47.585 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:27:47 smithi052 ceph-mon[32473]: from='osd.11 [v2:172.21.15.191:6824/2009632843,v1:172.21.15.191:6825/2009632843]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-03-04T09:27:47.586 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:27:47 smithi052 ceph-mon[32473]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-03-04T09:27:47.586 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:27:47 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:27:47.586 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:27:47 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:27:47.586 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:27:47 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T09:27:47.631 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:27:47 smithi106 ceph-mon[36484]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail 2024-03-04T09:27:47.631 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:27:47 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:27:47.631 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:27:47 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:27:47.631 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:27:47 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T09:27:47.632 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:27:47 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T09:27:47.632 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:27:47 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:27:47.632 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:27:47 smithi106 ceph-mon[36484]: from='osd.11 [v2:172.21.15.191:6824/2009632843,v1:172.21.15.191:6825/2009632843]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-03-04T09:27:47.632 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:27:47 smithi106 ceph-mon[36484]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-03-04T09:27:47.632 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:27:47 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:27:47.632 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:27:47 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:27:47.632 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:27:47 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T09:27:47.811 DEBUG:teuthology.orchestra.run.smithi191:osd.11> sudo journalctl -f -n 0 -u ceph-4e059f08-da08-11ee-95c6-87774f69a715@osd.11.service 2024-03-04T09:27:47.813 INFO:tasks.cephadm:Waiting for 12 OSDs to come up... 2024-03-04T09:27:47.813 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4e059f08-da08-11ee-95c6-87774f69a715 -- ceph osd stat -f json 2024-03-04T09:27:48.066 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/4e059f08-da08-11ee-95c6-87774f69a715/mon.a/config 2024-03-04T09:27:48.313 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:27:48 smithi191 ceph-mon[36816]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-03-04T09:27:48.313 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:27:48 smithi191 ceph-mon[36816]: osdmap e78: 12 total, 11 up, 12 in 2024-03-04T09:27:48.313 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:27:48 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-04T09:27:48.314 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:27:48 smithi191 ceph-mon[36816]: from='osd.11 [v2:172.21.15.191:6824/2009632843,v1:172.21.15.191:6825/2009632843]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi191", "root=default"]}]: dispatch 2024-03-04T09:27:48.314 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:27:48 smithi191 ceph-mon[36816]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi191", "root=default"]}]: dispatch 2024-03-04T09:27:48.314 INFO:journalctl@ceph.osd.11.smithi191.stdout:Mar 04 09:27:48 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-11[62160]: 2024-03-04T09:27:48.263+0000 7fdced6a1700 -1 osd.11 0 waiting for initial osdmap 2024-03-04T09:27:48.314 INFO:journalctl@ceph.osd.11.smithi191.stdout:Mar 04 09:27:48 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-11[62160]: 2024-03-04T09:27:48.277+0000 7fdce9494700 -1 osd.11 79 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-04T09:27:48.443 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:27:48 smithi052 ceph-mon[32473]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-03-04T09:27:48.443 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:27:48 smithi052 ceph-mon[32473]: osdmap e78: 12 total, 11 up, 12 in 2024-03-04T09:27:48.443 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:27:48 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-04T09:27:48.443 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:27:48 smithi052 ceph-mon[32473]: from='osd.11 [v2:172.21.15.191:6824/2009632843,v1:172.21.15.191:6825/2009632843]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi191", "root=default"]}]: dispatch 2024-03-04T09:27:48.443 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:27:48 smithi052 ceph-mon[32473]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi191", "root=default"]}]: dispatch 2024-03-04T09:27:48.631 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:27:48 smithi106 ceph-mon[36484]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-03-04T09:27:48.631 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:27:48 smithi106 ceph-mon[36484]: osdmap e78: 12 total, 11 up, 12 in 2024-03-04T09:27:48.631 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:27:48 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-04T09:27:48.631 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:27:48 smithi106 ceph-mon[36484]: from='osd.11 [v2:172.21.15.191:6824/2009632843,v1:172.21.15.191:6825/2009632843]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi191", "root=default"]}]: dispatch 2024-03-04T09:27:48.631 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:27:48 smithi106 ceph-mon[36484]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi191", "root=default"]}]: dispatch 2024-03-04T09:27:49.335 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:27:49 smithi052 ceph-mon[32473]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail 2024-03-04T09:27:49.335 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:27:49 smithi052 ceph-mon[32473]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi191", "root=default"]}]': finished 2024-03-04T09:27:49.335 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:27:49 smithi052 ceph-mon[32473]: osdmap e79: 12 total, 11 up, 12 in 2024-03-04T09:27:49.335 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:27:49 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-04T09:27:49.335 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:27:49 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-04T09:27:49.336 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:27:49 smithi191 ceph-mon[36816]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail 2024-03-04T09:27:49.336 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:27:49 smithi191 ceph-mon[36816]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi191", "root=default"]}]': finished 2024-03-04T09:27:49.336 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:27:49 smithi191 ceph-mon[36816]: osdmap e79: 12 total, 11 up, 12 in 2024-03-04T09:27:49.336 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:27:49 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-04T09:27:49.336 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:27:49 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-04T09:27:49.631 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:27:49 smithi106 ceph-mon[36484]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail 2024-03-04T09:27:49.631 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:27:49 smithi106 ceph-mon[36484]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi191", "root=default"]}]': finished 2024-03-04T09:27:49.631 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:27:49 smithi106 ceph-mon[36484]: osdmap e79: 12 total, 11 up, 12 in 2024-03-04T09:27:49.631 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:27:49 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-04T09:27:49.631 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:27:49 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-04T09:27:49.959 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:27:49.960+0000 7ff7506b2700 1 -- 172.21.15.52:0/3491611796 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff7481020f0 msgr2=0x7ff748102510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:27:49.959 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:27:49.960+0000 7ff7506b2700 1 --2- 172.21.15.52:0/3491611796 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff7481020f0 0x7ff748102510 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7ff73c0099c0 tx=0x7ff73c009cd0 comp rx=0 tx=0).stop 2024-03-04T09:27:49.959 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:27:49.960+0000 7ff7506b2700 1 -- 172.21.15.52:0/3491611796 shutdown_connections 2024-03-04T09:27:49.959 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:27:49.960+0000 7ff7506b2700 1 --2- 172.21.15.52:0/3491611796 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff748103c90 0x7ff74810a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:27:49.959 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:27:49.960+0000 7ff7506b2700 1 --2- 172.21.15.52:0/3491611796 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff7481020f0 0x7ff748102510 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:27:49.959 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:27:49.960+0000 7ff7506b2700 1 --2- 172.21.15.52:0/3491611796 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff7481032f0 0x7ff748103750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:27:49.960 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:27:49.960+0000 7ff7506b2700 1 -- 172.21.15.52:0/3491611796 >> 172.21.15.52:0/3491611796 conn(0x7ff7480fd6d0 msgr2=0x7ff7480ffaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:27:49.960 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:27:49.960+0000 7ff7506b2700 1 -- 172.21.15.52:0/3491611796 shutdown_connections 2024-03-04T09:27:49.960 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:27:49.960+0000 7ff7506b2700 1 -- 172.21.15.52:0/3491611796 wait complete. 2024-03-04T09:27:49.960 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:27:49.961+0000 7ff7506b2700 1 Processor -- start 2024-03-04T09:27:49.960 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:27:49.961+0000 7ff7506b2700 1 -- start start 2024-03-04T09:27:49.960 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:27:49.961+0000 7ff7506b2700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff7481020f0 0x7ff748198370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:27:49.960 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:27:49.961+0000 7ff7506b2700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff7481032f0 0x7ff7481988b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:27:49.960 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:27:49.961+0000 7ff7506b2700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff748103c90 0x7ff74819f930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:27:49.960 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:27:49.961+0000 7ff7506b2700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff74810dce0 con 0x7ff7481032f0 2024-03-04T09:27:49.960 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:27:49.961+0000 7ff7506b2700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff74810db60 con 0x7ff7481020f0 2024-03-04T09:27:49.960 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:27:49.961+0000 7ff7506b2700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff74810de60 con 0x7ff748103c90 2024-03-04T09:27:49.960 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:27:49.961+0000 7ff74dc4d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff7481032f0 0x7ff7481988b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:27:49.960 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:27:49.961+0000 7ff74dc4d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff7481032f0 0x7ff7481988b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:46630/0 (socket says 172.21.15.52:46630) 2024-03-04T09:27:49.961 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:27:49.961+0000 7ff74dc4d700 1 -- 172.21.15.52:0/1685560111 learned_addr learned my addr 172.21.15.52:0/1685560111 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-04T09:27:49.961 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:27:49.961+0000 7ff74ec4f700 1 --2- 172.21.15.52:0/1685560111 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff748103c90 0x7ff74819f930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:27:49.961 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:27:49.962+0000 7ff74e44e700 1 --2- 172.21.15.52:0/1685560111 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff7481020f0 0x7ff748198370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:27:49.961 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:27:49.962+0000 7ff74dc4d700 1 -- 172.21.15.52:0/1685560111 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff748103c90 msgr2=0x7ff74819f930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:27:49.961 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:27:49.962+0000 7ff74dc4d700 1 --2- 172.21.15.52:0/1685560111 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff748103c90 0x7ff74819f930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:27:49.961 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:27:49.962+0000 7ff74dc4d700 1 -- 172.21.15.52:0/1685560111 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff7481020f0 msgr2=0x7ff748198370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:27:49.961 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:27:49.962+0000 7ff74dc4d700 1 --2- 172.21.15.52:0/1685560111 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff7481020f0 0x7ff748198370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:27:49.961 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:27:49.962+0000 7ff74dc4d700 1 -- 172.21.15.52:0/1685560111 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff74819ffd0 con 0x7ff7481032f0 2024-03-04T09:27:49.961 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:27:49.962+0000 7ff74e44e700 1 --2- 172.21.15.52:0/1685560111 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff7481020f0 0x7ff748198370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T09:27:49.961 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:27:49.962+0000 7ff74ec4f700 1 --2- 172.21.15.52:0/1685560111 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff748103c90 0x7ff74819f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-04T09:27:49.961 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:27:49.962+0000 7ff74dc4d700 1 --2- 172.21.15.52:0/1685560111 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff7481032f0 0x7ff7481988b0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7ff74400e9f0 tx=0x7ff74400ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:27:49.961 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:27:49.962+0000 7ff73b7fe700 1 -- 172.21.15.52:0/1685560111 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff74400cbd0 con 0x7ff7481032f0 2024-03-04T09:27:49.961 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:27:49.962+0000 7ff7506b2700 1 -- 172.21.15.52:0/1685560111 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7481a02c0 con 0x7ff7481032f0 2024-03-04T09:27:49.961 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:27:49.962+0000 7ff73b7fe700 1 -- 172.21.15.52:0/1685560111 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff74400cd70 con 0x7ff7481032f0 2024-03-04T09:27:49.962 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:27:49.962+0000 7ff73b7fe700 1 -- 172.21.15.52:0/1685560111 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff744010430 con 0x7ff7481032f0 2024-03-04T09:27:49.962 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:27:49.962+0000 7ff7506b2700 1 -- 172.21.15.52:0/1685560111 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7481a07b0 con 0x7ff7481032f0 2024-03-04T09:27:49.962 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:27:49.963+0000 7ff7506b2700 1 -- 172.21.15.52:0/1685560111 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff748068d80 con 0x7ff7481032f0 2024-03-04T09:27:49.963 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:27:49.964+0000 7ff73b7fe700 1 -- 172.21.15.52:0/1685560111 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 16) v1 ==== 136756+0+0 (secure 0 0 0) 0x7ff744004da0 con 0x7ff7481032f0 2024-03-04T09:27:49.963 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:27:49.965+0000 7ff73b7fe700 1 --2- 172.21.15.52:0/1685560111 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7ff7340a1e10 0x7ff7340a42d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:27:49.963 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:27:49.965+0000 7ff74e44e700 1 --2- 172.21.15.52:0/1685560111 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7ff7340a1e10 0x7ff7340a42d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:27:49.963 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:27:49.965+0000 7ff73b7fe700 1 -- 172.21.15.52:0/1685560111 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7127+0+0 (secure 0 0 0) 0x7ff7440cde20 con 0x7ff7481032f0 2024-03-04T09:27:49.963 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:27:49.966+0000 7ff74e44e700 1 --2- 172.21.15.52:0/1685560111 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7ff7340a1e10 0x7ff7340a42d0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7ff73c00b580 tx=0x7ff73c005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:27:49.965 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:27:49.969+0000 7ff73b7fe700 1 -- 172.21.15.52:0/1685560111 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7ff744053280 con 0x7ff7481032f0 2024-03-04T09:27:50.173 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:27:50.176+0000 7ff7506b2700 1 -- 172.21.15.52:0/1685560111 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) v1 -- 0x7ff74802d9e0 con 0x7ff7481032f0 2024-03-04T09:27:50.174 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:27:50.177+0000 7ff73b7fe700 1 -- 172.21.15.52:0/1685560111 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v80) v1 ==== 74+0+133 (secure 0 0 0) 0x7ff74409b650 con 0x7ff7481032f0 2024-03-04T09:27:50.174 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-03-04T09:27:50.176 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:27:50.179+0000 7ff7506b2700 1 -- 172.21.15.52:0/1685560111 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7ff7340a1e10 msgr2=0x7ff7340a42d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:27:50.177 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:27:50.179+0000 7ff7506b2700 1 --2- 172.21.15.52:0/1685560111 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7ff7340a1e10 0x7ff7340a42d0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7ff73c00b580 tx=0x7ff73c005fb0 comp rx=0 tx=0).stop 2024-03-04T09:27:50.177 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:27:50.179+0000 7ff7506b2700 1 -- 172.21.15.52:0/1685560111 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff7481032f0 msgr2=0x7ff7481988b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:27:50.177 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:27:50.179+0000 7ff7506b2700 1 --2- 172.21.15.52:0/1685560111 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff7481032f0 0x7ff7481988b0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7ff74400e9f0 tx=0x7ff74400ed00 comp rx=0 tx=0).stop 2024-03-04T09:27:50.177 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:27:50.179+0000 7ff7506b2700 1 -- 172.21.15.52:0/1685560111 shutdown_connections 2024-03-04T09:27:50.177 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:27:50.179+0000 7ff7506b2700 1 --2- 172.21.15.52:0/1685560111 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7ff7340a1e10 0x7ff7340a42d0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:27:50.177 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:27:50.179+0000 7ff7506b2700 1 --2- 172.21.15.52:0/1685560111 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff748103c90 0x7ff74819f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:27:50.177 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:27:50.179+0000 7ff7506b2700 1 --2- 172.21.15.52:0/1685560111 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff7481020f0 0x7ff748198370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:27:50.177 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:27:50.179+0000 7ff7506b2700 1 --2- 172.21.15.52:0/1685560111 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff7481032f0 0x7ff7481988b0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:27:50.177 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:27:50.179+0000 7ff7506b2700 1 -- 172.21.15.52:0/1685560111 >> 172.21.15.52:0/1685560111 conn(0x7ff7480fd6d0 msgr2=0x7ff7481066c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:27:50.177 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:27:50.179+0000 7ff7506b2700 1 -- 172.21.15.52:0/1685560111 shutdown_connections 2024-03-04T09:27:50.177 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:27:50.180+0000 7ff7506b2700 1 -- 172.21.15.52:0/1685560111 wait complete. 2024-03-04T09:27:50.476 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:27:50 smithi191 ceph-mon[36816]: purged_snaps scrub starts 2024-03-04T09:27:50.476 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:27:50 smithi191 ceph-mon[36816]: purged_snaps scrub ok 2024-03-04T09:27:50.476 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:27:50 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-04T09:27:50.476 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:27:50 smithi191 ceph-mon[36816]: osd.11 [v2:172.21.15.191:6824/2009632843,v1:172.21.15.191:6825/2009632843] boot 2024-03-04T09:27:50.476 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:27:50 smithi191 ceph-mon[36816]: osdmap e80: 12 total, 12 up, 12 in 2024-03-04T09:27:50.476 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:27:50 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-04T09:27:50.476 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:27:50 smithi191 ceph-mon[36816]: from='client.? 172.21.15.52:0/1685560111' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-03-04T09:27:50.584 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:27:50 smithi052 ceph-mon[32473]: purged_snaps scrub starts 2024-03-04T09:27:50.585 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:27:50 smithi052 ceph-mon[32473]: purged_snaps scrub ok 2024-03-04T09:27:50.585 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:27:50 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-04T09:27:50.585 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:27:50 smithi052 ceph-mon[32473]: osd.11 [v2:172.21.15.191:6824/2009632843,v1:172.21.15.191:6825/2009632843] boot 2024-03-04T09:27:50.585 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:27:50 smithi052 ceph-mon[32473]: osdmap e80: 12 total, 12 up, 12 in 2024-03-04T09:27:50.585 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:27:50 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-04T09:27:50.585 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:27:50 smithi052 ceph-mon[32473]: from='client.? 172.21.15.52:0/1685560111' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-03-04T09:27:50.631 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:27:50 smithi106 ceph-mon[36484]: purged_snaps scrub starts 2024-03-04T09:27:50.631 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:27:50 smithi106 ceph-mon[36484]: purged_snaps scrub ok 2024-03-04T09:27:50.631 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:27:50 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-04T09:27:50.631 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:27:50 smithi106 ceph-mon[36484]: osd.11 [v2:172.21.15.191:6824/2009632843,v1:172.21.15.191:6825/2009632843] boot 2024-03-04T09:27:50.631 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:27:50 smithi106 ceph-mon[36484]: osdmap e80: 12 total, 12 up, 12 in 2024-03-04T09:27:50.632 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:27:50 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-04T09:27:50.632 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:27:50 smithi106 ceph-mon[36484]: from='client.? 172.21.15.52:0/1685560111' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-03-04T09:27:51.100 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":80,"num_osds":12,"num_up_osds":12,"osd_up_since":1709544469,"num_in_osds":12,"osd_in_since":1709544450,"num_remapped_pgs":0} 2024-03-04T09:27:51.101 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd shell --fsid 4e059f08-da08-11ee-95c6-87774f69a715 -- ceph osd dump --format=json 2024-03-04T09:27:51.335 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:27:51 smithi052 ceph-mon[32473]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail 2024-03-04T09:27:51.335 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:27:51 smithi052 ceph-mon[32473]: osdmap e81: 12 total, 12 up, 12 in 2024-03-04T09:27:51.335 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:27:51 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:27:51.335 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:27:51 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:27:51.335 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:27:51 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-04T09:27:51.335 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:27:51 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-04T09:27:51.335 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:27:51 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-04T09:27:51.335 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:27:51 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-04T09:27:51.335 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:27:51 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:27:51.335 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:27:51 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T09:27:51.335 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:27:51 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T09:27:51.336 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:27:51 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:27:51.345 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/4e059f08-da08-11ee-95c6-87774f69a715/mon.a/config 2024-03-04T09:27:51.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:27:51 smithi191 ceph-mon[36816]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail 2024-03-04T09:27:51.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:27:51 smithi191 ceph-mon[36816]: osdmap e81: 12 total, 12 up, 12 in 2024-03-04T09:27:51.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:27:51 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:27:51.587 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:27:51 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:27:51.587 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:27:51 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-04T09:27:51.587 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:27:51 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-04T09:27:51.587 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:27:51 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-04T09:27:51.587 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:27:51 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-04T09:27:51.587 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:27:51 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:27:51.588 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:27:51 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T09:27:51.588 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:27:51 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T09:27:51.588 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:27:51 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:27:51.631 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:27:51 smithi106 ceph-mon[36484]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail 2024-03-04T09:27:51.631 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:27:51 smithi106 ceph-mon[36484]: osdmap e81: 12 total, 12 up, 12 in 2024-03-04T09:27:51.631 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:27:51 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:27:51.632 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:27:51 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:27:51.632 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:27:51 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-04T09:27:51.632 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:27:51 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-04T09:27:51.632 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:27:51 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-04T09:27:51.632 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:27:51 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-04T09:27:51.632 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:27:51 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:27:51.632 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:27:51 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T09:27:51.632 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:27:51 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T09:27:51.632 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:27:51 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:27:52.471 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:27:52 smithi052 ceph-mon[32473]: Detected new or changed devices on smithi191 2024-03-04T09:27:52.472 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:27:52 smithi052 ceph-mon[32473]: Adjusting osd_memory_target on smithi191 to 4270M 2024-03-04T09:27:52.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:27:52 smithi191 ceph-mon[36816]: Detected new or changed devices on smithi191 2024-03-04T09:27:52.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:27:52 smithi191 ceph-mon[36816]: Adjusting osd_memory_target on smithi191 to 4270M 2024-03-04T09:27:52.631 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:27:52 smithi106 ceph-mon[36484]: Detected new or changed devices on smithi191 2024-03-04T09:27:52.631 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:27:52 smithi106 ceph-mon[36484]: Adjusting osd_memory_target on smithi191 to 4270M 2024-03-04T09:27:53.120 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:27:53.120+0000 7f0bde1bf700 1 -- 172.21.15.52:0/295075695 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0bd8101960 msgr2=0x7f0bd810de40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:27:53.121 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:27:53.120+0000 7f0bde1bf700 1 --2- 172.21.15.52:0/295075695 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0bd8101960 0x7f0bd810de40 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f0bd000b0a0 tx=0x7f0bd000b3b0 comp rx=0 tx=0).stop 2024-03-04T09:27:53.121 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:27:53.120+0000 7f0bde1bf700 1 -- 172.21.15.52:0/295075695 shutdown_connections 2024-03-04T09:27:53.121 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:27:53.120+0000 7f0bde1bf700 1 --2- 172.21.15.52:0/295075695 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0bd8107000 0x7f0bd81073e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:27:53.121 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:27:53.120+0000 7f0bde1bf700 1 --2- 172.21.15.52:0/295075695 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0bd8101960 0x7f0bd810de40 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:27:53.121 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:27:53.120+0000 7f0bde1bf700 1 --2- 172.21.15.52:0/295075695 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0bd8100fc0 0x7f0bd8101420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:27:53.121 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:27:53.120+0000 7f0bde1bf700 1 -- 172.21.15.52:0/295075695 >> 172.21.15.52:0/295075695 conn(0x7f0bd8077ef0 msgr2=0x7f0bd8078300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:27:53.122 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:27:53.120+0000 7f0bde1bf700 1 -- 172.21.15.52:0/295075695 shutdown_connections 2024-03-04T09:27:53.122 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:27:53.120+0000 7f0bde1bf700 1 -- 172.21.15.52:0/295075695 wait complete. 2024-03-04T09:27:53.122 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:27:53.121+0000 7f0bde1bf700 1 Processor -- start 2024-03-04T09:27:53.122 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:27:53.121+0000 7f0bde1bf700 1 -- start start 2024-03-04T09:27:53.122 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:27:53.121+0000 7f0bde1bf700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0bd8100fc0 0x7f0bd8199740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:27:53.122 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:27:53.121+0000 7f0bde1bf700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0bd8101960 0x7f0bd8199c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:27:53.123 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:27:53.121+0000 7f0bde1bf700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0bd8107000 0x7f0bd81a47c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:27:53.123 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:27:53.121+0000 7f0bde1bf700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bd8111300 con 0x7f0bd8107000 2024-03-04T09:27:53.123 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:27:53.121+0000 7f0bde1bf700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bd8111180 con 0x7f0bd8100fc0 2024-03-04T09:27:53.123 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:27:53.121+0000 7f0bde1bf700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bd8111480 con 0x7f0bd8101960 2024-03-04T09:27:53.123 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:27:53.121+0000 7f0bd7fff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0bd8107000 0x7f0bd81a47c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:27:53.123 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:27:53.121+0000 7f0bd7fff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0bd8107000 0x7f0bd81a47c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:46648/0 (socket says 172.21.15.52:46648) 2024-03-04T09:27:53.123 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:27:53.121+0000 7f0bd7fff700 1 -- 172.21.15.52:0/126199939 learned_addr learned my addr 172.21.15.52:0/126199939 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-04T09:27:53.124 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:27:53.121+0000 7f0bd6ffd700 1 --2- 172.21.15.52:0/126199939 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0bd8101960 0x7f0bd8199c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:27:53.124 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:27:53.121+0000 7f0bd77fe700 1 --2- 172.21.15.52:0/126199939 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0bd8100fc0 0x7f0bd8199740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:27:53.124 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:27:53.121+0000 7f0bd7fff700 1 -- 172.21.15.52:0/126199939 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0bd8101960 msgr2=0x7f0bd8199c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:27:53.124 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:27:53.121+0000 7f0bd7fff700 1 --2- 172.21.15.52:0/126199939 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0bd8101960 0x7f0bd8199c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:27:53.124 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:27:53.121+0000 7f0bd7fff700 1 -- 172.21.15.52:0/126199939 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0bd8100fc0 msgr2=0x7f0bd8199740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:27:53.124 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:27:53.121+0000 7f0bd7fff700 1 --2- 172.21.15.52:0/126199939 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0bd8100fc0 0x7f0bd8199740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:27:53.124 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:27:53.121+0000 7f0bd7fff700 1 -- 172.21.15.52:0/126199939 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0bd81a4d00 con 0x7f0bd8107000 2024-03-04T09:27:53.125 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:27:53.122+0000 7f0bd7fff700 1 --2- 172.21.15.52:0/126199939 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0bd8107000 0x7f0bd81a47c0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f0bd00062a0 tx=0x7f0bd0004d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:27:53.125 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:27:53.122+0000 7f0bd4ff9700 1 -- 172.21.15.52:0/126199939 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bd0021070 con 0x7f0bd8107000 2024-03-04T09:27:53.125 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:27:53.122+0000 7f0bd4ff9700 1 -- 172.21.15.52:0/126199939 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0bd0003c00 con 0x7f0bd8107000 2024-03-04T09:27:53.125 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:27:53.122+0000 7f0bde1bf700 1 -- 172.21.15.52:0/126199939 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0bd81a4ed0 con 0x7f0bd8107000 2024-03-04T09:27:53.125 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:27:53.122+0000 7f0bd4ff9700 1 -- 172.21.15.52:0/126199939 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bd0004880 con 0x7f0bd8107000 2024-03-04T09:27:53.125 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:27:53.122+0000 7f0bde1bf700 1 -- 172.21.15.52:0/126199939 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0bd81a5330 con 0x7f0bd8107000 2024-03-04T09:27:53.125 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:27:53.123+0000 7f0bde1bf700 1 -- 172.21.15.52:0/126199939 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0bd8068d80 con 0x7f0bd8107000 2024-03-04T09:27:53.127 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:27:53.124+0000 7f0bd4ff9700 1 -- 172.21.15.52:0/126199939 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 16) v1 ==== 136756+0+0 (secure 0 0 0) 0x7f0bd0007980 con 0x7f0bd8107000 2024-03-04T09:27:53.127 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:27:53.124+0000 7f0bd4ff9700 1 --2- 172.21.15.52:0/126199939 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f0bc00a1e10 0x7f0bc00a42d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:27:53.127 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:27:53.125+0000 7f0bd77fe700 1 --2- 172.21.15.52:0/126199939 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f0bc00a1e10 0x7f0bc00a42d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:27:53.127 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:27:53.125+0000 7f0bd4ff9700 1 -- 172.21.15.52:0/126199939 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f0bd00cec20 con 0x7f0bd8107000 2024-03-04T09:27:53.128 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:27:53.125+0000 7f0bd77fe700 1 --2- 172.21.15.52:0/126199939 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f0bc00a1e10 0x7f0bc00a42d0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f0bc8004630 tx=0x7f0bc800b3c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:27:53.128 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:27:53.129+0000 7f0bd4ff9700 1 -- 172.21.15.52:0/126199939 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f0bd009c580 con 0x7f0bd8107000 2024-03-04T09:27:53.335 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:27:53 smithi052 ceph-mon[32473]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-04T09:27:53.335 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:27:53.339+0000 7f0bde1bf700 1 -- 172.21.15.52:0/126199939 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0bd802d9b0 con 0x7f0bd8107000 2024-03-04T09:27:53.340 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:27:53.340+0000 7f0bd4ff9700 1 -- 172.21.15.52:0/126199939 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) v1 ==== 74+0+18492 (secure 0 0 0) 0x7f0bd009be80 con 0x7f0bd8107000 2024-03-04T09:27:53.341 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-03-04T09:27:53.341 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":81,"fsid":"4e059f08-da08-11ee-95c6-87774f69a715","created":"2024-03-04T09:20:46.104963+0000","modified":"2024-03-04T09:27:50.268367+0000","last_up_change":"2024-03-04T09:27:49.261365+0000","last_in_change":"2024-03-04T09:27:30.024964+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-04T09:24:01.742461+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"583172be-602f-4bf4-bfcb-42663cf5b277","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6802","nonce":1371221397},{"type":"v1","addr":"172.21.15.52:6803","nonce":1371221397}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6804","nonce":1371221397},{"type":"v1","addr":"172.21.15.52:6805","nonce":1371221397}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6808","nonce":1371221397},{"type":"v1","addr":"172.21.15.52:6809","nonce":1371221397}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6806","nonce":1371221397},{"type":"v1","addr":"172.21.15.52:6807","nonce":1371221397}]},"public_addr":"172.21.15.52:6803/1371221397","cluster_addr":"172.21.15.52:6805/1371221397","heartbeat_back_addr":"172.21.15.52:6809/1371221397","heartbeat_front_addr":"172.21.15.52:6807/1371221397","state":["exists","up"]},{"osd":1,"uuid":"5ae0081c-a2f6-46eb-ace0-0368208377b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":2540865537},{"type":"v1","addr":"172.21.15.106:6801","nonce":2540865537}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":2540865537},{"type":"v1","addr":"172.21.15.106:6803","nonce":2540865537}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":2540865537},{"type":"v1","addr":"172.21.15.106:6807","nonce":2540865537}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":2540865537},{"type":"v1","addr":"172.21.15.106:6805","nonce":2540865537}]},"public_addr":"172.21.15.106:6801/2540865537","cluster_addr":"172.21.15.106:6803/2540865537","heartbeat_back_addr":"172.21.15.106:6807/2540865537","heartbeat_front_addr":"172.21.15.106:6805/2540865537","state":["exists","up"]},{"osd":2,"uuid":"1948bac7-2bb2-4218-8b93-4e4743a0de04","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":1377083164},{"type":"v1","addr":"172.21.15.191:6801","nonce":1377083164}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":1377083164},{"type":"v1","addr":"172.21.15.191:6803","nonce":1377083164}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":1377083164},{"type":"v1","addr":"172.21.15.191:6807","nonce":1377083164}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":1377083164},{"type":"v1","addr":"172.21.15.191:6805","nonce":1377083164}]},"public_addr":"172.21.15.191:6801/1377083164","cluster_addr":"172.21.15.191:6803/1377083164","heartbeat_back_addr":"172.21.15.191:6807/1377083164","heartbeat_front_addr":"172.21.15.191:6805/1377083164","state":["exists","up"]},{"osd":3,"uuid":"16c9e9fe-7167-4992-bf12-1a7398e0498a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6810","nonce":2388571810},{"type":"v1","addr":"172.21.15.52:6811","nonce":2388571810}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6812","nonce":2388571810},{"type":"v1","addr":"172.21.15.52:6813","nonce":2388571810}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6816","nonce":2388571810},{"type":"v1","addr":"172.21.15.52:6817","nonce":2388571810}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6814","nonce":2388571810},{"type":"v1","addr":"172.21.15.52:6815","nonce":2388571810}]},"public_addr":"172.21.15.52:6811/2388571810","cluster_addr":"172.21.15.52:6813/2388571810","heartbeat_back_addr":"172.21.15.52:6817/2388571810","heartbeat_front_addr":"172.21.15.52:6815/2388571810","state":["exists","up"]},{"osd":4,"uuid":"c8ef48c7-7c78-44d3-a9e5-99ec750abd3d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6808","nonce":3465868015},{"type":"v1","addr":"172.21.15.106:6809","nonce":3465868015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6810","nonce":3465868015},{"type":"v1","addr":"172.21.15.106:6811","nonce":3465868015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6814","nonce":3465868015},{"type":"v1","addr":"172.21.15.106:6815","nonce":3465868015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6812","nonce":3465868015},{"type":"v1","addr":"172.21.15.106:6813","nonce":3465868015}]},"public_addr":"172.21.15.106:6809/3465868015","cluster_addr":"172.21.15.106:6811/3465868015","heartbeat_back_addr":"172.21.15.106:6815/3465868015","heartbeat_front_addr":"172.21.15.106:6813/3465868015","state":["exists","up"]},{"osd":5,"uuid":"9e41824a-c5a2-4f88-be50-2ba1dceb3220","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":1485092643},{"type":"v1","addr":"172.21.15.191:6809","nonce":1485092643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":1485092643},{"type":"v1","addr":"172.21.15.191:6811","nonce":1485092643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":1485092643},{"type":"v1","addr":"172.21.15.191:6815","nonce":1485092643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":1485092643},{"type":"v1","addr":"172.21.15.191:6813","nonce":1485092643}]},"public_addr":"172.21.15.191:6809/1485092643","cluster_addr":"172.21.15.191:6811/1485092643","heartbeat_back_addr":"172.21.15.191:6815/1485092643","heartbeat_front_addr":"172.21.15.191:6813/1485092643","state":["exists","up"]},{"osd":6,"uuid":"b35dccee-ec16-452e-ac69-3ac2fb7b27fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6818","nonce":1989797856},{"type":"v1","addr":"172.21.15.52:6819","nonce":1989797856}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6820","nonce":1989797856},{"type":"v1","addr":"172.21.15.52:6821","nonce":1989797856}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6824","nonce":1989797856},{"type":"v1","addr":"172.21.15.52:6825","nonce":1989797856}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6822","nonce":1989797856},{"type":"v1","addr":"172.21.15.52:6823","nonce":1989797856}]},"public_addr":"172.21.15.52:6819/1989797856","cluster_addr":"172.21.15.52:6821/1989797856","heartbeat_back_addr":"172.21.15.52:6825/1989797856","heartbeat_front_addr":"172.21.15.52:6823/1989797856","state":["exists","up"]},{"osd":7,"uuid":"71303e3d-14ec-4cbd-b19c-67645a1de3c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6816","nonce":1450870329},{"type":"v1","addr":"172.21.15.106:6817","nonce":1450870329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":1450870329},{"type":"v1","addr":"172.21.15.106:6819","nonce":1450870329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":1450870329},{"type":"v1","addr":"172.21.15.106:6823","nonce":1450870329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6820","nonce":1450870329},{"type":"v1","addr":"172.21.15.106:6821","nonce":1450870329}]},"public_addr":"172.21.15.106:6817/1450870329","cluster_addr":"172.21.15.106:6819/1450870329","heartbeat_back_addr":"172.21.15.106:6823/1450870329","heartbeat_front_addr":"172.21.15.106:6821/1450870329","state":["exists","up"]},{"osd":8,"uuid":"6f5786c4-4402-435a-bdad-adff46fe9473","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":1114119072},{"type":"v1","addr":"172.21.15.191:6817","nonce":1114119072}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6818","nonce":1114119072},{"type":"v1","addr":"172.21.15.191:6819","nonce":1114119072}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":1114119072},{"type":"v1","addr":"172.21.15.191:6823","nonce":1114119072}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":1114119072},{"type":"v1","addr":"172.21.15.191:6821","nonce":1114119072}]},"public_addr":"172.21.15.191:6817/1114119072","cluster_addr":"172.21.15.191:6819/1114119072","heartbeat_back_addr":"172.21.15.191:6823/1114119072","heartbeat_front_addr":"172.21.15.191:6821/1114119072","state":["exists","up"]},{"osd":9,"uuid":"062bbaa5-1227-4684-b859-450f5594a90c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6826","nonce":3485821914},{"type":"v1","addr":"172.21.15.52:6827","nonce":3485821914}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6828","nonce":3485821914},{"type":"v1","addr":"172.21.15.52:6829","nonce":3485821914}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6832","nonce":3485821914},{"type":"v1","addr":"172.21.15.52:6833","nonce":3485821914}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6830","nonce":3485821914},{"type":"v1","addr":"172.21.15.52:6831","nonce":3485821914}]},"public_addr":"172.21.15.52:6827/3485821914","cluster_addr":"172.21.15.52:6829/3485821914","heartbeat_back_addr":"172.21.15.52:6833/3485821914","heartbeat_front_addr":"172.21.15.52:6831/3485821914","state":["exists","up"]},{"osd":10,"uuid":"1a285a5a-f793-48da-a8eb-5326d9528cd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":3815586576},{"type":"v1","addr":"172.21.15.106:6825","nonce":3815586576}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6826","nonce":3815586576},{"type":"v1","addr":"172.21.15.106:6827","nonce":3815586576}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":3815586576},{"type":"v1","addr":"172.21.15.106:6831","nonce":3815586576}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":3815586576},{"type":"v1","addr":"172.21.15.106:6829","nonce":3815586576}]},"public_addr":"172.21.15.106:6825/3815586576","cluster_addr":"172.21.15.106:6827/3815586576","heartbeat_back_addr":"172.21.15.106:6831/3815586576","heartbeat_front_addr":"172.21.15.106:6829/3815586576","state":["exists","up"]},{"osd":11,"uuid":"1f8b9a43-57fa-41f4-9969-3b803a74bf45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":2009632843},{"type":"v1","addr":"172.21.15.191:6825","nonce":2009632843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":2009632843},{"type":"v1","addr":"172.21.15.191:6827","nonce":2009632843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":2009632843},{"type":"v1","addr":"172.21.15.191:6831","nonce":2009632843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":2009632843},{"type":"v1","addr":"172.21.15.191:6829","nonce":2009632843}]},"public_addr":"172.21.15.191:6825/2009632843","cluster_addr":"172.21.15.191:6827/2009632843","heartbeat_back_addr":"172.21.15.191:6831/2009632843","heartbeat_front_addr":"172.21.15.191:6829/2009632843","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T09:23:10.894763+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T09:23:34.420055+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T09:23:58.288324+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T09:24:23.673457+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T09:24:48.208561+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T09:25:13.405473+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T09:25:38.670915+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T09:26:04.451257+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T09:26:30.281380+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T09:26:56.543143+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T09:27:22.412930+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T09:27:48.003866+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.52:0/4082645718":"2024-03-05T09:21:39.401935+0000","172.21.15.52:0/1271949988":"2024-03-05T09:21:39.401935+0000","172.21.15.52:6801/2379361333":"2024-03-05T09:21:12.760938+0000","172.21.15.52:0/497075075":"2024-03-05T09:21:12.760938+0000","172.21.15.52:6800/2379361333":"2024-03-05T09:21:12.760938+0000","172.21.15.52:0/568242577":"2024-03-05T09:21:12.760938+0000","172.21.15.52:6801/2425730636":"2024-03-05T09:21:39.401935+0000","172.21.15.52:6800/2425730636":"2024-03-05T09:21:39.401935+0000","172.21.15.52:0/2743084243":"2024-03-05T09:21:39.401935+0000","172.21.15.52:0/1035878034":"2024-03-05T09:21:12.760938+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-04T09:27:53.343 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:27:53.342+0000 7f0bde1bf700 1 -- 172.21.15.52:0/126199939 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f0bc00a1e10 msgr2=0x7f0bc00a42d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:27:53.343 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:27:53.342+0000 7f0bde1bf700 1 --2- 172.21.15.52:0/126199939 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f0bc00a1e10 0x7f0bc00a42d0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f0bc8004630 tx=0x7f0bc800b3c0 comp rx=0 tx=0).stop 2024-03-04T09:27:53.343 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:27:53.342+0000 7f0bde1bf700 1 -- 172.21.15.52:0/126199939 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0bd8107000 msgr2=0x7f0bd81a47c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:27:53.343 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:27:53.342+0000 7f0bde1bf700 1 --2- 172.21.15.52:0/126199939 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0bd8107000 0x7f0bd81a47c0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f0bd00062a0 tx=0x7f0bd0004d70 comp rx=0 tx=0).stop 2024-03-04T09:27:53.344 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:27:53.342+0000 7f0bde1bf700 1 -- 172.21.15.52:0/126199939 shutdown_connections 2024-03-04T09:27:53.344 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:27:53.342+0000 7f0bde1bf700 1 --2- 172.21.15.52:0/126199939 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f0bc00a1e10 0x7f0bc00a42d0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:27:53.344 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:27:53.342+0000 7f0bde1bf700 1 --2- 172.21.15.52:0/126199939 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0bd8107000 0x7f0bd81a47c0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:27:53.344 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:27:53.342+0000 7f0bde1bf700 1 --2- 172.21.15.52:0/126199939 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0bd8100fc0 0x7f0bd8199740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:27:53.344 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:27:53.342+0000 7f0bde1bf700 1 --2- 172.21.15.52:0/126199939 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0bd8101960 0x7f0bd8199c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:27:53.344 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:27:53.342+0000 7f0bde1bf700 1 -- 172.21.15.52:0/126199939 >> 172.21.15.52:0/126199939 conn(0x7f0bd8077ef0 msgr2=0x7f0bd80ffcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:27:53.345 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:27:53.342+0000 7f0bde1bf700 1 -- 172.21.15.52:0/126199939 shutdown_connections 2024-03-04T09:27:53.345 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:27:53.342+0000 7f0bde1bf700 1 -- 172.21.15.52:0/126199939 wait complete. 2024-03-04T09:27:53.591 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:27:53 smithi191 ceph-mon[36816]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-04T09:27:53.634 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:27:53 smithi106 ceph-mon[36484]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-04T09:27:53.967 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-04T09:24:01.742461+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-04T09:27:53.967 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd shell --fsid 4e059f08-da08-11ee-95c6-87774f69a715 -- ceph osd pool get .mgr pg_num 2024-03-04T09:27:54.208 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/4e059f08-da08-11ee-95c6-87774f69a715/mon.a/config 2024-03-04T09:27:54.585 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:27:54 smithi052 ceph-mon[32473]: from='client.? 172.21.15.52:0/126199939' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-04T09:27:54.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:27:54 smithi191 ceph-mon[36816]: from='client.? 172.21.15.52:0/126199939' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-04T09:27:54.631 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:27:54 smithi106 ceph-mon[36484]: from='client.? 172.21.15.52:0/126199939' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-04T09:27:55.585 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:27:55 smithi052 ceph-mon[32473]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-04T09:27:55.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:27:55 smithi191 ceph-mon[36816]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-04T09:27:55.631 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:27:55 smithi106 ceph-mon[36484]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-04T09:27:56.087 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:27:56.087+0000 7fa3f34f8700 1 -- 172.21.15.52:0/2421232277 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa3ec06b410 msgr2=0x7fa3ec106dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:27:56.087 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:27:56.087+0000 7fa3f34f8700 1 --2- 172.21.15.52:0/2421232277 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa3ec06b410 0x7fa3ec106dd0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fa3dc009990 tx=0x7fa3dc009ca0 comp rx=0 tx=0).stop 2024-03-04T09:27:56.087 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:27:56.087+0000 7fa3f34f8700 1 -- 172.21.15.52:0/2421232277 shutdown_connections 2024-03-04T09:27:56.087 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:27:56.087+0000 7fa3f34f8700 1 --2- 172.21.15.52:0/2421232277 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa3ec107500 0x7fa3ec10f080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:27:56.087 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:27:56.087+0000 7fa3f34f8700 1 --2- 172.21.15.52:0/2421232277 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa3ec06aa60 0x7fa3ec06ae40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:27:56.087 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:27:56.087+0000 7fa3f34f8700 1 --2- 172.21.15.52:0/2421232277 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa3ec06b410 0x7fa3ec106dd0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:27:56.087 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:27:56.087+0000 7fa3f34f8700 1 -- 172.21.15.52:0/2421232277 >> 172.21.15.52:0/2421232277 conn(0x7fa3ec077e20 msgr2=0x7fa3ec078230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:27:56.087 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:27:56.087+0000 7fa3f34f8700 1 -- 172.21.15.52:0/2421232277 shutdown_connections 2024-03-04T09:27:56.088 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:27:56.088+0000 7fa3f34f8700 1 -- 172.21.15.52:0/2421232277 wait complete. 2024-03-04T09:27:56.088 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:27:56.088+0000 7fa3f34f8700 1 Processor -- start 2024-03-04T09:27:56.088 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:27:56.088+0000 7fa3f34f8700 1 -- start start 2024-03-04T09:27:56.088 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:27:56.088+0000 7fa3f34f8700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa3ec06aa60 0x7fa3ec101370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:27:56.088 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:27:56.088+0000 7fa3f34f8700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa3ec06b410 0x7fa3ec1018b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:27:56.088 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:27:56.088+0000 7fa3f34f8700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa3ec107500 0x7fa3ec101df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:27:56.088 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:27:56.088+0000 7fa3f34f8700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3ec112440 con 0x7fa3ec06b410 2024-03-04T09:27:56.088 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:27:56.088+0000 7fa3f34f8700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3ec1122c0 con 0x7fa3ec107500 2024-03-04T09:27:56.088 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:27:56.088+0000 7fa3f34f8700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3ec1125c0 con 0x7fa3ec06aa60 2024-03-04T09:27:56.088 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:27:56.088+0000 7fa3f0a93700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa3ec06b410 0x7fa3ec1018b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:27:56.088 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:27:56.088+0000 7fa3f0a93700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa3ec06b410 0x7fa3ec1018b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:33978/0 (socket says 172.21.15.52:33978) 2024-03-04T09:27:56.089 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:27:56.088+0000 7fa3f0a93700 1 -- 172.21.15.52:0/3087048787 learned_addr learned my addr 172.21.15.52:0/3087048787 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-04T09:27:56.089 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:27:56.088+0000 7fa3f1294700 1 --2- 172.21.15.52:0/3087048787 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa3ec06aa60 0x7fa3ec101370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:27:56.089 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:27:56.089+0000 7fa3f1a95700 1 --2- 172.21.15.52:0/3087048787 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa3ec107500 0x7fa3ec101df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:27:56.089 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:27:56.089+0000 7fa3f0a93700 1 -- 172.21.15.52:0/3087048787 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa3ec06aa60 msgr2=0x7fa3ec101370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:27:56.089 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:27:56.089+0000 7fa3f0a93700 1 --2- 172.21.15.52:0/3087048787 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa3ec06aa60 0x7fa3ec101370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:27:56.089 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:27:56.089+0000 7fa3f0a93700 1 -- 172.21.15.52:0/3087048787 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa3ec107500 msgr2=0x7fa3ec101df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:27:56.089 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:27:56.089+0000 7fa3f0a93700 1 --2- 172.21.15.52:0/3087048787 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa3ec107500 0x7fa3ec101df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:27:56.089 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:27:56.089+0000 7fa3f0a93700 1 -- 172.21.15.52:0/3087048787 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3ec1a6d90 con 0x7fa3ec06b410 2024-03-04T09:27:56.089 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:27:56.089+0000 7fa3f1a95700 1 --2- 172.21.15.52:0/3087048787 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa3ec107500 0x7fa3ec101df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T09:27:56.089 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:27:56.089+0000 7fa3f0a93700 1 --2- 172.21.15.52:0/3087048787 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa3ec06b410 0x7fa3ec1018b0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fa3dc000c00 tx=0x7fa3dc0047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:27:56.089 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:27:56.089+0000 7fa3e27fc700 1 -- 172.21.15.52:0/3087048787 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3dc027070 con 0x7fa3ec06b410 2024-03-04T09:27:56.089 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:27:56.089+0000 7fa3e27fc700 1 -- 172.21.15.52:0/3087048787 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa3dc003780 con 0x7fa3ec06b410 2024-03-04T09:27:56.089 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:27:56.089+0000 7fa3f34f8700 1 -- 172.21.15.52:0/3087048787 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3ec1a7020 con 0x7fa3ec06b410 2024-03-04T09:27:56.089 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:27:56.089+0000 7fa3e27fc700 1 -- 172.21.15.52:0/3087048787 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3dc022db0 con 0x7fa3ec06b410 2024-03-04T09:27:56.090 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:27:56.089+0000 7fa3f34f8700 1 -- 172.21.15.52:0/3087048787 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3ec1a74d0 con 0x7fa3ec06b410 2024-03-04T09:27:56.090 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:27:56.090+0000 7fa3f34f8700 1 -- 172.21.15.52:0/3087048787 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa3ec103880 con 0x7fa3ec06b410 2024-03-04T09:27:56.091 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:27:56.092+0000 7fa3e27fc700 1 -- 172.21.15.52:0/3087048787 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 16) v1 ==== 136756+0+0 (secure 0 0 0) 0x7fa3dc02a070 con 0x7fa3ec06b410 2024-03-04T09:27:56.091 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:27:56.093+0000 7fa3e27fc700 1 --2- 172.21.15.52:0/3087048787 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7fa3d80a1e90 0x7fa3d80a4350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:27:56.091 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:27:56.093+0000 7fa3f1294700 1 --2- 172.21.15.52:0/3087048787 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7fa3d80a1e90 0x7fa3d80a4350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:27:56.091 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:27:56.093+0000 7fa3e27fc700 1 -- 172.21.15.52:0/3087048787 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fa3dc0ce910 con 0x7fa3ec06b410 2024-03-04T09:27:56.091 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:27:56.093+0000 7fa3f1294700 1 --2- 172.21.15.52:0/3087048787 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7fa3d80a1e90 0x7fa3d80a4350 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fa3e8005950 tx=0x7fa3e800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:27:56.095 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:27:56.096+0000 7fa3e27fc700 1 -- 172.21.15.52:0/3087048787 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fa3dc013070 con 0x7fa3ec06b410 2024-03-04T09:27:56.296 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:27:56.299+0000 7fa3f34f8700 1 -- 172.21.15.52:0/3087048787 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fa3ec02d9b0 con 0x7fa3ec06b410 2024-03-04T09:27:56.296 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:27:56.299+0000 7fa3e27fc700 1 -- 172.21.15.52:0/3087048787 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v81) v1 ==== 93+0+10 (secure 0 0 0) 0x7fa3dc0bb070 con 0x7fa3ec06b410 2024-03-04T09:27:56.296 INFO:teuthology.orchestra.run.smithi052.stdout:pg_num: 1 2024-03-04T09:27:56.298 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:27:56.302+0000 7fa3f34f8700 1 -- 172.21.15.52:0/3087048787 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7fa3d80a1e90 msgr2=0x7fa3d80a4350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:27:56.299 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:27:56.302+0000 7fa3f34f8700 1 --2- 172.21.15.52:0/3087048787 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7fa3d80a1e90 0x7fa3d80a4350 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fa3e8005950 tx=0x7fa3e800b410 comp rx=0 tx=0).stop 2024-03-04T09:27:56.299 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:27:56.302+0000 7fa3f34f8700 1 -- 172.21.15.52:0/3087048787 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa3ec06b410 msgr2=0x7fa3ec1018b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:27:56.299 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:27:56.302+0000 7fa3f34f8700 1 --2- 172.21.15.52:0/3087048787 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa3ec06b410 0x7fa3ec1018b0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fa3dc000c00 tx=0x7fa3dc0047b0 comp rx=0 tx=0).stop 2024-03-04T09:27:56.299 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:27:56.302+0000 7fa3f34f8700 1 -- 172.21.15.52:0/3087048787 shutdown_connections 2024-03-04T09:27:56.299 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:27:56.302+0000 7fa3f34f8700 1 --2- 172.21.15.52:0/3087048787 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7fa3d80a1e90 0x7fa3d80a4350 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:27:56.299 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:27:56.302+0000 7fa3f34f8700 1 --2- 172.21.15.52:0/3087048787 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa3ec06aa60 0x7fa3ec101370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:27:56.299 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:27:56.302+0000 7fa3f34f8700 1 --2- 172.21.15.52:0/3087048787 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa3ec107500 0x7fa3ec101df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:27:56.299 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:27:56.302+0000 7fa3f34f8700 1 --2- 172.21.15.52:0/3087048787 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa3ec06b410 0x7fa3ec1018b0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:27:56.299 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:27:56.302+0000 7fa3f34f8700 1 -- 172.21.15.52:0/3087048787 >> 172.21.15.52:0/3087048787 conn(0x7fa3ec077e20 msgr2=0x7fa3ec0ff310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:27:56.299 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:27:56.302+0000 7fa3f34f8700 1 -- 172.21.15.52:0/3087048787 shutdown_connections 2024-03-04T09:27:56.299 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:27:56.302+0000 7fa3f34f8700 1 -- 172.21.15.52:0/3087048787 wait complete. 2024-03-04T09:27:56.978 INFO:tasks.cephadm:Adding mds.a on smithi052 2024-03-04T09:27:56.978 INFO:tasks.cephadm:Adding mds.d on smithi052 2024-03-04T09:27:56.978 INFO:tasks.cephadm:Adding mds.g on smithi052 2024-03-04T09:27:56.978 INFO:tasks.cephadm:Adding mds.j on smithi052 2024-03-04T09:27:56.978 INFO:tasks.cephadm:Adding mds.b on smithi106 2024-03-04T09:27:56.978 INFO:tasks.cephadm:Adding mds.e on smithi106 2024-03-04T09:27:56.978 INFO:tasks.cephadm:Adding mds.h on smithi106 2024-03-04T09:27:56.978 INFO:tasks.cephadm:Adding mds.k on smithi106 2024-03-04T09:27:56.979 INFO:tasks.cephadm:Adding mds.c on smithi191 2024-03-04T09:27:56.979 INFO:tasks.cephadm:Adding mds.f on smithi191 2024-03-04T09:27:56.979 INFO:tasks.cephadm:Adding mds.i on smithi191 2024-03-04T09:27:56.979 INFO:tasks.cephadm:Adding mds.l on smithi191 2024-03-04T09:27:56.979 DEBUG:teuthology.orchestra.run.smithi191:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4e059f08-da08-11ee-95c6-87774f69a715 -- ceph orch apply mds all '12;smithi052=a;smithi052=d;smithi052=g;smithi052=j;smithi106=b;smithi106=e;smithi106=h;smithi106=k;smithi191=c;smithi191=f;smithi191=i;smithi191=l' 2024-03-04T09:27:57.229 INFO:teuthology.orchestra.run.smithi191.stderr:Inferring config /var/lib/ceph/4e059f08-da08-11ee-95c6-87774f69a715/mon.c/config 2024-03-04T09:27:57.585 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:27:57 smithi052 ceph-mon[32473]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-04T09:27:57.585 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:27:57 smithi052 ceph-mon[32473]: from='client.? 172.21.15.52:0/3087048787' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-04T09:27:57.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:27:57 smithi191 ceph-mon[36816]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-04T09:27:57.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:27:57 smithi191 ceph-mon[36816]: from='client.? 172.21.15.52:0/3087048787' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-04T09:27:57.631 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:27:57 smithi106 ceph-mon[36484]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-04T09:27:57.631 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:27:57 smithi106 ceph-mon[36484]: from='client.? 172.21.15.52:0/3087048787' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-04T09:27:58.893 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:27:58.894+0000 7f5e025e8700 1 -- 172.21.15.191:0/3480949479 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5dfc101990 msgr2=0x7f5dfc101df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:27:58.893 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:27:58.894+0000 7f5e025e8700 1 --2- 172.21.15.191:0/3480949479 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5dfc101990 0x7f5dfc101df0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f5df0009990 tx=0x7f5df0009ca0 comp rx=0 tx=0).stop 2024-03-04T09:27:58.893 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:27:58.894+0000 7f5e025e8700 1 -- 172.21.15.191:0/3480949479 shutdown_connections 2024-03-04T09:27:58.893 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:27:58.894+0000 7f5e025e8700 1 --2- 172.21.15.191:0/3480949479 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5dfc102330 0x7f5dfc108be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:27:58.893 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:27:58.894+0000 7f5e025e8700 1 --2- 172.21.15.191:0/3480949479 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5dfc100790 0x7f5dfc100bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:27:58.893 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:27:58.894+0000 7f5e025e8700 1 --2- 172.21.15.191:0/3480949479 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5dfc101990 0x7f5dfc101df0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:27:58.893 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:27:58.894+0000 7f5e025e8700 1 -- 172.21.15.191:0/3480949479 >> 172.21.15.191:0/3480949479 conn(0x7f5dfc0fbd50 msgr2=0x7f5dfc0fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:27:58.893 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:27:58.894+0000 7f5e025e8700 1 -- 172.21.15.191:0/3480949479 shutdown_connections 2024-03-04T09:27:58.893 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:27:58.894+0000 7f5e025e8700 1 -- 172.21.15.191:0/3480949479 wait complete. 2024-03-04T09:27:58.894 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:27:58.894+0000 7f5e025e8700 1 Processor -- start 2024-03-04T09:27:58.894 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:27:58.894+0000 7f5e025e8700 1 -- start start 2024-03-04T09:27:58.894 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:27:58.895+0000 7f5e025e8700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5dfc100790 0x7f5dfc10a150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:27:58.894 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:27:58.895+0000 7f5e025e8700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5dfc101990 0x7f5dfc19d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:27:58.894 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:27:58.895+0000 7f5e025e8700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5dfc102330 0x7f5dfc19f6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:27:58.894 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:27:58.895+0000 7f5e025e8700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5dfc10cc70 con 0x7f5dfc100790 2024-03-04T09:27:58.894 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:27:58.895+0000 7f5e025e8700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5dfc10caf0 con 0x7f5dfc101990 2024-03-04T09:27:58.894 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:27:58.895+0000 7f5e025e8700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5dfc10cdf0 con 0x7f5dfc102330 2024-03-04T09:27:58.894 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:27:58.895+0000 7f5e00b85700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5dfc102330 0x7f5dfc19f6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:27:58.894 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:27:58.895+0000 7f5e00b85700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5dfc102330 0x7f5dfc19f6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.191:40624/0 (socket says 172.21.15.191:40624) 2024-03-04T09:27:58.894 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:27:58.895+0000 7f5e00b85700 1 -- 172.21.15.191:0/2819151049 learned_addr learned my addr 172.21.15.191:0/2819151049 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-03-04T09:27:58.894 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:27:58.895+0000 7f5dfbfff700 1 --2- 172.21.15.191:0/2819151049 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5dfc100790 0x7f5dfc10a150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:27:58.894 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:27:58.895+0000 7f5dfb7fe700 1 --2- 172.21.15.191:0/2819151049 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5dfc101990 0x7f5dfc19d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:27:58.895 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:27:58.895+0000 7f5e00b85700 1 -- 172.21.15.191:0/2819151049 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5dfc101990 msgr2=0x7f5dfc19d0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:27:58.895 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:27:58.895+0000 7f5e00b85700 1 --2- 172.21.15.191:0/2819151049 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5dfc101990 0x7f5dfc19d0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:27:58.895 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:27:58.895+0000 7f5e00b85700 1 -- 172.21.15.191:0/2819151049 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5dfc100790 msgr2=0x7f5dfc10a150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:27:58.895 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:27:58.895+0000 7f5e00b85700 1 --2- 172.21.15.191:0/2819151049 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5dfc100790 0x7f5dfc10a150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:27:58.895 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:27:58.895+0000 7f5e00b85700 1 -- 172.21.15.191:0/2819151049 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5dfc19ffc0 con 0x7f5dfc102330 2024-03-04T09:27:58.895 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:27:58.895+0000 7f5e00b85700 1 --2- 172.21.15.191:0/2819151049 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5dfc102330 0x7f5dfc19f6f0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f5df400ada0 tx=0x7f5df400c330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:27:58.895 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:27:58.895+0000 7f5df97fa700 1 -- 172.21.15.191:0/2819151049 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5df4017020 con 0x7f5dfc102330 2024-03-04T09:27:58.895 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:27:58.895+0000 7f5e025e8700 1 -- 172.21.15.191:0/2819151049 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5dfc1a02b0 con 0x7f5dfc102330 2024-03-04T09:27:58.895 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:27:58.895+0000 7f5df97fa700 1 -- 172.21.15.191:0/2819151049 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5df400a0d0 con 0x7f5dfc102330 2024-03-04T09:27:58.895 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:27:58.895+0000 7f5df97fa700 1 -- 172.21.15.191:0/2819151049 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5df4005480 con 0x7f5dfc102330 2024-03-04T09:27:58.895 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:27:58.896+0000 7f5e025e8700 1 -- 172.21.15.191:0/2819151049 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5dfc1a0870 con 0x7f5dfc102330 2024-03-04T09:27:58.896 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:27:58.897+0000 7f5e025e8700 1 -- 172.21.15.191:0/2819151049 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5dfc068d80 con 0x7f5dfc102330 2024-03-04T09:27:58.896 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:27:58.898+0000 7f5df97fa700 1 -- 172.21.15.191:0/2819151049 <== mon.1 v2:172.21.15.191:3300/0 4 ==== mgrmap(e 16) v1 ==== 136756+0+0 (secure 0 0 0) 0x7f5df4005620 con 0x7f5dfc102330 2024-03-04T09:27:58.897 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:27:58.898+0000 7f5df97fa700 1 --2- 172.21.15.191:0/2819151049 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f5de40a1e10 0x7f5de40a42d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:27:58.897 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:27:58.898+0000 7f5df97fa700 1 -- 172.21.15.191:0/2819151049 <== mon.1 v2:172.21.15.191:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f5df40cd090 con 0x7f5dfc102330 2024-03-04T09:27:58.897 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:27:58.898+0000 7f5dfbfff700 1 --2- 172.21.15.191:0/2819151049 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f5de40a1e10 0x7f5de40a42d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:27:58.897 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:27:58.899+0000 7f5dfbfff700 1 --2- 172.21.15.191:0/2819151049 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f5de40a1e10 0x7f5de40a42d0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f5dfc1017f0 tx=0x7f5dec009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:27:58.899 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:27:58.903+0000 7f5df97fa700 1 -- 172.21.15.191:0/2819151049 <== mon.1 v2:172.21.15.191:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f5df4021020 con 0x7f5dfc102330 2024-03-04T09:27:59.104 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:27:59.107+0000 7f5e025e8700 1 -- 172.21.15.191:0/2819151049 --> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] -- mgr_command(tid 0: {"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi052=a;smithi052=d;smithi052=g;smithi052=j;smithi106=b;smithi106=e;smithi106=h;smithi106=k;smithi191=c;smithi191=f;smithi191=i;smithi191=l", "target": ["mon-mgr", ""]}) v1 -- 0x7f5dfc062700 con 0x7f5de40a1e10 2024-03-04T09:27:59.112 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:27:59.115+0000 7f5df97fa700 1 -- 172.21.15.191:0/2819151049 <== mgr.14152 v2:172.21.15.52:6800/3036033923 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+28 (secure 0 0 0) 0x7f5dfc062700 con 0x7f5de40a1e10 2024-03-04T09:27:59.112 INFO:teuthology.orchestra.run.smithi191.stdout:Scheduled mds.all update... 2024-03-04T09:27:59.114 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:27:59.117+0000 7f5e025e8700 1 -- 172.21.15.191:0/2819151049 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f5de40a1e10 msgr2=0x7f5de40a42d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:27:59.114 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:27:59.117+0000 7f5e025e8700 1 --2- 172.21.15.191:0/2819151049 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f5de40a1e10 0x7f5de40a42d0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f5dfc1017f0 tx=0x7f5dec009190 comp rx=0 tx=0).stop 2024-03-04T09:27:59.114 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:27:59.117+0000 7f5e025e8700 1 -- 172.21.15.191:0/2819151049 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5dfc102330 msgr2=0x7f5dfc19f6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:27:59.114 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:27:59.117+0000 7f5e025e8700 1 --2- 172.21.15.191:0/2819151049 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5dfc102330 0x7f5dfc19f6f0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f5df400ada0 tx=0x7f5df400c330 comp rx=0 tx=0).stop 2024-03-04T09:27:59.115 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:27:59.117+0000 7f5e025e8700 1 -- 172.21.15.191:0/2819151049 shutdown_connections 2024-03-04T09:27:59.115 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:27:59.117+0000 7f5e025e8700 1 --2- 172.21.15.191:0/2819151049 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f5de40a1e10 0x7f5de40a42d0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:27:59.115 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:27:59.117+0000 7f5e025e8700 1 --2- 172.21.15.191:0/2819151049 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5dfc102330 0x7f5dfc19f6f0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:27:59.115 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:27:59.117+0000 7f5e025e8700 1 --2- 172.21.15.191:0/2819151049 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5dfc100790 0x7f5dfc10a150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:27:59.115 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:27:59.117+0000 7f5e025e8700 1 --2- 172.21.15.191:0/2819151049 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5dfc101990 0x7f5dfc19d0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:27:59.115 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:27:59.117+0000 7f5e025e8700 1 -- 172.21.15.191:0/2819151049 >> 172.21.15.191:0/2819151049 conn(0x7f5dfc0fbd50 msgr2=0x7f5dfc104c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:27:59.115 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:27:59.117+0000 7f5e025e8700 1 -- 172.21.15.191:0/2819151049 shutdown_connections 2024-03-04T09:27:59.115 INFO:teuthology.orchestra.run.smithi191.stderr:2024-03-04T09:27:59.117+0000 7f5e025e8700 1 -- 172.21.15.191:0/2819151049 wait complete. 2024-03-04T09:27:59.426 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:27:59 smithi191 ceph-mon[36816]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-04T09:27:59.427 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:27:59 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:27:59.427 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:27:59 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T09:27:59.585 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:27:59 smithi052 ceph-mon[32473]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-04T09:27:59.585 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:27:59 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:27:59.585 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:27:59 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T09:27:59.631 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:27:59 smithi106 ceph-mon[36484]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-04T09:27:59.631 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:27:59 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:27:59.631 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:27:59 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T09:27:59.747 DEBUG:teuthology.orchestra.run.smithi052:mds.a> sudo journalctl -f -n 0 -u ceph-4e059f08-da08-11ee-95c6-87774f69a715@mds.a.service 2024-03-04T09:27:59.749 DEBUG:teuthology.orchestra.run.smithi052:mds.d> sudo journalctl -f -n 0 -u ceph-4e059f08-da08-11ee-95c6-87774f69a715@mds.d.service 2024-03-04T09:27:59.793 DEBUG:teuthology.orchestra.run.smithi052:mds.g> sudo journalctl -f -n 0 -u ceph-4e059f08-da08-11ee-95c6-87774f69a715@mds.g.service 2024-03-04T09:27:59.836 DEBUG:teuthology.orchestra.run.smithi052:mds.j> sudo journalctl -f -n 0 -u ceph-4e059f08-da08-11ee-95c6-87774f69a715@mds.j.service 2024-03-04T09:27:59.880 DEBUG:teuthology.orchestra.run.smithi106:mds.b> sudo journalctl -f -n 0 -u ceph-4e059f08-da08-11ee-95c6-87774f69a715@mds.b.service 2024-03-04T09:27:59.882 DEBUG:teuthology.orchestra.run.smithi106:mds.e> sudo journalctl -f -n 0 -u ceph-4e059f08-da08-11ee-95c6-87774f69a715@mds.e.service 2024-03-04T09:27:59.925 DEBUG:teuthology.orchestra.run.smithi106:mds.h> sudo journalctl -f -n 0 -u ceph-4e059f08-da08-11ee-95c6-87774f69a715@mds.h.service 2024-03-04T09:27:59.968 DEBUG:teuthology.orchestra.run.smithi106:mds.k> sudo journalctl -f -n 0 -u ceph-4e059f08-da08-11ee-95c6-87774f69a715@mds.k.service 2024-03-04T09:28:00.011 DEBUG:teuthology.orchestra.run.smithi191:mds.c> sudo journalctl -f -n 0 -u ceph-4e059f08-da08-11ee-95c6-87774f69a715@mds.c.service 2024-03-04T09:28:00.014 DEBUG:teuthology.orchestra.run.smithi191:mds.f> sudo journalctl -f -n 0 -u ceph-4e059f08-da08-11ee-95c6-87774f69a715@mds.f.service 2024-03-04T09:28:00.058 DEBUG:teuthology.orchestra.run.smithi191:mds.i> sudo journalctl -f -n 0 -u ceph-4e059f08-da08-11ee-95c6-87774f69a715@mds.i.service 2024-03-04T09:28:00.101 DEBUG:teuthology.orchestra.run.smithi191:mds.l> sudo journalctl -f -n 0 -u ceph-4e059f08-da08-11ee-95c6-87774f69a715@mds.l.service 2024-03-04T09:28:00.143 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2024-03-04T09:28:00.144 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-04T09:28:00.270 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:00.270+0000 7f016375e640 1 -- 172.21.15.52:0/3909847512 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f015c104a20 msgr2=0x7f015c106e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:28:00.270 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:00.270+0000 7f016375e640 1 --2- 172.21.15.52:0/3909847512 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f015c104a20 0x7f015c106e10 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f0150009900 tx=0x7f015002bc30 comp rx=0 tx=0).stop 2024-03-04T09:28:00.270 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:00.270+0000 7f016375e640 1 -- 172.21.15.52:0/3909847512 shutdown_connections 2024-03-04T09:28:00.270 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:00.270+0000 7f016375e640 1 --2- 172.21.15.52:0/3909847512 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f015c107350 0x7f015c1097a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:00.270 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:00.270+0000 7f016375e640 1 --2- 172.21.15.52:0/3909847512 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f015c104a20 0x7f015c106e10 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:00.270 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:00.270+0000 7f016375e640 1 --2- 172.21.15.52:0/3909847512 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f015c1020f0 0x7f015c1044e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:00.270 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:00.270+0000 7f016375e640 1 -- 172.21.15.52:0/3909847512 >> 172.21.15.52:0/3909847512 conn(0x7f015c0fbb00 msgr2=0x7f015c0fdf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:28:00.270 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:00.270+0000 7f016375e640 1 -- 172.21.15.52:0/3909847512 shutdown_connections 2024-03-04T09:28:00.271 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:00.270+0000 7f016375e640 1 -- 172.21.15.52:0/3909847512 wait complete. 2024-03-04T09:28:00.271 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:00.271+0000 7f016375e640 1 Processor -- start 2024-03-04T09:28:00.271 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:00.271+0000 7f016375e640 1 -- start start 2024-03-04T09:28:00.271 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:00.271+0000 7f016375e640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f015c1020f0 0x7f015c197910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:28:00.271 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:00.271+0000 7f016375e640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f015c104a20 0x7f015c197e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:28:00.271 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:00.271+0000 7f016375e640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f015c107350 0x7f015c19eec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:28:00.271 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:00.271+0000 7f016375e640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f015c10d270 con 0x7f015c104a20 2024-03-04T09:28:00.271 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:00.271+0000 7f016375e640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f015c10d0f0 con 0x7f015c107350 2024-03-04T09:28:00.271 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:00.271+0000 7f016375e640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f015c10d3f0 con 0x7f015c1020f0 2024-03-04T09:28:00.271 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:00.271+0000 7f0160cf9640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f015c104a20 0x7f015c197e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:28:00.272 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:00.271+0000 7f0160cf9640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f015c104a20 0x7f015c197e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:33992/0 (socket says 172.21.15.52:33992) 2024-03-04T09:28:00.272 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:00.271+0000 7f0160cf9640 1 -- 172.21.15.52:0/3307155318 learned_addr learned my addr 172.21.15.52:0/3307155318 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-04T09:28:00.272 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:00.271+0000 7f0161cfb640 1 --2- 172.21.15.52:0/3307155318 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f015c107350 0x7f015c19eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:28:00.272 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:00.271+0000 7f01614fa640 1 --2- 172.21.15.52:0/3307155318 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f015c1020f0 0x7f015c197910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:28:00.272 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:00.271+0000 7f0160cf9640 1 -- 172.21.15.52:0/3307155318 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f015c1020f0 msgr2=0x7f015c197910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:28:00.272 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:00.271+0000 7f0160cf9640 1 --2- 172.21.15.52:0/3307155318 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f015c1020f0 0x7f015c197910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:00.272 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:00.271+0000 7f0160cf9640 1 -- 172.21.15.52:0/3307155318 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f015c107350 msgr2=0x7f015c19eec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:28:00.272 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:00.271+0000 7f0160cf9640 1 --2- 172.21.15.52:0/3307155318 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f015c107350 0x7f015c19eec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:00.272 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:00.271+0000 7f0160cf9640 1 -- 172.21.15.52:0/3307155318 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f015c19f530 con 0x7f015c104a20 2024-03-04T09:28:00.272 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:00.271+0000 7f0161cfb640 1 --2- 172.21.15.52:0/3307155318 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f015c107350 0x7f015c19eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-04T09:28:00.272 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:00.272+0000 7f0160cf9640 1 --2- 172.21.15.52:0/3307155318 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f015c104a20 0x7f015c197e50 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f0150000c00 tx=0x7f0150005d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:28:00.272 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:00.272+0000 7f015a7fc640 1 -- 172.21.15.52:0/3307155318 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0150041070 con 0x7f015c104a20 2024-03-04T09:28:00.272 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:00.272+0000 7f015a7fc640 1 -- 172.21.15.52:0/3307155318 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f01500044d0 con 0x7f015c104a20 2024-03-04T09:28:00.273 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:00.272+0000 7f016375e640 1 -- 172.21.15.52:0/3307155318 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f015c19f7c0 con 0x7f015c104a20 2024-03-04T09:28:00.273 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:00.272+0000 7f015a7fc640 1 -- 172.21.15.52:0/3307155318 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0150034440 con 0x7f015c104a20 2024-03-04T09:28:00.273 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:00.272+0000 7f016375e640 1 -- 172.21.15.52:0/3307155318 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f015c19fcb0 con 0x7f015c104a20 2024-03-04T09:28:00.273 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:00.273+0000 7f016375e640 1 -- 172.21.15.52:0/3307155318 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f015c0fd5f0 con 0x7f015c104a20 2024-03-04T09:28:00.274 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:00.274+0000 7f015a7fc640 1 -- 172.21.15.52:0/3307155318 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 16) v1 ==== 136756+0+0 (secure 0 0 0) 0x7f0150044070 con 0x7f015c104a20 2024-03-04T09:28:00.274 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:00.274+0000 7f015a7fc640 1 --2- 172.21.15.52:0/3307155318 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f01440a1d80 0x7f01440a4240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:28:00.274 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:00.275+0000 7f01614fa640 1 --2- 172.21.15.52:0/3307155318 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f01440a1d80 0x7f01440a4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:28:00.274 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:00.275+0000 7f015a7fc640 1 -- 172.21.15.52:0/3307155318 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f01500ef420 con 0x7f015c104a20 2024-03-04T09:28:00.274 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:00.275+0000 7f01614fa640 1 --2- 172.21.15.52:0/3307155318 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f01440a1d80 0x7f01440a4240 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f015400bbe0 tx=0x7f015400b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:28:00.276 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:00.279+0000 7f015a7fc640 1 -- 172.21.15.52:0/3307155318 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f01500bc6a0 con 0x7f015c104a20 2024-03-04T09:28:00.460 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:00.463+0000 7f016375e640 1 -- 172.21.15.52:0/3307155318 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f015c02d210 con 0x7f015c104a20 2024-03-04T09:28:00.461 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:00.464+0000 7f015a7fc640 1 -- 172.21.15.52:0/3307155318 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) v1 ==== 74+0+18492 (secure 0 0 0) 0x7f01500c0740 con 0x7f015c104a20 2024-03-04T09:28:00.461 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-03-04T09:28:00.461 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":81,"fsid":"4e059f08-da08-11ee-95c6-87774f69a715","created":"2024-03-04T09:20:46.104963+0000","modified":"2024-03-04T09:27:50.268367+0000","last_up_change":"2024-03-04T09:27:49.261365+0000","last_in_change":"2024-03-04T09:27:30.024964+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-04T09:24:01.742461+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"583172be-602f-4bf4-bfcb-42663cf5b277","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6802","nonce":1371221397},{"type":"v1","addr":"172.21.15.52:6803","nonce":1371221397}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6804","nonce":1371221397},{"type":"v1","addr":"172.21.15.52:6805","nonce":1371221397}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6808","nonce":1371221397},{"type":"v1","addr":"172.21.15.52:6809","nonce":1371221397}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6806","nonce":1371221397},{"type":"v1","addr":"172.21.15.52:6807","nonce":1371221397}]},"public_addr":"172.21.15.52:6803/1371221397","cluster_addr":"172.21.15.52:6805/1371221397","heartbeat_back_addr":"172.21.15.52:6809/1371221397","heartbeat_front_addr":"172.21.15.52:6807/1371221397","state":["exists","up"]},{"osd":1,"uuid":"5ae0081c-a2f6-46eb-ace0-0368208377b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":2540865537},{"type":"v1","addr":"172.21.15.106:6801","nonce":2540865537}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":2540865537},{"type":"v1","addr":"172.21.15.106:6803","nonce":2540865537}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":2540865537},{"type":"v1","addr":"172.21.15.106:6807","nonce":2540865537}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":2540865537},{"type":"v1","addr":"172.21.15.106:6805","nonce":2540865537}]},"public_addr":"172.21.15.106:6801/2540865537","cluster_addr":"172.21.15.106:6803/2540865537","heartbeat_back_addr":"172.21.15.106:6807/2540865537","heartbeat_front_addr":"172.21.15.106:6805/2540865537","state":["exists","up"]},{"osd":2,"uuid":"1948bac7-2bb2-4218-8b93-4e4743a0de04","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":1377083164},{"type":"v1","addr":"172.21.15.191:6801","nonce":1377083164}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":1377083164},{"type":"v1","addr":"172.21.15.191:6803","nonce":1377083164}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":1377083164},{"type":"v1","addr":"172.21.15.191:6807","nonce":1377083164}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":1377083164},{"type":"v1","addr":"172.21.15.191:6805","nonce":1377083164}]},"public_addr":"172.21.15.191:6801/1377083164","cluster_addr":"172.21.15.191:6803/1377083164","heartbeat_back_addr":"172.21.15.191:6807/1377083164","heartbeat_front_addr":"172.21.15.191:6805/1377083164","state":["exists","up"]},{"osd":3,"uuid":"16c9e9fe-7167-4992-bf12-1a7398e0498a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6810","nonce":2388571810},{"type":"v1","addr":"172.21.15.52:6811","nonce":2388571810}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6812","nonce":2388571810},{"type":"v1","addr":"172.21.15.52:6813","nonce":2388571810}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6816","nonce":2388571810},{"type":"v1","addr":"172.21.15.52:6817","nonce":2388571810}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6814","nonce":2388571810},{"type":"v1","addr":"172.21.15.52:6815","nonce":2388571810}]},"public_addr":"172.21.15.52:6811/2388571810","cluster_addr":"172.21.15.52:6813/2388571810","heartbeat_back_addr":"172.21.15.52:6817/2388571810","heartbeat_front_addr":"172.21.15.52:6815/2388571810","state":["exists","up"]},{"osd":4,"uuid":"c8ef48c7-7c78-44d3-a9e5-99ec750abd3d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6808","nonce":3465868015},{"type":"v1","addr":"172.21.15.106:6809","nonce":3465868015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6810","nonce":3465868015},{"type":"v1","addr":"172.21.15.106:6811","nonce":3465868015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6814","nonce":3465868015},{"type":"v1","addr":"172.21.15.106:6815","nonce":3465868015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6812","nonce":3465868015},{"type":"v1","addr":"172.21.15.106:6813","nonce":3465868015}]},"public_addr":"172.21.15.106:6809/3465868015","cluster_addr":"172.21.15.106:6811/3465868015","heartbeat_back_addr":"172.21.15.106:6815/3465868015","heartbeat_front_addr":"172.21.15.106:6813/3465868015","state":["exists","up"]},{"osd":5,"uuid":"9e41824a-c5a2-4f88-be50-2ba1dceb3220","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":1485092643},{"type":"v1","addr":"172.21.15.191:6809","nonce":1485092643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":1485092643},{"type":"v1","addr":"172.21.15.191:6811","nonce":1485092643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":1485092643},{"type":"v1","addr":"172.21.15.191:6815","nonce":1485092643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":1485092643},{"type":"v1","addr":"172.21.15.191:6813","nonce":1485092643}]},"public_addr":"172.21.15.191:6809/1485092643","cluster_addr":"172.21.15.191:6811/1485092643","heartbeat_back_addr":"172.21.15.191:6815/1485092643","heartbeat_front_addr":"172.21.15.191:6813/1485092643","state":["exists","up"]},{"osd":6,"uuid":"b35dccee-ec16-452e-ac69-3ac2fb7b27fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6818","nonce":1989797856},{"type":"v1","addr":"172.21.15.52:6819","nonce":1989797856}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6820","nonce":1989797856},{"type":"v1","addr":"172.21.15.52:6821","nonce":1989797856}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6824","nonce":1989797856},{"type":"v1","addr":"172.21.15.52:6825","nonce":1989797856}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6822","nonce":1989797856},{"type":"v1","addr":"172.21.15.52:6823","nonce":1989797856}]},"public_addr":"172.21.15.52:6819/1989797856","cluster_addr":"172.21.15.52:6821/1989797856","heartbeat_back_addr":"172.21.15.52:6825/1989797856","heartbeat_front_addr":"172.21.15.52:6823/1989797856","state":["exists","up"]},{"osd":7,"uuid":"71303e3d-14ec-4cbd-b19c-67645a1de3c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6816","nonce":1450870329},{"type":"v1","addr":"172.21.15.106:6817","nonce":1450870329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":1450870329},{"type":"v1","addr":"172.21.15.106:6819","nonce":1450870329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":1450870329},{"type":"v1","addr":"172.21.15.106:6823","nonce":1450870329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6820","nonce":1450870329},{"type":"v1","addr":"172.21.15.106:6821","nonce":1450870329}]},"public_addr":"172.21.15.106:6817/1450870329","cluster_addr":"172.21.15.106:6819/1450870329","heartbeat_back_addr":"172.21.15.106:6823/1450870329","heartbeat_front_addr":"172.21.15.106:6821/1450870329","state":["exists","up"]},{"osd":8,"uuid":"6f5786c4-4402-435a-bdad-adff46fe9473","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":1114119072},{"type":"v1","addr":"172.21.15.191:6817","nonce":1114119072}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6818","nonce":1114119072},{"type":"v1","addr":"172.21.15.191:6819","nonce":1114119072}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":1114119072},{"type":"v1","addr":"172.21.15.191:6823","nonce":1114119072}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":1114119072},{"type":"v1","addr":"172.21.15.191:6821","nonce":1114119072}]},"public_addr":"172.21.15.191:6817/1114119072","cluster_addr":"172.21.15.191:6819/1114119072","heartbeat_back_addr":"172.21.15.191:6823/1114119072","heartbeat_front_addr":"172.21.15.191:6821/1114119072","state":["exists","up"]},{"osd":9,"uuid":"062bbaa5-1227-4684-b859-450f5594a90c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6826","nonce":3485821914},{"type":"v1","addr":"172.21.15.52:6827","nonce":3485821914}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6828","nonce":3485821914},{"type":"v1","addr":"172.21.15.52:6829","nonce":3485821914}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6832","nonce":3485821914},{"type":"v1","addr":"172.21.15.52:6833","nonce":3485821914}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6830","nonce":3485821914},{"type":"v1","addr":"172.21.15.52:6831","nonce":3485821914}]},"public_addr":"172.21.15.52:6827/3485821914","cluster_addr":"172.21.15.52:6829/3485821914","heartbeat_back_addr":"172.21.15.52:6833/3485821914","heartbeat_front_addr":"172.21.15.52:6831/3485821914","state":["exists","up"]},{"osd":10,"uuid":"1a285a5a-f793-48da-a8eb-5326d9528cd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":3815586576},{"type":"v1","addr":"172.21.15.106:6825","nonce":3815586576}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6826","nonce":3815586576},{"type":"v1","addr":"172.21.15.106:6827","nonce":3815586576}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":3815586576},{"type":"v1","addr":"172.21.15.106:6831","nonce":3815586576}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":3815586576},{"type":"v1","addr":"172.21.15.106:6829","nonce":3815586576}]},"public_addr":"172.21.15.106:6825/3815586576","cluster_addr":"172.21.15.106:6827/3815586576","heartbeat_back_addr":"172.21.15.106:6831/3815586576","heartbeat_front_addr":"172.21.15.106:6829/3815586576","state":["exists","up"]},{"osd":11,"uuid":"1f8b9a43-57fa-41f4-9969-3b803a74bf45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":2009632843},{"type":"v1","addr":"172.21.15.191:6825","nonce":2009632843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":2009632843},{"type":"v1","addr":"172.21.15.191:6827","nonce":2009632843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":2009632843},{"type":"v1","addr":"172.21.15.191:6831","nonce":2009632843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":2009632843},{"type":"v1","addr":"172.21.15.191:6829","nonce":2009632843}]},"public_addr":"172.21.15.191:6825/2009632843","cluster_addr":"172.21.15.191:6827/2009632843","heartbeat_back_addr":"172.21.15.191:6831/2009632843","heartbeat_front_addr":"172.21.15.191:6829/2009632843","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T09:23:10.894763+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T09:23:34.420055+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T09:23:58.288324+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T09:24:23.673457+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T09:24:48.208561+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T09:25:13.405473+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T09:25:38.670915+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T09:26:04.451257+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T09:26:30.281380+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T09:26:56.543143+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T09:27:22.412930+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T09:27:48.003866+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.52:0/4082645718":"2024-03-05T09:21:39.401935+0000","172.21.15.52:0/1271949988":"2024-03-05T09:21:39.401935+0000","172.21.15.52:6801/2379361333":"2024-03-05T09:21:12.760938+0000","172.21.15.52:0/497075075":"2024-03-05T09:21:12.760938+0000","172.21.15.52:6800/2379361333":"2024-03-05T09:21:12.760938+0000","172.21.15.52:0/568242577":"2024-03-05T09:21:12.760938+0000","172.21.15.52:6801/2425730636":"2024-03-05T09:21:39.401935+0000","172.21.15.52:6800/2425730636":"2024-03-05T09:21:39.401935+0000","172.21.15.52:0/2743084243":"2024-03-05T09:21:39.401935+0000","172.21.15.52:0/1035878034":"2024-03-05T09:21:12.760938+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-04T09:28:00.463 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:00.466+0000 7f016375e640 1 -- 172.21.15.52:0/3307155318 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f01440a1d80 msgr2=0x7f01440a4240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:28:00.463 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:00.466+0000 7f016375e640 1 --2- 172.21.15.52:0/3307155318 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f01440a1d80 0x7f01440a4240 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f015400bbe0 tx=0x7f015400b390 comp rx=0 tx=0).stop 2024-03-04T09:28:00.463 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:00.466+0000 7f016375e640 1 -- 172.21.15.52:0/3307155318 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f015c104a20 msgr2=0x7f015c197e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:28:00.463 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:00.466+0000 7f016375e640 1 --2- 172.21.15.52:0/3307155318 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f015c104a20 0x7f015c197e50 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f0150000c00 tx=0x7f0150005d50 comp rx=0 tx=0).stop 2024-03-04T09:28:00.463 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:00.466+0000 7f016375e640 1 -- 172.21.15.52:0/3307155318 shutdown_connections 2024-03-04T09:28:00.463 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:00.466+0000 7f016375e640 1 --2- 172.21.15.52:0/3307155318 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f01440a1d80 0x7f01440a4240 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:00.464 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:00.466+0000 7f016375e640 1 --2- 172.21.15.52:0/3307155318 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f015c107350 0x7f015c19eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:00.464 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:00.466+0000 7f016375e640 1 --2- 172.21.15.52:0/3307155318 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f015c104a20 0x7f015c197e50 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:00.464 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:00.466+0000 7f016375e640 1 --2- 172.21.15.52:0/3307155318 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f015c1020f0 0x7f015c197910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:00.464 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:00.466+0000 7f016375e640 1 -- 172.21.15.52:0/3307155318 >> 172.21.15.52:0/3307155318 conn(0x7f015c0fbb00 msgr2=0x7f015c102a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:28:00.464 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:00.466+0000 7f016375e640 1 -- 172.21.15.52:0/3307155318 shutdown_connections 2024-03-04T09:28:00.464 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:00.466+0000 7f016375e640 1 -- 172.21.15.52:0/3307155318 wait complete. 2024-03-04T09:28:00.473 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:28:00 smithi052 ceph-mon[32473]: from='client.24391 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi052=a;smithi052=d;smithi052=g;smithi052=j;smithi106=b;smithi106=e;smithi106=h;smithi106=k;smithi191=c;smithi191=f;smithi191=i;smithi191=l", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T09:28:00.473 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:28:00 smithi052 ceph-mon[32473]: Saving service mds.all spec with placement smithi052=a;smithi052=d;smithi052=g;smithi052=j;smithi106=b;smithi106=e;smithi106=h;smithi106=k;smithi191=c;smithi191=f;smithi191=i;smithi191=l;count:12 2024-03-04T09:28:00.473 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:28:00 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T09:28:00.473 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:28:00 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T09:28:00.473 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:28:00 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:28:00.473 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:28:00 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-04T09:28:00.473 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:28:00 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-04T09:28:00.473 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:28:00 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T09:28:00.475 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-04T09:24:01.742461+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-04T09:28:00.475 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-04T09:28:00.584 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:00.584+0000 7fde84181640 1 -- 172.21.15.52:0/2333796484 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fde7c103190 msgr2=0x7fde7c109a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:28:00.585 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:00.584+0000 7fde84181640 1 --2- 172.21.15.52:0/2333796484 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fde7c103190 0x7fde7c109a20 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fde78009ca0 tx=0x7fde7802c0f0 comp rx=0 tx=0).stop 2024-03-04T09:28:00.585 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:00.584+0000 7fde84181640 1 -- 172.21.15.52:0/2333796484 shutdown_connections 2024-03-04T09:28:00.585 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:00.584+0000 7fde84181640 1 --2- 172.21.15.52:0/2333796484 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fde7c103190 0x7fde7c109a20 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:00.585 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:00.584+0000 7fde84181640 1 --2- 172.21.15.52:0/2333796484 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fde7c1027d0 0x7fde7c102c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:00.585 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:00.584+0000 7fde84181640 1 --2- 172.21.15.52:0/2333796484 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fde7c1015e0 0x7fde7c101a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:00.585 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:00.584+0000 7fde84181640 1 -- 172.21.15.52:0/2333796484 >> 172.21.15.52:0/2333796484 conn(0x7fde7c0fcb70 msgr2=0x7fde7c0fef90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:28:00.585 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:00.584+0000 7fde84181640 1 -- 172.21.15.52:0/2333796484 shutdown_connections 2024-03-04T09:28:00.585 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:00.584+0000 7fde84181640 1 -- 172.21.15.52:0/2333796484 wait complete. 2024-03-04T09:28:00.585 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:00.585+0000 7fde84181640 1 Processor -- start 2024-03-04T09:28:00.585 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:00.585+0000 7fde84181640 1 -- start start 2024-03-04T09:28:00.585 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:00.585+0000 7fde84181640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fde7c1015e0 0x7fde7c075380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:28:00.585 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:00.585+0000 7fde84181640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fde7c1027d0 0x7fde7c0739c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:28:00.585 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:00.585+0000 7fde84181640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fde7c103190 0x7fde7c073f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:28:00.586 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:00.585+0000 7fde84181640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde7c10d260 con 0x7fde7c1015e0 2024-03-04T09:28:00.586 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:00.585+0000 7fde84181640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde7c10d0e0 con 0x7fde7c103190 2024-03-04T09:28:00.586 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:00.585+0000 7fde84181640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde7c10d3e0 con 0x7fde7c1027d0 2024-03-04T09:28:00.586 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:00.585+0000 7fde81f1d640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fde7c1015e0 0x7fde7c075380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:28:00.586 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:00.585+0000 7fde81f1d640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fde7c1015e0 0x7fde7c075380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:34006/0 (socket says 172.21.15.52:34006) 2024-03-04T09:28:00.586 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:00.585+0000 7fde81f1d640 1 -- 172.21.15.52:0/3441287099 learned_addr learned my addr 172.21.15.52:0/3441287099 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-04T09:28:00.586 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:00.585+0000 7fde8171c640 1 --2- 172.21.15.52:0/3441287099 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fde7c1027d0 0x7fde7c0739c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:28:00.586 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:00.585+0000 7fde8271e640 1 --2- 172.21.15.52:0/3441287099 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fde7c103190 0x7fde7c073f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:28:00.586 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:00.585+0000 7fde81f1d640 1 -- 172.21.15.52:0/3441287099 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fde7c1027d0 msgr2=0x7fde7c0739c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:28:00.586 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:00.585+0000 7fde81f1d640 1 --2- 172.21.15.52:0/3441287099 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fde7c1027d0 0x7fde7c0739c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:00.586 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:00.585+0000 7fde81f1d640 1 -- 172.21.15.52:0/3441287099 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fde7c103190 msgr2=0x7fde7c073f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:28:00.586 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:00.585+0000 7fde81f1d640 1 --2- 172.21.15.52:0/3441287099 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fde7c103190 0x7fde7c073f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:00.586 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:00.585+0000 7fde81f1d640 1 -- 172.21.15.52:0/3441287099 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde7c074790 con 0x7fde7c1015e0 2024-03-04T09:28:00.586 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:00.585+0000 7fde8271e640 1 --2- 172.21.15.52:0/3441287099 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fde7c103190 0x7fde7c073f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T09:28:00.587 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:00.585+0000 7fde81f1d640 1 --2- 172.21.15.52:0/3441287099 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fde7c1015e0 0x7fde7c075380 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fde7400d770 tx=0x7fde7400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:28:00.587 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:00.585+0000 7fde72ffd640 1 -- 172.21.15.52:0/3441287099 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde74017070 con 0x7fde7c1015e0 2024-03-04T09:28:00.587 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:00.585+0000 7fde72ffd640 1 -- 172.21.15.52:0/3441287099 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fde74009c30 con 0x7fde7c1015e0 2024-03-04T09:28:00.587 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:00.585+0000 7fde84181640 1 -- 172.21.15.52:0/3441287099 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde7c1a19d0 con 0x7fde7c1015e0 2024-03-04T09:28:00.587 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:00.585+0000 7fde72ffd640 1 -- 172.21.15.52:0/3441287099 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde74003dd0 con 0x7fde7c1015e0 2024-03-04T09:28:00.587 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:00.586+0000 7fde84181640 1 -- 172.21.15.52:0/3441287099 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde7c1a1f20 con 0x7fde7c1015e0 2024-03-04T09:28:00.587 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:00.586+0000 7fde84181640 1 -- 172.21.15.52:0/3441287099 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde7c108900 con 0x7fde7c1015e0 2024-03-04T09:28:00.587 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:00.587+0000 7fde72ffd640 1 -- 172.21.15.52:0/3441287099 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 16) v1 ==== 136756+0+0 (secure 0 0 0) 0x7fde74003fb0 con 0x7fde7c1015e0 2024-03-04T09:28:00.588 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:28:00 smithi191 ceph-mon[36816]: from='client.24391 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi052=a;smithi052=d;smithi052=g;smithi052=j;smithi106=b;smithi106=e;smithi106=h;smithi106=k;smithi191=c;smithi191=f;smithi191=i;smithi191=l", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T09:28:00.588 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:28:00 smithi191 ceph-mon[36816]: Saving service mds.all spec with placement smithi052=a;smithi052=d;smithi052=g;smithi052=j;smithi106=b;smithi106=e;smithi106=h;smithi106=k;smithi191=c;smithi191=f;smithi191=i;smithi191=l;count:12 2024-03-04T09:28:00.588 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:28:00 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T09:28:00.588 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:28:00 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T09:28:00.588 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:28:00 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:28:00.588 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:28:00 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-04T09:28:00.588 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:28:00 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-04T09:28:00.588 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:28:00 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T09:28:00.589 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:00.588+0000 7fde72ffd640 1 --2- 172.21.15.52:0/3441287099 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7fde680a1d80 0x7fde680a4240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:28:00.589 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:00.588+0000 7fde72ffd640 1 -- 172.21.15.52:0/3441287099 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fde740cd6e0 con 0x7fde7c1015e0 2024-03-04T09:28:00.589 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:00.588+0000 7fde8171c640 1 --2- 172.21.15.52:0/3441287099 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7fde680a1d80 0x7fde680a4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:28:00.589 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:00.588+0000 7fde8171c640 1 --2- 172.21.15.52:0/3441287099 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7fde680a1d80 0x7fde680a4240 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fde7c0750f0 tx=0x7fde6c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:28:00.589 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:00.591+0000 7fde72ffd640 1 -- 172.21.15.52:0/3441287099 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fde74009de0 con 0x7fde7c1015e0 2024-03-04T09:28:00.631 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:28:00 smithi106 ceph-mon[36484]: from='client.24391 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi052=a;smithi052=d;smithi052=g;smithi052=j;smithi106=b;smithi106=e;smithi106=h;smithi106=k;smithi191=c;smithi191=f;smithi191=i;smithi191=l", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T09:28:00.631 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:28:00 smithi106 ceph-mon[36484]: Saving service mds.all spec with placement smithi052=a;smithi052=d;smithi052=g;smithi052=j;smithi106=b;smithi106=e;smithi106=h;smithi106=k;smithi191=c;smithi191=f;smithi191=i;smithi191=l;count:12 2024-03-04T09:28:00.631 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:28:00 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T09:28:00.631 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:28:00 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T09:28:00.631 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:28:00 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:28:00.632 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:28:00 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-04T09:28:00.632 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:28:00 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-04T09:28:00.632 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:28:00 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T09:28:00.772 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:00.775+0000 7fde84181640 1 -- 172.21.15.52:0/3441287099 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fde7c02d1a0 con 0x7fde7c1015e0 2024-03-04T09:28:00.772 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:00.775+0000 7fde72ffd640 1 -- 172.21.15.52:0/3441287099 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v81) v1 ==== 93+0+10 (secure 0 0 0) 0x7fde7409afb0 con 0x7fde7c1015e0 2024-03-04T09:28:00.772 INFO:teuthology.orchestra.run.smithi052.stdout:pg_num: 1 2024-03-04T09:28:00.774 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:00.777+0000 7fde84181640 1 -- 172.21.15.52:0/3441287099 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7fde680a1d80 msgr2=0x7fde680a4240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:28:00.774 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:00.777+0000 7fde84181640 1 --2- 172.21.15.52:0/3441287099 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7fde680a1d80 0x7fde680a4240 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fde7c0750f0 tx=0x7fde6c005f90 comp rx=0 tx=0).stop 2024-03-04T09:28:00.774 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:00.777+0000 7fde84181640 1 -- 172.21.15.52:0/3441287099 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fde7c1015e0 msgr2=0x7fde7c075380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:28:00.774 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:00.777+0000 7fde84181640 1 --2- 172.21.15.52:0/3441287099 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fde7c1015e0 0x7fde7c075380 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fde7400d770 tx=0x7fde7400dc30 comp rx=0 tx=0).stop 2024-03-04T09:28:00.774 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:00.777+0000 7fde84181640 1 -- 172.21.15.52:0/3441287099 shutdown_connections 2024-03-04T09:28:00.775 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:00.777+0000 7fde84181640 1 --2- 172.21.15.52:0/3441287099 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7fde680a1d80 0x7fde680a4240 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:00.775 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:00.777+0000 7fde84181640 1 --2- 172.21.15.52:0/3441287099 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fde7c103190 0x7fde7c073f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:00.775 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:00.777+0000 7fde84181640 1 --2- 172.21.15.52:0/3441287099 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fde7c1027d0 0x7fde7c0739c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:00.775 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:00.777+0000 7fde84181640 1 --2- 172.21.15.52:0/3441287099 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fde7c1015e0 0x7fde7c075380 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:00.775 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:00.777+0000 7fde84181640 1 -- 172.21.15.52:0/3441287099 >> 172.21.15.52:0/3441287099 conn(0x7fde7c0fcb70 msgr2=0x7fde7c105ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:28:00.775 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:00.777+0000 7fde84181640 1 -- 172.21.15.52:0/3441287099 shutdown_connections 2024-03-04T09:28:00.775 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:00.777+0000 7fde84181640 1 -- 172.21.15.52:0/3441287099 wait complete. 2024-03-04T09:28:00.785 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T09:28:00.890 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:00.890+0000 7f247bafb640 1 -- 172.21.15.52:0/316285873 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f24741031a0 msgr2=0x7f2474109a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:28:00.890 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:00.890+0000 7f247bafb640 1 --2- 172.21.15.52:0/316285873 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f24741031a0 0x7f2474109a30 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f2464009ca0 tx=0x7f246402c0f0 comp rx=0 tx=0).stop 2024-03-04T09:28:00.890 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:00.890+0000 7f247bafb640 1 -- 172.21.15.52:0/316285873 shutdown_connections 2024-03-04T09:28:00.890 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:00.890+0000 7f247bafb640 1 --2- 172.21.15.52:0/316285873 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f24741031a0 0x7f2474109a30 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:00.890 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:00.890+0000 7f247bafb640 1 --2- 172.21.15.52:0/316285873 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f24741027e0 0x7f2474102c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:00.890 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:00.890+0000 7f247bafb640 1 --2- 172.21.15.52:0/316285873 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f24741015f0 0x7f2474101a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:00.890 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:00.890+0000 7f247bafb640 1 -- 172.21.15.52:0/316285873 >> 172.21.15.52:0/316285873 conn(0x7f24740fcb80 msgr2=0x7f24740fefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:28:00.891 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:00.890+0000 7f247bafb640 1 -- 172.21.15.52:0/316285873 shutdown_connections 2024-03-04T09:28:00.891 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:00.890+0000 7f247bafb640 1 -- 172.21.15.52:0/316285873 wait complete. 2024-03-04T09:28:00.891 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:00.891+0000 7f247bafb640 1 Processor -- start 2024-03-04T09:28:00.891 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:00.891+0000 7f247bafb640 1 -- start start 2024-03-04T09:28:00.891 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:00.891+0000 7f247bafb640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f24741015f0 0x7f24741977e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:28:00.891 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:00.891+0000 7f247bafb640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f24741027e0 0x7f2474197d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:28:00.891 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:00.891+0000 7f247bafb640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f24741031a0 0x7f247419ed90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:28:00.891 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:00.891+0000 7f247bafb640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f247410d250 con 0x7f24741027e0 2024-03-04T09:28:00.891 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:00.891+0000 7f247bafb640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f247410d0d0 con 0x7f24741031a0 2024-03-04T09:28:00.891 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:00.891+0000 7f247bafb640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f247410d3d0 con 0x7f24741015f0 2024-03-04T09:28:00.891 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:00.891+0000 7f2479096640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f24741027e0 0x7f2474197d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:28:00.892 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:00.891+0000 7f2479096640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f24741027e0 0x7f2474197d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:34026/0 (socket says 172.21.15.52:34026) 2024-03-04T09:28:00.892 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:00.891+0000 7f2479096640 1 -- 172.21.15.52:0/3092957201 learned_addr learned my addr 172.21.15.52:0/3092957201 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-04T09:28:00.892 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:00.891+0000 7f2479897640 1 --2- 172.21.15.52:0/3092957201 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f24741015f0 0x7f24741977e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:28:00.892 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:00.891+0000 7f247a098640 1 --2- 172.21.15.52:0/3092957201 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f24741031a0 0x7f247419ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:28:00.892 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:00.892+0000 7f2479096640 1 -- 172.21.15.52:0/3092957201 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f24741015f0 msgr2=0x7f24741977e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:28:00.892 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:00.892+0000 7f2479096640 1 --2- 172.21.15.52:0/3092957201 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f24741015f0 0x7f24741977e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:00.892 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:00.892+0000 7f2479096640 1 -- 172.21.15.52:0/3092957201 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f24741031a0 msgr2=0x7f247419ed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:28:00.892 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:00.892+0000 7f2479096640 1 --2- 172.21.15.52:0/3092957201 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f24741031a0 0x7f247419ed90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:00.892 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:00.892+0000 7f2479096640 1 -- 172.21.15.52:0/3092957201 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f247419f400 con 0x7f24741027e0 2024-03-04T09:28:00.892 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:00.892+0000 7f2479096640 1 --2- 172.21.15.52:0/3092957201 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f24741027e0 0x7f2474197d20 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f246800e9f0 tx=0x7f246800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:28:00.892 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:00.892+0000 7f2472ffd640 1 -- 172.21.15.52:0/3092957201 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f246800cd80 con 0x7f24741027e0 2024-03-04T09:28:00.893 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:00.892+0000 7f2472ffd640 1 -- 172.21.15.52:0/3092957201 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2468010910 con 0x7f24741027e0 2024-03-04T09:28:00.893 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:00.892+0000 7f2472ffd640 1 -- 172.21.15.52:0/3092957201 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2468018990 con 0x7f24741027e0 2024-03-04T09:28:00.893 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:00.892+0000 7f247bafb640 1 -- 172.21.15.52:0/3092957201 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f247419f6f0 con 0x7f24741027e0 2024-03-04T09:28:00.893 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:00.892+0000 7f247bafb640 1 -- 172.21.15.52:0/3092957201 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f247419fc40 con 0x7f24741027e0 2024-03-04T09:28:00.893 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:00.893+0000 7f247bafb640 1 -- 172.21.15.52:0/3092957201 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2474108840 con 0x7f24741027e0 2024-03-04T09:28:00.894 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:00.894+0000 7f2472ffd640 1 -- 172.21.15.52:0/3092957201 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 16) v1 ==== 136756+0+0 (secure 0 0 0) 0x7f2468010ac0 con 0x7f24741027e0 2024-03-04T09:28:00.894 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:00.895+0000 7f2472ffd640 1 --2- 172.21.15.52:0/3092957201 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f245c0a1d80 0x7f245c0a4240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:28:00.894 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:00.895+0000 7f2472ffd640 1 -- 172.21.15.52:0/3092957201 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f24680ce0c0 con 0x7f24741027e0 2024-03-04T09:28:00.894 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:00.895+0000 7f2479897640 1 --2- 172.21.15.52:0/3092957201 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f245c0a1d80 0x7f245c0a4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:28:00.894 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:00.895+0000 7f2479897640 1 --2- 172.21.15.52:0/3092957201 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f245c0a1d80 0x7f245c0a4240 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f2474102640 tx=0x7f246c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:28:00.895 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:00.898+0000 7f2472ffd640 1 -- 172.21.15.52:0/3092957201 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f246809b990 con 0x7f24741027e0 2024-03-04T09:28:01.120 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:01.123+0000 7f247bafb640 1 -- 172.21.15.52:0/3092957201 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f247402d1b0 con 0x7f24741027e0 2024-03-04T09:28:01.120 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:01.123+0000 7f2472ffd640 1 -- 172.21.15.52:0/3092957201 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 1 v1) v1 ==== 93+0+535 (secure 0 0 0) 0x7f246809b290 con 0x7f24741027e0 2024-03-04T09:28:01.120 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-03-04T09:28:01.120 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":1,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[]} 2024-03-04T09:28:01.120 INFO:teuthology.orchestra.run.smithi052.stderr:dumped fsmap epoch 1 2024-03-04T09:28:01.122 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:01.125+0000 7f247bafb640 1 -- 172.21.15.52:0/3092957201 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f245c0a1d80 msgr2=0x7f245c0a4240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:28:01.122 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:01.125+0000 7f247bafb640 1 --2- 172.21.15.52:0/3092957201 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f245c0a1d80 0x7f245c0a4240 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f2474102640 tx=0x7f246c005f90 comp rx=0 tx=0).stop 2024-03-04T09:28:01.122 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:01.125+0000 7f247bafb640 1 -- 172.21.15.52:0/3092957201 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f24741027e0 msgr2=0x7f2474197d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:28:01.122 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:01.125+0000 7f247bafb640 1 --2- 172.21.15.52:0/3092957201 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f24741027e0 0x7f2474197d20 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f246800e9f0 tx=0x7f246800eeb0 comp rx=0 tx=0).stop 2024-03-04T09:28:01.122 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:01.125+0000 7f247bafb640 1 -- 172.21.15.52:0/3092957201 shutdown_connections 2024-03-04T09:28:01.122 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:01.125+0000 7f247bafb640 1 --2- 172.21.15.52:0/3092957201 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f245c0a1d80 0x7f245c0a4240 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:01.122 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:01.125+0000 7f247bafb640 1 --2- 172.21.15.52:0/3092957201 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f24741031a0 0x7f247419ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:01.122 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:01.125+0000 7f247bafb640 1 --2- 172.21.15.52:0/3092957201 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f24741027e0 0x7f2474197d20 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:01.123 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:01.125+0000 7f247bafb640 1 --2- 172.21.15.52:0/3092957201 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f24741015f0 0x7f24741977e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:01.123 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:01.125+0000 7f247bafb640 1 -- 172.21.15.52:0/3092957201 >> 172.21.15.52:0/3092957201 conn(0x7f24740fcb80 msgr2=0x7f2474105a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:28:01.123 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:01.125+0000 7f247bafb640 1 -- 172.21.15.52:0/3092957201 shutdown_connections 2024-03-04T09:28:01.123 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:01.125+0000 7f247bafb640 1 -- 172.21.15.52:0/3092957201 wait complete. 2024-03-04T09:28:01.585 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:28:01 smithi052 ceph-mon[32473]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-04T09:28:01.585 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:28:01 smithi052 ceph-mon[32473]: Deploying daemon mds.l on smithi191 2024-03-04T09:28:01.585 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:28:01 smithi052 ceph-mon[32473]: from='client.? 172.21.15.52:0/3307155318' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-04T09:28:01.585 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:28:01 smithi052 ceph-mon[32473]: from='client.? 172.21.15.52:0/3441287099' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-04T09:28:01.585 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:28:01 smithi052 ceph-mon[32473]: from='client.? 172.21.15.52:0/3092957201' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-04T09:28:01.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:28:01 smithi191 ceph-mon[36816]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-04T09:28:01.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:28:01 smithi191 ceph-mon[36816]: Deploying daemon mds.l on smithi191 2024-03-04T09:28:01.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:28:01 smithi191 ceph-mon[36816]: from='client.? 172.21.15.52:0/3307155318' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-04T09:28:01.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:28:01 smithi191 ceph-mon[36816]: from='client.? 172.21.15.52:0/3441287099' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-04T09:28:01.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:28:01 smithi191 ceph-mon[36816]: from='client.? 172.21.15.52:0/3092957201' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-04T09:28:01.631 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:28:01 smithi106 ceph-mon[36484]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-04T09:28:01.631 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:28:01 smithi106 ceph-mon[36484]: Deploying daemon mds.l on smithi191 2024-03-04T09:28:01.631 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:28:01 smithi106 ceph-mon[36484]: from='client.? 172.21.15.52:0/3307155318' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-04T09:28:01.631 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:28:01 smithi106 ceph-mon[36484]: from='client.? 172.21.15.52:0/3441287099' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-04T09:28:01.631 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:28:01 smithi106 ceph-mon[36484]: from='client.? 172.21.15.52:0/3092957201' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-04T09:28:02.087 INFO:journalctl@ceph.mds.l.smithi191.stdout:Mar 04 09:28:01 smithi191 systemd[1]: Starting Ceph mds.l for 4e059f08-da08-11ee-95c6-87774f69a715... 2024-03-04T09:28:02.418 INFO:journalctl@ceph.mds.l.smithi191.stdout:Mar 04 09:28:02 smithi191 podman[66443]: 2024-03-04 09:28:02.066135359 +0000 UTC m=+0.017537444 image pull f88ffb32aa6a7d5dfdc954d9f096f0c9cc47981d05ec8d117ba859a22d8185ba quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5 2024-03-04T09:28:02.418 INFO:journalctl@ceph.mds.l.smithi191.stdout:Mar 04 09:28:02 smithi191 podman[66443]: 2024-03-04 09:28:02.185547749 +0000 UTC m=+0.136949829 container create 08126d739d3a2e2f9c3140f21255c28092412552931135a4033cb0be331b541d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5, name=ceph-4e059f08-da08-11ee-95c6-87774f69a715-mds-l, org.label-schema.build-date=20240222, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, RELEASE=wip-lusov-qdb-1bc250c, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0) 2024-03-04T09:28:02.418 INFO:journalctl@ceph.mds.l.smithi191.stdout:Mar 04 09:28:02 smithi191 podman[66443]: 2024-03-04 09:28:02.422097345 +0000 UTC m=+0.373499437 container init 08126d739d3a2e2f9c3140f21255c28092412552931135a4033cb0be331b541d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5, name=ceph-4e059f08-da08-11ee-95c6-87774f69a715-mds-l, RELEASE=wip-lusov-qdb-1bc250c, org.label-schema.build-date=20240222, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8) 2024-03-04T09:28:02.836 INFO:journalctl@ceph.mds.l.smithi191.stdout:Mar 04 09:28:02 smithi191 podman[66443]: 2024-03-04 09:28:02.425390012 +0000 UTC m=+0.376792102 container start 08126d739d3a2e2f9c3140f21255c28092412552931135a4033cb0be331b541d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5, name=ceph-4e059f08-da08-11ee-95c6-87774f69a715-mds-l, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240222, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-qdb-1bc250c, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5) 2024-03-04T09:28:02.836 INFO:journalctl@ceph.mds.l.smithi191.stdout:Mar 04 09:28:02 smithi191 ceph-mds[66462]: -- 172.21.15.191:0/3694152534 <== mon.1 v2:172.21.15.191:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x560c1061fa40 con 0x560c105f2800 2024-03-04T09:28:02.836 INFO:journalctl@ceph.mds.l.smithi191.stdout:Mar 04 09:28:02 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-mds-l[66458]: starting mds.l at 2024-03-04T09:28:02.836 INFO:journalctl@ceph.mds.l.smithi191.stdout:Mar 04 09:28:02 smithi191 bash[66443]: 08126d739d3a2e2f9c3140f21255c28092412552931135a4033cb0be331b541d 2024-03-04T09:28:02.836 INFO:journalctl@ceph.mds.l.smithi191.stdout:Mar 04 09:28:02 smithi191 systemd[1]: Started Ceph mds.l for 4e059f08-da08-11ee-95c6-87774f69a715. 2024-03-04T09:28:03.133 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T09:28:03.250 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:03.251+0000 7f77eda81640 1 -- 172.21.15.52:0/2224653646 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f77e81031a0 msgr2=0x7f77e8109a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:28:03.251 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:03.251+0000 7f77eda81640 1 --2- 172.21.15.52:0/2224653646 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f77e81031a0 0x7f77e8109a30 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f77e0009900 tx=0x7f77e002bc30 comp rx=0 tx=0).stop 2024-03-04T09:28:03.251 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:03.251+0000 7f77eda81640 1 -- 172.21.15.52:0/2224653646 shutdown_connections 2024-03-04T09:28:03.251 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:03.251+0000 7f77eda81640 1 --2- 172.21.15.52:0/2224653646 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f77e81031a0 0x7f77e8109a30 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:03.251 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:03.251+0000 7f77eda81640 1 --2- 172.21.15.52:0/2224653646 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f77e81027e0 0x7f77e8102c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:03.251 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:03.251+0000 7f77eda81640 1 --2- 172.21.15.52:0/2224653646 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f77e81015f0 0x7f77e8101a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:03.251 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:03.251+0000 7f77eda81640 1 -- 172.21.15.52:0/2224653646 >> 172.21.15.52:0/2224653646 conn(0x7f77e80fcb80 msgr2=0x7f77e80fefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:28:03.251 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:03.251+0000 7f77eda81640 1 -- 172.21.15.52:0/2224653646 shutdown_connections 2024-03-04T09:28:03.251 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:03.251+0000 7f77eda81640 1 -- 172.21.15.52:0/2224653646 wait complete. 2024-03-04T09:28:03.251 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:03.252+0000 7f77eda81640 1 Processor -- start 2024-03-04T09:28:03.251 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:03.252+0000 7f77eda81640 1 -- start start 2024-03-04T09:28:03.251 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:03.252+0000 7f77eda81640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f77e81015f0 0x7f77e81978e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:28:03.251 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:03.252+0000 7f77eda81640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f77e81027e0 0x7f77e8197e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:28:03.251 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:03.252+0000 7f77eda81640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f77e81031a0 0x7f77e819ee90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:28:03.251 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:03.252+0000 7f77eda81640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77e810d270 con 0x7f77e81031a0 2024-03-04T09:28:03.252 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:03.252+0000 7f77eda81640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77e810d0f0 con 0x7f77e81027e0 2024-03-04T09:28:03.252 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:03.252+0000 7f77eda81640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77e810d3f0 con 0x7f77e81015f0 2024-03-04T09:28:03.252 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:03.252+0000 7f77e7fff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f77e81031a0 0x7f77e819ee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:28:03.252 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:03.252+0000 7f77e7fff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f77e81031a0 0x7f77e819ee90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:34048/0 (socket says 172.21.15.52:34048) 2024-03-04T09:28:03.252 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:03.252+0000 7f77e7fff640 1 -- 172.21.15.52:0/1484704540 learned_addr learned my addr 172.21.15.52:0/1484704540 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-04T09:28:03.252 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:03.253+0000 7f77e77fe640 1 --2- 172.21.15.52:0/1484704540 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f77e81015f0 0x7f77e81978e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:28:03.252 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:03.253+0000 7f77e7fff640 1 -- 172.21.15.52:0/1484704540 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f77e81015f0 msgr2=0x7f77e81978e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:28:03.252 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:03.253+0000 7f77e7fff640 1 --2- 172.21.15.52:0/1484704540 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f77e81015f0 0x7f77e81978e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:03.252 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:03.253+0000 7f77e7fff640 1 -- 172.21.15.52:0/1484704540 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f77e81027e0 msgr2=0x7f77e8197e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:28:03.252 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:03.253+0000 7f77e6ffd640 1 --2- 172.21.15.52:0/1484704540 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f77e81027e0 0x7f77e8197e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:28:03.252 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:03.253+0000 7f77e7fff640 1 --2- 172.21.15.52:0/1484704540 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f77e81027e0 0x7f77e8197e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:03.252 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:03.253+0000 7f77e7fff640 1 -- 172.21.15.52:0/1484704540 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77e819f500 con 0x7f77e81031a0 2024-03-04T09:28:03.253 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:03.253+0000 7f77e7fff640 1 --2- 172.21.15.52:0/1484704540 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f77e81031a0 0x7f77e819ee90 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f77e002ce70 tx=0x7f77e0005e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:28:03.253 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:03.253+0000 7f77e4ff9640 1 -- 172.21.15.52:0/1484704540 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77e0041070 con 0x7f77e81031a0 2024-03-04T09:28:03.253 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:03.253+0000 7f77e4ff9640 1 -- 172.21.15.52:0/1484704540 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f77e00054e0 con 0x7f77e81031a0 2024-03-04T09:28:03.253 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:03.253+0000 7f77eda81640 1 -- 172.21.15.52:0/1484704540 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77e819f790 con 0x7f77e81031a0 2024-03-04T09:28:03.253 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:03.253+0000 7f77e4ff9640 1 -- 172.21.15.52:0/1484704540 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77e00344a0 con 0x7f77e81031a0 2024-03-04T09:28:03.253 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:03.253+0000 7f77eda81640 1 -- 172.21.15.52:0/1484704540 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77e819fcd0 con 0x7f77e81031a0 2024-03-04T09:28:03.254 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:03.254+0000 7f77eda81640 1 -- 172.21.15.52:0/1484704540 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f77e8108910 con 0x7f77e81031a0 2024-03-04T09:28:03.254 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:03.255+0000 7f77e4ff9640 1 -- 172.21.15.52:0/1484704540 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 16) v1 ==== 136756+0+0 (secure 0 0 0) 0x7f77e0044070 con 0x7f77e81031a0 2024-03-04T09:28:03.254 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:03.256+0000 7f77e4ff9640 1 --2- 172.21.15.52:0/1484704540 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f77d40a1e00 0x7f77d40a42c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:28:03.254 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:03.256+0000 7f77e77fe640 1 --2- 172.21.15.52:0/1484704540 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f77d40a1e00 0x7f77d40a42c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:28:03.254 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:03.256+0000 7f77e4ff9640 1 -- 172.21.15.52:0/1484704540 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f77e00ef6f0 con 0x7f77e81031a0 2024-03-04T09:28:03.254 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:03.256+0000 7f77e77fe640 1 --2- 172.21.15.52:0/1484704540 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f77d40a1e00 0x7f77d40a42c0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f77e8102640 tx=0x7f77d8009670 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:28:03.257 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:03.260+0000 7f77e4ff9640 1 -- 172.21.15.52:0/1484704540 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f77e00bcf20 con 0x7f77e81031a0 2024-03-04T09:28:03.481 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:03.484+0000 7f77eda81640 1 -- 172.21.15.52:0/1484704540 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f77e80621d0 con 0x7f77e81031a0 2024-03-04T09:28:03.482 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:03.485+0000 7f77e4ff9640 1 -- 172.21.15.52:0/1484704540 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) v1 ==== 93+0+1325 (secure 0 0 0) 0x7f77e00bc820 con 0x7f77e81031a0 2024-03-04T09:28:03.482 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-03-04T09:28:03.482 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24412,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6833/765768204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":765768204},{"type":"v1","addr":"172.21.15.191:6833","nonce":765768204}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[]} 2024-03-04T09:28:03.483 INFO:teuthology.orchestra.run.smithi052.stderr:dumped fsmap epoch 2 2024-03-04T09:28:03.484 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:03.487+0000 7f77eda81640 1 -- 172.21.15.52:0/1484704540 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f77d40a1e00 msgr2=0x7f77d40a42c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:28:03.484 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:03.487+0000 7f77eda81640 1 --2- 172.21.15.52:0/1484704540 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f77d40a1e00 0x7f77d40a42c0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f77e8102640 tx=0x7f77d8009670 comp rx=0 tx=0).stop 2024-03-04T09:28:03.484 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:03.487+0000 7f77eda81640 1 -- 172.21.15.52:0/1484704540 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f77e81031a0 msgr2=0x7f77e819ee90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:28:03.485 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:03.487+0000 7f77eda81640 1 --2- 172.21.15.52:0/1484704540 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f77e81031a0 0x7f77e819ee90 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f77e002ce70 tx=0x7f77e0005e40 comp rx=0 tx=0).stop 2024-03-04T09:28:03.485 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:03.487+0000 7f77eda81640 1 -- 172.21.15.52:0/1484704540 shutdown_connections 2024-03-04T09:28:03.485 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:03.487+0000 7f77eda81640 1 --2- 172.21.15.52:0/1484704540 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f77d40a1e00 0x7f77d40a42c0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:03.485 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:03.487+0000 7f77eda81640 1 --2- 172.21.15.52:0/1484704540 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f77e81031a0 0x7f77e819ee90 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:03.485 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:03.487+0000 7f77eda81640 1 --2- 172.21.15.52:0/1484704540 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f77e81027e0 0x7f77e8197e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:03.485 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:03.487+0000 7f77eda81640 1 --2- 172.21.15.52:0/1484704540 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f77e81015f0 0x7f77e81978e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:03.485 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:03.487+0000 7f77eda81640 1 -- 172.21.15.52:0/1484704540 >> 172.21.15.52:0/1484704540 conn(0x7f77e80fcb80 msgr2=0x7f77e8105ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:28:03.485 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:03.487+0000 7f77eda81640 1 -- 172.21.15.52:0/1484704540 shutdown_connections 2024-03-04T09:28:03.485 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:03.487+0000 7f77eda81640 1 -- 172.21.15.52:0/1484704540 wait complete. 2024-03-04T09:28:03.494 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:28:03 smithi052 ceph-mon[32473]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-04T09:28:03.494 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:28:03 smithi052 ceph-mon[32473]: mds.? [v2:172.21.15.191:6832/765768204,v1:172.21.15.191:6833/765768204] up:boot 2024-03-04T09:28:03.494 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:28:03 smithi052 ceph-mon[32473]: fsmap 1 up:standby 2024-03-04T09:28:03.494 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:28:03 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-03-04T09:28:03.494 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:28:03 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:28:03.494 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:28:03 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:28:03.494 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:28:03 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:28:03.494 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:28:03 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-04T09:28:03.494 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:28:03 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-04T09:28:03.495 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:28:03 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T09:28:03.527 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:28:03 smithi106 ceph-mon[36484]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-04T09:28:03.528 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:28:03 smithi106 ceph-mon[36484]: mds.? [v2:172.21.15.191:6832/765768204,v1:172.21.15.191:6833/765768204] up:boot 2024-03-04T09:28:03.528 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:28:03 smithi106 ceph-mon[36484]: fsmap 1 up:standby 2024-03-04T09:28:03.528 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:28:03 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-03-04T09:28:03.528 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:28:03 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:28:03.528 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:28:03 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:28:03.528 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:28:03 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:28:03.528 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:28:03 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-04T09:28:03.528 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:28:03 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-04T09:28:03.529 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:28:03 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T09:28:03.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:28:03 smithi191 ceph-mon[36816]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-04T09:28:03.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:28:03 smithi191 ceph-mon[36816]: mds.? [v2:172.21.15.191:6832/765768204,v1:172.21.15.191:6833/765768204] up:boot 2024-03-04T09:28:03.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:28:03 smithi191 ceph-mon[36816]: fsmap 1 up:standby 2024-03-04T09:28:03.587 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:28:03 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-03-04T09:28:03.587 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:28:03 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:28:03.587 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:28:03 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:28:03.587 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:28:03 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:28:03.587 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:28:03 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-04T09:28:03.587 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:28:03 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-04T09:28:03.587 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:28:03 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T09:28:04.584 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:28:04 smithi106 ceph-mon[36484]: Deploying daemon mds.k on smithi106 2024-03-04T09:28:04.584 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:28:04 smithi106 ceph-mon[36484]: from='client.? 172.21.15.52:0/1484704540' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-04T09:28:04.585 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:28:04 smithi052 ceph-mon[32473]: Deploying daemon mds.k on smithi106 2024-03-04T09:28:04.585 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:28:04 smithi052 ceph-mon[32473]: from='client.? 172.21.15.52:0/1484704540' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-04T09:28:04.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:28:04 smithi191 ceph-mon[36816]: Deploying daemon mds.k on smithi106 2024-03-04T09:28:04.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:28:04 smithi191 ceph-mon[36816]: from='client.? 172.21.15.52:0/1484704540' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-04T09:28:04.881 INFO:journalctl@ceph.mds.k.smithi106.stdout:Mar 04 09:28:04 smithi106 systemd[1]: Starting Ceph mds.k for 4e059f08-da08-11ee-95c6-87774f69a715... 2024-03-04T09:28:05.229 INFO:journalctl@ceph.mds.k.smithi106.stdout:Mar 04 09:28:04 smithi106 podman[65689]: 2024-03-04 09:28:04.971829412 +0000 UTC m=+0.105394087 container create a3f695264ac9030fc69e3fe0c14043664f858181b11765e394b01d500dac2d4b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5, name=ceph-4e059f08-da08-11ee-95c6-87774f69a715-mds-k, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-qdb-1bc250c, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240222, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, GIT_CLEAN=True, io.buildah.version=1.33.5) 2024-03-04T09:28:05.229 INFO:journalctl@ceph.mds.k.smithi106.stdout:Mar 04 09:28:04 smithi106 podman[65689]: 2024-03-04 09:28:04.883161848 +0000 UTC m=+0.016726540 image pull f88ffb32aa6a7d5dfdc954d9f096f0c9cc47981d05ec8d117ba859a22d8185ba quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5 2024-03-04T09:28:05.229 INFO:journalctl@ceph.mds.k.smithi106.stdout:Mar 04 09:28:05 smithi106 podman[65689]: 2024-03-04 09:28:05.232687076 +0000 UTC m=+0.366251774 container init a3f695264ac9030fc69e3fe0c14043664f858181b11765e394b01d500dac2d4b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5, name=ceph-4e059f08-da08-11ee-95c6-87774f69a715-mds-k, org.label-schema.license=GPLv2, org.label-schema.build-date=20240222, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-qdb-1bc250c, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, maintainer=Guillaume Abrioux ) 2024-03-04T09:28:05.496 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T09:28:05.584 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:28:05 smithi052 ceph-mon[32473]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-04T09:28:05.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:28:05 smithi191 ceph-mon[36816]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-04T09:28:05.607 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:05.607+0000 7f0367325640 1 -- 172.21.15.52:0/541514034 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0360102070 msgr2=0x7f0360104460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:28:05.608 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:05.607+0000 7f0367325640 1 --2- 172.21.15.52:0/541514034 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0360102070 0x7f0360104460 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f0358009900 tx=0x7f035802bc30 comp rx=0 tx=0).stop 2024-03-04T09:28:05.608 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:05.608+0000 7f0367325640 1 -- 172.21.15.52:0/541514034 shutdown_connections 2024-03-04T09:28:05.608 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:05.608+0000 7f0367325640 1 --2- 172.21.15.52:0/541514034 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f03601072d0 0x7f0360109720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:05.608 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:05.608+0000 7f0367325640 1 --2- 172.21.15.52:0/541514034 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f03601049a0 0x7f0360106d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:05.608 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:05.608+0000 7f0367325640 1 --2- 172.21.15.52:0/541514034 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0360102070 0x7f0360104460 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:05.608 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:05.608+0000 7f0367325640 1 -- 172.21.15.52:0/541514034 >> 172.21.15.52:0/541514034 conn(0x7f03600fba60 msgr2=0x7f03600fdea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:28:05.608 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:05.608+0000 7f0367325640 1 -- 172.21.15.52:0/541514034 shutdown_connections 2024-03-04T09:28:05.608 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:05.608+0000 7f0367325640 1 -- 172.21.15.52:0/541514034 wait complete. 2024-03-04T09:28:05.608 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:05.608+0000 7f0367325640 1 Processor -- start 2024-03-04T09:28:05.608 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:05.608+0000 7f0367325640 1 -- start start 2024-03-04T09:28:05.609 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:05.608+0000 7f0367325640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0360102070 0x7f03601019e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:28:05.609 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:05.608+0000 7f0367325640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f03601049a0 0x7f0360100020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:28:05.609 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:05.608+0000 7f0367325640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f03601072d0 0x7f0360100560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:28:05.609 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:05.608+0000 7f0367325640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f036010d280 con 0x7f0360102070 2024-03-04T09:28:05.609 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:05.608+0000 7f0367325640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f036010d100 con 0x7f03601049a0 2024-03-04T09:28:05.609 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:05.608+0000 7f0367325640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f036010d400 con 0x7f03601072d0 2024-03-04T09:28:05.609 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:05.608+0000 7f03650c1640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0360102070 0x7f03601019e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:28:05.609 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:05.608+0000 7f03650c1640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0360102070 0x7f03601019e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:51174/0 (socket says 172.21.15.52:51174) 2024-03-04T09:28:05.609 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:05.608+0000 7f03650c1640 1 -- 172.21.15.52:0/1335778840 learned_addr learned my addr 172.21.15.52:0/1335778840 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-04T09:28:05.610 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:05.609+0000 7f03648c0640 1 --2- 172.21.15.52:0/1335778840 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f03601049a0 0x7f0360100020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:28:05.610 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:05.609+0000 7f03658c2640 1 --2- 172.21.15.52:0/1335778840 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f03601072d0 0x7f0360100560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:28:05.610 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:05.609+0000 7f03650c1640 1 -- 172.21.15.52:0/1335778840 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f03601072d0 msgr2=0x7f0360100560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:28:05.610 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:05.609+0000 7f03650c1640 1 --2- 172.21.15.52:0/1335778840 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f03601072d0 0x7f0360100560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:05.610 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:05.609+0000 7f03650c1640 1 -- 172.21.15.52:0/1335778840 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f03601049a0 msgr2=0x7f0360100020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:28:05.610 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:05.609+0000 7f03650c1640 1 --2- 172.21.15.52:0/1335778840 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f03601049a0 0x7f0360100020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:05.610 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:05.609+0000 7f03650c1640 1 -- 172.21.15.52:0/1335778840 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0360100df0 con 0x7f0360102070 2024-03-04T09:28:05.610 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:05.609+0000 7f03648c0640 1 --2- 172.21.15.52:0/1335778840 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f03601049a0 0x7f0360100020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T09:28:05.610 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:05.609+0000 7f03658c2640 1 --2- 172.21.15.52:0/1335778840 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f03601072d0 0x7f0360100560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-04T09:28:05.610 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:05.609+0000 7f03650c1640 1 --2- 172.21.15.52:0/1335778840 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0360102070 0x7f03601019e0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f0358009230 tx=0x7f0358005dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:28:05.610 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:05.609+0000 7f03567fc640 1 -- 172.21.15.52:0/1335778840 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0358041070 con 0x7f0360102070 2024-03-04T09:28:05.610 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:05.609+0000 7f0367325640 1 -- 172.21.15.52:0/1335778840 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03601a1a20 con 0x7f0360102070 2024-03-04T09:28:05.611 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:05.609+0000 7f03567fc640 1 -- 172.21.15.52:0/1335778840 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f03580054e0 con 0x7f0360102070 2024-03-04T09:28:05.611 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:05.609+0000 7f03567fc640 1 -- 172.21.15.52:0/1335778840 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0358034440 con 0x7f0360102070 2024-03-04T09:28:05.611 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:05.609+0000 7f0367325640 1 -- 172.21.15.52:0/1335778840 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03601a1fc0 con 0x7f0360102070 2024-03-04T09:28:05.611 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:05.610+0000 7f0367325640 1 -- 172.21.15.52:0/1335778840 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f03600fd550 con 0x7f0360102070 2024-03-04T09:28:05.612 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:05.612+0000 7f03567fc640 1 -- 172.21.15.52:0/1335778840 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 16) v1 ==== 136756+0+0 (secure 0 0 0) 0x7f0358044070 con 0x7f0360102070 2024-03-04T09:28:05.612 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:05.613+0000 7f03567fc640 1 --2- 172.21.15.52:0/1335778840 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f034c0a1d80 0x7f034c0a4240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:28:05.612 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:05.613+0000 7f03567fc640 1 -- 172.21.15.52:0/1335778840 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f03580efb50 con 0x7f0360102070 2024-03-04T09:28:05.612 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:05.613+0000 7f03648c0640 1 --2- 172.21.15.52:0/1335778840 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f034c0a1d80 0x7f034c0a4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:28:05.612 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:05.613+0000 7f03648c0640 1 --2- 172.21.15.52:0/1335778840 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f034c0a1d80 0x7f034c0a4240 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f03500058c0 tx=0x7f0350005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:28:05.613 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:05.615+0000 7f03567fc640 1 -- 172.21.15.52:0/1335778840 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f03580bd380 con 0x7f0360102070 2024-03-04T09:28:05.631 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:28:05 smithi106 ceph-mon[36484]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-04T09:28:05.631 INFO:journalctl@ceph.mds.k.smithi106.stdout:Mar 04 09:28:05 smithi106 podman[65689]: 2024-03-04 09:28:05.235579771 +0000 UTC m=+0.369144465 container start a3f695264ac9030fc69e3fe0c14043664f858181b11765e394b01d500dac2d4b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5, name=ceph-4e059f08-da08-11ee-95c6-87774f69a715-mds-k, org.label-schema.license=GPLv2, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240222, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-qdb-1bc250c, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, GIT_CLEAN=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5) 2024-03-04T09:28:05.631 INFO:journalctl@ceph.mds.k.smithi106.stdout:Mar 04 09:28:05 smithi106 ceph-mds[65708]: -- 172.21.15.106:0/1330426526 <== mon.2 v2:172.21.15.106:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5639a7ae5a40 con 0x5639a7ab8800 2024-03-04T09:28:05.631 INFO:journalctl@ceph.mds.k.smithi106.stdout:Mar 04 09:28:05 smithi106 ceph-4e059f08-da08-11ee-95c6-87774f69a715-mds-k[65704]: starting mds.k at 2024-03-04T09:28:05.632 INFO:journalctl@ceph.mds.k.smithi106.stdout:Mar 04 09:28:05 smithi106 bash[65689]: a3f695264ac9030fc69e3fe0c14043664f858181b11765e394b01d500dac2d4b 2024-03-04T09:28:05.632 INFO:journalctl@ceph.mds.k.smithi106.stdout:Mar 04 09:28:05 smithi106 systemd[1]: Started Ceph mds.k for 4e059f08-da08-11ee-95c6-87774f69a715. 2024-03-04T09:28:05.840 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:05.844+0000 7f0367325640 1 -- 172.21.15.52:0/1335778840 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f036002cff0 con 0x7f0360102070 2024-03-04T09:28:05.841 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:05.844+0000 7f03567fc640 1 -- 172.21.15.52:0/1335778840 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) v1 ==== 93+0+2119 (secure 0 0 0) 0x7f03580bcc80 con 0x7f0360102070 2024-03-04T09:28:05.841 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-03-04T09:28:05.841 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24407,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6833/1818480323","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6832","nonce":1818480323},{"type":"v1","addr":"172.21.15.106:6833","nonce":1818480323}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24412,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6833/765768204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":765768204},{"type":"v1","addr":"172.21.15.191:6833","nonce":765768204}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[]} 2024-03-04T09:28:05.842 INFO:teuthology.orchestra.run.smithi052.stderr:dumped fsmap epoch 3 2024-03-04T09:28:05.843 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:05.846+0000 7f0367325640 1 -- 172.21.15.52:0/1335778840 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f034c0a1d80 msgr2=0x7f034c0a4240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:28:05.843 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:05.846+0000 7f0367325640 1 --2- 172.21.15.52:0/1335778840 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f034c0a1d80 0x7f034c0a4240 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f03500058c0 tx=0x7f0350005850 comp rx=0 tx=0).stop 2024-03-04T09:28:05.843 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:05.846+0000 7f0367325640 1 -- 172.21.15.52:0/1335778840 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0360102070 msgr2=0x7f03601019e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:28:05.843 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:05.846+0000 7f0367325640 1 --2- 172.21.15.52:0/1335778840 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0360102070 0x7f03601019e0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f0358009230 tx=0x7f0358005dd0 comp rx=0 tx=0).stop 2024-03-04T09:28:05.843 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:05.846+0000 7f0367325640 1 -- 172.21.15.52:0/1335778840 shutdown_connections 2024-03-04T09:28:05.844 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:05.846+0000 7f0367325640 1 --2- 172.21.15.52:0/1335778840 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f034c0a1d80 0x7f034c0a4240 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:05.844 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:05.846+0000 7f0367325640 1 --2- 172.21.15.52:0/1335778840 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f03601072d0 0x7f0360100560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:05.844 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:05.846+0000 7f0367325640 1 --2- 172.21.15.52:0/1335778840 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f03601049a0 0x7f0360100020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:05.844 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:05.846+0000 7f0367325640 1 --2- 172.21.15.52:0/1335778840 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0360102070 0x7f03601019e0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:05.844 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:05.846+0000 7f0367325640 1 -- 172.21.15.52:0/1335778840 >> 172.21.15.52:0/1335778840 conn(0x7f03600fba60 msgr2=0x7f0360102ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:28:05.844 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:05.846+0000 7f0367325640 1 -- 172.21.15.52:0/1335778840 shutdown_connections 2024-03-04T09:28:05.844 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:05.846+0000 7f0367325640 1 -- 172.21.15.52:0/1335778840 wait complete. 2024-03-04T09:28:06.421 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:28:06 smithi052 ceph-mon[32473]: mds.? [v2:172.21.15.106:6832/1818480323,v1:172.21.15.106:6833/1818480323] up:boot 2024-03-04T09:28:06.421 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:28:06 smithi052 ceph-mon[32473]: fsmap 2 up:standby 2024-03-04T09:28:06.421 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:28:06 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-03-04T09:28:06.421 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:28:06 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:28:06.421 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:28:06 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:28:06.422 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:28:06 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:28:06.422 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:28:06 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-04T09:28:06.422 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:28:06 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-04T09:28:06.422 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:28:06 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T09:28:06.422 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:28:06 smithi052 ceph-mon[32473]: from='client.? 172.21.15.52:0/1335778840' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-04T09:28:06.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:28:06 smithi191 ceph-mon[36816]: mds.? [v2:172.21.15.106:6832/1818480323,v1:172.21.15.106:6833/1818480323] up:boot 2024-03-04T09:28:06.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:28:06 smithi191 ceph-mon[36816]: fsmap 2 up:standby 2024-03-04T09:28:06.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:28:06 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-03-04T09:28:06.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:28:06 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:28:06.587 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:28:06 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:28:06.587 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:28:06 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:28:06.587 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:28:06 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-04T09:28:06.587 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:28:06 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-04T09:28:06.587 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:28:06 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T09:28:06.587 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:28:06 smithi191 ceph-mon[36816]: from='client.? 172.21.15.52:0/1335778840' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-04T09:28:06.629 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:28:06 smithi106 ceph-mon[36484]: mds.? [v2:172.21.15.106:6832/1818480323,v1:172.21.15.106:6833/1818480323] up:boot 2024-03-04T09:28:06.629 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:28:06 smithi106 ceph-mon[36484]: fsmap 2 up:standby 2024-03-04T09:28:06.629 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:28:06 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-03-04T09:28:06.630 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:28:06 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:28:06.630 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:28:06 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:28:06.630 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:28:06 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:28:06.630 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:28:06 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-04T09:28:06.630 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:28:06 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-04T09:28:06.630 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:28:06 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T09:28:06.630 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:28:06 smithi106 ceph-mon[36484]: from='client.? 172.21.15.52:0/1335778840' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-04T09:28:07.416 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:28:07 smithi052 ceph-mon[32473]: Deploying daemon mds.j on smithi052 2024-03-04T09:28:07.417 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:28:07 smithi052 ceph-mon[32473]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-04T09:28:07.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:28:07 smithi191 ceph-mon[36816]: Deploying daemon mds.j on smithi052 2024-03-04T09:28:07.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:28:07 smithi191 ceph-mon[36816]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-04T09:28:07.631 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:28:07 smithi106 ceph-mon[36484]: Deploying daemon mds.j on smithi052 2024-03-04T09:28:07.631 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:28:07 smithi106 ceph-mon[36484]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-04T09:28:07.796 INFO:journalctl@ceph.mds.j.smithi052.stdout:Mar 04 09:28:07 smithi052 systemd[1]: Starting Ceph mds.j for 4e059f08-da08-11ee-95c6-87774f69a715... 2024-03-04T09:28:07.855 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T09:28:07.967 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:07.967+0000 7fa39f15c640 1 -- 172.21.15.52:0/4137526042 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa3981072f0 msgr2=0x7fa398109740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:28:07.967 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:07.967+0000 7fa39f15c640 1 --2- 172.21.15.52:0/4137526042 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa3981072f0 0x7fa398109740 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fa390009ca0 tx=0x7fa39002c0f0 comp rx=0 tx=0).stop 2024-03-04T09:28:07.967 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:07.967+0000 7fa39f15c640 1 -- 172.21.15.52:0/4137526042 shutdown_connections 2024-03-04T09:28:07.967 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:07.967+0000 7fa39f15c640 1 --2- 172.21.15.52:0/4137526042 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa3981072f0 0x7fa398109740 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:07.967 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:07.967+0000 7fa39f15c640 1 --2- 172.21.15.52:0/4137526042 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa3981049c0 0x7fa398106db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:07.967 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:07.967+0000 7fa39f15c640 1 --2- 172.21.15.52:0/4137526042 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa398102090 0x7fa398104480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:07.967 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:07.967+0000 7fa39f15c640 1 -- 172.21.15.52:0/4137526042 >> 172.21.15.52:0/4137526042 conn(0x7fa3980fba80 msgr2=0x7fa3980fdec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:28:07.968 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:07.967+0000 7fa39f15c640 1 -- 172.21.15.52:0/4137526042 shutdown_connections 2024-03-04T09:28:07.968 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:07.967+0000 7fa39f15c640 1 -- 172.21.15.52:0/4137526042 wait complete. 2024-03-04T09:28:07.968 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:07.968+0000 7fa39f15c640 1 Processor -- start 2024-03-04T09:28:07.968 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:07.968+0000 7fa39f15c640 1 -- start start 2024-03-04T09:28:07.968 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:07.968+0000 7fa39f15c640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa398102090 0x7fa398197860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:28:07.968 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:07.968+0000 7fa39f15c640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa3981049c0 0x7fa398197da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:28:07.968 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:07.968+0000 7fa39f15c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa3981072f0 0x7fa39819ee10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:28:07.968 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:07.968+0000 7fa39f15c640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa39810d280 con 0x7fa398102090 2024-03-04T09:28:07.968 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:07.968+0000 7fa39f15c640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa39810d100 con 0x7fa3981072f0 2024-03-04T09:28:07.968 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:07.968+0000 7fa39f15c640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa39810d400 con 0x7fa3981049c0 2024-03-04T09:28:07.968 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:07.968+0000 7fa39cef8640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa398102090 0x7fa398197860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:28:07.969 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:07.968+0000 7fa39cef8640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa398102090 0x7fa398197860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:51184/0 (socket says 172.21.15.52:51184) 2024-03-04T09:28:07.969 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:07.968+0000 7fa39cef8640 1 -- 172.21.15.52:0/2569544902 learned_addr learned my addr 172.21.15.52:0/2569544902 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-04T09:28:07.969 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:07.968+0000 7fa39d6f9640 1 --2- 172.21.15.52:0/2569544902 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa3981072f0 0x7fa39819ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:28:07.969 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:07.968+0000 7fa38ffff640 1 --2- 172.21.15.52:0/2569544902 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa3981049c0 0x7fa398197da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:28:07.969 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:07.968+0000 7fa39cef8640 1 -- 172.21.15.52:0/2569544902 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa3981049c0 msgr2=0x7fa398197da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:28:07.969 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:07.968+0000 7fa39cef8640 1 --2- 172.21.15.52:0/2569544902 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa3981049c0 0x7fa398197da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:07.969 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:07.968+0000 7fa39cef8640 1 -- 172.21.15.52:0/2569544902 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa3981072f0 msgr2=0x7fa39819ee10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:28:07.969 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:07.968+0000 7fa39cef8640 1 --2- 172.21.15.52:0/2569544902 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa3981072f0 0x7fa39819ee10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:07.969 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:07.968+0000 7fa39cef8640 1 -- 172.21.15.52:0/2569544902 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa39819f480 con 0x7fa398102090 2024-03-04T09:28:07.969 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:07.968+0000 7fa39d6f9640 1 --2- 172.21.15.52:0/2569544902 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa3981072f0 0x7fa39819ee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T09:28:07.969 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:07.969+0000 7fa39cef8640 1 --2- 172.21.15.52:0/2569544902 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa398102090 0x7fa398197860 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fa39400d950 tx=0x7fa39400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:28:07.969 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:07.969+0000 7fa38dffb640 1 -- 172.21.15.52:0/2569544902 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa394017030 con 0x7fa398102090 2024-03-04T09:28:07.970 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:07.969+0000 7fa38dffb640 1 -- 172.21.15.52:0/2569544902 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa394004020 con 0x7fa398102090 2024-03-04T09:28:07.970 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:07.969+0000 7fa39f15c640 1 -- 172.21.15.52:0/2569544902 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa39819f770 con 0x7fa398102090 2024-03-04T09:28:07.970 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:07.969+0000 7fa38dffb640 1 -- 172.21.15.52:0/2569544902 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3940047a0 con 0x7fa398102090 2024-03-04T09:28:07.970 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:07.969+0000 7fa39f15c640 1 -- 172.21.15.52:0/2569544902 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa39819fd10 con 0x7fa398102090 2024-03-04T09:28:07.970 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:07.970+0000 7fa39f15c640 1 -- 172.21.15.52:0/2569544902 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa3980fd570 con 0x7fa398102090 2024-03-04T09:28:07.971 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:07.971+0000 7fa38dffb640 1 -- 172.21.15.52:0/2569544902 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 16) v1 ==== 136756+0+0 (secure 0 0 0) 0x7fa3940041d0 con 0x7fa398102090 2024-03-04T09:28:07.971 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:07.971+0000 7fa38dffb640 1 --2- 172.21.15.52:0/2569544902 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7fa3800a1e00 0x7fa3800a42c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:28:07.971 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:07.971+0000 7fa38dffb640 1 -- 172.21.15.52:0/2569544902 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fa3940cd3e0 con 0x7fa398102090 2024-03-04T09:28:07.971 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:07.971+0000 7fa38ffff640 1 --2- 172.21.15.52:0/2569544902 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7fa3800a1e00 0x7fa3800a42c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:28:07.971 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:07.972+0000 7fa38ffff640 1 --2- 172.21.15.52:0/2569544902 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7fa3800a1e00 0x7fa3800a42c0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fa398198d70 tx=0x7fa388005c50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:28:07.972 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:07.976+0000 7fa38dffb640 1 -- 172.21.15.52:0/2569544902 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fa39409acb0 con 0x7fa398102090 2024-03-04T09:28:08.085 INFO:journalctl@ceph.mds.j.smithi052.stdout:Mar 04 09:28:07 smithi052 podman[70401]: 2024-03-04 09:28:07.799493749 +0000 UTC m=+0.100411858 container create 942ac769c5f25b486442cf5e61daacf8570415d4721ef8df4785a5d62bf29e57 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5, name=ceph-4e059f08-da08-11ee-95c6-87774f69a715-mds-j, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, org.label-schema.license=GPLv2, org.label-schema.build-date=20240222, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-qdb-1bc250c, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-03-04T09:28:08.085 INFO:journalctl@ceph.mds.j.smithi052.stdout:Mar 04 09:28:07 smithi052 podman[70401]: 2024-03-04 09:28:07.716438613 +0000 UTC m=+0.017356682 image pull f88ffb32aa6a7d5dfdc954d9f096f0c9cc47981d05ec8d117ba859a22d8185ba quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5 2024-03-04T09:28:08.085 INFO:journalctl@ceph.mds.j.smithi052.stdout:Mar 04 09:28:08 smithi052 podman[70401]: 2024-03-04 09:28:08.010552986 +0000 UTC m=+0.311471066 container init 942ac769c5f25b486442cf5e61daacf8570415d4721ef8df4785a5d62bf29e57 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5, name=ceph-4e059f08-da08-11ee-95c6-87774f69a715-mds-j, GIT_BRANCH=HEAD, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240222, RELEASE=wip-lusov-qdb-1bc250c, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-04T09:28:08.085 INFO:journalctl@ceph.mds.j.smithi052.stdout:Mar 04 09:28:08 smithi052 podman[70401]: 2024-03-04 09:28:08.014166035 +0000 UTC m=+0.315084104 container start 942ac769c5f25b486442cf5e61daacf8570415d4721ef8df4785a5d62bf29e57 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5, name=ceph-4e059f08-da08-11ee-95c6-87774f69a715-mds-j, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, org.label-schema.build-date=20240222, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_BRANCH=HEAD, RELEASE=wip-lusov-qdb-1bc250c, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-03-04T09:28:08.085 INFO:journalctl@ceph.mds.j.smithi052.stdout:Mar 04 09:28:08 smithi052 ceph-mds[70459]: -- 172.21.15.52:0/3974168338 <== mon.0 v2:172.21.15.52:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55d17ce13a40 con 0x55d17cde6c00 2024-03-04T09:28:08.085 INFO:journalctl@ceph.mds.j.smithi052.stdout:Mar 04 09:28:08 smithi052 ceph-4e059f08-da08-11ee-95c6-87774f69a715-mds-j[70434]: starting mds.j at 2024-03-04T09:28:08.224 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:08.227+0000 7fa39f15c640 1 -- 172.21.15.52:0/2569544902 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa39802d2a0 con 0x7fa398102090 2024-03-04T09:28:08.224 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:08.228+0000 7fa38dffb640 1 -- 172.21.15.52:0/2569544902 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) v1 ==== 93+0+2119 (secure 0 0 0) 0x7fa39409a5b0 con 0x7fa398102090 2024-03-04T09:28:08.225 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-03-04T09:28:08.225 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24407,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6833/1818480323","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6832","nonce":1818480323},{"type":"v1","addr":"172.21.15.106:6833","nonce":1818480323}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24412,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6833/765768204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":765768204},{"type":"v1","addr":"172.21.15.191:6833","nonce":765768204}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[]} 2024-03-04T09:28:08.225 INFO:teuthology.orchestra.run.smithi052.stderr:dumped fsmap epoch 3 2024-03-04T09:28:08.227 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:08.230+0000 7fa39f15c640 1 -- 172.21.15.52:0/2569544902 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7fa3800a1e00 msgr2=0x7fa3800a42c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:28:08.227 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:08.230+0000 7fa39f15c640 1 --2- 172.21.15.52:0/2569544902 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7fa3800a1e00 0x7fa3800a42c0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fa398198d70 tx=0x7fa388005c50 comp rx=0 tx=0).stop 2024-03-04T09:28:08.227 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:08.230+0000 7fa39f15c640 1 -- 172.21.15.52:0/2569544902 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa398102090 msgr2=0x7fa398197860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:28:08.227 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:08.230+0000 7fa39f15c640 1 --2- 172.21.15.52:0/2569544902 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa398102090 0x7fa398197860 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fa39400d950 tx=0x7fa39400de10 comp rx=0 tx=0).stop 2024-03-04T09:28:08.227 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:08.230+0000 7fa39f15c640 1 -- 172.21.15.52:0/2569544902 shutdown_connections 2024-03-04T09:28:08.227 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:08.230+0000 7fa39f15c640 1 --2- 172.21.15.52:0/2569544902 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7fa3800a1e00 0x7fa3800a42c0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:08.227 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:08.230+0000 7fa39f15c640 1 --2- 172.21.15.52:0/2569544902 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa3981072f0 0x7fa39819ee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:08.227 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:08.230+0000 7fa39f15c640 1 --2- 172.21.15.52:0/2569544902 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa3981049c0 0x7fa398197da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:08.227 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:08.230+0000 7fa39f15c640 1 --2- 172.21.15.52:0/2569544902 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa398102090 0x7fa398197860 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:08.227 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:08.230+0000 7fa39f15c640 1 -- 172.21.15.52:0/2569544902 >> 172.21.15.52:0/2569544902 conn(0x7fa3980fba80 msgr2=0x7fa3981029d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:28:08.227 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:08.230+0000 7fa39f15c640 1 -- 172.21.15.52:0/2569544902 shutdown_connections 2024-03-04T09:28:08.228 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:08.230+0000 7fa39f15c640 1 -- 172.21.15.52:0/2569544902 wait complete. 2024-03-04T09:28:08.587 INFO:journalctl@ceph.mds.j.smithi052.stdout:Mar 04 09:28:08 smithi052 bash[70401]: 942ac769c5f25b486442cf5e61daacf8570415d4721ef8df4785a5d62bf29e57 2024-03-04T09:28:08.587 INFO:journalctl@ceph.mds.j.smithi052.stdout:Mar 04 09:28:08 smithi052 systemd[1]: Started Ceph mds.j for 4e059f08-da08-11ee-95c6-87774f69a715. 2024-03-04T09:28:09.396 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:28:09 smithi052 ceph-mon[32473]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-04T09:28:09.397 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:28:09 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:28:09.397 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:28:09 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:28:09.397 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:28:09 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:28:09.397 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:28:09 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-04T09:28:09.397 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:28:09 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-04T09:28:09.397 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:28:09 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T09:28:09.397 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:28:09 smithi052 ceph-mon[32473]: Deploying daemon mds.d on smithi052 2024-03-04T09:28:09.397 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:28:09 smithi052 ceph-mon[32473]: from='client.? 172.21.15.52:0/2569544902' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-04T09:28:09.397 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:28:09 smithi052 ceph-mon[32473]: mds.? [v2:172.21.15.52:6834/1007961019,v1:172.21.15.52:6835/1007961019] up:boot 2024-03-04T09:28:09.397 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:28:09 smithi052 ceph-mon[32473]: fsmap 3 up:standby 2024-03-04T09:28:09.398 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:28:09 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-03-04T09:28:09.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:28:09 smithi191 ceph-mon[36816]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-04T09:28:09.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:28:09 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:28:09.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:28:09 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:28:09.587 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:28:09 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:28:09.587 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:28:09 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-04T09:28:09.587 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:28:09 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-04T09:28:09.587 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:28:09 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T09:28:09.587 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:28:09 smithi191 ceph-mon[36816]: Deploying daemon mds.d on smithi052 2024-03-04T09:28:09.587 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:28:09 smithi191 ceph-mon[36816]: from='client.? 172.21.15.52:0/2569544902' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-04T09:28:09.587 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:28:09 smithi191 ceph-mon[36816]: mds.? [v2:172.21.15.52:6834/1007961019,v1:172.21.15.52:6835/1007961019] up:boot 2024-03-04T09:28:09.587 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:28:09 smithi191 ceph-mon[36816]: fsmap 3 up:standby 2024-03-04T09:28:09.587 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:28:09 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-03-04T09:28:09.631 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:28:09 smithi106 ceph-mon[36484]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-04T09:28:09.631 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:28:09 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:28:09.631 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:28:09 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:28:09.631 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:28:09 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:28:09.632 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:28:09 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-04T09:28:09.632 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:28:09 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-04T09:28:09.632 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:28:09 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T09:28:09.632 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:28:09 smithi106 ceph-mon[36484]: Deploying daemon mds.d on smithi052 2024-03-04T09:28:09.632 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:28:09 smithi106 ceph-mon[36484]: from='client.? 172.21.15.52:0/2569544902' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-04T09:28:09.632 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:28:09 smithi106 ceph-mon[36484]: mds.? [v2:172.21.15.52:6834/1007961019,v1:172.21.15.52:6835/1007961019] up:boot 2024-03-04T09:28:09.632 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:28:09 smithi106 ceph-mon[36484]: fsmap 3 up:standby 2024-03-04T09:28:09.632 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:28:09 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-03-04T09:28:10.238 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T09:28:10.415 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:10.417+0000 7f3191cdf640 1 -- 172.21.15.52:0/35194041 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f318c1027e0 msgr2=0x7f318c102c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:28:10.415 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:10.417+0000 7f3191cdf640 1 --2- 172.21.15.52:0/35194041 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f318c1027e0 0x7f318c102c60 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f3180009900 tx=0x7f318002bc30 comp rx=0 tx=0).stop 2024-03-04T09:28:10.415 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:10.418+0000 7f3191cdf640 1 -- 172.21.15.52:0/35194041 shutdown_connections 2024-03-04T09:28:10.415 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:10.418+0000 7f3191cdf640 1 --2- 172.21.15.52:0/35194041 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f318c1031a0 0x7f318c109a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:10.415 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:10.418+0000 7f3191cdf640 1 --2- 172.21.15.52:0/35194041 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f318c1027e0 0x7f318c102c60 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:10.416 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:10.418+0000 7f3191cdf640 1 --2- 172.21.15.52:0/35194041 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f318c1015f0 0x7f318c101a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:10.416 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:10.418+0000 7f3191cdf640 1 -- 172.21.15.52:0/35194041 >> 172.21.15.52:0/35194041 conn(0x7f318c0fcb80 msgr2=0x7f318c0fefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:28:10.416 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:10.418+0000 7f3191cdf640 1 -- 172.21.15.52:0/35194041 shutdown_connections 2024-03-04T09:28:10.416 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:10.418+0000 7f3191cdf640 1 -- 172.21.15.52:0/35194041 wait complete. 2024-03-04T09:28:10.416 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:10.418+0000 7f3191cdf640 1 Processor -- start 2024-03-04T09:28:10.418 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:10.419+0000 7f3191cdf640 1 -- start start 2024-03-04T09:28:10.419 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:10.419+0000 7f3191cdf640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f318c1015f0 0x7f318c075380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:28:10.419 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:10.419+0000 7f3191cdf640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f318c1027e0 0x7f318c0739c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:28:10.419 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:10.419+0000 7f3191cdf640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f318c1031a0 0x7f318c073f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:28:10.419 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:10.419+0000 7f3191cdf640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f318c10d270 con 0x7f318c1027e0 2024-03-04T09:28:10.419 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:10.419+0000 7f3191cdf640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f318c10d0f0 con 0x7f318c1015f0 2024-03-04T09:28:10.419 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:10.419+0000 7f3191cdf640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f318c10d3f0 con 0x7f318c1031a0 2024-03-04T09:28:10.419 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:10.419+0000 7f318affd640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f318c1027e0 0x7f318c0739c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:28:10.419 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:10.419+0000 7f318affd640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f318c1027e0 0x7f318c0739c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:51228/0 (socket says 172.21.15.52:51228) 2024-03-04T09:28:10.419 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:10.419+0000 7f318affd640 1 -- 172.21.15.52:0/2004772381 learned_addr learned my addr 172.21.15.52:0/2004772381 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-04T09:28:10.419 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:10.419+0000 7f318b7fe640 1 --2- 172.21.15.52:0/2004772381 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f318c1015f0 0x7f318c075380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:28:10.420 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:10.419+0000 7f318bfff640 1 --2- 172.21.15.52:0/2004772381 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f318c1031a0 0x7f318c073f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:28:10.420 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:10.419+0000 7f318affd640 1 -- 172.21.15.52:0/2004772381 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f318c1031a0 msgr2=0x7f318c073f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:28:10.420 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:10.419+0000 7f318affd640 1 --2- 172.21.15.52:0/2004772381 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f318c1031a0 0x7f318c073f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:10.420 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:10.419+0000 7f318affd640 1 -- 172.21.15.52:0/2004772381 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f318c1015f0 msgr2=0x7f318c075380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:28:10.420 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:10.419+0000 7f318affd640 1 --2- 172.21.15.52:0/2004772381 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f318c1015f0 0x7f318c075380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:10.420 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:10.419+0000 7f318affd640 1 -- 172.21.15.52:0/2004772381 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f318c074790 con 0x7f318c1027e0 2024-03-04T09:28:10.420 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:10.419+0000 7f318affd640 1 --2- 172.21.15.52:0/2004772381 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f318c1027e0 0x7f318c0739c0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f3180009230 tx=0x7f3180005dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:28:10.420 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:10.420+0000 7f3188ff9640 1 -- 172.21.15.52:0/2004772381 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3180041070 con 0x7f318c1027e0 2024-03-04T09:28:10.420 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:10.420+0000 7f3188ff9640 1 -- 172.21.15.52:0/2004772381 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f31800054e0 con 0x7f318c1027e0 2024-03-04T09:28:10.420 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:10.420+0000 7f3191cdf640 1 -- 172.21.15.52:0/2004772381 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f318c1a19a0 con 0x7f318c1027e0 2024-03-04T09:28:10.420 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:10.420+0000 7f3188ff9640 1 -- 172.21.15.52:0/2004772381 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3180034440 con 0x7f318c1027e0 2024-03-04T09:28:10.420 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:10.420+0000 7f3191cdf640 1 -- 172.21.15.52:0/2004772381 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f318c1a1ef0 con 0x7f318c1027e0 2024-03-04T09:28:10.421 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:10.421+0000 7f3191cdf640 1 -- 172.21.15.52:0/2004772381 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f318c108910 con 0x7f318c1027e0 2024-03-04T09:28:10.421 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:10.422+0000 7f3188ff9640 1 -- 172.21.15.52:0/2004772381 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 16) v1 ==== 136756+0+0 (secure 0 0 0) 0x7f3180044070 con 0x7f318c1027e0 2024-03-04T09:28:10.421 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:10.422+0000 7f3188ff9640 1 --2- 172.21.15.52:0/2004772381 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f31740a1e00 0x7f31740a42c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:28:10.422 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:10.423+0000 7f3188ff9640 1 -- 172.21.15.52:0/2004772381 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f31800ef690 con 0x7f318c1027e0 2024-03-04T09:28:10.422 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:10.423+0000 7f318b7fe640 1 --2- 172.21.15.52:0/2004772381 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f31740a1e00 0x7f31740a42c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:28:10.422 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:10.423+0000 7f318b7fe640 1 --2- 172.21.15.52:0/2004772381 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f31740a1e00 0x7f31740a42c0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f318c102640 tx=0x7f317c00b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:28:10.424 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:10.427+0000 7f3188ff9640 1 -- 172.21.15.52:0/2004772381 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f31800bcec0 con 0x7f318c1027e0 2024-03-04T09:28:10.655 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:10.658+0000 7f3191cdf640 1 -- 172.21.15.52:0/2004772381 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f318c0621d0 con 0x7f318c1027e0 2024-03-04T09:28:10.656 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:10.659+0000 7f3188ff9640 1 -- 172.21.15.52:0/2004772381 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) v1 ==== 93+0+2910 (secure 0 0 0) 0x7f31800bc7c0 con 0x7f318c1027e0 2024-03-04T09:28:10.656 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-03-04T09:28:10.656 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":4,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6835/1007961019","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6834","nonce":1007961019},{"type":"v1","addr":"172.21.15.52:6835","nonce":1007961019}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":24407,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6833/1818480323","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6832","nonce":1818480323},{"type":"v1","addr":"172.21.15.106:6833","nonce":1818480323}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24412,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6833/765768204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":765768204},{"type":"v1","addr":"172.21.15.191:6833","nonce":765768204}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[]} 2024-03-04T09:28:10.657 INFO:teuthology.orchestra.run.smithi052.stderr:dumped fsmap epoch 4 2024-03-04T09:28:10.658 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:10.661+0000 7f3191cdf640 1 -- 172.21.15.52:0/2004772381 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f31740a1e00 msgr2=0x7f31740a42c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:28:10.658 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:10.661+0000 7f3191cdf640 1 --2- 172.21.15.52:0/2004772381 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f31740a1e00 0x7f31740a42c0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f318c102640 tx=0x7f317c00b390 comp rx=0 tx=0).stop 2024-03-04T09:28:10.658 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:10.661+0000 7f3191cdf640 1 -- 172.21.15.52:0/2004772381 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f318c1027e0 msgr2=0x7f318c0739c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:28:10.659 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:10.661+0000 7f3191cdf640 1 --2- 172.21.15.52:0/2004772381 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f318c1027e0 0x7f318c0739c0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f3180009230 tx=0x7f3180005dd0 comp rx=0 tx=0).stop 2024-03-04T09:28:10.659 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:10.661+0000 7f3191cdf640 1 -- 172.21.15.52:0/2004772381 shutdown_connections 2024-03-04T09:28:10.659 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:10.661+0000 7f3191cdf640 1 --2- 172.21.15.52:0/2004772381 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f31740a1e00 0x7f31740a42c0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:10.659 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:10.661+0000 7f3191cdf640 1 --2- 172.21.15.52:0/2004772381 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f318c1031a0 0x7f318c073f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:10.659 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:10.661+0000 7f3191cdf640 1 --2- 172.21.15.52:0/2004772381 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f318c1027e0 0x7f318c0739c0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:10.659 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:10.661+0000 7f3191cdf640 1 --2- 172.21.15.52:0/2004772381 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f318c1015f0 0x7f318c075380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:10.659 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:10.661+0000 7f3191cdf640 1 -- 172.21.15.52:0/2004772381 >> 172.21.15.52:0/2004772381 conn(0x7f318c0fcb80 msgr2=0x7f318c105ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:28:10.659 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:10.661+0000 7f3191cdf640 1 -- 172.21.15.52:0/2004772381 shutdown_connections 2024-03-04T09:28:10.659 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:10.661+0000 7f3191cdf640 1 -- 172.21.15.52:0/2004772381 wait complete. 2024-03-04T09:28:11.177 INFO:journalctl@ceph.mds.d.smithi052.stdout:Mar 04 09:28:10 smithi052 systemd[1]: Started Ceph mds.d for 4e059f08-da08-11ee-95c6-87774f69a715. 2024-03-04T09:28:11.585 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:28:11 smithi052 ceph-mon[32473]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-04T09:28:11.585 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:28:11 smithi052 ceph-mon[32473]: from='client.? 172.21.15.52:0/2004772381' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-04T09:28:11.585 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:28:11 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:28:11.585 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:28:11 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:28:11.585 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:28:11 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:28:11.585 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:28:11 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-04T09:28:11.585 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:28:11 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-04T09:28:11.585 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:28:11 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T09:28:11.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:28:11 smithi191 ceph-mon[36816]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-04T09:28:11.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:28:11 smithi191 ceph-mon[36816]: from='client.? 172.21.15.52:0/2004772381' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-04T09:28:11.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:28:11 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:28:11.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:28:11 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:28:11.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:28:11 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:28:11.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:28:11 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-04T09:28:11.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:28:11 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-04T09:28:11.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:28:11 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T09:28:11.612 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:28:11 smithi106 ceph-mon[36484]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-04T09:28:11.612 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:28:11 smithi106 ceph-mon[36484]: from='client.? 172.21.15.52:0/2004772381' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-04T09:28:11.613 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:28:11 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:28:11.613 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:28:11 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:28:11.613 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:28:11 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:28:11.613 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:28:11 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-04T09:28:11.613 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:28:11 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-04T09:28:11.613 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:28:11 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T09:28:12.585 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:28:12 smithi052 ceph-mon[32473]: Deploying daemon mds.e on smithi106 2024-03-04T09:28:12.585 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:28:12 smithi052 ceph-mon[32473]: mds.? [v2:172.21.15.52:6836/3753881283,v1:172.21.15.52:6837/3753881283] up:boot 2024-03-04T09:28:12.585 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:28:12 smithi052 ceph-mon[32473]: fsmap 4 up:standby 2024-03-04T09:28:12.585 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:28:12 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-03-04T09:28:12.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:28:12 smithi191 ceph-mon[36816]: Deploying daemon mds.e on smithi106 2024-03-04T09:28:12.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:28:12 smithi191 ceph-mon[36816]: mds.? [v2:172.21.15.52:6836/3753881283,v1:172.21.15.52:6837/3753881283] up:boot 2024-03-04T09:28:12.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:28:12 smithi191 ceph-mon[36816]: fsmap 4 up:standby 2024-03-04T09:28:12.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:28:12 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-03-04T09:28:12.631 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:28:12 smithi106 ceph-mon[36484]: Deploying daemon mds.e on smithi106 2024-03-04T09:28:12.631 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:28:12 smithi106 ceph-mon[36484]: mds.? [v2:172.21.15.52:6836/3753881283,v1:172.21.15.52:6837/3753881283] up:boot 2024-03-04T09:28:12.631 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:28:12 smithi106 ceph-mon[36484]: fsmap 4 up:standby 2024-03-04T09:28:12.631 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:28:12 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-03-04T09:28:12.670 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T09:28:12.789 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:12.790+0000 7f531cbb5640 1 -- 172.21.15.52:0/3589133430 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f53181004f0 msgr2=0x7f5318100910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:28:12.790 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:12.790+0000 7f531cbb5640 1 --2- 172.21.15.52:0/3589133430 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f53181004f0 0x7f5318100910 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f5310009900 tx=0x7f531002bc30 comp rx=0 tx=0).stop 2024-03-04T09:28:12.790 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:12.790+0000 7f531cbb5640 1 -- 172.21.15.52:0/3589133430 shutdown_connections 2024-03-04T09:28:12.790 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:12.790+0000 7f531cbb5640 1 --2- 172.21.15.52:0/3589133430 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f53181020a0 0x7f5318108930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:12.790 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:12.790+0000 7f531cbb5640 1 --2- 172.21.15.52:0/3589133430 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f53181016e0 0x7f5318101b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:12.790 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:12.790+0000 7f531cbb5640 1 --2- 172.21.15.52:0/3589133430 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f53181004f0 0x7f5318100910 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:12.790 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:12.790+0000 7f531cbb5640 1 -- 172.21.15.52:0/3589133430 >> 172.21.15.52:0/3589133430 conn(0x7f53180fba60 msgr2=0x7f53180fdea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:28:12.790 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:12.790+0000 7f531cbb5640 1 -- 172.21.15.52:0/3589133430 shutdown_connections 2024-03-04T09:28:12.790 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:12.790+0000 7f531cbb5640 1 -- 172.21.15.52:0/3589133430 wait complete. 2024-03-04T09:28:12.790 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:12.790+0000 7f531cbb5640 1 Processor -- start 2024-03-04T09:28:12.790 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:12.791+0000 7f531cbb5640 1 -- start start 2024-03-04T09:28:12.790 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:12.791+0000 7f531cbb5640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f53181004f0 0x7f53181977f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:28:12.791 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:12.791+0000 7f531cbb5640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f53181016e0 0x7f5318197d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:28:12.791 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:12.791+0000 7f531cbb5640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f53181020a0 0x7f531819eda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:28:12.791 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:12.791+0000 7f531cbb5640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f531810c180 con 0x7f53181020a0 2024-03-04T09:28:12.791 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:12.791+0000 7f531cbb5640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f531810c000 con 0x7f53181016e0 2024-03-04T09:28:12.791 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:12.791+0000 7f531cbb5640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f531810c300 con 0x7f53181004f0 2024-03-04T09:28:12.791 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:12.791+0000 7f5316d9d640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f53181020a0 0x7f531819eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:28:12.791 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:12.791+0000 7f5316d9d640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f53181020a0 0x7f531819eda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:51274/0 (socket says 172.21.15.52:51274) 2024-03-04T09:28:12.791 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:12.791+0000 7f5316d9d640 1 -- 172.21.15.52:0/3241194428 learned_addr learned my addr 172.21.15.52:0/3241194428 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-04T09:28:12.791 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:12.791+0000 7f5315d9b640 1 --2- 172.21.15.52:0/3241194428 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f53181016e0 0x7f5318197d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:28:12.791 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:12.791+0000 7f531659c640 1 --2- 172.21.15.52:0/3241194428 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f53181004f0 0x7f53181977f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:28:12.791 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:12.791+0000 7f5316d9d640 1 -- 172.21.15.52:0/3241194428 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f53181004f0 msgr2=0x7f53181977f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:28:12.791 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:12.791+0000 7f5316d9d640 1 --2- 172.21.15.52:0/3241194428 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f53181004f0 0x7f53181977f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:12.791 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:12.791+0000 7f5316d9d640 1 -- 172.21.15.52:0/3241194428 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f53181016e0 msgr2=0x7f5318197d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:28:12.791 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:12.791+0000 7f5316d9d640 1 --2- 172.21.15.52:0/3241194428 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f53181016e0 0x7f5318197d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:12.792 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:12.791+0000 7f5316d9d640 1 -- 172.21.15.52:0/3241194428 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f531819f410 con 0x7f53181020a0 2024-03-04T09:28:12.792 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:12.791+0000 7f5316d9d640 1 --2- 172.21.15.52:0/3241194428 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f53181020a0 0x7f531819eda0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f530800cce0 tx=0x7f530800a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:28:12.792 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:12.792+0000 7f530f7fe640 1 -- 172.21.15.52:0/3241194428 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f530800f450 con 0x7f53181020a0 2024-03-04T09:28:12.792 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:12.792+0000 7f530f7fe640 1 -- 172.21.15.52:0/3241194428 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f530800fad0 con 0x7f53181020a0 2024-03-04T09:28:12.792 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:12.792+0000 7f531cbb5640 1 -- 172.21.15.52:0/3241194428 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f531819f700 con 0x7f53181020a0 2024-03-04T09:28:12.792 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:12.792+0000 7f530f7fe640 1 -- 172.21.15.52:0/3241194428 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5308018b60 con 0x7f53181020a0 2024-03-04T09:28:12.792 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:12.792+0000 7f531cbb5640 1 -- 172.21.15.52:0/3241194428 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f531819fc50 con 0x7f53181020a0 2024-03-04T09:28:12.792 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:12.793+0000 7f531cbb5640 1 -- 172.21.15.52:0/3241194428 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f531819ff10 con 0x7f53181020a0 2024-03-04T09:28:12.793 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:12.794+0000 7f530f7fe640 1 -- 172.21.15.52:0/3241194428 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 16) v1 ==== 136756+0+0 (secure 0 0 0) 0x7f530800fc80 con 0x7f53181020a0 2024-03-04T09:28:12.793 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:12.794+0000 7f530f7fe640 1 --2- 172.21.15.52:0/3241194428 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f53040a1e00 0x7f53040a42c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:28:12.793 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:12.794+0000 7f531659c640 1 --2- 172.21.15.52:0/3241194428 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f53040a1e00 0x7f53040a42c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:28:12.793 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:12.794+0000 7f530f7fe640 1 -- 172.21.15.52:0/3241194428 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f53080ce090 con 0x7f53181020a0 2024-03-04T09:28:12.793 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:12.795+0000 7f531659c640 1 --2- 172.21.15.52:0/3241194428 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f53040a1e00 0x7f53040a42c0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f5310000c00 tx=0x7f5310004f10 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:28:12.795 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:12.798+0000 7f530f7fe640 1 -- 172.21.15.52:0/3241194428 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f5308014030 con 0x7f53181020a0 2024-03-04T09:28:13.016 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:13.019+0000 7f531cbb5640 1 -- 172.21.15.52:0/3241194428 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f53181a00e0 con 0x7f53181020a0 2024-03-04T09:28:13.016 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:13.019+0000 7f530f7fe640 1 -- 172.21.15.52:0/3241194428 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) v1 ==== 93+0+3701 (secure 0 0 0) 0x7f530809b9e0 con 0x7f53181020a0 2024-03-04T09:28:13.016 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-03-04T09:28:13.016 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":5,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6835/1007961019","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6834","nonce":1007961019},{"type":"v1","addr":"172.21.15.52:6835","nonce":1007961019}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6837/3753881283","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6836","nonce":3753881283},{"type":"v1","addr":"172.21.15.52:6837","nonce":3753881283}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24407,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6833/1818480323","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6832","nonce":1818480323},{"type":"v1","addr":"172.21.15.106:6833","nonce":1818480323}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24412,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6833/765768204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":765768204},{"type":"v1","addr":"172.21.15.191:6833","nonce":765768204}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[]} 2024-03-04T09:28:13.017 INFO:teuthology.orchestra.run.smithi052.stderr:dumped fsmap epoch 5 2024-03-04T09:28:13.018 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:13.021+0000 7f531cbb5640 1 -- 172.21.15.52:0/3241194428 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f53040a1e00 msgr2=0x7f53040a42c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:28:13.018 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:13.021+0000 7f531cbb5640 1 --2- 172.21.15.52:0/3241194428 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f53040a1e00 0x7f53040a42c0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f5310000c00 tx=0x7f5310004f10 comp rx=0 tx=0).stop 2024-03-04T09:28:13.018 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:13.021+0000 7f531cbb5640 1 -- 172.21.15.52:0/3241194428 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f53181020a0 msgr2=0x7f531819eda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:28:13.018 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:13.021+0000 7f531cbb5640 1 --2- 172.21.15.52:0/3241194428 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f53181020a0 0x7f531819eda0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f530800cce0 tx=0x7f530800a620 comp rx=0 tx=0).stop 2024-03-04T09:28:13.018 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:13.021+0000 7f531cbb5640 1 -- 172.21.15.52:0/3241194428 shutdown_connections 2024-03-04T09:28:13.018 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:13.021+0000 7f531cbb5640 1 --2- 172.21.15.52:0/3241194428 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f53040a1e00 0x7f53040a42c0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:13.018 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:13.021+0000 7f531cbb5640 1 --2- 172.21.15.52:0/3241194428 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f53181020a0 0x7f531819eda0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:13.019 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:13.021+0000 7f531cbb5640 1 --2- 172.21.15.52:0/3241194428 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f53181016e0 0x7f5318197d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:13.019 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:13.021+0000 7f531cbb5640 1 --2- 172.21.15.52:0/3241194428 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f53181004f0 0x7f53181977f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:13.019 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:13.021+0000 7f531cbb5640 1 -- 172.21.15.52:0/3241194428 >> 172.21.15.52:0/3241194428 conn(0x7f53180fba60 msgr2=0x7f5318068730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:28:13.019 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:13.021+0000 7f531cbb5640 1 -- 172.21.15.52:0/3241194428 shutdown_connections 2024-03-04T09:28:13.019 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:13.021+0000 7f531cbb5640 1 -- 172.21.15.52:0/3241194428 wait complete. 2024-03-04T09:28:13.335 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:28:13 smithi052 ceph-mon[32473]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-04T09:28:13.335 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:28:13 smithi052 ceph-mon[32473]: from='client.? 172.21.15.52:0/3241194428' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-04T09:28:13.382 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:28:13 smithi106 ceph-mon[36484]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-04T09:28:13.382 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:28:13 smithi106 ceph-mon[36484]: from='client.? 172.21.15.52:0/3241194428' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-04T09:28:13.382 INFO:journalctl@ceph.mds.e.smithi106.stdout:Mar 04 09:28:13 smithi106 systemd[1]: Starting Ceph mds.e for 4e059f08-da08-11ee-95c6-87774f69a715... 2024-03-04T09:28:13.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:28:13 smithi191 ceph-mon[36816]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-04T09:28:13.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:28:13 smithi191 ceph-mon[36816]: from='client.? 172.21.15.52:0/3241194428' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-04T09:28:13.710 INFO:journalctl@ceph.mds.e.smithi106.stdout:Mar 04 09:28:13 smithi106 podman[66012]: 2024-03-04 09:28:13.356325464 +0000 UTC m=+0.017739076 image pull f88ffb32aa6a7d5dfdc954d9f096f0c9cc47981d05ec8d117ba859a22d8185ba quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5 2024-03-04T09:28:13.710 INFO:journalctl@ceph.mds.e.smithi106.stdout:Mar 04 09:28:13 smithi106 podman[66012]: 2024-03-04 09:28:13.461254846 +0000 UTC m=+0.122668468 container create 2bbb1b9af83dec2c854defb30c77bfdc68abf11068f3aa49fd499116ce7870f9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5, name=ceph-4e059f08-da08-11ee-95c6-87774f69a715-mds-e, GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20240222, RELEASE=wip-lusov-qdb-1bc250c, org.label-schema.schema-version=1.0, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-04T09:28:13.711 INFO:journalctl@ceph.mds.e.smithi106.stdout:Mar 04 09:28:13 smithi106 podman[66012]: 2024-03-04 09:28:13.713927097 +0000 UTC m=+0.375340699 container init 2bbb1b9af83dec2c854defb30c77bfdc68abf11068f3aa49fd499116ce7870f9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5, name=ceph-4e059f08-da08-11ee-95c6-87774f69a715-mds-e, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240222, RELEASE=wip-lusov-qdb-1bc250c, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8) 2024-03-04T09:28:14.131 INFO:journalctl@ceph.mds.e.smithi106.stdout:Mar 04 09:28:13 smithi106 podman[66012]: 2024-03-04 09:28:13.716854129 +0000 UTC m=+0.378267736 container start 2bbb1b9af83dec2c854defb30c77bfdc68abf11068f3aa49fd499116ce7870f9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5, name=ceph-4e059f08-da08-11ee-95c6-87774f69a715-mds-e, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240222, RELEASE=wip-lusov-qdb-1bc250c, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-03-04T09:28:14.131 INFO:journalctl@ceph.mds.e.smithi106.stdout:Mar 04 09:28:13 smithi106 ceph-mds[66031]: -- 172.21.15.106:0/445531971 <== mon.2 v2:172.21.15.106:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55bfc6599a40 con 0x55bfc656c800 2024-03-04T09:28:14.131 INFO:journalctl@ceph.mds.e.smithi106.stdout:Mar 04 09:28:13 smithi106 ceph-4e059f08-da08-11ee-95c6-87774f69a715-mds-e[66027]: starting mds.e at 2024-03-04T09:28:14.131 INFO:journalctl@ceph.mds.e.smithi106.stdout:Mar 04 09:28:13 smithi106 bash[66012]: 2bbb1b9af83dec2c854defb30c77bfdc68abf11068f3aa49fd499116ce7870f9 2024-03-04T09:28:14.131 INFO:journalctl@ceph.mds.e.smithi106.stdout:Mar 04 09:28:13 smithi106 systemd[1]: Started Ceph mds.e for 4e059f08-da08-11ee-95c6-87774f69a715. 2024-03-04T09:28:15.030 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T09:28:15.086 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:28:14 smithi191 ceph-mon[36816]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-04T09:28:15.086 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:28:14 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:28:15.087 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:28:14 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:28:15.087 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:28:14 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:28:15.087 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:28:14 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-04T09:28:15.087 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:28:14 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-04T09:28:15.087 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:28:14 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T09:28:15.087 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:28:14 smithi191 ceph-mon[36816]: Deploying daemon mds.f on smithi191 2024-03-04T09:28:15.087 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:28:14 smithi191 ceph-mon[36816]: mds.? [v2:172.21.15.106:6834/1541069897,v1:172.21.15.106:6835/1541069897] up:boot 2024-03-04T09:28:15.087 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:28:14 smithi191 ceph-mon[36816]: fsmap 5 up:standby 2024-03-04T09:28:15.087 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:28:14 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-03-04T09:28:15.131 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:28:14 smithi106 ceph-mon[36484]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-04T09:28:15.132 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:28:14 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:28:15.132 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:28:14 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:28:15.132 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:28:14 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:28:15.132 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:28:14 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-04T09:28:15.132 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:28:14 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-04T09:28:15.132 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:28:14 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T09:28:15.132 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:28:14 smithi106 ceph-mon[36484]: Deploying daemon mds.f on smithi191 2024-03-04T09:28:15.132 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:28:14 smithi106 ceph-mon[36484]: mds.? [v2:172.21.15.106:6834/1541069897,v1:172.21.15.106:6835/1541069897] up:boot 2024-03-04T09:28:15.132 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:28:14 smithi106 ceph-mon[36484]: fsmap 5 up:standby 2024-03-04T09:28:15.132 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:28:14 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-03-04T09:28:15.146 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:15.149+0000 7faef6439640 1 -- 172.21.15.52:0/11600363 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7faef01020c0 msgr2=0x7faef0108950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:28:15.146 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:15.149+0000 7faef6439640 1 --2- 172.21.15.52:0/11600363 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7faef01020c0 0x7faef0108950 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7faee8009ca0 tx=0x7faee802c0f0 comp rx=0 tx=0).stop 2024-03-04T09:28:15.146 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:15.149+0000 7faef6439640 1 -- 172.21.15.52:0/11600363 shutdown_connections 2024-03-04T09:28:15.146 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:15.149+0000 7faef6439640 1 --2- 172.21.15.52:0/11600363 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7faef01020c0 0x7faef0108950 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:15.146 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:15.149+0000 7faef6439640 1 --2- 172.21.15.52:0/11600363 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7faef0101700 0x7faef0101b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:15.146 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:15.149+0000 7faef6439640 1 --2- 172.21.15.52:0/11600363 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7faef0100510 0x7faef0100930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:15.146 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:15.149+0000 7faef6439640 1 -- 172.21.15.52:0/11600363 >> 172.21.15.52:0/11600363 conn(0x7faef00fba80 msgr2=0x7faef00fdec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:28:15.147 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:15.149+0000 7faef6439640 1 -- 172.21.15.52:0/11600363 shutdown_connections 2024-03-04T09:28:15.147 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:15.149+0000 7faef6439640 1 -- 172.21.15.52:0/11600363 wait complete. 2024-03-04T09:28:15.147 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:15.149+0000 7faef6439640 1 Processor -- start 2024-03-04T09:28:15.147 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:15.149+0000 7faef6439640 1 -- start start 2024-03-04T09:28:15.149 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:15.150+0000 7faef6439640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7faef0100510 0x7faef0196700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:28:15.149 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:15.150+0000 7faef6439640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7faef0101700 0x7faef0196c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:28:15.149 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:15.150+0000 7faef6439640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7faef01020c0 0x7faef019dcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:28:15.149 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:15.150+0000 7faef6439640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faef010c170 con 0x7faef0100510 2024-03-04T09:28:15.149 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:15.150+0000 7faef6439640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faef010bff0 con 0x7faef01020c0 2024-03-04T09:28:15.149 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:15.150+0000 7faef6439640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faef010c2f0 con 0x7faef0101700 2024-03-04T09:28:15.149 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:15.150+0000 7faeeffff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7faef0100510 0x7faef0196700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:28:15.149 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:15.150+0000 7faeeffff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7faef0100510 0x7faef0196700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:41448/0 (socket says 172.21.15.52:41448) 2024-03-04T09:28:15.149 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:15.150+0000 7faeeffff640 1 -- 172.21.15.52:0/1049179793 learned_addr learned my addr 172.21.15.52:0/1049179793 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-04T09:28:15.150 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:15.150+0000 7faeef7fe640 1 --2- 172.21.15.52:0/1049179793 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7faef0101700 0x7faef0196c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:28:15.150 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:15.150+0000 7faef49d6640 1 --2- 172.21.15.52:0/1049179793 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7faef01020c0 0x7faef019dcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:28:15.150 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:15.150+0000 7faeeffff640 1 -- 172.21.15.52:0/1049179793 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7faef0101700 msgr2=0x7faef0196c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:28:15.150 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:15.150+0000 7faeeffff640 1 --2- 172.21.15.52:0/1049179793 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7faef0101700 0x7faef0196c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:15.150 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:15.150+0000 7faeeffff640 1 -- 172.21.15.52:0/1049179793 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7faef01020c0 msgr2=0x7faef019dcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:28:15.150 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:15.150+0000 7faeeffff640 1 --2- 172.21.15.52:0/1049179793 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7faef01020c0 0x7faef019dcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:15.150 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:15.150+0000 7faeeffff640 1 -- 172.21.15.52:0/1049179793 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faef019e320 con 0x7faef0100510 2024-03-04T09:28:15.150 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:15.150+0000 7faeeffff640 1 --2- 172.21.15.52:0/1049179793 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7faef0100510 0x7faef0196700 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7faed800c970 tx=0x7faed800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:28:15.150 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:15.150+0000 7faeed7fa640 1 -- 172.21.15.52:0/1049179793 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faed8007c10 con 0x7faef0100510 2024-03-04T09:28:15.150 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:15.151+0000 7faeed7fa640 1 -- 172.21.15.52:0/1049179793 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7faed8007db0 con 0x7faef0100510 2024-03-04T09:28:15.150 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:15.151+0000 7faef6439640 1 -- 172.21.15.52:0/1049179793 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faef019e610 con 0x7faef0100510 2024-03-04T09:28:15.150 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:15.151+0000 7faeed7fa640 1 -- 172.21.15.52:0/1049179793 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faed800f940 con 0x7faef0100510 2024-03-04T09:28:15.150 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:15.151+0000 7faef6439640 1 -- 172.21.15.52:0/1049179793 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faef019ebb0 con 0x7faef0100510 2024-03-04T09:28:15.150 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:15.152+0000 7faef6439640 1 -- 172.21.15.52:0/1049179793 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faef0107760 con 0x7faef0100510 2024-03-04T09:28:15.151 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:15.153+0000 7faeed7fa640 1 -- 172.21.15.52:0/1049179793 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 16) v1 ==== 136756+0+0 (secure 0 0 0) 0x7faed800fae0 con 0x7faef0100510 2024-03-04T09:28:15.151 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:15.153+0000 7faeed7fa640 1 --2- 172.21.15.52:0/1049179793 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7faedc0a1e00 0x7faedc0a42c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:28:15.151 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:15.153+0000 7faeef7fe640 1 --2- 172.21.15.52:0/1049179793 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7faedc0a1e00 0x7faedc0a42c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:28:15.151 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:15.153+0000 7faeed7fa640 1 -- 172.21.15.52:0/1049179793 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7faed80cdb90 con 0x7faef0100510 2024-03-04T09:28:15.152 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:15.154+0000 7faeef7fe640 1 --2- 172.21.15.52:0/1049179793 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7faedc0a1e00 0x7faedc0a42c0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7faef0197c10 tx=0x7faee0005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:28:15.154 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:15.158+0000 7faeed7fa640 1 -- 172.21.15.52:0/1049179793 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7faed809b4f0 con 0x7faef0100510 2024-03-04T09:28:15.334 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:28:14 smithi052 ceph-mon[32473]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-04T09:28:15.335 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:28:14 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:28:15.335 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:28:14 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:28:15.335 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:28:14 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:28:15.335 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:28:14 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-04T09:28:15.335 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:28:14 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-04T09:28:15.335 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:28:14 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T09:28:15.335 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:28:14 smithi052 ceph-mon[32473]: Deploying daemon mds.f on smithi191 2024-03-04T09:28:15.335 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:28:14 smithi052 ceph-mon[32473]: mds.? [v2:172.21.15.106:6834/1541069897,v1:172.21.15.106:6835/1541069897] up:boot 2024-03-04T09:28:15.335 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:28:14 smithi052 ceph-mon[32473]: fsmap 5 up:standby 2024-03-04T09:28:15.335 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:28:14 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-03-04T09:28:15.375 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:15.379+0000 7faef6439640 1 -- 172.21.15.52:0/1049179793 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7faef002cff0 con 0x7faef0100510 2024-03-04T09:28:15.376 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:15.379+0000 7faeed7fa640 1 -- 172.21.15.52:0/1049179793 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) v1 ==== 93+0+4495 (secure 0 0 0) 0x7faed809adf0 con 0x7faef0100510 2024-03-04T09:28:15.376 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-03-04T09:28:15.376 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":6,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6835/1007961019","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6834","nonce":1007961019},{"type":"v1","addr":"172.21.15.52:6835","nonce":1007961019}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6837/3753881283","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6836","nonce":3753881283},{"type":"v1","addr":"172.21.15.52:6837","nonce":3753881283}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24407,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6833/1818480323","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6832","nonce":1818480323},{"type":"v1","addr":"172.21.15.106:6833","nonce":1818480323}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24412,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6833/765768204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":765768204},{"type":"v1","addr":"172.21.15.191:6833","nonce":765768204}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24440,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6835/1541069897","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6834","nonce":1541069897},{"type":"v1","addr":"172.21.15.106:6835","nonce":1541069897}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6}],"filesystems":[]} 2024-03-04T09:28:15.377 INFO:teuthology.orchestra.run.smithi052.stderr:dumped fsmap epoch 6 2024-03-04T09:28:15.378 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:15.381+0000 7faef6439640 1 -- 172.21.15.52:0/1049179793 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7faedc0a1e00 msgr2=0x7faedc0a42c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:28:15.378 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:15.381+0000 7faef6439640 1 --2- 172.21.15.52:0/1049179793 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7faedc0a1e00 0x7faedc0a42c0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7faef0197c10 tx=0x7faee0005f90 comp rx=0 tx=0).stop 2024-03-04T09:28:15.378 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:15.381+0000 7faef6439640 1 -- 172.21.15.52:0/1049179793 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7faef0100510 msgr2=0x7faef0196700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:28:15.378 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:15.381+0000 7faef6439640 1 --2- 172.21.15.52:0/1049179793 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7faef0100510 0x7faef0196700 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7faed800c970 tx=0x7faed800ce30 comp rx=0 tx=0).stop 2024-03-04T09:28:15.378 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:15.381+0000 7faef6439640 1 -- 172.21.15.52:0/1049179793 shutdown_connections 2024-03-04T09:28:15.379 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:15.381+0000 7faef6439640 1 --2- 172.21.15.52:0/1049179793 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7faedc0a1e00 0x7faedc0a42c0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:15.379 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:15.381+0000 7faef6439640 1 --2- 172.21.15.52:0/1049179793 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7faef01020c0 0x7faef019dcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:15.379 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:15.381+0000 7faef6439640 1 --2- 172.21.15.52:0/1049179793 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7faef0101700 0x7faef0196c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:15.379 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:15.381+0000 7faef6439640 1 --2- 172.21.15.52:0/1049179793 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7faef0100510 0x7faef0196700 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:15.379 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:15.381+0000 7faef6439640 1 -- 172.21.15.52:0/1049179793 >> 172.21.15.52:0/1049179793 conn(0x7faef00fba80 msgr2=0x7faef0104930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:28:15.379 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:15.381+0000 7faef6439640 1 -- 172.21.15.52:0/1049179793 shutdown_connections 2024-03-04T09:28:15.379 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:15.381+0000 7faef6439640 1 -- 172.21.15.52:0/1049179793 wait complete. 2024-03-04T09:28:16.131 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:28:15 smithi106 ceph-mon[36484]: from='client.? 172.21.15.52:0/1049179793' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-04T09:28:16.271 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:28:15 smithi191 ceph-mon[36816]: from='client.? 172.21.15.52:0/1049179793' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-04T09:28:16.272 INFO:journalctl@ceph.mds.f.smithi191.stdout:Mar 04 09:28:16 smithi191 systemd[1]: Starting Ceph mds.f for 4e059f08-da08-11ee-95c6-87774f69a715... 2024-03-04T09:28:16.334 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:28:15 smithi052 ceph-mon[32473]: from='client.? 172.21.15.52:0/1049179793' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-04T09:28:16.528 INFO:journalctl@ceph.mds.f.smithi191.stdout:Mar 04 09:28:16 smithi191 podman[66756]: 2024-03-04 09:28:16.177573302 +0000 UTC m=+0.017639604 image pull f88ffb32aa6a7d5dfdc954d9f096f0c9cc47981d05ec8d117ba859a22d8185ba quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5 2024-03-04T09:28:16.528 INFO:journalctl@ceph.mds.f.smithi191.stdout:Mar 04 09:28:16 smithi191 podman[66756]: 2024-03-04 09:28:16.294053671 +0000 UTC m=+0.134119971 container create 701812dde8795ec976caad0e1f41395fb4d9513389a4202224d7e1b403636a6b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5, name=ceph-4e059f08-da08-11ee-95c6-87774f69a715-mds-f, io.buildah.version=1.33.5, RELEASE=wip-lusov-qdb-1bc250c, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, GIT_BRANCH=HEAD, org.label-schema.build-date=20240222, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-03-04T09:28:16.836 INFO:journalctl@ceph.mds.f.smithi191.stdout:Mar 04 09:28:16 smithi191 podman[66756]: 2024-03-04 09:28:16.53182197 +0000 UTC m=+0.371888279 container init 701812dde8795ec976caad0e1f41395fb4d9513389a4202224d7e1b403636a6b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5, name=ceph-4e059f08-da08-11ee-95c6-87774f69a715-mds-f, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240222, CEPH_POINT_RELEASE=, ceph=True, RELEASE=wip-lusov-qdb-1bc250c, org.label-schema.license=GPLv2) 2024-03-04T09:28:16.836 INFO:journalctl@ceph.mds.f.smithi191.stdout:Mar 04 09:28:16 smithi191 podman[66756]: 2024-03-04 09:28:16.534631982 +0000 UTC m=+0.374698298 container start 701812dde8795ec976caad0e1f41395fb4d9513389a4202224d7e1b403636a6b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5, name=ceph-4e059f08-da08-11ee-95c6-87774f69a715-mds-f, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, RELEASE=wip-lusov-qdb-1bc250c, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240222, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True) 2024-03-04T09:28:16.837 INFO:journalctl@ceph.mds.f.smithi191.stdout:Mar 04 09:28:16 smithi191 ceph-mds[66775]: -- 172.21.15.191:0/2387139162 <== mon.1 v2:172.21.15.191:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x561b41f6ba40 con 0x561b41f3f000 2024-03-04T09:28:16.837 INFO:journalctl@ceph.mds.f.smithi191.stdout:Mar 04 09:28:16 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-mds-f[66771]: starting mds.f at 2024-03-04T09:28:16.837 INFO:journalctl@ceph.mds.f.smithi191.stdout:Mar 04 09:28:16 smithi191 bash[66756]: 701812dde8795ec976caad0e1f41395fb4d9513389a4202224d7e1b403636a6b 2024-03-04T09:28:16.837 INFO:journalctl@ceph.mds.f.smithi191.stdout:Mar 04 09:28:16 smithi191 systemd[1]: Started Ceph mds.f for 4e059f08-da08-11ee-95c6-87774f69a715. 2024-03-04T09:28:17.131 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:28:16 smithi106 ceph-mon[36484]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-04T09:28:17.131 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:28:16 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:28:17.131 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:28:16 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:28:17.132 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:28:16 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:28:17.132 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:28:16 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-04T09:28:17.132 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:28:16 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-04T09:28:17.132 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:28:16 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T09:28:17.318 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:28:16 smithi191 ceph-mon[36816]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-04T09:28:17.318 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:28:16 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:28:17.318 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:28:16 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:28:17.318 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:28:16 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:28:17.319 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:28:16 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-04T09:28:17.319 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:28:16 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-04T09:28:17.319 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:28:16 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T09:28:17.334 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:28:16 smithi052 ceph-mon[32473]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-04T09:28:17.335 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:28:16 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:28:17.335 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:28:16 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:28:17.335 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:28:16 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:28:17.335 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:28:16 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-04T09:28:17.335 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:28:16 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-04T09:28:17.335 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:28:16 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T09:28:17.390 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T09:28:17.513 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:17.509+0000 7fc172e39640 1 -- 172.21.15.52:0/3846896157 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc16c1031a0 msgr2=0x7fc16c109a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:28:17.513 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:17.509+0000 7fc172e39640 1 --2- 172.21.15.52:0/3846896157 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc16c1031a0 0x7fc16c109a30 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fc164009ca0 tx=0x7fc16402c0f0 comp rx=0 tx=0).stop 2024-03-04T09:28:17.513 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:17.509+0000 7fc172e39640 1 -- 172.21.15.52:0/3846896157 shutdown_connections 2024-03-04T09:28:17.514 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:17.509+0000 7fc172e39640 1 --2- 172.21.15.52:0/3846896157 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc16c1031a0 0x7fc16c109a30 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:17.514 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:17.509+0000 7fc172e39640 1 --2- 172.21.15.52:0/3846896157 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc16c1027e0 0x7fc16c102c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:17.514 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:17.509+0000 7fc172e39640 1 --2- 172.21.15.52:0/3846896157 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc16c1015f0 0x7fc16c101a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:17.514 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:17.509+0000 7fc172e39640 1 -- 172.21.15.52:0/3846896157 >> 172.21.15.52:0/3846896157 conn(0x7fc16c0fcb80 msgr2=0x7fc16c0fefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:28:17.514 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:17.509+0000 7fc172e39640 1 -- 172.21.15.52:0/3846896157 shutdown_connections 2024-03-04T09:28:17.515 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:17.509+0000 7fc172e39640 1 -- 172.21.15.52:0/3846896157 wait complete. 2024-03-04T09:28:17.515 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:17.510+0000 7fc172e39640 1 Processor -- start 2024-03-04T09:28:17.515 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:17.510+0000 7fc172e39640 1 -- start start 2024-03-04T09:28:17.515 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:17.510+0000 7fc172e39640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc16c1015f0 0x7fc16c075380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:28:17.515 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:17.510+0000 7fc172e39640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc16c1027e0 0x7fc16c0739c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:28:17.515 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:17.510+0000 7fc172e39640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc16c1031a0 0x7fc16c073f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:28:17.516 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:17.510+0000 7fc172e39640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc16c10d2e0 con 0x7fc16c1015f0 2024-03-04T09:28:17.516 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:17.510+0000 7fc172e39640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc16c10d160 con 0x7fc16c1031a0 2024-03-04T09:28:17.516 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:17.510+0000 7fc172e39640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc16c10d460 con 0x7fc16c1027e0 2024-03-04T09:28:17.517 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:17.510+0000 7fc170bd5640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc16c1015f0 0x7fc16c075380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:28:17.517 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:17.510+0000 7fc170bd5640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc16c1015f0 0x7fc16c075380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:41456/0 (socket says 172.21.15.52:41456) 2024-03-04T09:28:17.517 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:17.510+0000 7fc170bd5640 1 -- 172.21.15.52:0/4019193613 learned_addr learned my addr 172.21.15.52:0/4019193613 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-04T09:28:17.517 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:17.510+0000 7fc16bfff640 1 --2- 172.21.15.52:0/4019193613 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc16c1027e0 0x7fc16c0739c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:28:17.517 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:17.511+0000 7fc170bd5640 1 -- 172.21.15.52:0/4019193613 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc16c1027e0 msgr2=0x7fc16c0739c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:28:17.517 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:17.511+0000 7fc170bd5640 1 --2- 172.21.15.52:0/4019193613 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc16c1027e0 0x7fc16c0739c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:17.518 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:17.511+0000 7fc1713d6640 1 --2- 172.21.15.52:0/4019193613 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc16c1031a0 0x7fc16c073f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:28:17.518 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:17.511+0000 7fc170bd5640 1 -- 172.21.15.52:0/4019193613 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc16c1031a0 msgr2=0x7fc16c073f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:28:17.518 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:17.511+0000 7fc170bd5640 1 --2- 172.21.15.52:0/4019193613 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc16c1031a0 0x7fc16c073f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:17.518 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:17.511+0000 7fc170bd5640 1 -- 172.21.15.52:0/4019193613 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc16c074790 con 0x7fc16c1015f0 2024-03-04T09:28:17.518 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:17.511+0000 7fc1713d6640 1 --2- 172.21.15.52:0/4019193613 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc16c1031a0 0x7fc16c073f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-04T09:28:17.518 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:17.511+0000 7fc170bd5640 1 --2- 172.21.15.52:0/4019193613 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc16c1015f0 0x7fc16c075380 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fc16000b450 tx=0x7fc16000b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:28:17.519 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:17.511+0000 7fc169ffb640 1 -- 172.21.15.52:0/4019193613 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc160015020 con 0x7fc16c1015f0 2024-03-04T09:28:17.519 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:17.511+0000 7fc169ffb640 1 -- 172.21.15.52:0/4019193613 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc160011d70 con 0x7fc16c1015f0 2024-03-04T09:28:17.519 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:17.511+0000 7fc172e39640 1 -- 172.21.15.52:0/4019193613 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc16c1a1a10 con 0x7fc16c1015f0 2024-03-04T09:28:17.519 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:17.511+0000 7fc169ffb640 1 -- 172.21.15.52:0/4019193613 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc16000f340 con 0x7fc16c1015f0 2024-03-04T09:28:17.519 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:17.511+0000 7fc172e39640 1 -- 172.21.15.52:0/4019193613 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc16c1a1f60 con 0x7fc16c1015f0 2024-03-04T09:28:17.519 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:17.512+0000 7fc172e39640 1 -- 172.21.15.52:0/4019193613 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc16c068850 con 0x7fc16c1015f0 2024-03-04T09:28:17.520 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:17.513+0000 7fc169ffb640 1 -- 172.21.15.52:0/4019193613 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 16) v1 ==== 136756+0+0 (secure 0 0 0) 0x7fc160018020 con 0x7fc16c1015f0 2024-03-04T09:28:17.520 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:17.514+0000 7fc169ffb640 1 --2- 172.21.15.52:0/4019193613 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7fc15c0a1d80 0x7fc15c0a4240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:28:17.520 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:17.514+0000 7fc16bfff640 1 --2- 172.21.15.52:0/4019193613 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7fc15c0a1d80 0x7fc15c0a4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:28:17.520 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:17.514+0000 7fc169ffb640 1 -- 172.21.15.52:0/4019193613 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fc1600cd460 con 0x7fc16c1015f0 2024-03-04T09:28:17.520 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:17.514+0000 7fc16bfff640 1 --2- 172.21.15.52:0/4019193613 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7fc15c0a1d80 0x7fc15c0a4240 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fc16c0750f0 tx=0x7fc15800b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:28:17.521 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:17.519+0000 7fc169ffb640 1 -- 172.21.15.52:0/4019193613 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fc16001f020 con 0x7fc16c1015f0 2024-03-04T09:28:17.737 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:17.739+0000 7fc172e39640 1 -- 172.21.15.52:0/4019193613 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc16c02d020 con 0x7fc16c1015f0 2024-03-04T09:28:17.737 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:17.740+0000 7fc169ffb640 1 -- 172.21.15.52:0/4019193613 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+5286 (secure 0 0 0) 0x7fc16000d040 con 0x7fc16c1015f0 2024-03-04T09:28:17.738 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-03-04T09:28:17.738 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":7,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6835/1007961019","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6834","nonce":1007961019},{"type":"v1","addr":"172.21.15.52:6835","nonce":1007961019}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6837/3753881283","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6836","nonce":3753881283},{"type":"v1","addr":"172.21.15.52:6837","nonce":3753881283}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24407,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6833/1818480323","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6832","nonce":1818480323},{"type":"v1","addr":"172.21.15.106:6833","nonce":1818480323}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24412,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6833/765768204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":765768204},{"type":"v1","addr":"172.21.15.191:6833","nonce":765768204}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24440,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6835/1541069897","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6834","nonce":1541069897},{"type":"v1","addr":"172.21.15.106:6835","nonce":1541069897}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24451,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6835/105048699","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":105048699},{"type":"v1","addr":"172.21.15.191:6835","nonce":105048699}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7}],"filesystems":[]} 2024-03-04T09:28:17.740 INFO:teuthology.orchestra.run.smithi052.stderr:dumped fsmap epoch 7 2024-03-04T09:28:17.740 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:17.742+0000 7fc172e39640 1 -- 172.21.15.52:0/4019193613 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7fc15c0a1d80 msgr2=0x7fc15c0a4240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:28:17.740 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:17.742+0000 7fc172e39640 1 --2- 172.21.15.52:0/4019193613 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7fc15c0a1d80 0x7fc15c0a4240 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fc16c0750f0 tx=0x7fc15800b390 comp rx=0 tx=0).stop 2024-03-04T09:28:17.740 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:17.742+0000 7fc172e39640 1 -- 172.21.15.52:0/4019193613 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc16c1015f0 msgr2=0x7fc16c075380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:28:17.741 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:17.742+0000 7fc172e39640 1 --2- 172.21.15.52:0/4019193613 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc16c1015f0 0x7fc16c075380 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fc16000b450 tx=0x7fc16000b910 comp rx=0 tx=0).stop 2024-03-04T09:28:17.741 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:17.742+0000 7fc172e39640 1 -- 172.21.15.52:0/4019193613 shutdown_connections 2024-03-04T09:28:17.741 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:17.742+0000 7fc172e39640 1 --2- 172.21.15.52:0/4019193613 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7fc15c0a1d80 0x7fc15c0a4240 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:17.741 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:17.742+0000 7fc172e39640 1 --2- 172.21.15.52:0/4019193613 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc16c1031a0 0x7fc16c073f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:17.741 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:17.742+0000 7fc172e39640 1 --2- 172.21.15.52:0/4019193613 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc16c1027e0 0x7fc16c0739c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:17.741 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:17.742+0000 7fc172e39640 1 --2- 172.21.15.52:0/4019193613 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc16c1015f0 0x7fc16c075380 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:17.742 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:17.742+0000 7fc172e39640 1 -- 172.21.15.52:0/4019193613 >> 172.21.15.52:0/4019193613 conn(0x7fc16c0fcb80 msgr2=0x7fc16c105ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:28:17.742 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:17.742+0000 7fc172e39640 1 -- 172.21.15.52:0/4019193613 shutdown_connections 2024-03-04T09:28:17.742 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:17.742+0000 7fc172e39640 1 -- 172.21.15.52:0/4019193613 wait complete. 2024-03-04T09:28:18.030 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:28:17 smithi191 ceph-mon[36816]: Deploying daemon mds.c on smithi191 2024-03-04T09:28:18.030 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:28:17 smithi191 ceph-mon[36816]: mds.? [v2:172.21.15.191:6834/105048699,v1:172.21.15.191:6835/105048699] up:boot 2024-03-04T09:28:18.030 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:28:17 smithi191 ceph-mon[36816]: fsmap 6 up:standby 2024-03-04T09:28:18.030 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:28:17 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-03-04T09:28:18.031 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:28:17 smithi191 ceph-mon[36816]: from='client.? 172.21.15.52:0/4019193613' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-04T09:28:18.085 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:28:17 smithi052 ceph-mon[32473]: Deploying daemon mds.c on smithi191 2024-03-04T09:28:18.086 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:28:17 smithi052 ceph-mon[32473]: mds.? [v2:172.21.15.191:6834/105048699,v1:172.21.15.191:6835/105048699] up:boot 2024-03-04T09:28:18.086 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:28:17 smithi052 ceph-mon[32473]: fsmap 6 up:standby 2024-03-04T09:28:18.086 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:28:17 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-03-04T09:28:18.086 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:28:17 smithi052 ceph-mon[32473]: from='client.? 172.21.15.52:0/4019193613' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-04T09:28:18.131 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:28:17 smithi106 ceph-mon[36484]: Deploying daemon mds.c on smithi191 2024-03-04T09:28:18.131 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:28:17 smithi106 ceph-mon[36484]: mds.? [v2:172.21.15.191:6834/105048699,v1:172.21.15.191:6835/105048699] up:boot 2024-03-04T09:28:18.131 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:28:17 smithi106 ceph-mon[36484]: fsmap 6 up:standby 2024-03-04T09:28:18.131 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:28:17 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-03-04T09:28:18.131 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:28:17 smithi106 ceph-mon[36484]: from='client.? 172.21.15.52:0/4019193613' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-04T09:28:19.068 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:28:18 smithi191 ceph-mon[36816]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-04T09:28:19.137 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:28:18 smithi106 ceph-mon[36484]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-04T09:28:19.335 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:28:18 smithi052 ceph-mon[32473]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-04T09:28:19.751 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T09:28:19.837 INFO:journalctl@ceph.mds.c.smithi191.stdout:Mar 04 09:28:19 smithi191 systemd[1]: Started Ceph mds.c for 4e059f08-da08-11ee-95c6-87774f69a715. 2024-03-04T09:28:19.883 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:19.881+0000 7f0df720f640 1 -- 172.21.15.52:0/3924457858 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0df0102070 msgr2=0x7f0df0104460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:28:19.883 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:19.881+0000 7f0df720f640 1 --2- 172.21.15.52:0/3924457858 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0df0102070 0x7f0df0104460 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f0de4009900 tx=0x7f0de402bc30 comp rx=0 tx=0).stop 2024-03-04T09:28:19.883 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:19.881+0000 7f0df720f640 1 -- 172.21.15.52:0/3924457858 shutdown_connections 2024-03-04T09:28:19.883 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:19.881+0000 7f0df720f640 1 --2- 172.21.15.52:0/3924457858 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0df01072d0 0x7f0df0109720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:19.883 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:19.881+0000 7f0df720f640 1 --2- 172.21.15.52:0/3924457858 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0df01049a0 0x7f0df0106d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:19.884 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:19.881+0000 7f0df720f640 1 --2- 172.21.15.52:0/3924457858 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0df0102070 0x7f0df0104460 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:19.884 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:19.881+0000 7f0df720f640 1 -- 172.21.15.52:0/3924457858 >> 172.21.15.52:0/3924457858 conn(0x7f0df00fba60 msgr2=0x7f0df00fdea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:28:19.884 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:19.881+0000 7f0df720f640 1 -- 172.21.15.52:0/3924457858 shutdown_connections 2024-03-04T09:28:19.884 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:19.881+0000 7f0df720f640 1 -- 172.21.15.52:0/3924457858 wait complete. 2024-03-04T09:28:19.884 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:19.882+0000 7f0df720f640 1 Processor -- start 2024-03-04T09:28:19.884 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:19.882+0000 7f0df720f640 1 -- start start 2024-03-04T09:28:19.884 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:19.882+0000 7f0df720f640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0df0102070 0x7f0df0197960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:28:19.884 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:19.882+0000 7f0df720f640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0df01049a0 0x7f0df0197ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:28:19.885 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:19.882+0000 7f0df720f640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0df01072d0 0x7f0df019ef10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:28:19.885 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:19.882+0000 7f0df720f640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0df010d280 con 0x7f0df01072d0 2024-03-04T09:28:19.885 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:19.882+0000 7f0df720f640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0df010d100 con 0x7f0df01049a0 2024-03-04T09:28:19.885 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:19.882+0000 7f0df720f640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0df010d400 con 0x7f0df0102070 2024-03-04T09:28:19.885 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:19.882+0000 7f0df57ac640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0df01072d0 0x7f0df019ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:28:19.885 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:19.882+0000 7f0df57ac640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0df01072d0 0x7f0df019ef10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:41480/0 (socket says 172.21.15.52:41480) 2024-03-04T09:28:19.886 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:19.882+0000 7f0df57ac640 1 -- 172.21.15.52:0/408188078 learned_addr learned my addr 172.21.15.52:0/408188078 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-04T09:28:19.886 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:19.882+0000 7f0deffff640 1 --2- 172.21.15.52:0/408188078 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0df01049a0 0x7f0df0197ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:28:19.886 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:19.882+0000 7f0df4fab640 1 --2- 172.21.15.52:0/408188078 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0df0102070 0x7f0df0197960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:28:19.886 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:19.882+0000 7f0df57ac640 1 -- 172.21.15.52:0/408188078 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0df0102070 msgr2=0x7f0df0197960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:28:19.886 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:19.882+0000 7f0df57ac640 1 --2- 172.21.15.52:0/408188078 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0df0102070 0x7f0df0197960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:19.886 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:19.882+0000 7f0df57ac640 1 -- 172.21.15.52:0/408188078 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0df01049a0 msgr2=0x7f0df0197ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:28:19.886 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:19.882+0000 7f0df57ac640 1 --2- 172.21.15.52:0/408188078 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0df01049a0 0x7f0df0197ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:19.886 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:19.882+0000 7f0df57ac640 1 -- 172.21.15.52:0/408188078 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0df019f580 con 0x7f0df01072d0 2024-03-04T09:28:19.887 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:19.883+0000 7f0df57ac640 1 --2- 172.21.15.52:0/408188078 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0df01072d0 0x7f0df019ef10 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f0de800eb10 tx=0x7f0de800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:28:19.887 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:19.883+0000 7f0dedffb640 1 -- 172.21.15.52:0/408188078 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0de8004b20 con 0x7f0df01072d0 2024-03-04T09:28:19.887 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:19.883+0000 7f0dedffb640 1 -- 172.21.15.52:0/408188078 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0de8004cc0 con 0x7f0df01072d0 2024-03-04T09:28:19.887 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:19.883+0000 7f0df720f640 1 -- 172.21.15.52:0/408188078 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0df019f870 con 0x7f0df01072d0 2024-03-04T09:28:19.887 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:19.883+0000 7f0dedffb640 1 -- 172.21.15.52:0/408188078 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0de8010760 con 0x7f0df01072d0 2024-03-04T09:28:19.887 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:19.883+0000 7f0df720f640 1 -- 172.21.15.52:0/408188078 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0df019fd70 con 0x7f0df01072d0 2024-03-04T09:28:19.887 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:19.884+0000 7f0df720f640 1 -- 172.21.15.52:0/408188078 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0df00fd550 con 0x7f0df01072d0 2024-03-04T09:28:19.887 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:19.885+0000 7f0dedffb640 1 -- 172.21.15.52:0/408188078 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 16) v1 ==== 136756+0+0 (secure 0 0 0) 0x7f0de8005020 con 0x7f0df01072d0 2024-03-04T09:28:19.888 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:19.886+0000 7f0dedffb640 1 --2- 172.21.15.52:0/408188078 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f0de00a1d80 0x7f0de00a4240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:28:19.888 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:19.886+0000 7f0df4fab640 1 --2- 172.21.15.52:0/408188078 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f0de00a1d80 0x7f0de00a4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:28:19.888 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:19.886+0000 7f0dedffb640 1 -- 172.21.15.52:0/408188078 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f0de80ce010 con 0x7f0df01072d0 2024-03-04T09:28:19.888 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:19.886+0000 7f0df4fab640 1 --2- 172.21.15.52:0/408188078 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f0de00a1d80 0x7f0de00a4240 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f0de4006010 tx=0x7f0de4000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:28:19.888 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:19.890+0000 7f0dedffb640 1 -- 172.21.15.52:0/408188078 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f0de809b840 con 0x7f0df01072d0 2024-03-04T09:28:20.108 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:20.111+0000 7f0df720f640 1 -- 172.21.15.52:0/408188078 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0df00621d0 con 0x7f0df01072d0 2024-03-04T09:28:20.109 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:20.112+0000 7f0dedffb640 1 -- 172.21.15.52:0/408188078 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+6080 (secure 0 0 0) 0x7f0de809b140 con 0x7f0df01072d0 2024-03-04T09:28:20.109 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-03-04T09:28:20.109 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":8,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6835/1007961019","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6834","nonce":1007961019},{"type":"v1","addr":"172.21.15.52:6835","nonce":1007961019}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6837/3753881283","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6836","nonce":3753881283},{"type":"v1","addr":"172.21.15.52:6837","nonce":3753881283}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24407,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6833/1818480323","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6832","nonce":1818480323},{"type":"v1","addr":"172.21.15.106:6833","nonce":1818480323}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24412,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6833/765768204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":765768204},{"type":"v1","addr":"172.21.15.191:6833","nonce":765768204}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24440,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6835/1541069897","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6834","nonce":1541069897},{"type":"v1","addr":"172.21.15.106:6835","nonce":1541069897}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24451,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6835/105048699","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":105048699},{"type":"v1","addr":"172.21.15.191:6835","nonce":105048699}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24463,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6837/4071064225","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6836","nonce":4071064225},{"type":"v1","addr":"172.21.15.191:6837","nonce":4071064225}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8}],"filesystems":[]} 2024-03-04T09:28:20.109 INFO:teuthology.orchestra.run.smithi052.stderr:dumped fsmap epoch 8 2024-03-04T09:28:20.111 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:20.114+0000 7f0df720f640 1 -- 172.21.15.52:0/408188078 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f0de00a1d80 msgr2=0x7f0de00a4240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:28:20.111 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:20.114+0000 7f0df720f640 1 --2- 172.21.15.52:0/408188078 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f0de00a1d80 0x7f0de00a4240 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f0de4006010 tx=0x7f0de4000bc0 comp rx=0 tx=0).stop 2024-03-04T09:28:20.111 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:20.114+0000 7f0df720f640 1 -- 172.21.15.52:0/408188078 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0df01072d0 msgr2=0x7f0df019ef10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:28:20.111 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:20.114+0000 7f0df720f640 1 --2- 172.21.15.52:0/408188078 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0df01072d0 0x7f0df019ef10 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f0de800eb10 tx=0x7f0de800c2d0 comp rx=0 tx=0).stop 2024-03-04T09:28:20.111 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:20.114+0000 7f0df720f640 1 -- 172.21.15.52:0/408188078 shutdown_connections 2024-03-04T09:28:20.112 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:20.114+0000 7f0df720f640 1 --2- 172.21.15.52:0/408188078 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f0de00a1d80 0x7f0de00a4240 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:20.112 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:20.114+0000 7f0df720f640 1 --2- 172.21.15.52:0/408188078 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0df01072d0 0x7f0df019ef10 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:20.112 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:20.114+0000 7f0df720f640 1 --2- 172.21.15.52:0/408188078 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0df01049a0 0x7f0df0197ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:20.112 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:20.114+0000 7f0df720f640 1 --2- 172.21.15.52:0/408188078 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0df0102070 0x7f0df0197960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:20.112 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:20.114+0000 7f0df720f640 1 -- 172.21.15.52:0/408188078 >> 172.21.15.52:0/408188078 conn(0x7f0df00fba60 msgr2=0x7f0df0102ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:28:20.112 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:20.114+0000 7f0df720f640 1 -- 172.21.15.52:0/408188078 shutdown_connections 2024-03-04T09:28:20.112 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:20.114+0000 7f0df720f640 1 -- 172.21.15.52:0/408188078 wait complete. 2024-03-04T09:28:20.835 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:28:20 smithi052 ceph-mon[32473]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-04T09:28:20.835 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:28:20 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:28:20.835 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:28:20 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:28:20.835 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:28:20 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:28:20.835 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:28:20 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-04T09:28:20.836 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:28:20 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-04T09:28:20.836 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:28:20 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T09:28:20.836 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:28:20 smithi052 ceph-mon[32473]: Deploying daemon mds.h on smithi106 2024-03-04T09:28:20.836 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:28:20 smithi052 ceph-mon[32473]: mds.? [v2:172.21.15.191:6836/4071064225,v1:172.21.15.191:6837/4071064225] up:boot 2024-03-04T09:28:20.836 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:28:20 smithi052 ceph-mon[32473]: fsmap 7 up:standby 2024-03-04T09:28:20.836 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:28:20 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-04T09:28:20.836 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:28:20 smithi052 ceph-mon[32473]: from='client.? 172.21.15.52:0/408188078' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-04T09:28:20.836 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:28:20 smithi191 ceph-mon[36816]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-04T09:28:20.837 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:28:20 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:28:20.837 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:28:20 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:28:20.837 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:28:20 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:28:20.837 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:28:20 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-04T09:28:20.837 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:28:20 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-04T09:28:20.837 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:28:20 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T09:28:20.837 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:28:20 smithi191 ceph-mon[36816]: Deploying daemon mds.h on smithi106 2024-03-04T09:28:20.837 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:28:20 smithi191 ceph-mon[36816]: mds.? [v2:172.21.15.191:6836/4071064225,v1:172.21.15.191:6837/4071064225] up:boot 2024-03-04T09:28:20.837 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:28:20 smithi191 ceph-mon[36816]: fsmap 7 up:standby 2024-03-04T09:28:20.837 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:28:20 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-04T09:28:20.837 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:28:20 smithi191 ceph-mon[36816]: from='client.? 172.21.15.52:0/408188078' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-04T09:28:20.881 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:28:20 smithi106 ceph-mon[36484]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-04T09:28:20.881 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:28:20 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:28:20.881 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:28:20 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:28:20.881 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:28:20 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:28:20.881 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:28:20 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-04T09:28:20.881 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:28:20 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-04T09:28:20.881 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:28:20 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T09:28:20.882 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:28:20 smithi106 ceph-mon[36484]: Deploying daemon mds.h on smithi106 2024-03-04T09:28:20.882 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:28:20 smithi106 ceph-mon[36484]: mds.? [v2:172.21.15.191:6836/4071064225,v1:172.21.15.191:6837/4071064225] up:boot 2024-03-04T09:28:20.882 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:28:20 smithi106 ceph-mon[36484]: fsmap 7 up:standby 2024-03-04T09:28:20.882 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:28:20 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-04T09:28:20.882 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:28:20 smithi106 ceph-mon[36484]: from='client.? 172.21.15.52:0/408188078' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-04T09:28:22.123 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T09:28:22.131 INFO:journalctl@ceph.mds.h.smithi106.stdout:Mar 04 09:28:22 smithi106 podman[66319]: 2024-03-04 09:28:22.058455298 +0000 UTC m=+0.119208645 container create 7a759fae8efb0fb8b3808f050eb58c5ec7ba9f3afe57f08007faaad34b1ad3d9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5, name=ceph-4e059f08-da08-11ee-95c6-87774f69a715-mds-h, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240222, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, ceph=True, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-qdb-1bc250c) 2024-03-04T09:28:22.241 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:22.241+0000 7fb271a2a640 1 -- 172.21.15.52:0/1814310679 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb26c1031a0 msgr2=0x7fb26c109a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:28:22.241 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:22.241+0000 7fb271a2a640 1 --2- 172.21.15.52:0/1814310679 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb26c1031a0 0x7fb26c109a30 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fb264009ca0 tx=0x7fb26402c0f0 comp rx=0 tx=0).stop 2024-03-04T09:28:22.241 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:22.241+0000 7fb271a2a640 1 -- 172.21.15.52:0/1814310679 shutdown_connections 2024-03-04T09:28:22.241 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:22.241+0000 7fb271a2a640 1 --2- 172.21.15.52:0/1814310679 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb26c1031a0 0x7fb26c109a30 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:22.241 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:22.241+0000 7fb271a2a640 1 --2- 172.21.15.52:0/1814310679 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb26c1027e0 0x7fb26c102c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:22.241 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:22.242+0000 7fb271a2a640 1 --2- 172.21.15.52:0/1814310679 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb26c1015f0 0x7fb26c101a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:22.242 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:22.242+0000 7fb271a2a640 1 -- 172.21.15.52:0/1814310679 >> 172.21.15.52:0/1814310679 conn(0x7fb26c0fcb80 msgr2=0x7fb26c0fefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:28:22.242 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:22.242+0000 7fb271a2a640 1 -- 172.21.15.52:0/1814310679 shutdown_connections 2024-03-04T09:28:22.242 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:22.242+0000 7fb271a2a640 1 -- 172.21.15.52:0/1814310679 wait complete. 2024-03-04T09:28:22.242 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:22.242+0000 7fb271a2a640 1 Processor -- start 2024-03-04T09:28:22.242 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:22.242+0000 7fb271a2a640 1 -- start start 2024-03-04T09:28:22.242 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:22.242+0000 7fb271a2a640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb26c1015f0 0x7fb26c197860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:28:22.242 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:22.242+0000 7fb271a2a640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb26c1027e0 0x7fb26c197da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:28:22.242 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:22.242+0000 7fb271a2a640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb26c1031a0 0x7fb26c19ee10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:28:22.242 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:22.242+0000 7fb271a2a640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb26c10d250 con 0x7fb26c1027e0 2024-03-04T09:28:22.242 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:22.242+0000 7fb271a2a640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb26c10d0d0 con 0x7fb26c1031a0 2024-03-04T09:28:22.242 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:22.242+0000 7fb271a2a640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb26c10d3d0 con 0x7fb26c1015f0 2024-03-04T09:28:22.242 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:22.242+0000 7fb263fff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb26c1027e0 0x7fb26c197da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:28:22.242 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:22.242+0000 7fb263fff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb26c1027e0 0x7fb26c197da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:41504/0 (socket says 172.21.15.52:41504) 2024-03-04T09:28:22.243 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:22.242+0000 7fb263fff640 1 -- 172.21.15.52:0/2561287578 learned_addr learned my addr 172.21.15.52:0/2561287578 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-04T09:28:22.243 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:22.243+0000 7fb26affd640 1 --2- 172.21.15.52:0/2561287578 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb26c1015f0 0x7fb26c197860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:28:22.243 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:22.243+0000 7fb26b7fe640 1 --2- 172.21.15.52:0/2561287578 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb26c1031a0 0x7fb26c19ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:28:22.243 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:22.243+0000 7fb263fff640 1 -- 172.21.15.52:0/2561287578 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb26c1015f0 msgr2=0x7fb26c197860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:28:22.243 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:22.243+0000 7fb263fff640 1 --2- 172.21.15.52:0/2561287578 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb26c1015f0 0x7fb26c197860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:22.243 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:22.243+0000 7fb263fff640 1 -- 172.21.15.52:0/2561287578 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb26c1031a0 msgr2=0x7fb26c19ee10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:28:22.243 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:22.243+0000 7fb263fff640 1 --2- 172.21.15.52:0/2561287578 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb26c1031a0 0x7fb26c19ee10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:22.243 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:22.243+0000 7fb263fff640 1 -- 172.21.15.52:0/2561287578 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb26c19f480 con 0x7fb26c1027e0 2024-03-04T09:28:22.243 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:22.243+0000 7fb26affd640 1 --2- 172.21.15.52:0/2561287578 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb26c1015f0 0x7fb26c197860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T09:28:22.243 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:22.243+0000 7fb263fff640 1 --2- 172.21.15.52:0/2561287578 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb26c1027e0 0x7fb26c197da0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fb25400d950 tx=0x7fb25400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:28:22.243 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:22.243+0000 7fb270a28640 1 -- 172.21.15.52:0/2561287578 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb254017070 con 0x7fb26c1027e0 2024-03-04T09:28:22.243 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:22.243+0000 7fb271a2a640 1 -- 172.21.15.52:0/2561287578 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb26c19f770 con 0x7fb26c1027e0 2024-03-04T09:28:22.243 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:22.243+0000 7fb270a28640 1 -- 172.21.15.52:0/2561287578 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb254009c30 con 0x7fb26c1027e0 2024-03-04T09:28:22.243 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:22.243+0000 7fb270a28640 1 -- 172.21.15.52:0/2561287578 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb25400f6b0 con 0x7fb26c1027e0 2024-03-04T09:28:22.244 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:22.243+0000 7fb271a2a640 1 -- 172.21.15.52:0/2561287578 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb26c19fcc0 con 0x7fb26c1027e0 2024-03-04T09:28:22.244 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:22.244+0000 7fb271a2a640 1 -- 172.21.15.52:0/2561287578 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb26c1088b0 con 0x7fb26c1027e0 2024-03-04T09:28:22.245 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:22.245+0000 7fb270a28640 1 -- 172.21.15.52:0/2561287578 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 16) v1 ==== 136756+0+0 (secure 0 0 0) 0x7fb2540109d0 con 0x7fb26c1027e0 2024-03-04T09:28:22.245 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:22.246+0000 7fb270a28640 1 --2- 172.21.15.52:0/2561287578 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7fb2580a1d80 0x7fb2580a4240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:28:22.245 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:22.246+0000 7fb26affd640 1 --2- 172.21.15.52:0/2561287578 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7fb2580a1d80 0x7fb2580a4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:28:22.245 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:22.246+0000 7fb270a28640 1 -- 172.21.15.52:0/2561287578 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fb2540cd630 con 0x7fb26c1027e0 2024-03-04T09:28:22.245 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:22.247+0000 7fb26affd640 1 --2- 172.21.15.52:0/2561287578 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7fb2580a1d80 0x7fb2580a4240 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fb25c0058c0 tx=0x7fb25c005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:28:22.248 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:22.251+0000 7fb270a28640 1 -- 172.21.15.52:0/2561287578 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fb254009de0 con 0x7fb26c1027e0 2024-03-04T09:28:22.471 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:22.474+0000 7fb271a2a640 1 -- 172.21.15.52:0/2561287578 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb26c02cff0 con 0x7fb26c1027e0 2024-03-04T09:28:22.471 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:22.474+0000 7fb270a28640 1 -- 172.21.15.52:0/2561287578 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+6080 (secure 0 0 0) 0x7fb25409ae60 con 0x7fb26c1027e0 2024-03-04T09:28:22.472 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-03-04T09:28:22.472 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":8,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6835/1007961019","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6834","nonce":1007961019},{"type":"v1","addr":"172.21.15.52:6835","nonce":1007961019}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6837/3753881283","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6836","nonce":3753881283},{"type":"v1","addr":"172.21.15.52:6837","nonce":3753881283}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24407,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6833/1818480323","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6832","nonce":1818480323},{"type":"v1","addr":"172.21.15.106:6833","nonce":1818480323}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24412,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6833/765768204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":765768204},{"type":"v1","addr":"172.21.15.191:6833","nonce":765768204}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24440,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6835/1541069897","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6834","nonce":1541069897},{"type":"v1","addr":"172.21.15.106:6835","nonce":1541069897}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24451,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6835/105048699","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":105048699},{"type":"v1","addr":"172.21.15.191:6835","nonce":105048699}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24463,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6837/4071064225","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6836","nonce":4071064225},{"type":"v1","addr":"172.21.15.191:6837","nonce":4071064225}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8}],"filesystems":[]} 2024-03-04T09:28:22.472 INFO:teuthology.orchestra.run.smithi052.stderr:dumped fsmap epoch 8 2024-03-04T09:28:22.473 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:22.476+0000 7fb271a2a640 1 -- 172.21.15.52:0/2561287578 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7fb2580a1d80 msgr2=0x7fb2580a4240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:28:22.474 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:22.476+0000 7fb271a2a640 1 --2- 172.21.15.52:0/2561287578 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7fb2580a1d80 0x7fb2580a4240 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fb25c0058c0 tx=0x7fb25c005850 comp rx=0 tx=0).stop 2024-03-04T09:28:22.474 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:22.476+0000 7fb271a2a640 1 -- 172.21.15.52:0/2561287578 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb26c1027e0 msgr2=0x7fb26c197da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:28:22.474 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:22.476+0000 7fb271a2a640 1 --2- 172.21.15.52:0/2561287578 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb26c1027e0 0x7fb26c197da0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fb25400d950 tx=0x7fb25400de10 comp rx=0 tx=0).stop 2024-03-04T09:28:22.474 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:22.476+0000 7fb271a2a640 1 -- 172.21.15.52:0/2561287578 shutdown_connections 2024-03-04T09:28:22.474 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:22.476+0000 7fb271a2a640 1 --2- 172.21.15.52:0/2561287578 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7fb2580a1d80 0x7fb2580a4240 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:22.474 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:22.476+0000 7fb271a2a640 1 --2- 172.21.15.52:0/2561287578 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb26c1031a0 0x7fb26c19ee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:22.474 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:22.476+0000 7fb271a2a640 1 --2- 172.21.15.52:0/2561287578 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb26c1027e0 0x7fb26c197da0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:22.474 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:22.476+0000 7fb271a2a640 1 --2- 172.21.15.52:0/2561287578 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb26c1015f0 0x7fb26c197860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:22.474 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:22.476+0000 7fb271a2a640 1 -- 172.21.15.52:0/2561287578 >> 172.21.15.52:0/2561287578 conn(0x7fb26c0fcb80 msgr2=0x7fb26c105a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:28:22.474 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:22.476+0000 7fb271a2a640 1 -- 172.21.15.52:0/2561287578 shutdown_connections 2024-03-04T09:28:22.474 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:22.477+0000 7fb271a2a640 1 -- 172.21.15.52:0/2561287578 wait complete. 2024-03-04T09:28:22.482 INFO:journalctl@ceph.mds.h.smithi106.stdout:Mar 04 09:28:22 smithi106 podman[66319]: 2024-03-04 09:28:22.294764075 +0000 UTC m=+0.355517434 container init 7a759fae8efb0fb8b3808f050eb58c5ec7ba9f3afe57f08007faaad34b1ad3d9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5, name=ceph-4e059f08-da08-11ee-95c6-87774f69a715-mds-h, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240222, RELEASE=wip-lusov-qdb-1bc250c, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.license=GPLv2) 2024-03-04T09:28:22.482 INFO:journalctl@ceph.mds.h.smithi106.stdout:Mar 04 09:28:22 smithi106 podman[66319]: 2024-03-04 09:28:22.297481793 +0000 UTC m=+0.358235146 container start 7a759fae8efb0fb8b3808f050eb58c5ec7ba9f3afe57f08007faaad34b1ad3d9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5, name=ceph-4e059f08-da08-11ee-95c6-87774f69a715-mds-h, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240222, RELEASE=wip-lusov-qdb-1bc250c, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-04T09:28:22.482 INFO:journalctl@ceph.mds.h.smithi106.stdout:Mar 04 09:28:22 smithi106 ceph-mds[66338]: -- 172.21.15.106:0/2581510605 <== mon.2 v2:172.21.15.106:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x559b186bfa40 con 0x559b18692800 2024-03-04T09:28:22.482 INFO:journalctl@ceph.mds.h.smithi106.stdout:Mar 04 09:28:22 smithi106 ceph-4e059f08-da08-11ee-95c6-87774f69a715-mds-h[66334]: starting mds.h at 2024-03-04T09:28:22.482 INFO:journalctl@ceph.mds.h.smithi106.stdout:Mar 04 09:28:22 smithi106 bash[66319]: 7a759fae8efb0fb8b3808f050eb58c5ec7ba9f3afe57f08007faaad34b1ad3d9 2024-03-04T09:28:22.482 INFO:journalctl@ceph.mds.h.smithi106.stdout:Mar 04 09:28:22 smithi106 systemd[1]: Started Ceph mds.h for 4e059f08-da08-11ee-95c6-87774f69a715. 2024-03-04T09:28:22.835 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:28:22 smithi052 ceph-mon[32473]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-04T09:28:22.835 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:28:22 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:28:22.835 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:28:22 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:28:22.835 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:28:22 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:28:22.835 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:28:22 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-04T09:28:22.835 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:28:22 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-04T09:28:22.835 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:28:22 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T09:28:22.835 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:28:22 smithi052 ceph-mon[32473]: from='client.? 172.21.15.52:0/2561287578' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-04T09:28:22.836 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:28:22 smithi191 ceph-mon[36816]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-04T09:28:22.836 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:28:22 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:28:22.836 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:28:22 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:28:22.836 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:28:22 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:28:22.836 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:28:22 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-04T09:28:22.836 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:28:22 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-04T09:28:22.836 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:28:22 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T09:28:22.836 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:28:22 smithi191 ceph-mon[36816]: from='client.? 172.21.15.52:0/2561287578' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-04T09:28:22.881 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:28:22 smithi106 ceph-mon[36484]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-04T09:28:22.881 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:28:22 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:28:22.881 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:28:22 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:28:22.881 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:28:22 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:28:22.881 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:28:22 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-04T09:28:22.881 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:28:22 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-04T09:28:22.881 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:28:22 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T09:28:22.882 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:28:22 smithi106 ceph-mon[36484]: from='client.? 172.21.15.52:0/2561287578' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-04T09:28:23.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:28:23 smithi191 ceph-mon[36816]: Deploying daemon mds.i on smithi191 2024-03-04T09:28:23.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:28:23 smithi191 ceph-mon[36816]: mds.? [v2:172.21.15.106:6836/2901755066,v1:172.21.15.106:6837/2901755066] up:boot 2024-03-04T09:28:23.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:28:23 smithi191 ceph-mon[36816]: fsmap 8 up:standby 2024-03-04T09:28:23.587 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:28:23 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-03-04T09:28:23.835 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:28:23 smithi052 ceph-mon[32473]: Deploying daemon mds.i on smithi191 2024-03-04T09:28:23.835 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:28:23 smithi052 ceph-mon[32473]: mds.? [v2:172.21.15.106:6836/2901755066,v1:172.21.15.106:6837/2901755066] up:boot 2024-03-04T09:28:23.835 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:28:23 smithi052 ceph-mon[32473]: fsmap 8 up:standby 2024-03-04T09:28:23.835 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:28:23 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-03-04T09:28:23.881 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:28:23 smithi106 ceph-mon[36484]: Deploying daemon mds.i on smithi191 2024-03-04T09:28:23.881 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:28:23 smithi106 ceph-mon[36484]: mds.? [v2:172.21.15.106:6836/2901755066,v1:172.21.15.106:6837/2901755066] up:boot 2024-03-04T09:28:23.881 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:28:23 smithi106 ceph-mon[36484]: fsmap 8 up:standby 2024-03-04T09:28:23.881 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:28:23 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-03-04T09:28:24.485 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T09:28:24.604 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:24.604+0000 7f8669cc2640 1 -- 172.21.15.52:0/338886939 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8664104a40 msgr2=0x7f8664106e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:28:24.604 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:24.604+0000 7f8669cc2640 1 --2- 172.21.15.52:0/338886939 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8664104a40 0x7f8664106e30 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f8658009900 tx=0x7f865802bc30 comp rx=0 tx=0).stop 2024-03-04T09:28:24.604 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:24.604+0000 7f8669cc2640 1 -- 172.21.15.52:0/338886939 shutdown_connections 2024-03-04T09:28:24.604 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:24.604+0000 7f8669cc2640 1 --2- 172.21.15.52:0/338886939 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8664107370 0x7f8664109760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:24.604 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:24.604+0000 7f8669cc2640 1 --2- 172.21.15.52:0/338886939 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8664104a40 0x7f8664106e30 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:24.604 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:24.604+0000 7f8669cc2640 1 --2- 172.21.15.52:0/338886939 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8664102110 0x7f8664104500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:24.604 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:24.604+0000 7f8669cc2640 1 -- 172.21.15.52:0/338886939 >> 172.21.15.52:0/338886939 conn(0x7f86640fba60 msgr2=0x7f86640fdea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:28:24.604 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:24.604+0000 7f8669cc2640 1 -- 172.21.15.52:0/338886939 shutdown_connections 2024-03-04T09:28:24.605 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:24.604+0000 7f8669cc2640 1 -- 172.21.15.52:0/338886939 wait complete. 2024-03-04T09:28:24.605 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:24.604+0000 7f8669cc2640 1 Processor -- start 2024-03-04T09:28:24.605 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:24.605+0000 7f8669cc2640 1 -- start start 2024-03-04T09:28:24.605 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:24.605+0000 7f8669cc2640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8664102110 0x7f8664199b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:28:24.605 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:24.605+0000 7f8669cc2640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8664104a40 0x7f866419a0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:28:24.605 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:24.605+0000 7f8669cc2640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8664107370 0x7f86641a1120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:28:24.605 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:24.605+0000 7f8669cc2640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f866410d250 con 0x7f8664107370 2024-03-04T09:28:24.605 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:24.605+0000 7f8669cc2640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f866410d0d0 con 0x7f8664104a40 2024-03-04T09:28:24.605 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:24.605+0000 7f8669cc2640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f866410d3d0 con 0x7f8664102110 2024-03-04T09:28:24.605 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:24.605+0000 7f8663fff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8664107370 0x7f86641a1120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:28:24.606 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:24.605+0000 7f8663fff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8664107370 0x7f86641a1120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:46020/0 (socket says 172.21.15.52:46020) 2024-03-04T09:28:24.606 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:24.605+0000 7f8663fff640 1 -- 172.21.15.52:0/3924214951 learned_addr learned my addr 172.21.15.52:0/3924214951 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-04T09:28:24.606 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:24.605+0000 7f8662ffd640 1 --2- 172.21.15.52:0/3924214951 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8664104a40 0x7f866419a0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:28:24.606 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:24.605+0000 7f86637fe640 1 --2- 172.21.15.52:0/3924214951 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8664102110 0x7f8664199b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:28:24.606 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:24.605+0000 7f8663fff640 1 -- 172.21.15.52:0/3924214951 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8664102110 msgr2=0x7f8664199b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:28:24.606 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:24.605+0000 7f8663fff640 1 --2- 172.21.15.52:0/3924214951 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8664102110 0x7f8664199b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:24.606 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:24.605+0000 7f8663fff640 1 -- 172.21.15.52:0/3924214951 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8664104a40 msgr2=0x7f866419a0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:28:24.606 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:24.605+0000 7f8663fff640 1 --2- 172.21.15.52:0/3924214951 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8664104a40 0x7f866419a0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:24.606 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:24.605+0000 7f8663fff640 1 -- 172.21.15.52:0/3924214951 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86641a1790 con 0x7f8664107370 2024-03-04T09:28:24.606 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:24.605+0000 7f86637fe640 1 --2- 172.21.15.52:0/3924214951 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8664102110 0x7f8664199b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T09:28:24.606 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:24.605+0000 7f8662ffd640 1 --2- 172.21.15.52:0/3924214951 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8664104a40 0x7f866419a0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T09:28:24.606 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:24.605+0000 7f8663fff640 1 --2- 172.21.15.52:0/3924214951 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8664107370 0x7f86641a1120 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f865c00efd0 tx=0x7f865c00c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:28:24.606 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:24.606+0000 7f8660ff9640 1 -- 172.21.15.52:0/3924214951 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f865c003b70 con 0x7f8664107370 2024-03-04T09:28:24.607 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:24.606+0000 7f8660ff9640 1 -- 172.21.15.52:0/3924214951 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f865c003d10 con 0x7f8664107370 2024-03-04T09:28:24.607 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:24.606+0000 7f8669cc2640 1 -- 172.21.15.52:0/3924214951 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86641a1a80 con 0x7f8664107370 2024-03-04T09:28:24.607 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:24.606+0000 7f8660ff9640 1 -- 172.21.15.52:0/3924214951 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f865c0109f0 con 0x7f8664107370 2024-03-04T09:28:24.607 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:24.606+0000 7f8669cc2640 1 -- 172.21.15.52:0/3924214951 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86641a1f50 con 0x7f8664107370 2024-03-04T09:28:24.607 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:24.607+0000 7f8669cc2640 1 -- 172.21.15.52:0/3924214951 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8664108e60 con 0x7f8664107370 2024-03-04T09:28:24.608 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:24.608+0000 7f8660ff9640 1 -- 172.21.15.52:0/3924214951 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 16) v1 ==== 136756+0+0 (secure 0 0 0) 0x7f865c004020 con 0x7f8664107370 2024-03-04T09:28:24.608 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:24.608+0000 7f8660ff9640 1 --2- 172.21.15.52:0/3924214951 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f864c0a1d80 0x7f864c0a4240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:28:24.608 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:24.609+0000 7f86637fe640 1 --2- 172.21.15.52:0/3924214951 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f864c0a1d80 0x7f864c0a4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:28:24.608 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:24.609+0000 7f8660ff9640 1 -- 172.21.15.52:0/3924214951 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f865c0cdf20 con 0x7f8664107370 2024-03-04T09:28:24.608 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:24.609+0000 7f86637fe640 1 --2- 172.21.15.52:0/3924214951 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f864c0a1d80 0x7f864c0a4240 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f86540058c0 tx=0x7f8654005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:28:24.610 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:24.614+0000 7f8660ff9640 1 -- 172.21.15.52:0/3924214951 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f865c09b750 con 0x7f8664107370 2024-03-04T09:28:24.830 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:24.833+0000 7f8669cc2640 1 -- 172.21.15.52:0/3924214951 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f866402cff0 con 0x7f8664107370 2024-03-04T09:28:24.830 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:24.833+0000 7f8660ff9640 1 -- 172.21.15.52:0/3924214951 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+6874 (secure 0 0 0) 0x7f865c09b050 con 0x7f8664107370 2024-03-04T09:28:24.831 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-03-04T09:28:24.831 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":9,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6835/1007961019","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6834","nonce":1007961019},{"type":"v1","addr":"172.21.15.52:6835","nonce":1007961019}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6837/3753881283","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6836","nonce":3753881283},{"type":"v1","addr":"172.21.15.52:6837","nonce":3753881283}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24407,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6833/1818480323","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6832","nonce":1818480323},{"type":"v1","addr":"172.21.15.106:6833","nonce":1818480323}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24412,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6833/765768204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":765768204},{"type":"v1","addr":"172.21.15.191:6833","nonce":765768204}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24440,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6835/1541069897","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6834","nonce":1541069897},{"type":"v1","addr":"172.21.15.106:6835","nonce":1541069897}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24451,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6835/105048699","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":105048699},{"type":"v1","addr":"172.21.15.191:6835","nonce":105048699}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24463,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6837/4071064225","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6836","nonce":4071064225},{"type":"v1","addr":"172.21.15.191:6837","nonce":4071064225}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24473,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6837/2901755066","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6836","nonce":2901755066},{"type":"v1","addr":"172.21.15.106:6837","nonce":2901755066}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9}],"filesystems":[]} 2024-03-04T09:28:24.831 INFO:teuthology.orchestra.run.smithi052.stderr:dumped fsmap epoch 9 2024-03-04T09:28:24.832 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:24.835+0000 7f8669cc2640 1 -- 172.21.15.52:0/3924214951 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f864c0a1d80 msgr2=0x7f864c0a4240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:28:24.832 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:24.835+0000 7f8669cc2640 1 --2- 172.21.15.52:0/3924214951 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f864c0a1d80 0x7f864c0a4240 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f86540058c0 tx=0x7f8654005850 comp rx=0 tx=0).stop 2024-03-04T09:28:24.832 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:24.835+0000 7f8669cc2640 1 -- 172.21.15.52:0/3924214951 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8664107370 msgr2=0x7f86641a1120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:28:24.833 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:24.835+0000 7f8669cc2640 1 --2- 172.21.15.52:0/3924214951 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8664107370 0x7f86641a1120 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f865c00efd0 tx=0x7f865c00c6a0 comp rx=0 tx=0).stop 2024-03-04T09:28:24.833 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:24.835+0000 7f8669cc2640 1 -- 172.21.15.52:0/3924214951 shutdown_connections 2024-03-04T09:28:24.833 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:24.835+0000 7f8669cc2640 1 --2- 172.21.15.52:0/3924214951 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f864c0a1d80 0x7f864c0a4240 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:24.833 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:24.835+0000 7f8669cc2640 1 --2- 172.21.15.52:0/3924214951 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8664107370 0x7f86641a1120 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:24.833 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:24.835+0000 7f8669cc2640 1 --2- 172.21.15.52:0/3924214951 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8664104a40 0x7f866419a0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:24.833 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:24.835+0000 7f8669cc2640 1 --2- 172.21.15.52:0/3924214951 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8664102110 0x7f8664199b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:24.833 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:24.835+0000 7f8669cc2640 1 -- 172.21.15.52:0/3924214951 >> 172.21.15.52:0/3924214951 conn(0x7f86640fba60 msgr2=0x7f86641018a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:28:24.833 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:24.836+0000 7f8669cc2640 1 -- 172.21.15.52:0/3924214951 shutdown_connections 2024-03-04T09:28:24.833 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:24.836+0000 7f8669cc2640 1 -- 172.21.15.52:0/3924214951 wait complete. 2024-03-04T09:28:24.834 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:28:24 smithi052 ceph-mon[32473]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-04T09:28:24.836 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:28:24 smithi191 ceph-mon[36816]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-04T09:28:24.836 INFO:journalctl@ceph.mds.i.smithi191.stdout:Mar 04 09:28:24 smithi191 systemd[1]: Starting Ceph mds.i for 4e059f08-da08-11ee-95c6-87774f69a715... 2024-03-04T09:28:24.881 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:28:24 smithi106 ceph-mon[36484]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-04T09:28:25.132 INFO:journalctl@ceph.mds.i.smithi191.stdout:Mar 04 09:28:24 smithi191 podman[67389]: 2024-03-04 09:28:24.782576961 +0000 UTC m=+0.017465313 image pull f88ffb32aa6a7d5dfdc954d9f096f0c9cc47981d05ec8d117ba859a22d8185ba quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5 2024-03-04T09:28:25.133 INFO:journalctl@ceph.mds.i.smithi191.stdout:Mar 04 09:28:24 smithi191 podman[67389]: 2024-03-04 09:28:24.899771092 +0000 UTC m=+0.134659421 container create a07ca64ad21d83b913c50c069d97f5a3dad12b121405c9a545b65e2cc2e5a2f5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5, name=ceph-4e059f08-da08-11ee-95c6-87774f69a715-mds-i, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, org.label-schema.build-date=20240222, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, RELEASE=wip-lusov-qdb-1bc250c, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-03-04T09:28:25.501 INFO:journalctl@ceph.mds.i.smithi191.stdout:Mar 04 09:28:25 smithi191 podman[67389]: 2024-03-04 09:28:25.136762004 +0000 UTC m=+0.371650340 container init a07ca64ad21d83b913c50c069d97f5a3dad12b121405c9a545b65e2cc2e5a2f5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5, name=ceph-4e059f08-da08-11ee-95c6-87774f69a715-mds-i, org.label-schema.build-date=20240222, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, ceph=True, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-qdb-1bc250c, io.buildah.version=1.33.5) 2024-03-04T09:28:25.501 INFO:journalctl@ceph.mds.i.smithi191.stdout:Mar 04 09:28:25 smithi191 podman[67389]: 2024-03-04 09:28:25.139454778 +0000 UTC m=+0.374343116 container start a07ca64ad21d83b913c50c069d97f5a3dad12b121405c9a545b65e2cc2e5a2f5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5, name=ceph-4e059f08-da08-11ee-95c6-87774f69a715-mds-i, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240222, GIT_CLEAN=True, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, RELEASE=wip-lusov-qdb-1bc250c, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-04T09:28:25.501 INFO:journalctl@ceph.mds.i.smithi191.stdout:Mar 04 09:28:25 smithi191 ceph-mds[67408]: -- 172.21.15.191:0/588086495 <== mon.1 v2:172.21.15.191:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55750116da40 con 0x557501140800 2024-03-04T09:28:25.501 INFO:journalctl@ceph.mds.i.smithi191.stdout:Mar 04 09:28:25 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-mds-i[67404]: starting mds.i at 2024-03-04T09:28:25.501 INFO:journalctl@ceph.mds.i.smithi191.stdout:Mar 04 09:28:25 smithi191 bash[67389]: a07ca64ad21d83b913c50c069d97f5a3dad12b121405c9a545b65e2cc2e5a2f5 2024-03-04T09:28:25.501 INFO:journalctl@ceph.mds.i.smithi191.stdout:Mar 04 09:28:25 smithi191 systemd[1]: Started Ceph mds.i for 4e059f08-da08-11ee-95c6-87774f69a715. 2024-03-04T09:28:25.835 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:28:25 smithi052 ceph-mon[32473]: from='client.? 172.21.15.52:0/3924214951' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-04T09:28:25.835 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:28:25 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:28:25.835 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:28:25 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:28:25.835 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:28:25 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:28:25.835 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:28:25 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-04T09:28:25.835 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:28:25 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-04T09:28:25.835 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:28:25 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T09:28:25.835 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:28:25 smithi052 ceph-mon[32473]: Deploying daemon mds.g on smithi052 2024-03-04T09:28:25.836 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:28:25 smithi191 ceph-mon[36816]: from='client.? 172.21.15.52:0/3924214951' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-04T09:28:25.836 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:28:25 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:28:25.836 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:28:25 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:28:25.836 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:28:25 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:28:25.836 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:28:25 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-04T09:28:25.836 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:28:25 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-04T09:28:25.836 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:28:25 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T09:28:25.836 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:28:25 smithi191 ceph-mon[36816]: Deploying daemon mds.g on smithi052 2024-03-04T09:28:25.881 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:28:25 smithi106 ceph-mon[36484]: from='client.? 172.21.15.52:0/3924214951' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-04T09:28:25.881 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:28:25 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:28:25.881 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:28:25 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:28:25.881 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:28:25 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:28:25.882 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:28:25 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-04T09:28:25.882 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:28:25 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-04T09:28:25.882 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:28:25 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T09:28:25.882 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:28:25 smithi106 ceph-mon[36484]: Deploying daemon mds.g on smithi052 2024-03-04T09:28:26.585 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:28:26 smithi052 ceph-mon[32473]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-04T09:28:26.585 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:28:26 smithi052 ceph-mon[32473]: mds.? [v2:172.21.15.191:6838/2194324891,v1:172.21.15.191:6839/2194324891] up:boot 2024-03-04T09:28:26.585 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:28:26 smithi052 ceph-mon[32473]: fsmap 9 up:standby 2024-03-04T09:28:26.585 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:28:26 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-03-04T09:28:26.836 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:28:26 smithi191 ceph-mon[36816]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-04T09:28:26.836 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:28:26 smithi191 ceph-mon[36816]: mds.? [v2:172.21.15.191:6838/2194324891,v1:172.21.15.191:6839/2194324891] up:boot 2024-03-04T09:28:26.836 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:28:26 smithi191 ceph-mon[36816]: fsmap 9 up:standby 2024-03-04T09:28:26.837 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:28:26 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-03-04T09:28:26.845 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T09:28:26.881 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:28:26 smithi106 ceph-mon[36484]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-04T09:28:26.881 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:28:26 smithi106 ceph-mon[36484]: mds.? [v2:172.21.15.191:6838/2194324891,v1:172.21.15.191:6839/2194324891] up:boot 2024-03-04T09:28:26.881 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:28:26 smithi106 ceph-mon[36484]: fsmap 9 up:standby 2024-03-04T09:28:26.881 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:28:26 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-03-04T09:28:26.969 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:26.967+0000 7f9c18a2a640 1 -- 172.21.15.52:0/127897905 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9c141016e0 msgr2=0x7f9c14101b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:28:26.970 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:26.967+0000 7f9c18a2a640 1 --2- 172.21.15.52:0/127897905 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9c141016e0 0x7f9c14101b60 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f9c08009900 tx=0x7f9c0802bc30 comp rx=0 tx=0).stop 2024-03-04T09:28:26.970 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:26.967+0000 7f9c18a2a640 1 -- 172.21.15.52:0/127897905 shutdown_connections 2024-03-04T09:28:26.970 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:26.967+0000 7f9c18a2a640 1 --2- 172.21.15.52:0/127897905 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9c141020a0 0x7f9c14108930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:26.970 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:26.967+0000 7f9c18a2a640 1 --2- 172.21.15.52:0/127897905 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9c141016e0 0x7f9c14101b60 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:26.970 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:26.967+0000 7f9c18a2a640 1 --2- 172.21.15.52:0/127897905 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9c141004f0 0x7f9c14100910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:26.970 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:26.967+0000 7f9c18a2a640 1 -- 172.21.15.52:0/127897905 >> 172.21.15.52:0/127897905 conn(0x7f9c140fba60 msgr2=0x7f9c140fdea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:28:26.970 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:26.967+0000 7f9c18a2a640 1 -- 172.21.15.52:0/127897905 shutdown_connections 2024-03-04T09:28:26.970 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:26.967+0000 7f9c18a2a640 1 -- 172.21.15.52:0/127897905 wait complete. 2024-03-04T09:28:26.970 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:26.968+0000 7f9c18a2a640 1 Processor -- start 2024-03-04T09:28:26.970 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:26.968+0000 7f9c18a2a640 1 -- start start 2024-03-04T09:28:26.970 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:26.968+0000 7f9c18a2a640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9c141004f0 0x7f9c14196750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:28:26.970 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:26.968+0000 7f9c18a2a640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9c141016e0 0x7f9c14196c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:28:26.971 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:26.968+0000 7f9c18a2a640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9c141020a0 0x7f9c1419dd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:28:26.971 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:26.968+0000 7f9c18a2a640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c1410c180 con 0x7f9c141016e0 2024-03-04T09:28:26.971 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:26.968+0000 7f9c18a2a640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c1410c000 con 0x7f9c141004f0 2024-03-04T09:28:26.971 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:26.968+0000 7f9c18a2a640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c1410c300 con 0x7f9c141020a0 2024-03-04T09:28:26.971 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:26.968+0000 7f9c11d9b640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9c141016e0 0x7f9c14196c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:28:26.971 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:26.968+0000 7f9c11d9b640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9c141016e0 0x7f9c14196c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:46046/0 (socket says 172.21.15.52:46046) 2024-03-04T09:28:26.971 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:26.968+0000 7f9c11d9b640 1 -- 172.21.15.52:0/2013412263 learned_addr learned my addr 172.21.15.52:0/2013412263 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-04T09:28:26.971 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:26.968+0000 7f9c1259c640 1 --2- 172.21.15.52:0/2013412263 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9c141004f0 0x7f9c14196750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:28:26.971 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:26.968+0000 7f9c12d9d640 1 --2- 172.21.15.52:0/2013412263 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9c141020a0 0x7f9c1419dd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:28:26.971 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:26.968+0000 7f9c11d9b640 1 -- 172.21.15.52:0/2013412263 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9c141020a0 msgr2=0x7f9c1419dd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:28:26.971 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:26.968+0000 7f9c11d9b640 1 --2- 172.21.15.52:0/2013412263 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9c141020a0 0x7f9c1419dd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:26.971 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:26.968+0000 7f9c11d9b640 1 -- 172.21.15.52:0/2013412263 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9c141004f0 msgr2=0x7f9c14196750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:28:26.971 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:26.968+0000 7f9c11d9b640 1 --2- 172.21.15.52:0/2013412263 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9c141004f0 0x7f9c14196750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:26.972 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:26.968+0000 7f9c11d9b640 1 -- 172.21.15.52:0/2013412263 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c1419e370 con 0x7f9c141016e0 2024-03-04T09:28:26.972 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:26.968+0000 7f9c12d9d640 1 --2- 172.21.15.52:0/2013412263 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9c141020a0 0x7f9c1419dd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T09:28:26.972 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:26.968+0000 7f9c11d9b640 1 --2- 172.21.15.52:0/2013412263 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9c141016e0 0x7f9c14196c90 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f9c08009230 tx=0x7f9c08005dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:28:26.972 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:26.969+0000 7f9c037fe640 1 -- 172.21.15.52:0/2013412263 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c08041070 con 0x7f9c141016e0 2024-03-04T09:28:26.972 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:26.969+0000 7f9c037fe640 1 -- 172.21.15.52:0/2013412263 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9c080054e0 con 0x7f9c141016e0 2024-03-04T09:28:26.972 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:26.969+0000 7f9c18a2a640 1 -- 172.21.15.52:0/2013412263 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c1419e600 con 0x7f9c141016e0 2024-03-04T09:28:26.972 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:26.969+0000 7f9c037fe640 1 -- 172.21.15.52:0/2013412263 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c08034440 con 0x7f9c141016e0 2024-03-04T09:28:26.972 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:26.969+0000 7f9c18a2a640 1 -- 172.21.15.52:0/2013412263 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c1419eaf0 con 0x7f9c141016e0 2024-03-04T09:28:26.972 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:26.970+0000 7f9c18a2a640 1 -- 172.21.15.52:0/2013412263 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9c141077b0 con 0x7f9c141016e0 2024-03-04T09:28:26.972 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:26.971+0000 7f9c037fe640 1 -- 172.21.15.52:0/2013412263 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 16) v1 ==== 136756+0+0 (secure 0 0 0) 0x7f9c08044070 con 0x7f9c141016e0 2024-03-04T09:28:26.972 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:26.971+0000 7f9c037fe640 1 --2- 172.21.15.52:0/2013412263 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f9bfc0a1d80 0x7f9bfc0a4240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:28:26.972 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:26.972+0000 7f9c1259c640 1 --2- 172.21.15.52:0/2013412263 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f9bfc0a1d80 0x7f9bfc0a4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:28:26.972 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:26.972+0000 7f9c037fe640 1 -- 172.21.15.52:0/2013412263 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f9c080efab0 con 0x7f9c141016e0 2024-03-04T09:28:26.973 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:26.972+0000 7f9c1259c640 1 --2- 172.21.15.52:0/2013412263 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f9bfc0a1d80 0x7f9bfc0a4240 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f9c0c0058c0 tx=0x7f9c0c005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:28:26.973 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:26.975+0000 7f9c037fe640 1 -- 172.21.15.52:0/2013412263 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f9c080bd380 con 0x7f9c141016e0 2024-03-04T09:28:27.195 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:27.199+0000 7f9c18a2a640 1 -- 172.21.15.52:0/2013412263 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9c1402cff0 con 0x7f9c141016e0 2024-03-04T09:28:27.196 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:27.199+0000 7f9c037fe640 1 -- 172.21.15.52:0/2013412263 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) v1 ==== 94+0+7670 (secure 0 0 0) 0x7f9c080bcc80 con 0x7f9c141016e0 2024-03-04T09:28:27.196 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-03-04T09:28:27.197 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":10,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6835/1007961019","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6834","nonce":1007961019},{"type":"v1","addr":"172.21.15.52:6835","nonce":1007961019}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6837/3753881283","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6836","nonce":3753881283},{"type":"v1","addr":"172.21.15.52:6837","nonce":3753881283}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24407,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6833/1818480323","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6832","nonce":1818480323},{"type":"v1","addr":"172.21.15.106:6833","nonce":1818480323}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24412,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6833/765768204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":765768204},{"type":"v1","addr":"172.21.15.191:6833","nonce":765768204}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24440,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6835/1541069897","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6834","nonce":1541069897},{"type":"v1","addr":"172.21.15.106:6835","nonce":1541069897}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24451,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6835/105048699","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":105048699},{"type":"v1","addr":"172.21.15.191:6835","nonce":105048699}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24463,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6837/4071064225","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6836","nonce":4071064225},{"type":"v1","addr":"172.21.15.191:6837","nonce":4071064225}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24473,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6837/2901755066","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6836","nonce":2901755066},{"type":"v1","addr":"172.21.15.106:6837","nonce":2901755066}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":24490,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6839/2194324891","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6838","nonce":2194324891},{"type":"v1","addr":"172.21.15.191:6839","nonce":2194324891}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10}],"filesystems":[]} 2024-03-04T09:28:27.197 INFO:teuthology.orchestra.run.smithi052.stderr:dumped fsmap epoch 10 2024-03-04T09:28:27.198 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:27.201+0000 7f9c18a2a640 1 -- 172.21.15.52:0/2013412263 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f9bfc0a1d80 msgr2=0x7f9bfc0a4240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:28:27.198 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:27.201+0000 7f9c18a2a640 1 --2- 172.21.15.52:0/2013412263 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f9bfc0a1d80 0x7f9bfc0a4240 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f9c0c0058c0 tx=0x7f9c0c005850 comp rx=0 tx=0).stop 2024-03-04T09:28:27.198 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:27.201+0000 7f9c18a2a640 1 -- 172.21.15.52:0/2013412263 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9c141016e0 msgr2=0x7f9c14196c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:28:27.199 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:27.201+0000 7f9c18a2a640 1 --2- 172.21.15.52:0/2013412263 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9c141016e0 0x7f9c14196c90 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f9c08009230 tx=0x7f9c08005dd0 comp rx=0 tx=0).stop 2024-03-04T09:28:27.199 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:27.201+0000 7f9c18a2a640 1 -- 172.21.15.52:0/2013412263 shutdown_connections 2024-03-04T09:28:27.199 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:27.201+0000 7f9c18a2a640 1 --2- 172.21.15.52:0/2013412263 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f9bfc0a1d80 0x7f9bfc0a4240 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:27.199 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:27.201+0000 7f9c18a2a640 1 --2- 172.21.15.52:0/2013412263 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9c141020a0 0x7f9c1419dd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:27.199 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:27.201+0000 7f9c18a2a640 1 --2- 172.21.15.52:0/2013412263 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9c141016e0 0x7f9c14196c90 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:27.199 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:27.201+0000 7f9c18a2a640 1 --2- 172.21.15.52:0/2013412263 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9c141004f0 0x7f9c14196750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:27.199 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:27.201+0000 7f9c18a2a640 1 -- 172.21.15.52:0/2013412263 >> 172.21.15.52:0/2013412263 conn(0x7f9c140fba60 msgr2=0x7f9c14104980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:28:27.199 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:27.202+0000 7f9c18a2a640 1 -- 172.21.15.52:0/2013412263 shutdown_connections 2024-03-04T09:28:27.199 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:27.202+0000 7f9c18a2a640 1 -- 172.21.15.52:0/2013412263 wait complete. 2024-03-04T09:28:27.787 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:28:27 smithi052 ceph-mon[32473]: from='client.? 172.21.15.52:0/2013412263' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-04T09:28:27.788 INFO:journalctl@ceph.mds.g.smithi052.stdout:Mar 04 09:28:27 smithi052 systemd[1]: Starting Ceph mds.g for 4e059f08-da08-11ee-95c6-87774f69a715... 2024-03-04T09:28:27.836 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:28:27 smithi191 ceph-mon[36816]: from='client.? 172.21.15.52:0/2013412263' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-04T09:28:27.881 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:28:27 smithi106 ceph-mon[36484]: from='client.? 172.21.15.52:0/2013412263' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-04T09:28:28.046 INFO:journalctl@ceph.mds.g.smithi052.stdout:Mar 04 09:28:27 smithi052 podman[71395]: 2024-03-04 09:28:27.693621766 +0000 UTC m=+0.017643561 image pull f88ffb32aa6a7d5dfdc954d9f096f0c9cc47981d05ec8d117ba859a22d8185ba quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5 2024-03-04T09:28:28.046 INFO:journalctl@ceph.mds.g.smithi052.stdout:Mar 04 09:28:27 smithi052 podman[71395]: 2024-03-04 09:28:27.813173381 +0000 UTC m=+0.137195158 container create 828128f1c5012dfee78709bdb7e0b2cce64a1ac65dbaab27e016c4ef89236fde (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5, name=ceph-4e059f08-da08-11ee-95c6-87774f69a715-mds-g, org.label-schema.vendor=CentOS, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, GIT_CLEAN=True, org.label-schema.build-date=20240222, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-qdb-1bc250c, GIT_BRANCH=HEAD) 2024-03-04T09:28:28.046 INFO:journalctl@ceph.mds.g.smithi052.stdout:Mar 04 09:28:28 smithi052 podman[71395]: 2024-03-04 09:28:28.049624169 +0000 UTC m=+0.373645953 container init 828128f1c5012dfee78709bdb7e0b2cce64a1ac65dbaab27e016c4ef89236fde (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5, name=ceph-4e059f08-da08-11ee-95c6-87774f69a715-mds-g, GIT_CLEAN=True, org.label-schema.build-date=20240222, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-qdb-1bc250c, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, maintainer=Guillaume Abrioux ) 2024-03-04T09:28:28.335 INFO:journalctl@ceph.mds.g.smithi052.stdout:Mar 04 09:28:28 smithi052 podman[71395]: 2024-03-04 09:28:28.052519116 +0000 UTC m=+0.376540897 container start 828128f1c5012dfee78709bdb7e0b2cce64a1ac65dbaab27e016c4ef89236fde (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5, name=ceph-4e059f08-da08-11ee-95c6-87774f69a715-mds-g, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-qdb-1bc250c, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, org.label-schema.build-date=20240222, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.33.5) 2024-03-04T09:28:28.335 INFO:journalctl@ceph.mds.g.smithi052.stdout:Mar 04 09:28:28 smithi052 ceph-mds[71414]: -- 172.21.15.52:0/2695020317 <== mon.0 v2:172.21.15.52:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x563eb9f7da40 con 0x563eb9f50800 2024-03-04T09:28:28.335 INFO:journalctl@ceph.mds.g.smithi052.stdout:Mar 04 09:28:28 smithi052 ceph-4e059f08-da08-11ee-95c6-87774f69a715-mds-g[71410]: starting mds.g at 2024-03-04T09:28:28.335 INFO:journalctl@ceph.mds.g.smithi052.stdout:Mar 04 09:28:28 smithi052 bash[71395]: 828128f1c5012dfee78709bdb7e0b2cce64a1ac65dbaab27e016c4ef89236fde 2024-03-04T09:28:28.335 INFO:journalctl@ceph.mds.g.smithi052.stdout:Mar 04 09:28:28 smithi052 systemd[1]: Started Ceph mds.g for 4e059f08-da08-11ee-95c6-87774f69a715. 2024-03-04T09:28:29.210 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T09:28:29.317 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:29.320+0000 7f71cc720640 1 -- 172.21.15.52:0/562151060 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f71c41004f0 msgr2=0x7f71c4100910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:28:29.318 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:29.320+0000 7f71cc720640 1 --2- 172.21.15.52:0/562151060 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f71c41004f0 0x7f71c4100910 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f71b4009900 tx=0x7f71b402bc30 comp rx=0 tx=0).stop 2024-03-04T09:28:29.318 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:29.320+0000 7f71cc720640 1 -- 172.21.15.52:0/562151060 shutdown_connections 2024-03-04T09:28:29.318 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:29.320+0000 7f71cc720640 1 --2- 172.21.15.52:0/562151060 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f71c41020a0 0x7f71c4108930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:29.318 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:29.320+0000 7f71cc720640 1 --2- 172.21.15.52:0/562151060 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f71c41016e0 0x7f71c4101b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:29.318 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:29.320+0000 7f71cc720640 1 --2- 172.21.15.52:0/562151060 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f71c41004f0 0x7f71c4100910 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:29.318 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:29.320+0000 7f71cc720640 1 -- 172.21.15.52:0/562151060 >> 172.21.15.52:0/562151060 conn(0x7f71c40fba60 msgr2=0x7f71c40fdea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:28:29.318 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:29.320+0000 7f71cc720640 1 -- 172.21.15.52:0/562151060 shutdown_connections 2024-03-04T09:28:29.318 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:29.320+0000 7f71cc720640 1 -- 172.21.15.52:0/562151060 wait complete. 2024-03-04T09:28:29.320 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:29.321+0000 7f71cc720640 1 Processor -- start 2024-03-04T09:28:29.321 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:29.321+0000 7f71cc720640 1 -- start start 2024-03-04T09:28:29.321 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:29.321+0000 7f71cc720640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f71c41004f0 0x7f71c4196800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:28:29.321 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:29.321+0000 7f71cc720640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f71c41016e0 0x7f71c4196d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:28:29.321 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:29.321+0000 7f71cc720640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f71c41020a0 0x7f71c419ddb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:28:29.321 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:29.321+0000 7f71cc720640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71c410c180 con 0x7f71c41016e0 2024-03-04T09:28:29.321 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:29.321+0000 7f71cc720640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71c410c000 con 0x7f71c41004f0 2024-03-04T09:28:29.321 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:29.321+0000 7f71cc720640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71c410c300 con 0x7f71c41020a0 2024-03-04T09:28:29.321 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:29.321+0000 7f71c9cbb640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f71c41016e0 0x7f71c4196d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:28:29.321 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:29.322+0000 7f71c9cbb640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f71c41016e0 0x7f71c4196d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:46070/0 (socket says 172.21.15.52:46070) 2024-03-04T09:28:29.321 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:29.322+0000 7f71c9cbb640 1 -- 172.21.15.52:0/3834936060 learned_addr learned my addr 172.21.15.52:0/3834936060 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-04T09:28:29.321 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:29.322+0000 7f71cacbd640 1 --2- 172.21.15.52:0/3834936060 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f71c41020a0 0x7f71c419ddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:28:29.321 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:29.322+0000 7f71ca4bc640 1 --2- 172.21.15.52:0/3834936060 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f71c41004f0 0x7f71c4196800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:28:29.321 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:29.322+0000 7f71c9cbb640 1 -- 172.21.15.52:0/3834936060 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f71c41020a0 msgr2=0x7f71c419ddb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:28:29.322 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:29.322+0000 7f71c9cbb640 1 --2- 172.21.15.52:0/3834936060 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f71c41020a0 0x7f71c419ddb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:29.322 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:29.322+0000 7f71c9cbb640 1 -- 172.21.15.52:0/3834936060 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f71c41004f0 msgr2=0x7f71c4196800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:28:29.322 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:29.322+0000 7f71c9cbb640 1 --2- 172.21.15.52:0/3834936060 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f71c41004f0 0x7f71c4196800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:29.322 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:29.322+0000 7f71c9cbb640 1 -- 172.21.15.52:0/3834936060 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71c419e420 con 0x7f71c41016e0 2024-03-04T09:28:29.322 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:29.322+0000 7f71cacbd640 1 --2- 172.21.15.52:0/3834936060 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f71c41020a0 0x7f71c419ddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-04T09:28:29.322 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:29.322+0000 7f71c9cbb640 1 --2- 172.21.15.52:0/3834936060 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f71c41016e0 0x7f71c4196d40 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f71c000c890 tx=0x7f71c000cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:28:29.322 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:29.322+0000 7f71bb7fe640 1 -- 172.21.15.52:0/3834936060 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71c0004220 con 0x7f71c41016e0 2024-03-04T09:28:29.322 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:29.322+0000 7f71bb7fe640 1 -- 172.21.15.52:0/3834936060 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f71c00043c0 con 0x7f71c41016e0 2024-03-04T09:28:29.322 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:29.322+0000 7f71bb7fe640 1 -- 172.21.15.52:0/3834936060 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71c00049e0 con 0x7f71c41016e0 2024-03-04T09:28:29.322 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:29.322+0000 7f71cc720640 1 -- 172.21.15.52:0/3834936060 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71c419e710 con 0x7f71c41016e0 2024-03-04T09:28:29.322 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:29.322+0000 7f71cc720640 1 -- 172.21.15.52:0/3834936060 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71c419ecb0 con 0x7f71c41016e0 2024-03-04T09:28:29.322 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:29.323+0000 7f71cc720640 1 -- 172.21.15.52:0/3834936060 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f71c4107890 con 0x7f71c41016e0 2024-03-04T09:28:29.323 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:29.324+0000 7f71bb7fe640 1 -- 172.21.15.52:0/3834936060 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 16) v1 ==== 136756+0+0 (secure 0 0 0) 0x7f71c00079b0 con 0x7f71c41016e0 2024-03-04T09:28:29.323 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:29.325+0000 7f71bb7fe640 1 --2- 172.21.15.52:0/3834936060 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f71b00a1e00 0x7f71b00a42c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:28:29.323 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:29.325+0000 7f71ca4bc640 1 --2- 172.21.15.52:0/3834936060 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f71b00a1e00 0x7f71b00a42c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:28:29.323 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:29.325+0000 7f71bb7fe640 1 -- 172.21.15.52:0/3834936060 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f71c00cda50 con 0x7f71c41016e0 2024-03-04T09:28:29.323 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:29.325+0000 7f71ca4bc640 1 --2- 172.21.15.52:0/3834936060 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f71b00a1e00 0x7f71b00a42c0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f71b402c7b0 tx=0x7f71b4005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:28:29.326 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:29.329+0000 7f71bb7fe640 1 -- 172.21.15.52:0/3834936060 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f71c0014030 con 0x7f71c41016e0 2024-03-04T09:28:29.390 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:28:29 smithi052 ceph-mon[32473]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-04T09:28:29.390 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:28:29 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:28:29.390 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:28:29 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:28:29.390 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:28:29 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:28:29.391 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:28:29 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-04T09:28:29.391 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:28:29 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-04T09:28:29.391 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:28:29 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T09:28:29.391 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:28:29 smithi052 ceph-mon[32473]: Deploying daemon mds.a on smithi052 2024-03-04T09:28:29.391 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:28:29 smithi052 ceph-mon[32473]: mds.? [v2:172.21.15.52:6838/3467897808,v1:172.21.15.52:6839/3467897808] up:boot 2024-03-04T09:28:29.391 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:28:29 smithi052 ceph-mon[32473]: fsmap 10 up:standby 2024-03-04T09:28:29.391 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:28:29 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-03-04T09:28:29.554 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:29.557+0000 7f71cc720640 1 -- 172.21.15.52:0/3834936060 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f71c402cff0 con 0x7f71c41016e0 2024-03-04T09:28:29.554 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:29.558+0000 7f71bb7fe640 1 -- 172.21.15.52:0/3834936060 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) v1 ==== 94+0+8462 (secure 0 0 0) 0x7f71c009b280 con 0x7f71c41016e0 2024-03-04T09:28:29.555 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-03-04T09:28:29.555 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":11,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6835/1007961019","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6834","nonce":1007961019},{"type":"v1","addr":"172.21.15.52:6835","nonce":1007961019}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6837/3753881283","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6836","nonce":3753881283},{"type":"v1","addr":"172.21.15.52:6837","nonce":3753881283}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14628,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6839/3467897808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6838","nonce":3467897808},{"type":"v1","addr":"172.21.15.52:6839","nonce":3467897808}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":24407,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6833/1818480323","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6832","nonce":1818480323},{"type":"v1","addr":"172.21.15.106:6833","nonce":1818480323}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24412,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6833/765768204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":765768204},{"type":"v1","addr":"172.21.15.191:6833","nonce":765768204}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24440,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6835/1541069897","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6834","nonce":1541069897},{"type":"v1","addr":"172.21.15.106:6835","nonce":1541069897}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24451,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6835/105048699","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":105048699},{"type":"v1","addr":"172.21.15.191:6835","nonce":105048699}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24463,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6837/4071064225","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6836","nonce":4071064225},{"type":"v1","addr":"172.21.15.191:6837","nonce":4071064225}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24473,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6837/2901755066","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6836","nonce":2901755066},{"type":"v1","addr":"172.21.15.106:6837","nonce":2901755066}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":24490,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6839/2194324891","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6838","nonce":2194324891},{"type":"v1","addr":"172.21.15.191:6839","nonce":2194324891}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10}],"filesystems":[]} 2024-03-04T09:28:29.555 INFO:teuthology.orchestra.run.smithi052.stderr:dumped fsmap epoch 11 2024-03-04T09:28:29.557 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:29.559+0000 7f71cc720640 1 -- 172.21.15.52:0/3834936060 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f71b00a1e00 msgr2=0x7f71b00a42c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:28:29.557 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:29.559+0000 7f71cc720640 1 --2- 172.21.15.52:0/3834936060 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f71b00a1e00 0x7f71b00a42c0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f71b402c7b0 tx=0x7f71b4005fd0 comp rx=0 tx=0).stop 2024-03-04T09:28:29.557 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:29.559+0000 7f71cc720640 1 -- 172.21.15.52:0/3834936060 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f71c41016e0 msgr2=0x7f71c4196d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:28:29.557 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:29.559+0000 7f71cc720640 1 --2- 172.21.15.52:0/3834936060 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f71c41016e0 0x7f71c4196d40 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f71c000c890 tx=0x7f71c000cd50 comp rx=0 tx=0).stop 2024-03-04T09:28:29.557 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:29.560+0000 7f71cc720640 1 -- 172.21.15.52:0/3834936060 shutdown_connections 2024-03-04T09:28:29.557 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:29.560+0000 7f71cc720640 1 --2- 172.21.15.52:0/3834936060 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f71b00a1e00 0x7f71b00a42c0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:29.557 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:29.560+0000 7f71cc720640 1 --2- 172.21.15.52:0/3834936060 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f71c41020a0 0x7f71c419ddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:29.557 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:29.560+0000 7f71cc720640 1 --2- 172.21.15.52:0/3834936060 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f71c41016e0 0x7f71c4196d40 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:29.557 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:29.560+0000 7f71cc720640 1 --2- 172.21.15.52:0/3834936060 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f71c41004f0 0x7f71c4196800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:29.557 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:29.560+0000 7f71cc720640 1 -- 172.21.15.52:0/3834936060 >> 172.21.15.52:0/3834936060 conn(0x7f71c40fba60 msgr2=0x7f71c4104a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:28:29.560 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:29.560+0000 7f71cc720640 1 -- 172.21.15.52:0/3834936060 shutdown_connections 2024-03-04T09:28:29.561 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:29.560+0000 7f71cc720640 1 -- 172.21.15.52:0/3834936060 wait complete. 2024-03-04T09:28:29.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:28:29 smithi191 ceph-mon[36816]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-04T09:28:29.587 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:28:29 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:28:29.587 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:28:29 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:28:29.587 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:28:29 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:28:29.587 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:28:29 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-04T09:28:29.587 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:28:29 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-04T09:28:29.587 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:28:29 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T09:28:29.587 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:28:29 smithi191 ceph-mon[36816]: Deploying daemon mds.a on smithi052 2024-03-04T09:28:29.587 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:28:29 smithi191 ceph-mon[36816]: mds.? [v2:172.21.15.52:6838/3467897808,v1:172.21.15.52:6839/3467897808] up:boot 2024-03-04T09:28:29.587 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:28:29 smithi191 ceph-mon[36816]: fsmap 10 up:standby 2024-03-04T09:28:29.587 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:28:29 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-03-04T09:28:29.631 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:28:29 smithi106 ceph-mon[36484]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-04T09:28:29.631 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:28:29 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:28:29.631 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:28:29 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:28:29.631 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:28:29 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:28:29.631 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:28:29 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-04T09:28:29.632 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:28:29 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-04T09:28:29.632 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:28:29 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T09:28:29.632 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:28:29 smithi106 ceph-mon[36484]: Deploying daemon mds.a on smithi052 2024-03-04T09:28:29.632 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:28:29 smithi106 ceph-mon[36484]: mds.? [v2:172.21.15.52:6838/3467897808,v1:172.21.15.52:6839/3467897808] up:boot 2024-03-04T09:28:29.632 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:28:29 smithi106 ceph-mon[36484]: fsmap 10 up:standby 2024-03-04T09:28:29.632 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:28:29 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-03-04T09:28:30.241 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:28:30 smithi052 ceph-mon[32473]: from='client.? 172.21.15.52:0/3834936060' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-04T09:28:30.585 INFO:journalctl@ceph.mds.a.smithi052.stdout:Mar 04 09:28:30 smithi052 systemd[1]: Starting Ceph mds.a for 4e059f08-da08-11ee-95c6-87774f69a715... 2024-03-04T09:28:30.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:28:30 smithi191 ceph-mon[36816]: from='client.? 172.21.15.52:0/3834936060' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-04T09:28:30.631 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:28:30 smithi106 ceph-mon[36484]: from='client.? 172.21.15.52:0/3834936060' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-04T09:28:30.954 INFO:journalctl@ceph.mds.a.smithi052.stdout:Mar 04 09:28:30 smithi052 podman[71758]: 2024-03-04 09:28:30.646466225 +0000 UTC m=+0.109182303 container create f1c2020db07bcf3b2c1b71521436149f1ce55ddae48dd6c52698c0bec66355ac (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5, name=ceph-4e059f08-da08-11ee-95c6-87774f69a715-mds-a, ceph=True, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-qdb-1bc250c, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.build-date=20240222, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-04T09:28:30.954 INFO:journalctl@ceph.mds.a.smithi052.stdout:Mar 04 09:28:30 smithi052 podman[71758]: 2024-03-04 09:28:30.554729552 +0000 UTC m=+0.017445630 image pull f88ffb32aa6a7d5dfdc954d9f096f0c9cc47981d05ec8d117ba859a22d8185ba quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5 2024-03-04T09:28:30.954 INFO:journalctl@ceph.mds.a.smithi052.stdout:Mar 04 09:28:30 smithi052 podman[71758]: 2024-03-04 09:28:30.857640586 +0000 UTC m=+0.320356666 container init f1c2020db07bcf3b2c1b71521436149f1ce55ddae48dd6c52698c0bec66355ac (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5, name=ceph-4e059f08-da08-11ee-95c6-87774f69a715-mds-a, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.build-date=20240222, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-qdb-1bc250c) 2024-03-04T09:28:30.954 INFO:journalctl@ceph.mds.a.smithi052.stdout:Mar 04 09:28:30 smithi052 podman[71758]: 2024-03-04 09:28:30.860261601 +0000 UTC m=+0.322977681 container start f1c2020db07bcf3b2c1b71521436149f1ce55ddae48dd6c52698c0bec66355ac (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5, name=ceph-4e059f08-da08-11ee-95c6-87774f69a715-mds-a, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=wip-lusov-qdb-1bc250c, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240222, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_CLEAN=True) 2024-03-04T09:28:30.954 INFO:journalctl@ceph.mds.a.smithi052.stdout:Mar 04 09:28:30 smithi052 ceph-mds[71777]: -- 172.21.15.52:0/409854563 <== mon.0 v2:172.21.15.52:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5556c417fa40 con 0x5556c4153000 2024-03-04T09:28:30.954 INFO:journalctl@ceph.mds.a.smithi052.stdout:Mar 04 09:28:30 smithi052 ceph-4e059f08-da08-11ee-95c6-87774f69a715-mds-a[71773]: starting mds.a at 2024-03-04T09:28:30.954 INFO:journalctl@ceph.mds.a.smithi052.stdout:Mar 04 09:28:30 smithi052 bash[71758]: f1c2020db07bcf3b2c1b71521436149f1ce55ddae48dd6c52698c0bec66355ac 2024-03-04T09:28:31.204 INFO:journalctl@ceph.mds.a.smithi052.stdout:Mar 04 09:28:30 smithi052 systemd[1]: Started Ceph mds.a for 4e059f08-da08-11ee-95c6-87774f69a715. 2024-03-04T09:28:31.568 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T09:28:31.584 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:28:31 smithi052 ceph-mon[32473]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-04T09:28:31.585 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:28:31 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:28:31.585 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:28:31 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:28:31.585 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:28:31 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:28:31.585 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:28:31 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-04T09:28:31.585 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:28:31 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-04T09:28:31.585 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:28:31 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T09:28:31.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:28:31 smithi191 ceph-mon[36816]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-04T09:28:31.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:28:31 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:28:31.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:28:31 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:28:31.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:28:31 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:28:31.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:28:31 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-04T09:28:31.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:28:31 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-04T09:28:31.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:28:31 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T09:28:31.631 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:28:31 smithi106 ceph-mon[36484]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-04T09:28:31.632 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:28:31 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:28:31.632 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:28:31 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:28:31.632 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:28:31 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:28:31.632 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:28:31 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-04T09:28:31.632 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:28:31 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-04T09:28:31.632 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:28:31 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T09:28:31.683 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:31.686+0000 7ff700bc7640 1 -- 172.21.15.52:0/439117088 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff6fc1020c0 msgr2=0x7ff6fc108950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:28:31.683 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:31.686+0000 7ff700bc7640 1 --2- 172.21.15.52:0/439117088 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff6fc1020c0 0x7ff6fc108950 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7ff6ec009ca0 tx=0x7ff6ec02c0f0 comp rx=0 tx=0).stop 2024-03-04T09:28:31.683 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:31.686+0000 7ff700bc7640 1 -- 172.21.15.52:0/439117088 shutdown_connections 2024-03-04T09:28:31.684 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:31.686+0000 7ff700bc7640 1 --2- 172.21.15.52:0/439117088 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff6fc1020c0 0x7ff6fc108950 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:31.684 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:31.686+0000 7ff700bc7640 1 --2- 172.21.15.52:0/439117088 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff6fc101700 0x7ff6fc101b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:31.684 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:31.686+0000 7ff700bc7640 1 --2- 172.21.15.52:0/439117088 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff6fc100510 0x7ff6fc100930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:31.684 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:31.686+0000 7ff700bc7640 1 -- 172.21.15.52:0/439117088 >> 172.21.15.52:0/439117088 conn(0x7ff6fc0fba80 msgr2=0x7ff6fc0fdec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:28:31.684 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:31.686+0000 7ff700bc7640 1 -- 172.21.15.52:0/439117088 shutdown_connections 2024-03-04T09:28:31.684 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:31.686+0000 7ff700bc7640 1 -- 172.21.15.52:0/439117088 wait complete. 2024-03-04T09:28:31.686 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:31.687+0000 7ff700bc7640 1 Processor -- start 2024-03-04T09:28:31.686 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:31.687+0000 7ff700bc7640 1 -- start start 2024-03-04T09:28:31.686 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:31.687+0000 7ff700bc7640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff6fc100510 0x7ff6fc06a790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:28:31.687 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:31.687+0000 7ff700bc7640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff6fc101700 0x7ff6fc19c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:28:31.687 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:31.687+0000 7ff700bc7640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff6fc1020c0 0x7ff6fc19ec10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:28:31.687 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:31.687+0000 7ff700bc7640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6fc10c1a0 con 0x7ff6fc1020c0 2024-03-04T09:28:31.687 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:31.687+0000 7ff700bc7640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6fc10c020 con 0x7ff6fc100510 2024-03-04T09:28:31.687 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:31.687+0000 7ff700bc7640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6fc10c320 con 0x7ff6fc101700 2024-03-04T09:28:31.687 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:31.687+0000 7ff6fad9d640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff6fc1020c0 0x7ff6fc19ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:28:31.687 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:31.687+0000 7ff6fad9d640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff6fc1020c0 0x7ff6fc19ec10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:46102/0 (socket says 172.21.15.52:46102) 2024-03-04T09:28:31.687 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:31.687+0000 7ff6fad9d640 1 -- 172.21.15.52:0/2740048892 learned_addr learned my addr 172.21.15.52:0/2740048892 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-04T09:28:31.687 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:31.687+0000 7ff6fa59c640 1 --2- 172.21.15.52:0/2740048892 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff6fc100510 0x7ff6fc06a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:28:31.687 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:31.688+0000 7ff6fad9d640 1 -- 172.21.15.52:0/2740048892 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff6fc101700 msgr2=0x7ff6fc19c5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:28:31.688 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:31.687+0000 7ff6f9d9b640 1 --2- 172.21.15.52:0/2740048892 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff6fc101700 0x7ff6fc19c5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:28:31.688 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:31.688+0000 7ff6fad9d640 1 --2- 172.21.15.52:0/2740048892 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff6fc101700 0x7ff6fc19c5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:31.688 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:31.688+0000 7ff6fad9d640 1 -- 172.21.15.52:0/2740048892 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff6fc100510 msgr2=0x7ff6fc06a790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:28:31.689 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:31.688+0000 7ff6fad9d640 1 --2- 172.21.15.52:0/2740048892 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff6fc100510 0x7ff6fc06a790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:31.689 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:31.688+0000 7ff6fad9d640 1 -- 172.21.15.52:0/2740048892 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6fc19f4d0 con 0x7ff6fc1020c0 2024-03-04T09:28:31.689 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:31.688+0000 7ff6fad9d640 1 --2- 172.21.15.52:0/2740048892 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff6fc1020c0 0x7ff6fc19ec10 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7ff6ec02c630 tx=0x7ff6ec032ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:28:31.689 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:31.688+0000 7ff6f37fe640 1 -- 172.21.15.52:0/2740048892 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6ec041020 con 0x7ff6fc1020c0 2024-03-04T09:28:31.689 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:31.688+0000 7ff700bc7640 1 -- 172.21.15.52:0/2740048892 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6fc19f760 con 0x7ff6fc1020c0 2024-03-04T09:28:31.689 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:31.688+0000 7ff6f37fe640 1 -- 172.21.15.52:0/2740048892 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff6ec034cd0 con 0x7ff6fc1020c0 2024-03-04T09:28:31.689 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:31.688+0000 7ff6f37fe640 1 -- 172.21.15.52:0/2740048892 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6ec039040 con 0x7ff6fc1020c0 2024-03-04T09:28:31.690 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:31.688+0000 7ff700bc7640 1 -- 172.21.15.52:0/2740048892 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6fc19fca0 con 0x7ff6fc1020c0 2024-03-04T09:28:31.690 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:31.689+0000 7ff700bc7640 1 -- 172.21.15.52:0/2740048892 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff6fc107830 con 0x7ff6fc1020c0 2024-03-04T09:28:31.691 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:31.690+0000 7ff6f37fe640 1 -- 172.21.15.52:0/2740048892 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 16) v1 ==== 136756+0+0 (secure 0 0 0) 0x7ff6ec044020 con 0x7ff6fc1020c0 2024-03-04T09:28:31.691 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:31.691+0000 7ff6f37fe640 1 --2- 172.21.15.52:0/2740048892 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7ff6e80a1d80 0x7ff6e80a4240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:28:31.691 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:31.691+0000 7ff6fa59c640 1 --2- 172.21.15.52:0/2740048892 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7ff6e80a1d80 0x7ff6e80a4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:28:31.691 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:31.691+0000 7ff6f37fe640 1 -- 172.21.15.52:0/2740048892 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7ff6ec0efd30 con 0x7ff6fc1020c0 2024-03-04T09:28:31.691 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:31.691+0000 7ff6fa59c640 1 --2- 172.21.15.52:0/2740048892 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7ff6e80a1d80 0x7ff6e80a4240 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7ff6fc101560 tx=0x7ff6f4005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:28:31.691 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:31.694+0000 7ff6f37fe640 1 -- 172.21.15.52:0/2740048892 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7ff6ec04b020 con 0x7ff6fc1020c0 2024-03-04T09:28:31.910 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:31.913+0000 7ff700bc7640 1 -- 172.21.15.52:0/2740048892 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff6fc02cff0 con 0x7ff6fc1020c0 2024-03-04T09:28:31.911 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:31.914+0000 7ff6f37fe640 1 -- 172.21.15.52:0/2740048892 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) v1 ==== 94+0+9254 (secure 0 0 0) 0x7ff6ec0bd560 con 0x7ff6fc1020c0 2024-03-04T09:28:31.911 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-03-04T09:28:31.911 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":12,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6835/1007961019","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6834","nonce":1007961019},{"type":"v1","addr":"172.21.15.52:6835","nonce":1007961019}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6837/3753881283","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6836","nonce":3753881283},{"type":"v1","addr":"172.21.15.52:6837","nonce":3753881283}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14628,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6839/3467897808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6838","nonce":3467897808},{"type":"v1","addr":"172.21.15.52:6839","nonce":3467897808}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14640,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6841/2804292291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6840","nonce":2804292291},{"type":"v1","addr":"172.21.15.52:6841","nonce":2804292291}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24407,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6833/1818480323","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6832","nonce":1818480323},{"type":"v1","addr":"172.21.15.106:6833","nonce":1818480323}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24412,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6833/765768204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":765768204},{"type":"v1","addr":"172.21.15.191:6833","nonce":765768204}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24440,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6835/1541069897","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6834","nonce":1541069897},{"type":"v1","addr":"172.21.15.106:6835","nonce":1541069897}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24451,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6835/105048699","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":105048699},{"type":"v1","addr":"172.21.15.191:6835","nonce":105048699}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24463,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6837/4071064225","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6836","nonce":4071064225},{"type":"v1","addr":"172.21.15.191:6837","nonce":4071064225}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24473,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6837/2901755066","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6836","nonce":2901755066},{"type":"v1","addr":"172.21.15.106:6837","nonce":2901755066}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":24490,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6839/2194324891","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6838","nonce":2194324891},{"type":"v1","addr":"172.21.15.191:6839","nonce":2194324891}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10}],"filesystems":[]} 2024-03-04T09:28:31.911 INFO:teuthology.orchestra.run.smithi052.stderr:dumped fsmap epoch 12 2024-03-04T09:28:31.913 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:31.916+0000 7ff700bc7640 1 -- 172.21.15.52:0/2740048892 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7ff6e80a1d80 msgr2=0x7ff6e80a4240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:28:31.913 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:31.916+0000 7ff700bc7640 1 --2- 172.21.15.52:0/2740048892 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7ff6e80a1d80 0x7ff6e80a4240 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7ff6fc101560 tx=0x7ff6f4005f90 comp rx=0 tx=0).stop 2024-03-04T09:28:31.913 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:31.916+0000 7ff700bc7640 1 -- 172.21.15.52:0/2740048892 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff6fc1020c0 msgr2=0x7ff6fc19ec10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:28:31.913 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:31.916+0000 7ff700bc7640 1 --2- 172.21.15.52:0/2740048892 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff6fc1020c0 0x7ff6fc19ec10 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7ff6ec02c630 tx=0x7ff6ec032ea0 comp rx=0 tx=0).stop 2024-03-04T09:28:31.913 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:31.916+0000 7ff700bc7640 1 -- 172.21.15.52:0/2740048892 shutdown_connections 2024-03-04T09:28:31.913 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:31.916+0000 7ff700bc7640 1 --2- 172.21.15.52:0/2740048892 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7ff6e80a1d80 0x7ff6e80a4240 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:31.913 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:31.916+0000 7ff700bc7640 1 --2- 172.21.15.52:0/2740048892 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff6fc1020c0 0x7ff6fc19ec10 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:31.913 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:31.916+0000 7ff700bc7640 1 --2- 172.21.15.52:0/2740048892 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff6fc101700 0x7ff6fc19c5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:31.913 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:31.916+0000 7ff700bc7640 1 --2- 172.21.15.52:0/2740048892 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff6fc100510 0x7ff6fc06a790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:31.913 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:31.916+0000 7ff700bc7640 1 -- 172.21.15.52:0/2740048892 >> 172.21.15.52:0/2740048892 conn(0x7ff6fc0fba80 msgr2=0x7ff6fc104a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:28:31.914 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:31.916+0000 7ff700bc7640 1 -- 172.21.15.52:0/2740048892 shutdown_connections 2024-03-04T09:28:31.914 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:31.916+0000 7ff700bc7640 1 -- 172.21.15.52:0/2740048892 wait complete. 2024-03-04T09:28:32.332 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:28:32 smithi106 ceph-mon[36484]: Deploying daemon mds.b on smithi106 2024-03-04T09:28:32.332 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:28:32 smithi106 ceph-mon[36484]: mds.? [v2:172.21.15.52:6840/2804292291,v1:172.21.15.52:6841/2804292291] up:boot 2024-03-04T09:28:32.332 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:28:32 smithi106 ceph-mon[36484]: fsmap 11 up:standby 2024-03-04T09:28:32.332 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:28:32 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-03-04T09:28:32.333 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:28:32 smithi106 ceph-mon[36484]: from='client.? 172.21.15.52:0/2740048892' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-04T09:28:32.585 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:28:32 smithi052 ceph-mon[32473]: Deploying daemon mds.b on smithi106 2024-03-04T09:28:32.585 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:28:32 smithi052 ceph-mon[32473]: mds.? [v2:172.21.15.52:6840/2804292291,v1:172.21.15.52:6841/2804292291] up:boot 2024-03-04T09:28:32.585 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:28:32 smithi052 ceph-mon[32473]: fsmap 11 up:standby 2024-03-04T09:28:32.585 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:28:32 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-03-04T09:28:32.585 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:28:32 smithi052 ceph-mon[32473]: from='client.? 172.21.15.52:0/2740048892' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-04T09:28:32.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:28:32 smithi191 ceph-mon[36816]: Deploying daemon mds.b on smithi106 2024-03-04T09:28:32.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:28:32 smithi191 ceph-mon[36816]: mds.? [v2:172.21.15.52:6840/2804292291,v1:172.21.15.52:6841/2804292291] up:boot 2024-03-04T09:28:32.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:28:32 smithi191 ceph-mon[36816]: fsmap 11 up:standby 2024-03-04T09:28:32.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:28:32 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-03-04T09:28:32.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:28:32 smithi191 ceph-mon[36816]: from='client.? 172.21.15.52:0/2740048892' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-04T09:28:33.381 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:28:33 smithi106 ceph-mon[36484]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-04T09:28:33.584 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:28:33 smithi052 ceph-mon[32473]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-04T09:28:33.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:28:33 smithi191 ceph-mon[36816]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-04T09:28:33.655 INFO:journalctl@ceph.mds.b.smithi106.stdout:Mar 04 09:28:33 smithi106 podman[66640]: 2024-03-04 09:28:33.636696948 +0000 UTC m=+0.350095467 container start a2424e9e9412a364f435c365e29cd1799fd06f352a296a66de3b493add01da77 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5, name=ceph-4e059f08-da08-11ee-95c6-87774f69a715-mds-b, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, org.label-schema.build-date=20240222, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , ceph=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, RELEASE=wip-lusov-qdb-1bc250c) 2024-03-04T09:28:33.925 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T09:28:34.037 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:34.038+0000 7f621c5ac640 1 -- 172.21.15.52:0/2820318596 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f62141028b0 msgr2=0x7f6214102d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:28:34.037 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:34.038+0000 7f621c5ac640 1 --2- 172.21.15.52:0/2820318596 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f62141028b0 0x7f6214102d30 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f6204009930 tx=0x7f620402bc30 comp rx=0 tx=0).stop 2024-03-04T09:28:34.037 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:34.038+0000 7f621c5ac640 1 -- 172.21.15.52:0/2820318596 shutdown_connections 2024-03-04T09:28:34.038 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:34.038+0000 7f621c5ac640 1 --2- 172.21.15.52:0/2820318596 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6214103270 0x7f6214109b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:34.038 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:34.038+0000 7f621c5ac640 1 --2- 172.21.15.52:0/2820318596 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f62141028b0 0x7f6214102d30 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:34.038 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:34.038+0000 7f621c5ac640 1 --2- 172.21.15.52:0/2820318596 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f62140ff8e0 0x7f62140ffd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:34.038 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:34.038+0000 7f621c5ac640 1 -- 172.21.15.52:0/2820318596 >> 172.21.15.52:0/2820318596 conn(0x7f62140fb210 msgr2=0x7f62140fd650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:28:34.038 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:34.038+0000 7f621c5ac640 1 -- 172.21.15.52:0/2820318596 shutdown_connections 2024-03-04T09:28:34.038 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:34.038+0000 7f621c5ac640 1 -- 172.21.15.52:0/2820318596 wait complete. 2024-03-04T09:28:34.038 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:34.039+0000 7f621c5ac640 1 Processor -- start 2024-03-04T09:28:34.038 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:34.039+0000 7f621c5ac640 1 -- start start 2024-03-04T09:28:34.038 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:34.039+0000 7f621c5ac640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f62140ff8e0 0x7f6214199b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:28:34.038 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:34.039+0000 7f621c5ac640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f62141028b0 0x7f621419a0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:28:34.038 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:34.039+0000 7f621c5ac640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6214103270 0x7f62141a1110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:28:34.038 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:34.039+0000 7f621c5ac640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f621410d2a0 con 0x7f6214103270 2024-03-04T09:28:34.038 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:34.039+0000 7f621c5ac640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f621410d120 con 0x7f62140ff8e0 2024-03-04T09:28:34.039 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:34.039+0000 7f621c5ac640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f621410d420 con 0x7f62141028b0 2024-03-04T09:28:34.039 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:34.039+0000 7f621ab49640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6214103270 0x7f62141a1110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:28:34.039 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:34.039+0000 7f621ab49640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6214103270 0x7f62141a1110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:46122/0 (socket says 172.21.15.52:46122) 2024-03-04T09:28:34.039 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:34.039+0000 7f621ab49640 1 -- 172.21.15.52:0/4057867528 learned_addr learned my addr 172.21.15.52:0/4057867528 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-04T09:28:34.039 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:34.039+0000 7f621a348640 1 --2- 172.21.15.52:0/4057867528 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f62140ff8e0 0x7f6214199b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:28:34.039 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:34.039+0000 7f621ab49640 1 -- 172.21.15.52:0/4057867528 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f62141028b0 msgr2=0x7f621419a0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:28:34.039 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:34.039+0000 7f6219b47640 1 --2- 172.21.15.52:0/4057867528 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f62141028b0 0x7f621419a0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:28:34.039 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:34.039+0000 7f621ab49640 1 --2- 172.21.15.52:0/4057867528 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f62141028b0 0x7f621419a0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:34.039 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:34.039+0000 7f621ab49640 1 -- 172.21.15.52:0/4057867528 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f62140ff8e0 msgr2=0x7f6214199b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:28:34.039 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:34.039+0000 7f621ab49640 1 --2- 172.21.15.52:0/4057867528 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f62140ff8e0 0x7f6214199b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:34.039 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:34.039+0000 7f621ab49640 1 -- 172.21.15.52:0/4057867528 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62141a1780 con 0x7f6214103270 2024-03-04T09:28:34.039 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:34.040+0000 7f621ab49640 1 --2- 172.21.15.52:0/4057867528 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6214103270 0x7f62141a1110 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f620c00eb10 tx=0x7f620c00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:28:34.039 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:34.040+0000 7f620b7fe640 1 -- 172.21.15.52:0/4057867528 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f620c004b20 con 0x7f6214103270 2024-03-04T09:28:34.039 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:34.040+0000 7f620b7fe640 1 -- 172.21.15.52:0/4057867528 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f620c004cc0 con 0x7f6214103270 2024-03-04T09:28:34.040 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:34.040+0000 7f621c5ac640 1 -- 172.21.15.52:0/4057867528 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62141a1a70 con 0x7f6214103270 2024-03-04T09:28:34.040 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:34.040+0000 7f620b7fe640 1 -- 172.21.15.52:0/4057867528 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f620c010760 con 0x7f6214103270 2024-03-04T09:28:34.040 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:34.040+0000 7f621c5ac640 1 -- 172.21.15.52:0/4057867528 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62141a1f70 con 0x7f6214103270 2024-03-04T09:28:34.041 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:34.041+0000 7f621c5ac640 1 -- 172.21.15.52:0/4057867528 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f62141a2260 con 0x7f6214103270 2024-03-04T09:28:34.041 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:34.042+0000 7f620b7fe640 1 -- 172.21.15.52:0/4057867528 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 16) v1 ==== 136756+0+0 (secure 0 0 0) 0x7f620c005020 con 0x7f6214103270 2024-03-04T09:28:34.041 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:34.043+0000 7f620b7fe640 1 --2- 172.21.15.52:0/4057867528 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f62000a1d80 0x7f62000a4240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:28:34.041 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:34.043+0000 7f621a348640 1 --2- 172.21.15.52:0/4057867528 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f62000a1d80 0x7f62000a4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:28:34.041 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:34.043+0000 7f620b7fe640 1 -- 172.21.15.52:0/4057867528 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f620c0cdf60 con 0x7f6214103270 2024-03-04T09:28:34.041 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:34.043+0000 7f621a348640 1 --2- 172.21.15.52:0/4057867528 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f62000a1d80 0x7f62000a4240 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f6214069a20 tx=0x7f6210005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:28:34.044 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:34.048+0000 7f620b7fe640 1 -- 172.21.15.52:0/4057867528 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f620c09b830 con 0x7f6214103270 2024-03-04T09:28:34.131 INFO:journalctl@ceph.mds.b.smithi106.stdout:Mar 04 09:28:33 smithi106 ceph-mds[66659]: -- 172.21.15.106:0/2709800020 <== mon.2 v2:172.21.15.106:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x557e6c721a40 con 0x557e6c6f4800 2024-03-04T09:28:34.131 INFO:journalctl@ceph.mds.b.smithi106.stdout:Mar 04 09:28:33 smithi106 ceph-4e059f08-da08-11ee-95c6-87774f69a715-mds-b[66655]: starting mds.b at 2024-03-04T09:28:34.131 INFO:journalctl@ceph.mds.b.smithi106.stdout:Mar 04 09:28:33 smithi106 bash[66640]: a2424e9e9412a364f435c365e29cd1799fd06f352a296a66de3b493add01da77 2024-03-04T09:28:34.131 INFO:journalctl@ceph.mds.b.smithi106.stdout:Mar 04 09:28:33 smithi106 systemd[1]: Started Ceph mds.b for 4e059f08-da08-11ee-95c6-87774f69a715. 2024-03-04T09:28:34.268 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:34.271+0000 7f621c5ac640 1 -- 172.21.15.52:0/4057867528 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f62141a2430 con 0x7f6214103270 2024-03-04T09:28:34.269 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:34.272+0000 7f620b7fe640 1 -- 172.21.15.52:0/4057867528 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) v1 ==== 94+0+10049 (secure 0 0 0) 0x7f620c09b130 con 0x7f6214103270 2024-03-04T09:28:34.269 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-03-04T09:28:34.270 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":13,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6835/1007961019","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6834","nonce":1007961019},{"type":"v1","addr":"172.21.15.52:6835","nonce":1007961019}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6837/3753881283","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6836","nonce":3753881283},{"type":"v1","addr":"172.21.15.52:6837","nonce":3753881283}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14628,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6839/3467897808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6838","nonce":3467897808},{"type":"v1","addr":"172.21.15.52:6839","nonce":3467897808}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14640,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6841/2804292291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6840","nonce":2804292291},{"type":"v1","addr":"172.21.15.52:6841","nonce":2804292291}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24407,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6833/1818480323","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6832","nonce":1818480323},{"type":"v1","addr":"172.21.15.106:6833","nonce":1818480323}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24412,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6833/765768204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":765768204},{"type":"v1","addr":"172.21.15.191:6833","nonce":765768204}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24440,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6835/1541069897","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6834","nonce":1541069897},{"type":"v1","addr":"172.21.15.106:6835","nonce":1541069897}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24451,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6835/105048699","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":105048699},{"type":"v1","addr":"172.21.15.191:6835","nonce":105048699}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24463,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6837/4071064225","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6836","nonce":4071064225},{"type":"v1","addr":"172.21.15.191:6837","nonce":4071064225}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24473,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6837/2901755066","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6836","nonce":2901755066},{"type":"v1","addr":"172.21.15.106:6837","nonce":2901755066}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":24490,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6839/2194324891","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6838","nonce":2194324891},{"type":"v1","addr":"172.21.15.191:6839","nonce":2194324891}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10},{"gid":24506,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6839/1995853093","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6838","nonce":1995853093},{"type":"v1","addr":"172.21.15.106:6839","nonce":1995853093}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13}],"filesystems":[]} 2024-03-04T09:28:34.270 INFO:teuthology.orchestra.run.smithi052.stderr:dumped fsmap epoch 13 2024-03-04T09:28:34.271 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:34.274+0000 7f621c5ac640 1 -- 172.21.15.52:0/4057867528 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f62000a1d80 msgr2=0x7f62000a4240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:28:34.272 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:34.274+0000 7f621c5ac640 1 --2- 172.21.15.52:0/4057867528 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f62000a1d80 0x7f62000a4240 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f6214069a20 tx=0x7f6210005f90 comp rx=0 tx=0).stop 2024-03-04T09:28:34.272 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:34.274+0000 7f621c5ac640 1 -- 172.21.15.52:0/4057867528 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6214103270 msgr2=0x7f62141a1110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:28:34.272 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:34.274+0000 7f621c5ac640 1 --2- 172.21.15.52:0/4057867528 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6214103270 0x7f62141a1110 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f620c00eb10 tx=0x7f620c00c2d0 comp rx=0 tx=0).stop 2024-03-04T09:28:34.272 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:34.275+0000 7f621c5ac640 1 -- 172.21.15.52:0/4057867528 shutdown_connections 2024-03-04T09:28:34.272 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:34.275+0000 7f621c5ac640 1 --2- 172.21.15.52:0/4057867528 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f62000a1d80 0x7f62000a4240 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:34.272 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:34.275+0000 7f621c5ac640 1 --2- 172.21.15.52:0/4057867528 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6214103270 0x7f62141a1110 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:34.272 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:34.275+0000 7f621c5ac640 1 --2- 172.21.15.52:0/4057867528 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f62141028b0 0x7f621419a0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:34.272 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:34.275+0000 7f621c5ac640 1 --2- 172.21.15.52:0/4057867528 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f62140ff8e0 0x7f6214199b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:34.272 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:34.275+0000 7f621c5ac640 1 -- 172.21.15.52:0/4057867528 >> 172.21.15.52:0/4057867528 conn(0x7f62140fb210 msgr2=0x7f62141006a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:28:34.272 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:34.275+0000 7f621c5ac640 1 -- 172.21.15.52:0/4057867528 shutdown_connections 2024-03-04T09:28:34.272 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:34.275+0000 7f621c5ac640 1 -- 172.21.15.52:0/4057867528 wait complete. 2024-03-04T09:28:34.283 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-04T09:28:34.389 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:34.390+0000 7f4762cfb640 1 -- 172.21.15.52:0/2203957452 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f475c1020a0 msgr2=0x7f475c108930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:28:34.389 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:34.390+0000 7f4762cfb640 1 --2- 172.21.15.52:0/2203957452 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f475c1020a0 0x7f475c108930 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f4754009ca0 tx=0x7f475402c0f0 comp rx=0 tx=0).stop 2024-03-04T09:28:34.389 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:34.390+0000 7f4762cfb640 1 -- 172.21.15.52:0/2203957452 shutdown_connections 2024-03-04T09:28:34.389 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:34.390+0000 7f4762cfb640 1 --2- 172.21.15.52:0/2203957452 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f475c1020a0 0x7f475c108930 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:34.389 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:34.390+0000 7f4762cfb640 1 --2- 172.21.15.52:0/2203957452 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f475c1016e0 0x7f475c101b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:34.389 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:34.390+0000 7f4762cfb640 1 --2- 172.21.15.52:0/2203957452 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f475c1004f0 0x7f475c100910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:34.389 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:34.390+0000 7f4762cfb640 1 -- 172.21.15.52:0/2203957452 >> 172.21.15.52:0/2203957452 conn(0x7f475c0fba60 msgr2=0x7f475c0fdea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:28:34.389 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:34.390+0000 7f4762cfb640 1 -- 172.21.15.52:0/2203957452 shutdown_connections 2024-03-04T09:28:34.389 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:34.390+0000 7f4762cfb640 1 -- 172.21.15.52:0/2203957452 wait complete. 2024-03-04T09:28:34.389 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:34.390+0000 7f4762cfb640 1 Processor -- start 2024-03-04T09:28:34.390 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:34.391+0000 7f4762cfb640 1 -- start start 2024-03-04T09:28:34.390 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:34.391+0000 7f4762cfb640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f475c1004f0 0x7f475c1978f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:28:34.390 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:34.391+0000 7f4762cfb640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f475c1016e0 0x7f475c197e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:28:34.390 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:34.391+0000 7f4762cfb640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f475c1020a0 0x7f475c19eea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:28:34.390 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:34.391+0000 7f4762cfb640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f475c10c180 con 0x7f475c1004f0 2024-03-04T09:28:34.390 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:34.391+0000 7f4762cfb640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f475c10c000 con 0x7f475c1016e0 2024-03-04T09:28:34.390 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:34.391+0000 7f4762cfb640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f475c10c300 con 0x7f475c1020a0 2024-03-04T09:28:34.390 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:34.391+0000 7f4760a97640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f475c1004f0 0x7f475c1978f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:28:34.390 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:34.391+0000 7f4760a97640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f475c1004f0 0x7f475c1978f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:57680/0 (socket says 172.21.15.52:57680) 2024-03-04T09:28:34.390 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:34.391+0000 7f4760a97640 1 -- 172.21.15.52:0/3388217884 learned_addr learned my addr 172.21.15.52:0/3388217884 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-04T09:28:34.390 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:34.391+0000 7f4761298640 1 --2- 172.21.15.52:0/3388217884 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f475c1020a0 0x7f475c19eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:28:34.390 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:34.391+0000 7f475bfff640 1 --2- 172.21.15.52:0/3388217884 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f475c1016e0 0x7f475c197e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:28:34.390 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:34.391+0000 7f4760a97640 1 -- 172.21.15.52:0/3388217884 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f475c1020a0 msgr2=0x7f475c19eea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:28:34.390 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:34.391+0000 7f4760a97640 1 --2- 172.21.15.52:0/3388217884 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f475c1020a0 0x7f475c19eea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:34.391 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:34.391+0000 7f4760a97640 1 -- 172.21.15.52:0/3388217884 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f475c1016e0 msgr2=0x7f475c197e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:28:34.391 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:34.391+0000 7f4760a97640 1 --2- 172.21.15.52:0/3388217884 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f475c1016e0 0x7f475c197e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:34.391 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:34.391+0000 7f4760a97640 1 -- 172.21.15.52:0/3388217884 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f475c19f510 con 0x7f475c1004f0 2024-03-04T09:28:34.391 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:34.391+0000 7f4760a97640 1 --2- 172.21.15.52:0/3388217884 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f475c1004f0 0x7f475c1978f0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f475000e9c0 tx=0x7f475000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:28:34.391 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:34.391+0000 7f4759ffb640 1 -- 172.21.15.52:0/3388217884 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f475000cde0 con 0x7f475c1004f0 2024-03-04T09:28:34.391 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:34.391+0000 7f4762cfb640 1 -- 172.21.15.52:0/3388217884 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f475c19f800 con 0x7f475c1004f0 2024-03-04T09:28:34.391 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:34.391+0000 7f4759ffb640 1 -- 172.21.15.52:0/3388217884 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4750010910 con 0x7f475c1004f0 2024-03-04T09:28:34.391 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:34.391+0000 7f4759ffb640 1 -- 172.21.15.52:0/3388217884 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4750018a10 con 0x7f475c1004f0 2024-03-04T09:28:34.391 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:34.392+0000 7f4762cfb640 1 -- 172.21.15.52:0/3388217884 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f475c19fda0 con 0x7f475c1004f0 2024-03-04T09:28:34.392 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:34.393+0000 7f4762cfb640 1 -- 172.21.15.52:0/3388217884 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f475c19ffa0 con 0x7f475c1004f0 2024-03-04T09:28:34.392 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:34.395+0000 7f4759ffb640 1 -- 172.21.15.52:0/3388217884 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 16) v1 ==== 136756+0+0 (secure 0 0 0) 0x7f475001f020 con 0x7f475c1004f0 2024-03-04T09:28:34.392 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:34.395+0000 7f4759ffb640 1 --2- 172.21.15.52:0/3388217884 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f474c0a1d80 0x7f474c0a4240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:28:34.392 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:34.395+0000 7f4759ffb640 1 -- 172.21.15.52:0/3388217884 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f47500ce0c0 con 0x7f475c1004f0 2024-03-04T09:28:34.392 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:34.395+0000 7f475bfff640 1 --2- 172.21.15.52:0/3388217884 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f474c0a1d80 0x7f474c0a4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:28:34.393 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:34.396+0000 7f475bfff640 1 --2- 172.21.15.52:0/3388217884 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f474c0a1d80 0x7f474c0a4240 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f475c198e00 tx=0x7f4748005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:28:34.394 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:34.398+0000 7f4759ffb640 1 -- 172.21.15.52:0/3388217884 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f475009b990 con 0x7f475c1004f0 2024-03-04T09:28:34.588 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:34.591+0000 7f4762cfb640 1 -- 172.21.15.52:0/3388217884 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f475c1a0330 con 0x7f475c1004f0 2024-03-04T09:28:34.590 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:34.593+0000 7f4759ffb640 1 -- 172.21.15.52:0/3388217884 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) v1 ==== 74+0+18492 (secure 0 0 0) 0x7f475009b290 con 0x7f475c1004f0 2024-03-04T09:28:34.590 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-03-04T09:28:34.590 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":81,"fsid":"4e059f08-da08-11ee-95c6-87774f69a715","created":"2024-03-04T09:20:46.104963+0000","modified":"2024-03-04T09:27:50.268367+0000","last_up_change":"2024-03-04T09:27:49.261365+0000","last_in_change":"2024-03-04T09:27:30.024964+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-04T09:24:01.742461+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"583172be-602f-4bf4-bfcb-42663cf5b277","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6802","nonce":1371221397},{"type":"v1","addr":"172.21.15.52:6803","nonce":1371221397}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6804","nonce":1371221397},{"type":"v1","addr":"172.21.15.52:6805","nonce":1371221397}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6808","nonce":1371221397},{"type":"v1","addr":"172.21.15.52:6809","nonce":1371221397}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6806","nonce":1371221397},{"type":"v1","addr":"172.21.15.52:6807","nonce":1371221397}]},"public_addr":"172.21.15.52:6803/1371221397","cluster_addr":"172.21.15.52:6805/1371221397","heartbeat_back_addr":"172.21.15.52:6809/1371221397","heartbeat_front_addr":"172.21.15.52:6807/1371221397","state":["exists","up"]},{"osd":1,"uuid":"5ae0081c-a2f6-46eb-ace0-0368208377b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":2540865537},{"type":"v1","addr":"172.21.15.106:6801","nonce":2540865537}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":2540865537},{"type":"v1","addr":"172.21.15.106:6803","nonce":2540865537}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":2540865537},{"type":"v1","addr":"172.21.15.106:6807","nonce":2540865537}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":2540865537},{"type":"v1","addr":"172.21.15.106:6805","nonce":2540865537}]},"public_addr":"172.21.15.106:6801/2540865537","cluster_addr":"172.21.15.106:6803/2540865537","heartbeat_back_addr":"172.21.15.106:6807/2540865537","heartbeat_front_addr":"172.21.15.106:6805/2540865537","state":["exists","up"]},{"osd":2,"uuid":"1948bac7-2bb2-4218-8b93-4e4743a0de04","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":1377083164},{"type":"v1","addr":"172.21.15.191:6801","nonce":1377083164}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":1377083164},{"type":"v1","addr":"172.21.15.191:6803","nonce":1377083164}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":1377083164},{"type":"v1","addr":"172.21.15.191:6807","nonce":1377083164}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":1377083164},{"type":"v1","addr":"172.21.15.191:6805","nonce":1377083164}]},"public_addr":"172.21.15.191:6801/1377083164","cluster_addr":"172.21.15.191:6803/1377083164","heartbeat_back_addr":"172.21.15.191:6807/1377083164","heartbeat_front_addr":"172.21.15.191:6805/1377083164","state":["exists","up"]},{"osd":3,"uuid":"16c9e9fe-7167-4992-bf12-1a7398e0498a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6810","nonce":2388571810},{"type":"v1","addr":"172.21.15.52:6811","nonce":2388571810}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6812","nonce":2388571810},{"type":"v1","addr":"172.21.15.52:6813","nonce":2388571810}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6816","nonce":2388571810},{"type":"v1","addr":"172.21.15.52:6817","nonce":2388571810}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6814","nonce":2388571810},{"type":"v1","addr":"172.21.15.52:6815","nonce":2388571810}]},"public_addr":"172.21.15.52:6811/2388571810","cluster_addr":"172.21.15.52:6813/2388571810","heartbeat_back_addr":"172.21.15.52:6817/2388571810","heartbeat_front_addr":"172.21.15.52:6815/2388571810","state":["exists","up"]},{"osd":4,"uuid":"c8ef48c7-7c78-44d3-a9e5-99ec750abd3d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6808","nonce":3465868015},{"type":"v1","addr":"172.21.15.106:6809","nonce":3465868015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6810","nonce":3465868015},{"type":"v1","addr":"172.21.15.106:6811","nonce":3465868015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6814","nonce":3465868015},{"type":"v1","addr":"172.21.15.106:6815","nonce":3465868015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6812","nonce":3465868015},{"type":"v1","addr":"172.21.15.106:6813","nonce":3465868015}]},"public_addr":"172.21.15.106:6809/3465868015","cluster_addr":"172.21.15.106:6811/3465868015","heartbeat_back_addr":"172.21.15.106:6815/3465868015","heartbeat_front_addr":"172.21.15.106:6813/3465868015","state":["exists","up"]},{"osd":5,"uuid":"9e41824a-c5a2-4f88-be50-2ba1dceb3220","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":1485092643},{"type":"v1","addr":"172.21.15.191:6809","nonce":1485092643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":1485092643},{"type":"v1","addr":"172.21.15.191:6811","nonce":1485092643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":1485092643},{"type":"v1","addr":"172.21.15.191:6815","nonce":1485092643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":1485092643},{"type":"v1","addr":"172.21.15.191:6813","nonce":1485092643}]},"public_addr":"172.21.15.191:6809/1485092643","cluster_addr":"172.21.15.191:6811/1485092643","heartbeat_back_addr":"172.21.15.191:6815/1485092643","heartbeat_front_addr":"172.21.15.191:6813/1485092643","state":["exists","up"]},{"osd":6,"uuid":"b35dccee-ec16-452e-ac69-3ac2fb7b27fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6818","nonce":1989797856},{"type":"v1","addr":"172.21.15.52:6819","nonce":1989797856}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6820","nonce":1989797856},{"type":"v1","addr":"172.21.15.52:6821","nonce":1989797856}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6824","nonce":1989797856},{"type":"v1","addr":"172.21.15.52:6825","nonce":1989797856}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6822","nonce":1989797856},{"type":"v1","addr":"172.21.15.52:6823","nonce":1989797856}]},"public_addr":"172.21.15.52:6819/1989797856","cluster_addr":"172.21.15.52:6821/1989797856","heartbeat_back_addr":"172.21.15.52:6825/1989797856","heartbeat_front_addr":"172.21.15.52:6823/1989797856","state":["exists","up"]},{"osd":7,"uuid":"71303e3d-14ec-4cbd-b19c-67645a1de3c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6816","nonce":1450870329},{"type":"v1","addr":"172.21.15.106:6817","nonce":1450870329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":1450870329},{"type":"v1","addr":"172.21.15.106:6819","nonce":1450870329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":1450870329},{"type":"v1","addr":"172.21.15.106:6823","nonce":1450870329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6820","nonce":1450870329},{"type":"v1","addr":"172.21.15.106:6821","nonce":1450870329}]},"public_addr":"172.21.15.106:6817/1450870329","cluster_addr":"172.21.15.106:6819/1450870329","heartbeat_back_addr":"172.21.15.106:6823/1450870329","heartbeat_front_addr":"172.21.15.106:6821/1450870329","state":["exists","up"]},{"osd":8,"uuid":"6f5786c4-4402-435a-bdad-adff46fe9473","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":1114119072},{"type":"v1","addr":"172.21.15.191:6817","nonce":1114119072}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6818","nonce":1114119072},{"type":"v1","addr":"172.21.15.191:6819","nonce":1114119072}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":1114119072},{"type":"v1","addr":"172.21.15.191:6823","nonce":1114119072}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":1114119072},{"type":"v1","addr":"172.21.15.191:6821","nonce":1114119072}]},"public_addr":"172.21.15.191:6817/1114119072","cluster_addr":"172.21.15.191:6819/1114119072","heartbeat_back_addr":"172.21.15.191:6823/1114119072","heartbeat_front_addr":"172.21.15.191:6821/1114119072","state":["exists","up"]},{"osd":9,"uuid":"062bbaa5-1227-4684-b859-450f5594a90c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6826","nonce":3485821914},{"type":"v1","addr":"172.21.15.52:6827","nonce":3485821914}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6828","nonce":3485821914},{"type":"v1","addr":"172.21.15.52:6829","nonce":3485821914}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6832","nonce":3485821914},{"type":"v1","addr":"172.21.15.52:6833","nonce":3485821914}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6830","nonce":3485821914},{"type":"v1","addr":"172.21.15.52:6831","nonce":3485821914}]},"public_addr":"172.21.15.52:6827/3485821914","cluster_addr":"172.21.15.52:6829/3485821914","heartbeat_back_addr":"172.21.15.52:6833/3485821914","heartbeat_front_addr":"172.21.15.52:6831/3485821914","state":["exists","up"]},{"osd":10,"uuid":"1a285a5a-f793-48da-a8eb-5326d9528cd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":3815586576},{"type":"v1","addr":"172.21.15.106:6825","nonce":3815586576}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6826","nonce":3815586576},{"type":"v1","addr":"172.21.15.106:6827","nonce":3815586576}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":3815586576},{"type":"v1","addr":"172.21.15.106:6831","nonce":3815586576}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":3815586576},{"type":"v1","addr":"172.21.15.106:6829","nonce":3815586576}]},"public_addr":"172.21.15.106:6825/3815586576","cluster_addr":"172.21.15.106:6827/3815586576","heartbeat_back_addr":"172.21.15.106:6831/3815586576","heartbeat_front_addr":"172.21.15.106:6829/3815586576","state":["exists","up"]},{"osd":11,"uuid":"1f8b9a43-57fa-41f4-9969-3b803a74bf45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":2009632843},{"type":"v1","addr":"172.21.15.191:6825","nonce":2009632843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":2009632843},{"type":"v1","addr":"172.21.15.191:6827","nonce":2009632843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":2009632843},{"type":"v1","addr":"172.21.15.191:6831","nonce":2009632843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":2009632843},{"type":"v1","addr":"172.21.15.191:6829","nonce":2009632843}]},"public_addr":"172.21.15.191:6825/2009632843","cluster_addr":"172.21.15.191:6827/2009632843","heartbeat_back_addr":"172.21.15.191:6831/2009632843","heartbeat_front_addr":"172.21.15.191:6829/2009632843","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T09:23:10.894763+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T09:23:34.420055+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T09:23:58.288324+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T09:24:23.673457+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T09:24:48.208561+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T09:25:13.405473+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T09:25:38.670915+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T09:26:04.451257+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T09:26:30.281380+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T09:26:56.543143+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T09:27:22.412930+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T09:27:48.003866+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.52:0/4082645718":"2024-03-05T09:21:39.401935+0000","172.21.15.52:0/1271949988":"2024-03-05T09:21:39.401935+0000","172.21.15.52:6801/2379361333":"2024-03-05T09:21:12.760938+0000","172.21.15.52:0/497075075":"2024-03-05T09:21:12.760938+0000","172.21.15.52:6800/2379361333":"2024-03-05T09:21:12.760938+0000","172.21.15.52:0/568242577":"2024-03-05T09:21:12.760938+0000","172.21.15.52:6801/2425730636":"2024-03-05T09:21:39.401935+0000","172.21.15.52:6800/2425730636":"2024-03-05T09:21:39.401935+0000","172.21.15.52:0/2743084243":"2024-03-05T09:21:39.401935+0000","172.21.15.52:0/1035878034":"2024-03-05T09:21:12.760938+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-04T09:28:34.592 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:34.595+0000 7f4762cfb640 1 -- 172.21.15.52:0/3388217884 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f474c0a1d80 msgr2=0x7f474c0a4240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:28:34.592 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:34.595+0000 7f4762cfb640 1 --2- 172.21.15.52:0/3388217884 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f474c0a1d80 0x7f474c0a4240 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f475c198e00 tx=0x7f4748005f90 comp rx=0 tx=0).stop 2024-03-04T09:28:34.592 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:34.595+0000 7f4762cfb640 1 -- 172.21.15.52:0/3388217884 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f475c1004f0 msgr2=0x7f475c1978f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:28:34.593 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:34.595+0000 7f4762cfb640 1 --2- 172.21.15.52:0/3388217884 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f475c1004f0 0x7f475c1978f0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f475000e9c0 tx=0x7f475000ee80 comp rx=0 tx=0).stop 2024-03-04T09:28:34.593 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:34.595+0000 7f4762cfb640 1 -- 172.21.15.52:0/3388217884 shutdown_connections 2024-03-04T09:28:34.593 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:34.595+0000 7f4762cfb640 1 --2- 172.21.15.52:0/3388217884 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f474c0a1d80 0x7f474c0a4240 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:34.593 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:34.595+0000 7f4762cfb640 1 --2- 172.21.15.52:0/3388217884 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f475c1020a0 0x7f475c19eea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:34.593 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:34.595+0000 7f4762cfb640 1 --2- 172.21.15.52:0/3388217884 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f475c1016e0 0x7f475c197e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:34.593 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:34.595+0000 7f4762cfb640 1 --2- 172.21.15.52:0/3388217884 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f475c1004f0 0x7f475c1978f0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:34.593 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:34.595+0000 7f4762cfb640 1 -- 172.21.15.52:0/3388217884 >> 172.21.15.52:0/3388217884 conn(0x7f475c0fba60 msgr2=0x7f475c0fd4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:28:34.593 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:34.595+0000 7f4762cfb640 1 -- 172.21.15.52:0/3388217884 shutdown_connections 2024-03-04T09:28:34.593 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:34.595+0000 7f4762cfb640 1 -- 172.21.15.52:0/3388217884 wait complete. 2024-03-04T09:28:34.605 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-04T09:24:01.742461+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-04T09:28:34.605 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-04T09:28:34.718 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:34.718+0000 7fad8ea0e640 1 -- 172.21.15.52:0/3171415483 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fad881027f0 msgr2=0x7fad88102c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:28:34.718 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:34.718+0000 7fad8ea0e640 1 --2- 172.21.15.52:0/3171415483 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fad881027f0 0x7fad88102c70 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fad7c009900 tx=0x7fad7c02bc30 comp rx=0 tx=0).stop 2024-03-04T09:28:34.719 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:34.718+0000 7fad8ea0e640 1 -- 172.21.15.52:0/3171415483 shutdown_connections 2024-03-04T09:28:34.719 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:34.718+0000 7fad8ea0e640 1 --2- 172.21.15.52:0/3171415483 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fad881031b0 0x7fad88109a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:34.719 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:34.718+0000 7fad8ea0e640 1 --2- 172.21.15.52:0/3171415483 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fad881027f0 0x7fad88102c70 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:34.719 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:34.718+0000 7fad8ea0e640 1 --2- 172.21.15.52:0/3171415483 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fad88101600 0x7fad88101a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:34.719 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:34.718+0000 7fad8ea0e640 1 -- 172.21.15.52:0/3171415483 >> 172.21.15.52:0/3171415483 conn(0x7fad880fcb70 msgr2=0x7fad880fefb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:28:34.719 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:34.718+0000 7fad8ea0e640 1 -- 172.21.15.52:0/3171415483 shutdown_connections 2024-03-04T09:28:34.719 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:34.718+0000 7fad8ea0e640 1 -- 172.21.15.52:0/3171415483 wait complete. 2024-03-04T09:28:34.719 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:34.719+0000 7fad8ea0e640 1 Processor -- start 2024-03-04T09:28:34.719 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:34.719+0000 7fad8ea0e640 1 -- start start 2024-03-04T09:28:34.719 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:34.719+0000 7fad8ea0e640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fad88101600 0x7fad88197930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:28:34.719 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:34.719+0000 7fad8ea0e640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fad881027f0 0x7fad88197e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:28:34.720 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:34.719+0000 7fad8ea0e640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fad881031b0 0x7fad8819eee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:28:34.720 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:34.719+0000 7fad8ea0e640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad8810d260 con 0x7fad88101600 2024-03-04T09:28:34.720 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:34.719+0000 7fad8ea0e640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad8810d0e0 con 0x7fad881027f0 2024-03-04T09:28:34.720 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:34.719+0000 7fad8ea0e640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad8810d3e0 con 0x7fad881031b0 2024-03-04T09:28:34.720 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:34.719+0000 7fad87fff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fad88101600 0x7fad88197930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:28:34.720 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:34.719+0000 7fad87fff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fad88101600 0x7fad88197930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:57698/0 (socket says 172.21.15.52:57698) 2024-03-04T09:28:34.720 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:34.719+0000 7fad87fff640 1 -- 172.21.15.52:0/4134124718 learned_addr learned my addr 172.21.15.52:0/4134124718 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-04T09:28:34.720 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:34.719+0000 7fad877fe640 1 --2- 172.21.15.52:0/4134124718 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fad881027f0 0x7fad88197e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:28:34.720 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:34.719+0000 7fad8cfab640 1 --2- 172.21.15.52:0/4134124718 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fad881031b0 0x7fad8819eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:28:34.720 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:34.719+0000 7fad87fff640 1 -- 172.21.15.52:0/4134124718 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fad881031b0 msgr2=0x7fad8819eee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:28:34.720 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:34.719+0000 7fad87fff640 1 --2- 172.21.15.52:0/4134124718 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fad881031b0 0x7fad8819eee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:34.720 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:34.719+0000 7fad87fff640 1 -- 172.21.15.52:0/4134124718 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fad881027f0 msgr2=0x7fad88197e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:28:34.720 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:34.719+0000 7fad87fff640 1 --2- 172.21.15.52:0/4134124718 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fad881027f0 0x7fad88197e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:34.721 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:34.719+0000 7fad87fff640 1 -- 172.21.15.52:0/4134124718 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad8819f550 con 0x7fad88101600 2024-03-04T09:28:34.721 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:34.719+0000 7fad8cfab640 1 --2- 172.21.15.52:0/4134124718 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fad881031b0 0x7fad8819eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T09:28:34.721 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:34.719+0000 7fad877fe640 1 --2- 172.21.15.52:0/4134124718 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fad881027f0 0x7fad88197e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T09:28:34.721 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:34.719+0000 7fad87fff640 1 --2- 172.21.15.52:0/4134124718 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fad88101600 0x7fad88197930 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fad7800e9c0 tx=0x7fad7800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:28:34.721 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:34.720+0000 7fad857fa640 1 -- 172.21.15.52:0/4134124718 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad7800cd80 con 0x7fad88101600 2024-03-04T09:28:34.721 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:34.720+0000 7fad857fa640 1 -- 172.21.15.52:0/4134124718 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fad780035f0 con 0x7fad88101600 2024-03-04T09:28:34.721 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:34.720+0000 7fad8ea0e640 1 -- 172.21.15.52:0/4134124718 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad8819f840 con 0x7fad88101600 2024-03-04T09:28:34.721 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:34.720+0000 7fad857fa640 1 -- 172.21.15.52:0/4134124718 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad78010690 con 0x7fad88101600 2024-03-04T09:28:34.721 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:34.720+0000 7fad8ea0e640 1 -- 172.21.15.52:0/4134124718 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad880739e0 con 0x7fad88101600 2024-03-04T09:28:34.721 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:34.721+0000 7fad8ea0e640 1 -- 172.21.15.52:0/4134124718 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad88108940 con 0x7fad88101600 2024-03-04T09:28:34.721 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:34.722+0000 7fad857fa640 1 -- 172.21.15.52:0/4134124718 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 16) v1 ==== 136756+0+0 (secure 0 0 0) 0x7fad780037a0 con 0x7fad88101600 2024-03-04T09:28:34.722 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:34.722+0000 7fad857fa640 1 --2- 172.21.15.52:0/4134124718 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7fad700a6210 0x7fad700a86d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:28:34.722 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:34.722+0000 7fad877fe640 1 --2- 172.21.15.52:0/4134124718 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7fad700a6210 0x7fad700a86d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:28:34.722 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:34.722+0000 7fad857fa640 1 -- 172.21.15.52:0/4134124718 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fad780cddd0 con 0x7fad88101600 2024-03-04T09:28:34.722 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:34.723+0000 7fad877fe640 1 --2- 172.21.15.52:0/4134124718 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7fad700a6210 0x7fad700a86d0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fad7c006010 tx=0x7fad7c02c8a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:28:34.723 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:34.727+0000 7fad857fa640 1 -- 172.21.15.52:0/4134124718 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fad7809b6a0 con 0x7fad88101600 2024-03-04T09:28:34.904 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:34.907+0000 7fad8ea0e640 1 -- 172.21.15.52:0/4134124718 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fad8802cff0 con 0x7fad88101600 2024-03-04T09:28:34.904 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:34.907+0000 7fad857fa640 1 -- 172.21.15.52:0/4134124718 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v81) v1 ==== 93+0+10 (secure 0 0 0) 0x7fad7809afa0 con 0x7fad88101600 2024-03-04T09:28:34.904 INFO:teuthology.orchestra.run.smithi052.stdout:pg_num: 1 2024-03-04T09:28:34.906 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:34.909+0000 7fad8ea0e640 1 -- 172.21.15.52:0/4134124718 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7fad700a6210 msgr2=0x7fad700a86d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:28:34.906 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:34.909+0000 7fad8ea0e640 1 --2- 172.21.15.52:0/4134124718 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7fad700a6210 0x7fad700a86d0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fad7c006010 tx=0x7fad7c02c8a0 comp rx=0 tx=0).stop 2024-03-04T09:28:34.906 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:34.909+0000 7fad8ea0e640 1 -- 172.21.15.52:0/4134124718 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fad88101600 msgr2=0x7fad88197930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:28:34.907 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:34.909+0000 7fad8ea0e640 1 --2- 172.21.15.52:0/4134124718 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fad88101600 0x7fad88197930 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fad7800e9c0 tx=0x7fad7800ee80 comp rx=0 tx=0).stop 2024-03-04T09:28:34.907 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:34.909+0000 7fad8ea0e640 1 -- 172.21.15.52:0/4134124718 shutdown_connections 2024-03-04T09:28:34.907 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:34.909+0000 7fad8ea0e640 1 --2- 172.21.15.52:0/4134124718 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7fad700a6210 0x7fad700a86d0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:34.907 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:34.909+0000 7fad8ea0e640 1 --2- 172.21.15.52:0/4134124718 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fad881031b0 0x7fad8819eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:34.907 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:34.909+0000 7fad8ea0e640 1 --2- 172.21.15.52:0/4134124718 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fad881027f0 0x7fad88197e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:34.907 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:34.909+0000 7fad8ea0e640 1 --2- 172.21.15.52:0/4134124718 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fad88101600 0x7fad88197930 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:34.907 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:34.909+0000 7fad8ea0e640 1 -- 172.21.15.52:0/4134124718 >> 172.21.15.52:0/4134124718 conn(0x7fad880fcb70 msgr2=0x7fad88105b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:28:34.907 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:34.909+0000 7fad8ea0e640 1 -- 172.21.15.52:0/4134124718 shutdown_connections 2024-03-04T09:28:34.907 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:34.909+0000 7fad8ea0e640 1 -- 172.21.15.52:0/4134124718 wait complete. 2024-03-04T09:28:34.916 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:28:34 smithi052 ceph-mon[32473]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-04T09:28:34.916 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:28:34 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:28:34.916 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:28:34 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:28:34.916 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:28:34 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:28:34.916 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:28:34 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:28:34.916 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:28:34 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:28:34.916 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:28:34 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T09:28:34.917 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:28:34 smithi052 ceph-mon[32473]: mds.? [v2:172.21.15.106:6838/1995853093,v1:172.21.15.106:6839/1995853093] up:boot 2024-03-04T09:28:34.917 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:28:34 smithi052 ceph-mon[32473]: fsmap 12 up:standby 2024-03-04T09:28:34.917 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:28:34 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-03-04T09:28:34.917 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:28:34 smithi052 ceph-mon[32473]: from='client.? 172.21.15.52:0/4057867528' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-04T09:28:34.917 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:28:34 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:28:34.917 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:28:34 smithi052 ceph-mon[32473]: from='client.? 172.21.15.52:0/3388217884' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-04T09:28:34.918 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-03-04T09:28:35.026 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:35.027+0000 7f4230440640 1 -- 172.21.15.52:0/69839659 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f42281016e0 msgr2=0x7f4228101b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:28:35.026 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:35.027+0000 7f4230440640 1 --2- 172.21.15.52:0/69839659 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f42281016e0 0x7f4228101b60 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f421c009900 tx=0x7f421c02bc30 comp rx=0 tx=0).stop 2024-03-04T09:28:35.026 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:35.027+0000 7f4230440640 1 -- 172.21.15.52:0/69839659 shutdown_connections 2024-03-04T09:28:35.026 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:35.027+0000 7f4230440640 1 --2- 172.21.15.52:0/69839659 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f42281020a0 0x7f4228108930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:35.026 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:35.027+0000 7f4230440640 1 --2- 172.21.15.52:0/69839659 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f42281016e0 0x7f4228101b60 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:35.026 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:35.027+0000 7f4230440640 1 --2- 172.21.15.52:0/69839659 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f42281004f0 0x7f4228100910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:35.027 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:35.027+0000 7f4230440640 1 -- 172.21.15.52:0/69839659 >> 172.21.15.52:0/69839659 conn(0x7f42280fba60 msgr2=0x7f42280fdea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:28:35.027 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:35.027+0000 7f4230440640 1 -- 172.21.15.52:0/69839659 shutdown_connections 2024-03-04T09:28:35.027 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:35.027+0000 7f4230440640 1 -- 172.21.15.52:0/69839659 wait complete. 2024-03-04T09:28:35.027 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:35.027+0000 7f4230440640 1 Processor -- start 2024-03-04T09:28:35.027 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:35.027+0000 7f4230440640 1 -- start start 2024-03-04T09:28:35.027 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:35.028+0000 7f4230440640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f42281004f0 0x7f4228197960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:28:35.027 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:35.028+0000 7f4230440640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f42281016e0 0x7f4228197ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:28:35.027 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:35.028+0000 7f4230440640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f42281020a0 0x7f422819ef10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:28:35.027 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:35.028+0000 7f4230440640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f422810c1c0 con 0x7f42281020a0 2024-03-04T09:28:35.027 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:35.028+0000 7f4230440640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f422810c040 con 0x7f42281004f0 2024-03-04T09:28:35.027 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:35.028+0000 7f4230440640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f422810c340 con 0x7f42281016e0 2024-03-04T09:28:35.028 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:35.028+0000 7f422e9dd640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f42281020a0 0x7f422819ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:28:35.028 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:35.028+0000 7f422e9dd640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f42281020a0 0x7f422819ef10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:57708/0 (socket says 172.21.15.52:57708) 2024-03-04T09:28:35.028 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:35.028+0000 7f422e9dd640 1 -- 172.21.15.52:0/467758584 learned_addr learned my addr 172.21.15.52:0/467758584 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-04T09:28:35.028 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:35.028+0000 7f422d9db640 1 --2- 172.21.15.52:0/467758584 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f42281016e0 0x7f4228197ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:28:35.028 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:35.028+0000 7f422e1dc640 1 --2- 172.21.15.52:0/467758584 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f42281004f0 0x7f4228197960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:28:35.028 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:35.028+0000 7f422e9dd640 1 -- 172.21.15.52:0/467758584 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f42281016e0 msgr2=0x7f4228197ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:28:35.029 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:35.028+0000 7f422e9dd640 1 --2- 172.21.15.52:0/467758584 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f42281016e0 0x7f4228197ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:35.029 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:35.028+0000 7f422e9dd640 1 -- 172.21.15.52:0/467758584 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f42281004f0 msgr2=0x7f4228197960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:28:35.029 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:35.028+0000 7f422e9dd640 1 --2- 172.21.15.52:0/467758584 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f42281004f0 0x7f4228197960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:35.029 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:35.028+0000 7f422e9dd640 1 -- 172.21.15.52:0/467758584 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f422819f580 con 0x7f42281020a0 2024-03-04T09:28:35.029 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:35.028+0000 7f422e9dd640 1 --2- 172.21.15.52:0/467758584 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f42281020a0 0x7f422819ef10 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f421800b970 tx=0x7f421800be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:28:35.029 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:35.028+0000 7f42277fe640 1 -- 172.21.15.52:0/467758584 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4218019070 con 0x7f42281020a0 2024-03-04T09:28:35.029 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:35.028+0000 7f4230440640 1 -- 172.21.15.52:0/467758584 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f422819f870 con 0x7f42281020a0 2024-03-04T09:28:35.029 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:35.028+0000 7f42277fe640 1 -- 172.21.15.52:0/467758584 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f421800cb40 con 0x7f42281020a0 2024-03-04T09:28:35.029 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:35.028+0000 7f42277fe640 1 -- 172.21.15.52:0/467758584 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4218007610 con 0x7f42281020a0 2024-03-04T09:28:35.029 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:35.028+0000 7f4230440640 1 -- 172.21.15.52:0/467758584 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f422819fde0 con 0x7f42281020a0 2024-03-04T09:28:35.030 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:35.030+0000 7f4230440640 1 -- 172.21.15.52:0/467758584 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f42281a0040 con 0x7f42281020a0 2024-03-04T09:28:35.030 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:35.030+0000 7f42277fe640 1 -- 172.21.15.52:0/467758584 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 16) v1 ==== 136756+0+0 (secure 0 0 0) 0x7f421801c070 con 0x7f42281020a0 2024-03-04T09:28:35.030 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:35.031+0000 7f42277fe640 1 --2- 172.21.15.52:0/467758584 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f42100a1d80 0x7f42100a4240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:28:35.030 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:35.031+0000 7f422e1dc640 1 --2- 172.21.15.52:0/467758584 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f42100a1d80 0x7f42100a4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:28:35.031 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:35.031+0000 7f42277fe640 1 -- 172.21.15.52:0/467758584 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f42180cce80 con 0x7f42281020a0 2024-03-04T09:28:35.031 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:35.032+0000 7f422e1dc640 1 --2- 172.21.15.52:0/467758584 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f42100a1d80 0x7f42100a4240 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f4228101540 tx=0x7f422000b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:28:35.032 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:35.036+0000 7f42277fe640 1 -- 172.21.15.52:0/467758584 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f421809a100 con 0x7f42281020a0 2024-03-04T09:28:35.079 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:28:34 smithi191 ceph-mon[36816]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-04T09:28:35.079 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:28:34 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:28:35.080 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:28:34 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:28:35.080 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:28:34 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:28:35.080 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:28:34 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:28:35.080 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:28:34 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:28:35.080 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:28:34 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T09:28:35.080 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:28:34 smithi191 ceph-mon[36816]: mds.? [v2:172.21.15.106:6838/1995853093,v1:172.21.15.106:6839/1995853093] up:boot 2024-03-04T09:28:35.080 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:28:34 smithi191 ceph-mon[36816]: fsmap 12 up:standby 2024-03-04T09:28:35.080 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:28:34 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-03-04T09:28:35.080 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:28:34 smithi191 ceph-mon[36816]: from='client.? 172.21.15.52:0/4057867528' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-04T09:28:35.080 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:28:34 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:28:35.080 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:28:34 smithi191 ceph-mon[36816]: from='client.? 172.21.15.52:0/3388217884' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-04T09:28:35.118 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:28:34 smithi106 ceph-mon[36484]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-04T09:28:35.118 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:28:34 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:28:35.118 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:28:34 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:28:35.118 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:28:34 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:28:35.118 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:28:34 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:28:35.118 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:28:34 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:28:35.118 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:28:34 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T09:28:35.118 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:28:34 smithi106 ceph-mon[36484]: mds.? [v2:172.21.15.106:6838/1995853093,v1:172.21.15.106:6839/1995853093] up:boot 2024-03-04T09:28:35.118 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:28:34 smithi106 ceph-mon[36484]: fsmap 12 up:standby 2024-03-04T09:28:35.118 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:28:34 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-03-04T09:28:35.118 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:28:34 smithi106 ceph-mon[36484]: from='client.? 172.21.15.52:0/4057867528' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-04T09:28:35.118 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:28:34 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:28:35.119 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:28:34 smithi106 ceph-mon[36484]: from='client.? 172.21.15.52:0/3388217884' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-04T09:28:35.223 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:35.227+0000 7f4230440640 1 -- 172.21.15.52:0/467758584 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f42281a0210 con 0x7f42281020a0 2024-03-04T09:28:35.224 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:35.227+0000 7f42277fe640 1 -- 172.21.15.52:0/467758584 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v81) v1 ==== 84+0+65 (secure 0 0 0) 0x7f421809e1a0 con 0x7f42281020a0 2024-03-04T09:28:35.224 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-03-04T09:28:35.224 INFO:teuthology.orchestra.run.smithi052.stdout:[ 2024-03-04T09:28:35.224 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-03-04T09:28:35.224 INFO:teuthology.orchestra.run.smithi052.stdout: "poolnum": 1, 2024-03-04T09:28:35.224 INFO:teuthology.orchestra.run.smithi052.stdout: "poolname": ".mgr" 2024-03-04T09:28:35.224 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-03-04T09:28:35.225 INFO:teuthology.orchestra.run.smithi052.stdout:] 2024-03-04T09:28:35.226 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:35.229+0000 7f4230440640 1 -- 172.21.15.52:0/467758584 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f42100a1d80 msgr2=0x7f42100a4240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:28:35.226 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:35.229+0000 7f4230440640 1 --2- 172.21.15.52:0/467758584 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f42100a1d80 0x7f42100a4240 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f4228101540 tx=0x7f422000b390 comp rx=0 tx=0).stop 2024-03-04T09:28:35.226 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:35.229+0000 7f4230440640 1 -- 172.21.15.52:0/467758584 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f42281020a0 msgr2=0x7f422819ef10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:28:35.226 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:35.229+0000 7f4230440640 1 --2- 172.21.15.52:0/467758584 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f42281020a0 0x7f422819ef10 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f421800b970 tx=0x7f421800be30 comp rx=0 tx=0).stop 2024-03-04T09:28:35.227 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:35.229+0000 7f4230440640 1 -- 172.21.15.52:0/467758584 shutdown_connections 2024-03-04T09:28:35.227 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:35.229+0000 7f4230440640 1 --2- 172.21.15.52:0/467758584 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f42100a1d80 0x7f42100a4240 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:35.227 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:35.229+0000 7f4230440640 1 --2- 172.21.15.52:0/467758584 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f42281020a0 0x7f422819ef10 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:35.227 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:35.229+0000 7f4230440640 1 --2- 172.21.15.52:0/467758584 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f42281016e0 0x7f4228197ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:35.227 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:35.229+0000 7f4230440640 1 --2- 172.21.15.52:0/467758584 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f42281004f0 0x7f4228197960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:35.227 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:35.229+0000 7f4230440640 1 -- 172.21.15.52:0/467758584 >> 172.21.15.52:0/467758584 conn(0x7f42280fba60 msgr2=0x7f42280fd520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:28:35.227 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:35.229+0000 7f4230440640 1 -- 172.21.15.52:0/467758584 shutdown_connections 2024-03-04T09:28:35.227 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:35.229+0000 7f4230440640 1 -- 172.21.15.52:0/467758584 wait complete. 2024-03-04T09:28:35.237 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-03-04T09:28:35.238 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-03-04T09:28:35.348 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:35.349+0000 7ff999ce1640 1 -- 172.21.15.52:0/1077427684 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff994102080 msgr2=0x7ff994108910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:28:35.349 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:35.349+0000 7ff999ce1640 1 --2- 172.21.15.52:0/1077427684 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff994102080 0x7ff994108910 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7ff98c009ca0 tx=0x7ff98c02c0f0 comp rx=0 tx=0).stop 2024-03-04T09:28:35.349 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:35.349+0000 7ff999ce1640 1 -- 172.21.15.52:0/1077427684 shutdown_connections 2024-03-04T09:28:35.349 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:35.349+0000 7ff999ce1640 1 --2- 172.21.15.52:0/1077427684 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff994102080 0x7ff994108910 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:35.349 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:35.349+0000 7ff999ce1640 1 --2- 172.21.15.52:0/1077427684 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff9941016c0 0x7ff994101b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:35.349 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:35.349+0000 7ff999ce1640 1 --2- 172.21.15.52:0/1077427684 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff9941004d0 0x7ff9941008f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:35.349 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:35.349+0000 7ff999ce1640 1 -- 172.21.15.52:0/1077427684 >> 172.21.15.52:0/1077427684 conn(0x7ff9940fba60 msgr2=0x7ff9940fde80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:28:35.349 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:35.349+0000 7ff999ce1640 1 -- 172.21.15.52:0/1077427684 shutdown_connections 2024-03-04T09:28:35.349 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:35.349+0000 7ff999ce1640 1 -- 172.21.15.52:0/1077427684 wait complete. 2024-03-04T09:28:35.349 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:35.349+0000 7ff999ce1640 1 Processor -- start 2024-03-04T09:28:35.349 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:35.350+0000 7ff999ce1640 1 -- start start 2024-03-04T09:28:35.349 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:35.350+0000 7ff999ce1640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff9941004d0 0x7ff9940692a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:28:35.350 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:35.350+0000 7ff999ce1640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff9941016c0 0x7ff99419c600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:28:35.350 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:35.350+0000 7ff999ce1640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff994102080 0x7ff99419ec20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:28:35.350 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:35.350+0000 7ff999ce1640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff99410c160 con 0x7ff9941016c0 2024-03-04T09:28:35.350 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:35.350+0000 7ff999ce1640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff99410bfe0 con 0x7ff9941004d0 2024-03-04T09:28:35.350 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:35.350+0000 7ff999ce1640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff99410c2e0 con 0x7ff994102080 2024-03-04T09:28:35.350 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:35.350+0000 7ff992ffd640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff9941016c0 0x7ff99419c600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:28:35.350 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:35.350+0000 7ff992ffd640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff9941016c0 0x7ff99419c600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:57726/0 (socket says 172.21.15.52:57726) 2024-03-04T09:28:35.350 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:35.350+0000 7ff992ffd640 1 -- 172.21.15.52:0/3291323285 learned_addr learned my addr 172.21.15.52:0/3291323285 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-04T09:28:35.350 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:35.350+0000 7ff9937fe640 1 --2- 172.21.15.52:0/3291323285 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff9941004d0 0x7ff9940692a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:28:35.351 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:35.350+0000 7ff993fff640 1 --2- 172.21.15.52:0/3291323285 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff994102080 0x7ff99419ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:28:35.351 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:35.350+0000 7ff992ffd640 1 -- 172.21.15.52:0/3291323285 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff994102080 msgr2=0x7ff99419ec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:28:35.351 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:35.350+0000 7ff992ffd640 1 --2- 172.21.15.52:0/3291323285 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff994102080 0x7ff99419ec20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:35.351 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:35.350+0000 7ff992ffd640 1 -- 172.21.15.52:0/3291323285 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff9941004d0 msgr2=0x7ff9940692a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:28:35.351 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:35.350+0000 7ff992ffd640 1 --2- 172.21.15.52:0/3291323285 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff9941004d0 0x7ff9940692a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:35.351 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:35.350+0000 7ff992ffd640 1 -- 172.21.15.52:0/3291323285 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff99419f530 con 0x7ff9941016c0 2024-03-04T09:28:35.351 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:35.350+0000 7ff992ffd640 1 --2- 172.21.15.52:0/3291323285 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff9941016c0 0x7ff99419c600 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7ff98800e9f0 tx=0x7ff98800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:28:35.351 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:35.350+0000 7ff990ff9640 1 -- 172.21.15.52:0/3291323285 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff98800cd80 con 0x7ff9941016c0 2024-03-04T09:28:35.351 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:35.351+0000 7ff990ff9640 1 -- 172.21.15.52:0/3291323285 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff988005500 con 0x7ff9941016c0 2024-03-04T09:28:35.351 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:35.351+0000 7ff999ce1640 1 -- 172.21.15.52:0/3291323285 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff99419f7c0 con 0x7ff9941016c0 2024-03-04T09:28:35.351 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:35.351+0000 7ff990ff9640 1 -- 172.21.15.52:0/3291323285 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff988010690 con 0x7ff9941016c0 2024-03-04T09:28:35.351 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:35.351+0000 7ff999ce1640 1 -- 172.21.15.52:0/3291323285 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff99419fd10 con 0x7ff9941016c0 2024-03-04T09:28:35.351 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:35.351+0000 7ff999ce1640 1 -- 172.21.15.52:0/3291323285 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff9941077f0 con 0x7ff9941016c0 2024-03-04T09:28:35.352 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:35.353+0000 7ff990ff9640 1 -- 172.21.15.52:0/3291323285 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 16) v1 ==== 136756+0+0 (secure 0 0 0) 0x7ff9880046c0 con 0x7ff9941016c0 2024-03-04T09:28:35.352 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:35.353+0000 7ff990ff9640 1 --2- 172.21.15.52:0/3291323285 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7ff97c0a1e00 0x7ff97c0a42c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:28:35.352 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:35.353+0000 7ff9937fe640 1 --2- 172.21.15.52:0/3291323285 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7ff97c0a1e00 0x7ff97c0a42c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:28:35.352 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:35.353+0000 7ff990ff9640 1 -- 172.21.15.52:0/3291323285 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7ff9880cdf10 con 0x7ff9941016c0 2024-03-04T09:28:35.353 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:35.354+0000 7ff9937fe640 1 --2- 172.21.15.52:0/3291323285 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7ff97c0a1e00 0x7ff97c0a42c0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7ff994101520 tx=0x7ff984005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:28:35.353 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:35.357+0000 7ff990ff9640 1 -- 172.21.15.52:0/3291323285 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7ff988053280 con 0x7ff9941016c0 2024-03-04T09:28:35.552 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:35.555+0000 7ff999ce1640 1 -- 172.21.15.52:0/3291323285 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7ff99402cff0 con 0x7ff9941016c0 2024-03-04T09:28:35.883 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:35.886+0000 7ff990ff9640 1 -- 172.21.15.52:0/3291323285 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v82) v1 ==== 138+0+0 (secure 0 0 0) 0x7ff98809b740 con 0x7ff9941016c0 2024-03-04T09:28:35.884 INFO:teuthology.orchestra.run.smithi052.stderr:pool 'cephfs_metadata' created 2024-03-04T09:28:35.887 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:35.889+0000 7ff999ce1640 1 -- 172.21.15.52:0/3291323285 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7ff97c0a1e00 msgr2=0x7ff97c0a42c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:28:35.887 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:35.889+0000 7ff999ce1640 1 --2- 172.21.15.52:0/3291323285 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7ff97c0a1e00 0x7ff97c0a42c0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7ff994101520 tx=0x7ff984005f90 comp rx=0 tx=0).stop 2024-03-04T09:28:35.887 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:35.889+0000 7ff999ce1640 1 -- 172.21.15.52:0/3291323285 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff9941016c0 msgr2=0x7ff99419c600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:28:35.887 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:35.889+0000 7ff999ce1640 1 --2- 172.21.15.52:0/3291323285 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff9941016c0 0x7ff99419c600 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7ff98800e9f0 tx=0x7ff98800edf0 comp rx=0 tx=0).stop 2024-03-04T09:28:35.887 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:35.889+0000 7ff999ce1640 1 -- 172.21.15.52:0/3291323285 shutdown_connections 2024-03-04T09:28:35.887 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:35.889+0000 7ff999ce1640 1 --2- 172.21.15.52:0/3291323285 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7ff97c0a1e00 0x7ff97c0a42c0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:35.887 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:35.889+0000 7ff999ce1640 1 --2- 172.21.15.52:0/3291323285 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff994102080 0x7ff99419ec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:35.887 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:35.889+0000 7ff999ce1640 1 --2- 172.21.15.52:0/3291323285 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff9941016c0 0x7ff99419c600 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:35.887 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:35.889+0000 7ff999ce1640 1 --2- 172.21.15.52:0/3291323285 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff9941004d0 0x7ff9940692a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:35.887 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:35.889+0000 7ff999ce1640 1 -- 172.21.15.52:0/3291323285 >> 172.21.15.52:0/3291323285 conn(0x7ff9940fba60 msgr2=0x7ff9941049c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:28:35.887 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:35.890+0000 7ff999ce1640 1 -- 172.21.15.52:0/3291323285 shutdown_connections 2024-03-04T09:28:35.887 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:35.890+0000 7ff999ce1640 1 -- 172.21.15.52:0/3291323285 wait complete. 2024-03-04T09:28:35.901 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.05 2024-03-04T09:28:36.012 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:36.013+0000 7f5fd3193640 1 -- 172.21.15.52:0/3113668167 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5fcc104980 msgr2=0x7f5fcc106d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:28:36.012 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:36.013+0000 7f5fd3193640 1 --2- 172.21.15.52:0/3113668167 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5fcc104980 0x7f5fcc106d70 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f5fc0009900 tx=0x7f5fc002bc30 comp rx=0 tx=0).stop 2024-03-04T09:28:36.012 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:36.013+0000 7f5fd3193640 1 -- 172.21.15.52:0/3113668167 shutdown_connections 2024-03-04T09:28:36.012 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:36.013+0000 7f5fd3193640 1 --2- 172.21.15.52:0/3113668167 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5fcc1072b0 0x7f5fcc109700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:36.013 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:36.013+0000 7f5fd3193640 1 --2- 172.21.15.52:0/3113668167 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5fcc104980 0x7f5fcc106d70 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:36.013 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:36.013+0000 7f5fd3193640 1 --2- 172.21.15.52:0/3113668167 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5fcc102050 0x7f5fcc104440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:36.013 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:36.013+0000 7f5fd3193640 1 -- 172.21.15.52:0/3113668167 >> 172.21.15.52:0/3113668167 conn(0x7f5fcc0fba60 msgr2=0x7f5fcc0fde80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:28:36.013 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:36.013+0000 7f5fd3193640 1 -- 172.21.15.52:0/3113668167 shutdown_connections 2024-03-04T09:28:36.013 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:36.013+0000 7f5fd3193640 1 -- 172.21.15.52:0/3113668167 wait complete. 2024-03-04T09:28:36.013 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:36.014+0000 7f5fd3193640 1 Processor -- start 2024-03-04T09:28:36.013 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:36.014+0000 7f5fd3193640 1 -- start start 2024-03-04T09:28:36.013 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:36.014+0000 7f5fd3193640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5fcc102050 0x7f5fcc197940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:28:36.013 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:36.014+0000 7f5fd3193640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5fcc104980 0x7f5fcc197e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:28:36.013 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:36.014+0000 7f5fd3193640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5fcc1072b0 0x7f5fcc19eef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:28:36.013 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:36.014+0000 7f5fd3193640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fcc10d260 con 0x7f5fcc1072b0 2024-03-04T09:28:36.013 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:36.014+0000 7f5fd3193640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fcc10d0e0 con 0x7f5fcc102050 2024-03-04T09:28:36.013 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:36.014+0000 7f5fd3193640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fcc10d3e0 con 0x7f5fcc104980 2024-03-04T09:28:36.014 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:36.014+0000 7f5fd1730640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5fcc1072b0 0x7f5fcc19eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:28:36.014 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:36.014+0000 7f5fd1730640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5fcc1072b0 0x7f5fcc19eef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:57750/0 (socket says 172.21.15.52:57750) 2024-03-04T09:28:36.014 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:36.014+0000 7f5fd1730640 1 -- 172.21.15.52:0/3102616459 learned_addr learned my addr 172.21.15.52:0/3102616459 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-04T09:28:36.014 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:36.014+0000 7f5fcbfff640 1 --2- 172.21.15.52:0/3102616459 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5fcc104980 0x7f5fcc197e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:28:36.014 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:36.014+0000 7f5fd0f2f640 1 --2- 172.21.15.52:0/3102616459 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5fcc102050 0x7f5fcc197940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:28:36.014 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:36.014+0000 7f5fd1730640 1 -- 172.21.15.52:0/3102616459 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5fcc104980 msgr2=0x7f5fcc197e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:28:36.014 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:36.014+0000 7f5fd1730640 1 --2- 172.21.15.52:0/3102616459 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5fcc104980 0x7f5fcc197e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:36.014 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:36.014+0000 7f5fd1730640 1 -- 172.21.15.52:0/3102616459 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5fcc102050 msgr2=0x7f5fcc197940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:28:36.014 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:36.014+0000 7f5fd1730640 1 --2- 172.21.15.52:0/3102616459 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5fcc102050 0x7f5fcc197940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:36.014 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:36.014+0000 7f5fd1730640 1 -- 172.21.15.52:0/3102616459 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fcc19f560 con 0x7f5fcc1072b0 2024-03-04T09:28:36.014 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:36.014+0000 7f5fcbfff640 1 --2- 172.21.15.52:0/3102616459 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5fcc104980 0x7f5fcc197e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T09:28:36.014 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:36.015+0000 7f5fd1730640 1 --2- 172.21.15.52:0/3102616459 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5fcc1072b0 0x7f5fcc19eef0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f5fbc00e970 tx=0x7f5fbc00ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:28:36.014 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:36.015+0000 7f5fc9ffb640 1 -- 172.21.15.52:0/3102616459 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fbc00ccc0 con 0x7f5fcc1072b0 2024-03-04T09:28:36.014 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:36.015+0000 7f5fd3193640 1 -- 172.21.15.52:0/3102616459 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5fcc19f850 con 0x7f5fcc1072b0 2024-03-04T09:28:36.015 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:36.015+0000 7f5fc9ffb640 1 -- 172.21.15.52:0/3102616459 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5fbc00ce60 con 0x7f5fcc1072b0 2024-03-04T09:28:36.015 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:36.015+0000 7f5fc9ffb640 1 -- 172.21.15.52:0/3102616459 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fbc010450 con 0x7f5fcc1072b0 2024-03-04T09:28:36.015 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:36.015+0000 7f5fd3193640 1 -- 172.21.15.52:0/3102616459 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5fcc19fdc0 con 0x7f5fcc1072b0 2024-03-04T09:28:36.016 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:36.016+0000 7f5fd3193640 1 -- 172.21.15.52:0/3102616459 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5fcc0fd550 con 0x7f5fcc1072b0 2024-03-04T09:28:36.016 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:36.018+0000 7f5fc9ffb640 1 -- 172.21.15.52:0/3102616459 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 16) v1 ==== 136756+0+0 (secure 0 0 0) 0x7f5fbc005000 con 0x7f5fcc1072b0 2024-03-04T09:28:36.016 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:36.018+0000 7f5fc9ffb640 1 --2- 172.21.15.52:0/3102616459 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f5fb40a1d80 0x7f5fb40a4240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:28:36.016 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:36.018+0000 7f5fd0f2f640 1 --2- 172.21.15.52:0/3102616459 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f5fb40a1d80 0x7f5fb40a4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:28:36.016 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:36.019+0000 7f5fc9ffb640 1 -- 172.21.15.52:0/3102616459 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7497+0+0 (secure 0 0 0) 0x7f5fbc0ce280 con 0x7f5fcc1072b0 2024-03-04T09:28:36.016 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:36.019+0000 7f5fd0f2f640 1 --2- 172.21.15.52:0/3102616459 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f5fb40a1d80 0x7f5fb40a4240 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f5fc4005fd0 tx=0x7f5fc4009870 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:28:36.019 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:36.022+0000 7f5fc9ffb640 1 -- 172.21.15.52:0/3102616459 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f5fbc053280 con 0x7f5fcc1072b0 2024-03-04T09:28:36.131 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:28:35 smithi106 ceph-mon[36484]: from='client.? 172.21.15.52:0/4134124718' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-04T09:28:36.131 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:28:35 smithi106 ceph-mon[36484]: from='client.? 172.21.15.52:0/467758584' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-03-04T09:28:36.201 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:36.204+0000 7f5fd3193640 1 -- 172.21.15.52:0/3102616459 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05} v 0) v1 -- 0x7f5fcc0621d0 con 0x7f5fcc1072b0 2024-03-04T09:28:36.334 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:28:35 smithi052 ceph-mon[32473]: from='client.? 172.21.15.52:0/4134124718' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-04T09:28:36.335 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:28:35 smithi052 ceph-mon[32473]: from='client.? 172.21.15.52:0/467758584' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-03-04T09:28:36.336 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:28:35 smithi191 ceph-mon[36816]: from='client.? 172.21.15.52:0/4134124718' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-04T09:28:36.336 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:28:35 smithi191 ceph-mon[36816]: from='client.? 172.21.15.52:0/467758584' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-03-04T09:28:36.885 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:36.888+0000 7f5fc9ffb640 1 -- 172.21.15.52:0/3102616459 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]=0 pool 'cephfs_data' created v83) v1 ==== 171+0+0 (secure 0 0 0) 0x7f5fbc09b940 con 0x7f5fcc1072b0 2024-03-04T09:28:36.885 INFO:teuthology.orchestra.run.smithi052.stderr:pool 'cephfs_data' created 2024-03-04T09:28:36.889 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:36.892+0000 7f5fd3193640 1 -- 172.21.15.52:0/3102616459 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f5fb40a1d80 msgr2=0x7f5fb40a4240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:28:36.889 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:36.892+0000 7f5fd3193640 1 --2- 172.21.15.52:0/3102616459 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f5fb40a1d80 0x7f5fb40a4240 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f5fc4005fd0 tx=0x7f5fc4009870 comp rx=0 tx=0).stop 2024-03-04T09:28:36.889 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:36.892+0000 7f5fd3193640 1 -- 172.21.15.52:0/3102616459 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5fcc1072b0 msgr2=0x7f5fcc19eef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:28:36.889 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:36.892+0000 7f5fd3193640 1 --2- 172.21.15.52:0/3102616459 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5fcc1072b0 0x7f5fcc19eef0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f5fbc00e970 tx=0x7f5fbc00ed70 comp rx=0 tx=0).stop 2024-03-04T09:28:36.891 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:36.893+0000 7f5fd3193640 1 -- 172.21.15.52:0/3102616459 shutdown_connections 2024-03-04T09:28:36.891 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:36.893+0000 7f5fd3193640 1 --2- 172.21.15.52:0/3102616459 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f5fb40a1d80 0x7f5fb40a4240 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:36.891 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:36.893+0000 7f5fd3193640 1 --2- 172.21.15.52:0/3102616459 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5fcc1072b0 0x7f5fcc19eef0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:36.891 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:36.893+0000 7f5fd3193640 1 --2- 172.21.15.52:0/3102616459 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5fcc104980 0x7f5fcc197e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:36.891 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:36.893+0000 7f5fd3193640 1 --2- 172.21.15.52:0/3102616459 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5fcc102050 0x7f5fcc197940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:36.891 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:36.893+0000 7f5fd3193640 1 -- 172.21.15.52:0/3102616459 >> 172.21.15.52:0/3102616459 conn(0x7f5fcc0fba60 msgr2=0x7f5fcc102ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:28:36.893 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:36.896+0000 7f5fd3193640 1 -- 172.21.15.52:0/3102616459 shutdown_connections 2024-03-04T09:28:36.893 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:36.896+0000 7f5fd3193640 1 -- 172.21.15.52:0/3102616459 wait complete. 2024-03-04T09:28:36.913 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-03-04T09:28:37.022 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:37.023+0000 7f410198a640 1 -- 172.21.15.52:0/856876942 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f40fc101660 msgr2=0x7f40fc101a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:28:37.022 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:37.023+0000 7f410198a640 1 --2- 172.21.15.52:0/856876942 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f40fc101660 0x7f40fc101a80 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f40ec009900 tx=0x7f40ec02bc30 comp rx=0 tx=0).stop 2024-03-04T09:28:37.022 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:37.023+0000 7f410198a640 1 -- 172.21.15.52:0/856876942 shutdown_connections 2024-03-04T09:28:37.022 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:37.023+0000 7f410198a640 1 --2- 172.21.15.52:0/856876942 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f40fc103210 0x7f40fc109aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:37.022 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:37.023+0000 7f410198a640 1 --2- 172.21.15.52:0/856876942 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f40fc102850 0x7f40fc102cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:37.022 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:37.023+0000 7f410198a640 1 --2- 172.21.15.52:0/856876942 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f40fc101660 0x7f40fc101a80 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:37.023 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:37.023+0000 7f410198a640 1 -- 172.21.15.52:0/856876942 >> 172.21.15.52:0/856876942 conn(0x7f40fc0fcbf0 msgr2=0x7f40fc0ff010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:28:37.023 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:37.023+0000 7f410198a640 1 -- 172.21.15.52:0/856876942 shutdown_connections 2024-03-04T09:28:37.023 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:37.023+0000 7f410198a640 1 -- 172.21.15.52:0/856876942 wait complete. 2024-03-04T09:28:37.023 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:37.024+0000 7f410198a640 1 Processor -- start 2024-03-04T09:28:37.023 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:37.024+0000 7f410198a640 1 -- start start 2024-03-04T09:28:37.023 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:37.024+0000 7f410198a640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f40fc101660 0x7f40fc197860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:28:37.023 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:37.024+0000 7f410198a640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f40fc102850 0x7f40fc197da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:28:37.023 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:37.024+0000 7f410198a640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f40fc103210 0x7f40fc19ee10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:28:37.023 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:37.024+0000 7f410198a640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40fc10d240 con 0x7f40fc103210 2024-03-04T09:28:37.023 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:37.024+0000 7f410198a640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40fc10d0c0 con 0x7f40fc102850 2024-03-04T09:28:37.023 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:37.024+0000 7f410198a640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40fc10d3c0 con 0x7f40fc101660 2024-03-04T09:28:37.023 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:37.024+0000 7f40fb7fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f40fc103210 0x7f40fc19ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:28:37.023 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:37.024+0000 7f40fb7fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f40fc103210 0x7f40fc19ee10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:57774/0 (socket says 172.21.15.52:57774) 2024-03-04T09:28:37.024 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:37.024+0000 7f40fb7fe640 1 -- 172.21.15.52:0/2808420848 learned_addr learned my addr 172.21.15.52:0/2808420848 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-04T09:28:37.024 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:37.024+0000 7f40faffd640 1 --2- 172.21.15.52:0/2808420848 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f40fc101660 0x7f40fc197860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:28:37.024 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:37.024+0000 7f40f3fff640 1 --2- 172.21.15.52:0/2808420848 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f40fc102850 0x7f40fc197da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:28:37.024 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:37.024+0000 7f40fb7fe640 1 -- 172.21.15.52:0/2808420848 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f40fc101660 msgr2=0x7f40fc197860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:28:37.024 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:37.024+0000 7f40fb7fe640 1 --2- 172.21.15.52:0/2808420848 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f40fc101660 0x7f40fc197860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:37.024 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:37.024+0000 7f40fb7fe640 1 -- 172.21.15.52:0/2808420848 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f40fc102850 msgr2=0x7f40fc197da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:28:37.024 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:37.024+0000 7f40fb7fe640 1 --2- 172.21.15.52:0/2808420848 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f40fc102850 0x7f40fc197da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:37.024 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:37.024+0000 7f40fb7fe640 1 -- 172.21.15.52:0/2808420848 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40fc19f480 con 0x7f40fc103210 2024-03-04T09:28:37.024 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:37.024+0000 7f40fb7fe640 1 --2- 172.21.15.52:0/2808420848 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f40fc103210 0x7f40fc19ee10 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f40f400eb10 tx=0x7f40f400ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:28:37.024 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:37.025+0000 7f4100988640 1 -- 172.21.15.52:0/2808420848 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40f400ce20 con 0x7f40fc103210 2024-03-04T09:28:37.024 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:37.025+0000 7f4100988640 1 -- 172.21.15.52:0/2808420848 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f40f4005020 con 0x7f40fc103210 2024-03-04T09:28:37.024 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:37.025+0000 7f410198a640 1 -- 172.21.15.52:0/2808420848 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40fc19f770 con 0x7f40fc103210 2024-03-04T09:28:37.024 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:37.025+0000 7f4100988640 1 -- 172.21.15.52:0/2808420848 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40f4010690 con 0x7f40fc103210 2024-03-04T09:28:37.024 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:37.025+0000 7f410198a640 1 -- 172.21.15.52:0/2808420848 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40fc19fcc0 con 0x7f40fc103210 2024-03-04T09:28:37.025 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:37.026+0000 7f410198a640 1 -- 172.21.15.52:0/2808420848 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f40fc108980 con 0x7f40fc103210 2024-03-04T09:28:37.025 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:37.027+0000 7f4100988640 1 -- 172.21.15.52:0/2808420848 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 16) v1 ==== 136756+0+0 (secure 0 0 0) 0x7f40f4005540 con 0x7f40fc103210 2024-03-04T09:28:37.026 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:37.027+0000 7f4100988640 1 --2- 172.21.15.52:0/2808420848 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f40e80a1e00 0x7f40e80a42c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:28:37.026 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:37.027+0000 7f40faffd640 1 --2- 172.21.15.52:0/2808420848 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f40e80a1e00 0x7f40e80a42c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:28:37.026 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:37.028+0000 7f4100988640 1 -- 172.21.15.52:0/2808420848 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7879+0+0 (secure 0 0 0) 0x7f40f40ce0f0 con 0x7f40fc103210 2024-03-04T09:28:37.026 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:37.028+0000 7f40faffd640 1 --2- 172.21.15.52:0/2808420848 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f40e80a1e00 0x7f40e80a42c0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f40ec000c00 tx=0x7f40ec005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:28:37.029 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:37.032+0000 7f4100988640 1 -- 172.21.15.52:0/2808420848 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f40f409b6d0 con 0x7f40fc103210 2024-03-04T09:28:37.131 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:28:36 smithi106 ceph-mon[36484]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-04T09:28:37.131 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:28:36 smithi106 ceph-mon[36484]: from='client.? 172.21.15.52:0/3291323285' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-03-04T09:28:37.131 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:28:36 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:28:37.131 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:28:36 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:28:37.131 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:28:36 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:28:37.131 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:28:36 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:28:37.131 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:28:36 smithi106 ceph-mon[36484]: from='client.? 172.21.15.52:0/3291323285' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-03-04T09:28:37.132 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:28:36 smithi106 ceph-mon[36484]: osdmap e82: 12 total, 12 up, 12 in 2024-03-04T09:28:37.132 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:28:36 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:28:37.132 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:28:36 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:28:37.132 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:28:36 smithi106 ceph-mon[36484]: from='client.? 172.21.15.52:0/3102616459' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-03-04T09:28:37.132 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:28:36 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T09:28:37.132 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:28:36 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T09:28:37.132 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:28:36 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:28:37.132 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:28:36 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T09:28:37.259 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:37.263+0000 7f410198a640 1 -- 172.21.15.52:0/2808420848 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f40fc062240 con 0x7f40fc103210 2024-03-04T09:28:37.335 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:28:36 smithi052 ceph-mon[32473]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-04T09:28:37.335 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:28:36 smithi052 ceph-mon[32473]: from='client.? 172.21.15.52:0/3291323285' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-03-04T09:28:37.335 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:28:36 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:28:37.336 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:28:36 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:28:37.336 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:28:36 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:28:37.336 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:28:36 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:28:37.336 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:28:36 smithi052 ceph-mon[32473]: from='client.? 172.21.15.52:0/3291323285' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-03-04T09:28:37.336 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:28:36 smithi052 ceph-mon[32473]: osdmap e82: 12 total, 12 up, 12 in 2024-03-04T09:28:37.336 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:28:36 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:28:37.336 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:28:36 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:28:37.336 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:28:36 smithi052 ceph-mon[32473]: from='client.? 172.21.15.52:0/3102616459' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-03-04T09:28:37.336 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:28:36 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T09:28:37.337 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:28:36 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T09:28:37.337 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:28:36 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:28:37.337 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:28:36 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T09:28:37.338 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:28:36 smithi191 ceph-mon[36816]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-04T09:28:37.338 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:28:36 smithi191 ceph-mon[36816]: from='client.? 172.21.15.52:0/3291323285' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-03-04T09:28:37.338 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:28:36 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:28:37.338 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:28:36 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:28:37.338 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:28:36 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:28:37.338 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:28:36 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:28:37.338 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:28:36 smithi191 ceph-mon[36816]: from='client.? 172.21.15.52:0/3291323285' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-03-04T09:28:37.339 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:28:36 smithi191 ceph-mon[36816]: osdmap e82: 12 total, 12 up, 12 in 2024-03-04T09:28:37.339 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:28:36 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:28:37.339 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:28:36 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:28:37.339 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:28:36 smithi191 ceph-mon[36816]: from='client.? 172.21.15.52:0/3102616459' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-03-04T09:28:37.340 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:28:36 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T09:28:37.340 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:28:36 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T09:28:37.340 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:28:36 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:28:37.340 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:28:36 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T09:28:37.900 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:37.903+0000 7f4100988640 1 -- 172.21.15.52:0/2808420848 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v14) v1 ==== 172+0+0 (secure 0 0 0) 0x7f40f409afd0 con 0x7f40fc103210 2024-03-04T09:28:37.904 INFO:teuthology.orchestra.run.smithi052.stderr:new fs with metadata pool 2 and data pool 3 2024-03-04T09:28:37.904 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:37.906+0000 7f410198a640 1 -- 172.21.15.52:0/2808420848 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f40e80a1e00 msgr2=0x7f40e80a42c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:28:37.904 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:37.906+0000 7f410198a640 1 --2- 172.21.15.52:0/2808420848 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f40e80a1e00 0x7f40e80a42c0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f40ec000c00 tx=0x7f40ec005fd0 comp rx=0 tx=0).stop 2024-03-04T09:28:37.905 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:37.906+0000 7f410198a640 1 -- 172.21.15.52:0/2808420848 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f40fc103210 msgr2=0x7f40fc19ee10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:28:37.905 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:37.906+0000 7f410198a640 1 --2- 172.21.15.52:0/2808420848 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f40fc103210 0x7f40fc19ee10 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f40f400eb10 tx=0x7f40f400ef10 comp rx=0 tx=0).stop 2024-03-04T09:28:37.905 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:37.907+0000 7f410198a640 1 -- 172.21.15.52:0/2808420848 shutdown_connections 2024-03-04T09:28:37.905 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:37.907+0000 7f410198a640 1 --2- 172.21.15.52:0/2808420848 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f40e80a1e00 0x7f40e80a42c0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:37.905 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:37.907+0000 7f410198a640 1 --2- 172.21.15.52:0/2808420848 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f40fc103210 0x7f40fc19ee10 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:37.905 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:37.907+0000 7f410198a640 1 --2- 172.21.15.52:0/2808420848 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f40fc102850 0x7f40fc197da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:37.905 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:37.907+0000 7f410198a640 1 --2- 172.21.15.52:0/2808420848 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f40fc101660 0x7f40fc197860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:37.905 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:37.907+0000 7f410198a640 1 -- 172.21.15.52:0/2808420848 >> 172.21.15.52:0/2808420848 conn(0x7f40fc0fcbf0 msgr2=0x7f40fc105b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:28:37.905 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:37.907+0000 7f410198a640 1 -- 172.21.15.52:0/2808420848 shutdown_connections 2024-03-04T09:28:37.905 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:37.907+0000 7f410198a640 1 -- 172.21.15.52:0/2808420848 wait complete. 2024-03-04T09:28:37.926 DEBUG:tasks.cephfs.filesystem:EC profile is ['m=2', 'k=2', 'crush-failure-domain=osd'] 2024-03-04T09:28:37.926 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd erasure-code-profile set cephfs_data_ec m=2 k=2 crush-failure-domain=osd 2024-03-04T09:28:38.043 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:38.044+0000 7f5e008b5640 1 -- 172.21.15.52:0/2824802824 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5dfc1016c0 msgr2=0x7f5dfc101b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:28:38.043 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:38.044+0000 7f5e008b5640 1 --2- 172.21.15.52:0/2824802824 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5dfc1016c0 0x7f5dfc101b40 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f5df0009900 tx=0x7f5df002bc30 comp rx=0 tx=0).stop 2024-03-04T09:28:38.043 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:38.044+0000 7f5e008b5640 1 -- 172.21.15.52:0/2824802824 shutdown_connections 2024-03-04T09:28:38.043 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:38.044+0000 7f5e008b5640 1 --2- 172.21.15.52:0/2824802824 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5dfc102080 0x7f5dfc108910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:38.043 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:38.044+0000 7f5e008b5640 1 --2- 172.21.15.52:0/2824802824 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5dfc1016c0 0x7f5dfc101b40 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:38.043 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:38.044+0000 7f5e008b5640 1 --2- 172.21.15.52:0/2824802824 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5dfc1004d0 0x7f5dfc1008f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:38.043 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:38.044+0000 7f5e008b5640 1 -- 172.21.15.52:0/2824802824 >> 172.21.15.52:0/2824802824 conn(0x7f5dfc0fba60 msgr2=0x7f5dfc0fde80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:28:38.043 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:38.044+0000 7f5e008b5640 1 -- 172.21.15.52:0/2824802824 shutdown_connections 2024-03-04T09:28:38.044 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:38.044+0000 7f5e008b5640 1 -- 172.21.15.52:0/2824802824 wait complete. 2024-03-04T09:28:38.044 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:38.044+0000 7f5e008b5640 1 Processor -- start 2024-03-04T09:28:38.044 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:38.044+0000 7f5e008b5640 1 -- start start 2024-03-04T09:28:38.044 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:38.045+0000 7f5e008b5640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5dfc1004d0 0x7f5dfc199b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:28:38.044 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:38.045+0000 7f5e008b5640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5dfc1016c0 0x7f5dfc19a040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:28:38.044 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:38.045+0000 7f5e008b5640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5dfc102080 0x7f5dfc1a10b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:28:38.044 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:38.045+0000 7f5e008b5640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5dfc10c160 con 0x7f5dfc1004d0 2024-03-04T09:28:38.044 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:38.045+0000 7f5e008b5640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5dfc10bfe0 con 0x7f5dfc1016c0 2024-03-04T09:28:38.044 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:38.045+0000 7f5e008b5640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5dfc10c2e0 con 0x7f5dfc102080 2024-03-04T09:28:38.045 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:38.045+0000 7f5dfa59c640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5dfc1004d0 0x7f5dfc199b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:28:38.045 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:38.045+0000 7f5dfa59c640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5dfc1004d0 0x7f5dfc199b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:57796/0 (socket says 172.21.15.52:57796) 2024-03-04T09:28:38.045 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:38.045+0000 7f5dfa59c640 1 -- 172.21.15.52:0/4114867410 learned_addr learned my addr 172.21.15.52:0/4114867410 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-04T09:28:38.045 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:38.045+0000 7f5dfad9d640 1 --2- 172.21.15.52:0/4114867410 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5dfc102080 0x7f5dfc1a10b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:28:38.045 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:38.045+0000 7f5df9d9b640 1 --2- 172.21.15.52:0/4114867410 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5dfc1016c0 0x7f5dfc19a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:28:38.045 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:38.045+0000 7f5dfa59c640 1 -- 172.21.15.52:0/4114867410 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5dfc102080 msgr2=0x7f5dfc1a10b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:28:38.045 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:38.045+0000 7f5dfa59c640 1 --2- 172.21.15.52:0/4114867410 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5dfc102080 0x7f5dfc1a10b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:38.045 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:38.045+0000 7f5dfa59c640 1 -- 172.21.15.52:0/4114867410 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5dfc1016c0 msgr2=0x7f5dfc19a040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:28:38.045 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:38.045+0000 7f5dfa59c640 1 --2- 172.21.15.52:0/4114867410 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5dfc1016c0 0x7f5dfc19a040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:38.045 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:38.045+0000 7f5dfa59c640 1 -- 172.21.15.52:0/4114867410 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5dfc1a1720 con 0x7f5dfc1004d0 2024-03-04T09:28:38.045 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:38.045+0000 7f5dfa59c640 1 --2- 172.21.15.52:0/4114867410 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5dfc1004d0 0x7f5dfc199b00 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f5dec00eb70 tx=0x7f5dec00c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:28:38.046 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:38.045+0000 7f5deb7fe640 1 -- 172.21.15.52:0/4114867410 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dec003b70 con 0x7f5dfc1004d0 2024-03-04T09:28:38.046 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:38.045+0000 7f5e008b5640 1 -- 172.21.15.52:0/4114867410 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5dfc1a1a10 con 0x7f5dfc1004d0 2024-03-04T09:28:38.046 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:38.045+0000 7f5deb7fe640 1 -- 172.21.15.52:0/4114867410 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5dec00ee70 con 0x7f5dfc1004d0 2024-03-04T09:28:38.046 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:38.045+0000 7f5deb7fe640 1 -- 172.21.15.52:0/4114867410 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dec010830 con 0x7f5dfc1004d0 2024-03-04T09:28:38.046 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:38.045+0000 7f5e008b5640 1 -- 172.21.15.52:0/4114867410 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5dfc1a1ee0 con 0x7f5dfc1004d0 2024-03-04T09:28:38.047 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:38.047+0000 7f5e008b5640 1 -- 172.21.15.52:0/4114867410 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5dfc1a2110 con 0x7f5dfc1004d0 2024-03-04T09:28:38.047 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:38.048+0000 7f5deb7fe640 1 -- 172.21.15.52:0/4114867410 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 16) v1 ==== 136756+0+0 (secure 0 0 0) 0x7f5dec004020 con 0x7f5dfc1004d0 2024-03-04T09:28:38.047 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:38.049+0000 7f5deb7fe640 1 --2- 172.21.15.52:0/4114867410 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f5de40a1d80 0x7f5de40a4240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:28:38.047 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:38.049+0000 7f5df9d9b640 1 --2- 172.21.15.52:0/4114867410 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f5de40a1d80 0x7f5de40a4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:28:38.047 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:38.049+0000 7f5deb7fe640 1 -- 172.21.15.52:0/4114867410 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f5dec0ce2e0 con 0x7f5dfc1004d0 2024-03-04T09:28:38.047 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:38.049+0000 7f5df9d9b640 1 --2- 172.21.15.52:0/4114867410 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f5de40a1d80 0x7f5de40a4240 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f5df002c7b0 tx=0x7f5df0005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:28:38.052 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:38.055+0000 7f5deb7fe640 1 -- 172.21.15.52:0/4114867410 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f5dec09b860 con 0x7f5dfc1004d0 2024-03-04T09:28:38.299 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:38.302+0000 7f5e008b5640 1 -- 172.21.15.52:0/4114867410 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]} v 0) v1 -- 0x7f5dfc1a2400 con 0x7f5dfc1004d0 2024-03-04T09:28:38.334 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:28:37 smithi052 ceph-mon[32473]: from='client.? 172.21.15.52:0/3102616459' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-03-04T09:28:38.335 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:28:37 smithi052 ceph-mon[32473]: osdmap e83: 12 total, 12 up, 12 in 2024-03-04T09:28:38.335 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:28:37 smithi052 ceph-mon[32473]: from='client.? 172.21.15.52:0/2808420848' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-03-04T09:28:38.335 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:28:37 smithi052 ceph-mon[32473]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-03-04T09:28:38.336 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:28:37 smithi191 ceph-mon[36816]: from='client.? 172.21.15.52:0/3102616459' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-03-04T09:28:38.336 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:28:37 smithi191 ceph-mon[36816]: osdmap e83: 12 total, 12 up, 12 in 2024-03-04T09:28:38.336 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:28:37 smithi191 ceph-mon[36816]: from='client.? 172.21.15.52:0/2808420848' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-03-04T09:28:38.336 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:28:37 smithi191 ceph-mon[36816]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-03-04T09:28:38.381 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:28:37 smithi106 ceph-mon[36484]: from='client.? 172.21.15.52:0/3102616459' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-03-04T09:28:38.381 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:28:37 smithi106 ceph-mon[36484]: osdmap e83: 12 total, 12 up, 12 in 2024-03-04T09:28:38.381 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:28:37 smithi106 ceph-mon[36484]: from='client.? 172.21.15.52:0/2808420848' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-03-04T09:28:38.381 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:28:37 smithi106 ceph-mon[36484]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-03-04T09:28:38.888 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:38.891+0000 7f5deb7fe640 1 -- 172.21.15.52:0/4114867410 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]=0 v85) v1 ==== 157+0+0 (secure 0 0 0) 0x7f5dec09b160 con 0x7f5dfc1004d0 2024-03-04T09:28:38.891 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:38.894+0000 7f5e008b5640 1 -- 172.21.15.52:0/4114867410 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f5de40a1d80 msgr2=0x7f5de40a4240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:28:38.891 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:38.894+0000 7f5e008b5640 1 --2- 172.21.15.52:0/4114867410 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f5de40a1d80 0x7f5de40a4240 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f5df002c7b0 tx=0x7f5df0005fd0 comp rx=0 tx=0).stop 2024-03-04T09:28:38.891 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:38.894+0000 7f5e008b5640 1 -- 172.21.15.52:0/4114867410 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5dfc1004d0 msgr2=0x7f5dfc199b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:28:38.891 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:38.894+0000 7f5e008b5640 1 --2- 172.21.15.52:0/4114867410 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5dfc1004d0 0x7f5dfc199b00 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f5dec00eb70 tx=0x7f5dec00c6a0 comp rx=0 tx=0).stop 2024-03-04T09:28:38.892 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:38.894+0000 7f5e008b5640 1 -- 172.21.15.52:0/4114867410 shutdown_connections 2024-03-04T09:28:38.892 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:38.894+0000 7f5e008b5640 1 --2- 172.21.15.52:0/4114867410 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f5de40a1d80 0x7f5de40a4240 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:38.892 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:38.894+0000 7f5e008b5640 1 --2- 172.21.15.52:0/4114867410 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5dfc102080 0x7f5dfc1a10b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:38.892 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:38.894+0000 7f5e008b5640 1 --2- 172.21.15.52:0/4114867410 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5dfc1016c0 0x7f5dfc19a040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:38.892 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:38.894+0000 7f5e008b5640 1 --2- 172.21.15.52:0/4114867410 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5dfc1004d0 0x7f5dfc199b00 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:38.892 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:38.894+0000 7f5e008b5640 1 -- 172.21.15.52:0/4114867410 >> 172.21.15.52:0/4114867410 conn(0x7f5dfc0fba60 msgr2=0x7f5dfc0fd540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:28:38.892 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:38.894+0000 7f5e008b5640 1 -- 172.21.15.52:0/4114867410 shutdown_connections 2024-03-04T09:28:38.892 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:38.894+0000 7f5e008b5640 1 -- 172.21.15.52:0/4114867410 wait complete. 2024-03-04T09:28:38.904 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data_ec erasure cephfs_data_ec --pg_num_min 64 --target_size_ratio 0.9 2024-03-04T09:28:39.021 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:39.022+0000 7fb1df25c640 1 -- 172.21.15.52:0/720671892 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb1d81031a0 msgr2=0x7fb1d8109a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:28:39.021 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:39.022+0000 7fb1df25c640 1 --2- 172.21.15.52:0/720671892 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb1d81031a0 0x7fb1d8109a30 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fb1d0009ca0 tx=0x7fb1d002c0f0 comp rx=0 tx=0).stop 2024-03-04T09:28:39.021 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:39.022+0000 7fb1df25c640 1 -- 172.21.15.52:0/720671892 shutdown_connections 2024-03-04T09:28:39.021 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:39.022+0000 7fb1df25c640 1 --2- 172.21.15.52:0/720671892 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb1d81031a0 0x7fb1d8109a30 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:39.022 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:39.022+0000 7fb1df25c640 1 --2- 172.21.15.52:0/720671892 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb1d81027e0 0x7fb1d8102c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:39.022 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:39.022+0000 7fb1df25c640 1 --2- 172.21.15.52:0/720671892 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb1d81015f0 0x7fb1d8101a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:39.022 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:39.022+0000 7fb1df25c640 1 -- 172.21.15.52:0/720671892 >> 172.21.15.52:0/720671892 conn(0x7fb1d80fcb80 msgr2=0x7fb1d80fefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:28:39.022 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:39.022+0000 7fb1df25c640 1 -- 172.21.15.52:0/720671892 shutdown_connections 2024-03-04T09:28:39.022 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:39.022+0000 7fb1df25c640 1 -- 172.21.15.52:0/720671892 wait complete. 2024-03-04T09:28:39.022 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:39.022+0000 7fb1df25c640 1 Processor -- start 2024-03-04T09:28:39.022 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:39.022+0000 7fb1df25c640 1 -- start start 2024-03-04T09:28:39.022 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:39.022+0000 7fb1df25c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb1d81015f0 0x7fb1d8197860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:28:39.022 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:39.022+0000 7fb1df25c640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb1d81027e0 0x7fb1d8197da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:28:39.022 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:39.022+0000 7fb1df25c640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb1d81031a0 0x7fb1d819ee10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:28:39.022 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:39.022+0000 7fb1df25c640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1d810d250 con 0x7fb1d81027e0 2024-03-04T09:28:39.022 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:39.022+0000 7fb1df25c640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1d810d0d0 con 0x7fb1d81015f0 2024-03-04T09:28:39.022 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:39.022+0000 7fb1df25c640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1d810d3d0 con 0x7fb1d81031a0 2024-03-04T09:28:39.023 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:39.023+0000 7fb1d7fff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb1d81027e0 0x7fb1d8197da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:28:39.023 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:39.023+0000 7fb1d7fff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb1d81027e0 0x7fb1d8197da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:57818/0 (socket says 172.21.15.52:57818) 2024-03-04T09:28:39.023 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:39.023+0000 7fb1d7fff640 1 -- 172.21.15.52:0/3269973034 learned_addr learned my addr 172.21.15.52:0/3269973034 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-04T09:28:39.023 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:39.023+0000 7fb1dcff8640 1 --2- 172.21.15.52:0/3269973034 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb1d81015f0 0x7fb1d8197860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:28:39.023 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:39.023+0000 7fb1dd7f9640 1 --2- 172.21.15.52:0/3269973034 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb1d81031a0 0x7fb1d819ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:28:39.023 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:39.023+0000 7fb1d7fff640 1 -- 172.21.15.52:0/3269973034 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb1d81031a0 msgr2=0x7fb1d819ee10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:28:39.023 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:39.023+0000 7fb1d7fff640 1 --2- 172.21.15.52:0/3269973034 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb1d81031a0 0x7fb1d819ee10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:39.023 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:39.023+0000 7fb1d7fff640 1 -- 172.21.15.52:0/3269973034 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb1d81015f0 msgr2=0x7fb1d8197860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:28:39.023 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:39.023+0000 7fb1d7fff640 1 --2- 172.21.15.52:0/3269973034 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb1d81015f0 0x7fb1d8197860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:39.023 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:39.023+0000 7fb1d7fff640 1 -- 172.21.15.52:0/3269973034 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1d819f480 con 0x7fb1d81027e0 2024-03-04T09:28:39.023 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:39.023+0000 7fb1d7fff640 1 --2- 172.21.15.52:0/3269973034 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb1d81027e0 0x7fb1d8197da0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fb1c400d950 tx=0x7fb1c400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:28:39.023 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:39.023+0000 7fb1d5ffb640 1 -- 172.21.15.52:0/3269973034 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1c4017070 con 0x7fb1d81027e0 2024-03-04T09:28:39.023 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:39.023+0000 7fb1d5ffb640 1 -- 172.21.15.52:0/3269973034 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb1c4009c30 con 0x7fb1d81027e0 2024-03-04T09:28:39.023 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:39.023+0000 7fb1df25c640 1 -- 172.21.15.52:0/3269973034 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1d819f770 con 0x7fb1d81027e0 2024-03-04T09:28:39.024 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:39.023+0000 7fb1d5ffb640 1 -- 172.21.15.52:0/3269973034 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1c400f6b0 con 0x7fb1d81027e0 2024-03-04T09:28:39.024 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:39.023+0000 7fb1df25c640 1 -- 172.21.15.52:0/3269973034 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1d819fcc0 con 0x7fb1d81027e0 2024-03-04T09:28:39.024 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:39.024+0000 7fb1df25c640 1 -- 172.21.15.52:0/3269973034 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb1d81088b0 con 0x7fb1d81027e0 2024-03-04T09:28:39.025 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:39.025+0000 7fb1d5ffb640 1 -- 172.21.15.52:0/3269973034 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 16) v1 ==== 136756+0+0 (secure 0 0 0) 0x7fb1c40109d0 con 0x7fb1d81027e0 2024-03-04T09:28:39.025 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:39.026+0000 7fb1d5ffb640 1 --2- 172.21.15.52:0/3269973034 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7fb1c80a1d80 0x7fb1c80a4240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:28:39.025 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:39.026+0000 7fb1dcff8640 1 --2- 172.21.15.52:0/3269973034 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7fb1c80a1d80 0x7fb1c80a4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:28:39.025 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:39.026+0000 7fb1d5ffb640 1 -- 172.21.15.52:0/3269973034 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8277+0+0 (secure 0 0 0) 0x7fb1c40cd9d0 con 0x7fb1d81027e0 2024-03-04T09:28:39.025 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:39.026+0000 7fb1dcff8640 1 --2- 172.21.15.52:0/3269973034 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7fb1c80a1d80 0x7fb1c80a4240 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fb1d8102640 tx=0x7fb1cc00b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:28:39.027 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:39.030+0000 7fb1d5ffb640 1 -- 172.21.15.52:0/3269973034 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fb1c4009de0 con 0x7fb1d81027e0 2024-03-04T09:28:39.209 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:39.212+0000 7fb1df25c640 1 -- 172.21.15.52:0/3269973034 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fb1d802cff0 con 0x7fb1d81027e0 2024-03-04T09:28:39.225 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:28:38 smithi106 ceph-mon[36484]: pgmap v282: 97 pgs: 23 creating+peering, 2 active+clean, 72 unknown; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-04T09:28:39.225 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:28:38 smithi106 ceph-mon[36484]: Health check failed: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-04T09:28:39.226 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:28:38 smithi106 ceph-mon[36484]: osdmap e84: 12 total, 12 up, 12 in 2024-03-04T09:28:39.226 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:28:38 smithi106 ceph-mon[36484]: from='client.? 172.21.15.52:0/2808420848' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-03-04T09:28:39.226 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:28:38 smithi106 ceph-mon[36484]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-03-04T09:28:39.226 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:28:38 smithi106 ceph-mon[36484]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-03-04T09:28:39.226 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:28:38 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:28:39.226 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:28:38 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:28:39.226 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:28:38 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:28:39.226 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:28:38 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:28:39.226 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:28:38 smithi106 ceph-mon[36484]: from='client.? 172.21.15.52:0/4114867410' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-03-04T09:28:39.226 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:28:38 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:28:39.226 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:28:38 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:28:39.226 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:28:38 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T09:28:39.226 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:28:38 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T09:28:39.227 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:28:38 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:28:39.227 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:28:38 smithi106 ceph-mon[36484]: from='client.? 172.21.15.52:0/4114867410' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-03-04T09:28:39.227 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:28:38 smithi106 ceph-mon[36484]: osdmap e85: 12 total, 12 up, 12 in 2024-03-04T09:28:39.335 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:28:38 smithi052 ceph-mon[32473]: pgmap v282: 97 pgs: 23 creating+peering, 2 active+clean, 72 unknown; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-04T09:28:39.335 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:28:38 smithi052 ceph-mon[32473]: Health check failed: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-04T09:28:39.335 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:28:38 smithi052 ceph-mon[32473]: osdmap e84: 12 total, 12 up, 12 in 2024-03-04T09:28:39.335 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:28:38 smithi052 ceph-mon[32473]: from='client.? 172.21.15.52:0/2808420848' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-03-04T09:28:39.335 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:28:38 smithi052 ceph-mon[32473]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-03-04T09:28:39.335 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:28:38 smithi052 ceph-mon[32473]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-03-04T09:28:39.335 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:28:38 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:28:39.336 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:28:38 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:28:39.336 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:28:38 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:28:39.336 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:28:38 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:28:39.336 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:28:38 smithi052 ceph-mon[32473]: from='client.? 172.21.15.52:0/4114867410' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-03-04T09:28:39.336 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:28:38 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:28:39.336 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:28:38 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:28:39.336 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:28:38 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T09:28:39.336 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:28:38 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T09:28:39.336 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:28:38 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:28:39.336 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:28:38 smithi052 ceph-mon[32473]: from='client.? 172.21.15.52:0/4114867410' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-03-04T09:28:39.336 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:28:38 smithi052 ceph-mon[32473]: osdmap e85: 12 total, 12 up, 12 in 2024-03-04T09:28:39.337 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:28:38 smithi191 ceph-mon[36816]: pgmap v282: 97 pgs: 23 creating+peering, 2 active+clean, 72 unknown; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-04T09:28:39.337 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:28:38 smithi191 ceph-mon[36816]: Health check failed: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-04T09:28:39.337 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:28:38 smithi191 ceph-mon[36816]: osdmap e84: 12 total, 12 up, 12 in 2024-03-04T09:28:39.337 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:28:38 smithi191 ceph-mon[36816]: from='client.? 172.21.15.52:0/2808420848' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-03-04T09:28:39.337 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:28:38 smithi191 ceph-mon[36816]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-03-04T09:28:39.337 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:28:38 smithi191 ceph-mon[36816]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-03-04T09:28:39.337 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:28:38 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:28:39.337 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:28:38 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:28:39.338 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:28:38 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:28:39.338 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:28:38 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:28:39.338 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:28:38 smithi191 ceph-mon[36816]: from='client.? 172.21.15.52:0/4114867410' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-03-04T09:28:39.338 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:28:38 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:28:39.338 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:28:38 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:28:39.338 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:28:38 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T09:28:39.338 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:28:38 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T09:28:39.338 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:28:38 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:28:39.338 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:28:38 smithi191 ceph-mon[36816]: from='client.? 172.21.15.52:0/4114867410' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-03-04T09:28:39.338 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:28:38 smithi191 ceph-mon[36816]: osdmap e85: 12 total, 12 up, 12 in 2024-03-04T09:28:39.631 INFO:journalctl@ceph.mds.b.smithi106.stdout:Mar 04 09:28:39 smithi106 ceph-4e059f08-da08-11ee-95c6-87774f69a715-mds-b[66655]: 2024-03-04T09:28:39.228+0000 7fd99c4b7700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-04T09:28:40.335 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:28:39 smithi052 ceph-mon[32473]: mds.? [v2:172.21.15.106:6838/1995853093,v1:172.21.15.106:6839/1995853093] up:active 2024-03-04T09:28:40.730 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:28:39 smithi052 ceph-mon[32473]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-04T09:28:40.731 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:28:39 smithi052 ceph-mon[32473]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-04T09:28:40.731 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:28:39 smithi052 ceph-mon[32473]: from='client.? 172.21.15.52:0/3269973034' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-03-04T09:28:40.732 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:28:39 smithi106 ceph-mon[36484]: mds.? [v2:172.21.15.106:6838/1995853093,v1:172.21.15.106:6839/1995853093] up:active 2024-03-04T09:28:40.732 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:28:39 smithi106 ceph-mon[36484]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-04T09:28:40.732 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:28:39 smithi106 ceph-mon[36484]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-04T09:28:40.732 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:28:39 smithi106 ceph-mon[36484]: from='client.? 172.21.15.52:0/3269973034' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-03-04T09:28:40.732 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:28:39 smithi191 ceph-mon[36816]: mds.? [v2:172.21.15.106:6838/1995853093,v1:172.21.15.106:6839/1995853093] up:active 2024-03-04T09:28:40.732 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:28:39 smithi191 ceph-mon[36816]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-04T09:28:40.732 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:28:39 smithi191 ceph-mon[36816]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-04T09:28:40.732 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:28:39 smithi191 ceph-mon[36816]: from='client.? 172.21.15.52:0/3269973034' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-03-04T09:28:40.919 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:40.921+0000 7fb1d5ffb640 1 -- 172.21.15.52:0/3269973034 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data_ec' created v87) v1 ==== 228+0+0 (secure 0 0 0) 0x7fb1c409a780 con 0x7fb1d81027e0 2024-03-04T09:28:40.919 INFO:teuthology.orchestra.run.smithi052.stderr:pool 'cephfs_data_ec' created 2024-03-04T09:28:40.927 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:40.925+0000 7fb1df25c640 1 -- 172.21.15.52:0/3269973034 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7fb1c80a1d80 msgr2=0x7fb1c80a4240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:28:40.927 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:40.925+0000 7fb1df25c640 1 --2- 172.21.15.52:0/3269973034 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7fb1c80a1d80 0x7fb1c80a4240 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fb1d8102640 tx=0x7fb1cc00b390 comp rx=0 tx=0).stop 2024-03-04T09:28:40.927 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:40.925+0000 7fb1df25c640 1 -- 172.21.15.52:0/3269973034 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb1d81027e0 msgr2=0x7fb1d8197da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:28:40.928 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:40.925+0000 7fb1df25c640 1 --2- 172.21.15.52:0/3269973034 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb1d81027e0 0x7fb1d8197da0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fb1c400d950 tx=0x7fb1c400de10 comp rx=0 tx=0).stop 2024-03-04T09:28:40.928 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:40.925+0000 7fb1df25c640 1 -- 172.21.15.52:0/3269973034 shutdown_connections 2024-03-04T09:28:40.928 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:40.925+0000 7fb1df25c640 1 --2- 172.21.15.52:0/3269973034 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7fb1c80a1d80 0x7fb1c80a4240 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:40.928 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:40.925+0000 7fb1df25c640 1 --2- 172.21.15.52:0/3269973034 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb1d81031a0 0x7fb1d819ee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:40.928 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:40.925+0000 7fb1df25c640 1 --2- 172.21.15.52:0/3269973034 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb1d81027e0 0x7fb1d8197da0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:40.928 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:40.925+0000 7fb1df25c640 1 --2- 172.21.15.52:0/3269973034 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb1d81015f0 0x7fb1d8197860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:40.928 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:40.925+0000 7fb1df25c640 1 -- 172.21.15.52:0/3269973034 >> 172.21.15.52:0/3269973034 conn(0x7fb1d80fcb80 msgr2=0x7fb1d8105a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:28:40.928 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:40.925+0000 7fb1df25c640 1 -- 172.21.15.52:0/3269973034 shutdown_connections 2024-03-04T09:28:40.928 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:40.925+0000 7fb1df25c640 1 -- 172.21.15.52:0/3269973034 wait complete. 2024-03-04T09:28:40.947 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool set cephfs_data_ec allow_ec_overwrites true 2024-03-04T09:28:41.073 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:41.074+0000 7fb5d493e640 1 -- 172.21.15.52:0/641699732 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb5d0103180 msgr2=0x7fb5d0109a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:28:41.074 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:41.074+0000 7fb5d493e640 1 --2- 172.21.15.52:0/641699732 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb5d0103180 0x7fb5d0109a10 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fb5c8009ca0 tx=0x7fb5c802c0f0 comp rx=0 tx=0).stop 2024-03-04T09:28:41.074 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:41.074+0000 7fb5d493e640 1 -- 172.21.15.52:0/641699732 shutdown_connections 2024-03-04T09:28:41.074 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:41.074+0000 7fb5d493e640 1 --2- 172.21.15.52:0/641699732 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb5d0103180 0x7fb5d0109a10 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:41.074 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:41.074+0000 7fb5d493e640 1 --2- 172.21.15.52:0/641699732 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb5d01027c0 0x7fb5d0102c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:41.074 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:41.074+0000 7fb5d493e640 1 --2- 172.21.15.52:0/641699732 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb5d01015d0 0x7fb5d01019f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:41.074 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:41.074+0000 7fb5d493e640 1 -- 172.21.15.52:0/641699732 >> 172.21.15.52:0/641699732 conn(0x7fb5d00fcb40 msgr2=0x7fb5d00fef80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:28:41.074 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:41.074+0000 7fb5d493e640 1 -- 172.21.15.52:0/641699732 shutdown_connections 2024-03-04T09:28:41.074 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:41.074+0000 7fb5d493e640 1 -- 172.21.15.52:0/641699732 wait complete. 2024-03-04T09:28:41.074 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:41.074+0000 7fb5d493e640 1 Processor -- start 2024-03-04T09:28:41.074 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:41.074+0000 7fb5d493e640 1 -- start start 2024-03-04T09:28:41.075 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:41.075+0000 7fb5d493e640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb5d01015d0 0x7fb5d0075380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:28:41.075 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:41.075+0000 7fb5d493e640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb5d01027c0 0x7fb5d00739c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:28:41.075 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:41.075+0000 7fb5d493e640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb5d0103180 0x7fb5d0073f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:28:41.075 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:41.075+0000 7fb5d493e640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5d010d230 con 0x7fb5d01027c0 2024-03-04T09:28:41.075 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:41.075+0000 7fb5d493e640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5d010d0b0 con 0x7fb5d0103180 2024-03-04T09:28:41.075 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:41.075+0000 7fb5d493e640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5d010d3b0 con 0x7fb5d01015d0 2024-03-04T09:28:41.075 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:41.075+0000 7fb5c7fff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb5d01027c0 0x7fb5d00739c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:28:41.075 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:41.075+0000 7fb5c7fff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb5d01027c0 0x7fb5d00739c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:57834/0 (socket says 172.21.15.52:57834) 2024-03-04T09:28:41.075 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:41.075+0000 7fb5c7fff640 1 -- 172.21.15.52:0/4283631061 learned_addr learned my addr 172.21.15.52:0/4283631061 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-04T09:28:41.075 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:41.075+0000 7fb5ced9d640 1 --2- 172.21.15.52:0/4283631061 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb5d0103180 0x7fb5d0073f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:28:41.076 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:41.075+0000 7fb5ce59c640 1 --2- 172.21.15.52:0/4283631061 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb5d01015d0 0x7fb5d0075380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:28:41.076 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:41.075+0000 7fb5c7fff640 1 -- 172.21.15.52:0/4283631061 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb5d01015d0 msgr2=0x7fb5d0075380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:28:41.076 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:41.075+0000 7fb5c7fff640 1 --2- 172.21.15.52:0/4283631061 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb5d01015d0 0x7fb5d0075380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:41.076 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:41.075+0000 7fb5c7fff640 1 -- 172.21.15.52:0/4283631061 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb5d0103180 msgr2=0x7fb5d0073f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:28:41.076 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:41.075+0000 7fb5c7fff640 1 --2- 172.21.15.52:0/4283631061 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb5d0103180 0x7fb5d0073f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:41.076 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:41.075+0000 7fb5c7fff640 1 -- 172.21.15.52:0/4283631061 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5d0074790 con 0x7fb5d01027c0 2024-03-04T09:28:41.076 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:41.075+0000 7fb5ce59c640 1 --2- 172.21.15.52:0/4283631061 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb5d01015d0 0x7fb5d0075380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T09:28:41.076 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:41.075+0000 7fb5c7fff640 1 --2- 172.21.15.52:0/4283631061 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb5d01027c0 0x7fb5d00739c0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fb5b800c920 tx=0x7fb5b800cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:28:41.076 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:41.075+0000 7fb5c77fe640 1 -- 172.21.15.52:0/4283631061 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5b8007c70 con 0x7fb5d01027c0 2024-03-04T09:28:41.076 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:41.075+0000 7fb5c77fe640 1 -- 172.21.15.52:0/4283631061 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb5b8007e10 con 0x7fb5d01027c0 2024-03-04T09:28:41.076 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:41.075+0000 7fb5d493e640 1 -- 172.21.15.52:0/4283631061 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5d01a1900 con 0x7fb5d01027c0 2024-03-04T09:28:41.076 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:41.075+0000 7fb5c77fe640 1 -- 172.21.15.52:0/4283631061 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5b8018890 con 0x7fb5d01027c0 2024-03-04T09:28:41.076 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:41.075+0000 7fb5d493e640 1 -- 172.21.15.52:0/4283631061 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5d01a1e50 con 0x7fb5d01027c0 2024-03-04T09:28:41.077 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:41.076+0000 7fb5d493e640 1 -- 172.21.15.52:0/4283631061 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb5d0108890 con 0x7fb5d01027c0 2024-03-04T09:28:41.077 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:41.077+0000 7fb5c77fe640 1 -- 172.21.15.52:0/4283631061 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 16) v1 ==== 136756+0+0 (secure 0 0 0) 0x7fb5b8018a50 con 0x7fb5d01027c0 2024-03-04T09:28:41.077 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:41.078+0000 7fb5c77fe640 1 --2- 172.21.15.52:0/4283631061 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7fb5bc0a1d80 0x7fb5bc0a4240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:28:41.078 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:41.078+0000 7fb5ce59c640 1 --2- 172.21.15.52:0/4283631061 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7fb5bc0a1d80 0x7fb5bc0a4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:28:41.078 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:41.078+0000 7fb5c77fe640 1 -- 172.21.15.52:0/4283631061 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8770+0+0 (secure 0 0 0) 0x7fb5b80ce410 con 0x7fb5d01027c0 2024-03-04T09:28:41.078 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:41.078+0000 7fb5ce59c640 1 --2- 172.21.15.52:0/4283631061 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7fb5bc0a1d80 0x7fb5bc0a4240 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fb5d0102620 tx=0x7fb5c0005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:28:41.078 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:41.082+0000 7fb5c77fe640 1 -- 172.21.15.52:0/4283631061 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fb5b8014030 con 0x7fb5d01027c0 2024-03-04T09:28:41.263 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:41.264+0000 7fb5d493e640 1 -- 172.21.15.52:0/4283631061 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"} v 0) v1 -- 0x7fb5d002cff0 con 0x7fb5d01027c0 2024-03-04T09:28:41.335 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:28:40 smithi052 ceph-mon[32473]: pgmap v285: 97 pgs: 23 creating+peering, 15 active+clean, 59 unknown; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 0 op/s 2024-03-04T09:28:41.335 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:28:40 smithi052 ceph-mon[32473]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-03-04T09:28:41.335 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:28:40 smithi052 ceph-mon[32473]: Cluster is now healthy 2024-03-04T09:28:41.335 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:28:40 smithi052 ceph-mon[32473]: osdmap e86: 12 total, 12 up, 12 in 2024-03-04T09:28:41.336 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:28:40 smithi191 ceph-mon[36816]: pgmap v285: 97 pgs: 23 creating+peering, 15 active+clean, 59 unknown; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 0 op/s 2024-03-04T09:28:41.336 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:28:40 smithi191 ceph-mon[36816]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-03-04T09:28:41.337 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:28:40 smithi191 ceph-mon[36816]: Cluster is now healthy 2024-03-04T09:28:41.337 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:28:40 smithi191 ceph-mon[36816]: osdmap e86: 12 total, 12 up, 12 in 2024-03-04T09:28:41.344 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:28:40 smithi106 ceph-mon[36484]: pgmap v285: 97 pgs: 23 creating+peering, 15 active+clean, 59 unknown; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 0 op/s 2024-03-04T09:28:41.344 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:28:40 smithi106 ceph-mon[36484]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-03-04T09:28:41.344 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:28:40 smithi106 ceph-mon[36484]: Cluster is now healthy 2024-03-04T09:28:41.344 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:28:40 smithi106 ceph-mon[36484]: osdmap e86: 12 total, 12 up, 12 in 2024-03-04T09:28:41.924 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:41.927+0000 7fb5c77fe640 1 -- 172.21.15.52:0/4283631061 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]=0 set pool 4 allow_ec_overwrites to true v88) v1 ==== 169+0+0 (secure 0 0 0) 0x7fb5b809b6b0 con 0x7fb5d01027c0 2024-03-04T09:28:41.938 INFO:teuthology.orchestra.run.smithi052.stderr:set pool 4 allow_ec_overwrites to true 2024-03-04T09:28:41.939 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:41.931+0000 7fb5d493e640 1 -- 172.21.15.52:0/4283631061 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7fb5bc0a1d80 msgr2=0x7fb5bc0a4240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:28:41.939 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:41.931+0000 7fb5d493e640 1 --2- 172.21.15.52:0/4283631061 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7fb5bc0a1d80 0x7fb5bc0a4240 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fb5d0102620 tx=0x7fb5c0005f90 comp rx=0 tx=0).stop 2024-03-04T09:28:41.939 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:41.931+0000 7fb5d493e640 1 -- 172.21.15.52:0/4283631061 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb5d01027c0 msgr2=0x7fb5d00739c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:28:41.939 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:41.931+0000 7fb5d493e640 1 --2- 172.21.15.52:0/4283631061 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb5d01027c0 0x7fb5d00739c0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fb5b800c920 tx=0x7fb5b800cde0 comp rx=0 tx=0).stop 2024-03-04T09:28:41.939 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:41.931+0000 7fb5d493e640 1 -- 172.21.15.52:0/4283631061 shutdown_connections 2024-03-04T09:28:41.939 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:41.931+0000 7fb5d493e640 1 --2- 172.21.15.52:0/4283631061 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7fb5bc0a1d80 0x7fb5bc0a4240 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:41.939 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:41.931+0000 7fb5d493e640 1 --2- 172.21.15.52:0/4283631061 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb5d0103180 0x7fb5d0073f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:41.939 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:41.931+0000 7fb5d493e640 1 --2- 172.21.15.52:0/4283631061 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb5d01027c0 0x7fb5d00739c0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:41.939 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:41.931+0000 7fb5d493e640 1 --2- 172.21.15.52:0/4283631061 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb5d01015d0 0x7fb5d0075380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:41.939 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:41.931+0000 7fb5d493e640 1 -- 172.21.15.52:0/4283631061 >> 172.21.15.52:0/4283631061 conn(0x7fb5d00fcb40 msgr2=0x7fb5d0105a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:28:41.939 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:41.931+0000 7fb5d493e640 1 -- 172.21.15.52:0/4283631061 shutdown_connections 2024-03-04T09:28:41.939 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:41.931+0000 7fb5d493e640 1 -- 172.21.15.52:0/4283631061 wait complete. 2024-03-04T09:28:41.950 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs add_data_pool cephfs cephfs_data_ec 2024-03-04T09:28:42.063 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:42.064+0000 7f1c79847640 1 -- 172.21.15.52:0/1516554121 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1c74101110 msgr2=0x7f1c741089b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:28:42.063 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:42.064+0000 7f1c79847640 1 --2- 172.21.15.52:0/1516554121 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1c74101110 0x7f1c741089b0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f1c6c009ca0 tx=0x7f1c6c02c0f0 comp rx=0 tx=0).stop 2024-03-04T09:28:42.063 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:42.064+0000 7f1c79847640 1 -- 172.21.15.52:0/1516554121 shutdown_connections 2024-03-04T09:28:42.063 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:42.064+0000 7f1c79847640 1 --2- 172.21.15.52:0/1516554121 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1c74101110 0x7f1c741089b0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:42.064 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:42.064+0000 7f1c79847640 1 --2- 172.21.15.52:0/1516554121 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1c74100750 0x7f1c74100bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:42.064 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:42.064+0000 7f1c79847640 1 --2- 172.21.15.52:0/1516554121 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1c74101b00 0x7f1c74101f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:42.064 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:42.064+0000 7f1c79847640 1 -- 172.21.15.52:0/1516554121 >> 172.21.15.52:0/1516554121 conn(0x7f1c740fb200 msgr2=0x7f1c740fd620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:28:42.064 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:42.064+0000 7f1c79847640 1 -- 172.21.15.52:0/1516554121 shutdown_connections 2024-03-04T09:28:42.064 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:42.064+0000 7f1c79847640 1 -- 172.21.15.52:0/1516554121 wait complete. 2024-03-04T09:28:42.064 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:42.064+0000 7f1c79847640 1 Processor -- start 2024-03-04T09:28:42.064 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:42.064+0000 7f1c79847640 1 -- start start 2024-03-04T09:28:42.064 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:42.065+0000 7f1c79847640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1c74100750 0x7f1c741966e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:28:42.064 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:42.065+0000 7f1c79847640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1c74101110 0x7f1c74196c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:28:42.064 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:42.065+0000 7f1c79847640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1c74101b00 0x7f1c7419dc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:28:42.064 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:42.065+0000 7f1c79847640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c7410c1a0 con 0x7f1c74101110 2024-03-04T09:28:42.064 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:42.065+0000 7f1c79847640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c7410c020 con 0x7f1c74101b00 2024-03-04T09:28:42.065 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:42.065+0000 7f1c79847640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c7410c320 con 0x7f1c74100750 2024-03-04T09:28:42.065 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:42.065+0000 7f1c6a7fc640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1c74101110 0x7f1c74196c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:28:42.065 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:42.065+0000 7f1c6a7fc640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1c74101110 0x7f1c74196c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:57852/0 (socket says 172.21.15.52:57852) 2024-03-04T09:28:42.065 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:42.065+0000 7f1c6a7fc640 1 -- 172.21.15.52:0/1387136786 learned_addr learned my addr 172.21.15.52:0/1387136786 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-04T09:28:42.065 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:42.065+0000 7f1c737fe640 1 --2- 172.21.15.52:0/1387136786 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1c74101b00 0x7f1c7419dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:28:42.065 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:42.065+0000 7f1c72ffd640 1 --2- 172.21.15.52:0/1387136786 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1c74100750 0x7f1c741966e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:28:42.065 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:42.065+0000 7f1c6a7fc640 1 -- 172.21.15.52:0/1387136786 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1c74100750 msgr2=0x7f1c741966e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:28:42.065 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:42.065+0000 7f1c6a7fc640 1 --2- 172.21.15.52:0/1387136786 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1c74100750 0x7f1c741966e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:42.065 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:42.065+0000 7f1c6a7fc640 1 -- 172.21.15.52:0/1387136786 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1c74101b00 msgr2=0x7f1c7419dc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:28:42.065 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:42.065+0000 7f1c6a7fc640 1 --2- 172.21.15.52:0/1387136786 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1c74101b00 0x7f1c7419dc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:42.065 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:42.065+0000 7f1c6a7fc640 1 -- 172.21.15.52:0/1387136786 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c7419e300 con 0x7f1c74101110 2024-03-04T09:28:42.066 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:42.065+0000 7f1c6a7fc640 1 --2- 172.21.15.52:0/1387136786 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1c74101110 0x7f1c74196c20 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f1c5c00e9c0 tx=0x7f1c5c00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:28:42.066 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:42.065+0000 7f1c78845640 1 -- 172.21.15.52:0/1387136786 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c5c00cd80 con 0x7f1c74101110 2024-03-04T09:28:42.066 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:42.065+0000 7f1c79847640 1 -- 172.21.15.52:0/1387136786 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c7419e5f0 con 0x7f1c74101110 2024-03-04T09:28:42.066 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:42.065+0000 7f1c78845640 1 -- 172.21.15.52:0/1387136786 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1c5c0035f0 con 0x7f1c74101110 2024-03-04T09:28:42.066 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:42.065+0000 7f1c78845640 1 -- 172.21.15.52:0/1387136786 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c5c010690 con 0x7f1c74101110 2024-03-04T09:28:42.066 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:42.065+0000 7f1c79847640 1 -- 172.21.15.52:0/1387136786 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c7419eb40 con 0x7f1c74101110 2024-03-04T09:28:42.066 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:42.066+0000 7f1c79847640 1 -- 172.21.15.52:0/1387136786 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1c74107830 con 0x7f1c74101110 2024-03-04T09:28:42.067 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:42.068+0000 7f1c78845640 1 -- 172.21.15.52:0/1387136786 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 16) v1 ==== 136756+0+0 (secure 0 0 0) 0x7f1c5c0037a0 con 0x7f1c74101110 2024-03-04T09:28:42.067 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:42.068+0000 7f1c78845640 1 --2- 172.21.15.52:0/1387136786 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f1c600a1d80 0x7f1c600a4240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:28:42.067 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:42.068+0000 7f1c72ffd640 1 --2- 172.21.15.52:0/1387136786 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f1c600a1d80 0x7f1c600a4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:28:42.067 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:42.068+0000 7f1c78845640 1 -- 172.21.15.52:0/1387136786 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8770+0+0 (secure 0 0 0) 0x7f1c5c0ce430 con 0x7f1c74101110 2024-03-04T09:28:42.067 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:42.069+0000 7f1c72ffd640 1 --2- 172.21.15.52:0/1387136786 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f1c600a1d80 0x7f1c600a4240 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f1c74069a00 tx=0x7f1c64005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:28:42.069 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:42.072+0000 7f1c78845640 1 -- 172.21.15.52:0/1387136786 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f1c5c09b6d0 con 0x7f1c74101110 2024-03-04T09:28:42.290 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:42.293+0000 7f1c79847640 1 -- 172.21.15.52:0/1387136786 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"} v 0) v1 -- 0x7f1c7402cff0 con 0x7f1c74101110 2024-03-04T09:28:42.335 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:28:41 smithi052 ceph-mon[32473]: from='client.? 172.21.15.52:0/3269973034' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-03-04T09:28:42.335 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:28:41 smithi052 ceph-mon[32473]: osdmap e87: 12 total, 12 up, 12 in 2024-03-04T09:28:42.335 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:28:41 smithi052 ceph-mon[32473]: from='client.? 172.21.15.52:0/4283631061' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-03-04T09:28:42.336 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:28:41 smithi191 ceph-mon[36816]: from='client.? 172.21.15.52:0/3269973034' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-03-04T09:28:42.336 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:28:41 smithi191 ceph-mon[36816]: osdmap e87: 12 total, 12 up, 12 in 2024-03-04T09:28:42.336 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:28:41 smithi191 ceph-mon[36816]: from='client.? 172.21.15.52:0/4283631061' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-03-04T09:28:42.381 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:28:41 smithi106 ceph-mon[36484]: from='client.? 172.21.15.52:0/3269973034' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-03-04T09:28:42.381 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:28:41 smithi106 ceph-mon[36484]: osdmap e87: 12 total, 12 up, 12 in 2024-03-04T09:28:42.381 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:28:41 smithi106 ceph-mon[36484]: from='client.? 172.21.15.52:0/4283631061' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-03-04T09:28:42.932 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:42.935+0000 7f1c78845640 1 -- 172.21.15.52:0/1387136786 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]=0 added data pool 4 to fsmap v17) v1 ==== 137+0+0 (secure 0 0 0) 0x7f1c5c09afd0 con 0x7f1c74101110 2024-03-04T09:28:42.932 INFO:teuthology.orchestra.run.smithi052.stderr:added data pool 4 to fsmap 2024-03-04T09:28:42.935 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:42.938+0000 7f1c79847640 1 -- 172.21.15.52:0/1387136786 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f1c600a1d80 msgr2=0x7f1c600a4240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:28:42.936 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:42.938+0000 7f1c79847640 1 --2- 172.21.15.52:0/1387136786 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f1c600a1d80 0x7f1c600a4240 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f1c74069a00 tx=0x7f1c64005f90 comp rx=0 tx=0).stop 2024-03-04T09:28:42.936 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:42.938+0000 7f1c79847640 1 -- 172.21.15.52:0/1387136786 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1c74101110 msgr2=0x7f1c74196c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:28:42.936 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:42.938+0000 7f1c79847640 1 --2- 172.21.15.52:0/1387136786 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1c74101110 0x7f1c74196c20 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f1c5c00e9c0 tx=0x7f1c5c00ee80 comp rx=0 tx=0).stop 2024-03-04T09:28:42.936 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:42.938+0000 7f1c79847640 1 -- 172.21.15.52:0/1387136786 shutdown_connections 2024-03-04T09:28:42.936 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:42.938+0000 7f1c79847640 1 --2- 172.21.15.52:0/1387136786 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f1c600a1d80 0x7f1c600a4240 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:42.936 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:42.938+0000 7f1c79847640 1 --2- 172.21.15.52:0/1387136786 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1c74101b00 0x7f1c7419dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:42.936 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:42.938+0000 7f1c79847640 1 --2- 172.21.15.52:0/1387136786 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1c74101110 0x7f1c74196c20 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:42.936 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:42.938+0000 7f1c79847640 1 --2- 172.21.15.52:0/1387136786 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1c74100750 0x7f1c741966e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:42.936 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:42.938+0000 7f1c79847640 1 -- 172.21.15.52:0/1387136786 >> 172.21.15.52:0/1387136786 conn(0x7f1c740fb200 msgr2=0x7f1c74104a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:28:42.936 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:42.938+0000 7f1c79847640 1 -- 172.21.15.52:0/1387136786 shutdown_connections 2024-03-04T09:28:42.936 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:42.938+0000 7f1c79847640 1 -- 172.21.15.52:0/1387136786 wait complete. 2024-03-04T09:28:42.948 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T09:28:43.066 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:43.066+0000 7f1267066640 1 -- 172.21.15.52:0/3865963064 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1260102810 msgr2=0x7f1260102c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:28:43.066 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:43.066+0000 7f1267066640 1 --2- 172.21.15.52:0/3865963064 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1260102810 0x7f1260102c90 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f1254009900 tx=0x7f125402bc30 comp rx=0 tx=0).stop 2024-03-04T09:28:43.066 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:43.067+0000 7f1267066640 1 -- 172.21.15.52:0/3865963064 shutdown_connections 2024-03-04T09:28:43.066 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:43.067+0000 7f1267066640 1 --2- 172.21.15.52:0/3865963064 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f12601031d0 0x7f1260109a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:43.066 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:43.067+0000 7f1267066640 1 --2- 172.21.15.52:0/3865963064 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1260102810 0x7f1260102c90 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:43.066 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:43.067+0000 7f1267066640 1 --2- 172.21.15.52:0/3865963064 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1260101620 0x7f1260101a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:43.066 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:43.067+0000 7f1267066640 1 -- 172.21.15.52:0/3865963064 >> 172.21.15.52:0/3865963064 conn(0x7f12600fcb70 msgr2=0x7f12600fefd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:28:43.067 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:43.067+0000 7f1267066640 1 -- 172.21.15.52:0/3865963064 shutdown_connections 2024-03-04T09:28:43.067 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:43.067+0000 7f1267066640 1 -- 172.21.15.52:0/3865963064 wait complete. 2024-03-04T09:28:43.067 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:43.067+0000 7f1267066640 1 Processor -- start 2024-03-04T09:28:43.067 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:43.067+0000 7f1267066640 1 -- start start 2024-03-04T09:28:43.067 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:43.067+0000 7f1267066640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1260101620 0x7f12601979a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:28:43.067 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:43.067+0000 7f1267066640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1260102810 0x7f1260197ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:28:43.067 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:43.067+0000 7f1267066640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f12601031d0 0x7f126019ef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:28:43.067 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:43.067+0000 7f1267066640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f126010d300 con 0x7f12601031d0 2024-03-04T09:28:43.067 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:43.067+0000 7f1267066640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f126010d180 con 0x7f1260101620 2024-03-04T09:28:43.067 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:43.067+0000 7f1267066640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f126010d480 con 0x7f1260102810 2024-03-04T09:28:43.067 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:43.067+0000 7f1265603640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f12601031d0 0x7f126019ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:28:43.067 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:43.067+0000 7f1265603640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f12601031d0 0x7f126019ef50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:57866/0 (socket says 172.21.15.52:57866) 2024-03-04T09:28:43.067 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:43.067+0000 7f1265603640 1 -- 172.21.15.52:0/3174265828 learned_addr learned my addr 172.21.15.52:0/3174265828 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-04T09:28:43.068 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:43.068+0000 7f1264e02640 1 --2- 172.21.15.52:0/3174265828 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1260101620 0x7f12601979a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:28:43.069 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:43.068+0000 7f125ffff640 1 --2- 172.21.15.52:0/3174265828 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1260102810 0x7f1260197ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:28:43.069 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:43.068+0000 7f1265603640 1 -- 172.21.15.52:0/3174265828 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1260102810 msgr2=0x7f1260197ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:28:43.069 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:43.068+0000 7f1265603640 1 --2- 172.21.15.52:0/3174265828 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1260102810 0x7f1260197ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:43.069 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:43.068+0000 7f1265603640 1 -- 172.21.15.52:0/3174265828 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1260101620 msgr2=0x7f12601979a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:28:43.069 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:43.068+0000 7f1265603640 1 --2- 172.21.15.52:0/3174265828 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1260101620 0x7f12601979a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:43.069 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:43.068+0000 7f1265603640 1 -- 172.21.15.52:0/3174265828 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f126019f5c0 con 0x7f12601031d0 2024-03-04T09:28:43.069 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:43.068+0000 7f1264e02640 1 --2- 172.21.15.52:0/3174265828 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1260101620 0x7f12601979a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-04T09:28:43.069 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:43.068+0000 7f1265603640 1 --2- 172.21.15.52:0/3174265828 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f12601031d0 0x7f126019ef50 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f1250009fd0 tx=0x7f125000b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:28:43.069 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:43.068+0000 7f125dffb640 1 -- 172.21.15.52:0/3174265828 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1250011830 con 0x7f12601031d0 2024-03-04T09:28:43.070 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:43.068+0000 7f1267066640 1 -- 172.21.15.52:0/3174265828 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f126019f8b0 con 0x7f12601031d0 2024-03-04T09:28:43.070 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:43.068+0000 7f125dffb640 1 -- 172.21.15.52:0/3174265828 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1250016070 con 0x7f12601031d0 2024-03-04T09:28:43.070 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:43.068+0000 7f125dffb640 1 -- 172.21.15.52:0/3174265828 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f125000f2e0 con 0x7f12601031d0 2024-03-04T09:28:43.070 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:43.068+0000 7f1267066640 1 -- 172.21.15.52:0/3174265828 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f126019fe00 con 0x7f12601031d0 2024-03-04T09:28:43.071 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:43.070+0000 7f1267066640 1 -- 172.21.15.52:0/3174265828 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1260108940 con 0x7f12601031d0 2024-03-04T09:28:43.071 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:43.070+0000 7f125dffb640 1 -- 172.21.15.52:0/3174265828 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 16) v1 ==== 136756+0+0 (secure 0 0 0) 0x7f12500119d0 con 0x7f12601031d0 2024-03-04T09:28:43.071 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:43.071+0000 7f125dffb640 1 --2- 172.21.15.52:0/3174265828 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f12480a1d80 0x7f12480a4240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:28:43.071 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:43.071+0000 7f125dffb640 1 -- 172.21.15.52:0/3174265828 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8802+0+0 (secure 0 0 0) 0x7f12500cd6a0 con 0x7f12601031d0 2024-03-04T09:28:43.071 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:43.071+0000 7f1264e02640 1 --2- 172.21.15.52:0/3174265828 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f12480a1d80 0x7f12480a4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:28:43.071 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:43.072+0000 7f1264e02640 1 --2- 172.21.15.52:0/3174265828 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f12480a1d80 0x7f12480a4240 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f1260102670 tx=0x7f1258009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:28:43.072 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:43.076+0000 7f125dffb640 1 -- 172.21.15.52:0/3174265828 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f125009a890 con 0x7f12601031d0 2024-03-04T09:28:43.293 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:43.296+0000 7f1267066640 1 -- 172.21.15.52:0/3174265828 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f12600621d0 con 0x7f12601031d0 2024-03-04T09:28:43.293 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:43.297+0000 7f125dffb640 1 -- 172.21.15.52:0/3174265828 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+11333 (secure 0 0 0) 0x7f125009a190 con 0x7f12601031d0 2024-03-04T09:28:43.294 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-03-04T09:28:43.294 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":17,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6835/1007961019","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6834","nonce":1007961019},{"type":"v1","addr":"172.21.15.52:6835","nonce":1007961019}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6837/3753881283","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6836","nonce":3753881283},{"type":"v1","addr":"172.21.15.52:6837","nonce":3753881283}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14628,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6839/3467897808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6838","nonce":3467897808},{"type":"v1","addr":"172.21.15.52:6839","nonce":3467897808}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14640,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6841/2804292291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6840","nonce":2804292291},{"type":"v1","addr":"172.21.15.52:6841","nonce":2804292291}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24407,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6833/1818480323","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6832","nonce":1818480323},{"type":"v1","addr":"172.21.15.106:6833","nonce":1818480323}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24412,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6833/765768204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":765768204},{"type":"v1","addr":"172.21.15.191:6833","nonce":765768204}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24440,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6835/1541069897","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6834","nonce":1541069897},{"type":"v1","addr":"172.21.15.106:6835","nonce":1541069897}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24451,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6835/105048699","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":105048699},{"type":"v1","addr":"172.21.15.191:6835","nonce":105048699}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24463,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6837/4071064225","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6836","nonce":4071064225},{"type":"v1","addr":"172.21.15.191:6837","nonce":4071064225}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24473,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6837/2901755066","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6836","nonce":2901755066},{"type":"v1","addr":"172.21.15.106:6837","nonce":2901755066}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":24490,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6839/2194324891","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6838","nonce":2194324891},{"type":"v1","addr":"172.21.15.191:6839","nonce":2194324891}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T09:28:37.264970+0000","modified":"2024-03-04T09:28:42.295443+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":24506},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24506":{"gid":24506,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.106:6839/1995853093","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6838","nonce":1995853093},{"type":"v1","addr":"172.21.15.106:6839","nonce":1995853093}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":24506,"qdb_cluster":[24506]},"id":1}]} 2024-03-04T09:28:43.294 INFO:teuthology.orchestra.run.smithi052.stderr:dumped fsmap epoch 17 2024-03-04T09:28:43.296 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:43.299+0000 7f1267066640 1 -- 172.21.15.52:0/3174265828 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f12480a1d80 msgr2=0x7f12480a4240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:28:43.296 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:43.299+0000 7f1267066640 1 --2- 172.21.15.52:0/3174265828 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f12480a1d80 0x7f12480a4240 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f1260102670 tx=0x7f1258009500 comp rx=0 tx=0).stop 2024-03-04T09:28:43.296 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:43.299+0000 7f1267066640 1 -- 172.21.15.52:0/3174265828 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f12601031d0 msgr2=0x7f126019ef50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:28:43.296 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:43.299+0000 7f1267066640 1 --2- 172.21.15.52:0/3174265828 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f12601031d0 0x7f126019ef50 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f1250009fd0 tx=0x7f125000b8e0 comp rx=0 tx=0).stop 2024-03-04T09:28:43.296 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:43.299+0000 7f1267066640 1 -- 172.21.15.52:0/3174265828 shutdown_connections 2024-03-04T09:28:43.296 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:43.299+0000 7f1267066640 1 --2- 172.21.15.52:0/3174265828 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f12480a1d80 0x7f12480a4240 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:43.297 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:43.299+0000 7f1267066640 1 --2- 172.21.15.52:0/3174265828 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f12601031d0 0x7f126019ef50 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:43.297 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:43.299+0000 7f1267066640 1 --2- 172.21.15.52:0/3174265828 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1260102810 0x7f1260197ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:43.297 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:43.299+0000 7f1267066640 1 --2- 172.21.15.52:0/3174265828 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1260101620 0x7f12601979a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:43.297 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:43.299+0000 7f1267066640 1 -- 172.21.15.52:0/3174265828 >> 172.21.15.52:0/3174265828 conn(0x7f12600fcb70 msgr2=0x7f1260105b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:28:43.297 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:43.299+0000 7f1267066640 1 -- 172.21.15.52:0/3174265828 shutdown_connections 2024-03-04T09:28:43.297 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:43.299+0000 7f1267066640 1 -- 172.21.15.52:0/3174265828 wait complete. 2024-03-04T09:28:43.306 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:28:42 smithi052 ceph-mon[32473]: pgmap v288: 129 pgs: 27 creating+peering, 81 active+clean, 21 unknown; 579 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s wr, 5 op/s 2024-03-04T09:28:43.306 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:28:42 smithi052 ceph-mon[32473]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-04T09:28:43.306 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:28:42 smithi052 ceph-mon[32473]: from='client.? 172.21.15.52:0/4283631061' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-03-04T09:28:43.306 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:28:42 smithi052 ceph-mon[32473]: osdmap e88: 12 total, 12 up, 12 in 2024-03-04T09:28:43.306 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:28:42 smithi052 ceph-mon[32473]: from='client.? 172.21.15.52:0/1387136786' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-03-04T09:28:43.307 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-04T09:28:43.336 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:28:42 smithi191 ceph-mon[36816]: pgmap v288: 129 pgs: 27 creating+peering, 81 active+clean, 21 unknown; 579 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s wr, 5 op/s 2024-03-04T09:28:43.336 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:28:42 smithi191 ceph-mon[36816]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-04T09:28:43.336 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:28:42 smithi191 ceph-mon[36816]: from='client.? 172.21.15.52:0/4283631061' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-03-04T09:28:43.336 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:28:42 smithi191 ceph-mon[36816]: osdmap e88: 12 total, 12 up, 12 in 2024-03-04T09:28:43.337 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:28:42 smithi191 ceph-mon[36816]: from='client.? 172.21.15.52:0/1387136786' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-03-04T09:28:43.381 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:28:42 smithi106 ceph-mon[36484]: pgmap v288: 129 pgs: 27 creating+peering, 81 active+clean, 21 unknown; 579 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s wr, 5 op/s 2024-03-04T09:28:43.381 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:28:42 smithi106 ceph-mon[36484]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-04T09:28:43.381 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:28:42 smithi106 ceph-mon[36484]: from='client.? 172.21.15.52:0/4283631061' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-03-04T09:28:43.381 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:28:42 smithi106 ceph-mon[36484]: osdmap e88: 12 total, 12 up, 12 in 2024-03-04T09:28:43.382 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:28:42 smithi106 ceph-mon[36484]: from='client.? 172.21.15.52:0/1387136786' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-03-04T09:28:43.454 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:43.455+0000 7f3ae909a640 1 -- 172.21.15.52:0/194998567 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3ae40ff8c0 msgr2=0x7f3ae40ffce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:28:43.454 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:43.455+0000 7f3ae909a640 1 --2- 172.21.15.52:0/194998567 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3ae40ff8c0 0x7f3ae40ffce0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f3ad4009930 tx=0x7f3ad402bc30 comp rx=0 tx=0).stop 2024-03-04T09:28:43.454 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:43.455+0000 7f3ae909a640 1 -- 172.21.15.52:0/194998567 shutdown_connections 2024-03-04T09:28:43.455 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:43.455+0000 7f3ae909a640 1 --2- 172.21.15.52:0/194998567 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3ae4103250 0x7f3ae4109ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:43.455 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:43.455+0000 7f3ae909a640 1 --2- 172.21.15.52:0/194998567 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3ae4102890 0x7f3ae4102d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:43.455 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:43.455+0000 7f3ae909a640 1 --2- 172.21.15.52:0/194998567 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3ae40ff8c0 0x7f3ae40ffce0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:43.455 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:43.455+0000 7f3ae909a640 1 -- 172.21.15.52:0/194998567 >> 172.21.15.52:0/194998567 conn(0x7f3ae40fb1f0 msgr2=0x7f3ae40fd630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:28:43.455 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:43.455+0000 7f3ae909a640 1 -- 172.21.15.52:0/194998567 shutdown_connections 2024-03-04T09:28:43.455 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:43.455+0000 7f3ae909a640 1 -- 172.21.15.52:0/194998567 wait complete. 2024-03-04T09:28:43.455 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:43.456+0000 7f3ae909a640 1 Processor -- start 2024-03-04T09:28:43.455 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:43.456+0000 7f3ae909a640 1 -- start start 2024-03-04T09:28:43.455 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:43.456+0000 7f3ae909a640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3ae40ff8c0 0x7f3ae41021b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:28:43.455 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:43.456+0000 7f3ae909a640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3ae4102890 0x7f3ae4100840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:28:43.455 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:43.456+0000 7f3ae909a640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3ae4103250 0x7f3ae4100d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:28:43.455 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:43.456+0000 7f3ae909a640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ae410d290 con 0x7f3ae4103250 2024-03-04T09:28:43.455 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:43.456+0000 7f3ae909a640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ae410d110 con 0x7f3ae4102890 2024-03-04T09:28:43.456 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:43.456+0000 7f3ae909a640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ae410d410 con 0x7f3ae40ff8c0 2024-03-04T09:28:43.456 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:43.456+0000 7f3ae359e640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3ae4103250 0x7f3ae4100d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:28:43.456 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:43.456+0000 7f3ae359e640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3ae4103250 0x7f3ae4100d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:57894/0 (socket says 172.21.15.52:57894) 2024-03-04T09:28:43.456 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:43.456+0000 7f3ae359e640 1 -- 172.21.15.52:0/1624286884 learned_addr learned my addr 172.21.15.52:0/1624286884 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-04T09:28:43.456 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:43.456+0000 7f3ae259c640 1 --2- 172.21.15.52:0/1624286884 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3ae4102890 0x7f3ae4100840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:28:43.456 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:43.456+0000 7f3ae2d9d640 1 --2- 172.21.15.52:0/1624286884 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3ae40ff8c0 0x7f3ae41021b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:28:43.456 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:43.457+0000 7f3ae359e640 1 -- 172.21.15.52:0/1624286884 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3ae40ff8c0 msgr2=0x7f3ae41021b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:28:43.456 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:43.457+0000 7f3ae359e640 1 --2- 172.21.15.52:0/1624286884 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3ae40ff8c0 0x7f3ae41021b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:43.456 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:43.457+0000 7f3ae359e640 1 -- 172.21.15.52:0/1624286884 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3ae4102890 msgr2=0x7f3ae4100840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:28:43.456 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:43.457+0000 7f3ae359e640 1 --2- 172.21.15.52:0/1624286884 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3ae4102890 0x7f3ae4100840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:43.456 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:43.457+0000 7f3ae359e640 1 -- 172.21.15.52:0/1624286884 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ae4101640 con 0x7f3ae4103250 2024-03-04T09:28:43.456 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:43.457+0000 7f3ae259c640 1 --2- 172.21.15.52:0/1624286884 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3ae4102890 0x7f3ae4100840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T09:28:43.456 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:43.457+0000 7f3ae359e640 1 --2- 172.21.15.52:0/1624286884 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3ae4103250 0x7f3ae4100d80 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f3adc00efd0 tx=0x7f3adc00c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:28:43.456 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:43.457+0000 7f3acbfff640 1 -- 172.21.15.52:0/1624286884 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3adc003b70 con 0x7f3ae4103250 2024-03-04T09:28:43.457 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:43.457+0000 7f3acbfff640 1 -- 172.21.15.52:0/1624286884 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3adc003d10 con 0x7f3ae4103250 2024-03-04T09:28:43.457 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:43.457+0000 7f3ae909a640 1 -- 172.21.15.52:0/1624286884 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ae41a1ac0 con 0x7f3ae4103250 2024-03-04T09:28:43.457 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:43.457+0000 7f3acbfff640 1 -- 172.21.15.52:0/1624286884 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3adc0109f0 con 0x7f3ae4103250 2024-03-04T09:28:43.457 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:43.457+0000 7f3ae909a640 1 -- 172.21.15.52:0/1624286884 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ae41a1f90 con 0x7f3ae4103250 2024-03-04T09:28:43.458 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:43.458+0000 7f3ae909a640 1 -- 172.21.15.52:0/1624286884 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ae4108ab0 con 0x7f3ae4103250 2024-03-04T09:28:43.458 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:43.459+0000 7f3acbfff640 1 -- 172.21.15.52:0/1624286884 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 16) v1 ==== 136756+0+0 (secure 0 0 0) 0x7f3adc004020 con 0x7f3ae4103250 2024-03-04T09:28:43.458 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:43.460+0000 7f3acbfff640 1 --2- 172.21.15.52:0/1624286884 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f3acc0a1d80 0x7f3acc0a4240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:28:43.458 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:43.460+0000 7f3ae2d9d640 1 --2- 172.21.15.52:0/1624286884 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f3acc0a1d80 0x7f3acc0a4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:28:43.458 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:43.460+0000 7f3acbfff640 1 -- 172.21.15.52:0/1624286884 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8802+0+0 (secure 0 0 0) 0x7f3adc0ce550 con 0x7f3ae4103250 2024-03-04T09:28:43.458 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:43.460+0000 7f3ae2d9d640 1 --2- 172.21.15.52:0/1624286884 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f3acc0a1d80 0x7f3acc0a4240 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f3ad402c7b0 tx=0x7f3ad4000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:28:43.461 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:43.464+0000 7f3acbfff640 1 -- 172.21.15.52:0/1624286884 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f3adc09b740 con 0x7f3ae4103250 2024-03-04T09:28:43.646 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:43.649+0000 7f3ae909a640 1 -- 172.21.15.52:0/1624286884 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3ae40621d0 con 0x7f3ae4103250 2024-03-04T09:28:43.647 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:43.651+0000 7f3acbfff640 1 -- 172.21.15.52:0/1624286884 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) v1 ==== 74+0+24164 (secure 0 0 0) 0x7f3adc09b040 con 0x7f3ae4103250 2024-03-04T09:28:43.647 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-03-04T09:28:43.648 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":89,"fsid":"4e059f08-da08-11ee-95c6-87774f69a715","created":"2024-03-04T09:20:46.104963+0000","modified":"2024-03-04T09:28:42.295274+0000","last_up_change":"2024-03-04T09:27:49.261365+0000","last_in_change":"2024-03-04T09:27:30.024964+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-04T09:24:01.742461+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-04T09:28:35.835600+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-04T09:28:36.453099+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-04T09:28:40.168831+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"583172be-602f-4bf4-bfcb-42663cf5b277","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6802","nonce":1371221397},{"type":"v1","addr":"172.21.15.52:6803","nonce":1371221397}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6804","nonce":1371221397},{"type":"v1","addr":"172.21.15.52:6805","nonce":1371221397}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6808","nonce":1371221397},{"type":"v1","addr":"172.21.15.52:6809","nonce":1371221397}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6806","nonce":1371221397},{"type":"v1","addr":"172.21.15.52:6807","nonce":1371221397}]},"public_addr":"172.21.15.52:6803/1371221397","cluster_addr":"172.21.15.52:6805/1371221397","heartbeat_back_addr":"172.21.15.52:6809/1371221397","heartbeat_front_addr":"172.21.15.52:6807/1371221397","state":["exists","up"]},{"osd":1,"uuid":"5ae0081c-a2f6-46eb-ace0-0368208377b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":2540865537},{"type":"v1","addr":"172.21.15.106:6801","nonce":2540865537}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":2540865537},{"type":"v1","addr":"172.21.15.106:6803","nonce":2540865537}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":2540865537},{"type":"v1","addr":"172.21.15.106:6807","nonce":2540865537}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":2540865537},{"type":"v1","addr":"172.21.15.106:6805","nonce":2540865537}]},"public_addr":"172.21.15.106:6801/2540865537","cluster_addr":"172.21.15.106:6803/2540865537","heartbeat_back_addr":"172.21.15.106:6807/2540865537","heartbeat_front_addr":"172.21.15.106:6805/2540865537","state":["exists","up"]},{"osd":2,"uuid":"1948bac7-2bb2-4218-8b93-4e4743a0de04","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":1377083164},{"type":"v1","addr":"172.21.15.191:6801","nonce":1377083164}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":1377083164},{"type":"v1","addr":"172.21.15.191:6803","nonce":1377083164}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":1377083164},{"type":"v1","addr":"172.21.15.191:6807","nonce":1377083164}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":1377083164},{"type":"v1","addr":"172.21.15.191:6805","nonce":1377083164}]},"public_addr":"172.21.15.191:6801/1377083164","cluster_addr":"172.21.15.191:6803/1377083164","heartbeat_back_addr":"172.21.15.191:6807/1377083164","heartbeat_front_addr":"172.21.15.191:6805/1377083164","state":["exists","up"]},{"osd":3,"uuid":"16c9e9fe-7167-4992-bf12-1a7398e0498a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6810","nonce":2388571810},{"type":"v1","addr":"172.21.15.52:6811","nonce":2388571810}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6812","nonce":2388571810},{"type":"v1","addr":"172.21.15.52:6813","nonce":2388571810}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6816","nonce":2388571810},{"type":"v1","addr":"172.21.15.52:6817","nonce":2388571810}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6814","nonce":2388571810},{"type":"v1","addr":"172.21.15.52:6815","nonce":2388571810}]},"public_addr":"172.21.15.52:6811/2388571810","cluster_addr":"172.21.15.52:6813/2388571810","heartbeat_back_addr":"172.21.15.52:6817/2388571810","heartbeat_front_addr":"172.21.15.52:6815/2388571810","state":["exists","up"]},{"osd":4,"uuid":"c8ef48c7-7c78-44d3-a9e5-99ec750abd3d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6808","nonce":3465868015},{"type":"v1","addr":"172.21.15.106:6809","nonce":3465868015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6810","nonce":3465868015},{"type":"v1","addr":"172.21.15.106:6811","nonce":3465868015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6814","nonce":3465868015},{"type":"v1","addr":"172.21.15.106:6815","nonce":3465868015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6812","nonce":3465868015},{"type":"v1","addr":"172.21.15.106:6813","nonce":3465868015}]},"public_addr":"172.21.15.106:6809/3465868015","cluster_addr":"172.21.15.106:6811/3465868015","heartbeat_back_addr":"172.21.15.106:6815/3465868015","heartbeat_front_addr":"172.21.15.106:6813/3465868015","state":["exists","up"]},{"osd":5,"uuid":"9e41824a-c5a2-4f88-be50-2ba1dceb3220","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":1485092643},{"type":"v1","addr":"172.21.15.191:6809","nonce":1485092643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":1485092643},{"type":"v1","addr":"172.21.15.191:6811","nonce":1485092643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":1485092643},{"type":"v1","addr":"172.21.15.191:6815","nonce":1485092643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":1485092643},{"type":"v1","addr":"172.21.15.191:6813","nonce":1485092643}]},"public_addr":"172.21.15.191:6809/1485092643","cluster_addr":"172.21.15.191:6811/1485092643","heartbeat_back_addr":"172.21.15.191:6815/1485092643","heartbeat_front_addr":"172.21.15.191:6813/1485092643","state":["exists","up"]},{"osd":6,"uuid":"b35dccee-ec16-452e-ac69-3ac2fb7b27fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6818","nonce":1989797856},{"type":"v1","addr":"172.21.15.52:6819","nonce":1989797856}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6820","nonce":1989797856},{"type":"v1","addr":"172.21.15.52:6821","nonce":1989797856}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6824","nonce":1989797856},{"type":"v1","addr":"172.21.15.52:6825","nonce":1989797856}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6822","nonce":1989797856},{"type":"v1","addr":"172.21.15.52:6823","nonce":1989797856}]},"public_addr":"172.21.15.52:6819/1989797856","cluster_addr":"172.21.15.52:6821/1989797856","heartbeat_back_addr":"172.21.15.52:6825/1989797856","heartbeat_front_addr":"172.21.15.52:6823/1989797856","state":["exists","up"]},{"osd":7,"uuid":"71303e3d-14ec-4cbd-b19c-67645a1de3c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6816","nonce":1450870329},{"type":"v1","addr":"172.21.15.106:6817","nonce":1450870329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":1450870329},{"type":"v1","addr":"172.21.15.106:6819","nonce":1450870329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":1450870329},{"type":"v1","addr":"172.21.15.106:6823","nonce":1450870329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6820","nonce":1450870329},{"type":"v1","addr":"172.21.15.106:6821","nonce":1450870329}]},"public_addr":"172.21.15.106:6817/1450870329","cluster_addr":"172.21.15.106:6819/1450870329","heartbeat_back_addr":"172.21.15.106:6823/1450870329","heartbeat_front_addr":"172.21.15.106:6821/1450870329","state":["exists","up"]},{"osd":8,"uuid":"6f5786c4-4402-435a-bdad-adff46fe9473","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":1114119072},{"type":"v1","addr":"172.21.15.191:6817","nonce":1114119072}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6818","nonce":1114119072},{"type":"v1","addr":"172.21.15.191:6819","nonce":1114119072}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":1114119072},{"type":"v1","addr":"172.21.15.191:6823","nonce":1114119072}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":1114119072},{"type":"v1","addr":"172.21.15.191:6821","nonce":1114119072}]},"public_addr":"172.21.15.191:6817/1114119072","cluster_addr":"172.21.15.191:6819/1114119072","heartbeat_back_addr":"172.21.15.191:6823/1114119072","heartbeat_front_addr":"172.21.15.191:6821/1114119072","state":["exists","up"]},{"osd":9,"uuid":"062bbaa5-1227-4684-b859-450f5594a90c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6826","nonce":3485821914},{"type":"v1","addr":"172.21.15.52:6827","nonce":3485821914}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6828","nonce":3485821914},{"type":"v1","addr":"172.21.15.52:6829","nonce":3485821914}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6832","nonce":3485821914},{"type":"v1","addr":"172.21.15.52:6833","nonce":3485821914}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6830","nonce":3485821914},{"type":"v1","addr":"172.21.15.52:6831","nonce":3485821914}]},"public_addr":"172.21.15.52:6827/3485821914","cluster_addr":"172.21.15.52:6829/3485821914","heartbeat_back_addr":"172.21.15.52:6833/3485821914","heartbeat_front_addr":"172.21.15.52:6831/3485821914","state":["exists","up"]},{"osd":10,"uuid":"1a285a5a-f793-48da-a8eb-5326d9528cd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":3815586576},{"type":"v1","addr":"172.21.15.106:6825","nonce":3815586576}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6826","nonce":3815586576},{"type":"v1","addr":"172.21.15.106:6827","nonce":3815586576}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":3815586576},{"type":"v1","addr":"172.21.15.106:6831","nonce":3815586576}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":3815586576},{"type":"v1","addr":"172.21.15.106:6829","nonce":3815586576}]},"public_addr":"172.21.15.106:6825/3815586576","cluster_addr":"172.21.15.106:6827/3815586576","heartbeat_back_addr":"172.21.15.106:6831/3815586576","heartbeat_front_addr":"172.21.15.106:6829/3815586576","state":["exists","up"]},{"osd":11,"uuid":"1f8b9a43-57fa-41f4-9969-3b803a74bf45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":2009632843},{"type":"v1","addr":"172.21.15.191:6825","nonce":2009632843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":2009632843},{"type":"v1","addr":"172.21.15.191:6827","nonce":2009632843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":2009632843},{"type":"v1","addr":"172.21.15.191:6831","nonce":2009632843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":2009632843},{"type":"v1","addr":"172.21.15.191:6829","nonce":2009632843}]},"public_addr":"172.21.15.191:6825/2009632843","cluster_addr":"172.21.15.191:6827/2009632843","heartbeat_back_addr":"172.21.15.191:6831/2009632843","heartbeat_front_addr":"172.21.15.191:6829/2009632843","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T09:23:10.894763+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T09:23:34.420055+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T09:23:58.288324+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T09:24:23.673457+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T09:24:48.208561+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T09:25:13.405473+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T09:25:38.670915+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T09:26:04.451257+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T09:26:30.281380+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T09:26:56.543143+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T09:27:22.412930+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T09:27:48.003866+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.52:0/4082645718":"2024-03-05T09:21:39.401935+0000","172.21.15.52:0/1271949988":"2024-03-05T09:21:39.401935+0000","172.21.15.52:6801/2379361333":"2024-03-05T09:21:12.760938+0000","172.21.15.52:0/497075075":"2024-03-05T09:21:12.760938+0000","172.21.15.52:6800/2379361333":"2024-03-05T09:21:12.760938+0000","172.21.15.52:0/568242577":"2024-03-05T09:21:12.760938+0000","172.21.15.52:6801/2425730636":"2024-03-05T09:21:39.401935+0000","172.21.15.52:6800/2425730636":"2024-03-05T09:21:39.401935+0000","172.21.15.52:0/2743084243":"2024-03-05T09:21:39.401935+0000","172.21.15.52:0/1035878034":"2024-03-05T09:21:12.760938+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-04T09:28:43.650 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:43.652+0000 7f3ae909a640 1 -- 172.21.15.52:0/1624286884 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f3acc0a1d80 msgr2=0x7f3acc0a4240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:28:43.650 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:43.652+0000 7f3ae909a640 1 --2- 172.21.15.52:0/1624286884 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f3acc0a1d80 0x7f3acc0a4240 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f3ad402c7b0 tx=0x7f3ad4000bc0 comp rx=0 tx=0).stop 2024-03-04T09:28:43.650 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:43.652+0000 7f3ae909a640 1 -- 172.21.15.52:0/1624286884 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3ae4103250 msgr2=0x7f3ae4100d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:28:43.650 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:43.652+0000 7f3ae909a640 1 --2- 172.21.15.52:0/1624286884 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3ae4103250 0x7f3ae4100d80 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f3adc00efd0 tx=0x7f3adc00c6a0 comp rx=0 tx=0).stop 2024-03-04T09:28:43.650 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:43.652+0000 7f3ae909a640 1 -- 172.21.15.52:0/1624286884 shutdown_connections 2024-03-04T09:28:43.650 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:43.652+0000 7f3ae909a640 1 --2- 172.21.15.52:0/1624286884 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f3acc0a1d80 0x7f3acc0a4240 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:43.650 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:43.652+0000 7f3ae909a640 1 --2- 172.21.15.52:0/1624286884 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3ae4103250 0x7f3ae4100d80 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:43.650 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:43.652+0000 7f3ae909a640 1 --2- 172.21.15.52:0/1624286884 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3ae4102890 0x7f3ae4100840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:43.651 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:43.652+0000 7f3ae909a640 1 --2- 172.21.15.52:0/1624286884 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3ae40ff8c0 0x7f3ae41021b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:43.651 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:43.653+0000 7f3ae909a640 1 -- 172.21.15.52:0/1624286884 >> 172.21.15.52:0/1624286884 conn(0x7f3ae40fb1f0 msgr2=0x7f3ae4105c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:28:43.651 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:43.653+0000 7f3ae909a640 1 -- 172.21.15.52:0/1624286884 shutdown_connections 2024-03-04T09:28:43.651 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:43.653+0000 7f3ae909a640 1 -- 172.21.15.52:0/1624286884 wait complete. 2024-03-04T09:28:43.661 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-04T09:28:43.767 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:43.767+0000 7f45a25c7640 1 -- 172.21.15.52:0/1346616620 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f459c1031a0 msgr2=0x7f459c109a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:28:43.767 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:43.767+0000 7f45a25c7640 1 --2- 172.21.15.52:0/1346616620 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f459c1031a0 0x7f459c109a30 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f4594009ca0 tx=0x7f459402c0f0 comp rx=0 tx=0).stop 2024-03-04T09:28:43.768 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:43.767+0000 7f45a25c7640 1 -- 172.21.15.52:0/1346616620 shutdown_connections 2024-03-04T09:28:43.768 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:43.767+0000 7f45a25c7640 1 --2- 172.21.15.52:0/1346616620 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f459c1031a0 0x7f459c109a30 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:43.768 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:43.767+0000 7f45a25c7640 1 --2- 172.21.15.52:0/1346616620 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f459c1027e0 0x7f459c102c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:43.768 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:43.767+0000 7f45a25c7640 1 --2- 172.21.15.52:0/1346616620 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f459c1015f0 0x7f459c101a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:43.768 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:43.767+0000 7f45a25c7640 1 -- 172.21.15.52:0/1346616620 >> 172.21.15.52:0/1346616620 conn(0x7f459c0fcb80 msgr2=0x7f459c0fefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:28:43.768 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:43.767+0000 7f45a25c7640 1 -- 172.21.15.52:0/1346616620 shutdown_connections 2024-03-04T09:28:43.768 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:43.768+0000 7f45a25c7640 1 -- 172.21.15.52:0/1346616620 wait complete. 2024-03-04T09:28:43.768 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:43.768+0000 7f45a25c7640 1 Processor -- start 2024-03-04T09:28:43.768 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:43.768+0000 7f45a25c7640 1 -- start start 2024-03-04T09:28:43.768 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:43.768+0000 7f45a25c7640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f459c1015f0 0x7f459c197880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:28:43.768 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:43.768+0000 7f45a25c7640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f459c1027e0 0x7f459c197dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:28:43.768 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:43.768+0000 7f45a25c7640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f459c1031a0 0x7f459c19ee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:28:43.768 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:43.768+0000 7f45a25c7640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f459c10d270 con 0x7f459c1031a0 2024-03-04T09:28:43.769 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:43.768+0000 7f45a25c7640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f459c10d0f0 con 0x7f459c1027e0 2024-03-04T09:28:43.769 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:43.768+0000 7f45a25c7640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f459c10d3f0 con 0x7f459c1015f0 2024-03-04T09:28:43.769 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:43.768+0000 7f45a0b64640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f459c1031a0 0x7f459c19ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:28:43.769 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:43.769+0000 7f45a0b64640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f459c1031a0 0x7f459c19ee30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:57914/0 (socket says 172.21.15.52:57914) 2024-03-04T09:28:43.769 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:43.769+0000 7f45a0b64640 1 -- 172.21.15.52:0/1014113513 learned_addr learned my addr 172.21.15.52:0/1014113513 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-04T09:28:43.769 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:43.769+0000 7f459b7fe640 1 --2- 172.21.15.52:0/1014113513 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f459c1027e0 0x7f459c197dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:28:43.769 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:43.769+0000 7f45a0b64640 1 -- 172.21.15.52:0/1014113513 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f459c1015f0 msgr2=0x7f459c197880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:28:43.769 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:43.769+0000 7f45a0b64640 1 --2- 172.21.15.52:0/1014113513 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f459c1015f0 0x7f459c197880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:43.769 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:43.769+0000 7f45a0b64640 1 -- 172.21.15.52:0/1014113513 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f459c1027e0 msgr2=0x7f459c197dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:28:43.769 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:43.769+0000 7f45a0b64640 1 --2- 172.21.15.52:0/1014113513 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f459c1027e0 0x7f459c197dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:43.769 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:43.769+0000 7f45a0b64640 1 -- 172.21.15.52:0/1014113513 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f459c19f4a0 con 0x7f459c1031a0 2024-03-04T09:28:43.769 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:43.769+0000 7f45a0b64640 1 --2- 172.21.15.52:0/1014113513 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f459c1031a0 0x7f459c19ee30 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f4594037040 tx=0x7f4594032f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:28:43.769 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:43.769+0000 7f45997fa640 1 -- 172.21.15.52:0/1014113513 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4594041020 con 0x7f459c1031a0 2024-03-04T09:28:43.769 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:43.769+0000 7f45997fa640 1 -- 172.21.15.52:0/1014113513 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4594034d90 con 0x7f459c1031a0 2024-03-04T09:28:43.770 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:43.769+0000 7f45a25c7640 1 -- 172.21.15.52:0/1014113513 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f459c19f730 con 0x7f459c1031a0 2024-03-04T09:28:43.770 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:43.769+0000 7f45997fa640 1 -- 172.21.15.52:0/1014113513 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4594039040 con 0x7f459c1031a0 2024-03-04T09:28:43.770 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:43.769+0000 7f45a25c7640 1 -- 172.21.15.52:0/1014113513 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f459c19fc70 con 0x7f459c1031a0 2024-03-04T09:28:43.770 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:43.770+0000 7f45a25c7640 1 -- 172.21.15.52:0/1014113513 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f459c108910 con 0x7f459c1031a0 2024-03-04T09:28:43.771 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:43.771+0000 7f45997fa640 1 -- 172.21.15.52:0/1014113513 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 16) v1 ==== 136756+0+0 (secure 0 0 0) 0x7f4594044020 con 0x7f459c1031a0 2024-03-04T09:28:43.771 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:43.772+0000 7f45997fa640 1 --2- 172.21.15.52:0/1014113513 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f45840a1e00 0x7f45840a42c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:28:43.771 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:43.772+0000 7f459bfff640 1 --2- 172.21.15.52:0/1014113513 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f45840a1e00 0x7f45840a42c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:28:43.771 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:43.772+0000 7f45997fa640 1 -- 172.21.15.52:0/1014113513 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8802+0+0 (secure 0 0 0) 0x7f45940eff50 con 0x7f459c1031a0 2024-03-04T09:28:43.771 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:43.772+0000 7f459bfff640 1 --2- 172.21.15.52:0/1014113513 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f45840a1e00 0x7f45840a42c0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f458c005fd0 tx=0x7f458c00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:28:43.773 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:43.776+0000 7f45997fa640 1 -- 172.21.15.52:0/1014113513 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f459404b020 con 0x7f459c1031a0 2024-03-04T09:28:43.957 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:43.960+0000 7f45a25c7640 1 -- 172.21.15.52:0/1014113513 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f459c0621d0 con 0x7f459c1031a0 2024-03-04T09:28:43.959 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:43.962+0000 7f45997fa640 1 -- 172.21.15.52:0/1014113513 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) v1 ==== 74+0+24164 (secure 0 0 0) 0x7f45940bd0e0 con 0x7f459c1031a0 2024-03-04T09:28:43.959 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-03-04T09:28:43.959 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":89,"fsid":"4e059f08-da08-11ee-95c6-87774f69a715","created":"2024-03-04T09:20:46.104963+0000","modified":"2024-03-04T09:28:42.295274+0000","last_up_change":"2024-03-04T09:27:49.261365+0000","last_in_change":"2024-03-04T09:27:30.024964+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-04T09:24:01.742461+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-04T09:28:35.835600+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-04T09:28:36.453099+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-04T09:28:40.168831+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"583172be-602f-4bf4-bfcb-42663cf5b277","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6802","nonce":1371221397},{"type":"v1","addr":"172.21.15.52:6803","nonce":1371221397}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6804","nonce":1371221397},{"type":"v1","addr":"172.21.15.52:6805","nonce":1371221397}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6808","nonce":1371221397},{"type":"v1","addr":"172.21.15.52:6809","nonce":1371221397}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6806","nonce":1371221397},{"type":"v1","addr":"172.21.15.52:6807","nonce":1371221397}]},"public_addr":"172.21.15.52:6803/1371221397","cluster_addr":"172.21.15.52:6805/1371221397","heartbeat_back_addr":"172.21.15.52:6809/1371221397","heartbeat_front_addr":"172.21.15.52:6807/1371221397","state":["exists","up"]},{"osd":1,"uuid":"5ae0081c-a2f6-46eb-ace0-0368208377b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":2540865537},{"type":"v1","addr":"172.21.15.106:6801","nonce":2540865537}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":2540865537},{"type":"v1","addr":"172.21.15.106:6803","nonce":2540865537}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":2540865537},{"type":"v1","addr":"172.21.15.106:6807","nonce":2540865537}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":2540865537},{"type":"v1","addr":"172.21.15.106:6805","nonce":2540865537}]},"public_addr":"172.21.15.106:6801/2540865537","cluster_addr":"172.21.15.106:6803/2540865537","heartbeat_back_addr":"172.21.15.106:6807/2540865537","heartbeat_front_addr":"172.21.15.106:6805/2540865537","state":["exists","up"]},{"osd":2,"uuid":"1948bac7-2bb2-4218-8b93-4e4743a0de04","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":1377083164},{"type":"v1","addr":"172.21.15.191:6801","nonce":1377083164}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":1377083164},{"type":"v1","addr":"172.21.15.191:6803","nonce":1377083164}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":1377083164},{"type":"v1","addr":"172.21.15.191:6807","nonce":1377083164}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":1377083164},{"type":"v1","addr":"172.21.15.191:6805","nonce":1377083164}]},"public_addr":"172.21.15.191:6801/1377083164","cluster_addr":"172.21.15.191:6803/1377083164","heartbeat_back_addr":"172.21.15.191:6807/1377083164","heartbeat_front_addr":"172.21.15.191:6805/1377083164","state":["exists","up"]},{"osd":3,"uuid":"16c9e9fe-7167-4992-bf12-1a7398e0498a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6810","nonce":2388571810},{"type":"v1","addr":"172.21.15.52:6811","nonce":2388571810}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6812","nonce":2388571810},{"type":"v1","addr":"172.21.15.52:6813","nonce":2388571810}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6816","nonce":2388571810},{"type":"v1","addr":"172.21.15.52:6817","nonce":2388571810}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6814","nonce":2388571810},{"type":"v1","addr":"172.21.15.52:6815","nonce":2388571810}]},"public_addr":"172.21.15.52:6811/2388571810","cluster_addr":"172.21.15.52:6813/2388571810","heartbeat_back_addr":"172.21.15.52:6817/2388571810","heartbeat_front_addr":"172.21.15.52:6815/2388571810","state":["exists","up"]},{"osd":4,"uuid":"c8ef48c7-7c78-44d3-a9e5-99ec750abd3d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6808","nonce":3465868015},{"type":"v1","addr":"172.21.15.106:6809","nonce":3465868015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6810","nonce":3465868015},{"type":"v1","addr":"172.21.15.106:6811","nonce":3465868015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6814","nonce":3465868015},{"type":"v1","addr":"172.21.15.106:6815","nonce":3465868015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6812","nonce":3465868015},{"type":"v1","addr":"172.21.15.106:6813","nonce":3465868015}]},"public_addr":"172.21.15.106:6809/3465868015","cluster_addr":"172.21.15.106:6811/3465868015","heartbeat_back_addr":"172.21.15.106:6815/3465868015","heartbeat_front_addr":"172.21.15.106:6813/3465868015","state":["exists","up"]},{"osd":5,"uuid":"9e41824a-c5a2-4f88-be50-2ba1dceb3220","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":1485092643},{"type":"v1","addr":"172.21.15.191:6809","nonce":1485092643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":1485092643},{"type":"v1","addr":"172.21.15.191:6811","nonce":1485092643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":1485092643},{"type":"v1","addr":"172.21.15.191:6815","nonce":1485092643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":1485092643},{"type":"v1","addr":"172.21.15.191:6813","nonce":1485092643}]},"public_addr":"172.21.15.191:6809/1485092643","cluster_addr":"172.21.15.191:6811/1485092643","heartbeat_back_addr":"172.21.15.191:6815/1485092643","heartbeat_front_addr":"172.21.15.191:6813/1485092643","state":["exists","up"]},{"osd":6,"uuid":"b35dccee-ec16-452e-ac69-3ac2fb7b27fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6818","nonce":1989797856},{"type":"v1","addr":"172.21.15.52:6819","nonce":1989797856}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6820","nonce":1989797856},{"type":"v1","addr":"172.21.15.52:6821","nonce":1989797856}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6824","nonce":1989797856},{"type":"v1","addr":"172.21.15.52:6825","nonce":1989797856}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6822","nonce":1989797856},{"type":"v1","addr":"172.21.15.52:6823","nonce":1989797856}]},"public_addr":"172.21.15.52:6819/1989797856","cluster_addr":"172.21.15.52:6821/1989797856","heartbeat_back_addr":"172.21.15.52:6825/1989797856","heartbeat_front_addr":"172.21.15.52:6823/1989797856","state":["exists","up"]},{"osd":7,"uuid":"71303e3d-14ec-4cbd-b19c-67645a1de3c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6816","nonce":1450870329},{"type":"v1","addr":"172.21.15.106:6817","nonce":1450870329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":1450870329},{"type":"v1","addr":"172.21.15.106:6819","nonce":1450870329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":1450870329},{"type":"v1","addr":"172.21.15.106:6823","nonce":1450870329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6820","nonce":1450870329},{"type":"v1","addr":"172.21.15.106:6821","nonce":1450870329}]},"public_addr":"172.21.15.106:6817/1450870329","cluster_addr":"172.21.15.106:6819/1450870329","heartbeat_back_addr":"172.21.15.106:6823/1450870329","heartbeat_front_addr":"172.21.15.106:6821/1450870329","state":["exists","up"]},{"osd":8,"uuid":"6f5786c4-4402-435a-bdad-adff46fe9473","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":1114119072},{"type":"v1","addr":"172.21.15.191:6817","nonce":1114119072}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6818","nonce":1114119072},{"type":"v1","addr":"172.21.15.191:6819","nonce":1114119072}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":1114119072},{"type":"v1","addr":"172.21.15.191:6823","nonce":1114119072}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":1114119072},{"type":"v1","addr":"172.21.15.191:6821","nonce":1114119072}]},"public_addr":"172.21.15.191:6817/1114119072","cluster_addr":"172.21.15.191:6819/1114119072","heartbeat_back_addr":"172.21.15.191:6823/1114119072","heartbeat_front_addr":"172.21.15.191:6821/1114119072","state":["exists","up"]},{"osd":9,"uuid":"062bbaa5-1227-4684-b859-450f5594a90c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6826","nonce":3485821914},{"type":"v1","addr":"172.21.15.52:6827","nonce":3485821914}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6828","nonce":3485821914},{"type":"v1","addr":"172.21.15.52:6829","nonce":3485821914}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6832","nonce":3485821914},{"type":"v1","addr":"172.21.15.52:6833","nonce":3485821914}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6830","nonce":3485821914},{"type":"v1","addr":"172.21.15.52:6831","nonce":3485821914}]},"public_addr":"172.21.15.52:6827/3485821914","cluster_addr":"172.21.15.52:6829/3485821914","heartbeat_back_addr":"172.21.15.52:6833/3485821914","heartbeat_front_addr":"172.21.15.52:6831/3485821914","state":["exists","up"]},{"osd":10,"uuid":"1a285a5a-f793-48da-a8eb-5326d9528cd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":3815586576},{"type":"v1","addr":"172.21.15.106:6825","nonce":3815586576}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6826","nonce":3815586576},{"type":"v1","addr":"172.21.15.106:6827","nonce":3815586576}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":3815586576},{"type":"v1","addr":"172.21.15.106:6831","nonce":3815586576}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":3815586576},{"type":"v1","addr":"172.21.15.106:6829","nonce":3815586576}]},"public_addr":"172.21.15.106:6825/3815586576","cluster_addr":"172.21.15.106:6827/3815586576","heartbeat_back_addr":"172.21.15.106:6831/3815586576","heartbeat_front_addr":"172.21.15.106:6829/3815586576","state":["exists","up"]},{"osd":11,"uuid":"1f8b9a43-57fa-41f4-9969-3b803a74bf45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":2009632843},{"type":"v1","addr":"172.21.15.191:6825","nonce":2009632843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":2009632843},{"type":"v1","addr":"172.21.15.191:6827","nonce":2009632843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":2009632843},{"type":"v1","addr":"172.21.15.191:6831","nonce":2009632843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":2009632843},{"type":"v1","addr":"172.21.15.191:6829","nonce":2009632843}]},"public_addr":"172.21.15.191:6825/2009632843","cluster_addr":"172.21.15.191:6827/2009632843","heartbeat_back_addr":"172.21.15.191:6831/2009632843","heartbeat_front_addr":"172.21.15.191:6829/2009632843","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T09:23:10.894763+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T09:23:34.420055+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T09:23:58.288324+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T09:24:23.673457+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T09:24:48.208561+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T09:25:13.405473+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T09:25:38.670915+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T09:26:04.451257+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T09:26:30.281380+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T09:26:56.543143+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T09:27:22.412930+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T09:27:48.003866+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.52:0/4082645718":"2024-03-05T09:21:39.401935+0000","172.21.15.52:0/1271949988":"2024-03-05T09:21:39.401935+0000","172.21.15.52:6801/2379361333":"2024-03-05T09:21:12.760938+0000","172.21.15.52:0/497075075":"2024-03-05T09:21:12.760938+0000","172.21.15.52:6800/2379361333":"2024-03-05T09:21:12.760938+0000","172.21.15.52:0/568242577":"2024-03-05T09:21:12.760938+0000","172.21.15.52:6801/2425730636":"2024-03-05T09:21:39.401935+0000","172.21.15.52:6800/2425730636":"2024-03-05T09:21:39.401935+0000","172.21.15.52:0/2743084243":"2024-03-05T09:21:39.401935+0000","172.21.15.52:0/1035878034":"2024-03-05T09:21:12.760938+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-04T09:28:43.961 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:43.964+0000 7f45a25c7640 1 -- 172.21.15.52:0/1014113513 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f45840a1e00 msgr2=0x7f45840a42c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:28:43.961 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:43.964+0000 7f45a25c7640 1 --2- 172.21.15.52:0/1014113513 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f45840a1e00 0x7f45840a42c0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f458c005fd0 tx=0x7f458c00b410 comp rx=0 tx=0).stop 2024-03-04T09:28:43.961 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:43.964+0000 7f45a25c7640 1 -- 172.21.15.52:0/1014113513 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f459c1031a0 msgr2=0x7f459c19ee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:28:43.961 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:43.964+0000 7f45a25c7640 1 --2- 172.21.15.52:0/1014113513 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f459c1031a0 0x7f459c19ee30 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f4594037040 tx=0x7f4594032f20 comp rx=0 tx=0).stop 2024-03-04T09:28:43.961 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:43.964+0000 7f45a25c7640 1 -- 172.21.15.52:0/1014113513 shutdown_connections 2024-03-04T09:28:43.961 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:43.964+0000 7f45a25c7640 1 --2- 172.21.15.52:0/1014113513 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f45840a1e00 0x7f45840a42c0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:43.961 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:43.964+0000 7f45a25c7640 1 --2- 172.21.15.52:0/1014113513 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f459c1031a0 0x7f459c19ee30 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:43.961 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:43.964+0000 7f45a25c7640 1 --2- 172.21.15.52:0/1014113513 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f459c1027e0 0x7f459c197dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:43.961 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:43.964+0000 7f45a25c7640 1 --2- 172.21.15.52:0/1014113513 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f459c1015f0 0x7f459c197880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:43.961 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:43.964+0000 7f45a25c7640 1 -- 172.21.15.52:0/1014113513 >> 172.21.15.52:0/1014113513 conn(0x7f459c0fcb80 msgr2=0x7f459c105ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:28:43.961 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:43.964+0000 7f45a25c7640 1 -- 172.21.15.52:0/1014113513 shutdown_connections 2024-03-04T09:28:43.962 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:43.964+0000 7f45a25c7640 1 -- 172.21.15.52:0/1014113513 wait complete. 2024-03-04T09:28:43.986 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T09:28:44.091 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:44.092+0000 7fe986815640 1 -- 172.21.15.52:0/744984717 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe9801027d0 msgr2=0x7fe980102c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:28:44.091 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:44.092+0000 7fe986815640 1 --2- 172.21.15.52:0/744984717 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe9801027d0 0x7fe980102c50 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fe970009860 tx=0x7fe97002bb70 comp rx=0 tx=0).stop 2024-03-04T09:28:44.091 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:44.092+0000 7fe986815640 1 -- 172.21.15.52:0/744984717 shutdown_connections 2024-03-04T09:28:44.091 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:44.092+0000 7fe986815640 1 --2- 172.21.15.52:0/744984717 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe980103190 0x7fe980109a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:44.091 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:44.092+0000 7fe986815640 1 --2- 172.21.15.52:0/744984717 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe9801027d0 0x7fe980102c50 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:44.091 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:44.092+0000 7fe986815640 1 --2- 172.21.15.52:0/744984717 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe9801015e0 0x7fe980101a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:44.091 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:44.092+0000 7fe986815640 1 -- 172.21.15.52:0/744984717 >> 172.21.15.52:0/744984717 conn(0x7fe9800fcbb0 msgr2=0x7fe9800fefd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:28:44.091 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:44.092+0000 7fe986815640 1 -- 172.21.15.52:0/744984717 shutdown_connections 2024-03-04T09:28:44.091 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:44.092+0000 7fe986815640 1 -- 172.21.15.52:0/744984717 wait complete. 2024-03-04T09:28:44.091 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:44.092+0000 7fe986815640 1 Processor -- start 2024-03-04T09:28:44.091 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:44.092+0000 7fe986815640 1 -- start start 2024-03-04T09:28:44.091 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:44.093+0000 7fe986815640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe9801015e0 0x7fe980197900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:28:44.092 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:44.093+0000 7fe986815640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe9801027d0 0x7fe980197e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:28:44.092 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:44.093+0000 7fe986815640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe980103190 0x7fe98019eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:28:44.092 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:44.093+0000 7fe986815640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe98010d260 con 0x7fe9801027d0 2024-03-04T09:28:44.092 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:44.093+0000 7fe986815640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe98010d0e0 con 0x7fe980103190 2024-03-04T09:28:44.092 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:44.093+0000 7fe986815640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe98010d3e0 con 0x7fe9801015e0 2024-03-04T09:28:44.092 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:44.093+0000 7fe97f7fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe9801027d0 0x7fe980197e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:28:44.092 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:44.093+0000 7fe97f7fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe9801027d0 0x7fe980197e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:57940/0 (socket says 172.21.15.52:57940) 2024-03-04T09:28:44.092 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:44.093+0000 7fe97f7fe640 1 -- 172.21.15.52:0/3453046344 learned_addr learned my addr 172.21.15.52:0/3453046344 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-04T09:28:44.092 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:44.093+0000 7fe984db2640 1 --2- 172.21.15.52:0/3453046344 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe980103190 0x7fe98019eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:28:44.093 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:44.093+0000 7fe97ffff640 1 --2- 172.21.15.52:0/3453046344 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe9801015e0 0x7fe980197900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:28:44.093 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:44.093+0000 7fe97f7fe640 1 -- 172.21.15.52:0/3453046344 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe9801015e0 msgr2=0x7fe980197900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:28:44.093 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:44.093+0000 7fe97f7fe640 1 --2- 172.21.15.52:0/3453046344 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe9801015e0 0x7fe980197900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:44.093 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:44.093+0000 7fe97f7fe640 1 -- 172.21.15.52:0/3453046344 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe980103190 msgr2=0x7fe98019eeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:28:44.093 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:44.093+0000 7fe97f7fe640 1 --2- 172.21.15.52:0/3453046344 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe980103190 0x7fe98019eeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:44.093 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:44.093+0000 7fe97f7fe640 1 -- 172.21.15.52:0/3453046344 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe98019f520 con 0x7fe9801027d0 2024-03-04T09:28:44.093 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:44.093+0000 7fe97f7fe640 1 --2- 172.21.15.52:0/3453046344 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe9801027d0 0x7fe980197e40 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fe970005940 tx=0x7fe9700042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:28:44.093 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:44.093+0000 7fe97d7fa640 1 -- 172.21.15.52:0/3453046344 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe970041020 con 0x7fe9801027d0 2024-03-04T09:28:44.093 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:44.093+0000 7fe986815640 1 -- 172.21.15.52:0/3453046344 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe98019f7b0 con 0x7fe9801027d0 2024-03-04T09:28:44.093 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:44.093+0000 7fe97d7fa640 1 -- 172.21.15.52:0/3453046344 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe9700323f0 con 0x7fe9801027d0 2024-03-04T09:28:44.094 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:44.093+0000 7fe97d7fa640 1 -- 172.21.15.52:0/3453046344 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe970039040 con 0x7fe9801027d0 2024-03-04T09:28:44.094 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:44.093+0000 7fe986815640 1 -- 172.21.15.52:0/3453046344 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe98019fca0 con 0x7fe9801027d0 2024-03-04T09:28:44.095 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:44.095+0000 7fe986815640 1 -- 172.21.15.52:0/3453046344 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe980108900 con 0x7fe9801027d0 2024-03-04T09:28:44.095 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:44.096+0000 7fe97d7fa640 1 -- 172.21.15.52:0/3453046344 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 16) v1 ==== 136756+0+0 (secure 0 0 0) 0x7fe970044020 con 0x7fe9801027d0 2024-03-04T09:28:44.095 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:44.096+0000 7fe97d7fa640 1 --2- 172.21.15.52:0/3453046344 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7fe96c0a1d80 0x7fe96c0a4240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:28:44.096 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:44.096+0000 7fe97ffff640 1 --2- 172.21.15.52:0/3453046344 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7fe96c0a1d80 0x7fe96c0a4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:28:44.096 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:44.096+0000 7fe97d7fa640 1 -- 172.21.15.52:0/3453046344 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8802+0+0 (secure 0 0 0) 0x7fe9700efb00 con 0x7fe9801027d0 2024-03-04T09:28:44.096 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:44.097+0000 7fe97ffff640 1 --2- 172.21.15.52:0/3453046344 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7fe96c0a1d80 0x7fe96c0a4240 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fe968005fd0 tx=0x7fe968005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:28:44.097 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:44.100+0000 7fe97d7fa640 1 -- 172.21.15.52:0/3453046344 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fe97004b020 con 0x7fe9801027d0 2024-03-04T09:28:44.316 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:44.320+0000 7fe986815640 1 -- 172.21.15.52:0/3453046344 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe9800621d0 con 0x7fe9801027d0 2024-03-04T09:28:44.317 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:44.320+0000 7fe97d7fa640 1 -- 172.21.15.52:0/3453046344 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+11333 (secure 0 0 0) 0x7fe9700bccf0 con 0x7fe9801027d0 2024-03-04T09:28:44.317 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-03-04T09:28:44.317 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":17,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6835/1007961019","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6834","nonce":1007961019},{"type":"v1","addr":"172.21.15.52:6835","nonce":1007961019}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6837/3753881283","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6836","nonce":3753881283},{"type":"v1","addr":"172.21.15.52:6837","nonce":3753881283}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14628,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6839/3467897808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6838","nonce":3467897808},{"type":"v1","addr":"172.21.15.52:6839","nonce":3467897808}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14640,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6841/2804292291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6840","nonce":2804292291},{"type":"v1","addr":"172.21.15.52:6841","nonce":2804292291}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24407,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6833/1818480323","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6832","nonce":1818480323},{"type":"v1","addr":"172.21.15.106:6833","nonce":1818480323}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24412,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6833/765768204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":765768204},{"type":"v1","addr":"172.21.15.191:6833","nonce":765768204}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24440,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6835/1541069897","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6834","nonce":1541069897},{"type":"v1","addr":"172.21.15.106:6835","nonce":1541069897}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24451,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6835/105048699","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":105048699},{"type":"v1","addr":"172.21.15.191:6835","nonce":105048699}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24463,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6837/4071064225","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6836","nonce":4071064225},{"type":"v1","addr":"172.21.15.191:6837","nonce":4071064225}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24473,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6837/2901755066","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6836","nonce":2901755066},{"type":"v1","addr":"172.21.15.106:6837","nonce":2901755066}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":24490,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6839/2194324891","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6838","nonce":2194324891},{"type":"v1","addr":"172.21.15.191:6839","nonce":2194324891}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T09:28:37.264970+0000","modified":"2024-03-04T09:28:42.295443+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":24506},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24506":{"gid":24506,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.106:6839/1995853093","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6838","nonce":1995853093},{"type":"v1","addr":"172.21.15.106:6839","nonce":1995853093}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":24506,"qdb_cluster":[24506]},"id":1}]} 2024-03-04T09:28:44.318 INFO:teuthology.orchestra.run.smithi052.stderr:dumped fsmap epoch 17 2024-03-04T09:28:44.319 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:44.322+0000 7fe986815640 1 -- 172.21.15.52:0/3453046344 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7fe96c0a1d80 msgr2=0x7fe96c0a4240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:28:44.320 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:44.322+0000 7fe986815640 1 --2- 172.21.15.52:0/3453046344 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7fe96c0a1d80 0x7fe96c0a4240 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fe968005fd0 tx=0x7fe968005850 comp rx=0 tx=0).stop 2024-03-04T09:28:44.320 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:44.322+0000 7fe986815640 1 -- 172.21.15.52:0/3453046344 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe9801027d0 msgr2=0x7fe980197e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:28:44.320 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:44.322+0000 7fe986815640 1 --2- 172.21.15.52:0/3453046344 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe9801027d0 0x7fe980197e40 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fe970005940 tx=0x7fe9700042f0 comp rx=0 tx=0).stop 2024-03-04T09:28:44.320 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:44.323+0000 7fe986815640 1 -- 172.21.15.52:0/3453046344 shutdown_connections 2024-03-04T09:28:44.320 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:44.323+0000 7fe986815640 1 --2- 172.21.15.52:0/3453046344 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7fe96c0a1d80 0x7fe96c0a4240 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:44.320 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:44.323+0000 7fe986815640 1 --2- 172.21.15.52:0/3453046344 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe980103190 0x7fe98019eeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:44.320 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:44.323+0000 7fe986815640 1 --2- 172.21.15.52:0/3453046344 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe9801027d0 0x7fe980197e40 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:44.320 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:44.323+0000 7fe986815640 1 --2- 172.21.15.52:0/3453046344 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe9801015e0 0x7fe980197900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:44.320 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:44.323+0000 7fe986815640 1 -- 172.21.15.52:0/3453046344 >> 172.21.15.52:0/3453046344 conn(0x7fe9800fcbb0 msgr2=0x7fe980105ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:28:44.321 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:44.323+0000 7fe986815640 1 -- 172.21.15.52:0/3453046344 shutdown_connections 2024-03-04T09:28:44.321 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:44.323+0000 7fe986815640 1 -- 172.21.15.52:0/3453046344 wait complete. 2024-03-04T09:28:44.329 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:28:43 smithi052 ceph-mon[32473]: osdmap e89: 12 total, 12 up, 12 in 2024-03-04T09:28:44.329 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:28:43 smithi052 ceph-mon[32473]: from='client.? 172.21.15.52:0/1387136786' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-03-04T09:28:44.330 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:28:43 smithi052 ceph-mon[32473]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-04T09:28:44.330 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:28:43 smithi052 ceph-mon[32473]: from='client.? 172.21.15.52:0/3174265828' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-04T09:28:44.330 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:28:43 smithi052 ceph-mon[32473]: from='client.? 172.21.15.52:0/1624286884' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-04T09:28:44.331 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-04T09:28:44.336 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:28:43 smithi191 ceph-mon[36816]: osdmap e89: 12 total, 12 up, 12 in 2024-03-04T09:28:44.336 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:28:43 smithi191 ceph-mon[36816]: from='client.? 172.21.15.52:0/1387136786' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-03-04T09:28:44.336 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:28:43 smithi191 ceph-mon[36816]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-04T09:28:44.336 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:28:43 smithi191 ceph-mon[36816]: from='client.? 172.21.15.52:0/3174265828' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-04T09:28:44.337 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:28:43 smithi191 ceph-mon[36816]: from='client.? 172.21.15.52:0/1624286884' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-04T09:28:44.381 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:28:43 smithi106 ceph-mon[36484]: osdmap e89: 12 total, 12 up, 12 in 2024-03-04T09:28:44.381 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:28:43 smithi106 ceph-mon[36484]: from='client.? 172.21.15.52:0/1387136786' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-03-04T09:28:44.381 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:28:43 smithi106 ceph-mon[36484]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-04T09:28:44.381 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:28:43 smithi106 ceph-mon[36484]: from='client.? 172.21.15.52:0/3174265828' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-04T09:28:44.381 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:28:43 smithi106 ceph-mon[36484]: from='client.? 172.21.15.52:0/1624286884' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-04T09:28:44.436 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:44.436+0000 7f9d8eb1e640 1 -- 172.21.15.52:0/834072548 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9d880ff8e0 msgr2=0x7f9d880ffd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:28:44.436 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:44.436+0000 7f9d8eb1e640 1 --2- 172.21.15.52:0/834072548 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9d880ff8e0 0x7f9d880ffd00 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f9d80009cd0 tx=0x7f9d8002c0f0 comp rx=0 tx=0).stop 2024-03-04T09:28:44.436 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:44.436+0000 7f9d8eb1e640 1 -- 172.21.15.52:0/834072548 shutdown_connections 2024-03-04T09:28:44.436 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:44.436+0000 7f9d8eb1e640 1 --2- 172.21.15.52:0/834072548 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9d88103270 0x7f9d88109b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:44.436 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:44.436+0000 7f9d8eb1e640 1 --2- 172.21.15.52:0/834072548 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9d881028b0 0x7f9d88102d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:44.436 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:44.436+0000 7f9d8eb1e640 1 --2- 172.21.15.52:0/834072548 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9d880ff8e0 0x7f9d880ffd00 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:44.436 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:44.436+0000 7f9d8eb1e640 1 -- 172.21.15.52:0/834072548 >> 172.21.15.52:0/834072548 conn(0x7f9d880fb210 msgr2=0x7f9d880fd650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:28:44.437 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:44.436+0000 7f9d8eb1e640 1 -- 172.21.15.52:0/834072548 shutdown_connections 2024-03-04T09:28:44.437 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:44.436+0000 7f9d8eb1e640 1 -- 172.21.15.52:0/834072548 wait complete. 2024-03-04T09:28:44.437 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:44.437+0000 7f9d8eb1e640 1 Processor -- start 2024-03-04T09:28:44.437 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:44.437+0000 7f9d8eb1e640 1 -- start start 2024-03-04T09:28:44.437 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:44.437+0000 7f9d8eb1e640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9d880ff8e0 0x7f9d88197980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:28:44.437 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:44.437+0000 7f9d8eb1e640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9d881028b0 0x7f9d88197ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:28:44.437 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:44.437+0000 7f9d8eb1e640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9d88103270 0x7f9d8819ef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:28:44.437 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:44.437+0000 7f9d8eb1e640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d8810d2a0 con 0x7f9d881028b0 2024-03-04T09:28:44.437 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:44.437+0000 7f9d8eb1e640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d8810d120 con 0x7f9d88103270 2024-03-04T09:28:44.437 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:44.437+0000 7f9d8eb1e640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d8810d420 con 0x7f9d880ff8e0 2024-03-04T09:28:44.437 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:44.437+0000 7f9d87fff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9d881028b0 0x7f9d88197ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:28:44.437 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:44.437+0000 7f9d87fff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9d881028b0 0x7f9d88197ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53362/0 (socket says 172.21.15.52:53362) 2024-03-04T09:28:44.437 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:44.437+0000 7f9d87fff640 1 -- 172.21.15.52:0/2311411945 learned_addr learned my addr 172.21.15.52:0/2311411945 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-04T09:28:44.438 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:44.437+0000 7f9d8d0bb640 1 --2- 172.21.15.52:0/2311411945 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9d88103270 0x7f9d8819ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:28:44.438 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:44.437+0000 7f9d87fff640 1 -- 172.21.15.52:0/2311411945 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9d880ff8e0 msgr2=0x7f9d88197980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:28:44.438 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:44.437+0000 7f9d8c8ba640 1 --2- 172.21.15.52:0/2311411945 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9d880ff8e0 0x7f9d88197980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:28:44.438 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:44.437+0000 7f9d87fff640 1 --2- 172.21.15.52:0/2311411945 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9d880ff8e0 0x7f9d88197980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:44.438 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:44.438+0000 7f9d87fff640 1 -- 172.21.15.52:0/2311411945 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9d88103270 msgr2=0x7f9d8819ef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:28:44.438 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:44.438+0000 7f9d87fff640 1 --2- 172.21.15.52:0/2311411945 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9d88103270 0x7f9d8819ef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:44.438 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:44.438+0000 7f9d87fff640 1 -- 172.21.15.52:0/2311411945 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d8819f5a0 con 0x7f9d881028b0 2024-03-04T09:28:44.438 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:44.438+0000 7f9d8c8ba640 1 --2- 172.21.15.52:0/2311411945 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9d880ff8e0 0x7f9d88197980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-04T09:28:44.438 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:44.438+0000 7f9d87fff640 1 --2- 172.21.15.52:0/2311411945 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9d881028b0 0x7f9d88197ec0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f9d7000ea60 tx=0x7f9d7000ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:28:44.438 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:44.438+0000 7f9d85ffb640 1 -- 172.21.15.52:0/2311411945 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d7000cde0 con 0x7f9d881028b0 2024-03-04T09:28:44.438 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:44.438+0000 7f9d85ffb640 1 -- 172.21.15.52:0/2311411945 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9d700035f0 con 0x7f9d881028b0 2024-03-04T09:28:44.438 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:44.438+0000 7f9d8eb1e640 1 -- 172.21.15.52:0/2311411945 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d8819f890 con 0x7f9d881028b0 2024-03-04T09:28:44.438 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:44.438+0000 7f9d85ffb640 1 -- 172.21.15.52:0/2311411945 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d70010670 con 0x7f9d881028b0 2024-03-04T09:28:44.438 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:44.438+0000 7f9d8eb1e640 1 -- 172.21.15.52:0/2311411945 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d8819fe30 con 0x7f9d881028b0 2024-03-04T09:28:44.439 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:44.439+0000 7f9d8eb1e640 1 -- 172.21.15.52:0/2311411945 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d88108a60 con 0x7f9d881028b0 2024-03-04T09:28:44.439 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:44.440+0000 7f9d85ffb640 1 -- 172.21.15.52:0/2311411945 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 16) v1 ==== 136756+0+0 (secure 0 0 0) 0x7f9d700037a0 con 0x7f9d881028b0 2024-03-04T09:28:44.440 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:44.441+0000 7f9d85ffb640 1 --2- 172.21.15.52:0/2311411945 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f9d740a1e00 0x7f9d740a42c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:28:44.440 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:44.441+0000 7f9d8c8ba640 1 --2- 172.21.15.52:0/2311411945 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f9d740a1e00 0x7f9d740a42c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:28:44.440 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:44.441+0000 7f9d85ffb640 1 -- 172.21.15.52:0/2311411945 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8802+0+0 (secure 0 0 0) 0x7f9d700ce310 con 0x7f9d881028b0 2024-03-04T09:28:44.440 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:44.441+0000 7f9d8c8ba640 1 --2- 172.21.15.52:0/2311411945 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f9d740a1e00 0x7f9d740a42c0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f9d80006010 tx=0x7f9d80009940 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:28:44.441 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:44.445+0000 7f9d85ffb640 1 -- 172.21.15.52:0/2311411945 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f9d7009b500 con 0x7f9d881028b0 2024-03-04T09:28:44.625 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:44.628+0000 7f9d8eb1e640 1 -- 172.21.15.52:0/2311411945 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9d880621d0 con 0x7f9d881028b0 2024-03-04T09:28:44.627 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:44.630+0000 7f9d85ffb640 1 -- 172.21.15.52:0/2311411945 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) v1 ==== 74+0+24164 (secure 0 0 0) 0x7f9d7009ae00 con 0x7f9d881028b0 2024-03-04T09:28:44.627 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-03-04T09:28:44.627 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":89,"fsid":"4e059f08-da08-11ee-95c6-87774f69a715","created":"2024-03-04T09:20:46.104963+0000","modified":"2024-03-04T09:28:42.295274+0000","last_up_change":"2024-03-04T09:27:49.261365+0000","last_in_change":"2024-03-04T09:27:30.024964+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-04T09:24:01.742461+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-04T09:28:35.835600+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-04T09:28:36.453099+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-04T09:28:40.168831+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"583172be-602f-4bf4-bfcb-42663cf5b277","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6802","nonce":1371221397},{"type":"v1","addr":"172.21.15.52:6803","nonce":1371221397}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6804","nonce":1371221397},{"type":"v1","addr":"172.21.15.52:6805","nonce":1371221397}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6808","nonce":1371221397},{"type":"v1","addr":"172.21.15.52:6809","nonce":1371221397}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6806","nonce":1371221397},{"type":"v1","addr":"172.21.15.52:6807","nonce":1371221397}]},"public_addr":"172.21.15.52:6803/1371221397","cluster_addr":"172.21.15.52:6805/1371221397","heartbeat_back_addr":"172.21.15.52:6809/1371221397","heartbeat_front_addr":"172.21.15.52:6807/1371221397","state":["exists","up"]},{"osd":1,"uuid":"5ae0081c-a2f6-46eb-ace0-0368208377b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":2540865537},{"type":"v1","addr":"172.21.15.106:6801","nonce":2540865537}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":2540865537},{"type":"v1","addr":"172.21.15.106:6803","nonce":2540865537}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":2540865537},{"type":"v1","addr":"172.21.15.106:6807","nonce":2540865537}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":2540865537},{"type":"v1","addr":"172.21.15.106:6805","nonce":2540865537}]},"public_addr":"172.21.15.106:6801/2540865537","cluster_addr":"172.21.15.106:6803/2540865537","heartbeat_back_addr":"172.21.15.106:6807/2540865537","heartbeat_front_addr":"172.21.15.106:6805/2540865537","state":["exists","up"]},{"osd":2,"uuid":"1948bac7-2bb2-4218-8b93-4e4743a0de04","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":1377083164},{"type":"v1","addr":"172.21.15.191:6801","nonce":1377083164}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":1377083164},{"type":"v1","addr":"172.21.15.191:6803","nonce":1377083164}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":1377083164},{"type":"v1","addr":"172.21.15.191:6807","nonce":1377083164}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":1377083164},{"type":"v1","addr":"172.21.15.191:6805","nonce":1377083164}]},"public_addr":"172.21.15.191:6801/1377083164","cluster_addr":"172.21.15.191:6803/1377083164","heartbeat_back_addr":"172.21.15.191:6807/1377083164","heartbeat_front_addr":"172.21.15.191:6805/1377083164","state":["exists","up"]},{"osd":3,"uuid":"16c9e9fe-7167-4992-bf12-1a7398e0498a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6810","nonce":2388571810},{"type":"v1","addr":"172.21.15.52:6811","nonce":2388571810}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6812","nonce":2388571810},{"type":"v1","addr":"172.21.15.52:6813","nonce":2388571810}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6816","nonce":2388571810},{"type":"v1","addr":"172.21.15.52:6817","nonce":2388571810}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6814","nonce":2388571810},{"type":"v1","addr":"172.21.15.52:6815","nonce":2388571810}]},"public_addr":"172.21.15.52:6811/2388571810","cluster_addr":"172.21.15.52:6813/2388571810","heartbeat_back_addr":"172.21.15.52:6817/2388571810","heartbeat_front_addr":"172.21.15.52:6815/2388571810","state":["exists","up"]},{"osd":4,"uuid":"c8ef48c7-7c78-44d3-a9e5-99ec750abd3d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6808","nonce":3465868015},{"type":"v1","addr":"172.21.15.106:6809","nonce":3465868015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6810","nonce":3465868015},{"type":"v1","addr":"172.21.15.106:6811","nonce":3465868015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6814","nonce":3465868015},{"type":"v1","addr":"172.21.15.106:6815","nonce":3465868015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6812","nonce":3465868015},{"type":"v1","addr":"172.21.15.106:6813","nonce":3465868015}]},"public_addr":"172.21.15.106:6809/3465868015","cluster_addr":"172.21.15.106:6811/3465868015","heartbeat_back_addr":"172.21.15.106:6815/3465868015","heartbeat_front_addr":"172.21.15.106:6813/3465868015","state":["exists","up"]},{"osd":5,"uuid":"9e41824a-c5a2-4f88-be50-2ba1dceb3220","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":1485092643},{"type":"v1","addr":"172.21.15.191:6809","nonce":1485092643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":1485092643},{"type":"v1","addr":"172.21.15.191:6811","nonce":1485092643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":1485092643},{"type":"v1","addr":"172.21.15.191:6815","nonce":1485092643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":1485092643},{"type":"v1","addr":"172.21.15.191:6813","nonce":1485092643}]},"public_addr":"172.21.15.191:6809/1485092643","cluster_addr":"172.21.15.191:6811/1485092643","heartbeat_back_addr":"172.21.15.191:6815/1485092643","heartbeat_front_addr":"172.21.15.191:6813/1485092643","state":["exists","up"]},{"osd":6,"uuid":"b35dccee-ec16-452e-ac69-3ac2fb7b27fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6818","nonce":1989797856},{"type":"v1","addr":"172.21.15.52:6819","nonce":1989797856}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6820","nonce":1989797856},{"type":"v1","addr":"172.21.15.52:6821","nonce":1989797856}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6824","nonce":1989797856},{"type":"v1","addr":"172.21.15.52:6825","nonce":1989797856}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6822","nonce":1989797856},{"type":"v1","addr":"172.21.15.52:6823","nonce":1989797856}]},"public_addr":"172.21.15.52:6819/1989797856","cluster_addr":"172.21.15.52:6821/1989797856","heartbeat_back_addr":"172.21.15.52:6825/1989797856","heartbeat_front_addr":"172.21.15.52:6823/1989797856","state":["exists","up"]},{"osd":7,"uuid":"71303e3d-14ec-4cbd-b19c-67645a1de3c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6816","nonce":1450870329},{"type":"v1","addr":"172.21.15.106:6817","nonce":1450870329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":1450870329},{"type":"v1","addr":"172.21.15.106:6819","nonce":1450870329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":1450870329},{"type":"v1","addr":"172.21.15.106:6823","nonce":1450870329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6820","nonce":1450870329},{"type":"v1","addr":"172.21.15.106:6821","nonce":1450870329}]},"public_addr":"172.21.15.106:6817/1450870329","cluster_addr":"172.21.15.106:6819/1450870329","heartbeat_back_addr":"172.21.15.106:6823/1450870329","heartbeat_front_addr":"172.21.15.106:6821/1450870329","state":["exists","up"]},{"osd":8,"uuid":"6f5786c4-4402-435a-bdad-adff46fe9473","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":1114119072},{"type":"v1","addr":"172.21.15.191:6817","nonce":1114119072}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6818","nonce":1114119072},{"type":"v1","addr":"172.21.15.191:6819","nonce":1114119072}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":1114119072},{"type":"v1","addr":"172.21.15.191:6823","nonce":1114119072}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":1114119072},{"type":"v1","addr":"172.21.15.191:6821","nonce":1114119072}]},"public_addr":"172.21.15.191:6817/1114119072","cluster_addr":"172.21.15.191:6819/1114119072","heartbeat_back_addr":"172.21.15.191:6823/1114119072","heartbeat_front_addr":"172.21.15.191:6821/1114119072","state":["exists","up"]},{"osd":9,"uuid":"062bbaa5-1227-4684-b859-450f5594a90c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6826","nonce":3485821914},{"type":"v1","addr":"172.21.15.52:6827","nonce":3485821914}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6828","nonce":3485821914},{"type":"v1","addr":"172.21.15.52:6829","nonce":3485821914}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6832","nonce":3485821914},{"type":"v1","addr":"172.21.15.52:6833","nonce":3485821914}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6830","nonce":3485821914},{"type":"v1","addr":"172.21.15.52:6831","nonce":3485821914}]},"public_addr":"172.21.15.52:6827/3485821914","cluster_addr":"172.21.15.52:6829/3485821914","heartbeat_back_addr":"172.21.15.52:6833/3485821914","heartbeat_front_addr":"172.21.15.52:6831/3485821914","state":["exists","up"]},{"osd":10,"uuid":"1a285a5a-f793-48da-a8eb-5326d9528cd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":3815586576},{"type":"v1","addr":"172.21.15.106:6825","nonce":3815586576}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6826","nonce":3815586576},{"type":"v1","addr":"172.21.15.106:6827","nonce":3815586576}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":3815586576},{"type":"v1","addr":"172.21.15.106:6831","nonce":3815586576}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":3815586576},{"type":"v1","addr":"172.21.15.106:6829","nonce":3815586576}]},"public_addr":"172.21.15.106:6825/3815586576","cluster_addr":"172.21.15.106:6827/3815586576","heartbeat_back_addr":"172.21.15.106:6831/3815586576","heartbeat_front_addr":"172.21.15.106:6829/3815586576","state":["exists","up"]},{"osd":11,"uuid":"1f8b9a43-57fa-41f4-9969-3b803a74bf45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":2009632843},{"type":"v1","addr":"172.21.15.191:6825","nonce":2009632843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":2009632843},{"type":"v1","addr":"172.21.15.191:6827","nonce":2009632843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":2009632843},{"type":"v1","addr":"172.21.15.191:6831","nonce":2009632843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":2009632843},{"type":"v1","addr":"172.21.15.191:6829","nonce":2009632843}]},"public_addr":"172.21.15.191:6825/2009632843","cluster_addr":"172.21.15.191:6827/2009632843","heartbeat_back_addr":"172.21.15.191:6831/2009632843","heartbeat_front_addr":"172.21.15.191:6829/2009632843","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T09:23:10.894763+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T09:23:34.420055+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T09:23:58.288324+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T09:24:23.673457+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T09:24:48.208561+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T09:25:13.405473+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T09:25:38.670915+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T09:26:04.451257+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T09:26:30.281380+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T09:26:56.543143+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T09:27:22.412930+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T09:27:48.003866+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.52:0/4082645718":"2024-03-05T09:21:39.401935+0000","172.21.15.52:0/1271949988":"2024-03-05T09:21:39.401935+0000","172.21.15.52:6801/2379361333":"2024-03-05T09:21:12.760938+0000","172.21.15.52:0/497075075":"2024-03-05T09:21:12.760938+0000","172.21.15.52:6800/2379361333":"2024-03-05T09:21:12.760938+0000","172.21.15.52:0/568242577":"2024-03-05T09:21:12.760938+0000","172.21.15.52:6801/2425730636":"2024-03-05T09:21:39.401935+0000","172.21.15.52:6800/2425730636":"2024-03-05T09:21:39.401935+0000","172.21.15.52:0/2743084243":"2024-03-05T09:21:39.401935+0000","172.21.15.52:0/1035878034":"2024-03-05T09:21:12.760938+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-04T09:28:44.629 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:44.632+0000 7f9d8eb1e640 1 -- 172.21.15.52:0/2311411945 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f9d740a1e00 msgr2=0x7f9d740a42c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:28:44.629 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:44.632+0000 7f9d8eb1e640 1 --2- 172.21.15.52:0/2311411945 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f9d740a1e00 0x7f9d740a42c0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f9d80006010 tx=0x7f9d80009940 comp rx=0 tx=0).stop 2024-03-04T09:28:44.629 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:44.632+0000 7f9d8eb1e640 1 -- 172.21.15.52:0/2311411945 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9d881028b0 msgr2=0x7f9d88197ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:28:44.629 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:44.632+0000 7f9d8eb1e640 1 --2- 172.21.15.52:0/2311411945 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9d881028b0 0x7f9d88197ec0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f9d7000ea60 tx=0x7f9d7000ef20 comp rx=0 tx=0).stop 2024-03-04T09:28:44.629 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:44.632+0000 7f9d8eb1e640 1 -- 172.21.15.52:0/2311411945 shutdown_connections 2024-03-04T09:28:44.629 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:44.632+0000 7f9d8eb1e640 1 --2- 172.21.15.52:0/2311411945 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f9d740a1e00 0x7f9d740a42c0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:44.630 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:44.632+0000 7f9d8eb1e640 1 --2- 172.21.15.52:0/2311411945 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9d88103270 0x7f9d8819ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:44.630 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:44.632+0000 7f9d8eb1e640 1 --2- 172.21.15.52:0/2311411945 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9d881028b0 0x7f9d88197ec0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:44.630 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:44.632+0000 7f9d8eb1e640 1 --2- 172.21.15.52:0/2311411945 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9d880ff8e0 0x7f9d88197980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:44.630 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:44.632+0000 7f9d8eb1e640 1 -- 172.21.15.52:0/2311411945 >> 172.21.15.52:0/2311411945 conn(0x7f9d880fb210 msgr2=0x7f9d88105c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:28:44.630 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:44.632+0000 7f9d8eb1e640 1 -- 172.21.15.52:0/2311411945 shutdown_connections 2024-03-04T09:28:44.630 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:44.632+0000 7f9d8eb1e640 1 -- 172.21.15.52:0/2311411945 wait complete. 2024-03-04T09:28:44.641 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 17, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-04T09:28:37.264970+0000', 'modified': '2024-03-04T09:28:42.295443+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 24506}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24506': {'gid': 24506, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.106:6839/1995853093', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.106:6838', 'nonce': 1995853093}, {'type': 'v1', 'addr': '172.21.15.106:6839', 'nonce': 1995853093}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 1, 'qdb_leader': 24506, 'qdb_cluster': [24506]} 2024-03-04T09:28:44.641 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-03-04T09:28:44.641 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-03-04T09:28:44.759 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:44.747+0000 7efd493a1640 1 -- 172.21.15.52:0/2047532565 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efd44103230 msgr2=0x7efd44109ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:28:44.760 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:44.747+0000 7efd493a1640 1 --2- 172.21.15.52:0/2047532565 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efd44103230 0x7efd44109ac0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7efd34009cd0 tx=0x7efd3402c0f0 comp rx=0 tx=0).stop 2024-03-04T09:28:44.760 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:44.747+0000 7efd493a1640 1 -- 172.21.15.52:0/2047532565 shutdown_connections 2024-03-04T09:28:44.760 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:44.747+0000 7efd493a1640 1 --2- 172.21.15.52:0/2047532565 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efd44103230 0x7efd44109ac0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:44.760 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:44.747+0000 7efd493a1640 1 --2- 172.21.15.52:0/2047532565 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7efd44102870 0x7efd44102cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:44.760 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:44.747+0000 7efd493a1640 1 --2- 172.21.15.52:0/2047532565 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7efd440ff8a0 0x7efd440ffcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:44.760 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:44.747+0000 7efd493a1640 1 -- 172.21.15.52:0/2047532565 >> 172.21.15.52:0/2047532565 conn(0x7efd440fb1f0 msgr2=0x7efd440fd610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:28:44.760 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:44.747+0000 7efd493a1640 1 -- 172.21.15.52:0/2047532565 shutdown_connections 2024-03-04T09:28:44.760 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:44.748+0000 7efd493a1640 1 -- 172.21.15.52:0/2047532565 wait complete. 2024-03-04T09:28:44.760 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:44.748+0000 7efd493a1640 1 Processor -- start 2024-03-04T09:28:44.760 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:44.748+0000 7efd493a1640 1 -- start start 2024-03-04T09:28:44.761 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:44.748+0000 7efd493a1640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7efd440ff8a0 0x7efd44197890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:28:44.761 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:44.748+0000 7efd493a1640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efd44102870 0x7efd44197dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:28:44.761 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:44.748+0000 7efd493a1640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7efd44103230 0x7efd4419ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:28:44.761 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:44.748+0000 7efd493a1640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd4410d270 con 0x7efd44102870 2024-03-04T09:28:44.761 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:44.748+0000 7efd493a1640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd4410d0f0 con 0x7efd440ff8a0 2024-03-04T09:28:44.761 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:44.748+0000 7efd493a1640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd4410d3f0 con 0x7efd44103230 2024-03-04T09:28:44.761 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:44.748+0000 7efd427fc640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efd44102870 0x7efd44197dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:28:44.761 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:44.748+0000 7efd427fc640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efd44102870 0x7efd44197dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53366/0 (socket says 172.21.15.52:53366) 2024-03-04T09:28:44.761 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:44.748+0000 7efd427fc640 1 -- 172.21.15.52:0/461347884 learned_addr learned my addr 172.21.15.52:0/461347884 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-04T09:28:44.762 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:44.748+0000 7efd42ffd640 1 --2- 172.21.15.52:0/461347884 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7efd440ff8a0 0x7efd44197890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:28:44.762 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:44.749+0000 7efd427fc640 1 -- 172.21.15.52:0/461347884 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7efd44103230 msgr2=0x7efd4419ee40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:28:44.762 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:44.749+0000 7efd437fe640 1 --2- 172.21.15.52:0/461347884 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7efd44103230 0x7efd4419ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:28:44.762 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:44.749+0000 7efd427fc640 1 --2- 172.21.15.52:0/461347884 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7efd44103230 0x7efd4419ee40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:44.762 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:44.749+0000 7efd427fc640 1 -- 172.21.15.52:0/461347884 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7efd440ff8a0 msgr2=0x7efd44197890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:28:44.762 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:44.749+0000 7efd427fc640 1 --2- 172.21.15.52:0/461347884 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7efd440ff8a0 0x7efd44197890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:44.762 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:44.749+0000 7efd427fc640 1 -- 172.21.15.52:0/461347884 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd4419f4b0 con 0x7efd44102870 2024-03-04T09:28:44.762 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:44.749+0000 7efd42ffd640 1 --2- 172.21.15.52:0/461347884 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7efd440ff8a0 0x7efd44197890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-04T09:28:44.762 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:44.749+0000 7efd437fe640 1 --2- 172.21.15.52:0/461347884 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7efd44103230 0x7efd4419ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-04T09:28:44.763 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:44.749+0000 7efd427fc640 1 --2- 172.21.15.52:0/461347884 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efd44102870 0x7efd44197dd0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7efd2c00d770 tx=0x7efd2c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:28:44.763 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:44.749+0000 7efd3bfff640 1 -- 172.21.15.52:0/461347884 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd2c017070 con 0x7efd44102870 2024-03-04T09:28:44.763 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:44.749+0000 7efd493a1640 1 -- 172.21.15.52:0/461347884 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd4419f7a0 con 0x7efd44102870 2024-03-04T09:28:44.763 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:44.749+0000 7efd3bfff640 1 -- 172.21.15.52:0/461347884 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7efd2c005500 con 0x7efd44102870 2024-03-04T09:28:44.763 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:44.749+0000 7efd3bfff640 1 -- 172.21.15.52:0/461347884 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd2c004e00 con 0x7efd44102870 2024-03-04T09:28:44.763 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:44.749+0000 7efd493a1640 1 -- 172.21.15.52:0/461347884 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd4419fcf0 con 0x7efd44102870 2024-03-04T09:28:44.763 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:44.751+0000 7efd3bfff640 1 -- 172.21.15.52:0/461347884 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 16) v1 ==== 136756+0+0 (secure 0 0 0) 0x7efd2c004fa0 con 0x7efd44102870 2024-03-04T09:28:44.763 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:44.752+0000 7efd4bf89740 1 Processor -- start 2024-03-04T09:28:44.763 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:44.752+0000 7efd4bf89740 1 -- start start 2024-03-04T09:28:44.763 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:44.752+0000 7efd4bf89740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a3342797b0 0x55a334279bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:28:44.763 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:44.752+0000 7efd4bf89740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a33427a980 0x55a33427ae00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:28:44.764 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:44.752+0000 7efd4bf89740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a33427b340 0x55a334281bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:28:44.764 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:44.752+0000 7efd4bf89740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a3341de6b0 con 0x55a33427b340 2024-03-04T09:28:44.764 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:44.752+0000 7efd4bf89740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a334187a60 con 0x55a3342797b0 2024-03-04T09:28:44.764 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:44.752+0000 7efd4bf89740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a3341da750 con 0x55a33427a980 2024-03-04T09:28:44.764 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:44.752+0000 7efd427fc640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a33427b340 0x55a334281bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:28:44.764 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:44.752+0000 7efd427fc640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a33427b340 0x55a334281bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53378/0 (socket says 172.21.15.52:53378) 2024-03-04T09:28:44.764 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:44.752+0000 7efd427fc640 1 -- 172.21.15.52:0/625802626 learned_addr learned my addr 172.21.15.52:0/625802626 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-04T09:28:44.765 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:44.752+0000 7efd437fe640 1 --2- 172.21.15.52:0/625802626 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a3342797b0 0x55a334279bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:28:44.765 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:44.752+0000 7efd3bfff640 1 --2- 172.21.15.52:0/461347884 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7efd300a1d80 0x7efd300a4240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:28:44.765 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:44.753+0000 7efd42ffd640 1 --2- 172.21.15.52:0/625802626 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a33427a980 0x55a33427ae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:28:44.765 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:44.753+0000 7efd437fe640 1 --2- 172.21.15.52:0/461347884 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7efd300a1d80 0x7efd300a4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:28:44.765 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:44.753+0000 7efd3bfff640 1 -- 172.21.15.52:0/461347884 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8802+0+0 (secure 0 0 0) 0x7efd2c0cda80 con 0x7efd44102870 2024-03-04T09:28:44.765 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:44.753+0000 7efd437fe640 1 -- 172.21.15.52:0/625802626 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a33427a980 msgr2=0x55a33427ae00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T09:28:44.765 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:44.753+0000 7efd437fe640 1 --2- 172.21.15.52:0/625802626 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a33427a980 0x55a33427ae00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:44.766 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:44.753+0000 7efd437fe640 1 -- 172.21.15.52:0/625802626 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a33427b340 msgr2=0x55a334281bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T09:28:44.766 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:44.753+0000 7efd437fe640 1 --2- 172.21.15.52:0/625802626 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a33427b340 0x55a334281bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:44.766 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:44.753+0000 7efd437fe640 1 -- 172.21.15.52:0/625802626 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a3341a5f30 con 0x55a3342797b0 2024-03-04T09:28:44.766 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:44.753+0000 7efd427fc640 1 --2- 172.21.15.52:0/625802626 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a33427b340 0x55a334281bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-04T09:28:44.766 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:44.753+0000 7efd437fe640 1 --2- 172.21.15.52:0/625802626 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a3342797b0 0x55a334279bd0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7efd34005e30 tx=0x7efd34032f00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=35572d42075c9517 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:28:44.766 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:44.753+0000 7efd437fe640 1 --2- 172.21.15.52:0/461347884 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7efd300a1d80 0x7efd300a4240 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7efd34045000 tx=0x7efd34048000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:28:44.767 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:44.753+0000 7efd397fa640 1 -- 172.21.15.52:0/625802626 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd34039070 con 0x55a3342797b0 2024-03-04T09:28:44.767 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:44.753+0000 7efd397fa640 1 -- 172.21.15.52:0/625802626 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7efd34004520 con 0x55a3342797b0 2024-03-04T09:28:44.767 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:44.753+0000 7efd397fa640 1 -- 172.21.15.52:0/625802626 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd34034710 con 0x55a3342797b0 2024-03-04T09:28:44.767 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:44.754+0000 7efd4bf89740 1 -- 172.21.15.52:0/625802626 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a3342797b0 msgr2=0x55a334279bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:28:44.767 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:44.754+0000 7efd4bf89740 1 --2- 172.21.15.52:0/625802626 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a3342797b0 0x55a334279bd0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7efd34005e30 tx=0x7efd34032f00 comp rx=0 tx=0).stop 2024-03-04T09:28:44.767 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:44.754+0000 7efd4bf89740 1 -- 172.21.15.52:0/625802626 shutdown_connections 2024-03-04T09:28:44.767 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:44.754+0000 7efd4bf89740 1 --2- 172.21.15.52:0/625802626 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a33427b340 0x55a334281bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:44.767 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:44.754+0000 7efd4bf89740 1 --2- 172.21.15.52:0/625802626 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a33427a980 0x55a33427ae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:44.767 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:44.754+0000 7efd4bf89740 1 --2- 172.21.15.52:0/625802626 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a3342797b0 0x55a334279bd0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:44.767 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:44.754+0000 7efd4bf89740 1 -- 172.21.15.52:0/625802626 >> 172.21.15.52:0/625802626 conn(0x55a334274ba0 msgr2=0x55a334274fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:28:44.768 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:44.754+0000 7efd4bf89740 1 -- 172.21.15.52:0/625802626 shutdown_connections 2024-03-04T09:28:44.768 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:44.754+0000 7efd4bf89740 1 -- 172.21.15.52:0/625802626 wait complete. 2024-03-04T09:28:44.768 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:44.754+0000 7efd4bf89740 1 -- start start 2024-03-04T09:28:44.768 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:44.754+0000 7efd4bf89740 1 Processor -- start 2024-03-04T09:28:44.768 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:44.755+0000 7efd4bf89740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a3342797b0 0x55a334283db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:28:44.768 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:44.755+0000 7efd4bf89740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a33427a980 0x55a3342842f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:28:44.768 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:44.755+0000 7efd4bf89740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a33427b340 0x55a33428b330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:28:44.768 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:44.755+0000 7efd4bf89740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a3342822f0 con 0x55a33427a980 2024-03-04T09:28:44.768 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:44.755+0000 7efd4bf89740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a33419aea0 con 0x55a3342797b0 2024-03-04T09:28:44.768 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:44.755+0000 7efd4bf89740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a334282470 con 0x55a33427b340 2024-03-04T09:28:44.768 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:44.755+0000 7efd4bf89740 10 client.0 _ll_register_callbacks cb 0x55a33419a870 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-04T09:28:44.768 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:44.755+0000 7efd427fc640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a33427a980 0x55a3342842f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:28:44.768 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:44.755+0000 7efd427fc640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a33427a980 0x55a3342842f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53380/0 (socket says 172.21.15.52:53380) 2024-03-04T09:28:44.769 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:44.755+0000 7efd427fc640 1 -- 172.21.15.52:0/1333585702 learned_addr learned my addr 172.21.15.52:0/1333585702 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-04T09:28:44.769 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:44.755+0000 7efd42ffd640 1 --2- 172.21.15.52:0/1333585702 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a3342797b0 0x55a334283db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:28:44.769 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:44.755+0000 7efd437fe640 1 --2- 172.21.15.52:0/1333585702 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a33427b340 0x55a33428b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:28:44.769 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:44.755+0000 7efd427fc640 1 -- 172.21.15.52:0/1333585702 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a33427b340 msgr2=0x55a33428b330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T09:28:44.769 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:44.755+0000 7efd427fc640 1 --2- 172.21.15.52:0/1333585702 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a33427b340 0x55a33428b330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:44.769 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:44.755+0000 7efd427fc640 1 -- 172.21.15.52:0/1333585702 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a3342797b0 msgr2=0x55a334283db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T09:28:44.769 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:44.755+0000 7efd427fc640 1 --2- 172.21.15.52:0/1333585702 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a3342797b0 0x55a334283db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:44.769 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:44.755+0000 7efd427fc640 1 -- 172.21.15.52:0/1333585702 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a33419b270 con 0x55a33427a980 2024-03-04T09:28:44.769 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:44.755+0000 7efd427fc640 1 --2- 172.21.15.52:0/1333585702 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a33427a980 0x55a3342842f0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7efd2c00d740 tx=0x7efd2c096180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82864e01f4bfa25c server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:28:44.769 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:44.755+0000 7efd2a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-03-04T09:28:44.769 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:44.755+0000 7efd2a7fc640 1 -- 172.21.15.52:0/1333585702 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd2c020070 con 0x55a33427a980 2024-03-04T09:28:44.770 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:44.755+0000 7efd2a7fc640 1 -- 172.21.15.52:0/1333585702 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8802+0+0 (secure 0 0 0) 0x7efd2c0cb3f0 con 0x55a33427a980 2024-03-04T09:28:44.770 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:44.755+0000 7efd4bf89740 1 -- 172.21.15.52:0/1333585702 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7efd34004520 con 0x55a33427a980 2024-03-04T09:28:44.770 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:44.755+0000 7efd2a7fc640 1 -- 172.21.15.52:0/1333585702 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) v2 ==== 24+0+0 (secure 0 0 0) 0x7efd2c09a050 con 0x55a33427a980 2024-03-04T09:28:44.770 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:44.755+0000 7efd4bf89740 10 client.14757 fetch_fsmap learned FSMap version 17 2024-03-04T09:28:44.770 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:44.755+0000 7efd4bf89740 1 -- 172.21.15.52:0/1333585702 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=4+}) v3 -- 0x55a333ef32d0 con 0x55a33427a980 2024-03-04T09:28:44.770 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:44.755+0000 7efd2a7fc640 1 -- 172.21.15.52:0/1333585702 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7efd34004520 con 0x55a33427a980 2024-03-04T09:28:44.770 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:44.756+0000 7efd2a7fc640 1 -- 172.21.15.52:0/1333585702 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 17) v1 ==== 7922+0+0 (secure 0 0 0) 0x7efd2c09e150 con 0x55a33427a980 2024-03-04T09:28:44.770 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:44.756+0000 7efd4bf89740 10 client.14757 fetch_fsmap finished waiting for FSMap version 17 2024-03-04T09:28:44.770 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:44.756+0000 7efd4bf89740 10 client.14757 resolve_mds: resolved name 'b' to daemon mds.b 2024-03-04T09:28:44.771 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:44.756+0000 7efd4bf89740 20 client.14757 populate_metadata read hostname 'smithi052' 2024-03-04T09:28:44.771 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:44.756+0000 7efd4bf89740 1 --2- 172.21.15.52:0/1333585702 >> [v2:172.21.15.106:6838/1995853093,v1:172.21.15.106:6839/1995853093] conn(0x55a3342909a0 0x55a33428fa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:28:44.771 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:44.756+0000 7efd4bf89740 4 client.14757 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-04T09:28:44.771 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:44.756+0000 7efd4bf89740 1 -- 172.21.15.52:0/1333585702 --> [v2:172.21.15.106:6838/1995853093,v1:172.21.15.106:6839/1995853093] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a3341ebbf0 con 0x55a3342909a0 2024-03-04T09:28:44.771 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:44.756+0000 7efd42ffd640 1 --2- 172.21.15.52:0/1333585702 >> [v2:172.21.15.106:6838/1995853093,v1:172.21.15.106:6839/1995853093] conn(0x55a3342909a0 0x55a33428fa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:28:44.771 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:44.756+0000 7efd2a7fc640 10 client.14757 ms_handle_connect on v2:172.21.15.106:6838/1995853093 2024-03-04T09:28:44.771 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:44.756+0000 7efd42ffd640 1 --2- 172.21.15.52:0/1333585702 >> [v2:172.21.15.106:6838/1995853093,v1:172.21.15.106:6839/1995853093] conn(0x55a3342909a0 0x55a33428fa60 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8007822359c5d5fc server_cookie=a78a3874a8daa838 in_seq=0 out_seq=0 2024-03-04T09:28:44.772 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:44.757+0000 7efd2a7fc640 1 -- 172.21.15.52:0/1333585702 <== mds.0 v2:172.21.15.106:6838/1995853093 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55a3341ebbf0 con 0x55a3342909a0 2024-03-04T09:28:44.772 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:44.757+0000 7efd2a7fc640 10 client.14757 handle_command_reply: tid=0 2024-03-04T09:28:44.772 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:44.757+0000 7efd4bf89740 1 client.14757 shutdown 2024-03-04T09:28:44.772 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:44.757+0000 7efd4bf89740 1 -- 172.21.15.52:0/1333585702 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a33427a980 msgr2=0x55a3342842f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:28:44.772 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:44.757+0000 7efd4bf89740 1 --2- 172.21.15.52:0/1333585702 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a33427a980 0x55a3342842f0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7efd2c00d740 tx=0x7efd2c096180 comp rx=0 tx=0).stop 2024-03-04T09:28:44.772 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:44.757+0000 7efd4bf89740 1 -- 172.21.15.52:0/1333585702 shutdown_connections 2024-03-04T09:28:44.772 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:44.757+0000 7efd4bf89740 1 --2- 172.21.15.52:0/1333585702 >> [v2:172.21.15.106:6838/1995853093,v1:172.21.15.106:6839/1995853093] conn(0x55a3342909a0 0x55a33428fa60 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:44.772 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:44.757+0000 7efd4bf89740 1 --2- 172.21.15.52:0/1333585702 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a33427b340 0x55a33428b330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:44.773 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:44.757+0000 7efd4bf89740 1 --2- 172.21.15.52:0/1333585702 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a33427a980 0x55a3342842f0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:44.773 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:44.757+0000 7efd4bf89740 1 --2- 172.21.15.52:0/1333585702 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a3342797b0 0x55a334283db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:44.773 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:44.757+0000 7efd4bf89740 1 -- 172.21.15.52:0/1333585702 >> 172.21.15.52:0/1333585702 conn(0x55a334274ba0 msgr2=0x55a334273da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:28:44.773 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:44.757+0000 7efd2a7fc640 0 client.14757 ms_handle_reset on v2:172.21.15.106:6838/1995853093 2024-03-04T09:28:44.773 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:44.757+0000 7efd4bf89740 1 -- 172.21.15.52:0/1333585702 shutdown_connections 2024-03-04T09:28:44.773 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:44.757+0000 7efd4bf89740 1 -- 172.21.15.52:0/1333585702 wait complete. 2024-03-04T09:28:44.773 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:44.757+0000 7efd4bf89740 20 client.14757 trim_cache size 0 max 16384 2024-03-04T09:28:44.784 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:44.774+0000 7efd4bf89740 1 Processor -- start 2024-03-04T09:28:44.784 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:44.774+0000 7efd4bf89740 1 -- start start 2024-03-04T09:28:44.784 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:44.774+0000 7efd4bf89740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a3342797b0 0x55a33427da20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:28:44.784 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:44.774+0000 7efd4bf89740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a33427a980 0x55a33428b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:28:44.784 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:44.774+0000 7efd4bf89740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a33427b340 0x55a3342839c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:28:44.785 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:44.774+0000 7efd4bf89740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a3342901c0 con 0x55a33427a980 2024-03-04T09:28:44.785 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:44.774+0000 7efd4bf89740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a334290820 con 0x55a33427b340 2024-03-04T09:28:44.785 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:44.774+0000 7efd4bf89740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a334290e80 con 0x55a3342797b0 2024-03-04T09:28:44.785 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:44.774+0000 7efd427fc640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a33427a980 0x55a33428b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:28:44.785 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:44.774+0000 7efd427fc640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a33427a980 0x55a33428b2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53392/0 (socket says 172.21.15.52:53392) 2024-03-04T09:28:44.785 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:44.774+0000 7efd427fc640 1 -- 172.21.15.52:0/3174724612 learned_addr learned my addr 172.21.15.52:0/3174724612 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-04T09:28:44.785 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:44.774+0000 7efd437fe640 1 --2- 172.21.15.52:0/3174724612 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a33427b340 0x55a3342839c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:28:44.785 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:44.775+0000 7efd42ffd640 1 --2- 172.21.15.52:0/3174724612 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a3342797b0 0x55a33427da20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:28:44.785 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:44.775+0000 7efd427fc640 1 -- 172.21.15.52:0/3174724612 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a3342797b0 msgr2=0x55a33427da20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T09:28:44.785 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:44.775+0000 7efd427fc640 1 --2- 172.21.15.52:0/3174724612 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a3342797b0 0x55a33427da20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:44.785 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:44.775+0000 7efd427fc640 1 -- 172.21.15.52:0/3174724612 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a33427b340 msgr2=0x55a3342839c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T09:28:44.786 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:44.775+0000 7efd427fc640 1 --2- 172.21.15.52:0/3174724612 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a33427b340 0x55a3342839c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:44.786 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:44.775+0000 7efd427fc640 1 -- 172.21.15.52:0/3174724612 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a333fbd6b0 con 0x55a33427a980 2024-03-04T09:28:44.786 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:44.775+0000 7efd427fc640 1 --2- 172.21.15.52:0/3174724612 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a33427a980 0x55a33428b2f0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7efd2c09a210 tx=0x7efd2c009740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=540f945ac56cd021 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:28:44.786 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:44.775+0000 7efd38ff9640 1 -- 172.21.15.52:0/3174724612 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd2c020070 con 0x55a33427a980 2024-03-04T09:28:44.786 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:44.775+0000 7efd38ff9640 1 -- 172.21.15.52:0/3174724612 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7efd2c0046c0 con 0x55a33427a980 2024-03-04T09:28:44.786 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:44.775+0000 7efd38ff9640 1 -- 172.21.15.52:0/3174724612 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd2c0d3080 con 0x55a33427a980 2024-03-04T09:28:44.786 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:44.775+0000 7efd4bf89740 1 -- 172.21.15.52:0/3174724612 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a33427a980 msgr2=0x55a33428b2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:28:44.786 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:44.775+0000 7efd4bf89740 1 --2- 172.21.15.52:0/3174724612 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a33427a980 0x55a33428b2f0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7efd2c09a210 tx=0x7efd2c009740 comp rx=0 tx=0).stop 2024-03-04T09:28:44.786 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:44.776+0000 7efd4bf89740 1 -- 172.21.15.52:0/3174724612 shutdown_connections 2024-03-04T09:28:44.787 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:44.776+0000 7efd4bf89740 1 --2- 172.21.15.52:0/3174724612 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a33427b340 0x55a3342839c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:44.787 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:44.776+0000 7efd4bf89740 1 --2- 172.21.15.52:0/3174724612 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a33427a980 0x55a33428b2f0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:44.787 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:44.776+0000 7efd4bf89740 1 --2- 172.21.15.52:0/3174724612 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a3342797b0 0x55a33427da20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:44.787 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:44.776+0000 7efd4bf89740 1 -- 172.21.15.52:0/3174724612 >> 172.21.15.52:0/3174724612 conn(0x55a334274ba0 msgr2=0x55a334274f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:28:44.787 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:44.776+0000 7efd4bf89740 1 -- 172.21.15.52:0/3174724612 shutdown_connections 2024-03-04T09:28:44.787 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:44.776+0000 7efd4bf89740 1 -- 172.21.15.52:0/3174724612 wait complete. 2024-03-04T09:28:44.788 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:44.776+0000 7efd4bf89740 1 -- start start 2024-03-04T09:28:44.788 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:44.776+0000 7efd4bf89740 1 Processor -- start 2024-03-04T09:28:44.788 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:44.776+0000 7efd4bf89740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a3342797b0 0x55a334285510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:28:44.788 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:44.776+0000 7efd4bf89740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a33427a980 0x55a334285a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:28:44.788 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:44.776+0000 7efd4bf89740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a33427b340 0x55a33428a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:28:44.788 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:44.776+0000 7efd4bf89740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a334291b40 con 0x55a3342797b0 2024-03-04T09:28:44.788 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:44.776+0000 7efd4bf89740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a3342914e0 con 0x55a33427b340 2024-03-04T09:28:44.788 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:44.776+0000 7efd4bf89740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a3342921a0 con 0x55a33427a980 2024-03-04T09:28:44.788 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:44.776+0000 7efd4bf89740 10 client.0 _ll_register_callbacks cb 0x7efd2c00de80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-04T09:28:44.788 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:44.776+0000 7efd42ffd640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a3342797b0 0x55a334285510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:28:44.789 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:44.776+0000 7efd42ffd640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a3342797b0 0x55a334285510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53404/0 (socket says 172.21.15.52:53404) 2024-03-04T09:28:44.789 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:44.776+0000 7efd42ffd640 1 -- 172.21.15.52:0/2188507699 learned_addr learned my addr 172.21.15.52:0/2188507699 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-04T09:28:44.789 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:44.776+0000 7efd437fe640 1 --2- 172.21.15.52:0/2188507699 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a33427b340 0x55a33428a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:28:44.789 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:44.776+0000 7efd427fc640 1 --2- 172.21.15.52:0/2188507699 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a33427a980 0x55a334285a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:28:44.789 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:44.776+0000 7efd42ffd640 1 -- 172.21.15.52:0/2188507699 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a33427a980 msgr2=0x55a334285a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T09:28:44.789 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:44.776+0000 7efd42ffd640 1 --2- 172.21.15.52:0/2188507699 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a33427a980 0x55a334285a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:44.789 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:44.776+0000 7efd42ffd640 1 -- 172.21.15.52:0/2188507699 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a33427b340 msgr2=0x55a33428a9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T09:28:44.789 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:44.776+0000 7efd42ffd640 1 --2- 172.21.15.52:0/2188507699 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a33427b340 0x55a33428a9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:44.790 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:44.776+0000 7efd42ffd640 1 -- 172.21.15.52:0/2188507699 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a33428b010 con 0x55a3342797b0 2024-03-04T09:28:44.790 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:44.776+0000 7efd427fc640 1 --2- 172.21.15.52:0/2188507699 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a33427a980 0x55a334285a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-04T09:28:44.790 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:44.777+0000 7efd42ffd640 1 --2- 172.21.15.52:0/2188507699 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a3342797b0 0x55a334285510 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7efd3c005e00 tx=0x7efd3c012050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd611f8b713b56dd server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:28:44.791 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:44.777+0000 7efd2a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-03-04T09:28:44.791 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:44.777+0000 7efd2a7fc640 1 -- 172.21.15.52:0/2188507699 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd3c023070 con 0x55a3342797b0 2024-03-04T09:28:44.791 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:44.777+0000 7efd2a7fc640 1 -- 172.21.15.52:0/2188507699 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8802+0+0 (secure 0 0 0) 0x7efd3c012da0 con 0x55a3342797b0 2024-03-04T09:28:44.792 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:44.777+0000 7efd4bf89740 1 -- 172.21.15.52:0/2188507699 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a3341e5f40 con 0x55a3342797b0 2024-03-04T09:28:44.792 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:44.777+0000 7efd2a7fc640 1 -- 172.21.15.52:0/2188507699 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) v2 ==== 24+0+0 (secure 0 0 0) 0x7efd3c02f050 con 0x55a3342797b0 2024-03-04T09:28:44.792 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:44.777+0000 7efd4bf89740 10 client.14763 fetch_fsmap learned FSMap version 17 2024-03-04T09:28:44.793 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:44.777+0000 7efd4bf89740 1 -- 172.21.15.52:0/2188507699 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=4+}) v3 -- 0x55a3342ec560 con 0x55a3342797b0 2024-03-04T09:28:44.793 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:44.777+0000 7efd2a7fc640 1 -- 172.21.15.52:0/2188507699 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55a3341e5f40 con 0x55a3342797b0 2024-03-04T09:28:44.793 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:44.777+0000 7efd2a7fc640 1 -- 172.21.15.52:0/2188507699 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 17) v1 ==== 7922+0+0 (secure 0 0 0) 0x7efd3c003ae0 con 0x55a3342797b0 2024-03-04T09:28:44.793 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:44.777+0000 7efd4bf89740 10 client.14763 fetch_fsmap finished waiting for FSMap version 17 2024-03-04T09:28:44.794 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:44.777+0000 7efd4bf89740 10 client.14763 resolve_mds: resolved name 'b' to daemon mds.b 2024-03-04T09:28:44.794 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:44.777+0000 7efd4bf89740 20 client.14763 populate_metadata read hostname 'smithi052' 2024-03-04T09:28:44.794 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:44.777+0000 7efd4bf89740 1 --2- 172.21.15.52:0/2188507699 >> [v2:172.21.15.106:6838/1995853093,v1:172.21.15.106:6839/1995853093] conn(0x55a3342f0b90 0x55a3342ec850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:28:44.794 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:44.777+0000 7efd4bf89740 4 client.14763 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-04T09:28:44.794 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:44.777+0000 7efd4bf89740 1 -- 172.21.15.52:0/2188507699 --> [v2:172.21.15.106:6838/1995853093,v1:172.21.15.106:6839/1995853093] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55a3341649a0 con 0x55a3342f0b90 2024-03-04T09:28:44.795 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:44.777+0000 7efd427fc640 1 --2- 172.21.15.52:0/2188507699 >> [v2:172.21.15.106:6838/1995853093,v1:172.21.15.106:6839/1995853093] conn(0x55a3342f0b90 0x55a3342ec850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:28:44.795 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:44.778+0000 7efd427fc640 1 --2- 172.21.15.52:0/2188507699 >> [v2:172.21.15.106:6838/1995853093,v1:172.21.15.106:6839/1995853093] conn(0x55a3342f0b90 0x55a3342ec850 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=26e28de3ac191972 server_cookie=aed6685541606819 in_seq=0 out_seq=0 2024-03-04T09:28:44.795 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:44.778+0000 7efd2a7fc640 10 client.14763 ms_handle_connect on v2:172.21.15.106:6838/1995853093 2024-03-04T09:28:44.795 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:44.778+0000 7efd2a7fc640 1 -- 172.21.15.52:0/2188507699 <== mds.0 v2:172.21.15.106:6838/1995853093 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x55a3341649a0 con 0x55a3342f0b90 2024-03-04T09:28:44.795 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:44.778+0000 7efd2a7fc640 10 client.14763 handle_command_reply: tid=0 2024-03-04T09:28:44.795 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:44.778+0000 7efd4bf89740 1 client.14763 shutdown 2024-03-04T09:28:44.796 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:44.778+0000 7efd4bf89740 1 -- 172.21.15.52:0/2188507699 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a3342797b0 msgr2=0x55a334285510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:28:44.796 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:44.778+0000 7efd4bf89740 1 --2- 172.21.15.52:0/2188507699 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a3342797b0 0x55a334285510 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7efd3c005e00 tx=0x7efd3c012050 comp rx=0 tx=0).stop 2024-03-04T09:28:44.796 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:44.778+0000 7efd4bf89740 1 -- 172.21.15.52:0/2188507699 shutdown_connections 2024-03-04T09:28:44.796 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:44.778+0000 7efd4bf89740 1 --2- 172.21.15.52:0/2188507699 >> [v2:172.21.15.106:6838/1995853093,v1:172.21.15.106:6839/1995853093] conn(0x55a3342f0b90 0x55a3342ec850 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:44.796 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:44.778+0000 7efd4bf89740 1 --2- 172.21.15.52:0/2188507699 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a33427b340 0x55a33428a9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:44.796 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:44.778+0000 7efd4bf89740 1 --2- 172.21.15.52:0/2188507699 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a33427a980 0x55a334285a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:44.797 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:44.778+0000 7efd4bf89740 1 --2- 172.21.15.52:0/2188507699 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a3342797b0 0x55a334285510 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:44.797 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:44.778+0000 7efd2a7fc640 0 client.14763 ms_handle_reset on v2:172.21.15.106:6838/1995853093 2024-03-04T09:28:44.797 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:44.778+0000 7efd4bf89740 1 -- 172.21.15.52:0/2188507699 >> 172.21.15.52:0/2188507699 conn(0x55a334274ba0 msgr2=0x55a3342839c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:28:44.797 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:44.778+0000 7efd4bf89740 1 -- 172.21.15.52:0/2188507699 shutdown_connections 2024-03-04T09:28:44.797 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:44.778+0000 7efd4bf89740 1 -- 172.21.15.52:0/2188507699 wait complete. 2024-03-04T09:28:44.797 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:44.778+0000 7efd4bf89740 20 client.14763 trim_cache size 0 max 16384 2024-03-04T09:28:44.798 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2024-03-04T09:28:44.798 INFO:teuthology.orchestra.run.smithi052.stdout: "cluster_fsid": "4e059f08-da08-11ee-95c6-87774f69a715", 2024-03-04T09:28:44.798 INFO:teuthology.orchestra.run.smithi052.stdout: "whoami": 0, 2024-03-04T09:28:44.799 INFO:teuthology.orchestra.run.smithi052.stdout: "id": 24506, 2024-03-04T09:28:44.799 INFO:teuthology.orchestra.run.smithi052.stdout: "want_state": "up:active", 2024-03-04T09:28:44.799 INFO:teuthology.orchestra.run.smithi052.stdout: "state": "up:active", 2024-03-04T09:28:44.799 INFO:teuthology.orchestra.run.smithi052.stdout: "fs_name": "cephfs", 2024-03-04T09:28:44.799 INFO:teuthology.orchestra.run.smithi052.stdout: "rank_uptime": 6.8734145560000002, 2024-03-04T09:28:44.799 INFO:teuthology.orchestra.run.smithi052.stdout: "mdsmap_epoch": 17, 2024-03-04T09:28:44.800 INFO:teuthology.orchestra.run.smithi052.stdout: "osdmap_epoch": 89, 2024-03-04T09:28:44.800 INFO:teuthology.orchestra.run.smithi052.stdout: "osdmap_epoch_barrier": 89, 2024-03-04T09:28:44.800 INFO:teuthology.orchestra.run.smithi052.stdout: "uptime": 11.117461045000001 2024-03-04T09:28:44.800 INFO:teuthology.orchestra.run.smithi052.stdout:} 2024-03-04T09:28:44.802 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:44.779+0000 7efd493a1640 1 -- 172.21.15.52:0/461347884 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7efd300a1d80 msgr2=0x7efd300a4240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:28:44.803 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:44.779+0000 7efd493a1640 1 --2- 172.21.15.52:0/461347884 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7efd300a1d80 0x7efd300a4240 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7efd34045000 tx=0x7efd34048000 comp rx=0 tx=0).stop 2024-03-04T09:28:44.803 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:44.779+0000 7efd493a1640 1 -- 172.21.15.52:0/461347884 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efd44102870 msgr2=0x7efd44197dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:28:44.803 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:44.779+0000 7efd493a1640 1 --2- 172.21.15.52:0/461347884 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efd44102870 0x7efd44197dd0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7efd2c00d770 tx=0x7efd2c00dc30 comp rx=0 tx=0).stop 2024-03-04T09:28:44.803 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:44.779+0000 7efd493a1640 1 -- 172.21.15.52:0/461347884 shutdown_connections 2024-03-04T09:28:44.803 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:44.779+0000 7efd493a1640 1 --2- 172.21.15.52:0/461347884 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7efd300a1d80 0x7efd300a4240 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:44.804 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:44.779+0000 7efd493a1640 1 --2- 172.21.15.52:0/461347884 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7efd44103230 0x7efd4419ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:44.804 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:44.779+0000 7efd493a1640 1 --2- 172.21.15.52:0/461347884 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efd44102870 0x7efd44197dd0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:44.804 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:44.779+0000 7efd493a1640 1 --2- 172.21.15.52:0/461347884 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7efd440ff8a0 0x7efd44197890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:44.804 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:44.779+0000 7efd493a1640 1 -- 172.21.15.52:0/461347884 >> 172.21.15.52:0/461347884 conn(0x7efd440fb1f0 msgr2=0x7efd44105b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:28:44.805 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:44.779+0000 7efd493a1640 1 -- 172.21.15.52:0/461347884 shutdown_connections 2024-03-04T09:28:44.805 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:44.779+0000 7efd493a1640 1 -- 172.21.15.52:0/461347884 wait complete. 2024-03-04T09:28:44.805 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-03-04T09:28:44.806 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-03-04T09:28:44.806 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd shell --fsid 4e059f08-da08-11ee-95c6-87774f69a715 -- ceph fs ls 2024-03-04T09:28:45.047 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/4e059f08-da08-11ee-95c6-87774f69a715/mon.a/config 2024-03-04T09:28:45.299 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:28:44 smithi052 ceph-mon[32473]: pgmap v291: 129 pgs: 16 creating+peering, 100 active+clean, 13 unknown; 579 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s wr, 9 op/s 2024-03-04T09:28:45.299 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:28:44 smithi052 ceph-mon[32473]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-03-04T09:28:45.299 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:28:44 smithi052 ceph-mon[32473]: Cluster is now healthy 2024-03-04T09:28:45.299 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:28:44 smithi052 ceph-mon[32473]: from='client.? 172.21.15.52:0/1014113513' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-04T09:28:45.299 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:28:44 smithi052 ceph-mon[32473]: from='client.? 172.21.15.52:0/3453046344' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-04T09:28:45.299 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:28:44 smithi052 ceph-mon[32473]: from='client.? 172.21.15.52:0/2311411945' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-04T09:28:45.336 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:28:44 smithi191 ceph-mon[36816]: pgmap v291: 129 pgs: 16 creating+peering, 100 active+clean, 13 unknown; 579 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s wr, 9 op/s 2024-03-04T09:28:45.336 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:28:44 smithi191 ceph-mon[36816]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-03-04T09:28:45.337 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:28:44 smithi191 ceph-mon[36816]: Cluster is now healthy 2024-03-04T09:28:45.337 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:28:44 smithi191 ceph-mon[36816]: from='client.? 172.21.15.52:0/1014113513' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-04T09:28:45.337 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:28:44 smithi191 ceph-mon[36816]: from='client.? 172.21.15.52:0/3453046344' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-04T09:28:45.337 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:28:44 smithi191 ceph-mon[36816]: from='client.? 172.21.15.52:0/2311411945' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-04T09:28:45.381 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:28:44 smithi106 ceph-mon[36484]: pgmap v291: 129 pgs: 16 creating+peering, 100 active+clean, 13 unknown; 579 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s wr, 9 op/s 2024-03-04T09:28:45.381 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:28:44 smithi106 ceph-mon[36484]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-03-04T09:28:45.381 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:28:44 smithi106 ceph-mon[36484]: Cluster is now healthy 2024-03-04T09:28:45.381 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:28:44 smithi106 ceph-mon[36484]: from='client.? 172.21.15.52:0/1014113513' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-04T09:28:45.381 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:28:44 smithi106 ceph-mon[36484]: from='client.? 172.21.15.52:0/3453046344' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-04T09:28:45.381 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:28:44 smithi106 ceph-mon[36484]: from='client.? 172.21.15.52:0/2311411945' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-04T09:28:46.969 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:46.966+0000 7f23fdfed700 1 -- 172.21.15.52:0/334222189 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f23f81080f0 msgr2=0x7f23f81084d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:28:46.969 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:46.966+0000 7f23fdfed700 1 --2- 172.21.15.52:0/334222189 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f23f81080f0 0x7f23f81084d0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f23e800b0d0 tx=0x7f23e800b3e0 comp rx=0 tx=0).stop 2024-03-04T09:28:46.969 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:46.966+0000 7f23fdfed700 1 -- 172.21.15.52:0/334222189 shutdown_connections 2024-03-04T09:28:46.969 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:46.966+0000 7f23fdfed700 1 --2- 172.21.15.52:0/334222189 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f23f8102ad0 0x7f23f810ef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:46.969 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:46.966+0000 7f23fdfed700 1 --2- 172.21.15.52:0/334222189 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f23f81080f0 0x7f23f81084d0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:46.969 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:46.966+0000 7f23fdfed700 1 --2- 172.21.15.52:0/334222189 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f23f8102110 0x7f23f8102590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:46.969 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:46.966+0000 7f23fdfed700 1 -- 172.21.15.52:0/334222189 >> 172.21.15.52:0/334222189 conn(0x7f23f80791f0 msgr2=0x7f23f8079600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:28:46.970 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:46.966+0000 7f23fdfed700 1 -- 172.21.15.52:0/334222189 shutdown_connections 2024-03-04T09:28:46.970 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:46.966+0000 7f23fdfed700 1 -- 172.21.15.52:0/334222189 wait complete. 2024-03-04T09:28:46.970 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:46.967+0000 7f23fdfed700 1 Processor -- start 2024-03-04T09:28:46.970 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:46.967+0000 7f23fdfed700 1 -- start start 2024-03-04T09:28:46.970 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:46.967+0000 7f23fdfed700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f23f8102110 0x7f23f819e3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:28:46.970 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:46.967+0000 7f23fdfed700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f23f8102ad0 0x7f23f819e920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:28:46.970 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:46.967+0000 7f23fdfed700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f23f81080f0 0x7f23f81985a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:28:46.970 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:46.967+0000 7f23fdfed700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23f81123a0 con 0x7f23f8102110 2024-03-04T09:28:46.970 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:46.967+0000 7f23fdfed700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23f8112220 con 0x7f23f81080f0 2024-03-04T09:28:46.970 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:46.967+0000 7f23fdfed700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23f8112520 con 0x7f23f8102ad0 2024-03-04T09:28:46.970 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:46.967+0000 7f23f77fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f23f8102110 0x7f23f819e3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:28:46.970 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:46.967+0000 7f23f77fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f23f8102110 0x7f23f819e3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53412/0 (socket says 172.21.15.52:53412) 2024-03-04T09:28:46.970 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:46.967+0000 7f23f77fe700 1 -- 172.21.15.52:0/3687393019 learned_addr learned my addr 172.21.15.52:0/3687393019 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-04T09:28:46.970 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:46.967+0000 7f23f7fff700 1 --2- 172.21.15.52:0/3687393019 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f23f81080f0 0x7f23f81985a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:28:46.971 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:46.967+0000 7f23f6ffd700 1 --2- 172.21.15.52:0/3687393019 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f23f8102ad0 0x7f23f819e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:28:46.971 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:46.967+0000 7f23f77fe700 1 -- 172.21.15.52:0/3687393019 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f23f8102ad0 msgr2=0x7f23f819e920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:28:46.971 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:46.967+0000 7f23f77fe700 1 --2- 172.21.15.52:0/3687393019 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f23f8102ad0 0x7f23f819e920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:46.971 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:46.967+0000 7f23f77fe700 1 -- 172.21.15.52:0/3687393019 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f23f81080f0 msgr2=0x7f23f81985a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:28:46.971 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:46.967+0000 7f23f77fe700 1 --2- 172.21.15.52:0/3687393019 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f23f81080f0 0x7f23f81985a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:46.971 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:46.967+0000 7f23f77fe700 1 -- 172.21.15.52:0/3687393019 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23f8198d00 con 0x7f23f8102110 2024-03-04T09:28:46.971 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:46.967+0000 7f23f7fff700 1 --2- 172.21.15.52:0/3687393019 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f23f81080f0 0x7f23f81985a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T09:28:46.971 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:46.968+0000 7f23f77fe700 1 --2- 172.21.15.52:0/3687393019 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f23f8102110 0x7f23f819e3e0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f23e800d010 tx=0x7f23e8007ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:28:46.971 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:46.968+0000 7f23f4ff9700 1 -- 172.21.15.52:0/3687393019 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23e8021070 con 0x7f23f8102110 2024-03-04T09:28:46.971 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:46.968+0000 7f23fdfed700 1 -- 172.21.15.52:0/3687393019 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23f8198f90 con 0x7f23f8102110 2024-03-04T09:28:46.971 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:46.968+0000 7f23f4ff9700 1 -- 172.21.15.52:0/3687393019 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f23e8007de0 con 0x7f23f8102110 2024-03-04T09:28:46.971 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:46.968+0000 7f23f4ff9700 1 -- 172.21.15.52:0/3687393019 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23e801bc40 con 0x7f23f8102110 2024-03-04T09:28:46.971 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:46.968+0000 7f23fdfed700 1 -- 172.21.15.52:0/3687393019 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23f81a52b0 con 0x7f23f8102110 2024-03-04T09:28:46.972 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:46.969+0000 7f23fdfed700 1 -- 172.21.15.52:0/3687393019 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f23f8068d80 con 0x7f23f8102110 2024-03-04T09:28:46.972 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:46.970+0000 7f23f4ff9700 1 -- 172.21.15.52:0/3687393019 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 16) v1 ==== 136756+0+0 (secure 0 0 0) 0x7f23e8024070 con 0x7f23f8102110 2024-03-04T09:28:46.972 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:46.970+0000 7f23f4ff9700 1 --2- 172.21.15.52:0/3687393019 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f23e40a1e10 0x7f23e40a42d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:28:46.972 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:46.971+0000 7f23f6ffd700 1 --2- 172.21.15.52:0/3687393019 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f23e40a1e10 0x7f23e40a42d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:28:46.972 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:46.971+0000 7f23f4ff9700 1 -- 172.21.15.52:0/3687393019 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8802+0+0 (secure 0 0 0) 0x7f23e800f020 con 0x7f23f8102110 2024-03-04T09:28:46.972 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:46.971+0000 7f23f6ffd700 1 --2- 172.21.15.52:0/3687393019 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f23e40a1e10 0x7f23e40a42d0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f23f8079780 tx=0x7f23e0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:28:46.972 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:46.974+0000 7f23f4ff9700 1 -- 172.21.15.52:0/3687393019 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f23e802b030 con 0x7f23f8102110 2024-03-04T09:28:47.225 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:47.227+0000 7f23fdfed700 1 -- 172.21.15.52:0/3687393019 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f23f8199ad0 con 0x7f23f8102110 2024-03-04T09:28:47.225 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:28:46 smithi052 ceph-mon[32473]: pgmap v292: 129 pgs: 16 creating+peering, 109 active+clean, 4 unknown; 579 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s wr, 7 op/s 2024-03-04T09:28:47.225 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:47.228+0000 7f23f4ff9700 1 -- 172.21.15.52:0/3687393019 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v17) v1 ==== 53+0+88 (secure 0 0 0) 0x7f23e809cf10 con 0x7f23f8102110 2024-03-04T09:28:47.225 INFO:teuthology.orchestra.run.smithi052.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-03-04T09:28:47.227 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:47.230+0000 7f23fdfed700 1 -- 172.21.15.52:0/3687393019 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f23e40a1e10 msgr2=0x7f23e40a42d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:28:47.227 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:47.230+0000 7f23fdfed700 1 --2- 172.21.15.52:0/3687393019 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f23e40a1e10 0x7f23e40a42d0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f23f8079780 tx=0x7f23e0009260 comp rx=0 tx=0).stop 2024-03-04T09:28:47.227 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:47.230+0000 7f23fdfed700 1 -- 172.21.15.52:0/3687393019 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f23f8102110 msgr2=0x7f23f819e3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:28:47.227 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:47.230+0000 7f23fdfed700 1 --2- 172.21.15.52:0/3687393019 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f23f8102110 0x7f23f819e3e0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f23e800d010 tx=0x7f23e8007ad0 comp rx=0 tx=0).stop 2024-03-04T09:28:47.227 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:47.230+0000 7f23fdfed700 1 -- 172.21.15.52:0/3687393019 shutdown_connections 2024-03-04T09:28:47.227 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:47.230+0000 7f23fdfed700 1 --2- 172.21.15.52:0/3687393019 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f23e40a1e10 0x7f23e40a42d0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:47.227 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:47.230+0000 7f23fdfed700 1 --2- 172.21.15.52:0/3687393019 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f23f81080f0 0x7f23f81985a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:47.228 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:47.230+0000 7f23fdfed700 1 --2- 172.21.15.52:0/3687393019 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f23f8102110 0x7f23f819e3e0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:47.228 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:47.230+0000 7f23fdfed700 1 --2- 172.21.15.52:0/3687393019 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f23f8102ad0 0x7f23f819e920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:47.228 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:47.230+0000 7f23fdfed700 1 -- 172.21.15.52:0/3687393019 >> 172.21.15.52:0/3687393019 conn(0x7f23f80791f0 msgr2=0x7f23f8100cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:28:47.228 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:47.230+0000 7f23fdfed700 1 -- 172.21.15.52:0/3687393019 shutdown_connections 2024-03-04T09:28:47.228 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:47.230+0000 7f23fdfed700 1 -- 172.21.15.52:0/3687393019 wait complete. 2024-03-04T09:28:47.336 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:28:46 smithi191 ceph-mon[36816]: pgmap v292: 129 pgs: 16 creating+peering, 109 active+clean, 4 unknown; 579 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s wr, 7 op/s 2024-03-04T09:28:47.381 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:28:46 smithi106 ceph-mon[36484]: pgmap v292: 129 pgs: 16 creating+peering, 109 active+clean, 4 unknown; 579 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s wr, 7 op/s 2024-03-04T09:28:47.934 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-03-04T09:28:47.934 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-03-04T09:28:47.934 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi052.front.sepia.ceph.com 2024-03-04T09:28:47.934 INFO:tasks.cephfs.mount:self.client.name = client.admin 2024-03-04T09:28:47.934 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-03-04T09:28:47.934 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-03-04T09:28:47.934 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-03-04T09:28:47.934 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-03-04T09:28:47.934 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.admin' 2024-03-04T09:28:47.935 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-04T09:28:47.935 DEBUG:teuthology.orchestra.run.smithi052:> ip addr 2024-03-04T09:28:47.951 INFO:teuthology.orchestra.run.smithi052.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-04T09:28:47.951 INFO:teuthology.orchestra.run.smithi052.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-04T09:28:47.951 INFO:teuthology.orchestra.run.smithi052.stdout: inet 127.0.0.1/8 scope host lo 2024-03-04T09:28:47.951 INFO:teuthology.orchestra.run.smithi052.stdout: valid_lft forever preferred_lft forever 2024-03-04T09:28:47.951 INFO:teuthology.orchestra.run.smithi052.stdout: inet6 ::1/128 scope host 2024-03-04T09:28:47.951 INFO:teuthology.orchestra.run.smithi052.stdout: valid_lft forever preferred_lft forever 2024-03-04T09:28:47.951 INFO:teuthology.orchestra.run.smithi052.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-04T09:28:47.951 INFO:teuthology.orchestra.run.smithi052.stdout: link/ether 0c:c4:7a:6b:fd:d2 brd ff:ff:ff:ff:ff:ff 2024-03-04T09:28:47.952 INFO:teuthology.orchestra.run.smithi052.stdout: altname enp7s0f0 2024-03-04T09:28:47.952 INFO:teuthology.orchestra.run.smithi052.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-04T09:28:47.952 INFO:teuthology.orchestra.run.smithi052.stdout: link/ether 0c:c4:7a:6b:fd:d3 brd ff:ff:ff:ff:ff:ff 2024-03-04T09:28:47.952 INFO:teuthology.orchestra.run.smithi052.stdout: altname enp7s0f1 2024-03-04T09:28:47.952 INFO:teuthology.orchestra.run.smithi052.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-03-04T09:28:47.952 INFO:teuthology.orchestra.run.smithi052.stdout: link/ether 0c:c4:7a:bd:15:de brd ff:ff:ff:ff:ff:ff 2024-03-04T09:28:47.952 INFO:teuthology.orchestra.run.smithi052.stdout: altname enp1s0f0 2024-03-04T09:28:47.952 INFO:teuthology.orchestra.run.smithi052.stdout: inet 172.21.15.52/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-03-04T09:28:47.952 INFO:teuthology.orchestra.run.smithi052.stdout: valid_lft 42203sec preferred_lft 42203sec 2024-03-04T09:28:47.952 INFO:teuthology.orchestra.run.smithi052.stdout: inet6 fe80::ec4:7aff:febd:15de/64 scope link noprefixroute 2024-03-04T09:28:47.952 INFO:teuthology.orchestra.run.smithi052.stdout: valid_lft forever preferred_lft forever 2024-03-04T09:28:47.952 INFO:teuthology.orchestra.run.smithi052.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-04T09:28:47.952 INFO:teuthology.orchestra.run.smithi052.stdout: link/ether 0c:c4:7a:bd:15:df brd ff:ff:ff:ff:ff:ff 2024-03-04T09:28:47.953 INFO:teuthology.orchestra.run.smithi052.stdout: altname enp1s0f1 2024-03-04T09:28:47.953 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-03-04T09:28:47.953 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-04T09:28:47.953 DEBUG:teuthology.orchestra.run.smithi052:> (cd / && exec bash -c ' 2024-03-04T09:28:47.953 DEBUG:teuthology.orchestra.run.smithi052:> set -e 2024-03-04T09:28:47.954 DEBUG:teuthology.orchestra.run.smithi052:> sudo ip link add name ceph-brx type bridge 2024-03-04T09:28:47.954 DEBUG:teuthology.orchestra.run.smithi052:> sudo ip addr flush dev ceph-brx 2024-03-04T09:28:47.954 DEBUG:teuthology.orchestra.run.smithi052:> sudo ip link set ceph-brx up 2024-03-04T09:28:47.954 DEBUG:teuthology.orchestra.run.smithi052:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-03-04T09:28:47.954 DEBUG:teuthology.orchestra.run.smithi052:> ') 2024-03-04T09:28:48.202 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:28:47 smithi052 ceph-mon[32473]: from='client.? 172.21.15.52:0/3687393019' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-03-04T09:28:48.244 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-04T09:28:48.244 DEBUG:teuthology.orchestra.run.smithi052:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-03-04T09:28:48.272 INFO:teuthology.orchestra.run.smithi052.stdout:1 2024-03-04T09:28:48.273 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-04T09:28:48.273 DEBUG:teuthology.orchestra.run.smithi052:> route 2024-03-04T09:28:48.336 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:28:47 smithi191 ceph-mon[36816]: from='client.? 172.21.15.52:0/3687393019' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-03-04T09:28:48.341 INFO:teuthology.orchestra.run.smithi052.stdout:Kernel IP routing table 2024-03-04T09:28:48.342 INFO:teuthology.orchestra.run.smithi052.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-03-04T09:28:48.342 INFO:teuthology.orchestra.run.smithi052.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-03-04T09:28:48.342 INFO:teuthology.orchestra.run.smithi052.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-03-04T09:28:48.342 INFO:teuthology.orchestra.run.smithi052.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-03-04T09:28:48.342 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-04T09:28:48.342 DEBUG:teuthology.orchestra.run.smithi052:> (cd / && exec bash -c ' 2024-03-04T09:28:48.343 DEBUG:teuthology.orchestra.run.smithi052:> set -e 2024-03-04T09:28:48.343 DEBUG:teuthology.orchestra.run.smithi052:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-03-04T09:28:48.343 DEBUG:teuthology.orchestra.run.smithi052:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-03-04T09:28:48.343 DEBUG:teuthology.orchestra.run.smithi052:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-03-04T09:28:48.343 DEBUG:teuthology.orchestra.run.smithi052:> ') 2024-03-04T09:28:48.381 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:28:47 smithi106 ceph-mon[36484]: from='client.? 172.21.15.52:0/3687393019' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-03-04T09:28:48.593 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-04T09:28:48.593 DEBUG:teuthology.orchestra.run.smithi052:> ip netns list 2024-03-04T09:28:48.608 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-04T09:28:48.608 DEBUG:teuthology.orchestra.run.smithi052:> ip netns list-id 2024-03-04T09:28:48.662 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-04T09:28:48.662 DEBUG:teuthology.orchestra.run.smithi052:> (cd / && exec bash -c ' 2024-03-04T09:28:48.662 DEBUG:teuthology.orchestra.run.smithi052:> set -e 2024-03-04T09:28:48.662 DEBUG:teuthology.orchestra.run.smithi052:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-03-04T09:28:48.662 DEBUG:teuthology.orchestra.run.smithi052:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.admin 0 2024-03-04T09:28:48.663 DEBUG:teuthology.orchestra.run.smithi052:> ') 2024-03-04T09:28:48.745 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.admin' with 192.168.0.1/16 2024-03-04T09:28:48.745 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-04T09:28:48.745 DEBUG:teuthology.orchestra.run.smithi052:> (cd / && exec bash -c ' 2024-03-04T09:28:48.746 DEBUG:teuthology.orchestra.run.smithi052:> set -e 2024-03-04T09:28:48.746 DEBUG:teuthology.orchestra.run.smithi052:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.admin type veth peer name brx.0 2024-03-04T09:28:48.746 DEBUG:teuthology.orchestra.run.smithi052:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-03-04T09:28:48.746 DEBUG:teuthology.orchestra.run.smithi052:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set veth0 up 2024-03-04T09:28:48.746 DEBUG:teuthology.orchestra.run.smithi052:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set lo up 2024-03-04T09:28:48.746 DEBUG:teuthology.orchestra.run.smithi052:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip route add default via 192.168.255.254 2024-03-04T09:28:48.746 DEBUG:teuthology.orchestra.run.smithi052:> ') 2024-03-04T09:28:48.887 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-04T09:28:48.887 DEBUG:teuthology.orchestra.run.smithi052:> (cd / && exec bash -c ' 2024-03-04T09:28:48.887 DEBUG:teuthology.orchestra.run.smithi052:> set -e 2024-03-04T09:28:48.887 DEBUG:teuthology.orchestra.run.smithi052:> sudo ip link set brx.0 up 2024-03-04T09:28:48.887 DEBUG:teuthology.orchestra.run.smithi052:> sudo ip link set dev brx.0 master ceph-brx 2024-03-04T09:28:48.887 DEBUG:teuthology.orchestra.run.smithi052:> ') 2024-03-04T09:28:48.929 INFO:tasks.cephfs.fuse_mount:Client client.admin config is {} 2024-03-04T09:28:48.929 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-04T09:28:48.929 DEBUG:teuthology.orchestra.run.smithi052:> mkdir -p -v /home/ubuntu/cephtest/mnt.admin 2024-03-04T09:28:48.984 INFO:teuthology.orchestra.run.smithi052.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.admin' 2024-03-04T09:28:48.984 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-04T09:28:48.984 DEBUG:teuthology.orchestra.run.smithi052:> chmod 0000 /home/ubuntu/cephtest/mnt.admin 2024-03-04T09:28:49.038 DEBUG:teuthology.orchestra.run.smithi052:> sudo modprobe fuse 2024-03-04T09:28:49.084 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:28:48 smithi052 ceph-mon[32473]: pgmap v293: 129 pgs: 129 active+clean; 579 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s wr, 4 op/s 2024-03-04T09:28:49.105 DEBUG:teuthology.orchestra.run.smithi052:> cat /proc/self/mounts | awk '{print $2}' 2024-03-04T09:28:49.161 INFO:teuthology.orchestra.run.smithi052.stdout:/proc 2024-03-04T09:28:49.161 INFO:teuthology.orchestra.run.smithi052.stdout:/sys 2024-03-04T09:28:49.161 INFO:teuthology.orchestra.run.smithi052.stdout:/dev 2024-03-04T09:28:49.161 INFO:teuthology.orchestra.run.smithi052.stdout:/sys/kernel/security 2024-03-04T09:28:49.161 INFO:teuthology.orchestra.run.smithi052.stdout:/dev/shm 2024-03-04T09:28:49.162 INFO:teuthology.orchestra.run.smithi052.stdout:/dev/pts 2024-03-04T09:28:49.162 INFO:teuthology.orchestra.run.smithi052.stdout:/run 2024-03-04T09:28:49.162 INFO:teuthology.orchestra.run.smithi052.stdout:/sys/fs/cgroup 2024-03-04T09:28:49.162 INFO:teuthology.orchestra.run.smithi052.stdout:/sys/fs/pstore 2024-03-04T09:28:49.162 INFO:teuthology.orchestra.run.smithi052.stdout:/sys/fs/bpf 2024-03-04T09:28:49.162 INFO:teuthology.orchestra.run.smithi052.stdout:/ 2024-03-04T09:28:49.162 INFO:teuthology.orchestra.run.smithi052.stdout:/sys/fs/selinux 2024-03-04T09:28:49.162 INFO:teuthology.orchestra.run.smithi052.stdout:/proc/sys/fs/binfmt_misc 2024-03-04T09:28:49.162 INFO:teuthology.orchestra.run.smithi052.stdout:/dev/hugepages 2024-03-04T09:28:49.162 INFO:teuthology.orchestra.run.smithi052.stdout:/dev/mqueue 2024-03-04T09:28:49.162 INFO:teuthology.orchestra.run.smithi052.stdout:/sys/kernel/debug 2024-03-04T09:28:49.162 INFO:teuthology.orchestra.run.smithi052.stdout:/sys/kernel/tracing 2024-03-04T09:28:49.162 INFO:teuthology.orchestra.run.smithi052.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-03-04T09:28:49.162 INFO:teuthology.orchestra.run.smithi052.stdout:/sys/fs/fuse/connections 2024-03-04T09:28:49.163 INFO:teuthology.orchestra.run.smithi052.stdout:/sys/kernel/config 2024-03-04T09:28:49.163 INFO:teuthology.orchestra.run.smithi052.stdout:/run/credentials/systemd-sysctl.service 2024-03-04T09:28:49.163 INFO:teuthology.orchestra.run.smithi052.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-03-04T09:28:49.163 INFO:teuthology.orchestra.run.smithi052.stdout:/var/lib/nfs/rpc_pipefs 2024-03-04T09:28:49.163 INFO:teuthology.orchestra.run.smithi052.stdout:/run/user/1000 2024-03-04T09:28:49.163 INFO:teuthology.orchestra.run.smithi052.stdout:/var/lib/ceph 2024-03-04T09:28:49.163 INFO:teuthology.orchestra.run.smithi052.stdout:/var/lib/containers/storage/overlay 2024-03-04T09:28:49.163 INFO:teuthology.orchestra.run.smithi052.stdout:/var/lib/containers/storage/overlay/28fa81928f7b73985e9c40377e62b816d263da1f674239c6129fb41df6b6ef53/merged 2024-03-04T09:28:49.163 INFO:teuthology.orchestra.run.smithi052.stdout:/var/lib/containers/storage/overlay/0bce1dd133fe836b76f1cb406dff8e165414ea5f2a823932691245dadab15f1e/merged 2024-03-04T09:28:49.163 INFO:teuthology.orchestra.run.smithi052.stdout:/run/user/0 2024-03-04T09:28:49.163 INFO:teuthology.orchestra.run.smithi052.stdout:/proc/sys/fs/binfmt_misc 2024-03-04T09:28:49.163 INFO:teuthology.orchestra.run.smithi052.stdout:/var/lib/containers/storage/overlay/d816c81d2aeed21266c288384ccb88de9076a667cd03ec8f9978e2af9c574d1a/merged 2024-03-04T09:28:49.163 INFO:teuthology.orchestra.run.smithi052.stdout:/var/lib/containers/storage/overlay/4ef9b280c8e90d23b98798c11ef26be7bd047ee2c5ccc8fc738b787e4ab41d62/merged 2024-03-04T09:28:49.163 INFO:teuthology.orchestra.run.smithi052.stdout:/var/lib/containers/storage/overlay/c86ea3f3802e792e891b596da42826fd2e33901c2bebab843fa7ea32b14ae13f/merged 2024-03-04T09:28:49.163 INFO:teuthology.orchestra.run.smithi052.stdout:/var/lib/containers/storage/overlay/eb0e8ccd92501f6d260e3db4db7dff22c9d69b82bfa906d6ad0340fc7c6a3a13/merged 2024-03-04T09:28:49.164 INFO:teuthology.orchestra.run.smithi052.stdout:/var/lib/containers/storage/overlay/cb71d9f98c5cf6bb09120f66b32886333a2d35960a00425b873b54daf56a4ea2/merged 2024-03-04T09:28:49.164 INFO:teuthology.orchestra.run.smithi052.stdout:/var/lib/containers/storage/overlay/9c73af4912a4ed03a9cfc88b84f04d6565dced049a2f87682f6e85db88fb2a26/merged 2024-03-04T09:28:49.164 INFO:teuthology.orchestra.run.smithi052.stdout:/var/lib/containers/storage/overlay/66809b9231f85b72ab276e8c1a5f6a79b21c3cfa1d3ecc9e2e076641393f259c/merged 2024-03-04T09:28:49.164 INFO:teuthology.orchestra.run.smithi052.stdout:/var/lib/containers/storage/overlay/2fc0e2d8d5cb411340efd593408207c397b1dd17ca36d8ab8cc2744248faa162/merged 2024-03-04T09:28:49.164 INFO:teuthology.orchestra.run.smithi052.stdout:/run/netns 2024-03-04T09:28:49.164 INFO:teuthology.orchestra.run.smithi052.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-03-04T09:28:49.164 INFO:teuthology.orchestra.run.smithi052.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-03-04T09:28:49.164 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-04T09:28:49.164 DEBUG:teuthology.orchestra.run.smithi052:> ls /sys/fs/fuse/connections 2024-03-04T09:28:49.216 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-03-04T09:28:49.216 DEBUG:teuthology.orchestra.run.smithi052:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.admin --id admin --client_fs=cephfs) 2024-03-04T09:28:49.259 DEBUG:teuthology.orchestra.run.smithi052:> sudo modprobe fuse 2024-03-04T09:28:49.285 DEBUG:teuthology.orchestra.run.smithi052:> cat /proc/self/mounts | awk '{print $2}' 2024-03-04T09:28:49.336 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:28:48 smithi191 ceph-mon[36816]: pgmap v293: 129 pgs: 129 active+clean; 579 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s wr, 4 op/s 2024-03-04T09:28:49.341 INFO:teuthology.orchestra.run.smithi052.stdout:/proc 2024-03-04T09:28:49.341 INFO:teuthology.orchestra.run.smithi052.stdout:/sys 2024-03-04T09:28:49.341 INFO:teuthology.orchestra.run.smithi052.stdout:/dev 2024-03-04T09:28:49.341 INFO:teuthology.orchestra.run.smithi052.stdout:/sys/kernel/security 2024-03-04T09:28:49.341 INFO:teuthology.orchestra.run.smithi052.stdout:/dev/shm 2024-03-04T09:28:49.341 INFO:teuthology.orchestra.run.smithi052.stdout:/dev/pts 2024-03-04T09:28:49.341 INFO:teuthology.orchestra.run.smithi052.stdout:/run 2024-03-04T09:28:49.342 INFO:teuthology.orchestra.run.smithi052.stdout:/sys/fs/cgroup 2024-03-04T09:28:49.342 INFO:teuthology.orchestra.run.smithi052.stdout:/sys/fs/pstore 2024-03-04T09:28:49.342 INFO:teuthology.orchestra.run.smithi052.stdout:/sys/fs/bpf 2024-03-04T09:28:49.342 INFO:teuthology.orchestra.run.smithi052.stdout:/ 2024-03-04T09:28:49.342 INFO:teuthology.orchestra.run.smithi052.stdout:/sys/fs/selinux 2024-03-04T09:28:49.342 INFO:teuthology.orchestra.run.smithi052.stdout:/proc/sys/fs/binfmt_misc 2024-03-04T09:28:49.342 INFO:teuthology.orchestra.run.smithi052.stdout:/dev/hugepages 2024-03-04T09:28:49.342 INFO:teuthology.orchestra.run.smithi052.stdout:/dev/mqueue 2024-03-04T09:28:49.342 INFO:teuthology.orchestra.run.smithi052.stdout:/sys/kernel/debug 2024-03-04T09:28:49.342 INFO:teuthology.orchestra.run.smithi052.stdout:/sys/kernel/tracing 2024-03-04T09:28:49.342 INFO:teuthology.orchestra.run.smithi052.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-03-04T09:28:49.342 INFO:teuthology.orchestra.run.smithi052.stdout:/sys/fs/fuse/connections 2024-03-04T09:28:49.342 INFO:teuthology.orchestra.run.smithi052.stdout:/sys/kernel/config 2024-03-04T09:28:49.342 INFO:teuthology.orchestra.run.smithi052.stdout:/run/credentials/systemd-sysctl.service 2024-03-04T09:28:49.343 INFO:teuthology.orchestra.run.smithi052.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-03-04T09:28:49.343 INFO:teuthology.orchestra.run.smithi052.stdout:/var/lib/nfs/rpc_pipefs 2024-03-04T09:28:49.343 INFO:teuthology.orchestra.run.smithi052.stdout:/run/user/1000 2024-03-04T09:28:49.343 INFO:teuthology.orchestra.run.smithi052.stdout:/var/lib/ceph 2024-03-04T09:28:49.343 INFO:teuthology.orchestra.run.smithi052.stdout:/var/lib/containers/storage/overlay 2024-03-04T09:28:49.343 INFO:teuthology.orchestra.run.smithi052.stdout:/var/lib/containers/storage/overlay/28fa81928f7b73985e9c40377e62b816d263da1f674239c6129fb41df6b6ef53/merged 2024-03-04T09:28:49.343 INFO:teuthology.orchestra.run.smithi052.stdout:/var/lib/containers/storage/overlay/0bce1dd133fe836b76f1cb406dff8e165414ea5f2a823932691245dadab15f1e/merged 2024-03-04T09:28:49.343 INFO:teuthology.orchestra.run.smithi052.stdout:/run/user/0 2024-03-04T09:28:49.343 INFO:teuthology.orchestra.run.smithi052.stdout:/proc/sys/fs/binfmt_misc 2024-03-04T09:28:49.343 INFO:teuthology.orchestra.run.smithi052.stdout:/var/lib/containers/storage/overlay/d816c81d2aeed21266c288384ccb88de9076a667cd03ec8f9978e2af9c574d1a/merged 2024-03-04T09:28:49.343 INFO:teuthology.orchestra.run.smithi052.stdout:/var/lib/containers/storage/overlay/4ef9b280c8e90d23b98798c11ef26be7bd047ee2c5ccc8fc738b787e4ab41d62/merged 2024-03-04T09:28:49.343 INFO:teuthology.orchestra.run.smithi052.stdout:/var/lib/containers/storage/overlay/c86ea3f3802e792e891b596da42826fd2e33901c2bebab843fa7ea32b14ae13f/merged 2024-03-04T09:28:49.343 INFO:teuthology.orchestra.run.smithi052.stdout:/var/lib/containers/storage/overlay/eb0e8ccd92501f6d260e3db4db7dff22c9d69b82bfa906d6ad0340fc7c6a3a13/merged 2024-03-04T09:28:49.344 INFO:teuthology.orchestra.run.smithi052.stdout:/var/lib/containers/storage/overlay/cb71d9f98c5cf6bb09120f66b32886333a2d35960a00425b873b54daf56a4ea2/merged 2024-03-04T09:28:49.344 INFO:teuthology.orchestra.run.smithi052.stdout:/var/lib/containers/storage/overlay/9c73af4912a4ed03a9cfc88b84f04d6565dced049a2f87682f6e85db88fb2a26/merged 2024-03-04T09:28:49.344 INFO:teuthology.orchestra.run.smithi052.stdout:/var/lib/containers/storage/overlay/66809b9231f85b72ab276e8c1a5f6a79b21c3cfa1d3ecc9e2e076641393f259c/merged 2024-03-04T09:28:49.344 INFO:teuthology.orchestra.run.smithi052.stdout:/var/lib/containers/storage/overlay/2fc0e2d8d5cb411340efd593408207c397b1dd17ca36d8ab8cc2744248faa162/merged 2024-03-04T09:28:49.344 INFO:teuthology.orchestra.run.smithi052.stdout:/run/netns 2024-03-04T09:28:49.344 INFO:teuthology.orchestra.run.smithi052.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-03-04T09:28:49.344 INFO:teuthology.orchestra.run.smithi052.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-03-04T09:28:49.344 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-04T09:28:49.344 DEBUG:teuthology.orchestra.run.smithi052:> ls /sys/fs/fuse/connections 2024-03-04T09:28:49.357 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi052.stderr:2024-03-04T09:28:49.360+0000 7fda9fe4b480 -1 init, newargv = 0x564063fc1a30 newargc=15 2024-03-04T09:28:49.357 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi052.stderr:ceph-fuse[73880]: starting ceph client 2024-03-04T09:28:49.362 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi052.stderr:ceph-fuse[73880]: starting fuse 2024-03-04T09:28:49.372 INFO:teuthology.orchestra.run.smithi052.stdout:119 2024-03-04T09:28:49.372 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [119] 2024-03-04T09:28:49.372 DEBUG:teuthology.orchestra.run.smithi052:> sudo stdin-killer -- python3 -c ' 2024-03-04T09:28:49.372 DEBUG:teuthology.orchestra.run.smithi052:> import glob 2024-03-04T09:28:49.372 DEBUG:teuthology.orchestra.run.smithi052:> import re 2024-03-04T09:28:49.372 DEBUG:teuthology.orchestra.run.smithi052:> import os 2024-03-04T09:28:49.373 DEBUG:teuthology.orchestra.run.smithi052:> import subprocess 2024-03-04T09:28:49.373 DEBUG:teuthology.orchestra.run.smithi052:> 2024-03-04T09:28:49.373 DEBUG:teuthology.orchestra.run.smithi052:> def _find_admin_socket(client_name): 2024-03-04T09:28:49.373 DEBUG:teuthology.orchestra.run.smithi052:> asok_path = "/var/run/ceph/ceph-client.admin.*.asok" 2024-03-04T09:28:49.373 DEBUG:teuthology.orchestra.run.smithi052:> files = glob.glob(asok_path) 2024-03-04T09:28:49.373 DEBUG:teuthology.orchestra.run.smithi052:> mountpoint = "/home/ubuntu/cephtest/mnt.admin" 2024-03-04T09:28:49.373 DEBUG:teuthology.orchestra.run.smithi052:> 2024-03-04T09:28:49.373 DEBUG:teuthology.orchestra.run.smithi052:> # Given a non-glob path, it better be there 2024-03-04T09:28:49.373 DEBUG:teuthology.orchestra.run.smithi052:> if "*" not in asok_path: 2024-03-04T09:28:49.373 DEBUG:teuthology.orchestra.run.smithi052:> assert(len(files) == 1) 2024-03-04T09:28:49.373 DEBUG:teuthology.orchestra.run.smithi052:> return files[0] 2024-03-04T09:28:49.373 DEBUG:teuthology.orchestra.run.smithi052:> 2024-03-04T09:28:49.373 DEBUG:teuthology.orchestra.run.smithi052:> for f in files: 2024-03-04T09:28:49.373 DEBUG:teuthology.orchestra.run.smithi052:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-03-04T09:28:49.373 DEBUG:teuthology.orchestra.run.smithi052:> if os.path.exists("/proc/{0}".format(pid)): 2024-03-04T09:28:49.373 DEBUG:teuthology.orchestra.run.smithi052:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-03-04T09:28:49.374 DEBUG:teuthology.orchestra.run.smithi052:> contents = proc_f.read() 2024-03-04T09:28:49.374 DEBUG:teuthology.orchestra.run.smithi052:> if mountpoint in contents: 2024-03-04T09:28:49.374 DEBUG:teuthology.orchestra.run.smithi052:> return f 2024-03-04T09:28:49.374 DEBUG:teuthology.orchestra.run.smithi052:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-03-04T09:28:49.374 DEBUG:teuthology.orchestra.run.smithi052:> 2024-03-04T09:28:49.374 DEBUG:teuthology.orchestra.run.smithi052:> print(_find_admin_socket("client.admin")) 2024-03-04T09:28:49.374 DEBUG:teuthology.orchestra.run.smithi052:> ' 2024-03-04T09:28:49.384 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:28:48 smithi106 ceph-mon[36484]: pgmap v293: 129 pgs: 129 active+clean; 579 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s wr, 4 op/s 2024-03-04T09:28:49.494 INFO:teuthology.orchestra.run.smithi052.stdout:/var/run/ceph/ceph-client.admin.73880.asok 2024-03-04T09:28:49.497 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:49 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-04T09:28:49.504 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.admin.73880.asok 2024-03-04T09:28:49.505 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-04T09:28:49.505 DEBUG:teuthology.orchestra.run.smithi052:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.admin.73880.asok status 2024-03-04T09:28:49.642 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2024-03-04T09:28:49.642 INFO:teuthology.orchestra.run.smithi052.stdout: "metadata": { 2024-03-04T09:28:49.642 INFO:teuthology.orchestra.run.smithi052.stdout: "ceph_sha1": "1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd", 2024-03-04T09:28:49.642 INFO:teuthology.orchestra.run.smithi052.stdout: "ceph_version": "ceph version 19.0.0-1739-g1bc250c7 (1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd) squid (dev)", 2024-03-04T09:28:49.643 INFO:teuthology.orchestra.run.smithi052.stdout: "entity_id": "admin", 2024-03-04T09:28:49.643 INFO:teuthology.orchestra.run.smithi052.stdout: "hostname": "smithi052", 2024-03-04T09:28:49.643 INFO:teuthology.orchestra.run.smithi052.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.admin", 2024-03-04T09:28:49.643 INFO:teuthology.orchestra.run.smithi052.stdout: "pid": "73880", 2024-03-04T09:28:49.643 INFO:teuthology.orchestra.run.smithi052.stdout: "root": "/" 2024-03-04T09:28:49.643 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:28:49.643 INFO:teuthology.orchestra.run.smithi052.stdout: "dentry_count": 0, 2024-03-04T09:28:49.643 INFO:teuthology.orchestra.run.smithi052.stdout: "dentry_pinned_count": 0, 2024-03-04T09:28:49.643 INFO:teuthology.orchestra.run.smithi052.stdout: "id": 14775, 2024-03-04T09:28:49.643 INFO:teuthology.orchestra.run.smithi052.stdout: "inst": { 2024-03-04T09:28:49.643 INFO:teuthology.orchestra.run.smithi052.stdout: "name": { 2024-03-04T09:28:49.643 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "client", 2024-03-04T09:28:49.643 INFO:teuthology.orchestra.run.smithi052.stdout: "num": 14775 2024-03-04T09:28:49.643 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:28:49.644 INFO:teuthology.orchestra.run.smithi052.stdout: "addr": { 2024-03-04T09:28:49.644 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "v1", 2024-03-04T09:28:49.644 INFO:teuthology.orchestra.run.smithi052.stdout: "addr": "192.168.0.1:0", 2024-03-04T09:28:49.644 INFO:teuthology.orchestra.run.smithi052.stdout: "nonce": 2532057314 2024-03-04T09:28:49.644 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-03-04T09:28:49.644 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:28:49.644 INFO:teuthology.orchestra.run.smithi052.stdout: "addr": { 2024-03-04T09:28:49.644 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "v1", 2024-03-04T09:28:49.644 INFO:teuthology.orchestra.run.smithi052.stdout: "addr": "192.168.0.1:0", 2024-03-04T09:28:49.644 INFO:teuthology.orchestra.run.smithi052.stdout: "nonce": 2532057314 2024-03-04T09:28:49.644 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:28:49.644 INFO:teuthology.orchestra.run.smithi052.stdout: "inst_str": "client.14775 192.168.0.1:0/2532057314", 2024-03-04T09:28:49.644 INFO:teuthology.orchestra.run.smithi052.stdout: "addr_str": "192.168.0.1:0/2532057314", 2024-03-04T09:28:49.644 INFO:teuthology.orchestra.run.smithi052.stdout: "inode_count": 1, 2024-03-04T09:28:49.645 INFO:teuthology.orchestra.run.smithi052.stdout: "mds_epoch": 17, 2024-03-04T09:28:49.645 INFO:teuthology.orchestra.run.smithi052.stdout: "osd_epoch": 89, 2024-03-04T09:28:49.645 INFO:teuthology.orchestra.run.smithi052.stdout: "osd_epoch_barrier": 0, 2024-03-04T09:28:49.645 INFO:teuthology.orchestra.run.smithi052.stdout: "blocklisted": false, 2024-03-04T09:28:49.645 INFO:teuthology.orchestra.run.smithi052.stdout: "fs_name": "cephfs" 2024-03-04T09:28:49.645 INFO:teuthology.orchestra.run.smithi052.stdout:} 2024-03-04T09:28:49.651 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-04T09:28:49.651 DEBUG:teuthology.orchestra.run.smithi052:> stat --file-system '--printf=%T 2024-03-04T09:28:49.651 DEBUG:teuthology.orchestra.run.smithi052:> ' -- /home/ubuntu/cephtest/mnt.admin 2024-03-04T09:28:49.667 INFO:teuthology.orchestra.run.smithi052.stdout:fuseblk 2024-03-04T09:28:49.667 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.admin 2024-03-04T09:28:49.667 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-04T09:28:49.667 DEBUG:teuthology.orchestra.run.smithi052:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.admin 2024-03-04T09:28:49.734 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-04T09:28:49.735 DEBUG:teuthology.orchestra.run.smithi052:> (cd /home/ubuntu/cephtest/mnt.admin && exec bash -c 'setfattr -n ceph.dir.layout.pool -v cephfs_data_ec . && getfattr -n ceph.dir.layout .') 2024-03-04T09:28:49.808 INFO:teuthology.orchestra.run.smithi052.stdout:# file: . 2024-03-04T09:28:49.808 INFO:teuthology.orchestra.run.smithi052.stdout:ceph.dir.layout="stripe_unit=4194304 stripe_count=1 object_size=4194304 pool=cephfs_data_ec" 2024-03-04T09:28:49.808 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-03-04T09:28:49.809 DEBUG:teuthology.orchestra.run.smithi052:> set -ex 2024-03-04T09:28:49.809 DEBUG:teuthology.orchestra.run.smithi052:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-04T09:28:49.865 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-04T09:28:50.013 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:50.013+0000 7f9305f93640 1 -- 172.21.15.52:0/2633263573 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f93000ffe60 msgr2=0x7f9300100280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:28:50.013 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:50.013+0000 7f9305f93640 1 --2- 172.21.15.52:0/2633263573 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f93000ffe60 0x7f9300100280 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f92f0009ca0 tx=0x7f92f002c0f0 comp rx=0 tx=0).stop 2024-03-04T09:28:50.013 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:50.013+0000 7f9305f93640 1 -- 172.21.15.52:0/2633263573 shutdown_connections 2024-03-04T09:28:50.014 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:50.013+0000 7f9305f93640 1 --2- 172.21.15.52:0/2633263573 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9300101160 0x7f93001015e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:50.014 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:50.013+0000 7f9305f93640 1 --2- 172.21.15.52:0/2633263573 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f93001007c0 0x7f9300100c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:50.014 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:50.013+0000 7f9305f93640 1 --2- 172.21.15.52:0/2633263573 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f93000ffe60 0x7f9300100280 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:50.014 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:50.013+0000 7f9305f93640 1 -- 172.21.15.52:0/2633263573 >> 172.21.15.52:0/2633263573 conn(0x7f93000fba60 msgr2=0x7f93000fdea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:28:50.014 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:50.013+0000 7f9305f93640 1 -- 172.21.15.52:0/2633263573 shutdown_connections 2024-03-04T09:28:50.014 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:50.013+0000 7f9305f93640 1 -- 172.21.15.52:0/2633263573 wait complete. 2024-03-04T09:28:50.014 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:50.014+0000 7f9305f93640 1 Processor -- start 2024-03-04T09:28:50.014 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:50.014+0000 7f9305f93640 1 -- start start 2024-03-04T09:28:50.014 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:50.014+0000 7f9305f93640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f93000ffe60 0x7f9300199ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:28:50.015 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:50.014+0000 7f9305f93640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f93001007c0 0x7f930019a000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:28:50.015 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:50.014+0000 7f9305f93640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9300101160 0x7f93001a1070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:28:50.015 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:50.014+0000 7f9305f93640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f930010f450 con 0x7f93001007c0 2024-03-04T09:28:50.015 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:50.014+0000 7f9305f93640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f930010f2d0 con 0x7f9300101160 2024-03-04T09:28:50.015 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:50.014+0000 7f9305f93640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f930010f5d0 con 0x7f93000ffe60 2024-03-04T09:28:50.015 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:50.014+0000 7f92feffd640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f93001007c0 0x7f930019a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:28:50.015 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:50.014+0000 7f92feffd640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f93001007c0 0x7f930019a000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53440/0 (socket says 172.21.15.52:53440) 2024-03-04T09:28:50.015 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:50.015+0000 7f92fffff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9300101160 0x7f93001a1070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:28:50.015 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:50.015+0000 7f92feffd640 1 -- 172.21.15.52:0/654699875 learned_addr learned my addr 172.21.15.52:0/654699875 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-04T09:28:50.015 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:50.015+0000 7f92ff7fe640 1 --2- 172.21.15.52:0/654699875 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f93000ffe60 0x7f9300199ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:28:50.015 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:50.015+0000 7f92feffd640 1 -- 172.21.15.52:0/654699875 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f93000ffe60 msgr2=0x7f9300199ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:28:50.016 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:50.015+0000 7f92feffd640 1 --2- 172.21.15.52:0/654699875 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f93000ffe60 0x7f9300199ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:50.016 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:50.015+0000 7f92feffd640 1 -- 172.21.15.52:0/654699875 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9300101160 msgr2=0x7f93001a1070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:28:50.016 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:50.015+0000 7f92feffd640 1 --2- 172.21.15.52:0/654699875 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9300101160 0x7f93001a1070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:50.016 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:50.015+0000 7f92feffd640 1 -- 172.21.15.52:0/654699875 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93001a16e0 con 0x7f93001007c0 2024-03-04T09:28:50.016 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:50.015+0000 7f92fffff640 1 --2- 172.21.15.52:0/654699875 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9300101160 0x7f93001a1070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-04T09:28:50.016 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:50.015+0000 7f92feffd640 1 --2- 172.21.15.52:0/654699875 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f93001007c0 0x7f930019a000 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f92f400e9f0 tx=0x7f92f400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:28:50.016 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:50.015+0000 7f92fcff9640 1 -- 172.21.15.52:0/654699875 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92f400cd80 con 0x7f93001007c0 2024-03-04T09:28:50.016 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:50.015+0000 7f9305f93640 1 -- 172.21.15.52:0/654699875 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93001a19d0 con 0x7f93001007c0 2024-03-04T09:28:50.016 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:50.015+0000 7f92fcff9640 1 -- 172.21.15.52:0/654699875 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f92f4010910 con 0x7f93001007c0 2024-03-04T09:28:50.016 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:50.015+0000 7f92fcff9640 1 -- 172.21.15.52:0/654699875 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92f40188f0 con 0x7f93001007c0 2024-03-04T09:28:50.017 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:50.015+0000 7f9305f93640 1 -- 172.21.15.52:0/654699875 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93001a1f20 con 0x7f93001007c0 2024-03-04T09:28:50.017 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:50.016+0000 7f9305f93640 1 -- 172.21.15.52:0/654699875 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f930010ad50 con 0x7f93001007c0 2024-03-04T09:28:50.017 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:50.017+0000 7f92fcff9640 1 -- 172.21.15.52:0/654699875 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 16) v1 ==== 136756+0+0 (secure 0 0 0) 0x7f92f4010ac0 con 0x7f93001007c0 2024-03-04T09:28:50.018 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:50.018+0000 7f92fcff9640 1 --2- 172.21.15.52:0/654699875 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f92e80a1d80 0x7f92e80a4240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:28:50.018 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:50.018+0000 7f92ff7fe640 1 --2- 172.21.15.52:0/654699875 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f92e80a1d80 0x7f92e80a4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:28:50.018 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:50.018+0000 7f92fcff9640 1 -- 172.21.15.52:0/654699875 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8802+0+0 (secure 0 0 0) 0x7f92f40ce890 con 0x7f93001007c0 2024-03-04T09:28:50.018 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:50.018+0000 7f92ff7fe640 1 --2- 172.21.15.52:0/654699875 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f92e80a1d80 0x7f92e80a4240 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f92f0006010 tx=0x7f92f00093f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:28:50.019 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:50.022+0000 7f92fcff9640 1 -- 172.21.15.52:0/654699875 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f92f409ba20 con 0x7f93001007c0 2024-03-04T09:28:50.203 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:50.204+0000 7f9305f93640 1 -- 172.21.15.52:0/654699875 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f930002d240 con 0x7f93001007c0 2024-03-04T09:28:50.203 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:50.205+0000 7f92fcff9640 1 -- 172.21.15.52:0/654699875 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) v1 ==== 74+0+24164 (secure 0 0 0) 0x7f92f409b320 con 0x7f93001007c0 2024-03-04T09:28:50.204 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-03-04T09:28:50.204 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":89,"fsid":"4e059f08-da08-11ee-95c6-87774f69a715","created":"2024-03-04T09:20:46.104963+0000","modified":"2024-03-04T09:28:42.295274+0000","last_up_change":"2024-03-04T09:27:49.261365+0000","last_in_change":"2024-03-04T09:27:30.024964+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-04T09:24:01.742461+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-04T09:28:35.835600+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-04T09:28:36.453099+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-04T09:28:40.168831+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"583172be-602f-4bf4-bfcb-42663cf5b277","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6802","nonce":1371221397},{"type":"v1","addr":"172.21.15.52:6803","nonce":1371221397}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6804","nonce":1371221397},{"type":"v1","addr":"172.21.15.52:6805","nonce":1371221397}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6808","nonce":1371221397},{"type":"v1","addr":"172.21.15.52:6809","nonce":1371221397}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6806","nonce":1371221397},{"type":"v1","addr":"172.21.15.52:6807","nonce":1371221397}]},"public_addr":"172.21.15.52:6803/1371221397","cluster_addr":"172.21.15.52:6805/1371221397","heartbeat_back_addr":"172.21.15.52:6809/1371221397","heartbeat_front_addr":"172.21.15.52:6807/1371221397","state":["exists","up"]},{"osd":1,"uuid":"5ae0081c-a2f6-46eb-ace0-0368208377b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":2540865537},{"type":"v1","addr":"172.21.15.106:6801","nonce":2540865537}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":2540865537},{"type":"v1","addr":"172.21.15.106:6803","nonce":2540865537}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":2540865537},{"type":"v1","addr":"172.21.15.106:6807","nonce":2540865537}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":2540865537},{"type":"v1","addr":"172.21.15.106:6805","nonce":2540865537}]},"public_addr":"172.21.15.106:6801/2540865537","cluster_addr":"172.21.15.106:6803/2540865537","heartbeat_back_addr":"172.21.15.106:6807/2540865537","heartbeat_front_addr":"172.21.15.106:6805/2540865537","state":["exists","up"]},{"osd":2,"uuid":"1948bac7-2bb2-4218-8b93-4e4743a0de04","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":1377083164},{"type":"v1","addr":"172.21.15.191:6801","nonce":1377083164}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":1377083164},{"type":"v1","addr":"172.21.15.191:6803","nonce":1377083164}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":1377083164},{"type":"v1","addr":"172.21.15.191:6807","nonce":1377083164}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":1377083164},{"type":"v1","addr":"172.21.15.191:6805","nonce":1377083164}]},"public_addr":"172.21.15.191:6801/1377083164","cluster_addr":"172.21.15.191:6803/1377083164","heartbeat_back_addr":"172.21.15.191:6807/1377083164","heartbeat_front_addr":"172.21.15.191:6805/1377083164","state":["exists","up"]},{"osd":3,"uuid":"16c9e9fe-7167-4992-bf12-1a7398e0498a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6810","nonce":2388571810},{"type":"v1","addr":"172.21.15.52:6811","nonce":2388571810}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6812","nonce":2388571810},{"type":"v1","addr":"172.21.15.52:6813","nonce":2388571810}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6816","nonce":2388571810},{"type":"v1","addr":"172.21.15.52:6817","nonce":2388571810}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6814","nonce":2388571810},{"type":"v1","addr":"172.21.15.52:6815","nonce":2388571810}]},"public_addr":"172.21.15.52:6811/2388571810","cluster_addr":"172.21.15.52:6813/2388571810","heartbeat_back_addr":"172.21.15.52:6817/2388571810","heartbeat_front_addr":"172.21.15.52:6815/2388571810","state":["exists","up"]},{"osd":4,"uuid":"c8ef48c7-7c78-44d3-a9e5-99ec750abd3d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6808","nonce":3465868015},{"type":"v1","addr":"172.21.15.106:6809","nonce":3465868015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6810","nonce":3465868015},{"type":"v1","addr":"172.21.15.106:6811","nonce":3465868015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6814","nonce":3465868015},{"type":"v1","addr":"172.21.15.106:6815","nonce":3465868015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6812","nonce":3465868015},{"type":"v1","addr":"172.21.15.106:6813","nonce":3465868015}]},"public_addr":"172.21.15.106:6809/3465868015","cluster_addr":"172.21.15.106:6811/3465868015","heartbeat_back_addr":"172.21.15.106:6815/3465868015","heartbeat_front_addr":"172.21.15.106:6813/3465868015","state":["exists","up"]},{"osd":5,"uuid":"9e41824a-c5a2-4f88-be50-2ba1dceb3220","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":1485092643},{"type":"v1","addr":"172.21.15.191:6809","nonce":1485092643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":1485092643},{"type":"v1","addr":"172.21.15.191:6811","nonce":1485092643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":1485092643},{"type":"v1","addr":"172.21.15.191:6815","nonce":1485092643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":1485092643},{"type":"v1","addr":"172.21.15.191:6813","nonce":1485092643}]},"public_addr":"172.21.15.191:6809/1485092643","cluster_addr":"172.21.15.191:6811/1485092643","heartbeat_back_addr":"172.21.15.191:6815/1485092643","heartbeat_front_addr":"172.21.15.191:6813/1485092643","state":["exists","up"]},{"osd":6,"uuid":"b35dccee-ec16-452e-ac69-3ac2fb7b27fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6818","nonce":1989797856},{"type":"v1","addr":"172.21.15.52:6819","nonce":1989797856}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6820","nonce":1989797856},{"type":"v1","addr":"172.21.15.52:6821","nonce":1989797856}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6824","nonce":1989797856},{"type":"v1","addr":"172.21.15.52:6825","nonce":1989797856}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6822","nonce":1989797856},{"type":"v1","addr":"172.21.15.52:6823","nonce":1989797856}]},"public_addr":"172.21.15.52:6819/1989797856","cluster_addr":"172.21.15.52:6821/1989797856","heartbeat_back_addr":"172.21.15.52:6825/1989797856","heartbeat_front_addr":"172.21.15.52:6823/1989797856","state":["exists","up"]},{"osd":7,"uuid":"71303e3d-14ec-4cbd-b19c-67645a1de3c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6816","nonce":1450870329},{"type":"v1","addr":"172.21.15.106:6817","nonce":1450870329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":1450870329},{"type":"v1","addr":"172.21.15.106:6819","nonce":1450870329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":1450870329},{"type":"v1","addr":"172.21.15.106:6823","nonce":1450870329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6820","nonce":1450870329},{"type":"v1","addr":"172.21.15.106:6821","nonce":1450870329}]},"public_addr":"172.21.15.106:6817/1450870329","cluster_addr":"172.21.15.106:6819/1450870329","heartbeat_back_addr":"172.21.15.106:6823/1450870329","heartbeat_front_addr":"172.21.15.106:6821/1450870329","state":["exists","up"]},{"osd":8,"uuid":"6f5786c4-4402-435a-bdad-adff46fe9473","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":1114119072},{"type":"v1","addr":"172.21.15.191:6817","nonce":1114119072}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6818","nonce":1114119072},{"type":"v1","addr":"172.21.15.191:6819","nonce":1114119072}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":1114119072},{"type":"v1","addr":"172.21.15.191:6823","nonce":1114119072}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":1114119072},{"type":"v1","addr":"172.21.15.191:6821","nonce":1114119072}]},"public_addr":"172.21.15.191:6817/1114119072","cluster_addr":"172.21.15.191:6819/1114119072","heartbeat_back_addr":"172.21.15.191:6823/1114119072","heartbeat_front_addr":"172.21.15.191:6821/1114119072","state":["exists","up"]},{"osd":9,"uuid":"062bbaa5-1227-4684-b859-450f5594a90c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6826","nonce":3485821914},{"type":"v1","addr":"172.21.15.52:6827","nonce":3485821914}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6828","nonce":3485821914},{"type":"v1","addr":"172.21.15.52:6829","nonce":3485821914}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6832","nonce":3485821914},{"type":"v1","addr":"172.21.15.52:6833","nonce":3485821914}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6830","nonce":3485821914},{"type":"v1","addr":"172.21.15.52:6831","nonce":3485821914}]},"public_addr":"172.21.15.52:6827/3485821914","cluster_addr":"172.21.15.52:6829/3485821914","heartbeat_back_addr":"172.21.15.52:6833/3485821914","heartbeat_front_addr":"172.21.15.52:6831/3485821914","state":["exists","up"]},{"osd":10,"uuid":"1a285a5a-f793-48da-a8eb-5326d9528cd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":3815586576},{"type":"v1","addr":"172.21.15.106:6825","nonce":3815586576}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6826","nonce":3815586576},{"type":"v1","addr":"172.21.15.106:6827","nonce":3815586576}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":3815586576},{"type":"v1","addr":"172.21.15.106:6831","nonce":3815586576}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":3815586576},{"type":"v1","addr":"172.21.15.106:6829","nonce":3815586576}]},"public_addr":"172.21.15.106:6825/3815586576","cluster_addr":"172.21.15.106:6827/3815586576","heartbeat_back_addr":"172.21.15.106:6831/3815586576","heartbeat_front_addr":"172.21.15.106:6829/3815586576","state":["exists","up"]},{"osd":11,"uuid":"1f8b9a43-57fa-41f4-9969-3b803a74bf45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":2009632843},{"type":"v1","addr":"172.21.15.191:6825","nonce":2009632843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":2009632843},{"type":"v1","addr":"172.21.15.191:6827","nonce":2009632843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":2009632843},{"type":"v1","addr":"172.21.15.191:6831","nonce":2009632843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":2009632843},{"type":"v1","addr":"172.21.15.191:6829","nonce":2009632843}]},"public_addr":"172.21.15.191:6825/2009632843","cluster_addr":"172.21.15.191:6827/2009632843","heartbeat_back_addr":"172.21.15.191:6831/2009632843","heartbeat_front_addr":"172.21.15.191:6829/2009632843","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T09:23:10.894763+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T09:23:34.420055+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T09:23:58.288324+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T09:24:23.673457+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T09:24:48.208561+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T09:25:13.405473+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T09:25:38.670915+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T09:26:04.451257+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T09:26:30.281380+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T09:26:56.543143+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T09:27:22.412930+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T09:27:48.003866+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.52:0/4082645718":"2024-03-05T09:21:39.401935+0000","172.21.15.52:0/1271949988":"2024-03-05T09:21:39.401935+0000","172.21.15.52:6801/2379361333":"2024-03-05T09:21:12.760938+0000","172.21.15.52:0/497075075":"2024-03-05T09:21:12.760938+0000","172.21.15.52:6800/2379361333":"2024-03-05T09:21:12.760938+0000","172.21.15.52:0/568242577":"2024-03-05T09:21:12.760938+0000","172.21.15.52:6801/2425730636":"2024-03-05T09:21:39.401935+0000","172.21.15.52:6800/2425730636":"2024-03-05T09:21:39.401935+0000","172.21.15.52:0/2743084243":"2024-03-05T09:21:39.401935+0000","172.21.15.52:0/1035878034":"2024-03-05T09:21:12.760938+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-04T09:28:50.205 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:50.207+0000 7f9305f93640 1 -- 172.21.15.52:0/654699875 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f92e80a1d80 msgr2=0x7f92e80a4240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:28:50.205 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:50.207+0000 7f9305f93640 1 --2- 172.21.15.52:0/654699875 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f92e80a1d80 0x7f92e80a4240 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f92f0006010 tx=0x7f92f00093f0 comp rx=0 tx=0).stop 2024-03-04T09:28:50.206 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:50.207+0000 7f9305f93640 1 -- 172.21.15.52:0/654699875 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f93001007c0 msgr2=0x7f930019a000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:28:50.206 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:50.207+0000 7f9305f93640 1 --2- 172.21.15.52:0/654699875 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f93001007c0 0x7f930019a000 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f92f400e9f0 tx=0x7f92f400eeb0 comp rx=0 tx=0).stop 2024-03-04T09:28:50.206 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:50.207+0000 7f9305f93640 1 -- 172.21.15.52:0/654699875 shutdown_connections 2024-03-04T09:28:50.206 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:50.207+0000 7f9305f93640 1 --2- 172.21.15.52:0/654699875 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f92e80a1d80 0x7f92e80a4240 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:50.206 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:50.207+0000 7f9305f93640 1 --2- 172.21.15.52:0/654699875 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9300101160 0x7f93001a1070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:50.206 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:50.207+0000 7f9305f93640 1 --2- 172.21.15.52:0/654699875 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f93001007c0 0x7f930019a000 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:50.206 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:50.207+0000 7f9305f93640 1 --2- 172.21.15.52:0/654699875 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f93000ffe60 0x7f9300199ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:50.207 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:50.207+0000 7f9305f93640 1 -- 172.21.15.52:0/654699875 >> 172.21.15.52:0/654699875 conn(0x7f93000fba60 msgr2=0x7f93000fd150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:28:50.207 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:50.207+0000 7f9305f93640 1 -- 172.21.15.52:0/654699875 shutdown_connections 2024-03-04T09:28:50.207 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:50.207+0000 7f9305f93640 1 -- 172.21.15.52:0/654699875 wait complete. 2024-03-04T09:28:50.215 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-04T09:24:01.742461+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-04T09:28:35.835600+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-04T09:28:36.453099+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-04T09:28:40.168831+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '89', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-04T09:28:50.216 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-04T09:28:50.323 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:50.324+0000 7fd3ebca4640 1 -- 172.21.15.52:0/1893775515 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd3e4102070 msgr2=0x7fd3e4104460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:28:50.323 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:50.324+0000 7fd3ebca4640 1 --2- 172.21.15.52:0/1893775515 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd3e4102070 0x7fd3e4104460 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fd3dc009900 tx=0x7fd3dc02bc30 comp rx=0 tx=0).stop 2024-03-04T09:28:50.323 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:50.324+0000 7fd3ebca4640 1 -- 172.21.15.52:0/1893775515 shutdown_connections 2024-03-04T09:28:50.324 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:50.324+0000 7fd3ebca4640 1 --2- 172.21.15.52:0/1893775515 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd3e41072d0 0x7fd3e4109720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:50.324 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:50.324+0000 7fd3ebca4640 1 --2- 172.21.15.52:0/1893775515 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd3e41049a0 0x7fd3e4106d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:50.324 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:50.324+0000 7fd3ebca4640 1 --2- 172.21.15.52:0/1893775515 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd3e4102070 0x7fd3e4104460 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:50.324 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:50.324+0000 7fd3ebca4640 1 -- 172.21.15.52:0/1893775515 >> 172.21.15.52:0/1893775515 conn(0x7fd3e40fba80 msgr2=0x7fd3e40fdea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:28:50.324 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:50.324+0000 7fd3ebca4640 1 -- 172.21.15.52:0/1893775515 shutdown_connections 2024-03-04T09:28:50.324 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:50.324+0000 7fd3ebca4640 1 -- 172.21.15.52:0/1893775515 wait complete. 2024-03-04T09:28:50.324 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:50.325+0000 7fd3ebca4640 1 Processor -- start 2024-03-04T09:28:50.324 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:50.325+0000 7fd3ebca4640 1 -- start start 2024-03-04T09:28:50.324 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:50.325+0000 7fd3ebca4640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd3e4102070 0x7fd3e4197960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:28:50.324 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:50.325+0000 7fd3ebca4640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd3e41049a0 0x7fd3e4197ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:28:50.324 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:50.325+0000 7fd3ebca4640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd3e41072d0 0x7fd3e419ef10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:28:50.325 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:50.325+0000 7fd3ebca4640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3e410d280 con 0x7fd3e41049a0 2024-03-04T09:28:50.325 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:50.325+0000 7fd3ebca4640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3e410d100 con 0x7fd3e41072d0 2024-03-04T09:28:50.325 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:50.325+0000 7fd3ebca4640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3e410d400 con 0x7fd3e4102070 2024-03-04T09:28:50.325 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:50.325+0000 7fd3e923f640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd3e41049a0 0x7fd3e4197ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:28:50.325 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:50.325+0000 7fd3e923f640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd3e41049a0 0x7fd3e4197ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53460/0 (socket says 172.21.15.52:53460) 2024-03-04T09:28:50.325 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:50.325+0000 7fd3e923f640 1 -- 172.21.15.52:0/1857533500 learned_addr learned my addr 172.21.15.52:0/1857533500 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-04T09:28:50.325 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:50.325+0000 7fd3ea241640 1 --2- 172.21.15.52:0/1857533500 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd3e41072d0 0x7fd3e419ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:28:50.325 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:50.325+0000 7fd3e9a40640 1 --2- 172.21.15.52:0/1857533500 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd3e4102070 0x7fd3e4197960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:28:50.325 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:50.326+0000 7fd3e923f640 1 -- 172.21.15.52:0/1857533500 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd3e4102070 msgr2=0x7fd3e4197960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:28:50.325 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:50.326+0000 7fd3e923f640 1 --2- 172.21.15.52:0/1857533500 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd3e4102070 0x7fd3e4197960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:50.325 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:50.326+0000 7fd3e923f640 1 -- 172.21.15.52:0/1857533500 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd3e41072d0 msgr2=0x7fd3e419ef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:28:50.326 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:50.326+0000 7fd3e923f640 1 --2- 172.21.15.52:0/1857533500 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd3e41072d0 0x7fd3e419ef10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:50.326 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:50.326+0000 7fd3e923f640 1 -- 172.21.15.52:0/1857533500 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3e419f580 con 0x7fd3e41049a0 2024-03-04T09:28:50.326 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:50.326+0000 7fd3e9a40640 1 --2- 172.21.15.52:0/1857533500 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd3e4102070 0x7fd3e4197960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-04T09:28:50.326 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:50.326+0000 7fd3ea241640 1 --2- 172.21.15.52:0/1857533500 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd3e41072d0 0x7fd3e419ef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-04T09:28:50.326 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:50.326+0000 7fd3e923f640 1 --2- 172.21.15.52:0/1857533500 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd3e41049a0 0x7fd3e4197ea0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fd3d400e9c0 tx=0x7fd3d400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:28:50.326 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:50.326+0000 7fd3daffd640 1 -- 172.21.15.52:0/1857533500 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3d400cd80 con 0x7fd3e41049a0 2024-03-04T09:28:50.326 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:50.326+0000 7fd3daffd640 1 -- 172.21.15.52:0/1857533500 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd3d4005500 con 0x7fd3e41049a0 2024-03-04T09:28:50.326 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:50.326+0000 7fd3ebca4640 1 -- 172.21.15.52:0/1857533500 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd3e419f870 con 0x7fd3e41049a0 2024-03-04T09:28:50.326 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:50.326+0000 7fd3daffd640 1 -- 172.21.15.52:0/1857533500 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3d4010690 con 0x7fd3e41049a0 2024-03-04T09:28:50.327 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:50.326+0000 7fd3ebca4640 1 -- 172.21.15.52:0/1857533500 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3e419fdc0 con 0x7fd3e41049a0 2024-03-04T09:28:50.327 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:50.327+0000 7fd3ebca4640 1 -- 172.21.15.52:0/1857533500 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd3e40fd570 con 0x7fd3e41049a0 2024-03-04T09:28:50.328 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:50.328+0000 7fd3daffd640 1 -- 172.21.15.52:0/1857533500 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 16) v1 ==== 136756+0+0 (secure 0 0 0) 0x7fd3d40046c0 con 0x7fd3e41049a0 2024-03-04T09:28:50.328 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:50.329+0000 7fd3daffd640 1 --2- 172.21.15.52:0/1857533500 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7fd3d00a1d80 0x7fd3d00a4240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:28:50.328 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:50.329+0000 7fd3daffd640 1 -- 172.21.15.52:0/1857533500 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8802+0+0 (secure 0 0 0) 0x7fd3d40ce470 con 0x7fd3e41049a0 2024-03-04T09:28:50.328 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:50.329+0000 7fd3e9a40640 1 --2- 172.21.15.52:0/1857533500 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7fd3d00a1d80 0x7fd3d00a4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:28:50.328 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:50.329+0000 7fd3e9a40640 1 --2- 172.21.15.52:0/1857533500 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7fd3d00a1d80 0x7fd3d00a4240 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fd3dc0096c0 tx=0x7fd3dc000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:28:50.330 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:50.333+0000 7fd3daffd640 1 -- 172.21.15.52:0/1857533500 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fd3d409b600 con 0x7fd3e41049a0 2024-03-04T09:28:50.512 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:50.515+0000 7fd3ebca4640 1 -- 172.21.15.52:0/1857533500 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fd3e40621d0 con 0x7fd3e41049a0 2024-03-04T09:28:50.512 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:50.515+0000 7fd3daffd640 1 -- 172.21.15.52:0/1857533500 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v89) v1 ==== 93+0+10 (secure 0 0 0) 0x7fd3d409af00 con 0x7fd3e41049a0 2024-03-04T09:28:50.513 INFO:teuthology.orchestra.run.smithi052.stdout:pg_num: 1 2024-03-04T09:28:50.514 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:50.517+0000 7fd3ebca4640 1 -- 172.21.15.52:0/1857533500 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7fd3d00a1d80 msgr2=0x7fd3d00a4240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:28:50.514 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:50.517+0000 7fd3ebca4640 1 --2- 172.21.15.52:0/1857533500 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7fd3d00a1d80 0x7fd3d00a4240 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fd3dc0096c0 tx=0x7fd3dc000bc0 comp rx=0 tx=0).stop 2024-03-04T09:28:50.514 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:50.517+0000 7fd3ebca4640 1 -- 172.21.15.52:0/1857533500 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd3e41049a0 msgr2=0x7fd3e4197ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:28:50.515 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:50.517+0000 7fd3ebca4640 1 --2- 172.21.15.52:0/1857533500 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd3e41049a0 0x7fd3e4197ea0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fd3d400e9c0 tx=0x7fd3d400ee80 comp rx=0 tx=0).stop 2024-03-04T09:28:50.515 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:50.517+0000 7fd3ebca4640 1 -- 172.21.15.52:0/1857533500 shutdown_connections 2024-03-04T09:28:50.515 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:50.517+0000 7fd3ebca4640 1 --2- 172.21.15.52:0/1857533500 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7fd3d00a1d80 0x7fd3d00a4240 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:50.515 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:50.517+0000 7fd3ebca4640 1 --2- 172.21.15.52:0/1857533500 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd3e41072d0 0x7fd3e419ef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:50.515 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:50.517+0000 7fd3ebca4640 1 --2- 172.21.15.52:0/1857533500 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd3e41049a0 0x7fd3e4197ea0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:50.515 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:50.517+0000 7fd3ebca4640 1 --2- 172.21.15.52:0/1857533500 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd3e4102070 0x7fd3e4197960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:50.515 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:50.517+0000 7fd3ebca4640 1 -- 172.21.15.52:0/1857533500 >> 172.21.15.52:0/1857533500 conn(0x7fd3e40fba80 msgr2=0x7fd3e41029f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:28:50.515 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:50.517+0000 7fd3ebca4640 1 -- 172.21.15.52:0/1857533500 shutdown_connections 2024-03-04T09:28:50.515 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:50.517+0000 7fd3ebca4640 1 -- 172.21.15.52:0/1857533500 wait complete. 2024-03-04T09:28:50.525 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-04T09:28:50.633 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:50.633+0000 7fa1d51a0640 1 -- 172.21.15.52:0/2545766981 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa1d0101f60 msgr2=0x7fa1d01087f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:28:50.633 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:50.633+0000 7fa1d51a0640 1 --2- 172.21.15.52:0/2545766981 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa1d0101f60 0x7fa1d01087f0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fa1c8009ca0 tx=0x7fa1c802c0f0 comp rx=0 tx=0).stop 2024-03-04T09:28:50.633 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:50.633+0000 7fa1d51a0640 1 -- 172.21.15.52:0/2545766981 shutdown_connections 2024-03-04T09:28:50.633 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:50.634+0000 7fa1d51a0640 1 --2- 172.21.15.52:0/2545766981 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa1d0101f60 0x7fa1d01087f0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:50.633 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:50.634+0000 7fa1d51a0640 1 --2- 172.21.15.52:0/2545766981 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa1d01015a0 0x7fa1d0101a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:50.633 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:50.634+0000 7fa1d51a0640 1 --2- 172.21.15.52:0/2545766981 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa1d01003b0 0x7fa1d01007d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:50.633 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:50.634+0000 7fa1d51a0640 1 -- 172.21.15.52:0/2545766981 >> 172.21.15.52:0/2545766981 conn(0x7fa1d00fb940 msgr2=0x7fa1d00fdd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:28:50.634 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:50.634+0000 7fa1d51a0640 1 -- 172.21.15.52:0/2545766981 shutdown_connections 2024-03-04T09:28:50.634 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:50.634+0000 7fa1d51a0640 1 -- 172.21.15.52:0/2545766981 wait complete. 2024-03-04T09:28:50.634 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:50.634+0000 7fa1d51a0640 1 Processor -- start 2024-03-04T09:28:50.634 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:50.634+0000 7fa1d51a0640 1 -- start start 2024-03-04T09:28:50.634 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:50.634+0000 7fa1d51a0640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa1d01003b0 0x7fa1d00692a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:28:50.634 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:50.634+0000 7fa1d51a0640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa1d01015a0 0x7fa1d019c430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:28:50.634 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:50.634+0000 7fa1d51a0640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa1d0101f60 0x7fa1d019ea50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:28:50.634 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:50.634+0000 7fa1d51a0640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1d010c040 con 0x7fa1d01015a0 2024-03-04T09:28:50.634 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:50.634+0000 7fa1d51a0640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1d010bec0 con 0x7fa1d01003b0 2024-03-04T09:28:50.634 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:50.634+0000 7fa1d51a0640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1d010c1c0 con 0x7fa1d0101f60 2024-03-04T09:28:50.634 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:50.635+0000 7fa1cf7fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa1d01015a0 0x7fa1d019c430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:28:50.634 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:50.635+0000 7fa1cf7fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa1d01015a0 0x7fa1d019c430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53474/0 (socket says 172.21.15.52:53474) 2024-03-04T09:28:50.634 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:50.635+0000 7fa1cf7fe640 1 -- 172.21.15.52:0/1814018454 learned_addr learned my addr 172.21.15.52:0/1814018454 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-04T09:28:50.634 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:50.635+0000 7fa1cffff640 1 --2- 172.21.15.52:0/1814018454 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa1d01003b0 0x7fa1d00692a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:28:50.635 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:50.635+0000 7fa1cf7fe640 1 -- 172.21.15.52:0/1814018454 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa1d0101f60 msgr2=0x7fa1d019ea50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:28:50.635 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:50.635+0000 7fa1cf7fe640 1 --2- 172.21.15.52:0/1814018454 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa1d0101f60 0x7fa1d019ea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:50.635 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:50.635+0000 7fa1cf7fe640 1 -- 172.21.15.52:0/1814018454 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa1d01003b0 msgr2=0x7fa1d00692a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:28:50.635 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:50.635+0000 7fa1cf7fe640 1 --2- 172.21.15.52:0/1814018454 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa1d01003b0 0x7fa1d00692a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:50.635 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:50.635+0000 7fa1cf7fe640 1 -- 172.21.15.52:0/1814018454 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1d019f360 con 0x7fa1d01015a0 2024-03-04T09:28:50.635 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:50.635+0000 7fa1cf7fe640 1 --2- 172.21.15.52:0/1814018454 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa1d01015a0 0x7fa1d019c430 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fa1c400c970 tx=0x7fa1c400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:28:50.635 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:50.635+0000 7fa1cd7fa640 1 -- 172.21.15.52:0/1814018454 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1c4007bb0 con 0x7fa1d01015a0 2024-03-04T09:28:50.635 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:50.635+0000 7fa1cd7fa640 1 -- 172.21.15.52:0/1814018454 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa1c4007d50 con 0x7fa1d01015a0 2024-03-04T09:28:50.635 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:50.635+0000 7fa1d51a0640 1 -- 172.21.15.52:0/1814018454 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1d019f5f0 con 0x7fa1d01015a0 2024-03-04T09:28:50.635 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:50.635+0000 7fa1cd7fa640 1 -- 172.21.15.52:0/1814018454 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1c400f450 con 0x7fa1d01015a0 2024-03-04T09:28:50.635 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:50.635+0000 7fa1d51a0640 1 -- 172.21.15.52:0/1814018454 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1d019fb40 con 0x7fa1d01015a0 2024-03-04T09:28:50.635 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:50.636+0000 7fa1d51a0640 1 -- 172.21.15.52:0/1814018454 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa1d01076d0 con 0x7fa1d01015a0 2024-03-04T09:28:50.636 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:50.637+0000 7fa1cd7fa640 1 -- 172.21.15.52:0/1814018454 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 16) v1 ==== 136756+0+0 (secure 0 0 0) 0x7fa1c4004fe0 con 0x7fa1d01015a0 2024-03-04T09:28:50.637 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:50.638+0000 7fa1cd7fa640 1 --2- 172.21.15.52:0/1814018454 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7fa1b80a1e00 0x7fa1b80a42c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:28:50.637 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:50.638+0000 7fa1cffff640 1 --2- 172.21.15.52:0/1814018454 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7fa1b80a1e00 0x7fa1b80a42c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:28:50.637 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:50.638+0000 7fa1cd7fa640 1 -- 172.21.15.52:0/1814018454 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8802+0+0 (secure 0 0 0) 0x7fa1c40ce150 con 0x7fa1d01015a0 2024-03-04T09:28:50.637 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:50.638+0000 7fa1cffff640 1 --2- 172.21.15.52:0/1814018454 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7fa1b80a1e00 0x7fa1b80a42c0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fa1c00095f0 tx=0x7fa1c0005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:28:50.638 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:50.641+0000 7fa1cd7fa640 1 -- 172.21.15.52:0/1814018454 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fa1c4014030 con 0x7fa1d01015a0 2024-03-04T09:28:50.820 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:50.823+0000 7fa1d51a0640 1 -- 172.21.15.52:0/1814018454 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fa1d002d210 con 0x7fa1d01015a0 2024-03-04T09:28:50.820 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:50.823+0000 7fa1cd7fa640 1 -- 172.21.15.52:0/1814018454 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v89) v1 ==== 104+0+11 (secure 0 0 0) 0x7fa1c409b3c0 con 0x7fa1d01015a0 2024-03-04T09:28:50.820 INFO:teuthology.orchestra.run.smithi052.stdout:pg_num: 32 2024-03-04T09:28:50.822 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:50.825+0000 7fa1d51a0640 1 -- 172.21.15.52:0/1814018454 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7fa1b80a1e00 msgr2=0x7fa1b80a42c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:28:50.822 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:50.825+0000 7fa1d51a0640 1 --2- 172.21.15.52:0/1814018454 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7fa1b80a1e00 0x7fa1b80a42c0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fa1c00095f0 tx=0x7fa1c0005f90 comp rx=0 tx=0).stop 2024-03-04T09:28:50.822 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:50.825+0000 7fa1d51a0640 1 -- 172.21.15.52:0/1814018454 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa1d01015a0 msgr2=0x7fa1d019c430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:28:50.822 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:50.825+0000 7fa1d51a0640 1 --2- 172.21.15.52:0/1814018454 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa1d01015a0 0x7fa1d019c430 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fa1c400c970 tx=0x7fa1c400ce30 comp rx=0 tx=0).stop 2024-03-04T09:28:50.822 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:50.825+0000 7fa1d51a0640 1 -- 172.21.15.52:0/1814018454 shutdown_connections 2024-03-04T09:28:50.822 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:50.825+0000 7fa1d51a0640 1 --2- 172.21.15.52:0/1814018454 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7fa1b80a1e00 0x7fa1b80a42c0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:50.822 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:50.825+0000 7fa1d51a0640 1 --2- 172.21.15.52:0/1814018454 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa1d0101f60 0x7fa1d019ea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:50.822 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:50.825+0000 7fa1d51a0640 1 --2- 172.21.15.52:0/1814018454 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa1d01015a0 0x7fa1d019c430 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:50.823 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:50.825+0000 7fa1d51a0640 1 --2- 172.21.15.52:0/1814018454 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa1d01003b0 0x7fa1d00692a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:50.823 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:50.825+0000 7fa1d51a0640 1 -- 172.21.15.52:0/1814018454 >> 172.21.15.52:0/1814018454 conn(0x7fa1d00fb940 msgr2=0x7fa1d01048a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:28:50.823 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:50.825+0000 7fa1d51a0640 1 -- 172.21.15.52:0/1814018454 shutdown_connections 2024-03-04T09:28:50.823 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:50.825+0000 7fa1d51a0640 1 -- 172.21.15.52:0/1814018454 wait complete. 2024-03-04T09:28:50.833 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-04T09:28:50.942 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:50.945+0000 7f431caed640 1 -- 172.21.15.52:0/1986973724 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f43181004d0 msgr2=0x7f43181008f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:28:50.942 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:50.945+0000 7f431caed640 1 --2- 172.21.15.52:0/1986973724 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f43181004d0 0x7f43181008f0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f4308009900 tx=0x7f430802bc30 comp rx=0 tx=0).stop 2024-03-04T09:28:50.942 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:50.945+0000 7f431caed640 1 -- 172.21.15.52:0/1986973724 shutdown_connections 2024-03-04T09:28:50.942 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:50.945+0000 7f431caed640 1 --2- 172.21.15.52:0/1986973724 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4318102080 0x7f4318108910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:50.942 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:50.945+0000 7f431caed640 1 --2- 172.21.15.52:0/1986973724 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f43181016c0 0x7f4318101b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:50.943 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:50.945+0000 7f431caed640 1 --2- 172.21.15.52:0/1986973724 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f43181004d0 0x7f43181008f0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:50.943 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:50.945+0000 7f431caed640 1 -- 172.21.15.52:0/1986973724 >> 172.21.15.52:0/1986973724 conn(0x7f43180fba60 msgr2=0x7f43180fde80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:28:50.943 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:50.945+0000 7f431caed640 1 -- 172.21.15.52:0/1986973724 shutdown_connections 2024-03-04T09:28:50.943 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:50.945+0000 7f431caed640 1 -- 172.21.15.52:0/1986973724 wait complete. 2024-03-04T09:28:50.945 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:50.946+0000 7f431caed640 1 Processor -- start 2024-03-04T09:28:50.945 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:50.946+0000 7f431caed640 1 -- start start 2024-03-04T09:28:50.945 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:50.946+0000 7f431caed640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f43181004d0 0x7f4318199bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:28:50.945 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:50.946+0000 7f431caed640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f43181016c0 0x7f431819a100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:28:50.945 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:50.946+0000 7f431caed640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4318102080 0x7f43181a1170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:28:50.945 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:50.946+0000 7f431caed640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f431810c1b0 con 0x7f43181016c0 2024-03-04T09:28:50.945 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:50.946+0000 7f431caed640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f431810c030 con 0x7f43181004d0 2024-03-04T09:28:50.945 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:50.946+0000 7f431caed640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f431810c330 con 0x7f4318102080 2024-03-04T09:28:50.946 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:50.946+0000 7f430ffff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f43181016c0 0x7f431819a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:28:50.946 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:50.946+0000 7f430ffff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f43181016c0 0x7f431819a100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53482/0 (socket says 172.21.15.52:53482) 2024-03-04T09:28:50.946 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:50.946+0000 7f431659c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f43181004d0 0x7f4318199bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:28:50.946 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:50.946+0000 7f430ffff640 1 -- 172.21.15.52:0/2920646061 learned_addr learned my addr 172.21.15.52:0/2920646061 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-04T09:28:50.946 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:50.946+0000 7f4316d9d640 1 --2- 172.21.15.52:0/2920646061 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4318102080 0x7f43181a1170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:28:50.946 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:50.946+0000 7f430ffff640 1 -- 172.21.15.52:0/2920646061 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4318102080 msgr2=0x7f43181a1170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:28:50.946 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:50.946+0000 7f430ffff640 1 --2- 172.21.15.52:0/2920646061 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4318102080 0x7f43181a1170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:50.946 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:50.946+0000 7f430ffff640 1 -- 172.21.15.52:0/2920646061 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f43181004d0 msgr2=0x7f4318199bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:28:50.946 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:50.946+0000 7f430ffff640 1 --2- 172.21.15.52:0/2920646061 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f43181004d0 0x7f4318199bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:50.946 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:50.946+0000 7f430ffff640 1 -- 172.21.15.52:0/2920646061 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43181a17e0 con 0x7f43181016c0 2024-03-04T09:28:50.946 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:50.946+0000 7f431659c640 1 --2- 172.21.15.52:0/2920646061 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f43181004d0 0x7f4318199bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-04T09:28:50.946 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:50.947+0000 7f430ffff640 1 --2- 172.21.15.52:0/2920646061 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f43181016c0 0x7f431819a100 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f430000d770 tx=0x7f430000db70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:28:50.946 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:50.947+0000 7f430f7fe640 1 -- 172.21.15.52:0/2920646061 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4300017070 con 0x7f43181016c0 2024-03-04T09:28:50.946 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:50.947+0000 7f430f7fe640 1 -- 172.21.15.52:0/2920646061 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4300009820 con 0x7f43181016c0 2024-03-04T09:28:50.947 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:50.947+0000 7f431caed640 1 -- 172.21.15.52:0/2920646061 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43181a1ad0 con 0x7f43181016c0 2024-03-04T09:28:50.947 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:50.947+0000 7f430f7fe640 1 -- 172.21.15.52:0/2920646061 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4300004cd0 con 0x7f43181016c0 2024-03-04T09:28:50.947 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:50.947+0000 7f431caed640 1 -- 172.21.15.52:0/2920646061 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43181a1fd0 con 0x7f43181016c0 2024-03-04T09:28:50.947 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:50.948+0000 7f431caed640 1 -- 172.21.15.52:0/2920646061 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f43181a2200 con 0x7f43181016c0 2024-03-04T09:28:50.947 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:50.949+0000 7f430f7fe640 1 -- 172.21.15.52:0/2920646061 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 16) v1 ==== 136756+0+0 (secure 0 0 0) 0x7f43000099d0 con 0x7f43181016c0 2024-03-04T09:28:50.947 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:50.949+0000 7f430f7fe640 1 --2- 172.21.15.52:0/2920646061 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f43040a1d80 0x7f43040a4240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:28:50.948 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:50.950+0000 7f431659c640 1 --2- 172.21.15.52:0/2920646061 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f43040a1d80 0x7f43040a4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:28:50.948 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:50.950+0000 7f430f7fe640 1 -- 172.21.15.52:0/2920646061 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8802+0+0 (secure 0 0 0) 0x7f43000cdd90 con 0x7f43181016c0 2024-03-04T09:28:50.948 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:50.950+0000 7f431659c640 1 --2- 172.21.15.52:0/2920646061 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f43040a1d80 0x7f43040a4240 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f430802c170 tx=0x7f430802df90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:28:50.950 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:50.953+0000 7f430f7fe640 1 -- 172.21.15.52:0/2920646061 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f430009af80 con 0x7f43181016c0 2024-03-04T09:28:51.085 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:28:50 smithi052 ceph-mon[32473]: pgmap v294: 129 pgs: 129 active+clean; 579 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 127 B/s wr, 2 op/s 2024-03-04T09:28:51.085 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:28:50 smithi052 ceph-mon[32473]: from='client.? 172.21.15.52:0/654699875' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-04T09:28:51.085 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:28:50 smithi052 ceph-mon[32473]: from='client.? 172.21.15.52:0/1857533500' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-04T09:28:51.085 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:28:50 smithi052 ceph-mon[32473]: from='client.? 172.21.15.52:0/1814018454' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-04T09:28:51.132 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:51.135+0000 7f431caed640 1 -- 172.21.15.52:0/2920646061 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f43181a23d0 con 0x7f43181016c0 2024-03-04T09:28:51.132 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:51.135+0000 7f430f7fe640 1 -- 172.21.15.52:0/2920646061 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v89) v1 ==== 100+0+11 (secure 0 0 0) 0x7f430009a880 con 0x7f43181016c0 2024-03-04T09:28:51.132 INFO:teuthology.orchestra.run.smithi052.stdout:pg_num: 64 2024-03-04T09:28:51.134 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:51.137+0000 7f431caed640 1 -- 172.21.15.52:0/2920646061 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f43040a1d80 msgr2=0x7f43040a4240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:28:51.134 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:51.137+0000 7f431caed640 1 --2- 172.21.15.52:0/2920646061 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f43040a1d80 0x7f43040a4240 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f430802c170 tx=0x7f430802df90 comp rx=0 tx=0).stop 2024-03-04T09:28:51.134 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:51.137+0000 7f431caed640 1 -- 172.21.15.52:0/2920646061 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f43181016c0 msgr2=0x7f431819a100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:28:51.134 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:51.137+0000 7f431caed640 1 --2- 172.21.15.52:0/2920646061 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f43181016c0 0x7f431819a100 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f430000d770 tx=0x7f430000db70 comp rx=0 tx=0).stop 2024-03-04T09:28:51.134 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:51.137+0000 7f431caed640 1 -- 172.21.15.52:0/2920646061 shutdown_connections 2024-03-04T09:28:51.134 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:51.137+0000 7f431caed640 1 --2- 172.21.15.52:0/2920646061 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f43040a1d80 0x7f43040a4240 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:51.135 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:51.137+0000 7f431caed640 1 --2- 172.21.15.52:0/2920646061 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4318102080 0x7f43181a1170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:51.135 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:51.137+0000 7f431caed640 1 --2- 172.21.15.52:0/2920646061 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f43181016c0 0x7f431819a100 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:51.135 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:51.137+0000 7f431caed640 1 --2- 172.21.15.52:0/2920646061 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f43181004d0 0x7f4318199bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:51.135 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:51.137+0000 7f431caed640 1 -- 172.21.15.52:0/2920646061 >> 172.21.15.52:0/2920646061 conn(0x7f43180fba60 msgr2=0x7f43180fd570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:28:51.135 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:51.137+0000 7f431caed640 1 -- 172.21.15.52:0/2920646061 shutdown_connections 2024-03-04T09:28:51.135 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:51.137+0000 7f431caed640 1 -- 172.21.15.52:0/2920646061 wait complete. 2024-03-04T09:28:51.145 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-04T09:28:51.253 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:51.253+0000 7f96ec556640 1 -- 172.21.15.52:0/361681941 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f96e41020b0 msgr2=0x7f96e41044a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:28:51.253 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:51.253+0000 7f96ec556640 1 --2- 172.21.15.52:0/361681941 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f96e41020b0 0x7f96e41044a0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f96e0009900 tx=0x7f96e002bc30 comp rx=0 tx=0).stop 2024-03-04T09:28:51.253 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:51.254+0000 7f96ec556640 1 -- 172.21.15.52:0/361681941 shutdown_connections 2024-03-04T09:28:51.253 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:51.254+0000 7f96ec556640 1 --2- 172.21.15.52:0/361681941 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f96e4107310 0x7f96e4109700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:51.253 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:51.254+0000 7f96ec556640 1 --2- 172.21.15.52:0/361681941 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f96e41049e0 0x7f96e4106dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:51.253 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:51.254+0000 7f96ec556640 1 --2- 172.21.15.52:0/361681941 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f96e41020b0 0x7f96e41044a0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:51.253 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:51.254+0000 7f96ec556640 1 -- 172.21.15.52:0/361681941 >> 172.21.15.52:0/361681941 conn(0x7f96e40fba60 msgr2=0x7f96e40fde80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:28:51.253 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:51.254+0000 7f96ec556640 1 -- 172.21.15.52:0/361681941 shutdown_connections 2024-03-04T09:28:51.253 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:51.254+0000 7f96ec556640 1 -- 172.21.15.52:0/361681941 wait complete. 2024-03-04T09:28:51.253 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:51.254+0000 7f96ec556640 1 Processor -- start 2024-03-04T09:28:51.254 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:51.254+0000 7f96ec556640 1 -- start start 2024-03-04T09:28:51.254 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:51.254+0000 7f96ec556640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f96e41020b0 0x7f96e4101a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:28:51.254 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:51.254+0000 7f96ec556640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f96e41049e0 0x7f96e4100060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:28:51.254 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:51.254+0000 7f96ec556640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f96e4107310 0x7f96e41005a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:28:51.254 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:51.254+0000 7f96ec556640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96e410d260 con 0x7f96e41020b0 2024-03-04T09:28:51.254 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:51.254+0000 7f96ec556640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96e410d0e0 con 0x7f96e41049e0 2024-03-04T09:28:51.254 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:51.254+0000 7f96ec556640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96e410d3e0 con 0x7f96e4107310 2024-03-04T09:28:51.254 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:51.254+0000 7f96ea2f2640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f96e41020b0 0x7f96e4101a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:28:51.254 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:51.254+0000 7f96ea2f2640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f96e41020b0 0x7f96e4101a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53510/0 (socket says 172.21.15.52:53510) 2024-03-04T09:28:51.254 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:51.254+0000 7f96ea2f2640 1 -- 172.21.15.52:0/3863474526 learned_addr learned my addr 172.21.15.52:0/3863474526 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-04T09:28:51.254 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:51.255+0000 7f96e9af1640 1 --2- 172.21.15.52:0/3863474526 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f96e41049e0 0x7f96e4100060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:28:51.254 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:51.255+0000 7f96eaaf3640 1 --2- 172.21.15.52:0/3863474526 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f96e4107310 0x7f96e41005a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:28:51.254 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:51.255+0000 7f96ea2f2640 1 -- 172.21.15.52:0/3863474526 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f96e4107310 msgr2=0x7f96e41005a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:28:51.255 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:51.255+0000 7f96ea2f2640 1 --2- 172.21.15.52:0/3863474526 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f96e4107310 0x7f96e41005a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:51.255 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:51.255+0000 7f96ea2f2640 1 -- 172.21.15.52:0/3863474526 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f96e41049e0 msgr2=0x7f96e4100060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:28:51.255 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:51.255+0000 7f96ea2f2640 1 --2- 172.21.15.52:0/3863474526 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f96e41049e0 0x7f96e4100060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:51.255 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:51.255+0000 7f96ea2f2640 1 -- 172.21.15.52:0/3863474526 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96e4100e30 con 0x7f96e41020b0 2024-03-04T09:28:51.255 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:51.255+0000 7f96eaaf3640 1 --2- 172.21.15.52:0/3863474526 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f96e4107310 0x7f96e41005a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T09:28:51.255 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:51.255+0000 7f96ea2f2640 1 --2- 172.21.15.52:0/3863474526 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f96e41020b0 0x7f96e4101a20 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f96e0009230 tx=0x7f96e0005d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:28:51.255 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:51.255+0000 7f96db7fe640 1 -- 172.21.15.52:0/3863474526 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96e0041070 con 0x7f96e41020b0 2024-03-04T09:28:51.255 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:51.255+0000 7f96ec556640 1 -- 172.21.15.52:0/3863474526 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96e41a1a10 con 0x7f96e41020b0 2024-03-04T09:28:51.255 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:51.255+0000 7f96db7fe640 1 -- 172.21.15.52:0/3863474526 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f96e00044d0 con 0x7f96e41020b0 2024-03-04T09:28:51.255 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:51.255+0000 7f96db7fe640 1 -- 172.21.15.52:0/3863474526 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96e0034440 con 0x7f96e41020b0 2024-03-04T09:28:51.255 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:51.255+0000 7f96ec556640 1 -- 172.21.15.52:0/3863474526 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96e41a1f60 con 0x7f96e41020b0 2024-03-04T09:28:51.255 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:51.256+0000 7f96ec556640 1 -- 172.21.15.52:0/3863474526 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f96e40fd550 con 0x7f96e41020b0 2024-03-04T09:28:51.256 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:51.258+0000 7f96db7fe640 1 -- 172.21.15.52:0/3863474526 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 16) v1 ==== 136756+0+0 (secure 0 0 0) 0x7f96e0044070 con 0x7f96e41020b0 2024-03-04T09:28:51.256 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:51.258+0000 7f96db7fe640 1 --2- 172.21.15.52:0/3863474526 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f96d00a1d80 0x7f96d00a4240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:28:51.256 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:51.259+0000 7f96e9af1640 1 --2- 172.21.15.52:0/3863474526 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f96d00a1d80 0x7f96d00a4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:28:51.256 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:51.259+0000 7f96db7fe640 1 -- 172.21.15.52:0/3863474526 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8802+0+0 (secure 0 0 0) 0x7f96e00effe0 con 0x7f96e41020b0 2024-03-04T09:28:51.257 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:51.259+0000 7f96e9af1640 1 --2- 172.21.15.52:0/3863474526 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f96d00a1d80 0x7f96d00a4240 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f96e4101790 tx=0x7f96d4005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:28:51.258 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:51.262+0000 7f96db7fe640 1 -- 172.21.15.52:0/3863474526 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f96e00bd1d0 con 0x7f96e41020b0 2024-03-04T09:28:51.336 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:28:50 smithi191 ceph-mon[36816]: pgmap v294: 129 pgs: 129 active+clean; 579 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 127 B/s wr, 2 op/s 2024-03-04T09:28:51.336 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:28:50 smithi191 ceph-mon[36816]: from='client.? 172.21.15.52:0/654699875' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-04T09:28:51.336 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:28:50 smithi191 ceph-mon[36816]: from='client.? 172.21.15.52:0/1857533500' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-04T09:28:51.336 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:28:50 smithi191 ceph-mon[36816]: from='client.? 172.21.15.52:0/1814018454' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-04T09:28:51.381 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:28:50 smithi106 ceph-mon[36484]: pgmap v294: 129 pgs: 129 active+clean; 579 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 127 B/s wr, 2 op/s 2024-03-04T09:28:51.381 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:28:50 smithi106 ceph-mon[36484]: from='client.? 172.21.15.52:0/654699875' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-04T09:28:51.381 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:28:50 smithi106 ceph-mon[36484]: from='client.? 172.21.15.52:0/1857533500' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-04T09:28:51.381 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:28:50 smithi106 ceph-mon[36484]: from='client.? 172.21.15.52:0/1814018454' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-04T09:28:51.441 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:51.444+0000 7f96ec556640 1 -- 172.21.15.52:0/3863474526 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f96e40621d0 con 0x7f96e41020b0 2024-03-04T09:28:51.441 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:51.444+0000 7f96db7fe640 1 -- 172.21.15.52:0/3863474526 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v89) v1 ==== 103+0+11 (secure 0 0 0) 0x7f96e00bcad0 con 0x7f96e41020b0 2024-03-04T09:28:51.442 INFO:teuthology.orchestra.run.smithi052.stdout:pg_num: 32 2024-03-04T09:28:51.443 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:51.446+0000 7f96ec556640 1 -- 172.21.15.52:0/3863474526 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f96d00a1d80 msgr2=0x7f96d00a4240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:28:51.443 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:51.446+0000 7f96ec556640 1 --2- 172.21.15.52:0/3863474526 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f96d00a1d80 0x7f96d00a4240 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f96e4101790 tx=0x7f96d4005f90 comp rx=0 tx=0).stop 2024-03-04T09:28:51.444 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:51.446+0000 7f96ec556640 1 -- 172.21.15.52:0/3863474526 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f96e41020b0 msgr2=0x7f96e4101a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:28:51.444 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:51.446+0000 7f96ec556640 1 --2- 172.21.15.52:0/3863474526 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f96e41020b0 0x7f96e4101a20 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f96e0009230 tx=0x7f96e0005d50 comp rx=0 tx=0).stop 2024-03-04T09:28:51.444 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:51.446+0000 7f96ec556640 1 -- 172.21.15.52:0/3863474526 shutdown_connections 2024-03-04T09:28:51.444 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:51.446+0000 7f96ec556640 1 --2- 172.21.15.52:0/3863474526 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f96d00a1d80 0x7f96d00a4240 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:51.444 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:51.446+0000 7f96ec556640 1 --2- 172.21.15.52:0/3863474526 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f96e4107310 0x7f96e41005a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:51.444 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:51.446+0000 7f96ec556640 1 --2- 172.21.15.52:0/3863474526 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f96e41049e0 0x7f96e4100060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:51.444 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:51.446+0000 7f96ec556640 1 --2- 172.21.15.52:0/3863474526 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f96e41020b0 0x7f96e4101a20 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:51.444 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:51.446+0000 7f96ec556640 1 -- 172.21.15.52:0/3863474526 >> 172.21.15.52:0/3863474526 conn(0x7f96e40fba60 msgr2=0x7f96e4102a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:28:51.444 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:51.446+0000 7f96ec556640 1 -- 172.21.15.52:0/3863474526 shutdown_connections 2024-03-04T09:28:51.444 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:51.446+0000 7f96ec556640 1 -- 172.21.15.52:0/3863474526 wait complete. 2024-03-04T09:28:51.455 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-04T09:28:51.561 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:51.561+0000 7f4d9b17a640 1 -- 172.21.15.52:0/276815034 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4d94107370 msgr2=0x7f4d94109760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:28:51.561 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:51.561+0000 7f4d9b17a640 1 --2- 172.21.15.52:0/276815034 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4d94107370 0x7f4d94109760 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f4d8c009ca0 tx=0x7f4d8c02c0f0 comp rx=0 tx=0).stop 2024-03-04T09:28:51.561 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:51.562+0000 7f4d9b17a640 1 -- 172.21.15.52:0/276815034 shutdown_connections 2024-03-04T09:28:51.561 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:51.562+0000 7f4d9b17a640 1 --2- 172.21.15.52:0/276815034 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4d94107370 0x7f4d94109760 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:51.561 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:51.562+0000 7f4d9b17a640 1 --2- 172.21.15.52:0/276815034 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4d94104a40 0x7f4d94106e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:51.561 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:51.562+0000 7f4d9b17a640 1 --2- 172.21.15.52:0/276815034 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4d94102110 0x7f4d94104500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:51.561 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:51.562+0000 7f4d9b17a640 1 -- 172.21.15.52:0/276815034 >> 172.21.15.52:0/276815034 conn(0x7f4d940fba80 msgr2=0x7f4d940fdea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:28:51.561 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:51.562+0000 7f4d9b17a640 1 -- 172.21.15.52:0/276815034 shutdown_connections 2024-03-04T09:28:51.561 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:51.562+0000 7f4d9b17a640 1 -- 172.21.15.52:0/276815034 wait complete. 2024-03-04T09:28:51.562 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:51.562+0000 7f4d9b17a640 1 Processor -- start 2024-03-04T09:28:51.562 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:51.562+0000 7f4d9b17a640 1 -- start start 2024-03-04T09:28:51.562 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:51.562+0000 7f4d9b17a640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4d94102110 0x7f4d941978a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:28:51.562 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:51.562+0000 7f4d9b17a640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4d94104a40 0x7f4d94197de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:28:51.562 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:51.562+0000 7f4d9b17a640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4d94107370 0x7f4d9419ee50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:28:51.562 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:51.562+0000 7f4d9b17a640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d9410d280 con 0x7f4d94102110 2024-03-04T09:28:51.562 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:51.562+0000 7f4d9b17a640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d9410d100 con 0x7f4d94107370 2024-03-04T09:28:51.562 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:51.562+0000 7f4d9b17a640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d9410d400 con 0x7f4d94104a40 2024-03-04T09:28:51.562 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:51.562+0000 7f4d98f16640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4d94102110 0x7f4d941978a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:28:51.562 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:51.562+0000 7f4d98f16640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4d94102110 0x7f4d941978a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53534/0 (socket says 172.21.15.52:53534) 2024-03-04T09:28:51.562 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:51.562+0000 7f4d98f16640 1 -- 172.21.15.52:0/281599294 learned_addr learned my addr 172.21.15.52:0/281599294 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-04T09:28:51.562 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:51.563+0000 7f4d99717640 1 --2- 172.21.15.52:0/281599294 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4d94107370 0x7f4d9419ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:28:51.562 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:51.563+0000 7f4d98f16640 1 -- 172.21.15.52:0/281599294 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4d94104a40 msgr2=0x7f4d94197de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:28:51.563 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:51.563+0000 7f4d93fff640 1 --2- 172.21.15.52:0/281599294 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4d94104a40 0x7f4d94197de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:28:51.563 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:51.563+0000 7f4d98f16640 1 --2- 172.21.15.52:0/281599294 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4d94104a40 0x7f4d94197de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:51.563 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:51.563+0000 7f4d98f16640 1 -- 172.21.15.52:0/281599294 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4d94107370 msgr2=0x7f4d9419ee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:28:51.563 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:51.563+0000 7f4d98f16640 1 --2- 172.21.15.52:0/281599294 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4d94107370 0x7f4d9419ee50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:51.563 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:51.563+0000 7f4d98f16640 1 -- 172.21.15.52:0/281599294 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d9419f4c0 con 0x7f4d94102110 2024-03-04T09:28:51.563 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:51.563+0000 7f4d99717640 1 --2- 172.21.15.52:0/281599294 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4d94107370 0x7f4d9419ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-04T09:28:51.563 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:51.563+0000 7f4d98f16640 1 --2- 172.21.15.52:0/281599294 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4d94102110 0x7f4d941978a0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f4d8800e9f0 tx=0x7f4d8800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:28:51.563 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:51.563+0000 7f4d91ffb640 1 -- 172.21.15.52:0/281599294 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d8800cd80 con 0x7f4d94102110 2024-03-04T09:28:51.563 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:51.563+0000 7f4d91ffb640 1 -- 172.21.15.52:0/281599294 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4d88010910 con 0x7f4d94102110 2024-03-04T09:28:51.563 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:51.563+0000 7f4d91ffb640 1 -- 172.21.15.52:0/281599294 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d88018990 con 0x7f4d94102110 2024-03-04T09:28:51.563 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:51.563+0000 7f4d9b17a640 1 -- 172.21.15.52:0/281599294 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d9419f7b0 con 0x7f4d94102110 2024-03-04T09:28:51.563 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:51.563+0000 7f4d9b17a640 1 -- 172.21.15.52:0/281599294 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d9419fd00 con 0x7f4d94102110 2024-03-04T09:28:51.563 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:51.564+0000 7f4d9b17a640 1 -- 172.21.15.52:0/281599294 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d940fd570 con 0x7f4d94102110 2024-03-04T09:28:51.564 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:51.565+0000 7f4d91ffb640 1 -- 172.21.15.52:0/281599294 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 16) v1 ==== 136756+0+0 (secure 0 0 0) 0x7f4d88010ac0 con 0x7f4d94102110 2024-03-04T09:28:51.564 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:51.566+0000 7f4d91ffb640 1 --2- 172.21.15.52:0/281599294 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f4d840a1d80 0x7f4d840a4240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:28:51.564 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:51.566+0000 7f4d93fff640 1 --2- 172.21.15.52:0/281599294 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f4d840a1d80 0x7f4d840a4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:28:51.564 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:51.566+0000 7f4d91ffb640 1 -- 172.21.15.52:0/281599294 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8802+0+0 (secure 0 0 0) 0x7f4d880ce800 con 0x7f4d94102110 2024-03-04T09:28:51.565 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:51.566+0000 7f4d93fff640 1 --2- 172.21.15.52:0/281599294 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f4d840a1d80 0x7f4d840a4240 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f4d94198db0 tx=0x7f4d80005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:28:51.566 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:51.570+0000 7f4d91ffb640 1 -- 172.21.15.52:0/281599294 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f4d8809b990 con 0x7f4d94102110 2024-03-04T09:28:51.760 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:51.763+0000 7f4d9b17a640 1 -- 172.21.15.52:0/281599294 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f4d9402cff0 con 0x7f4d94102110 2024-03-04T09:28:51.761 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:51.764+0000 7f4d91ffb640 1 -- 172.21.15.52:0/281599294 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v89) v1 ==== 81+0+590 (secure 0 0 0) 0x7f4d8809b290 con 0x7f4d94102110 2024-03-04T09:28:51.761 INFO:teuthology.orchestra.run.smithi052.stdout:172.21.15.52:0/4082645718 2024-03-05T09:21:39.401935+0000 2024-03-04T09:28:51.761 INFO:teuthology.orchestra.run.smithi052.stdout:172.21.15.52:0/1271949988 2024-03-05T09:21:39.401935+0000 2024-03-04T09:28:51.761 INFO:teuthology.orchestra.run.smithi052.stdout:172.21.15.52:6801/2379361333 2024-03-05T09:21:12.760938+0000 2024-03-04T09:28:51.761 INFO:teuthology.orchestra.run.smithi052.stdout:172.21.15.52:0/497075075 2024-03-05T09:21:12.760938+0000 2024-03-04T09:28:51.762 INFO:teuthology.orchestra.run.smithi052.stdout:172.21.15.52:6800/2379361333 2024-03-05T09:21:12.760938+0000 2024-03-04T09:28:51.762 INFO:teuthology.orchestra.run.smithi052.stdout:172.21.15.52:0/568242577 2024-03-05T09:21:12.760938+0000 2024-03-04T09:28:51.762 INFO:teuthology.orchestra.run.smithi052.stdout:172.21.15.52:6801/2425730636 2024-03-05T09:21:39.401935+0000 2024-03-04T09:28:51.762 INFO:teuthology.orchestra.run.smithi052.stdout:172.21.15.52:6800/2425730636 2024-03-05T09:21:39.401935+0000 2024-03-04T09:28:51.762 INFO:teuthology.orchestra.run.smithi052.stdout:172.21.15.52:0/2743084243 2024-03-05T09:21:39.401935+0000 2024-03-04T09:28:51.762 INFO:teuthology.orchestra.run.smithi052.stdout:172.21.15.52:0/1035878034 2024-03-05T09:21:12.760938+0000 2024-03-04T09:28:51.763 INFO:teuthology.orchestra.run.smithi052.stderr:listed 10 entries 2024-03-04T09:28:51.763 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:51.766+0000 7f4d9b17a640 1 -- 172.21.15.52:0/281599294 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f4d840a1d80 msgr2=0x7f4d840a4240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:28:51.763 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:51.766+0000 7f4d9b17a640 1 --2- 172.21.15.52:0/281599294 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f4d840a1d80 0x7f4d840a4240 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f4d94198db0 tx=0x7f4d80005f90 comp rx=0 tx=0).stop 2024-03-04T09:28:51.764 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:51.766+0000 7f4d9b17a640 1 -- 172.21.15.52:0/281599294 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4d94102110 msgr2=0x7f4d941978a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:28:51.764 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:51.766+0000 7f4d9b17a640 1 --2- 172.21.15.52:0/281599294 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4d94102110 0x7f4d941978a0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f4d8800e9f0 tx=0x7f4d8800eeb0 comp rx=0 tx=0).stop 2024-03-04T09:28:51.765 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:51.766+0000 7f4d9b17a640 1 -- 172.21.15.52:0/281599294 shutdown_connections 2024-03-04T09:28:51.765 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:51.766+0000 7f4d9b17a640 1 --2- 172.21.15.52:0/281599294 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f4d840a1d80 0x7f4d840a4240 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:51.765 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:51.766+0000 7f4d9b17a640 1 --2- 172.21.15.52:0/281599294 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4d94107370 0x7f4d9419ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:51.765 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:51.766+0000 7f4d9b17a640 1 --2- 172.21.15.52:0/281599294 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4d94104a40 0x7f4d94197de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:51.765 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:51.766+0000 7f4d9b17a640 1 --2- 172.21.15.52:0/281599294 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4d94102110 0x7f4d941978a0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:51.765 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:51.766+0000 7f4d9b17a640 1 -- 172.21.15.52:0/281599294 >> 172.21.15.52:0/281599294 conn(0x7f4d940fba80 msgr2=0x7f4d94102a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:28:51.765 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:51.766+0000 7f4d9b17a640 1 -- 172.21.15.52:0/281599294 shutdown_connections 2024-03-04T09:28:51.766 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:51.766+0000 7f4d9b17a640 1 -- 172.21.15.52:0/281599294 wait complete. 2024-03-04T09:28:51.773 DEBUG:teuthology.orchestra.run.smithi052:> set -ex 2024-03-04T09:28:51.774 DEBUG:teuthology.orchestra.run.smithi052:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-04T09:28:51.789 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-04T09:28:51.935 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:51.936+0000 7f288db25640 1 -- 172.21.15.52:0/2959349493 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f28881027e0 msgr2=0x7f2888102c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:28:51.935 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:51.936+0000 7f288db25640 1 --2- 172.21.15.52:0/2959349493 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f28881027e0 0x7f2888102c60 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f287c009900 tx=0x7f287c02bc30 comp rx=0 tx=0).stop 2024-03-04T09:28:51.935 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:51.936+0000 7f288db25640 1 -- 172.21.15.52:0/2959349493 shutdown_connections 2024-03-04T09:28:51.935 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:51.936+0000 7f288db25640 1 --2- 172.21.15.52:0/2959349493 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f28881031a0 0x7f2888109a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:51.935 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:51.936+0000 7f288db25640 1 --2- 172.21.15.52:0/2959349493 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f28881027e0 0x7f2888102c60 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:51.936 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:51.936+0000 7f288db25640 1 --2- 172.21.15.52:0/2959349493 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f28881015f0 0x7f2888101a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:51.936 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:51.936+0000 7f288db25640 1 -- 172.21.15.52:0/2959349493 >> 172.21.15.52:0/2959349493 conn(0x7f28880fcb80 msgr2=0x7f28880fefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:28:51.936 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:51.936+0000 7f288db25640 1 -- 172.21.15.52:0/2959349493 shutdown_connections 2024-03-04T09:28:51.936 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:51.936+0000 7f288db25640 1 -- 172.21.15.52:0/2959349493 wait complete. 2024-03-04T09:28:51.936 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:51.937+0000 7f288db25640 1 Processor -- start 2024-03-04T09:28:51.936 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:51.937+0000 7f288db25640 1 -- start start 2024-03-04T09:28:51.936 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:51.937+0000 7f288db25640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f28881015f0 0x7f28881978e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:28:51.936 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:51.937+0000 7f288db25640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f28881027e0 0x7f2888197e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:28:51.936 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:51.937+0000 7f288db25640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f28881031a0 0x7f288819ee90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:28:51.936 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:51.937+0000 7f288db25640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f288810d270 con 0x7f28881015f0 2024-03-04T09:28:51.936 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:51.937+0000 7f288db25640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f288810d0f0 con 0x7f28881027e0 2024-03-04T09:28:51.936 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:51.937+0000 7f288db25640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f288810d3f0 con 0x7f28881031a0 2024-03-04T09:28:51.937 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:51.937+0000 7f28877fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f28881015f0 0x7f28881978e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:28:51.937 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:51.937+0000 7f28877fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f28881015f0 0x7f28881978e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53546/0 (socket says 172.21.15.52:53546) 2024-03-04T09:28:51.937 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:51.937+0000 7f28877fe640 1 -- 172.21.15.52:0/1314992918 learned_addr learned my addr 172.21.15.52:0/1314992918 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-04T09:28:51.937 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:51.937+0000 7f2886ffd640 1 --2- 172.21.15.52:0/1314992918 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f28881027e0 0x7f2888197e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:28:51.937 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:51.937+0000 7f2887fff640 1 --2- 172.21.15.52:0/1314992918 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f28881031a0 0x7f288819ee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:28:51.937 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:51.937+0000 7f28877fe640 1 -- 172.21.15.52:0/1314992918 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f28881031a0 msgr2=0x7f288819ee90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:28:51.937 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:51.937+0000 7f28877fe640 1 --2- 172.21.15.52:0/1314992918 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f28881031a0 0x7f288819ee90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:51.937 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:51.937+0000 7f28877fe640 1 -- 172.21.15.52:0/1314992918 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f28881027e0 msgr2=0x7f2888197e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:28:51.937 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:51.937+0000 7f28877fe640 1 --2- 172.21.15.52:0/1314992918 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f28881027e0 0x7f2888197e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:51.937 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:51.937+0000 7f28877fe640 1 -- 172.21.15.52:0/1314992918 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f288819f500 con 0x7f28881015f0 2024-03-04T09:28:51.937 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:51.937+0000 7f28877fe640 1 --2- 172.21.15.52:0/1314992918 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f28881015f0 0x7f28881978e0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f287800e9c0 tx=0x7f287800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:28:51.937 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:51.938+0000 7f2884ff9640 1 -- 172.21.15.52:0/1314992918 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f287800cde0 con 0x7f28881015f0 2024-03-04T09:28:51.937 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:51.938+0000 7f2884ff9640 1 -- 172.21.15.52:0/1314992918 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2878010910 con 0x7f28881015f0 2024-03-04T09:28:51.937 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:51.938+0000 7f288db25640 1 -- 172.21.15.52:0/1314992918 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f288819f7f0 con 0x7f28881015f0 2024-03-04T09:28:51.938 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:51.938+0000 7f2884ff9640 1 -- 172.21.15.52:0/1314992918 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2878018a00 con 0x7f28881015f0 2024-03-04T09:28:51.938 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:51.938+0000 7f288db25640 1 -- 172.21.15.52:0/1314992918 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f288819fd10 con 0x7f28881015f0 2024-03-04T09:28:51.938 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:51.939+0000 7f288db25640 1 -- 172.21.15.52:0/1314992918 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2888108910 con 0x7f28881015f0 2024-03-04T09:28:51.939 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:51.940+0000 7f2884ff9640 1 -- 172.21.15.52:0/1314992918 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 16) v1 ==== 136756+0+0 (secure 0 0 0) 0x7f287801f020 con 0x7f28881015f0 2024-03-04T09:28:51.939 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:51.940+0000 7f2884ff9640 1 --2- 172.21.15.52:0/1314992918 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f28700a1d80 0x7f28700a4240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:28:51.939 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:51.940+0000 7f2884ff9640 1 -- 172.21.15.52:0/1314992918 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8802+0+0 (secure 0 0 0) 0x7f28780ce270 con 0x7f28881015f0 2024-03-04T09:28:51.939 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:51.940+0000 7f2886ffd640 1 --2- 172.21.15.52:0/1314992918 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f28700a1d80 0x7f28700a4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:28:51.939 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:51.941+0000 7f2886ffd640 1 --2- 172.21.15.52:0/1314992918 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f28700a1d80 0x7f28700a4240 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f287c02c7b0 tx=0x7f287c005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:28:51.941 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:51.945+0000 7f2884ff9640 1 -- 172.21.15.52:0/1314992918 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f287809b460 con 0x7f28881015f0 2024-03-04T09:28:52.085 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:28:51 smithi052 ceph-mon[32473]: from='client.? 172.21.15.52:0/2920646061' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-04T09:28:52.085 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:28:51 smithi052 ceph-mon[32473]: from='client.? 172.21.15.52:0/3863474526' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-04T09:28:52.085 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:28:51 smithi052 ceph-mon[32473]: from='client.? 172.21.15.52:0/281599294' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-03-04T09:28:52.129 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:52.132+0000 7f288db25640 1 -- 172.21.15.52:0/1314992918 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f28880621d0 con 0x7f28881015f0 2024-03-04T09:28:52.130 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:52.133+0000 7f2884ff9640 1 -- 172.21.15.52:0/1314992918 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) v1 ==== 74+0+24164 (secure 0 0 0) 0x7f287809ad60 con 0x7f28881015f0 2024-03-04T09:28:52.131 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-03-04T09:28:52.131 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":89,"fsid":"4e059f08-da08-11ee-95c6-87774f69a715","created":"2024-03-04T09:20:46.104963+0000","modified":"2024-03-04T09:28:42.295274+0000","last_up_change":"2024-03-04T09:27:49.261365+0000","last_in_change":"2024-03-04T09:27:30.024964+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-04T09:24:01.742461+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-04T09:28:35.835600+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-04T09:28:36.453099+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-04T09:28:40.168831+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"583172be-602f-4bf4-bfcb-42663cf5b277","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6802","nonce":1371221397},{"type":"v1","addr":"172.21.15.52:6803","nonce":1371221397}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6804","nonce":1371221397},{"type":"v1","addr":"172.21.15.52:6805","nonce":1371221397}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6808","nonce":1371221397},{"type":"v1","addr":"172.21.15.52:6809","nonce":1371221397}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6806","nonce":1371221397},{"type":"v1","addr":"172.21.15.52:6807","nonce":1371221397}]},"public_addr":"172.21.15.52:6803/1371221397","cluster_addr":"172.21.15.52:6805/1371221397","heartbeat_back_addr":"172.21.15.52:6809/1371221397","heartbeat_front_addr":"172.21.15.52:6807/1371221397","state":["exists","up"]},{"osd":1,"uuid":"5ae0081c-a2f6-46eb-ace0-0368208377b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":2540865537},{"type":"v1","addr":"172.21.15.106:6801","nonce":2540865537}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":2540865537},{"type":"v1","addr":"172.21.15.106:6803","nonce":2540865537}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":2540865537},{"type":"v1","addr":"172.21.15.106:6807","nonce":2540865537}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":2540865537},{"type":"v1","addr":"172.21.15.106:6805","nonce":2540865537}]},"public_addr":"172.21.15.106:6801/2540865537","cluster_addr":"172.21.15.106:6803/2540865537","heartbeat_back_addr":"172.21.15.106:6807/2540865537","heartbeat_front_addr":"172.21.15.106:6805/2540865537","state":["exists","up"]},{"osd":2,"uuid":"1948bac7-2bb2-4218-8b93-4e4743a0de04","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":1377083164},{"type":"v1","addr":"172.21.15.191:6801","nonce":1377083164}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":1377083164},{"type":"v1","addr":"172.21.15.191:6803","nonce":1377083164}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":1377083164},{"type":"v1","addr":"172.21.15.191:6807","nonce":1377083164}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":1377083164},{"type":"v1","addr":"172.21.15.191:6805","nonce":1377083164}]},"public_addr":"172.21.15.191:6801/1377083164","cluster_addr":"172.21.15.191:6803/1377083164","heartbeat_back_addr":"172.21.15.191:6807/1377083164","heartbeat_front_addr":"172.21.15.191:6805/1377083164","state":["exists","up"]},{"osd":3,"uuid":"16c9e9fe-7167-4992-bf12-1a7398e0498a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6810","nonce":2388571810},{"type":"v1","addr":"172.21.15.52:6811","nonce":2388571810}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6812","nonce":2388571810},{"type":"v1","addr":"172.21.15.52:6813","nonce":2388571810}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6816","nonce":2388571810},{"type":"v1","addr":"172.21.15.52:6817","nonce":2388571810}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6814","nonce":2388571810},{"type":"v1","addr":"172.21.15.52:6815","nonce":2388571810}]},"public_addr":"172.21.15.52:6811/2388571810","cluster_addr":"172.21.15.52:6813/2388571810","heartbeat_back_addr":"172.21.15.52:6817/2388571810","heartbeat_front_addr":"172.21.15.52:6815/2388571810","state":["exists","up"]},{"osd":4,"uuid":"c8ef48c7-7c78-44d3-a9e5-99ec750abd3d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6808","nonce":3465868015},{"type":"v1","addr":"172.21.15.106:6809","nonce":3465868015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6810","nonce":3465868015},{"type":"v1","addr":"172.21.15.106:6811","nonce":3465868015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6814","nonce":3465868015},{"type":"v1","addr":"172.21.15.106:6815","nonce":3465868015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6812","nonce":3465868015},{"type":"v1","addr":"172.21.15.106:6813","nonce":3465868015}]},"public_addr":"172.21.15.106:6809/3465868015","cluster_addr":"172.21.15.106:6811/3465868015","heartbeat_back_addr":"172.21.15.106:6815/3465868015","heartbeat_front_addr":"172.21.15.106:6813/3465868015","state":["exists","up"]},{"osd":5,"uuid":"9e41824a-c5a2-4f88-be50-2ba1dceb3220","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":1485092643},{"type":"v1","addr":"172.21.15.191:6809","nonce":1485092643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":1485092643},{"type":"v1","addr":"172.21.15.191:6811","nonce":1485092643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":1485092643},{"type":"v1","addr":"172.21.15.191:6815","nonce":1485092643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":1485092643},{"type":"v1","addr":"172.21.15.191:6813","nonce":1485092643}]},"public_addr":"172.21.15.191:6809/1485092643","cluster_addr":"172.21.15.191:6811/1485092643","heartbeat_back_addr":"172.21.15.191:6815/1485092643","heartbeat_front_addr":"172.21.15.191:6813/1485092643","state":["exists","up"]},{"osd":6,"uuid":"b35dccee-ec16-452e-ac69-3ac2fb7b27fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6818","nonce":1989797856},{"type":"v1","addr":"172.21.15.52:6819","nonce":1989797856}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6820","nonce":1989797856},{"type":"v1","addr":"172.21.15.52:6821","nonce":1989797856}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6824","nonce":1989797856},{"type":"v1","addr":"172.21.15.52:6825","nonce":1989797856}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6822","nonce":1989797856},{"type":"v1","addr":"172.21.15.52:6823","nonce":1989797856}]},"public_addr":"172.21.15.52:6819/1989797856","cluster_addr":"172.21.15.52:6821/1989797856","heartbeat_back_addr":"172.21.15.52:6825/1989797856","heartbeat_front_addr":"172.21.15.52:6823/1989797856","state":["exists","up"]},{"osd":7,"uuid":"71303e3d-14ec-4cbd-b19c-67645a1de3c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6816","nonce":1450870329},{"type":"v1","addr":"172.21.15.106:6817","nonce":1450870329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":1450870329},{"type":"v1","addr":"172.21.15.106:6819","nonce":1450870329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":1450870329},{"type":"v1","addr":"172.21.15.106:6823","nonce":1450870329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6820","nonce":1450870329},{"type":"v1","addr":"172.21.15.106:6821","nonce":1450870329}]},"public_addr":"172.21.15.106:6817/1450870329","cluster_addr":"172.21.15.106:6819/1450870329","heartbeat_back_addr":"172.21.15.106:6823/1450870329","heartbeat_front_addr":"172.21.15.106:6821/1450870329","state":["exists","up"]},{"osd":8,"uuid":"6f5786c4-4402-435a-bdad-adff46fe9473","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":1114119072},{"type":"v1","addr":"172.21.15.191:6817","nonce":1114119072}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6818","nonce":1114119072},{"type":"v1","addr":"172.21.15.191:6819","nonce":1114119072}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":1114119072},{"type":"v1","addr":"172.21.15.191:6823","nonce":1114119072}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":1114119072},{"type":"v1","addr":"172.21.15.191:6821","nonce":1114119072}]},"public_addr":"172.21.15.191:6817/1114119072","cluster_addr":"172.21.15.191:6819/1114119072","heartbeat_back_addr":"172.21.15.191:6823/1114119072","heartbeat_front_addr":"172.21.15.191:6821/1114119072","state":["exists","up"]},{"osd":9,"uuid":"062bbaa5-1227-4684-b859-450f5594a90c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6826","nonce":3485821914},{"type":"v1","addr":"172.21.15.52:6827","nonce":3485821914}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6828","nonce":3485821914},{"type":"v1","addr":"172.21.15.52:6829","nonce":3485821914}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6832","nonce":3485821914},{"type":"v1","addr":"172.21.15.52:6833","nonce":3485821914}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6830","nonce":3485821914},{"type":"v1","addr":"172.21.15.52:6831","nonce":3485821914}]},"public_addr":"172.21.15.52:6827/3485821914","cluster_addr":"172.21.15.52:6829/3485821914","heartbeat_back_addr":"172.21.15.52:6833/3485821914","heartbeat_front_addr":"172.21.15.52:6831/3485821914","state":["exists","up"]},{"osd":10,"uuid":"1a285a5a-f793-48da-a8eb-5326d9528cd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":3815586576},{"type":"v1","addr":"172.21.15.106:6825","nonce":3815586576}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6826","nonce":3815586576},{"type":"v1","addr":"172.21.15.106:6827","nonce":3815586576}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":3815586576},{"type":"v1","addr":"172.21.15.106:6831","nonce":3815586576}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":3815586576},{"type":"v1","addr":"172.21.15.106:6829","nonce":3815586576}]},"public_addr":"172.21.15.106:6825/3815586576","cluster_addr":"172.21.15.106:6827/3815586576","heartbeat_back_addr":"172.21.15.106:6831/3815586576","heartbeat_front_addr":"172.21.15.106:6829/3815586576","state":["exists","up"]},{"osd":11,"uuid":"1f8b9a43-57fa-41f4-9969-3b803a74bf45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":2009632843},{"type":"v1","addr":"172.21.15.191:6825","nonce":2009632843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":2009632843},{"type":"v1","addr":"172.21.15.191:6827","nonce":2009632843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":2009632843},{"type":"v1","addr":"172.21.15.191:6831","nonce":2009632843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":2009632843},{"type":"v1","addr":"172.21.15.191:6829","nonce":2009632843}]},"public_addr":"172.21.15.191:6825/2009632843","cluster_addr":"172.21.15.191:6827/2009632843","heartbeat_back_addr":"172.21.15.191:6831/2009632843","heartbeat_front_addr":"172.21.15.191:6829/2009632843","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T09:23:10.894763+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T09:23:34.420055+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T09:23:58.288324+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T09:24:23.673457+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T09:24:48.208561+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T09:25:13.405473+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T09:25:38.670915+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T09:26:04.451257+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T09:26:30.281380+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T09:26:56.543143+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T09:27:22.412930+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T09:27:48.003866+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.52:0/4082645718":"2024-03-05T09:21:39.401935+0000","172.21.15.52:0/1271949988":"2024-03-05T09:21:39.401935+0000","172.21.15.52:6801/2379361333":"2024-03-05T09:21:12.760938+0000","172.21.15.52:0/497075075":"2024-03-05T09:21:12.760938+0000","172.21.15.52:6800/2379361333":"2024-03-05T09:21:12.760938+0000","172.21.15.52:0/568242577":"2024-03-05T09:21:12.760938+0000","172.21.15.52:6801/2425730636":"2024-03-05T09:21:39.401935+0000","172.21.15.52:6800/2425730636":"2024-03-05T09:21:39.401935+0000","172.21.15.52:0/2743084243":"2024-03-05T09:21:39.401935+0000","172.21.15.52:0/1035878034":"2024-03-05T09:21:12.760938+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-04T09:28:52.133 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:52.135+0000 7f288db25640 1 -- 172.21.15.52:0/1314992918 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f28700a1d80 msgr2=0x7f28700a4240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:28:52.133 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:52.135+0000 7f288db25640 1 --2- 172.21.15.52:0/1314992918 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f28700a1d80 0x7f28700a4240 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f287c02c7b0 tx=0x7f287c005fd0 comp rx=0 tx=0).stop 2024-03-04T09:28:52.133 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:52.135+0000 7f288db25640 1 -- 172.21.15.52:0/1314992918 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f28881015f0 msgr2=0x7f28881978e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:28:52.133 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:52.135+0000 7f288db25640 1 --2- 172.21.15.52:0/1314992918 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f28881015f0 0x7f28881978e0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f287800e9c0 tx=0x7f287800ee80 comp rx=0 tx=0).stop 2024-03-04T09:28:52.133 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:52.135+0000 7f288db25640 1 -- 172.21.15.52:0/1314992918 shutdown_connections 2024-03-04T09:28:52.133 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:52.135+0000 7f288db25640 1 --2- 172.21.15.52:0/1314992918 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f28700a1d80 0x7f28700a4240 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:52.133 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:52.135+0000 7f288db25640 1 --2- 172.21.15.52:0/1314992918 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f28881031a0 0x7f288819ee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:52.133 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:52.135+0000 7f288db25640 1 --2- 172.21.15.52:0/1314992918 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f28881027e0 0x7f2888197e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:52.133 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:52.135+0000 7f288db25640 1 --2- 172.21.15.52:0/1314992918 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f28881015f0 0x7f28881978e0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:52.134 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:52.136+0000 7f288db25640 1 -- 172.21.15.52:0/1314992918 >> 172.21.15.52:0/1314992918 conn(0x7f28880fcb80 msgr2=0x7f2888105ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:28:52.134 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:52.136+0000 7f288db25640 1 -- 172.21.15.52:0/1314992918 shutdown_connections 2024-03-04T09:28:52.134 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:52.136+0000 7f288db25640 1 -- 172.21.15.52:0/1314992918 wait complete. 2024-03-04T09:28:52.144 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-04T09:24:01.742461+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-04T09:28:35.835600+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-04T09:28:36.453099+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-04T09:28:40.168831+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '89', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-04T09:28:52.145 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-04T09:28:52.240 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:28:51 smithi191 ceph-mon[36816]: from='client.? 172.21.15.52:0/2920646061' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-04T09:28:52.240 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:28:51 smithi191 ceph-mon[36816]: from='client.? 172.21.15.52:0/3863474526' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-04T09:28:52.240 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:28:51 smithi191 ceph-mon[36816]: from='client.? 172.21.15.52:0/281599294' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-03-04T09:28:52.253 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:52.253+0000 7f9c2a00e640 1 -- 172.21.15.52:0/2030173526 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9c241007b0 msgr2=0x7f9c24100c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:28:52.253 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:52.253+0000 7f9c2a00e640 1 --2- 172.21.15.52:0/2030173526 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9c241007b0 0x7f9c24100c10 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f9c18009900 tx=0x7f9c1802bc30 comp rx=0 tx=0).stop 2024-03-04T09:28:52.253 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:52.253+0000 7f9c2a00e640 1 -- 172.21.15.52:0/2030173526 shutdown_connections 2024-03-04T09:28:52.253 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:52.253+0000 7f9c2a00e640 1 --2- 172.21.15.52:0/2030173526 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9c24101150 0x7f9c241015d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:52.253 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:52.253+0000 7f9c2a00e640 1 --2- 172.21.15.52:0/2030173526 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9c241007b0 0x7f9c24100c10 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:52.254 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:52.253+0000 7f9c2a00e640 1 --2- 172.21.15.52:0/2030173526 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9c240ffe50 0x7f9c24100270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:52.254 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:52.253+0000 7f9c2a00e640 1 -- 172.21.15.52:0/2030173526 >> 172.21.15.52:0/2030173526 conn(0x7f9c240fba70 msgr2=0x7f9c240fde90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:28:52.254 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:52.253+0000 7f9c2a00e640 1 -- 172.21.15.52:0/2030173526 shutdown_connections 2024-03-04T09:28:52.254 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:52.253+0000 7f9c2a00e640 1 -- 172.21.15.52:0/2030173526 wait complete. 2024-03-04T09:28:52.254 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:52.254+0000 7f9c2a00e640 1 Processor -- start 2024-03-04T09:28:52.254 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:52.254+0000 7f9c2a00e640 1 -- start start 2024-03-04T09:28:52.254 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:52.254+0000 7f9c2a00e640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9c240ffe50 0x7f9c24199b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:28:52.254 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:52.254+0000 7f9c2a00e640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9c241007b0 0x7f9c2419a040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:28:52.254 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:52.254+0000 7f9c2a00e640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9c24101150 0x7f9c241a10b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:28:52.254 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:52.254+0000 7f9c2a00e640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c2410f420 con 0x7f9c24101150 2024-03-04T09:28:52.255 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:52.254+0000 7f9c2a00e640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c2410f2a0 con 0x7f9c240ffe50 2024-03-04T09:28:52.255 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:52.254+0000 7f9c2a00e640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c2410f5a0 con 0x7f9c241007b0 2024-03-04T09:28:52.255 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:52.254+0000 7f9c23fff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9c24101150 0x7f9c241a10b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:28:52.255 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:52.254+0000 7f9c23fff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9c24101150 0x7f9c241a10b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53560/0 (socket says 172.21.15.52:53560) 2024-03-04T09:28:52.255 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:52.254+0000 7f9c23fff640 1 -- 172.21.15.52:0/1484823310 learned_addr learned my addr 172.21.15.52:0/1484823310 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-04T09:28:52.255 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:52.254+0000 7f9c237fe640 1 --2- 172.21.15.52:0/1484823310 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9c240ffe50 0x7f9c24199b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:28:52.255 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:52.254+0000 7f9c22ffd640 1 --2- 172.21.15.52:0/1484823310 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9c241007b0 0x7f9c2419a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:28:52.255 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:52.255+0000 7f9c23fff640 1 -- 172.21.15.52:0/1484823310 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9c241007b0 msgr2=0x7f9c2419a040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:28:52.255 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:52.255+0000 7f9c23fff640 1 --2- 172.21.15.52:0/1484823310 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9c241007b0 0x7f9c2419a040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:52.255 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:52.255+0000 7f9c23fff640 1 -- 172.21.15.52:0/1484823310 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9c240ffe50 msgr2=0x7f9c24199b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:28:52.255 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:52.255+0000 7f9c23fff640 1 --2- 172.21.15.52:0/1484823310 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9c240ffe50 0x7f9c24199b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:52.255 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:52.255+0000 7f9c23fff640 1 -- 172.21.15.52:0/1484823310 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c241a1720 con 0x7f9c24101150 2024-03-04T09:28:52.256 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:52.255+0000 7f9c22ffd640 1 --2- 172.21.15.52:0/1484823310 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9c241007b0 0x7f9c2419a040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T09:28:52.256 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:52.255+0000 7f9c23fff640 1 --2- 172.21.15.52:0/1484823310 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9c24101150 0x7f9c241a10b0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f9c1c00cad0 tx=0x7f9c1c00cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:28:52.256 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:52.255+0000 7f9c20ff9640 1 -- 172.21.15.52:0/1484823310 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c1c00ae20 con 0x7f9c24101150 2024-03-04T09:28:52.256 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:52.255+0000 7f9c20ff9640 1 -- 172.21.15.52:0/1484823310 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9c1c005020 con 0x7f9c24101150 2024-03-04T09:28:52.256 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:52.255+0000 7f9c2a00e640 1 -- 172.21.15.52:0/1484823310 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c241a1a10 con 0x7f9c24101150 2024-03-04T09:28:52.256 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:52.255+0000 7f9c20ff9640 1 -- 172.21.15.52:0/1484823310 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c1c00f6f0 con 0x7f9c24101150 2024-03-04T09:28:52.256 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:52.255+0000 7f9c2a00e640 1 -- 172.21.15.52:0/1484823310 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c241a1ee0 con 0x7f9c24101150 2024-03-04T09:28:52.256 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:52.256+0000 7f9c2a00e640 1 -- 172.21.15.52:0/1484823310 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9c2410adb0 con 0x7f9c24101150 2024-03-04T09:28:52.257 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:52.257+0000 7f9c20ff9640 1 -- 172.21.15.52:0/1484823310 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 16) v1 ==== 136756+0+0 (secure 0 0 0) 0x7f9c1c0051d0 con 0x7f9c24101150 2024-03-04T09:28:52.257 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:52.257+0000 7f9c20ff9640 1 --2- 172.21.15.52:0/1484823310 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f9c0c0a1d80 0x7f9c0c0a4240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:28:52.257 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:52.258+0000 7f9c237fe640 1 --2- 172.21.15.52:0/1484823310 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f9c0c0a1d80 0x7f9c0c0a4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:28:52.257 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:52.258+0000 7f9c20ff9640 1 -- 172.21.15.52:0/1484823310 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8802+0+0 (secure 0 0 0) 0x7f9c1c0ce040 con 0x7f9c24101150 2024-03-04T09:28:52.258 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:52.258+0000 7f9c237fe640 1 --2- 172.21.15.52:0/1484823310 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f9c0c0a1d80 0x7f9c0c0a4240 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f9c140095f0 tx=0x7f9c14005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:28:52.259 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:52.262+0000 7f9c20ff9640 1 -- 172.21.15.52:0/1484823310 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f9c1c014030 con 0x7f9c24101150 2024-03-04T09:28:52.381 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:28:51 smithi106 ceph-mon[36484]: from='client.? 172.21.15.52:0/2920646061' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-04T09:28:52.381 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:28:51 smithi106 ceph-mon[36484]: from='client.? 172.21.15.52:0/3863474526' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-04T09:28:52.381 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:28:51 smithi106 ceph-mon[36484]: from='client.? 172.21.15.52:0/281599294' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-03-04T09:28:52.444 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:52.444+0000 7f9c2a00e640 1 -- 172.21.15.52:0/1484823310 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f9c240621d0 con 0x7f9c24101150 2024-03-04T09:28:52.444 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:52.445+0000 7f9c20ff9640 1 -- 172.21.15.52:0/1484823310 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v89) v1 ==== 93+0+10 (secure 0 0 0) 0x7f9c1c09b230 con 0x7f9c24101150 2024-03-04T09:28:52.445 INFO:teuthology.orchestra.run.smithi052.stdout:pg_num: 1 2024-03-04T09:28:52.445 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:52.447+0000 7f9c2a00e640 1 -- 172.21.15.52:0/1484823310 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f9c0c0a1d80 msgr2=0x7f9c0c0a4240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:28:52.445 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:52.447+0000 7f9c2a00e640 1 --2- 172.21.15.52:0/1484823310 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f9c0c0a1d80 0x7f9c0c0a4240 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f9c140095f0 tx=0x7f9c14005f90 comp rx=0 tx=0).stop 2024-03-04T09:28:52.446 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:52.447+0000 7f9c2a00e640 1 -- 172.21.15.52:0/1484823310 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9c24101150 msgr2=0x7f9c241a10b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:28:52.446 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:52.447+0000 7f9c2a00e640 1 --2- 172.21.15.52:0/1484823310 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9c24101150 0x7f9c241a10b0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f9c1c00cad0 tx=0x7f9c1c00cf90 comp rx=0 tx=0).stop 2024-03-04T09:28:52.446 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:52.447+0000 7f9c2a00e640 1 -- 172.21.15.52:0/1484823310 shutdown_connections 2024-03-04T09:28:52.446 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:52.447+0000 7f9c2a00e640 1 --2- 172.21.15.52:0/1484823310 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f9c0c0a1d80 0x7f9c0c0a4240 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:52.446 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:52.447+0000 7f9c2a00e640 1 --2- 172.21.15.52:0/1484823310 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9c24101150 0x7f9c241a10b0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:52.446 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:52.447+0000 7f9c2a00e640 1 --2- 172.21.15.52:0/1484823310 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9c241007b0 0x7f9c2419a040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:52.446 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:52.447+0000 7f9c2a00e640 1 --2- 172.21.15.52:0/1484823310 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9c240ffe50 0x7f9c24199b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:52.446 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:52.447+0000 7f9c2a00e640 1 -- 172.21.15.52:0/1484823310 >> 172.21.15.52:0/1484823310 conn(0x7f9c240fba70 msgr2=0x7f9c240fd230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:28:52.446 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:52.447+0000 7f9c2a00e640 1 -- 172.21.15.52:0/1484823310 shutdown_connections 2024-03-04T09:28:52.446 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:52.447+0000 7f9c2a00e640 1 -- 172.21.15.52:0/1484823310 wait complete. 2024-03-04T09:28:52.455 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-04T09:28:52.562 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:52.563+0000 7f431cabb640 1 -- 172.21.15.52:0/3932510754 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4318101830 msgr2=0x7f43181080c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:28:52.562 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:52.563+0000 7f431cabb640 1 --2- 172.21.15.52:0/3932510754 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4318101830 0x7f43181080c0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f4310009ca0 tx=0x7f431002c0f0 comp rx=0 tx=0).stop 2024-03-04T09:28:52.562 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:52.563+0000 7f431cabb640 1 -- 172.21.15.52:0/3932510754 shutdown_connections 2024-03-04T09:28:52.563 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:52.563+0000 7f431cabb640 1 --2- 172.21.15.52:0/3932510754 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4318101830 0x7f43181080c0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:52.563 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:52.563+0000 7f431cabb640 1 --2- 172.21.15.52:0/3932510754 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4318100e70 0x7f43181012f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:52.563 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:52.563+0000 7f431cabb640 1 --2- 172.21.15.52:0/3932510754 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f43180ffc80 0x7f43181000a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:52.563 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:52.563+0000 7f431cabb640 1 -- 172.21.15.52:0/3932510754 >> 172.21.15.52:0/3932510754 conn(0x7f43180fb1d0 msgr2=0x7f43180fd630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:28:52.563 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:52.563+0000 7f431cabb640 1 -- 172.21.15.52:0/3932510754 shutdown_connections 2024-03-04T09:28:52.563 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:52.563+0000 7f431cabb640 1 -- 172.21.15.52:0/3932510754 wait complete. 2024-03-04T09:28:52.563 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:52.564+0000 7f431cabb640 1 Processor -- start 2024-03-04T09:28:52.563 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:52.564+0000 7f431cabb640 1 -- start start 2024-03-04T09:28:52.563 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:52.564+0000 7f431cabb640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f43180ffc80 0x7f4318196780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:28:52.563 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:52.564+0000 7f431cabb640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4318100e70 0x7f4318196cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:28:52.563 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:52.564+0000 7f431cabb640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4318101830 0x7f431819dd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:28:52.563 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:52.564+0000 7f431cabb640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f431810c1c0 con 0x7f4318101830 2024-03-04T09:28:52.563 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:52.564+0000 7f431cabb640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f431810c040 con 0x7f4318100e70 2024-03-04T09:28:52.563 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:52.564+0000 7f431cabb640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f431810c340 con 0x7f43180ffc80 2024-03-04T09:28:52.564 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:52.564+0000 7f4316d9d640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4318101830 0x7f431819dd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:28:52.564 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:52.564+0000 7f4316d9d640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4318101830 0x7f431819dd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53584/0 (socket says 172.21.15.52:53584) 2024-03-04T09:28:52.564 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:52.564+0000 7f4316d9d640 1 -- 172.21.15.52:0/1021343700 learned_addr learned my addr 172.21.15.52:0/1021343700 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-04T09:28:52.564 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:52.564+0000 7f430ffff640 1 --2- 172.21.15.52:0/1021343700 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4318100e70 0x7f4318196cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:28:52.564 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:52.564+0000 7f431659c640 1 --2- 172.21.15.52:0/1021343700 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f43180ffc80 0x7f4318196780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:28:52.564 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:52.564+0000 7f4316d9d640 1 -- 172.21.15.52:0/1021343700 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f43180ffc80 msgr2=0x7f4318196780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:28:52.564 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:52.564+0000 7f4316d9d640 1 --2- 172.21.15.52:0/1021343700 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f43180ffc80 0x7f4318196780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:52.564 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:52.564+0000 7f4316d9d640 1 -- 172.21.15.52:0/1021343700 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4318100e70 msgr2=0x7f4318196cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:28:52.564 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:52.564+0000 7f4316d9d640 1 --2- 172.21.15.52:0/1021343700 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4318100e70 0x7f4318196cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:52.564 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:52.564+0000 7f4316d9d640 1 -- 172.21.15.52:0/1021343700 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f431819e3a0 con 0x7f4318101830 2024-03-04T09:28:52.564 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:52.565+0000 7f4316d9d640 1 --2- 172.21.15.52:0/1021343700 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4318101830 0x7f431819dd30 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f4310037040 tx=0x7f4310032f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:28:52.564 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:52.565+0000 7f430f7fe640 1 -- 172.21.15.52:0/1021343700 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4310041020 con 0x7f4318101830 2024-03-04T09:28:52.564 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:52.565+0000 7f431cabb640 1 -- 172.21.15.52:0/1021343700 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f431819e630 con 0x7f4318101830 2024-03-04T09:28:52.564 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:52.565+0000 7f430f7fe640 1 -- 172.21.15.52:0/1021343700 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4310034d90 con 0x7f4318101830 2024-03-04T09:28:52.565 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:52.565+0000 7f430f7fe640 1 -- 172.21.15.52:0/1021343700 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4310039040 con 0x7f4318101830 2024-03-04T09:28:52.565 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:52.565+0000 7f431cabb640 1 -- 172.21.15.52:0/1021343700 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f431819eb70 con 0x7f4318101830 2024-03-04T09:28:52.565 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:52.566+0000 7f431cabb640 1 -- 172.21.15.52:0/1021343700 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4318106f40 con 0x7f4318101830 2024-03-04T09:28:52.566 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:52.567+0000 7f430f7fe640 1 -- 172.21.15.52:0/1021343700 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 16) v1 ==== 136756+0+0 (secure 0 0 0) 0x7f4310044020 con 0x7f4318101830 2024-03-04T09:28:52.566 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:52.568+0000 7f430f7fe640 1 --2- 172.21.15.52:0/1021343700 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f43040a1d80 0x7f43040a4240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:28:52.566 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:52.568+0000 7f431659c640 1 --2- 172.21.15.52:0/1021343700 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f43040a1d80 0x7f43040a4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:28:52.566 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:52.568+0000 7f430f7fe640 1 -- 172.21.15.52:0/1021343700 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8802+0+0 (secure 0 0 0) 0x7f43100f0390 con 0x7f4318101830 2024-03-04T09:28:52.566 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:52.568+0000 7f431659c640 1 --2- 172.21.15.52:0/1021343700 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f43040a1d80 0x7f43040a4240 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f4308005fd0 tx=0x7f4308005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:28:52.568 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:52.571+0000 7f430f7fe640 1 -- 172.21.15.52:0/1021343700 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f431004b020 con 0x7f4318101830 2024-03-04T09:28:52.750 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:52.753+0000 7f431cabb640 1 -- 172.21.15.52:0/1021343700 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f43180621d0 con 0x7f4318101830 2024-03-04T09:28:52.750 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:52.753+0000 7f430f7fe640 1 -- 172.21.15.52:0/1021343700 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v89) v1 ==== 104+0+11 (secure 0 0 0) 0x7f43100bd580 con 0x7f4318101830 2024-03-04T09:28:52.750 INFO:teuthology.orchestra.run.smithi052.stdout:pg_num: 32 2024-03-04T09:28:52.752 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:52.755+0000 7f431cabb640 1 -- 172.21.15.52:0/1021343700 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f43040a1d80 msgr2=0x7f43040a4240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:28:52.753 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:52.755+0000 7f431cabb640 1 --2- 172.21.15.52:0/1021343700 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f43040a1d80 0x7f43040a4240 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f4308005fd0 tx=0x7f4308005850 comp rx=0 tx=0).stop 2024-03-04T09:28:52.753 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:52.755+0000 7f431cabb640 1 -- 172.21.15.52:0/1021343700 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4318101830 msgr2=0x7f431819dd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:28:52.753 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:52.755+0000 7f431cabb640 1 --2- 172.21.15.52:0/1021343700 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4318101830 0x7f431819dd30 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f4310037040 tx=0x7f4310032f20 comp rx=0 tx=0).stop 2024-03-04T09:28:52.753 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:52.755+0000 7f431cabb640 1 -- 172.21.15.52:0/1021343700 shutdown_connections 2024-03-04T09:28:52.753 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:52.756+0000 7f431cabb640 1 --2- 172.21.15.52:0/1021343700 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f43040a1d80 0x7f43040a4240 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:52.753 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:52.756+0000 7f431cabb640 1 --2- 172.21.15.52:0/1021343700 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4318101830 0x7f431819dd30 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:52.753 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:52.756+0000 7f431cabb640 1 --2- 172.21.15.52:0/1021343700 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4318100e70 0x7f4318196cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:52.753 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:52.756+0000 7f431cabb640 1 --2- 172.21.15.52:0/1021343700 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f43180ffc80 0x7f4318196780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:52.753 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:52.756+0000 7f431cabb640 1 -- 172.21.15.52:0/1021343700 >> 172.21.15.52:0/1021343700 conn(0x7f43180fb1d0 msgr2=0x7f4318104110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:28:52.753 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:52.756+0000 7f431cabb640 1 -- 172.21.15.52:0/1021343700 shutdown_connections 2024-03-04T09:28:52.753 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:52.756+0000 7f431cabb640 1 -- 172.21.15.52:0/1021343700 wait complete. 2024-03-04T09:28:52.763 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-04T09:28:52.871 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:52.872+0000 7f33f4cb8640 1 -- 172.21.15.52:0/3810492638 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f33f01027d0 msgr2=0x7f33f0102c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:28:52.871 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:52.872+0000 7f33f4cb8640 1 --2- 172.21.15.52:0/3810492638 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f33f01027d0 0x7f33f0102c50 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f33e4009900 tx=0x7f33e402bc30 comp rx=0 tx=0).stop 2024-03-04T09:28:52.871 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:52.872+0000 7f33f4cb8640 1 -- 172.21.15.52:0/3810492638 shutdown_connections 2024-03-04T09:28:52.871 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:52.872+0000 7f33f4cb8640 1 --2- 172.21.15.52:0/3810492638 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f33f0103190 0x7f33f0109a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:52.871 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:52.872+0000 7f33f4cb8640 1 --2- 172.21.15.52:0/3810492638 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f33f01027d0 0x7f33f0102c50 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:52.872 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:52.872+0000 7f33f4cb8640 1 --2- 172.21.15.52:0/3810492638 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f33f01015e0 0x7f33f0101a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:52.872 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:52.872+0000 7f33f4cb8640 1 -- 172.21.15.52:0/3810492638 >> 172.21.15.52:0/3810492638 conn(0x7f33f00fcb70 msgr2=0x7f33f00fef90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:28:52.872 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:52.872+0000 7f33f4cb8640 1 -- 172.21.15.52:0/3810492638 shutdown_connections 2024-03-04T09:28:52.872 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:52.872+0000 7f33f4cb8640 1 -- 172.21.15.52:0/3810492638 wait complete. 2024-03-04T09:28:52.872 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:52.873+0000 7f33f4cb8640 1 Processor -- start 2024-03-04T09:28:52.872 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:52.873+0000 7f33f4cb8640 1 -- start start 2024-03-04T09:28:52.872 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:52.873+0000 7f33f4cb8640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f33f01015e0 0x7f33f0075380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:28:52.872 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:52.873+0000 7f33f4cb8640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f33f01027d0 0x7f33f00739c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:28:52.872 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:52.873+0000 7f33f4cb8640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f33f0103190 0x7f33f0073f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:28:52.872 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:52.873+0000 7f33f4cb8640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33f010d240 con 0x7f33f01027d0 2024-03-04T09:28:52.872 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:52.873+0000 7f33f4cb8640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33f010d0c0 con 0x7f33f0103190 2024-03-04T09:28:52.872 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:52.873+0000 7f33f4cb8640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33f010d3c0 con 0x7f33f01015e0 2024-03-04T09:28:52.873 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:52.873+0000 7f33edd9b640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f33f01027d0 0x7f33f00739c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:28:52.873 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:52.873+0000 7f33edd9b640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f33f01027d0 0x7f33f00739c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53602/0 (socket says 172.21.15.52:53602) 2024-03-04T09:28:52.873 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:52.873+0000 7f33edd9b640 1 -- 172.21.15.52:0/2008241723 learned_addr learned my addr 172.21.15.52:0/2008241723 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-04T09:28:52.873 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:52.873+0000 7f33ee59c640 1 --2- 172.21.15.52:0/2008241723 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f33f01015e0 0x7f33f0075380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:28:52.873 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:52.873+0000 7f33eed9d640 1 --2- 172.21.15.52:0/2008241723 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f33f0103190 0x7f33f0073f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:28:52.873 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:52.873+0000 7f33edd9b640 1 -- 172.21.15.52:0/2008241723 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f33f01015e0 msgr2=0x7f33f0075380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:28:52.873 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:52.873+0000 7f33edd9b640 1 --2- 172.21.15.52:0/2008241723 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f33f01015e0 0x7f33f0075380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:52.873 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:52.873+0000 7f33edd9b640 1 -- 172.21.15.52:0/2008241723 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f33f0103190 msgr2=0x7f33f0073f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:28:52.873 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:52.873+0000 7f33edd9b640 1 --2- 172.21.15.52:0/2008241723 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f33f0103190 0x7f33f0073f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:52.873 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:52.873+0000 7f33edd9b640 1 -- 172.21.15.52:0/2008241723 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33f0074790 con 0x7f33f01027d0 2024-03-04T09:28:52.873 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:52.874+0000 7f33edd9b640 1 --2- 172.21.15.52:0/2008241723 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f33f01027d0 0x7f33f00739c0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f33e4009230 tx=0x7f33e4005d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:28:52.873 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:52.874+0000 7f33df7fe640 1 -- 172.21.15.52:0/2008241723 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33e4041070 con 0x7f33f01027d0 2024-03-04T09:28:52.873 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:52.874+0000 7f33f4cb8640 1 -- 172.21.15.52:0/2008241723 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33f01a1a00 con 0x7f33f01027d0 2024-03-04T09:28:52.873 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:52.874+0000 7f33df7fe640 1 -- 172.21.15.52:0/2008241723 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f33e40044d0 con 0x7f33f01027d0 2024-03-04T09:28:52.874 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:52.874+0000 7f33df7fe640 1 -- 172.21.15.52:0/2008241723 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33e4034440 con 0x7f33f01027d0 2024-03-04T09:28:52.874 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:52.874+0000 7f33f4cb8640 1 -- 172.21.15.52:0/2008241723 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33f01a1fa0 con 0x7f33f01027d0 2024-03-04T09:28:52.874 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:52.875+0000 7f33f4cb8640 1 -- 172.21.15.52:0/2008241723 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f33f0108980 con 0x7f33f01027d0 2024-03-04T09:28:52.875 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:52.876+0000 7f33df7fe640 1 -- 172.21.15.52:0/2008241723 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 16) v1 ==== 136756+0+0 (secure 0 0 0) 0x7f33e4044070 con 0x7f33f01027d0 2024-03-04T09:28:52.875 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:52.876+0000 7f33df7fe640 1 --2- 172.21.15.52:0/2008241723 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f33d80a1d80 0x7f33d80a4240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:28:52.875 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:52.877+0000 7f33ee59c640 1 --2- 172.21.15.52:0/2008241723 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f33d80a1d80 0x7f33d80a4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:28:52.875 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:52.877+0000 7f33df7fe640 1 -- 172.21.15.52:0/2008241723 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8802+0+0 (secure 0 0 0) 0x7f33e40efbf0 con 0x7f33f01027d0 2024-03-04T09:28:52.875 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:52.877+0000 7f33ee59c640 1 --2- 172.21.15.52:0/2008241723 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f33d80a1d80 0x7f33d80a4240 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f33e00095f0 tx=0x7f33e0005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:28:52.878 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:52.881+0000 7f33df7fe640 1 -- 172.21.15.52:0/2008241723 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f33e40bc730 con 0x7f33f01027d0 2024-03-04T09:28:53.063 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:53.065+0000 7f33f4cb8640 1 -- 172.21.15.52:0/2008241723 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f33f00621d0 con 0x7f33f01027d0 2024-03-04T09:28:53.063 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:53.065+0000 7f33df7fe640 1 -- 172.21.15.52:0/2008241723 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v89) v1 ==== 100+0+11 (secure 0 0 0) 0x7f33e40c07d0 con 0x7f33f01027d0 2024-03-04T09:28:53.063 INFO:teuthology.orchestra.run.smithi052.stdout:pg_num: 64 2024-03-04T09:28:53.065 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:53.067+0000 7f33f4cb8640 1 -- 172.21.15.52:0/2008241723 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f33d80a1d80 msgr2=0x7f33d80a4240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:28:53.066 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:53.067+0000 7f33f4cb8640 1 --2- 172.21.15.52:0/2008241723 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f33d80a1d80 0x7f33d80a4240 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f33e00095f0 tx=0x7f33e0005f90 comp rx=0 tx=0).stop 2024-03-04T09:28:53.066 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:53.067+0000 7f33f4cb8640 1 -- 172.21.15.52:0/2008241723 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f33f01027d0 msgr2=0x7f33f00739c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:28:53.066 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:53.067+0000 7f33f4cb8640 1 --2- 172.21.15.52:0/2008241723 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f33f01027d0 0x7f33f00739c0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f33e4009230 tx=0x7f33e4005d50 comp rx=0 tx=0).stop 2024-03-04T09:28:53.066 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:53.067+0000 7f33f4cb8640 1 -- 172.21.15.52:0/2008241723 shutdown_connections 2024-03-04T09:28:53.066 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:53.067+0000 7f33f4cb8640 1 --2- 172.21.15.52:0/2008241723 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f33d80a1d80 0x7f33d80a4240 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:53.066 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:53.067+0000 7f33f4cb8640 1 --2- 172.21.15.52:0/2008241723 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f33f0103190 0x7f33f0073f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:53.066 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:53.067+0000 7f33f4cb8640 1 --2- 172.21.15.52:0/2008241723 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f33f01027d0 0x7f33f00739c0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:53.066 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:53.067+0000 7f33f4cb8640 1 --2- 172.21.15.52:0/2008241723 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f33f01015e0 0x7f33f0075380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:53.066 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:53.067+0000 7f33f4cb8640 1 -- 172.21.15.52:0/2008241723 >> 172.21.15.52:0/2008241723 conn(0x7f33f00fcb70 msgr2=0x7f33f0105b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:28:53.067 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:53.067+0000 7f33f4cb8640 1 -- 172.21.15.52:0/2008241723 shutdown_connections 2024-03-04T09:28:53.067 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:53.067+0000 7f33f4cb8640 1 -- 172.21.15.52:0/2008241723 wait complete. 2024-03-04T09:28:53.074 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:28:52 smithi052 ceph-mon[32473]: pgmap v295: 129 pgs: 129 active+clean; 581 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 321 B/s wr, 0 op/s 2024-03-04T09:28:53.074 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:28:52 smithi052 ceph-mon[32473]: from='client.? 172.21.15.52:0/1314992918' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-04T09:28:53.074 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:28:52 smithi052 ceph-mon[32473]: from='client.? 172.21.15.52:0/1484823310' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-04T09:28:53.074 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:28:52 smithi052 ceph-mon[32473]: from='client.? 172.21.15.52:0/1021343700' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-04T09:28:53.075 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-04T09:28:53.182 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:53.184+0000 7fadb4725640 1 -- 172.21.15.52:0/685498665 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fadac102090 msgr2=0x7fadac108920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:28:53.183 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:53.184+0000 7fadb4725640 1 --2- 172.21.15.52:0/685498665 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fadac102090 0x7fadac108920 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fada4009ca0 tx=0x7fada402c0f0 comp rx=0 tx=0).stop 2024-03-04T09:28:53.183 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:53.184+0000 7fadb4725640 1 -- 172.21.15.52:0/685498665 shutdown_connections 2024-03-04T09:28:53.183 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:53.184+0000 7fadb4725640 1 --2- 172.21.15.52:0/685498665 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fadac102090 0x7fadac108920 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:53.183 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:53.184+0000 7fadb4725640 1 --2- 172.21.15.52:0/685498665 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fadac1016d0 0x7fadac101b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:53.183 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:53.184+0000 7fadb4725640 1 --2- 172.21.15.52:0/685498665 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fadac1004e0 0x7fadac100900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:53.183 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:53.184+0000 7fadb4725640 1 -- 172.21.15.52:0/685498665 >> 172.21.15.52:0/685498665 conn(0x7fadac0fba70 msgr2=0x7fadac0fde90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:28:53.183 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:53.184+0000 7fadb4725640 1 -- 172.21.15.52:0/685498665 shutdown_connections 2024-03-04T09:28:53.183 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:53.184+0000 7fadb4725640 1 -- 172.21.15.52:0/685498665 wait complete. 2024-03-04T09:28:53.183 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:53.184+0000 7fadb4725640 1 Processor -- start 2024-03-04T09:28:53.184 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:53.184+0000 7fadb4725640 1 -- start start 2024-03-04T09:28:53.184 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:53.185+0000 7fadb4725640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fadac1004e0 0x7fadac06a750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:28:53.184 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:53.185+0000 7fadb4725640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fadac1016d0 0x7fadac19c5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:28:53.184 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:53.185+0000 7fadb4725640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fadac102090 0x7fadac19ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:28:53.184 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:53.185+0000 7fadb4725640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadac10c170 con 0x7fadac1016d0 2024-03-04T09:28:53.184 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:53.185+0000 7fadb4725640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadac10bff0 con 0x7fadac102090 2024-03-04T09:28:53.184 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:53.185+0000 7fadb4725640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadac10c2f0 con 0x7fadac1004e0 2024-03-04T09:28:53.184 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:53.185+0000 7fadb1cc0640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fadac1016d0 0x7fadac19c5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:28:53.184 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:53.185+0000 7fadb1cc0640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fadac1016d0 0x7fadac19c5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53612/0 (socket says 172.21.15.52:53612) 2024-03-04T09:28:53.184 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:53.185+0000 7fadb1cc0640 1 -- 172.21.15.52:0/1736715048 learned_addr learned my addr 172.21.15.52:0/1736715048 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-04T09:28:53.184 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:53.185+0000 7fadb2cc2640 1 --2- 172.21.15.52:0/1736715048 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fadac102090 0x7fadac19ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:28:53.185 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:53.185+0000 7fadb24c1640 1 --2- 172.21.15.52:0/1736715048 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fadac1004e0 0x7fadac06a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:28:53.185 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:53.185+0000 7fadb1cc0640 1 -- 172.21.15.52:0/1736715048 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fadac1004e0 msgr2=0x7fadac06a750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:28:53.185 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:53.185+0000 7fadb1cc0640 1 --2- 172.21.15.52:0/1736715048 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fadac1004e0 0x7fadac06a750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:53.185 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:53.185+0000 7fadb1cc0640 1 -- 172.21.15.52:0/1736715048 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fadac102090 msgr2=0x7fadac19ebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:28:53.185 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:53.185+0000 7fadb1cc0640 1 --2- 172.21.15.52:0/1736715048 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fadac102090 0x7fadac19ebf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:53.185 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:53.185+0000 7fadb1cc0640 1 -- 172.21.15.52:0/1736715048 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fadac19f4b0 con 0x7fadac1016d0 2024-03-04T09:28:53.185 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:53.185+0000 7fadb24c1640 1 --2- 172.21.15.52:0/1736715048 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fadac1004e0 0x7fadac06a750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-04T09:28:53.185 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:53.185+0000 7fadb1cc0640 1 --2- 172.21.15.52:0/1736715048 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fadac1016d0 0x7fadac19c5b0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fad9c00d950 tx=0x7fad9c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:28:53.185 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:53.185+0000 7fada37fe640 1 -- 172.21.15.52:0/1736715048 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad9c017030 con 0x7fadac1016d0 2024-03-04T09:28:53.185 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:53.185+0000 7fada37fe640 1 -- 172.21.15.52:0/1736715048 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fad9c004020 con 0x7fadac1016d0 2024-03-04T09:28:53.185 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:53.185+0000 7fadb4725640 1 -- 172.21.15.52:0/1736715048 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fadac19f7a0 con 0x7fadac1016d0 2024-03-04T09:28:53.185 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:53.185+0000 7fada37fe640 1 -- 172.21.15.52:0/1736715048 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad9c0047a0 con 0x7fadac1016d0 2024-03-04T09:28:53.185 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:53.185+0000 7fadb4725640 1 -- 172.21.15.52:0/1736715048 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fadac19fcf0 con 0x7fadac1016d0 2024-03-04T09:28:53.187 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:53.186+0000 7fadb4725640 1 -- 172.21.15.52:0/1736715048 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fadac107850 con 0x7fadac1016d0 2024-03-04T09:28:53.187 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:53.188+0000 7fada37fe640 1 -- 172.21.15.52:0/1736715048 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 16) v1 ==== 136756+0+0 (secure 0 0 0) 0x7fad9c0041d0 con 0x7fadac1016d0 2024-03-04T09:28:53.187 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:53.188+0000 7fada37fe640 1 --2- 172.21.15.52:0/1736715048 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7fad980a1d80 0x7fad980a4240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:28:53.187 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:53.188+0000 7fadb24c1640 1 --2- 172.21.15.52:0/1736715048 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7fad980a1d80 0x7fad980a4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:28:53.187 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:53.188+0000 7fada37fe640 1 -- 172.21.15.52:0/1736715048 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8802+0+0 (secure 0 0 0) 0x7fad9c0cda00 con 0x7fadac1016d0 2024-03-04T09:28:53.187 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:53.189+0000 7fadb24c1640 1 --2- 172.21.15.52:0/1736715048 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7fad980a1d80 0x7fad980a4240 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fadac101530 tx=0x7fada8005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:28:53.189 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:53.193+0000 7fada37fe640 1 -- 172.21.15.52:0/1736715048 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fad9c09abf0 con 0x7fadac1016d0 2024-03-04T09:28:53.256 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:28:52 smithi191 ceph-mon[36816]: pgmap v295: 129 pgs: 129 active+clean; 581 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 321 B/s wr, 0 op/s 2024-03-04T09:28:53.256 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:28:52 smithi191 ceph-mon[36816]: from='client.? 172.21.15.52:0/1314992918' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-04T09:28:53.256 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:28:52 smithi191 ceph-mon[36816]: from='client.? 172.21.15.52:0/1484823310' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-04T09:28:53.256 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:28:52 smithi191 ceph-mon[36816]: from='client.? 172.21.15.52:0/1021343700' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-04T09:28:53.371 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:53.374+0000 7fadb4725640 1 -- 172.21.15.52:0/1736715048 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fadac062200 con 0x7fadac1016d0 2024-03-04T09:28:53.371 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:53.374+0000 7fada37fe640 1 -- 172.21.15.52:0/1736715048 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v89) v1 ==== 103+0+11 (secure 0 0 0) 0x7fad9c09a4f0 con 0x7fadac1016d0 2024-03-04T09:28:53.371 INFO:teuthology.orchestra.run.smithi052.stdout:pg_num: 32 2024-03-04T09:28:53.373 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:53.376+0000 7fadb4725640 1 -- 172.21.15.52:0/1736715048 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7fad980a1d80 msgr2=0x7fad980a4240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:28:53.373 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:53.376+0000 7fadb4725640 1 --2- 172.21.15.52:0/1736715048 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7fad980a1d80 0x7fad980a4240 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fadac101530 tx=0x7fada8005f90 comp rx=0 tx=0).stop 2024-03-04T09:28:53.374 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:53.376+0000 7fadb4725640 1 -- 172.21.15.52:0/1736715048 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fadac1016d0 msgr2=0x7fadac19c5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:28:53.374 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:53.376+0000 7fadb4725640 1 --2- 172.21.15.52:0/1736715048 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fadac1016d0 0x7fadac19c5b0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fad9c00d950 tx=0x7fad9c00de10 comp rx=0 tx=0).stop 2024-03-04T09:28:53.374 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:53.376+0000 7fadb4725640 1 -- 172.21.15.52:0/1736715048 shutdown_connections 2024-03-04T09:28:53.374 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:53.376+0000 7fadb4725640 1 --2- 172.21.15.52:0/1736715048 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7fad980a1d80 0x7fad980a4240 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:53.374 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:53.376+0000 7fadb4725640 1 --2- 172.21.15.52:0/1736715048 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fadac102090 0x7fadac19ebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:53.374 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:53.376+0000 7fadb4725640 1 --2- 172.21.15.52:0/1736715048 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fadac1016d0 0x7fadac19c5b0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:53.374 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:53.376+0000 7fadb4725640 1 --2- 172.21.15.52:0/1736715048 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fadac1004e0 0x7fadac06a750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:53.374 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:53.376+0000 7fadb4725640 1 -- 172.21.15.52:0/1736715048 >> 172.21.15.52:0/1736715048 conn(0x7fadac0fba70 msgr2=0x7fadac104a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:28:53.374 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:53.376+0000 7fadb4725640 1 -- 172.21.15.52:0/1736715048 shutdown_connections 2024-03-04T09:28:53.374 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:53.376+0000 7fadb4725640 1 -- 172.21.15.52:0/1736715048 wait complete. 2024-03-04T09:28:53.381 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:28:52 smithi106 ceph-mon[36484]: pgmap v295: 129 pgs: 129 active+clean; 581 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 321 B/s wr, 0 op/s 2024-03-04T09:28:53.381 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:28:52 smithi106 ceph-mon[36484]: from='client.? 172.21.15.52:0/1314992918' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-04T09:28:53.381 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:28:52 smithi106 ceph-mon[36484]: from='client.? 172.21.15.52:0/1484823310' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-04T09:28:53.381 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:28:52 smithi106 ceph-mon[36484]: from='client.? 172.21.15.52:0/1021343700' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-04T09:28:53.384 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-04T09:28:53.490 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:53.491+0000 7fd8c3b7a640 1 -- 172.21.15.52:0/2366152917 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd8bc1020c0 msgr2=0x7fd8bc1044b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:28:53.491 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:53.491+0000 7fd8c3b7a640 1 --2- 172.21.15.52:0/2366152917 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd8bc1020c0 0x7fd8bc1044b0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fd8b8009860 tx=0x7fd8b802bb70 comp rx=0 tx=0).stop 2024-03-04T09:28:53.491 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:53.491+0000 7fd8c3b7a640 1 -- 172.21.15.52:0/2366152917 shutdown_connections 2024-03-04T09:28:53.492 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:53.491+0000 7fd8c3b7a640 1 --2- 172.21.15.52:0/2366152917 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd8bc107320 0x7fd8bc109710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:53.492 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:53.491+0000 7fd8c3b7a640 1 --2- 172.21.15.52:0/2366152917 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd8bc1049f0 0x7fd8bc106de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:53.492 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:53.491+0000 7fd8c3b7a640 1 --2- 172.21.15.52:0/2366152917 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd8bc1020c0 0x7fd8bc1044b0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:53.492 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:53.491+0000 7fd8c3b7a640 1 -- 172.21.15.52:0/2366152917 >> 172.21.15.52:0/2366152917 conn(0x7fd8bc0fba70 msgr2=0x7fd8bc0fde90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:28:53.492 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:53.491+0000 7fd8c3b7a640 1 -- 172.21.15.52:0/2366152917 shutdown_connections 2024-03-04T09:28:53.492 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:53.491+0000 7fd8c3b7a640 1 -- 172.21.15.52:0/2366152917 wait complete. 2024-03-04T09:28:53.492 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:53.492+0000 7fd8c3b7a640 1 Processor -- start 2024-03-04T09:28:53.492 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:53.492+0000 7fd8c3b7a640 1 -- start start 2024-03-04T09:28:53.492 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:53.492+0000 7fd8c3b7a640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd8bc1020c0 0x7fd8bc197960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:28:53.492 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:53.492+0000 7fd8c3b7a640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd8bc1049f0 0x7fd8bc197ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:28:53.492 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:53.492+0000 7fd8c3b7a640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd8bc107320 0x7fd8bc19ef10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:28:53.492 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:53.492+0000 7fd8c3b7a640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8bc10d250 con 0x7fd8bc107320 2024-03-04T09:28:53.492 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:53.492+0000 7fd8c3b7a640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8bc10d0d0 con 0x7fd8bc1049f0 2024-03-04T09:28:53.493 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:53.492+0000 7fd8c3b7a640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8bc10d3d0 con 0x7fd8bc1020c0 2024-03-04T09:28:53.493 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:53.492+0000 7fd8c1115640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd8bc1049f0 0x7fd8bc197ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:28:53.493 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:53.492+0000 7fd8c1916640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd8bc1020c0 0x7fd8bc197960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:28:53.493 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:53.492+0000 7fd8c2117640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd8bc107320 0x7fd8bc19ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:28:53.493 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:53.492+0000 7fd8c1115640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd8bc1049f0 0x7fd8bc197ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.52:47608/0 (socket says 172.21.15.52:47608) 2024-03-04T09:28:53.493 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:53.492+0000 7fd8c1916640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd8bc1020c0 0x7fd8bc197960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.52:36804/0 (socket says 172.21.15.52:36804) 2024-03-04T09:28:53.493 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:53.492+0000 7fd8c1115640 1 -- 172.21.15.52:0/1886398778 learned_addr learned my addr 172.21.15.52:0/1886398778 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-04T09:28:53.493 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:53.492+0000 7fd8c1115640 1 -- 172.21.15.52:0/1886398778 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd8bc1020c0 msgr2=0x7fd8bc197960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:28:53.493 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:53.492+0000 7fd8c1115640 1 --2- 172.21.15.52:0/1886398778 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd8bc1020c0 0x7fd8bc197960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:53.493 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:53.492+0000 7fd8c1115640 1 -- 172.21.15.52:0/1886398778 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd8bc107320 msgr2=0x7fd8bc19ef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:28:53.493 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:53.492+0000 7fd8c1115640 1 --2- 172.21.15.52:0/1886398778 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd8bc107320 0x7fd8bc19ef10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:53.493 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:53.492+0000 7fd8c1115640 1 -- 172.21.15.52:0/1886398778 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8bc19f580 con 0x7fd8bc1049f0 2024-03-04T09:28:53.493 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:53.492+0000 7fd8c2117640 1 --2- 172.21.15.52:0/1886398778 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd8bc107320 0x7fd8bc19ef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-04T09:28:53.493 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:53.492+0000 7fd8c1115640 1 --2- 172.21.15.52:0/1886398778 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd8bc1049f0 0x7fd8bc197ea0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fd8ac00d770 tx=0x7fd8ac00dc30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:28:53.494 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:53.493+0000 7fd8b2ffd640 1 -- 172.21.15.52:0/1886398778 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8ac017070 con 0x7fd8bc1049f0 2024-03-04T09:28:53.494 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:53.493+0000 7fd8c3b7a640 1 -- 172.21.15.52:0/1886398778 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8bc19f870 con 0x7fd8bc1049f0 2024-03-04T09:28:53.494 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:53.493+0000 7fd8b2ffd640 1 -- 172.21.15.52:0/1886398778 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd8ac009c30 con 0x7fd8bc1049f0 2024-03-04T09:28:53.494 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:53.493+0000 7fd8b2ffd640 1 -- 172.21.15.52:0/1886398778 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8ac00f620 con 0x7fd8bc1049f0 2024-03-04T09:28:53.494 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:53.493+0000 7fd8c3b7a640 1 -- 172.21.15.52:0/1886398778 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8bc19fdc0 con 0x7fd8bc1049f0 2024-03-04T09:28:53.494 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:53.494+0000 7fd8c3b7a640 1 -- 172.21.15.52:0/1886398778 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd8bc0fd560 con 0x7fd8bc1049f0 2024-03-04T09:28:53.495 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:53.495+0000 7fd8b2ffd640 1 -- 172.21.15.52:0/1886398778 <== mon.2 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 16) v1 ==== 136756+0+0 (secure 0 0 0) 0x7fd8ac01a070 con 0x7fd8bc1049f0 2024-03-04T09:28:53.495 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:53.495+0000 7fd8b2ffd640 1 --2- 172.21.15.52:0/1886398778 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7fd8a80a1d80 0x7fd8a80a4240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:28:53.495 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:53.495+0000 7fd8c1916640 1 --2- 172.21.15.52:0/1886398778 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7fd8a80a1d80 0x7fd8a80a4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:28:53.495 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:53.495+0000 7fd8b2ffd640 1 -- 172.21.15.52:0/1886398778 <== mon.2 v2:172.21.15.106:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8802+0+0 (secure 0 0 0) 0x7fd8ac0cda40 con 0x7fd8bc1049f0 2024-03-04T09:28:53.495 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:53.496+0000 7fd8c1916640 1 --2- 172.21.15.52:0/1886398778 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7fd8a80a1d80 0x7fd8a80a4240 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fd8b802c180 tx=0x7fd8b8034ad0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:28:53.496 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:53.500+0000 7fd8b2ffd640 1 -- 172.21.15.52:0/1886398778 <== mon.2 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fd8ac009de0 con 0x7fd8bc1049f0 2024-03-04T09:28:53.693 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:53.696+0000 7fd8c3b7a640 1 -- 172.21.15.52:0/1886398778 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fd8bc02cff0 con 0x7fd8bc1049f0 2024-03-04T09:28:53.694 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:53.697+0000 7fd8b2ffd640 1 -- 172.21.15.52:0/1886398778 <== mon.2 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v89) v1 ==== 81+0+590 (secure 0 0 0) 0x7fd8ac09a5e0 con 0x7fd8bc1049f0 2024-03-04T09:28:53.694 INFO:teuthology.orchestra.run.smithi052.stdout:172.21.15.52:0/4082645718 2024-03-05T09:21:39.401935+0000 2024-03-04T09:28:53.694 INFO:teuthology.orchestra.run.smithi052.stdout:172.21.15.52:0/1271949988 2024-03-05T09:21:39.401935+0000 2024-03-04T09:28:53.694 INFO:teuthology.orchestra.run.smithi052.stdout:172.21.15.52:6801/2379361333 2024-03-05T09:21:12.760938+0000 2024-03-04T09:28:53.694 INFO:teuthology.orchestra.run.smithi052.stdout:172.21.15.52:0/497075075 2024-03-05T09:21:12.760938+0000 2024-03-04T09:28:53.694 INFO:teuthology.orchestra.run.smithi052.stdout:172.21.15.52:6800/2379361333 2024-03-05T09:21:12.760938+0000 2024-03-04T09:28:53.694 INFO:teuthology.orchestra.run.smithi052.stdout:172.21.15.52:0/568242577 2024-03-05T09:21:12.760938+0000 2024-03-04T09:28:53.694 INFO:teuthology.orchestra.run.smithi052.stdout:172.21.15.52:6801/2425730636 2024-03-05T09:21:39.401935+0000 2024-03-04T09:28:53.694 INFO:teuthology.orchestra.run.smithi052.stdout:172.21.15.52:6800/2425730636 2024-03-05T09:21:39.401935+0000 2024-03-04T09:28:53.695 INFO:teuthology.orchestra.run.smithi052.stdout:172.21.15.52:0/2743084243 2024-03-05T09:21:39.401935+0000 2024-03-04T09:28:53.695 INFO:teuthology.orchestra.run.smithi052.stdout:172.21.15.52:0/1035878034 2024-03-05T09:21:12.760938+0000 2024-03-04T09:28:53.695 INFO:teuthology.orchestra.run.smithi052.stderr:listed 10 entries 2024-03-04T09:28:53.696 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:53.699+0000 7fd8c3b7a640 1 -- 172.21.15.52:0/1886398778 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7fd8a80a1d80 msgr2=0x7fd8a80a4240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:28:53.696 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:53.699+0000 7fd8c3b7a640 1 --2- 172.21.15.52:0/1886398778 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7fd8a80a1d80 0x7fd8a80a4240 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fd8b802c180 tx=0x7fd8b8034ad0 comp rx=0 tx=0).stop 2024-03-04T09:28:53.696 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:53.699+0000 7fd8c3b7a640 1 -- 172.21.15.52:0/1886398778 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd8bc1049f0 msgr2=0x7fd8bc197ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:28:53.696 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:53.699+0000 7fd8c3b7a640 1 --2- 172.21.15.52:0/1886398778 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd8bc1049f0 0x7fd8bc197ea0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fd8ac00d770 tx=0x7fd8ac00dc30 comp rx=0 tx=0).stop 2024-03-04T09:28:53.696 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:53.699+0000 7fd8c3b7a640 1 -- 172.21.15.52:0/1886398778 shutdown_connections 2024-03-04T09:28:53.696 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:53.699+0000 7fd8c3b7a640 1 --2- 172.21.15.52:0/1886398778 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7fd8a80a1d80 0x7fd8a80a4240 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:53.696 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:53.699+0000 7fd8c3b7a640 1 --2- 172.21.15.52:0/1886398778 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd8bc107320 0x7fd8bc19ef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:53.696 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:53.699+0000 7fd8c3b7a640 1 --2- 172.21.15.52:0/1886398778 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd8bc1049f0 0x7fd8bc197ea0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:53.696 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:53.699+0000 7fd8c3b7a640 1 --2- 172.21.15.52:0/1886398778 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd8bc1020c0 0x7fd8bc197960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:28:53.696 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:53.699+0000 7fd8c3b7a640 1 -- 172.21.15.52:0/1886398778 >> 172.21.15.52:0/1886398778 conn(0x7fd8bc0fba70 msgr2=0x7fd8bc102e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:28:53.696 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:53.699+0000 7fd8c3b7a640 1 -- 172.21.15.52:0/1886398778 shutdown_connections 2024-03-04T09:28:53.697 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:28:53.699+0000 7fd8c3b7a640 1 -- 172.21.15.52:0/1886398778 wait complete. 2024-03-04T09:28:53.707 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi052.front.sepia.ceph.com... 2024-03-04T09:28:53.707 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-04T09:28:53.707 DEBUG:teuthology.orchestra.run.smithi052:> sudo fusermount -u /home/ubuntu/cephtest/mnt.admin 2024-03-04T09:28:53.732 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi052.stderr:ceph-fuse[73880]: fuse finished with error 0 and tester_r 0 2024-03-04T09:28:53.733 INFO:teuthology.orchestra.run:waiting for 300 2024-03-04T09:28:54.085 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:28:53 smithi052 ceph-mon[32473]: from='client.? 172.21.15.52:0/2008241723' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-04T09:28:54.085 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:28:53 smithi052 ceph-mon[32473]: from='client.? 172.21.15.52:0/1736715048' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-04T09:28:54.085 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:28:53 smithi052 ceph-mon[32473]: from='client.? 172.21.15.52:0/1886398778' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-03-04T09:28:54.336 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:28:53 smithi191 ceph-mon[36816]: from='client.? 172.21.15.52:0/2008241723' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-04T09:28:54.336 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:28:53 smithi191 ceph-mon[36816]: from='client.? 172.21.15.52:0/1736715048' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-04T09:28:54.336 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:28:53 smithi191 ceph-mon[36816]: from='client.? 172.21.15.52:0/1886398778' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-03-04T09:28:54.381 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:28:53 smithi106 ceph-mon[36484]: from='client.? 172.21.15.52:0/2008241723' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-04T09:28:54.381 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:28:53 smithi106 ceph-mon[36484]: from='client.? 172.21.15.52:0/1736715048' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-04T09:28:54.381 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:28:53 smithi106 ceph-mon[36484]: from='client.? 172.21.15.52:0/1886398778' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-03-04T09:28:55.335 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:28:54 smithi052 ceph-mon[32473]: pgmap v296: 129 pgs: 129 active+clean; 581 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 291 B/s wr, 0 op/s 2024-03-04T09:28:55.336 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:28:54 smithi191 ceph-mon[36816]: pgmap v296: 129 pgs: 129 active+clean; 581 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 291 B/s wr, 0 op/s 2024-03-04T09:28:55.381 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:28:54 smithi106 ceph-mon[36484]: pgmap v296: 129 pgs: 129 active+clean; 581 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 291 B/s wr, 0 op/s 2024-03-04T09:28:57.335 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:28:56 smithi052 ceph-mon[32473]: pgmap v297: 129 pgs: 129 active+clean; 581 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-03-04T09:28:57.336 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:28:56 smithi191 ceph-mon[36816]: pgmap v297: 129 pgs: 129 active+clean; 581 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-03-04T09:28:57.381 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:28:56 smithi106 ceph-mon[36484]: pgmap v297: 129 pgs: 129 active+clean; 581 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-03-04T09:28:59.335 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:28:58 smithi052 ceph-mon[32473]: pgmap v298: 129 pgs: 129 active+clean; 581 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-03-04T09:28:59.336 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:28:58 smithi191 ceph-mon[36816]: pgmap v298: 129 pgs: 129 active+clean; 581 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-03-04T09:28:59.381 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:28:58 smithi106 ceph-mon[36484]: pgmap v298: 129 pgs: 129 active+clean; 581 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-03-04T09:28:59.834 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi052.front.sepia.ceph.com 2024-03-04T09:28:59.835 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-04T09:28:59.835 DEBUG:teuthology.orchestra.run.smithi052:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.admin) 2024-03-04T09:28:59.863 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-04T09:29:00.020 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:00.021+0000 7f03d4ff7640 1 -- 172.21.15.52:0/2501381309 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f03d0100570 msgr2=0x7f03d0100990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:29:00.021 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:00.021+0000 7f03d4ff7640 1 --2- 172.21.15.52:0/2501381309 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f03d0100570 0x7f03d0100990 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f03c0009900 tx=0x7f03c002bc30 comp rx=0 tx=0).stop 2024-03-04T09:29:00.021 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:00.021+0000 7f03d4ff7640 1 -- 172.21.15.52:0/2501381309 shutdown_connections 2024-03-04T09:29:00.021 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:00.021+0000 7f03d4ff7640 1 --2- 172.21.15.52:0/2501381309 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f03d0102120 0x7f03d01089b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:29:00.021 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:00.021+0000 7f03d4ff7640 1 --2- 172.21.15.52:0/2501381309 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f03d0101760 0x7f03d0101be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:29:00.021 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:00.021+0000 7f03d4ff7640 1 --2- 172.21.15.52:0/2501381309 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f03d0100570 0x7f03d0100990 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:29:00.021 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:00.021+0000 7f03d4ff7640 1 -- 172.21.15.52:0/2501381309 >> 172.21.15.52:0/2501381309 conn(0x7f03d00fbb00 msgr2=0x7f03d00fdf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:29:00.021 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:00.021+0000 7f03d4ff7640 1 -- 172.21.15.52:0/2501381309 shutdown_connections 2024-03-04T09:29:00.021 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:00.021+0000 7f03d4ff7640 1 -- 172.21.15.52:0/2501381309 wait complete. 2024-03-04T09:29:00.021 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:00.022+0000 7f03d4ff7640 1 Processor -- start 2024-03-04T09:29:00.021 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:00.022+0000 7f03d4ff7640 1 -- start start 2024-03-04T09:29:00.021 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:00.022+0000 7f03d4ff7640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f03d0100570 0x7f03d006a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:29:00.021 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:00.022+0000 7f03d4ff7640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f03d0101760 0x7f03d019c610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:29:00.021 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:00.022+0000 7f03d4ff7640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f03d0102120 0x7f03d019ec80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:29:00.022 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:00.022+0000 7f03d4ff7640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03d010c190 con 0x7f03d0102120 2024-03-04T09:29:00.022 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:00.022+0000 7f03d4ff7640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03d010c010 con 0x7f03d0100570 2024-03-04T09:29:00.022 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:00.022+0000 7f03d4ff7640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03d010c310 con 0x7f03d0101760 2024-03-04T09:29:00.022 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:00.022+0000 7f03ced9d640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f03d0102120 0x7f03d019ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:29:00.022 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:00.022+0000 7f03ced9d640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f03d0102120 0x7f03d019ec80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:52762/0 (socket says 172.21.15.52:52762) 2024-03-04T09:29:00.022 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:00.022+0000 7f03ced9d640 1 -- 172.21.15.52:0/780665599 learned_addr learned my addr 172.21.15.52:0/780665599 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-04T09:29:00.022 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:00.022+0000 7f03ce59c640 1 --2- 172.21.15.52:0/780665599 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f03d0100570 0x7f03d006a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:29:00.022 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:00.023+0000 7f03c7fff640 1 --2- 172.21.15.52:0/780665599 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f03d0101760 0x7f03d019c610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:29:00.022 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:00.023+0000 7f03ced9d640 1 -- 172.21.15.52:0/780665599 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f03d0101760 msgr2=0x7f03d019c610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:29:00.022 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:00.023+0000 7f03ced9d640 1 --2- 172.21.15.52:0/780665599 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f03d0101760 0x7f03d019c610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:29:00.022 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:00.023+0000 7f03ced9d640 1 -- 172.21.15.52:0/780665599 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f03d0100570 msgr2=0x7f03d006a7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:29:00.022 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:00.023+0000 7f03ced9d640 1 --2- 172.21.15.52:0/780665599 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f03d0100570 0x7f03d006a7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:29:00.022 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:00.023+0000 7f03ced9d640 1 -- 172.21.15.52:0/780665599 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03d019f540 con 0x7f03d0102120 2024-03-04T09:29:00.022 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:00.023+0000 7f03c7fff640 1 --2- 172.21.15.52:0/780665599 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f03d0101760 0x7f03d019c610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-04T09:29:00.023 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:00.023+0000 7f03ced9d640 1 --2- 172.21.15.52:0/780665599 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f03d0102120 0x7f03d019ec80 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f03c800eb10 tx=0x7f03c800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:29:00.023 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:00.023+0000 7f03c77fe640 1 -- 172.21.15.52:0/780665599 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03c8004b20 con 0x7f03d0102120 2024-03-04T09:29:00.023 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:00.023+0000 7f03c77fe640 1 -- 172.21.15.52:0/780665599 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f03c8004cc0 con 0x7f03d0102120 2024-03-04T09:29:00.023 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:00.023+0000 7f03d4ff7640 1 -- 172.21.15.52:0/780665599 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03d019f830 con 0x7f03d0102120 2024-03-04T09:29:00.023 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:00.023+0000 7f03c77fe640 1 -- 172.21.15.52:0/780665599 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03c8010760 con 0x7f03d0102120 2024-03-04T09:29:00.023 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:00.023+0000 7f03d4ff7640 1 -- 172.21.15.52:0/780665599 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03d019fd80 con 0x7f03d0102120 2024-03-04T09:29:00.024 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:00.024+0000 7f03d4ff7640 1 -- 172.21.15.52:0/780665599 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f03d0107910 con 0x7f03d0102120 2024-03-04T09:29:00.024 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:00.025+0000 7f03c77fe640 1 -- 172.21.15.52:0/780665599 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 16) v1 ==== 136756+0+0 (secure 0 0 0) 0x7f03c8005000 con 0x7f03d0102120 2024-03-04T09:29:00.024 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:00.026+0000 7f03c77fe640 1 --2- 172.21.15.52:0/780665599 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f03bc0a1e00 0x7f03bc0a42c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:29:00.024 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:00.026+0000 7f03ce59c640 1 --2- 172.21.15.52:0/780665599 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f03bc0a1e00 0x7f03bc0a42c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:29:00.024 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:00.026+0000 7f03c77fe640 1 -- 172.21.15.52:0/780665599 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8802+0+0 (secure 0 0 0) 0x7f03c80ce7f0 con 0x7f03d0102120 2024-03-04T09:29:00.024 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:00.026+0000 7f03ce59c640 1 --2- 172.21.15.52:0/780665599 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f03bc0a1e00 0x7f03bc0a42c0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f03c0000c00 tx=0x7f03c0005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:29:00.026 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:00.029+0000 7f03c77fe640 1 -- 172.21.15.52:0/780665599 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f03c8053280 con 0x7f03d0102120 2024-03-04T09:29:00.209 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:00.213+0000 7f03d4ff7640 1 -- 172.21.15.52:0/780665599 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f03d00621d0 con 0x7f03d0102120 2024-03-04T09:29:00.211 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:00.214+0000 7f03c77fe640 1 -- 172.21.15.52:0/780665599 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) v1 ==== 74+0+24164 (secure 0 0 0) 0x7f03c809b980 con 0x7f03d0102120 2024-03-04T09:29:00.211 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-03-04T09:29:00.212 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":89,"fsid":"4e059f08-da08-11ee-95c6-87774f69a715","created":"2024-03-04T09:20:46.104963+0000","modified":"2024-03-04T09:28:42.295274+0000","last_up_change":"2024-03-04T09:27:49.261365+0000","last_in_change":"2024-03-04T09:27:30.024964+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-04T09:24:01.742461+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-04T09:28:35.835600+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-04T09:28:36.453099+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-04T09:28:40.168831+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"583172be-602f-4bf4-bfcb-42663cf5b277","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6802","nonce":1371221397},{"type":"v1","addr":"172.21.15.52:6803","nonce":1371221397}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6804","nonce":1371221397},{"type":"v1","addr":"172.21.15.52:6805","nonce":1371221397}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6808","nonce":1371221397},{"type":"v1","addr":"172.21.15.52:6809","nonce":1371221397}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6806","nonce":1371221397},{"type":"v1","addr":"172.21.15.52:6807","nonce":1371221397}]},"public_addr":"172.21.15.52:6803/1371221397","cluster_addr":"172.21.15.52:6805/1371221397","heartbeat_back_addr":"172.21.15.52:6809/1371221397","heartbeat_front_addr":"172.21.15.52:6807/1371221397","state":["exists","up"]},{"osd":1,"uuid":"5ae0081c-a2f6-46eb-ace0-0368208377b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":2540865537},{"type":"v1","addr":"172.21.15.106:6801","nonce":2540865537}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":2540865537},{"type":"v1","addr":"172.21.15.106:6803","nonce":2540865537}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":2540865537},{"type":"v1","addr":"172.21.15.106:6807","nonce":2540865537}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":2540865537},{"type":"v1","addr":"172.21.15.106:6805","nonce":2540865537}]},"public_addr":"172.21.15.106:6801/2540865537","cluster_addr":"172.21.15.106:6803/2540865537","heartbeat_back_addr":"172.21.15.106:6807/2540865537","heartbeat_front_addr":"172.21.15.106:6805/2540865537","state":["exists","up"]},{"osd":2,"uuid":"1948bac7-2bb2-4218-8b93-4e4743a0de04","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":1377083164},{"type":"v1","addr":"172.21.15.191:6801","nonce":1377083164}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":1377083164},{"type":"v1","addr":"172.21.15.191:6803","nonce":1377083164}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":1377083164},{"type":"v1","addr":"172.21.15.191:6807","nonce":1377083164}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":1377083164},{"type":"v1","addr":"172.21.15.191:6805","nonce":1377083164}]},"public_addr":"172.21.15.191:6801/1377083164","cluster_addr":"172.21.15.191:6803/1377083164","heartbeat_back_addr":"172.21.15.191:6807/1377083164","heartbeat_front_addr":"172.21.15.191:6805/1377083164","state":["exists","up"]},{"osd":3,"uuid":"16c9e9fe-7167-4992-bf12-1a7398e0498a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6810","nonce":2388571810},{"type":"v1","addr":"172.21.15.52:6811","nonce":2388571810}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6812","nonce":2388571810},{"type":"v1","addr":"172.21.15.52:6813","nonce":2388571810}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6816","nonce":2388571810},{"type":"v1","addr":"172.21.15.52:6817","nonce":2388571810}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6814","nonce":2388571810},{"type":"v1","addr":"172.21.15.52:6815","nonce":2388571810}]},"public_addr":"172.21.15.52:6811/2388571810","cluster_addr":"172.21.15.52:6813/2388571810","heartbeat_back_addr":"172.21.15.52:6817/2388571810","heartbeat_front_addr":"172.21.15.52:6815/2388571810","state":["exists","up"]},{"osd":4,"uuid":"c8ef48c7-7c78-44d3-a9e5-99ec750abd3d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6808","nonce":3465868015},{"type":"v1","addr":"172.21.15.106:6809","nonce":3465868015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6810","nonce":3465868015},{"type":"v1","addr":"172.21.15.106:6811","nonce":3465868015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6814","nonce":3465868015},{"type":"v1","addr":"172.21.15.106:6815","nonce":3465868015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6812","nonce":3465868015},{"type":"v1","addr":"172.21.15.106:6813","nonce":3465868015}]},"public_addr":"172.21.15.106:6809/3465868015","cluster_addr":"172.21.15.106:6811/3465868015","heartbeat_back_addr":"172.21.15.106:6815/3465868015","heartbeat_front_addr":"172.21.15.106:6813/3465868015","state":["exists","up"]},{"osd":5,"uuid":"9e41824a-c5a2-4f88-be50-2ba1dceb3220","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":1485092643},{"type":"v1","addr":"172.21.15.191:6809","nonce":1485092643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":1485092643},{"type":"v1","addr":"172.21.15.191:6811","nonce":1485092643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":1485092643},{"type":"v1","addr":"172.21.15.191:6815","nonce":1485092643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":1485092643},{"type":"v1","addr":"172.21.15.191:6813","nonce":1485092643}]},"public_addr":"172.21.15.191:6809/1485092643","cluster_addr":"172.21.15.191:6811/1485092643","heartbeat_back_addr":"172.21.15.191:6815/1485092643","heartbeat_front_addr":"172.21.15.191:6813/1485092643","state":["exists","up"]},{"osd":6,"uuid":"b35dccee-ec16-452e-ac69-3ac2fb7b27fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6818","nonce":1989797856},{"type":"v1","addr":"172.21.15.52:6819","nonce":1989797856}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6820","nonce":1989797856},{"type":"v1","addr":"172.21.15.52:6821","nonce":1989797856}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6824","nonce":1989797856},{"type":"v1","addr":"172.21.15.52:6825","nonce":1989797856}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6822","nonce":1989797856},{"type":"v1","addr":"172.21.15.52:6823","nonce":1989797856}]},"public_addr":"172.21.15.52:6819/1989797856","cluster_addr":"172.21.15.52:6821/1989797856","heartbeat_back_addr":"172.21.15.52:6825/1989797856","heartbeat_front_addr":"172.21.15.52:6823/1989797856","state":["exists","up"]},{"osd":7,"uuid":"71303e3d-14ec-4cbd-b19c-67645a1de3c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6816","nonce":1450870329},{"type":"v1","addr":"172.21.15.106:6817","nonce":1450870329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":1450870329},{"type":"v1","addr":"172.21.15.106:6819","nonce":1450870329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":1450870329},{"type":"v1","addr":"172.21.15.106:6823","nonce":1450870329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6820","nonce":1450870329},{"type":"v1","addr":"172.21.15.106:6821","nonce":1450870329}]},"public_addr":"172.21.15.106:6817/1450870329","cluster_addr":"172.21.15.106:6819/1450870329","heartbeat_back_addr":"172.21.15.106:6823/1450870329","heartbeat_front_addr":"172.21.15.106:6821/1450870329","state":["exists","up"]},{"osd":8,"uuid":"6f5786c4-4402-435a-bdad-adff46fe9473","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":1114119072},{"type":"v1","addr":"172.21.15.191:6817","nonce":1114119072}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6818","nonce":1114119072},{"type":"v1","addr":"172.21.15.191:6819","nonce":1114119072}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":1114119072},{"type":"v1","addr":"172.21.15.191:6823","nonce":1114119072}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":1114119072},{"type":"v1","addr":"172.21.15.191:6821","nonce":1114119072}]},"public_addr":"172.21.15.191:6817/1114119072","cluster_addr":"172.21.15.191:6819/1114119072","heartbeat_back_addr":"172.21.15.191:6823/1114119072","heartbeat_front_addr":"172.21.15.191:6821/1114119072","state":["exists","up"]},{"osd":9,"uuid":"062bbaa5-1227-4684-b859-450f5594a90c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6826","nonce":3485821914},{"type":"v1","addr":"172.21.15.52:6827","nonce":3485821914}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6828","nonce":3485821914},{"type":"v1","addr":"172.21.15.52:6829","nonce":3485821914}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6832","nonce":3485821914},{"type":"v1","addr":"172.21.15.52:6833","nonce":3485821914}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6830","nonce":3485821914},{"type":"v1","addr":"172.21.15.52:6831","nonce":3485821914}]},"public_addr":"172.21.15.52:6827/3485821914","cluster_addr":"172.21.15.52:6829/3485821914","heartbeat_back_addr":"172.21.15.52:6833/3485821914","heartbeat_front_addr":"172.21.15.52:6831/3485821914","state":["exists","up"]},{"osd":10,"uuid":"1a285a5a-f793-48da-a8eb-5326d9528cd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":3815586576},{"type":"v1","addr":"172.21.15.106:6825","nonce":3815586576}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6826","nonce":3815586576},{"type":"v1","addr":"172.21.15.106:6827","nonce":3815586576}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":3815586576},{"type":"v1","addr":"172.21.15.106:6831","nonce":3815586576}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":3815586576},{"type":"v1","addr":"172.21.15.106:6829","nonce":3815586576}]},"public_addr":"172.21.15.106:6825/3815586576","cluster_addr":"172.21.15.106:6827/3815586576","heartbeat_back_addr":"172.21.15.106:6831/3815586576","heartbeat_front_addr":"172.21.15.106:6829/3815586576","state":["exists","up"]},{"osd":11,"uuid":"1f8b9a43-57fa-41f4-9969-3b803a74bf45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":2009632843},{"type":"v1","addr":"172.21.15.191:6825","nonce":2009632843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":2009632843},{"type":"v1","addr":"172.21.15.191:6827","nonce":2009632843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":2009632843},{"type":"v1","addr":"172.21.15.191:6831","nonce":2009632843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":2009632843},{"type":"v1","addr":"172.21.15.191:6829","nonce":2009632843}]},"public_addr":"172.21.15.191:6825/2009632843","cluster_addr":"172.21.15.191:6827/2009632843","heartbeat_back_addr":"172.21.15.191:6831/2009632843","heartbeat_front_addr":"172.21.15.191:6829/2009632843","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T09:23:10.894763+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T09:23:34.420055+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T09:23:58.288324+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T09:24:23.673457+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T09:24:48.208561+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T09:25:13.405473+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T09:25:38.670915+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T09:26:04.451257+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T09:26:30.281380+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T09:26:56.543143+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T09:27:22.412930+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T09:27:48.003866+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.52:0/4082645718":"2024-03-05T09:21:39.401935+0000","172.21.15.52:0/1271949988":"2024-03-05T09:21:39.401935+0000","172.21.15.52:6801/2379361333":"2024-03-05T09:21:12.760938+0000","172.21.15.52:0/497075075":"2024-03-05T09:21:12.760938+0000","172.21.15.52:6800/2379361333":"2024-03-05T09:21:12.760938+0000","172.21.15.52:0/568242577":"2024-03-05T09:21:12.760938+0000","172.21.15.52:6801/2425730636":"2024-03-05T09:21:39.401935+0000","172.21.15.52:6800/2425730636":"2024-03-05T09:21:39.401935+0000","172.21.15.52:0/2743084243":"2024-03-05T09:21:39.401935+0000","172.21.15.52:0/1035878034":"2024-03-05T09:21:12.760938+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-04T09:29:00.213 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:00.216+0000 7f03d4ff7640 1 -- 172.21.15.52:0/780665599 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f03bc0a1e00 msgr2=0x7f03bc0a42c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:29:00.213 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:00.216+0000 7f03d4ff7640 1 --2- 172.21.15.52:0/780665599 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f03bc0a1e00 0x7f03bc0a42c0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f03c0000c00 tx=0x7f03c0005fd0 comp rx=0 tx=0).stop 2024-03-04T09:29:00.214 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:00.216+0000 7f03d4ff7640 1 -- 172.21.15.52:0/780665599 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f03d0102120 msgr2=0x7f03d019ec80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:29:00.214 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:00.216+0000 7f03d4ff7640 1 --2- 172.21.15.52:0/780665599 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f03d0102120 0x7f03d019ec80 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f03c800eb10 tx=0x7f03c800c2d0 comp rx=0 tx=0).stop 2024-03-04T09:29:00.214 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:00.216+0000 7f03d4ff7640 1 -- 172.21.15.52:0/780665599 shutdown_connections 2024-03-04T09:29:00.214 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:00.216+0000 7f03d4ff7640 1 --2- 172.21.15.52:0/780665599 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f03bc0a1e00 0x7f03bc0a42c0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:29:00.214 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:00.216+0000 7f03d4ff7640 1 --2- 172.21.15.52:0/780665599 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f03d0102120 0x7f03d019ec80 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:29:00.214 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:00.216+0000 7f03d4ff7640 1 --2- 172.21.15.52:0/780665599 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f03d0101760 0x7f03d019c610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:29:00.214 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:00.216+0000 7f03d4ff7640 1 --2- 172.21.15.52:0/780665599 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f03d0100570 0x7f03d006a7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:29:00.214 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:00.216+0000 7f03d4ff7640 1 -- 172.21.15.52:0/780665599 >> 172.21.15.52:0/780665599 conn(0x7f03d00fbb00 msgr2=0x7f03d0104ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:29:00.214 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:00.216+0000 7f03d4ff7640 1 -- 172.21.15.52:0/780665599 shutdown_connections 2024-03-04T09:29:00.214 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:00.216+0000 7f03d4ff7640 1 -- 172.21.15.52:0/780665599 wait complete. 2024-03-04T09:29:00.225 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-04T09:29:00.329 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:00.330+0000 7f7f9c7d0640 1 -- 172.21.15.52:0/2538472373 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7f941020c0 msgr2=0x7f7f94108950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:29:00.329 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:00.330+0000 7f7f9c7d0640 1 --2- 172.21.15.52:0/2538472373 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7f941020c0 0x7f7f94108950 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f7f8c009ca0 tx=0x7f7f8c02c0f0 comp rx=0 tx=0).stop 2024-03-04T09:29:00.329 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:00.330+0000 7f7f9c7d0640 1 -- 172.21.15.52:0/2538472373 shutdown_connections 2024-03-04T09:29:00.329 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:00.330+0000 7f7f9c7d0640 1 --2- 172.21.15.52:0/2538472373 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7f941020c0 0x7f7f94108950 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:29:00.330 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:00.330+0000 7f7f9c7d0640 1 --2- 172.21.15.52:0/2538472373 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7f94101700 0x7f7f94101b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:29:00.330 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:00.330+0000 7f7f9c7d0640 1 --2- 172.21.15.52:0/2538472373 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7f94100510 0x7f7f94100930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:29:00.330 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:00.330+0000 7f7f9c7d0640 1 -- 172.21.15.52:0/2538472373 >> 172.21.15.52:0/2538472373 conn(0x7f7f940fba80 msgr2=0x7f7f940fdec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:29:00.330 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:00.330+0000 7f7f9c7d0640 1 -- 172.21.15.52:0/2538472373 shutdown_connections 2024-03-04T09:29:00.330 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:00.330+0000 7f7f9c7d0640 1 -- 172.21.15.52:0/2538472373 wait complete. 2024-03-04T09:29:00.330 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:00.331+0000 7f7f9c7d0640 1 Processor -- start 2024-03-04T09:29:00.330 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:00.331+0000 7f7f9c7d0640 1 -- start start 2024-03-04T09:29:00.330 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:00.331+0000 7f7f9c7d0640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7f94100510 0x7f7f941044a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:29:00.330 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:00.331+0000 7f7f9c7d0640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7f94101700 0x7f7f94102ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:29:00.330 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:00.331+0000 7f7f9c7d0640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7f941020c0 0x7f7f94103020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:29:00.330 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:00.331+0000 7f7f9c7d0640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f9410c1a0 con 0x7f7f94100510 2024-03-04T09:29:00.330 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:00.331+0000 7f7f9c7d0640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f9410c020 con 0x7f7f941020c0 2024-03-04T09:29:00.330 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:00.331+0000 7f7f9c7d0640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f9410c320 con 0x7f7f94101700 2024-03-04T09:29:00.330 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:00.331+0000 7f7f9a56c640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7f94100510 0x7f7f941044a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:29:00.331 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:00.331+0000 7f7f9a56c640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7f94100510 0x7f7f941044a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:52778/0 (socket says 172.21.15.52:52778) 2024-03-04T09:29:00.331 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:00.331+0000 7f7f9a56c640 1 -- 172.21.15.52:0/2708909486 learned_addr learned my addr 172.21.15.52:0/2708909486 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-04T09:29:00.331 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:00.331+0000 7f7f9ad6d640 1 --2- 172.21.15.52:0/2708909486 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7f941020c0 0x7f7f94103020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:29:00.331 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:00.331+0000 7f7f99d6b640 1 --2- 172.21.15.52:0/2708909486 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7f94101700 0x7f7f94102ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:29:00.331 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:00.331+0000 7f7f9a56c640 1 -- 172.21.15.52:0/2708909486 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7f94101700 msgr2=0x7f7f94102ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:29:00.331 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:00.331+0000 7f7f9a56c640 1 --2- 172.21.15.52:0/2708909486 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7f94101700 0x7f7f94102ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:29:00.331 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:00.331+0000 7f7f9a56c640 1 -- 172.21.15.52:0/2708909486 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7f941020c0 msgr2=0x7f7f94103020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:29:00.331 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:00.331+0000 7f7f9a56c640 1 --2- 172.21.15.52:0/2708909486 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7f941020c0 0x7f7f94103020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:29:00.331 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:00.331+0000 7f7f9a56c640 1 -- 172.21.15.52:0/2708909486 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f941038b0 con 0x7f7f94100510 2024-03-04T09:29:00.331 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:00.331+0000 7f7f99d6b640 1 --2- 172.21.15.52:0/2708909486 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7f94101700 0x7f7f94102ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T09:29:00.331 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:00.331+0000 7f7f9ad6d640 1 --2- 172.21.15.52:0/2708909486 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7f941020c0 0x7f7f94103020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T09:29:00.331 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:00.331+0000 7f7f9a56c640 1 --2- 172.21.15.52:0/2708909486 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7f94100510 0x7f7f941044a0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f7f8400d770 tx=0x7f7f8400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:29:00.331 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:00.332+0000 7f7f8b7fe640 1 -- 172.21.15.52:0/2708909486 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f84017070 con 0x7f7f94100510 2024-03-04T09:29:00.331 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:00.332+0000 7f7f8b7fe640 1 -- 172.21.15.52:0/2708909486 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7f84009c30 con 0x7f7f94100510 2024-03-04T09:29:00.332 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:00.332+0000 7f7f9c7d0640 1 -- 172.21.15.52:0/2708909486 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f941a19e0 con 0x7f7f94100510 2024-03-04T09:29:00.332 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:00.332+0000 7f7f8b7fe640 1 -- 172.21.15.52:0/2708909486 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f84003dd0 con 0x7f7f94100510 2024-03-04T09:29:00.332 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:00.332+0000 7f7f9c7d0640 1 -- 172.21.15.52:0/2708909486 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f941a1f30 con 0x7f7f94100510 2024-03-04T09:29:00.332 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:00.333+0000 7f7f9c7d0640 1 -- 172.21.15.52:0/2708909486 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f94068810 con 0x7f7f94100510 2024-03-04T09:29:00.332 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:00.334+0000 7f7f8b7fe640 1 -- 172.21.15.52:0/2708909486 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 16) v1 ==== 136756+0+0 (secure 0 0 0) 0x7f7f84003fb0 con 0x7f7f94100510 2024-03-04T09:29:00.333 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:00.334+0000 7f7f8b7fe640 1 --2- 172.21.15.52:0/2708909486 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f7f800a1d80 0x7f7f800a4240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:29:00.333 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:00.334+0000 7f7f99d6b640 1 --2- 172.21.15.52:0/2708909486 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f7f800a1d80 0x7f7f800a4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:29:00.333 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:00.334+0000 7f7f8b7fe640 1 -- 172.21.15.52:0/2708909486 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8802+0+0 (secure 0 0 0) 0x7f7f840cda00 con 0x7f7f94100510 2024-03-04T09:29:00.333 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:00.335+0000 7f7f99d6b640 1 --2- 172.21.15.52:0/2708909486 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f7f800a1d80 0x7f7f800a4240 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f7f94104210 tx=0x7f7f90005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:29:00.335 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:00.339+0000 7f7f8b7fe640 1 -- 172.21.15.52:0/2708909486 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f7f8409abf0 con 0x7f7f94100510 2024-03-04T09:29:00.518 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:00.521+0000 7f7f9c7d0640 1 -- 172.21.15.52:0/2708909486 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7f941a2930 con 0x7f7f94100510 2024-03-04T09:29:00.519 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:00.522+0000 7f7f8b7fe640 1 -- 172.21.15.52:0/2708909486 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) v1 ==== 74+0+24164 (secure 0 0 0) 0x7f7f84009de0 con 0x7f7f94100510 2024-03-04T09:29:00.520 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-03-04T09:29:00.520 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":89,"fsid":"4e059f08-da08-11ee-95c6-87774f69a715","created":"2024-03-04T09:20:46.104963+0000","modified":"2024-03-04T09:28:42.295274+0000","last_up_change":"2024-03-04T09:27:49.261365+0000","last_in_change":"2024-03-04T09:27:30.024964+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-04T09:24:01.742461+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-04T09:28:35.835600+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-04T09:28:36.453099+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-04T09:28:40.168831+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"583172be-602f-4bf4-bfcb-42663cf5b277","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6802","nonce":1371221397},{"type":"v1","addr":"172.21.15.52:6803","nonce":1371221397}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6804","nonce":1371221397},{"type":"v1","addr":"172.21.15.52:6805","nonce":1371221397}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6808","nonce":1371221397},{"type":"v1","addr":"172.21.15.52:6809","nonce":1371221397}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6806","nonce":1371221397},{"type":"v1","addr":"172.21.15.52:6807","nonce":1371221397}]},"public_addr":"172.21.15.52:6803/1371221397","cluster_addr":"172.21.15.52:6805/1371221397","heartbeat_back_addr":"172.21.15.52:6809/1371221397","heartbeat_front_addr":"172.21.15.52:6807/1371221397","state":["exists","up"]},{"osd":1,"uuid":"5ae0081c-a2f6-46eb-ace0-0368208377b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":2540865537},{"type":"v1","addr":"172.21.15.106:6801","nonce":2540865537}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":2540865537},{"type":"v1","addr":"172.21.15.106:6803","nonce":2540865537}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":2540865537},{"type":"v1","addr":"172.21.15.106:6807","nonce":2540865537}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":2540865537},{"type":"v1","addr":"172.21.15.106:6805","nonce":2540865537}]},"public_addr":"172.21.15.106:6801/2540865537","cluster_addr":"172.21.15.106:6803/2540865537","heartbeat_back_addr":"172.21.15.106:6807/2540865537","heartbeat_front_addr":"172.21.15.106:6805/2540865537","state":["exists","up"]},{"osd":2,"uuid":"1948bac7-2bb2-4218-8b93-4e4743a0de04","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":1377083164},{"type":"v1","addr":"172.21.15.191:6801","nonce":1377083164}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":1377083164},{"type":"v1","addr":"172.21.15.191:6803","nonce":1377083164}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":1377083164},{"type":"v1","addr":"172.21.15.191:6807","nonce":1377083164}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":1377083164},{"type":"v1","addr":"172.21.15.191:6805","nonce":1377083164}]},"public_addr":"172.21.15.191:6801/1377083164","cluster_addr":"172.21.15.191:6803/1377083164","heartbeat_back_addr":"172.21.15.191:6807/1377083164","heartbeat_front_addr":"172.21.15.191:6805/1377083164","state":["exists","up"]},{"osd":3,"uuid":"16c9e9fe-7167-4992-bf12-1a7398e0498a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6810","nonce":2388571810},{"type":"v1","addr":"172.21.15.52:6811","nonce":2388571810}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6812","nonce":2388571810},{"type":"v1","addr":"172.21.15.52:6813","nonce":2388571810}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6816","nonce":2388571810},{"type":"v1","addr":"172.21.15.52:6817","nonce":2388571810}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6814","nonce":2388571810},{"type":"v1","addr":"172.21.15.52:6815","nonce":2388571810}]},"public_addr":"172.21.15.52:6811/2388571810","cluster_addr":"172.21.15.52:6813/2388571810","heartbeat_back_addr":"172.21.15.52:6817/2388571810","heartbeat_front_addr":"172.21.15.52:6815/2388571810","state":["exists","up"]},{"osd":4,"uuid":"c8ef48c7-7c78-44d3-a9e5-99ec750abd3d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6808","nonce":3465868015},{"type":"v1","addr":"172.21.15.106:6809","nonce":3465868015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6810","nonce":3465868015},{"type":"v1","addr":"172.21.15.106:6811","nonce":3465868015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6814","nonce":3465868015},{"type":"v1","addr":"172.21.15.106:6815","nonce":3465868015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6812","nonce":3465868015},{"type":"v1","addr":"172.21.15.106:6813","nonce":3465868015}]},"public_addr":"172.21.15.106:6809/3465868015","cluster_addr":"172.21.15.106:6811/3465868015","heartbeat_back_addr":"172.21.15.106:6815/3465868015","heartbeat_front_addr":"172.21.15.106:6813/3465868015","state":["exists","up"]},{"osd":5,"uuid":"9e41824a-c5a2-4f88-be50-2ba1dceb3220","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":1485092643},{"type":"v1","addr":"172.21.15.191:6809","nonce":1485092643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":1485092643},{"type":"v1","addr":"172.21.15.191:6811","nonce":1485092643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":1485092643},{"type":"v1","addr":"172.21.15.191:6815","nonce":1485092643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":1485092643},{"type":"v1","addr":"172.21.15.191:6813","nonce":1485092643}]},"public_addr":"172.21.15.191:6809/1485092643","cluster_addr":"172.21.15.191:6811/1485092643","heartbeat_back_addr":"172.21.15.191:6815/1485092643","heartbeat_front_addr":"172.21.15.191:6813/1485092643","state":["exists","up"]},{"osd":6,"uuid":"b35dccee-ec16-452e-ac69-3ac2fb7b27fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6818","nonce":1989797856},{"type":"v1","addr":"172.21.15.52:6819","nonce":1989797856}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6820","nonce":1989797856},{"type":"v1","addr":"172.21.15.52:6821","nonce":1989797856}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6824","nonce":1989797856},{"type":"v1","addr":"172.21.15.52:6825","nonce":1989797856}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6822","nonce":1989797856},{"type":"v1","addr":"172.21.15.52:6823","nonce":1989797856}]},"public_addr":"172.21.15.52:6819/1989797856","cluster_addr":"172.21.15.52:6821/1989797856","heartbeat_back_addr":"172.21.15.52:6825/1989797856","heartbeat_front_addr":"172.21.15.52:6823/1989797856","state":["exists","up"]},{"osd":7,"uuid":"71303e3d-14ec-4cbd-b19c-67645a1de3c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6816","nonce":1450870329},{"type":"v1","addr":"172.21.15.106:6817","nonce":1450870329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":1450870329},{"type":"v1","addr":"172.21.15.106:6819","nonce":1450870329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":1450870329},{"type":"v1","addr":"172.21.15.106:6823","nonce":1450870329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6820","nonce":1450870329},{"type":"v1","addr":"172.21.15.106:6821","nonce":1450870329}]},"public_addr":"172.21.15.106:6817/1450870329","cluster_addr":"172.21.15.106:6819/1450870329","heartbeat_back_addr":"172.21.15.106:6823/1450870329","heartbeat_front_addr":"172.21.15.106:6821/1450870329","state":["exists","up"]},{"osd":8,"uuid":"6f5786c4-4402-435a-bdad-adff46fe9473","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":1114119072},{"type":"v1","addr":"172.21.15.191:6817","nonce":1114119072}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6818","nonce":1114119072},{"type":"v1","addr":"172.21.15.191:6819","nonce":1114119072}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":1114119072},{"type":"v1","addr":"172.21.15.191:6823","nonce":1114119072}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":1114119072},{"type":"v1","addr":"172.21.15.191:6821","nonce":1114119072}]},"public_addr":"172.21.15.191:6817/1114119072","cluster_addr":"172.21.15.191:6819/1114119072","heartbeat_back_addr":"172.21.15.191:6823/1114119072","heartbeat_front_addr":"172.21.15.191:6821/1114119072","state":["exists","up"]},{"osd":9,"uuid":"062bbaa5-1227-4684-b859-450f5594a90c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6826","nonce":3485821914},{"type":"v1","addr":"172.21.15.52:6827","nonce":3485821914}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6828","nonce":3485821914},{"type":"v1","addr":"172.21.15.52:6829","nonce":3485821914}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6832","nonce":3485821914},{"type":"v1","addr":"172.21.15.52:6833","nonce":3485821914}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6830","nonce":3485821914},{"type":"v1","addr":"172.21.15.52:6831","nonce":3485821914}]},"public_addr":"172.21.15.52:6827/3485821914","cluster_addr":"172.21.15.52:6829/3485821914","heartbeat_back_addr":"172.21.15.52:6833/3485821914","heartbeat_front_addr":"172.21.15.52:6831/3485821914","state":["exists","up"]},{"osd":10,"uuid":"1a285a5a-f793-48da-a8eb-5326d9528cd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":3815586576},{"type":"v1","addr":"172.21.15.106:6825","nonce":3815586576}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6826","nonce":3815586576},{"type":"v1","addr":"172.21.15.106:6827","nonce":3815586576}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":3815586576},{"type":"v1","addr":"172.21.15.106:6831","nonce":3815586576}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":3815586576},{"type":"v1","addr":"172.21.15.106:6829","nonce":3815586576}]},"public_addr":"172.21.15.106:6825/3815586576","cluster_addr":"172.21.15.106:6827/3815586576","heartbeat_back_addr":"172.21.15.106:6831/3815586576","heartbeat_front_addr":"172.21.15.106:6829/3815586576","state":["exists","up"]},{"osd":11,"uuid":"1f8b9a43-57fa-41f4-9969-3b803a74bf45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":2009632843},{"type":"v1","addr":"172.21.15.191:6825","nonce":2009632843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":2009632843},{"type":"v1","addr":"172.21.15.191:6827","nonce":2009632843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":2009632843},{"type":"v1","addr":"172.21.15.191:6831","nonce":2009632843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":2009632843},{"type":"v1","addr":"172.21.15.191:6829","nonce":2009632843}]},"public_addr":"172.21.15.191:6825/2009632843","cluster_addr":"172.21.15.191:6827/2009632843","heartbeat_back_addr":"172.21.15.191:6831/2009632843","heartbeat_front_addr":"172.21.15.191:6829/2009632843","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T09:23:10.894763+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T09:23:34.420055+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T09:23:58.288324+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T09:24:23.673457+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T09:24:48.208561+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T09:25:13.405473+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T09:25:38.670915+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T09:26:04.451257+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T09:26:30.281380+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T09:26:56.543143+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T09:27:22.412930+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T09:27:48.003866+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.52:0/4082645718":"2024-03-05T09:21:39.401935+0000","172.21.15.52:0/1271949988":"2024-03-05T09:21:39.401935+0000","172.21.15.52:6801/2379361333":"2024-03-05T09:21:12.760938+0000","172.21.15.52:0/497075075":"2024-03-05T09:21:12.760938+0000","172.21.15.52:6800/2379361333":"2024-03-05T09:21:12.760938+0000","172.21.15.52:0/568242577":"2024-03-05T09:21:12.760938+0000","172.21.15.52:6801/2425730636":"2024-03-05T09:21:39.401935+0000","172.21.15.52:6800/2425730636":"2024-03-05T09:21:39.401935+0000","172.21.15.52:0/2743084243":"2024-03-05T09:21:39.401935+0000","172.21.15.52:0/1035878034":"2024-03-05T09:21:12.760938+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-04T09:29:00.521 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:00.524+0000 7f7f9c7d0640 1 -- 172.21.15.52:0/2708909486 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f7f800a1d80 msgr2=0x7f7f800a4240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:29:00.522 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:00.524+0000 7f7f9c7d0640 1 --2- 172.21.15.52:0/2708909486 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f7f800a1d80 0x7f7f800a4240 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f7f94104210 tx=0x7f7f90005f90 comp rx=0 tx=0).stop 2024-03-04T09:29:00.522 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:00.524+0000 7f7f9c7d0640 1 -- 172.21.15.52:0/2708909486 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7f94100510 msgr2=0x7f7f941044a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:29:00.522 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:00.524+0000 7f7f9c7d0640 1 --2- 172.21.15.52:0/2708909486 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7f94100510 0x7f7f941044a0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f7f8400d770 tx=0x7f7f8400dc30 comp rx=0 tx=0).stop 2024-03-04T09:29:00.522 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:00.524+0000 7f7f9c7d0640 1 -- 172.21.15.52:0/2708909486 shutdown_connections 2024-03-04T09:29:00.522 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:00.524+0000 7f7f9c7d0640 1 --2- 172.21.15.52:0/2708909486 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f7f800a1d80 0x7f7f800a4240 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:29:00.522 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:00.524+0000 7f7f9c7d0640 1 --2- 172.21.15.52:0/2708909486 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7f941020c0 0x7f7f94103020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:29:00.522 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:00.524+0000 7f7f9c7d0640 1 --2- 172.21.15.52:0/2708909486 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7f94101700 0x7f7f94102ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:29:00.522 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:00.524+0000 7f7f9c7d0640 1 --2- 172.21.15.52:0/2708909486 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7f94100510 0x7f7f941044a0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:29:00.522 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:00.524+0000 7f7f9c7d0640 1 -- 172.21.15.52:0/2708909486 >> 172.21.15.52:0/2708909486 conn(0x7f7f940fba80 msgr2=0x7f7f940fd4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:29:00.522 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:00.525+0000 7f7f9c7d0640 1 -- 172.21.15.52:0/2708909486 shutdown_connections 2024-03-04T09:29:00.522 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:00.525+0000 7f7f9c7d0640 1 -- 172.21.15.52:0/2708909486 wait complete. 2024-03-04T09:29:00.533 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-03-04T09:29:00.640 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:00.641+0000 7f7c34bb5640 1 -- 172.21.15.52:0/1615876919 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7c301053c0 msgr2=0x7f7c3010bc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:29:00.641 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:00.641+0000 7f7c34bb5640 1 --2- 172.21.15.52:0/1615876919 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7c301053c0 0x7f7c3010bc50 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f7c20009ca0 tx=0x7f7c2002c0f0 comp rx=0 tx=0).stop 2024-03-04T09:29:00.641 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:00.641+0000 7f7c34bb5640 1 -- 172.21.15.52:0/1615876919 shutdown_connections 2024-03-04T09:29:00.641 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:00.641+0000 7f7c34bb5640 1 --2- 172.21.15.52:0/1615876919 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7c301053c0 0x7f7c3010bc50 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:29:00.641 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:00.641+0000 7f7c34bb5640 1 --2- 172.21.15.52:0/1615876919 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7c30104a00 0x7f7c30104e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:29:00.641 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:00.641+0000 7f7c34bb5640 1 --2- 172.21.15.52:0/1615876919 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7c30103810 0x7f7c30103c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:29:00.641 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:00.641+0000 7f7c34bb5640 1 -- 172.21.15.52:0/1615876919 >> 172.21.15.52:0/1615876919 conn(0x7f7c300fed80 msgr2=0x7f7c301011c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:29:00.641 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:00.641+0000 7f7c34bb5640 1 -- 172.21.15.52:0/1615876919 shutdown_connections 2024-03-04T09:29:00.641 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:00.641+0000 7f7c34bb5640 1 -- 172.21.15.52:0/1615876919 wait complete. 2024-03-04T09:29:00.641 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:00.642+0000 7f7c34bb5640 1 Processor -- start 2024-03-04T09:29:00.641 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:00.642+0000 7f7c34bb5640 1 -- start start 2024-03-04T09:29:00.641 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:00.642+0000 7f7c34bb5640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7c30103810 0x7f7c301999f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:29:00.641 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:00.642+0000 7f7c34bb5640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7c30104a00 0x7f7c30199f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:29:00.642 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:00.642+0000 7f7c34bb5640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7c301053c0 0x7f7c301a0fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:29:00.642 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:00.642+0000 7f7c34bb5640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c3010f470 con 0x7f7c30103810 2024-03-04T09:29:00.642 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:00.642+0000 7f7c34bb5640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c3010f2f0 con 0x7f7c301053c0 2024-03-04T09:29:00.642 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:00.642+0000 7f7c34bb5640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c3010f5f0 con 0x7f7c30104a00 2024-03-04T09:29:00.642 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:00.642+0000 7f7c2e59c640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7c30103810 0x7f7c301999f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:29:00.642 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:00.642+0000 7f7c2e59c640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7c30103810 0x7f7c301999f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:52792/0 (socket says 172.21.15.52:52792) 2024-03-04T09:29:00.642 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:00.642+0000 7f7c2e59c640 1 -- 172.21.15.52:0/3595817015 learned_addr learned my addr 172.21.15.52:0/3595817015 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-04T09:29:00.642 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:00.642+0000 7f7c2ed9d640 1 --2- 172.21.15.52:0/3595817015 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7c301053c0 0x7f7c301a0fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:29:00.642 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:00.643+0000 7f7c2dd9b640 1 --2- 172.21.15.52:0/3595817015 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7c30104a00 0x7f7c30199f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:29:00.642 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:00.643+0000 7f7c2e59c640 1 -- 172.21.15.52:0/3595817015 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7c30104a00 msgr2=0x7f7c30199f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:29:00.642 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:00.643+0000 7f7c2e59c640 1 --2- 172.21.15.52:0/3595817015 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7c30104a00 0x7f7c30199f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:29:00.642 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:00.643+0000 7f7c2e59c640 1 -- 172.21.15.52:0/3595817015 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7c301053c0 msgr2=0x7f7c301a0fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:29:00.642 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:00.643+0000 7f7c2e59c640 1 --2- 172.21.15.52:0/3595817015 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7c301053c0 0x7f7c301a0fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:29:00.643 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:00.643+0000 7f7c2e59c640 1 -- 172.21.15.52:0/3595817015 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c301a1610 con 0x7f7c30103810 2024-03-04T09:29:00.643 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:00.643+0000 7f7c2dd9b640 1 --2- 172.21.15.52:0/3595817015 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7c30104a00 0x7f7c30199f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-04T09:29:00.643 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:00.643+0000 7f7c2e59c640 1 --2- 172.21.15.52:0/3595817015 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7c30103810 0x7f7c301999f0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f7c2800c970 tx=0x7f7c2800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:29:00.643 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:00.643+0000 7f7c277fe640 1 -- 172.21.15.52:0/3595817015 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c28007c10 con 0x7f7c30103810 2024-03-04T09:29:00.643 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:00.643+0000 7f7c277fe640 1 -- 172.21.15.52:0/3595817015 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7c28007db0 con 0x7f7c30103810 2024-03-04T09:29:00.643 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:00.643+0000 7f7c277fe640 1 -- 172.21.15.52:0/3595817015 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c2800f940 con 0x7f7c30103810 2024-03-04T09:29:00.643 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:00.643+0000 7f7c34bb5640 1 -- 172.21.15.52:0/3595817015 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c301a1900 con 0x7f7c30103810 2024-03-04T09:29:00.643 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:00.643+0000 7f7c34bb5640 1 -- 172.21.15.52:0/3595817015 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c301a1e50 con 0x7f7c30103810 2024-03-04T09:29:00.644 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:00.644+0000 7f7c34bb5640 1 -- 172.21.15.52:0/3595817015 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7c30068810 con 0x7f7c30103810 2024-03-04T09:29:00.644 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:00.645+0000 7f7c277fe640 1 -- 172.21.15.52:0/3595817015 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 16) v1 ==== 136756+0+0 (secure 0 0 0) 0x7f7c2800fae0 con 0x7f7c30103810 2024-03-04T09:29:00.644 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:00.646+0000 7f7c277fe640 1 --2- 172.21.15.52:0/3595817015 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f7c1c0a1d80 0x7f7c1c0a4240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:29:00.644 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:00.646+0000 7f7c2dd9b640 1 --2- 172.21.15.52:0/3595817015 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f7c1c0a1d80 0x7f7c1c0a4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:29:00.644 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:00.646+0000 7f7c277fe640 1 -- 172.21.15.52:0/3595817015 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8802+0+0 (secure 0 0 0) 0x7f7c280ce650 con 0x7f7c30103810 2024-03-04T09:29:00.644 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:00.646+0000 7f7c2dd9b640 1 --2- 172.21.15.52:0/3595817015 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f7c1c0a1d80 0x7f7c1c0a4240 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f7c3019af00 tx=0x7f7c18005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:29:00.646 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:00.649+0000 7f7c277fe640 1 -- 172.21.15.52:0/3595817015 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f7c2809b900 con 0x7f7c30103810 2024-03-04T09:29:00.865 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:00.869+0000 7f7c34bb5640 1 -- 172.21.15.52:0/3595817015 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f7c3010aad0 con 0x7f7c30103810 2024-03-04T09:29:00.968 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:00.971+0000 7f7c277fe640 1 -- 172.21.15.52:0/3595817015 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v18) v1 ==== 118+0+0 (secure 0 0 0) 0x7f7c2809b200 con 0x7f7c30103810 2024-03-04T09:29:00.970 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:00.973+0000 7f7c34bb5640 1 -- 172.21.15.52:0/3595817015 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f7c1c0a1d80 msgr2=0x7f7c1c0a4240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:29:00.970 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:00.973+0000 7f7c34bb5640 1 --2- 172.21.15.52:0/3595817015 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f7c1c0a1d80 0x7f7c1c0a4240 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f7c3019af00 tx=0x7f7c18005f90 comp rx=0 tx=0).stop 2024-03-04T09:29:00.970 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:00.973+0000 7f7c34bb5640 1 -- 172.21.15.52:0/3595817015 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7c30103810 msgr2=0x7f7c301999f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:29:00.970 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:00.973+0000 7f7c34bb5640 1 --2- 172.21.15.52:0/3595817015 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7c30103810 0x7f7c301999f0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f7c2800c970 tx=0x7f7c2800ce30 comp rx=0 tx=0).stop 2024-03-04T09:29:00.970 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:00.973+0000 7f7c34bb5640 1 -- 172.21.15.52:0/3595817015 shutdown_connections 2024-03-04T09:29:00.970 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:00.973+0000 7f7c34bb5640 1 --2- 172.21.15.52:0/3595817015 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f7c1c0a1d80 0x7f7c1c0a4240 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:29:00.970 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:00.973+0000 7f7c34bb5640 1 --2- 172.21.15.52:0/3595817015 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7c301053c0 0x7f7c301a0fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:29:00.970 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:00.973+0000 7f7c34bb5640 1 --2- 172.21.15.52:0/3595817015 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7c30104a00 0x7f7c30199f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:29:00.970 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:00.973+0000 7f7c34bb5640 1 --2- 172.21.15.52:0/3595817015 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7c30103810 0x7f7c301999f0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:29:00.971 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:00.973+0000 7f7c34bb5640 1 -- 172.21.15.52:0/3595817015 >> 172.21.15.52:0/3595817015 conn(0x7f7c300fed80 msgr2=0x7f7c30107ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:29:00.971 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:00.973+0000 7f7c34bb5640 1 -- 172.21.15.52:0/3595817015 shutdown_connections 2024-03-04T09:29:00.971 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:00.973+0000 7f7c34bb5640 1 -- 172.21.15.52:0/3595817015 wait complete. 2024-03-04T09:29:00.981 DEBUG:tasks.cephfs.filesystem:fs_config: {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'max_mds': 3, 'session_timeout': 300, 'standby_replay': True, 'subvols': {'create': 2, 'subvol_options': '--size 25000000000'}} 2024-03-04T09:29:00.981 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs max_mds 3 2024-03-04T09:29:01.089 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:01.090+0000 7ff7aaab5640 1 -- 172.21.15.52:0/3040093046 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff7a41027d0 msgr2=0x7ff7a4102c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:29:01.089 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:01.090+0000 7ff7aaab5640 1 --2- 172.21.15.52:0/3040093046 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff7a41027d0 0x7ff7a4102c50 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7ff790009900 tx=0x7ff79002bc30 comp rx=0 tx=0).stop 2024-03-04T09:29:01.089 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:01.090+0000 7ff7aaab5640 1 -- 172.21.15.52:0/3040093046 shutdown_connections 2024-03-04T09:29:01.089 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:01.090+0000 7ff7aaab5640 1 --2- 172.21.15.52:0/3040093046 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff7a4103190 0x7ff7a4109a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:29:01.089 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:01.090+0000 7ff7aaab5640 1 --2- 172.21.15.52:0/3040093046 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff7a41027d0 0x7ff7a4102c50 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:29:01.089 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:01.090+0000 7ff7aaab5640 1 --2- 172.21.15.52:0/3040093046 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff7a41015e0 0x7ff7a4101a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:29:01.089 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:01.090+0000 7ff7aaab5640 1 -- 172.21.15.52:0/3040093046 >> 172.21.15.52:0/3040093046 conn(0x7ff7a40fcb70 msgr2=0x7ff7a40fef90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:29:01.089 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:01.090+0000 7ff7aaab5640 1 -- 172.21.15.52:0/3040093046 shutdown_connections 2024-03-04T09:29:01.089 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:01.090+0000 7ff7aaab5640 1 -- 172.21.15.52:0/3040093046 wait complete. 2024-03-04T09:29:01.090 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:01.090+0000 7ff7aaab5640 1 Processor -- start 2024-03-04T09:29:01.090 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:01.090+0000 7ff7aaab5640 1 -- start start 2024-03-04T09:29:01.090 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:01.090+0000 7ff7aaab5640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff7a41015e0 0x7ff7a41978f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:29:01.090 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:01.090+0000 7ff7aaab5640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff7a41027d0 0x7ff7a4197e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:29:01.090 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:01.090+0000 7ff7aaab5640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff7a4103190 0x7ff7a419eea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:29:01.090 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:01.090+0000 7ff7aaab5640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7a410d240 con 0x7ff7a41015e0 2024-03-04T09:29:01.090 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:01.090+0000 7ff7aaab5640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7a410d0c0 con 0x7ff7a41027d0 2024-03-04T09:29:01.090 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:01.090+0000 7ff7aaab5640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7a410d3c0 con 0x7ff7a4103190 2024-03-04T09:29:01.090 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:01.091+0000 7ff7a8851640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff7a41015e0 0x7ff7a41978f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:29:01.090 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:01.091+0000 7ff7a8851640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff7a41015e0 0x7ff7a41978f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:52808/0 (socket says 172.21.15.52:52808) 2024-03-04T09:29:01.090 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:01.091+0000 7ff7a8851640 1 -- 172.21.15.52:0/936356160 learned_addr learned my addr 172.21.15.52:0/936356160 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-04T09:29:01.090 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:01.091+0000 7ff7a3fff640 1 --2- 172.21.15.52:0/936356160 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff7a41027d0 0x7ff7a4197e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:29:01.090 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:01.091+0000 7ff7a9052640 1 --2- 172.21.15.52:0/936356160 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff7a4103190 0x7ff7a419eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:29:01.091 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:01.091+0000 7ff7a8851640 1 -- 172.21.15.52:0/936356160 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff7a4103190 msgr2=0x7ff7a419eea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:29:01.091 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:01.091+0000 7ff7a8851640 1 --2- 172.21.15.52:0/936356160 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff7a4103190 0x7ff7a419eea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:29:01.091 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:01.091+0000 7ff7a8851640 1 -- 172.21.15.52:0/936356160 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff7a41027d0 msgr2=0x7ff7a4197e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:29:01.091 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:01.091+0000 7ff7a8851640 1 --2- 172.21.15.52:0/936356160 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff7a41027d0 0x7ff7a4197e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:29:01.091 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:01.091+0000 7ff7a8851640 1 -- 172.21.15.52:0/936356160 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7a419f510 con 0x7ff7a41015e0 2024-03-04T09:29:01.091 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:01.091+0000 7ff7a9052640 1 --2- 172.21.15.52:0/936356160 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff7a4103190 0x7ff7a419eea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-04T09:29:01.091 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:01.091+0000 7ff7a8851640 1 --2- 172.21.15.52:0/936356160 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff7a41015e0 0x7ff7a41978f0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7ff79800e9c0 tx=0x7ff79800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:29:01.091 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:01.091+0000 7ff7a1ffb640 1 -- 172.21.15.52:0/936356160 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff79800cd80 con 0x7ff7a41015e0 2024-03-04T09:29:01.091 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:01.091+0000 7ff7aaab5640 1 -- 172.21.15.52:0/936356160 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7a419f800 con 0x7ff7a41015e0 2024-03-04T09:29:01.091 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:01.091+0000 7ff7a1ffb640 1 -- 172.21.15.52:0/936356160 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff7980035f0 con 0x7ff7a41015e0 2024-03-04T09:29:01.091 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:01.091+0000 7ff7a1ffb640 1 -- 172.21.15.52:0/936356160 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff798010690 con 0x7ff7a41015e0 2024-03-04T09:29:01.091 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:01.091+0000 7ff7aaab5640 1 -- 172.21.15.52:0/936356160 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7a419fd70 con 0x7ff7a41015e0 2024-03-04T09:29:01.092 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:01.092+0000 7ff7aaab5640 1 -- 172.21.15.52:0/936356160 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff7a4068810 con 0x7ff7a41015e0 2024-03-04T09:29:01.092 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:01.093+0000 7ff7a1ffb640 1 -- 172.21.15.52:0/936356160 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 16) v1 ==== 136756+0+0 (secure 0 0 0) 0x7ff7980037a0 con 0x7ff7a41015e0 2024-03-04T09:29:01.092 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:01.094+0000 7ff7a1ffb640 1 --2- 172.21.15.52:0/936356160 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7ff7940a1d80 0x7ff7940a4240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:29:01.092 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:01.094+0000 7ff7a3fff640 1 --2- 172.21.15.52:0/936356160 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7ff7940a1d80 0x7ff7940a4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:29:01.093 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:01.094+0000 7ff7a1ffb640 1 -- 172.21.15.52:0/936356160 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8802+0+0 (secure 0 0 0) 0x7ff7980ce490 con 0x7ff7a41015e0 2024-03-04T09:29:01.093 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:01.094+0000 7ff7a3fff640 1 --2- 172.21.15.52:0/936356160 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7ff7940a1d80 0x7ff7940a4240 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7ff79002c7b0 tx=0x7ff790005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:29:01.095 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:01.098+0000 7ff7a1ffb640 1 -- 172.21.15.52:0/936356160 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7ff79809b620 con 0x7ff7a41015e0 2024-03-04T09:29:01.318 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:01.321+0000 7ff7aaab5640 1 -- 172.21.15.52:0/936356160 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"} v 0) v1 -- 0x7ff7a40621d0 con 0x7ff7a41015e0 2024-03-04T09:29:01.334 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:29:00 smithi052 ceph-mon[32473]: pgmap v299: 129 pgs: 129 active+clean; 581 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-03-04T09:29:01.335 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:29:00 smithi052 ceph-mon[32473]: from='client.? 172.21.15.52:0/780665599' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-04T09:29:01.335 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:29:00 smithi052 ceph-mon[32473]: from='client.? 172.21.15.52:0/2708909486' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-04T09:29:01.335 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:29:00 smithi052 ceph-mon[32473]: from='client.? 172.21.15.52:0/3595817015' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-03-04T09:29:01.336 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:29:00 smithi191 ceph-mon[36816]: pgmap v299: 129 pgs: 129 active+clean; 581 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-03-04T09:29:01.336 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:29:00 smithi191 ceph-mon[36816]: from='client.? 172.21.15.52:0/780665599' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-04T09:29:01.336 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:29:00 smithi191 ceph-mon[36816]: from='client.? 172.21.15.52:0/2708909486' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-04T09:29:01.336 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:29:00 smithi191 ceph-mon[36816]: from='client.? 172.21.15.52:0/3595817015' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-03-04T09:29:01.381 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:29:00 smithi106 ceph-mon[36484]: pgmap v299: 129 pgs: 129 active+clean; 581 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-03-04T09:29:01.381 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:29:00 smithi106 ceph-mon[36484]: from='client.? 172.21.15.52:0/780665599' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-04T09:29:01.382 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:29:00 smithi106 ceph-mon[36484]: from='client.? 172.21.15.52:0/2708909486' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-04T09:29:01.382 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:29:00 smithi106 ceph-mon[36484]: from='client.? 172.21.15.52:0/3595817015' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-03-04T09:29:01.973 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:01.976+0000 7ff7a1ffb640 1 -- 172.21.15.52:0/936356160 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]=0 v19) v1 ==== 105+0+0 (secure 0 0 0) 0x7ff79809af20 con 0x7ff7a41015e0 2024-03-04T09:29:01.976 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:01.979+0000 7ff7aaab5640 1 -- 172.21.15.52:0/936356160 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7ff7940a1d80 msgr2=0x7ff7940a4240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:29:01.976 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:01.979+0000 7ff7aaab5640 1 --2- 172.21.15.52:0/936356160 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7ff7940a1d80 0x7ff7940a4240 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7ff79002c7b0 tx=0x7ff790005fd0 comp rx=0 tx=0).stop 2024-03-04T09:29:01.976 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:01.979+0000 7ff7aaab5640 1 -- 172.21.15.52:0/936356160 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff7a41015e0 msgr2=0x7ff7a41978f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:29:01.977 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:01.979+0000 7ff7aaab5640 1 --2- 172.21.15.52:0/936356160 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff7a41015e0 0x7ff7a41978f0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7ff79800e9c0 tx=0x7ff79800ee80 comp rx=0 tx=0).stop 2024-03-04T09:29:01.977 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:01.979+0000 7ff7aaab5640 1 -- 172.21.15.52:0/936356160 shutdown_connections 2024-03-04T09:29:01.977 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:01.979+0000 7ff7aaab5640 1 --2- 172.21.15.52:0/936356160 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7ff7940a1d80 0x7ff7940a4240 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:29:01.977 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:01.979+0000 7ff7aaab5640 1 --2- 172.21.15.52:0/936356160 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff7a4103190 0x7ff7a419eea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:29:01.977 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:01.979+0000 7ff7aaab5640 1 --2- 172.21.15.52:0/936356160 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff7a41027d0 0x7ff7a4197e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:29:01.977 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:01.979+0000 7ff7aaab5640 1 --2- 172.21.15.52:0/936356160 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff7a41015e0 0x7ff7a41978f0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:29:01.977 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:01.979+0000 7ff7aaab5640 1 -- 172.21.15.52:0/936356160 >> 172.21.15.52:0/936356160 conn(0x7ff7a40fcb70 msgr2=0x7ff7a4105b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:29:01.977 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:01.979+0000 7ff7aaab5640 1 -- 172.21.15.52:0/936356160 shutdown_connections 2024-03-04T09:29:01.977 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:01.979+0000 7ff7aaab5640 1 -- 172.21.15.52:0/936356160 wait complete. 2024-03-04T09:29:01.991 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay true 2024-03-04T09:29:02.106 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:02.107+0000 7fe867b8c640 1 -- 172.21.15.52:0/2496040674 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe860102090 msgr2=0x7fe860108920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:29:02.106 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:02.107+0000 7fe867b8c640 1 --2- 172.21.15.52:0/2496040674 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe860102090 0x7fe860108920 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fe858009ca0 tx=0x7fe85802c0f0 comp rx=0 tx=0).stop 2024-03-04T09:29:02.106 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:02.107+0000 7fe867b8c640 1 -- 172.21.15.52:0/2496040674 shutdown_connections 2024-03-04T09:29:02.106 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:02.107+0000 7fe867b8c640 1 --2- 172.21.15.52:0/2496040674 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe860102090 0x7fe860108920 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:29:02.106 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:02.107+0000 7fe867b8c640 1 --2- 172.21.15.52:0/2496040674 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe8601016d0 0x7fe860101b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:29:02.106 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:02.107+0000 7fe867b8c640 1 --2- 172.21.15.52:0/2496040674 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe8601004e0 0x7fe860100900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:29:02.106 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:02.107+0000 7fe867b8c640 1 -- 172.21.15.52:0/2496040674 >> 172.21.15.52:0/2496040674 conn(0x7fe8600fba70 msgr2=0x7fe8600fde90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:29:02.107 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:02.107+0000 7fe867b8c640 1 -- 172.21.15.52:0/2496040674 shutdown_connections 2024-03-04T09:29:02.107 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:02.107+0000 7fe867b8c640 1 -- 172.21.15.52:0/2496040674 wait complete. 2024-03-04T09:29:02.107 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:02.107+0000 7fe867b8c640 1 Processor -- start 2024-03-04T09:29:02.107 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:02.107+0000 7fe867b8c640 1 -- start start 2024-03-04T09:29:02.107 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:02.108+0000 7fe867b8c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe8601004e0 0x7fe86010ae60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:29:02.107 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:02.108+0000 7fe867b8c640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe8601016d0 0x7fe8601094a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:29:02.107 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:02.108+0000 7fe867b8c640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe860102090 0x7fe8601099e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:29:02.107 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:02.108+0000 7fe867b8c640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe86010d280 con 0x7fe8601016d0 2024-03-04T09:29:02.107 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:02.108+0000 7fe867b8c640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe86010d100 con 0x7fe8601004e0 2024-03-04T09:29:02.107 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:02.108+0000 7fe867b8c640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe86010d400 con 0x7fe860102090 2024-03-04T09:29:02.107 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:02.108+0000 7fe865127640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe8601016d0 0x7fe8601094a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:29:02.107 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:02.108+0000 7fe865127640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe8601016d0 0x7fe8601094a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:52818/0 (socket says 172.21.15.52:52818) 2024-03-04T09:29:02.107 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:02.108+0000 7fe865127640 1 -- 172.21.15.52:0/1336623953 learned_addr learned my addr 172.21.15.52:0/1336623953 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-04T09:29:02.108 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:02.108+0000 7fe865928640 1 --2- 172.21.15.52:0/1336623953 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe8601004e0 0x7fe86010ae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:29:02.108 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:02.108+0000 7fe866129640 1 --2- 172.21.15.52:0/1336623953 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe860102090 0x7fe8601099e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:29:02.108 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:02.108+0000 7fe865127640 1 -- 172.21.15.52:0/1336623953 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe860102090 msgr2=0x7fe8601099e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:29:02.108 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:02.108+0000 7fe865127640 1 --2- 172.21.15.52:0/1336623953 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe860102090 0x7fe8601099e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:29:02.108 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:02.108+0000 7fe865127640 1 -- 172.21.15.52:0/1336623953 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe8601004e0 msgr2=0x7fe86010ae60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:29:02.108 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:02.108+0000 7fe865127640 1 --2- 172.21.15.52:0/1336623953 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe8601004e0 0x7fe86010ae60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:29:02.108 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:02.108+0000 7fe865127640 1 -- 172.21.15.52:0/1336623953 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe86010a270 con 0x7fe8601016d0 2024-03-04T09:29:02.108 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:02.108+0000 7fe866129640 1 --2- 172.21.15.52:0/1336623953 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe860102090 0x7fe8601099e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T09:29:02.108 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:02.108+0000 7fe865127640 1 --2- 172.21.15.52:0/1336623953 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe8601016d0 0x7fe8601094a0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fe85000e9f0 tx=0x7fe85000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:29:02.108 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:02.108+0000 7fe856ffd640 1 -- 172.21.15.52:0/1336623953 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe85000cd80 con 0x7fe8601016d0 2024-03-04T09:29:02.108 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:02.108+0000 7fe856ffd640 1 -- 172.21.15.52:0/1336623953 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe850005500 con 0x7fe8601016d0 2024-03-04T09:29:02.108 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:02.108+0000 7fe856ffd640 1 -- 172.21.15.52:0/1336623953 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe850010690 con 0x7fe8601016d0 2024-03-04T09:29:02.108 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:02.108+0000 7fe867b8c640 1 -- 172.21.15.52:0/1336623953 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8601a1a20 con 0x7fe8601016d0 2024-03-04T09:29:02.109 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:02.108+0000 7fe867b8c640 1 -- 172.21.15.52:0/1336623953 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8601a1f70 con 0x7fe8601016d0 2024-03-04T09:29:02.109 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:02.109+0000 7fe867b8c640 1 -- 172.21.15.52:0/1336623953 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe860107800 con 0x7fe8601016d0 2024-03-04T09:29:02.109 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:02.110+0000 7fe856ffd640 1 -- 172.21.15.52:0/1336623953 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 16) v1 ==== 136756+0+0 (secure 0 0 0) 0x7fe8500046c0 con 0x7fe8601016d0 2024-03-04T09:29:02.110 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:02.111+0000 7fe856ffd640 1 --2- 172.21.15.52:0/1336623953 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7fe84c0a1e00 0x7fe84c0a42c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:29:02.110 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:02.111+0000 7fe865928640 1 --2- 172.21.15.52:0/1336623953 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7fe84c0a1e00 0x7fe84c0a42c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:29:02.110 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:02.111+0000 7fe856ffd640 1 -- 172.21.15.52:0/1336623953 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8802+0+0 (secure 0 0 0) 0x7fe8500ce420 con 0x7fe8601016d0 2024-03-04T09:29:02.110 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:02.111+0000 7fe865928640 1 --2- 172.21.15.52:0/1336623953 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7fe84c0a1e00 0x7fe84c0a42c0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fe85c005fd0 tx=0x7fe85c00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:29:02.112 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:02.115+0000 7fe856ffd640 1 -- 172.21.15.52:0/1336623953 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fe85009b610 con 0x7fe8601016d0 2024-03-04T09:29:02.334 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:29:01 smithi052 ceph-mon[32473]: from='client.? 172.21.15.52:0/3595817015' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-03-04T09:29:02.335 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:29:01 smithi052 ceph-mon[32473]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-04T09:29:02.335 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:29:01 smithi052 ceph-mon[32473]: from='client.? 172.21.15.52:0/936356160' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-03-04T09:29:02.336 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:29:01 smithi191 ceph-mon[36816]: from='client.? 172.21.15.52:0/3595817015' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-03-04T09:29:02.336 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:29:01 smithi191 ceph-mon[36816]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-04T09:29:02.337 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:29:01 smithi191 ceph-mon[36816]: from='client.? 172.21.15.52:0/936356160' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-03-04T09:29:02.338 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:02.342+0000 7fe867b8c640 1 -- 172.21.15.52:0/1336623953 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) v1 -- 0x7fe86002cff0 con 0x7fe8601016d0 2024-03-04T09:29:02.381 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:29:01 smithi106 ceph-mon[36484]: from='client.? 172.21.15.52:0/3595817015' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-03-04T09:29:02.381 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:29:01 smithi106 ceph-mon[36484]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-04T09:29:02.381 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:29:01 smithi106 ceph-mon[36484]: from='client.? 172.21.15.52:0/936356160' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-03-04T09:29:02.982 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:02.985+0000 7fe856ffd640 1 -- 172.21.15.52:0/1336623953 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v21) v1 ==== 121+0+0 (secure 0 0 0) 0x7fe85009af10 con 0x7fe8601016d0 2024-03-04T09:29:02.985 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:02.988+0000 7fe867b8c640 1 -- 172.21.15.52:0/1336623953 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7fe84c0a1e00 msgr2=0x7fe84c0a42c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:29:02.985 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:02.988+0000 7fe867b8c640 1 --2- 172.21.15.52:0/1336623953 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7fe84c0a1e00 0x7fe84c0a42c0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fe85c005fd0 tx=0x7fe85c00b410 comp rx=0 tx=0).stop 2024-03-04T09:29:02.985 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:02.988+0000 7fe867b8c640 1 -- 172.21.15.52:0/1336623953 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe8601016d0 msgr2=0x7fe8601094a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:29:02.985 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:02.988+0000 7fe867b8c640 1 --2- 172.21.15.52:0/1336623953 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe8601016d0 0x7fe8601094a0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fe85000e9f0 tx=0x7fe85000edf0 comp rx=0 tx=0).stop 2024-03-04T09:29:02.986 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:02.988+0000 7fe867b8c640 1 -- 172.21.15.52:0/1336623953 shutdown_connections 2024-03-04T09:29:02.986 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:02.988+0000 7fe867b8c640 1 --2- 172.21.15.52:0/1336623953 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7fe84c0a1e00 0x7fe84c0a42c0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:29:02.986 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:02.988+0000 7fe867b8c640 1 --2- 172.21.15.52:0/1336623953 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe860102090 0x7fe8601099e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:29:02.986 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:02.988+0000 7fe867b8c640 1 --2- 172.21.15.52:0/1336623953 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe8601016d0 0x7fe8601094a0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:29:02.986 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:02.988+0000 7fe867b8c640 1 --2- 172.21.15.52:0/1336623953 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe8601004e0 0x7fe86010ae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:29:02.986 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:02.988+0000 7fe867b8c640 1 -- 172.21.15.52:0/1336623953 >> 172.21.15.52:0/1336623953 conn(0x7fe8600fba70 msgr2=0x7fe8601049d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:29:02.986 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:02.988+0000 7fe867b8c640 1 -- 172.21.15.52:0/1336623953 shutdown_connections 2024-03-04T09:29:02.986 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:02.988+0000 7fe867b8c640 1 -- 172.21.15.52:0/1336623953 wait complete. 2024-03-04T09:29:03.002 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs session_timeout 300 2024-03-04T09:29:03.116 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:03.117+0000 7fe204f20640 1 -- 172.21.15.52:0/4140746698 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe2001015f0 msgr2=0x7fe200101a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:29:03.116 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:03.117+0000 7fe204f20640 1 --2- 172.21.15.52:0/4140746698 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe2001015f0 0x7fe200101a10 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fe1f8009900 tx=0x7fe1f802bc30 comp rx=0 tx=0).stop 2024-03-04T09:29:03.116 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:03.117+0000 7fe204f20640 1 -- 172.21.15.52:0/4140746698 shutdown_connections 2024-03-04T09:29:03.116 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:03.117+0000 7fe204f20640 1 --2- 172.21.15.52:0/4140746698 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe2001031a0 0x7fe200109a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:29:03.116 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:03.117+0000 7fe204f20640 1 --2- 172.21.15.52:0/4140746698 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe2001027e0 0x7fe200102c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:29:03.117 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:03.117+0000 7fe204f20640 1 --2- 172.21.15.52:0/4140746698 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe2001015f0 0x7fe200101a10 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:29:03.117 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:03.117+0000 7fe204f20640 1 -- 172.21.15.52:0/4140746698 >> 172.21.15.52:0/4140746698 conn(0x7fe2000fcb60 msgr2=0x7fe2000fefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:29:03.117 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:03.117+0000 7fe204f20640 1 -- 172.21.15.52:0/4140746698 shutdown_connections 2024-03-04T09:29:03.117 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:03.117+0000 7fe204f20640 1 -- 172.21.15.52:0/4140746698 wait complete. 2024-03-04T09:29:03.117 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:03.118+0000 7fe204f20640 1 Processor -- start 2024-03-04T09:29:03.117 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:03.118+0000 7fe204f20640 1 -- start start 2024-03-04T09:29:03.117 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:03.118+0000 7fe204f20640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe2001015f0 0x7fe2001978c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:29:03.117 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:03.118+0000 7fe204f20640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe2001027e0 0x7fe200197e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:29:03.117 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:03.118+0000 7fe204f20640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe2001031a0 0x7fe20019ee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:29:03.117 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:03.118+0000 7fe204f20640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe20010d240 con 0x7fe2001015f0 2024-03-04T09:29:03.117 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:03.118+0000 7fe204f20640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe20010d0c0 con 0x7fe2001027e0 2024-03-04T09:29:03.117 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:03.118+0000 7fe204f20640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe20010d3c0 con 0x7fe2001031a0 2024-03-04T09:29:03.117 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:03.118+0000 7fe1fe59c640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe2001015f0 0x7fe2001978c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:29:03.117 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:03.118+0000 7fe1fe59c640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe2001015f0 0x7fe2001978c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:52836/0 (socket says 172.21.15.52:52836) 2024-03-04T09:29:03.118 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:03.118+0000 7fe1fe59c640 1 -- 172.21.15.52:0/1676378979 learned_addr learned my addr 172.21.15.52:0/1676378979 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-04T09:29:03.118 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:03.118+0000 7fe1fdd9b640 1 --2- 172.21.15.52:0/1676378979 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe2001027e0 0x7fe200197e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:29:03.118 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:03.118+0000 7fe1fed9d640 1 --2- 172.21.15.52:0/1676378979 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe2001031a0 0x7fe20019ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:29:03.118 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:03.118+0000 7fe1fe59c640 1 -- 172.21.15.52:0/1676378979 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe2001031a0 msgr2=0x7fe20019ee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:29:03.118 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:03.118+0000 7fe1fe59c640 1 --2- 172.21.15.52:0/1676378979 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe2001031a0 0x7fe20019ee70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:29:03.118 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:03.118+0000 7fe1fe59c640 1 -- 172.21.15.52:0/1676378979 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe2001027e0 msgr2=0x7fe200197e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:29:03.118 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:03.118+0000 7fe1fe59c640 1 --2- 172.21.15.52:0/1676378979 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe2001027e0 0x7fe200197e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:29:03.118 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:03.118+0000 7fe1fe59c640 1 -- 172.21.15.52:0/1676378979 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe20019f4e0 con 0x7fe2001015f0 2024-03-04T09:29:03.118 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:03.118+0000 7fe1fed9d640 1 --2- 172.21.15.52:0/1676378979 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe2001031a0 0x7fe20019ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T09:29:03.118 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:03.118+0000 7fe1fe59c640 1 --2- 172.21.15.52:0/1676378979 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe2001015f0 0x7fe2001978c0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fe1f8000c00 tx=0x7fe1f8005d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:29:03.118 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:03.119+0000 7fe1f77fe640 1 -- 172.21.15.52:0/1676378979 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1f8041070 con 0x7fe2001015f0 2024-03-04T09:29:03.118 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:03.119+0000 7fe1f77fe640 1 -- 172.21.15.52:0/1676378979 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe1f80044d0 con 0x7fe2001015f0 2024-03-04T09:29:03.118 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:03.119+0000 7fe204f20640 1 -- 172.21.15.52:0/1676378979 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe20019f770 con 0x7fe2001015f0 2024-03-04T09:29:03.119 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:03.119+0000 7fe1f77fe640 1 -- 172.21.15.52:0/1676378979 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1f8034440 con 0x7fe2001015f0 2024-03-04T09:29:03.119 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:03.119+0000 7fe204f20640 1 -- 172.21.15.52:0/1676378979 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe20019fc60 con 0x7fe2001015f0 2024-03-04T09:29:03.119 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:03.120+0000 7fe204f20640 1 -- 172.21.15.52:0/1676378979 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe2001089e0 con 0x7fe2001015f0 2024-03-04T09:29:03.119 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:03.121+0000 7fe1f77fe640 1 -- 172.21.15.52:0/1676378979 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 16) v1 ==== 136756+0+0 (secure 0 0 0) 0x7fe1f8044070 con 0x7fe2001015f0 2024-03-04T09:29:03.120 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:03.121+0000 7fe1f77fe640 1 --2- 172.21.15.52:0/1676378979 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7fe1ec0a1d80 0x7fe1ec0a4240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:29:03.120 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:03.121+0000 7fe1fdd9b640 1 --2- 172.21.15.52:0/1676378979 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7fe1ec0a1d80 0x7fe1ec0a4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:29:03.120 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:03.121+0000 7fe1f77fe640 1 -- 172.21.15.52:0/1676378979 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8802+0+0 (secure 0 0 0) 0x7fe1f80efb90 con 0x7fe2001015f0 2024-03-04T09:29:03.120 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:03.122+0000 7fe1fdd9b640 1 --2- 172.21.15.52:0/1676378979 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7fe1ec0a1d80 0x7fe1ec0a4240 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fe200198dd0 tx=0x7fe1e8005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:29:03.122 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:03.126+0000 7fe1f77fe640 1 -- 172.21.15.52:0/1676378979 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fe1f80bcd80 con 0x7fe2001015f0 2024-03-04T09:29:03.334 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:29:02 smithi052 ceph-mon[32473]: pgmap v300: 129 pgs: 129 active+clean; 581 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-03-04T09:29:03.335 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:29:02 smithi052 ceph-mon[32473]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-03-04T09:29:03.335 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:29:02 smithi052 ceph-mon[32473]: from='client.? 172.21.15.52:0/936356160' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-03-04T09:29:03.335 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:29:02 smithi052 ceph-mon[32473]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-03-04T09:29:03.335 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:29:02 smithi052 ceph-mon[32473]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-04T09:29:03.335 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:29:02 smithi052 ceph-mon[32473]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2024-03-04T09:29:03.335 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:29:02 smithi052 ceph-mon[32473]: daemon mds.i is now active in filesystem cephfs as rank 1 2024-03-04T09:29:03.335 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:29:02 smithi052 ceph-mon[32473]: from='client.? 172.21.15.52:0/1336623953' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-03-04T09:29:03.336 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:29:02 smithi191 ceph-mon[36816]: pgmap v300: 129 pgs: 129 active+clean; 581 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-03-04T09:29:03.336 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:29:02 smithi191 ceph-mon[36816]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-03-04T09:29:03.336 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:29:02 smithi191 ceph-mon[36816]: from='client.? 172.21.15.52:0/936356160' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-03-04T09:29:03.336 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:29:02 smithi191 ceph-mon[36816]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-03-04T09:29:03.336 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:29:02 smithi191 ceph-mon[36816]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-04T09:29:03.336 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:29:02 smithi191 ceph-mon[36816]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2024-03-04T09:29:03.336 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:29:02 smithi191 ceph-mon[36816]: daemon mds.i is now active in filesystem cephfs as rank 1 2024-03-04T09:29:03.337 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:29:02 smithi191 ceph-mon[36816]: from='client.? 172.21.15.52:0/1336623953' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-03-04T09:29:03.345 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:03.348+0000 7fe204f20640 1 -- 172.21.15.52:0/1676378979 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) v1 -- 0x7fe20002d210 con 0x7fe2001015f0 2024-03-04T09:29:03.381 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:29:02 smithi106 ceph-mon[36484]: pgmap v300: 129 pgs: 129 active+clean; 581 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-03-04T09:29:03.381 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:29:02 smithi106 ceph-mon[36484]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-03-04T09:29:03.381 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:29:02 smithi106 ceph-mon[36484]: from='client.? 172.21.15.52:0/936356160' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-03-04T09:29:03.381 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:29:02 smithi106 ceph-mon[36484]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-03-04T09:29:03.381 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:29:02 smithi106 ceph-mon[36484]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-04T09:29:03.381 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:29:02 smithi106 ceph-mon[36484]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2024-03-04T09:29:03.381 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:29:02 smithi106 ceph-mon[36484]: daemon mds.i is now active in filesystem cephfs as rank 1 2024-03-04T09:29:03.381 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:29:02 smithi106 ceph-mon[36484]: from='client.? 172.21.15.52:0/1336623953' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-03-04T09:29:03.994 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:03.997+0000 7fe1f77fe640 1 -- 172.21.15.52:0/1676378979 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v23) v1 ==== 115+0+0 (secure 0 0 0) 0x7fe1f80bc680 con 0x7fe2001015f0 2024-03-04T09:29:03.997 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:04.000+0000 7fe204f20640 1 -- 172.21.15.52:0/1676378979 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7fe1ec0a1d80 msgr2=0x7fe1ec0a4240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:29:03.997 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:04.000+0000 7fe204f20640 1 --2- 172.21.15.52:0/1676378979 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7fe1ec0a1d80 0x7fe1ec0a4240 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fe200198dd0 tx=0x7fe1e8005f90 comp rx=0 tx=0).stop 2024-03-04T09:29:03.997 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:04.000+0000 7fe204f20640 1 -- 172.21.15.52:0/1676378979 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe2001015f0 msgr2=0x7fe2001978c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:29:03.997 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:04.000+0000 7fe204f20640 1 --2- 172.21.15.52:0/1676378979 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe2001015f0 0x7fe2001978c0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fe1f8000c00 tx=0x7fe1f8005d50 comp rx=0 tx=0).stop 2024-03-04T09:29:03.997 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:04.000+0000 7fe204f20640 1 -- 172.21.15.52:0/1676378979 shutdown_connections 2024-03-04T09:29:03.997 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:04.000+0000 7fe204f20640 1 --2- 172.21.15.52:0/1676378979 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7fe1ec0a1d80 0x7fe1ec0a4240 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:29:03.997 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:04.000+0000 7fe204f20640 1 --2- 172.21.15.52:0/1676378979 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe2001031a0 0x7fe20019ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:29:03.997 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:04.000+0000 7fe204f20640 1 --2- 172.21.15.52:0/1676378979 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe2001027e0 0x7fe200197e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:29:03.997 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:04.000+0000 7fe204f20640 1 --2- 172.21.15.52:0/1676378979 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe2001015f0 0x7fe2001978c0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:29:03.997 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:04.000+0000 7fe204f20640 1 -- 172.21.15.52:0/1676378979 >> 172.21.15.52:0/1676378979 conn(0x7fe2000fcb60 msgr2=0x7fe200105bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:29:03.998 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:04.000+0000 7fe204f20640 1 -- 172.21.15.52:0/1676378979 shutdown_connections 2024-03-04T09:29:03.998 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:04.000+0000 7fe204f20640 1 -- 172.21.15.52:0/1676378979 wait complete. 2024-03-04T09:29:04.011 DEBUG:tasks.cephfs.filesystem:Creating {'create': 2, 'subvol_options': '--size 25000000000'} subvols for filesystem 'cephfs' 2024-03-04T09:29:04.012 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup create cephfs qa 2024-03-04T09:29:04.126 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:04.126+0000 7fdd20b1e640 1 -- 172.21.15.52:0/3818837358 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdd1c1016e0 msgr2=0x7fdd1c101b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:29:04.127 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:04.126+0000 7fdd20b1e640 1 --2- 172.21.15.52:0/3818837358 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdd1c1016e0 0x7fdd1c101b60 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fdd04009900 tx=0x7fdd0402bc30 comp rx=0 tx=0).stop 2024-03-04T09:29:04.127 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:04.126+0000 7fdd20b1e640 1 -- 172.21.15.52:0/3818837358 shutdown_connections 2024-03-04T09:29:04.127 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:04.126+0000 7fdd20b1e640 1 --2- 172.21.15.52:0/3818837358 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdd1c1020a0 0x7fdd1c108930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:29:04.127 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:04.126+0000 7fdd20b1e640 1 --2- 172.21.15.52:0/3818837358 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdd1c1016e0 0x7fdd1c101b60 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:29:04.127 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:04.126+0000 7fdd20b1e640 1 --2- 172.21.15.52:0/3818837358 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdd1c1004f0 0x7fdd1c100910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:29:04.127 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:04.126+0000 7fdd20b1e640 1 -- 172.21.15.52:0/3818837358 >> 172.21.15.52:0/3818837358 conn(0x7fdd1c0fba80 msgr2=0x7fdd1c0fdea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:29:04.127 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:04.127+0000 7fdd20b1e640 1 -- 172.21.15.52:0/3818837358 shutdown_connections 2024-03-04T09:29:04.127 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:04.127+0000 7fdd20b1e640 1 -- 172.21.15.52:0/3818837358 wait complete. 2024-03-04T09:29:04.127 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:04.127+0000 7fdd20b1e640 1 Processor -- start 2024-03-04T09:29:04.127 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:04.127+0000 7fdd20b1e640 1 -- start start 2024-03-04T09:29:04.127 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:04.127+0000 7fdd20b1e640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdd1c1004f0 0x7fdd1c197940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:29:04.127 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:04.127+0000 7fdd20b1e640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdd1c1016e0 0x7fdd1c197e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:29:04.128 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:04.127+0000 7fdd20b1e640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdd1c1020a0 0x7fdd1c19eea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:29:04.128 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:04.127+0000 7fdd20b1e640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd1c10c180 con 0x7fdd1c1016e0 2024-03-04T09:29:04.128 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:04.127+0000 7fdd20b1e640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd1c10c000 con 0x7fdd1c1020a0 2024-03-04T09:29:04.128 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:04.127+0000 7fdd20b1e640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd1c10c300 con 0x7fdd1c1004f0 2024-03-04T09:29:04.128 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:04.128+0000 7fdd19d9b640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdd1c1016e0 0x7fdd1c197e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:29:04.128 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:04.128+0000 7fdd19d9b640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdd1c1016e0 0x7fdd1c197e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:52854/0 (socket says 172.21.15.52:52854) 2024-03-04T09:29:04.128 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:04.128+0000 7fdd19d9b640 1 -- 172.21.15.52:0/184488527 learned_addr learned my addr 172.21.15.52:0/184488527 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-04T09:29:04.128 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:04.128+0000 7fdd1ad9d640 1 --2- 172.21.15.52:0/184488527 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdd1c1020a0 0x7fdd1c19eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:29:04.128 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:04.128+0000 7fdd1a59c640 1 --2- 172.21.15.52:0/184488527 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdd1c1004f0 0x7fdd1c197940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:29:04.128 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:04.128+0000 7fdd19d9b640 1 -- 172.21.15.52:0/184488527 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdd1c1004f0 msgr2=0x7fdd1c197940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:29:04.128 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:04.128+0000 7fdd19d9b640 1 --2- 172.21.15.52:0/184488527 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdd1c1004f0 0x7fdd1c197940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:29:04.128 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:04.128+0000 7fdd19d9b640 1 -- 172.21.15.52:0/184488527 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdd1c1020a0 msgr2=0x7fdd1c19eea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:29:04.128 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:04.128+0000 7fdd19d9b640 1 --2- 172.21.15.52:0/184488527 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdd1c1020a0 0x7fdd1c19eea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:29:04.129 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:04.128+0000 7fdd19d9b640 1 -- 172.21.15.52:0/184488527 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd1c19f510 con 0x7fdd1c1016e0 2024-03-04T09:29:04.129 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:04.128+0000 7fdd1ad9d640 1 --2- 172.21.15.52:0/184488527 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdd1c1020a0 0x7fdd1c19eea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-04T09:29:04.129 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:04.128+0000 7fdd19d9b640 1 --2- 172.21.15.52:0/184488527 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdd1c1016e0 0x7fdd1c197e80 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fdd04009230 tx=0x7fdd0402c650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:29:04.129 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:04.128+0000 7fdd137fe640 1 -- 172.21.15.52:0/184488527 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd04041070 con 0x7fdd1c1016e0 2024-03-04T09:29:04.129 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:04.128+0000 7fdd137fe640 1 -- 172.21.15.52:0/184488527 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fdd04004450 con 0x7fdd1c1016e0 2024-03-04T09:29:04.129 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:04.128+0000 7fdd20b1e640 1 -- 172.21.15.52:0/184488527 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd1c19f7a0 con 0x7fdd1c1016e0 2024-03-04T09:29:04.129 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:04.128+0000 7fdd137fe640 1 -- 172.21.15.52:0/184488527 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd04003db0 con 0x7fdd1c1016e0 2024-03-04T09:29:04.129 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:04.128+0000 7fdd20b1e640 1 -- 172.21.15.52:0/184488527 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd1c19fc90 con 0x7fdd1c1016e0 2024-03-04T09:29:04.129 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:04.129+0000 7fdd20b1e640 1 -- 172.21.15.52:0/184488527 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd1c19ffb0 con 0x7fdd1c1016e0 2024-03-04T09:29:04.130 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:04.130+0000 7fdd137fe640 1 -- 172.21.15.52:0/184488527 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 16) v1 ==== 136756+0+0 (secure 0 0 0) 0x7fdd04044070 con 0x7fdd1c1016e0 2024-03-04T09:29:04.130 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:04.131+0000 7fdd137fe640 1 --2- 172.21.15.52:0/184488527 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7fdd080a1d80 0x7fdd080a4240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:29:04.130 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:04.131+0000 7fdd137fe640 1 -- 172.21.15.52:0/184488527 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8802+0+0 (secure 0 0 0) 0x7fdd040eff50 con 0x7fdd1c1016e0 2024-03-04T09:29:04.130 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:04.131+0000 7fdd1a59c640 1 --2- 172.21.15.52:0/184488527 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7fdd080a1d80 0x7fdd080a4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:29:04.130 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:04.131+0000 7fdd1a59c640 1 --2- 172.21.15.52:0/184488527 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7fdd080a1d80 0x7fdd080a4240 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fdd1c101540 tx=0x7fdd14005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:29:04.131 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:04.135+0000 7fdd137fe640 1 -- 172.21.15.52:0/184488527 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fdd040bd140 con 0x7fdd1c1016e0 2024-03-04T09:29:04.335 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:29:03 smithi052 ceph-mon[32473]: from='client.? 172.21.15.52:0/1336623953' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-03-04T09:29:04.335 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:29:03 smithi052 ceph-mon[32473]: mds.? [v2:172.21.15.191:6838/2194324891,v1:172.21.15.191:6839/2194324891] up:active 2024-03-04T09:29:04.335 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:29:03 smithi052 ceph-mon[32473]: daemon mds.h assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-03-04T09:29:04.335 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:29:03 smithi052 ceph-mon[32473]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-03-04T09:29:04.335 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:29:03 smithi052 ceph-mon[32473]: Cluster is now healthy 2024-03-04T09:29:04.335 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:29:03 smithi052 ceph-mon[32473]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2024-03-04T09:29:04.335 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:29:03 smithi052 ceph-mon[32473]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:creating} 9 up:standby 2024-03-04T09:29:04.336 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:29:03 smithi052 ceph-mon[32473]: daemon mds.h is now active in filesystem cephfs as rank 2 2024-03-04T09:29:04.336 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:29:03 smithi052 ceph-mon[32473]: from='client.? 172.21.15.52:0/1676378979' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-03-04T09:29:04.336 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:29:03 smithi191 ceph-mon[36816]: from='client.? 172.21.15.52:0/1336623953' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-03-04T09:29:04.337 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:29:03 smithi191 ceph-mon[36816]: mds.? [v2:172.21.15.191:6838/2194324891,v1:172.21.15.191:6839/2194324891] up:active 2024-03-04T09:29:04.337 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:29:03 smithi191 ceph-mon[36816]: daemon mds.h assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-03-04T09:29:04.337 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:29:03 smithi191 ceph-mon[36816]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-03-04T09:29:04.337 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:29:03 smithi191 ceph-mon[36816]: Cluster is now healthy 2024-03-04T09:29:04.337 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:29:03 smithi191 ceph-mon[36816]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2024-03-04T09:29:04.337 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:29:03 smithi191 ceph-mon[36816]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:creating} 9 up:standby 2024-03-04T09:29:04.337 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:29:03 smithi191 ceph-mon[36816]: daemon mds.h is now active in filesystem cephfs as rank 2 2024-03-04T09:29:04.337 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:29:03 smithi191 ceph-mon[36816]: from='client.? 172.21.15.52:0/1676378979' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-03-04T09:29:04.339 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:04.342+0000 7fdd20b1e640 1 -- 172.21.15.52:0/184488527 --> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}) v1 -- 0x7fdd1c0fdc70 con 0x7fdd080a1d80 2024-03-04T09:29:04.358 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:04.361+0000 7fdd137fe640 1 -- 172.21.15.52:0/184488527 <== mgr.14152 v2:172.21.15.52:6800/3036033923 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fdd1c0fdc70 con 0x7fdd080a1d80 2024-03-04T09:29:04.360 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:04.363+0000 7fdd20b1e640 1 -- 172.21.15.52:0/184488527 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7fdd080a1d80 msgr2=0x7fdd080a4240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:29:04.360 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:04.363+0000 7fdd20b1e640 1 --2- 172.21.15.52:0/184488527 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7fdd080a1d80 0x7fdd080a4240 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fdd1c101540 tx=0x7fdd14005f90 comp rx=0 tx=0).stop 2024-03-04T09:29:04.361 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:04.363+0000 7fdd20b1e640 1 -- 172.21.15.52:0/184488527 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdd1c1016e0 msgr2=0x7fdd1c197e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:29:04.361 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:04.363+0000 7fdd20b1e640 1 --2- 172.21.15.52:0/184488527 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdd1c1016e0 0x7fdd1c197e80 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fdd04009230 tx=0x7fdd0402c650 comp rx=0 tx=0).stop 2024-03-04T09:29:04.361 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:04.363+0000 7fdd20b1e640 1 -- 172.21.15.52:0/184488527 shutdown_connections 2024-03-04T09:29:04.361 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:04.363+0000 7fdd20b1e640 1 --2- 172.21.15.52:0/184488527 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7fdd080a1d80 0x7fdd080a4240 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:29:04.361 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:04.363+0000 7fdd20b1e640 1 --2- 172.21.15.52:0/184488527 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdd1c1020a0 0x7fdd1c19eea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:29:04.361 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:04.363+0000 7fdd20b1e640 1 --2- 172.21.15.52:0/184488527 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdd1c1016e0 0x7fdd1c197e80 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:29:04.361 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:04.363+0000 7fdd20b1e640 1 --2- 172.21.15.52:0/184488527 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdd1c1004f0 0x7fdd1c197940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:29:04.362 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:04.363+0000 7fdd20b1e640 1 -- 172.21.15.52:0/184488527 >> 172.21.15.52:0/184488527 conn(0x7fdd1c0fba80 msgr2=0x7fdd1c0fd560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:29:04.362 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:04.363+0000 7fdd20b1e640 1 -- 172.21.15.52:0/184488527 shutdown_connections 2024-03-04T09:29:04.362 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:04.363+0000 7fdd20b1e640 1 -- 172.21.15.52:0/184488527 wait complete. 2024-03-04T09:29:04.371 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_0 --group_name qa 2024-03-04T09:29:04.381 INFO:journalctl@ceph.mds.b.smithi106.stdout:Mar 04 09:29:04 smithi106 ceph-4e059f08-da08-11ee-95c6-87774f69a715-mds-b[66655]: 2024-03-04T09:29:04.228+0000 7fd99c4b7700 -1 mds.pinger is_rank_lagging: rank=2 was never sent ping request. 2024-03-04T09:29:04.381 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:29:03 smithi106 ceph-mon[36484]: from='client.? 172.21.15.52:0/1336623953' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-03-04T09:29:04.382 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:29:03 smithi106 ceph-mon[36484]: mds.? [v2:172.21.15.191:6838/2194324891,v1:172.21.15.191:6839/2194324891] up:active 2024-03-04T09:29:04.382 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:29:03 smithi106 ceph-mon[36484]: daemon mds.h assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-03-04T09:29:04.382 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:29:03 smithi106 ceph-mon[36484]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-03-04T09:29:04.382 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:29:03 smithi106 ceph-mon[36484]: Cluster is now healthy 2024-03-04T09:29:04.382 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:29:03 smithi106 ceph-mon[36484]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2024-03-04T09:29:04.382 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:29:03 smithi106 ceph-mon[36484]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:creating} 9 up:standby 2024-03-04T09:29:04.383 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:29:03 smithi106 ceph-mon[36484]: daemon mds.h is now active in filesystem cephfs as rank 2 2024-03-04T09:29:04.383 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:29:03 smithi106 ceph-mon[36484]: from='client.? 172.21.15.52:0/1676378979' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-03-04T09:29:04.481 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:04.482+0000 7f3754d5e640 1 -- 172.21.15.52:0/2970916496 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f37501004e0 msgr2=0x7f3750100900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:29:04.481 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:04.482+0000 7f3754d5e640 1 --2- 172.21.15.52:0/2970916496 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f37501004e0 0x7f3750100900 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f3748009860 tx=0x7f374802bb70 comp rx=0 tx=0).stop 2024-03-04T09:29:04.481 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:04.482+0000 7f3754d5e640 1 -- 172.21.15.52:0/2970916496 shutdown_connections 2024-03-04T09:29:04.481 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:04.482+0000 7f3754d5e640 1 --2- 172.21.15.52:0/2970916496 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3750102090 0x7f3750108920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:29:04.482 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:04.482+0000 7f3754d5e640 1 --2- 172.21.15.52:0/2970916496 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f37501016d0 0x7f3750101b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:29:04.482 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:04.482+0000 7f3754d5e640 1 --2- 172.21.15.52:0/2970916496 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f37501004e0 0x7f3750100900 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:29:04.482 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:04.482+0000 7f3754d5e640 1 -- 172.21.15.52:0/2970916496 >> 172.21.15.52:0/2970916496 conn(0x7f37500fba70 msgr2=0x7f37500fde90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:29:04.482 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:04.482+0000 7f3754d5e640 1 -- 172.21.15.52:0/2970916496 shutdown_connections 2024-03-04T09:29:04.482 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:04.482+0000 7f3754d5e640 1 -- 172.21.15.52:0/2970916496 wait complete. 2024-03-04T09:29:04.482 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:04.483+0000 7f3754d5e640 1 Processor -- start 2024-03-04T09:29:04.482 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:04.483+0000 7f3754d5e640 1 -- start start 2024-03-04T09:29:04.482 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:04.483+0000 7f3754d5e640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f37501004e0 0x7f3750196880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:29:04.482 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:04.483+0000 7f3754d5e640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f37501016d0 0x7f3750196dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:29:04.482 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:04.483+0000 7f3754d5e640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3750102090 0x7f375019de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:29:04.482 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:04.483+0000 7f3754d5e640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f375010c170 con 0x7f37501016d0 2024-03-04T09:29:04.482 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:04.483+0000 7f3754d5e640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f375010bff0 con 0x7f3750102090 2024-03-04T09:29:04.482 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:04.483+0000 7f3754d5e640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f375010c2f0 con 0x7f37501004e0 2024-03-04T09:29:04.483 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:04.483+0000 7f374dd9b640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f37501016d0 0x7f3750196dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:29:04.483 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:04.483+0000 7f374dd9b640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f37501016d0 0x7f3750196dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:46922/0 (socket says 172.21.15.52:46922) 2024-03-04T09:29:04.483 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:04.483+0000 7f374dd9b640 1 -- 172.21.15.52:0/4000677434 learned_addr learned my addr 172.21.15.52:0/4000677434 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-04T09:29:04.483 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:04.483+0000 7f374e59c640 1 --2- 172.21.15.52:0/4000677434 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f37501004e0 0x7f3750196880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:29:04.483 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:04.483+0000 7f374ed9d640 1 --2- 172.21.15.52:0/4000677434 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3750102090 0x7f375019de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:29:04.483 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:04.483+0000 7f374dd9b640 1 -- 172.21.15.52:0/4000677434 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f37501004e0 msgr2=0x7f3750196880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:29:04.483 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:04.483+0000 7f374dd9b640 1 --2- 172.21.15.52:0/4000677434 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f37501004e0 0x7f3750196880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:29:04.483 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:04.483+0000 7f374dd9b640 1 -- 172.21.15.52:0/4000677434 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3750102090 msgr2=0x7f375019de30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:29:04.483 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:04.483+0000 7f374dd9b640 1 --2- 172.21.15.52:0/4000677434 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3750102090 0x7f375019de30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:29:04.483 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:04.483+0000 7f374dd9b640 1 -- 172.21.15.52:0/4000677434 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f375019e4a0 con 0x7f37501016d0 2024-03-04T09:29:04.483 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:04.483+0000 7f374e59c640 1 --2- 172.21.15.52:0/4000677434 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f37501004e0 0x7f3750196880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-04T09:29:04.483 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:04.483+0000 7f374ed9d640 1 --2- 172.21.15.52:0/4000677434 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3750102090 0x7f375019de30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-04T09:29:04.483 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:04.483+0000 7f374dd9b640 1 --2- 172.21.15.52:0/4000677434 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f37501016d0 0x7f3750196dc0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f373800e9f0 tx=0x7f373800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:29:04.483 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:04.484+0000 7f37477fe640 1 -- 172.21.15.52:0/4000677434 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f373800cd80 con 0x7f37501016d0 2024-03-04T09:29:04.483 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:04.484+0000 7f3754d5e640 1 -- 172.21.15.52:0/4000677434 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f375019e790 con 0x7f37501016d0 2024-03-04T09:29:04.484 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:04.484+0000 7f37477fe640 1 -- 172.21.15.52:0/4000677434 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3738005500 con 0x7f37501016d0 2024-03-04T09:29:04.484 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:04.484+0000 7f37477fe640 1 -- 172.21.15.52:0/4000677434 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3738010690 con 0x7f37501016d0 2024-03-04T09:29:04.484 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:04.484+0000 7f3754d5e640 1 -- 172.21.15.52:0/4000677434 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f375019ece0 con 0x7f37501016d0 2024-03-04T09:29:04.485 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:04.485+0000 7f3754d5e640 1 -- 172.21.15.52:0/4000677434 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3750107820 con 0x7f37501016d0 2024-03-04T09:29:04.485 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:04.486+0000 7f37477fe640 1 -- 172.21.15.52:0/4000677434 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 16) v1 ==== 136756+0+0 (secure 0 0 0) 0x7f37380046c0 con 0x7f37501016d0 2024-03-04T09:29:04.485 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:04.486+0000 7f37477fe640 1 --2- 172.21.15.52:0/4000677434 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f373c0a1d80 0x7f373c0a4240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:29:04.485 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:04.486+0000 7f374e59c640 1 --2- 172.21.15.52:0/4000677434 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f373c0a1d80 0x7f373c0a4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:29:04.485 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:04.487+0000 7f37477fe640 1 -- 172.21.15.52:0/4000677434 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8802+0+0 (secure 0 0 0) 0x7f37380ce3c0 con 0x7f37501016d0 2024-03-04T09:29:04.485 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:04.487+0000 7f374e59c640 1 --2- 172.21.15.52:0/4000677434 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f373c0a1d80 0x7f373c0a4240 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f3748009830 tx=0x7f374802e040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:29:04.487 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:04.491+0000 7f37477fe640 1 -- 172.21.15.52:0/4000677434 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f373809b5b0 con 0x7f37501016d0 2024-03-04T09:29:04.701 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:04.704+0000 7f3754d5e640 1 -- 172.21.15.52:0/4000677434 --> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}) v1 -- 0x7f3750061a10 con 0x7f373c0a1d80 2024-03-04T09:29:04.720 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:04.723+0000 7f37477fe640 1 -- 172.21.15.52:0/4000677434 <== mgr.14152 v2:172.21.15.52:6800/3036033923 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f3750061a10 con 0x7f373c0a1d80 2024-03-04T09:29:04.722 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:04.725+0000 7f3754d5e640 1 -- 172.21.15.52:0/4000677434 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f373c0a1d80 msgr2=0x7f373c0a4240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:29:04.723 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:04.725+0000 7f3754d5e640 1 --2- 172.21.15.52:0/4000677434 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f373c0a1d80 0x7f373c0a4240 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f3748009830 tx=0x7f374802e040 comp rx=0 tx=0).stop 2024-03-04T09:29:04.723 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:04.725+0000 7f3754d5e640 1 -- 172.21.15.52:0/4000677434 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f37501016d0 msgr2=0x7f3750196dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:29:04.723 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:04.725+0000 7f3754d5e640 1 --2- 172.21.15.52:0/4000677434 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f37501016d0 0x7f3750196dc0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f373800e9f0 tx=0x7f373800edf0 comp rx=0 tx=0).stop 2024-03-04T09:29:04.723 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:04.725+0000 7f3754d5e640 1 -- 172.21.15.52:0/4000677434 shutdown_connections 2024-03-04T09:29:04.723 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:04.725+0000 7f3754d5e640 1 --2- 172.21.15.52:0/4000677434 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f373c0a1d80 0x7f373c0a4240 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:29:04.723 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:04.725+0000 7f3754d5e640 1 --2- 172.21.15.52:0/4000677434 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3750102090 0x7f375019de30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:29:04.723 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:04.725+0000 7f3754d5e640 1 --2- 172.21.15.52:0/4000677434 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f37501016d0 0x7f3750196dc0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:29:04.723 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:04.725+0000 7f3754d5e640 1 --2- 172.21.15.52:0/4000677434 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f37501004e0 0x7f3750196880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:29:04.723 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:04.725+0000 7f3754d5e640 1 -- 172.21.15.52:0/4000677434 >> 172.21.15.52:0/4000677434 conn(0x7f37500fba70 msgr2=0x7f37501049f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:29:04.723 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:04.725+0000 7f3754d5e640 1 -- 172.21.15.52:0/4000677434 shutdown_connections 2024-03-04T09:29:04.723 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:04.726+0000 7f3754d5e640 1 -- 172.21.15.52:0/4000677434 wait complete. 2024-03-04T09:29:04.732 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Mar 04 09:29:04 smithi052 ceph-4e059f08-da08-11ee-95c6-87774f69a715-mgr-x[32705]: 2024-03-04T09:29:04.347+0000 7f9c12c24700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-04T09:29:04.732 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Mar 04 09:29:04 smithi052 ceph-4e059f08-da08-11ee-95c6-87774f69a715-mgr-x[32705]: 2024-03-04T09:29:04.347+0000 7f9c12c24700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-04T09:29:04.732 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Mar 04 09:29:04 smithi052 ceph-4e059f08-da08-11ee-95c6-87774f69a715-mgr-x[32705]: 2024-03-04T09:29:04.347+0000 7f9c12c24700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-04T09:29:04.732 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Mar 04 09:29:04 smithi052 ceph-4e059f08-da08-11ee-95c6-87774f69a715-mgr-x[32705]: 2024-03-04T09:29:04.347+0000 7f9c12c24700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-04T09:29:04.732 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Mar 04 09:29:04 smithi052 ceph-4e059f08-da08-11ee-95c6-87774f69a715-mgr-x[32705]: 2024-03-04T09:29:04.347+0000 7f9c12c24700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-04T09:29:04.733 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs --group_name qa sv_0 2024-03-04T09:29:04.843 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:04.844+0000 7f3b2118a640 1 -- 172.21.15.52:0/2873097410 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3b1c1004f0 msgr2=0x7f3b1c100910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:29:04.843 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:04.844+0000 7f3b2118a640 1 --2- 172.21.15.52:0/2873097410 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3b1c1004f0 0x7f3b1c100910 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f3b0c009900 tx=0x7f3b0c02bc30 comp rx=0 tx=0).stop 2024-03-04T09:29:04.843 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:04.844+0000 7f3b2118a640 1 -- 172.21.15.52:0/2873097410 shutdown_connections 2024-03-04T09:29:04.843 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:04.844+0000 7f3b2118a640 1 --2- 172.21.15.52:0/2873097410 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3b1c1020a0 0x7f3b1c108930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:29:04.843 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:04.844+0000 7f3b2118a640 1 --2- 172.21.15.52:0/2873097410 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3b1c1016e0 0x7f3b1c101b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:29:04.843 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:04.844+0000 7f3b2118a640 1 --2- 172.21.15.52:0/2873097410 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3b1c1004f0 0x7f3b1c100910 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:29:04.843 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:04.844+0000 7f3b2118a640 1 -- 172.21.15.52:0/2873097410 >> 172.21.15.52:0/2873097410 conn(0x7f3b1c0fba80 msgr2=0x7f3b1c0fdea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:29:04.843 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:04.844+0000 7f3b2118a640 1 -- 172.21.15.52:0/2873097410 shutdown_connections 2024-03-04T09:29:04.843 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:04.844+0000 7f3b2118a640 1 -- 172.21.15.52:0/2873097410 wait complete. 2024-03-04T09:29:04.843 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:04.844+0000 7f3b2118a640 1 Processor -- start 2024-03-04T09:29:04.843 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:04.844+0000 7f3b2118a640 1 -- start start 2024-03-04T09:29:04.844 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:04.845+0000 7f3b2118a640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3b1c1004f0 0x7f3b1c197960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:29:04.844 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:04.845+0000 7f3b2118a640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3b1c1016e0 0x7f3b1c197ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:29:04.844 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:04.845+0000 7f3b2118a640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3b1c1020a0 0x7f3b1c19ef10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:29:04.844 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:04.845+0000 7f3b2118a640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b1c10c180 con 0x7f3b1c1020a0 2024-03-04T09:29:04.844 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:04.845+0000 7f3b2118a640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b1c10c000 con 0x7f3b1c1016e0 2024-03-04T09:29:04.844 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:04.845+0000 7f3b2118a640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b1c10c300 con 0x7f3b1c1004f0 2024-03-04T09:29:04.844 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:04.845+0000 7f3b1b59e640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3b1c1020a0 0x7f3b1c19ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:29:04.844 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:04.845+0000 7f3b1b59e640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3b1c1020a0 0x7f3b1c19ef10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:46938/0 (socket says 172.21.15.52:46938) 2024-03-04T09:29:04.844 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:04.845+0000 7f3b1b59e640 1 -- 172.21.15.52:0/3965727234 learned_addr learned my addr 172.21.15.52:0/3965727234 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-04T09:29:04.844 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:04.845+0000 7f3b13fff640 1 --2- 172.21.15.52:0/3965727234 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3b1c1016e0 0x7f3b1c197ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:29:04.844 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:04.845+0000 7f3b1ad9d640 1 --2- 172.21.15.52:0/3965727234 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3b1c1004f0 0x7f3b1c197960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:29:04.844 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:04.845+0000 7f3b1b59e640 1 -- 172.21.15.52:0/3965727234 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3b1c1004f0 msgr2=0x7f3b1c197960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:29:04.844 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:04.845+0000 7f3b1b59e640 1 --2- 172.21.15.52:0/3965727234 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3b1c1004f0 0x7f3b1c197960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:29:04.845 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:04.845+0000 7f3b1b59e640 1 -- 172.21.15.52:0/3965727234 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3b1c1016e0 msgr2=0x7f3b1c197ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:29:04.845 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:04.845+0000 7f3b1b59e640 1 --2- 172.21.15.52:0/3965727234 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3b1c1016e0 0x7f3b1c197ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:29:04.845 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:04.845+0000 7f3b1b59e640 1 -- 172.21.15.52:0/3965727234 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b1c19f580 con 0x7f3b1c1020a0 2024-03-04T09:29:04.845 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:04.845+0000 7f3b1b59e640 1 --2- 172.21.15.52:0/3965727234 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3b1c1020a0 0x7f3b1c19ef10 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f3b14009fd0 tx=0x7f3b1400b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:29:04.845 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:04.845+0000 7f3b18d99640 1 -- 172.21.15.52:0/3965727234 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b14017070 con 0x7f3b1c1020a0 2024-03-04T09:29:04.845 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:04.845+0000 7f3b18d99640 1 -- 172.21.15.52:0/3965727234 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3b14003e50 con 0x7f3b1c1020a0 2024-03-04T09:29:04.845 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:04.845+0000 7f3b2118a640 1 -- 172.21.15.52:0/3965727234 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b1c19f870 con 0x7f3b1c1020a0 2024-03-04T09:29:04.845 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:04.845+0000 7f3b18d99640 1 -- 172.21.15.52:0/3965727234 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b140106c0 con 0x7f3b1c1020a0 2024-03-04T09:29:04.845 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:04.845+0000 7f3b2118a640 1 -- 172.21.15.52:0/3965727234 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b1c19fd70 con 0x7f3b1c1020a0 2024-03-04T09:29:04.846 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:04.846+0000 7f3b2118a640 1 -- 172.21.15.52:0/3965727234 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3b1c19ffd0 con 0x7f3b1c1020a0 2024-03-04T09:29:04.846 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:04.847+0000 7f3b18d99640 1 -- 172.21.15.52:0/3965727234 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 16) v1 ==== 136756+0+0 (secure 0 0 0) 0x7f3b14003970 con 0x7f3b1c1020a0 2024-03-04T09:29:04.846 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:04.848+0000 7f3b18d99640 1 --2- 172.21.15.52:0/3965727234 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f3b080a1d80 0x7f3b080a4240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:29:04.846 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:04.848+0000 7f3b1ad9d640 1 --2- 172.21.15.52:0/3965727234 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f3b080a1d80 0x7f3b080a4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:29:04.846 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:04.848+0000 7f3b18d99640 1 -- 172.21.15.52:0/3965727234 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8802+0+0 (secure 0 0 0) 0x7f3b140cdd10 con 0x7f3b1c1020a0 2024-03-04T09:29:04.847 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:04.848+0000 7f3b1ad9d640 1 --2- 172.21.15.52:0/3965727234 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f3b080a1d80 0x7f3b080a4240 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f3b0c000c00 tx=0x7f3b0c005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:29:04.849 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:04.852+0000 7f3b18d99640 1 -- 172.21.15.52:0/3965727234 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f3b1409a8b0 con 0x7f3b1c1020a0 2024-03-04T09:29:05.061 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:05.064+0000 7f3b2118a640 1 -- 172.21.15.52:0/3965727234 --> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}) v1 -- 0x7f3b1c0fdca0 con 0x7f3b080a1d80 2024-03-04T09:29:05.181 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:05.068+0000 7f3b18d99640 1 -- 172.21.15.52:0/3965727234 <== mgr.14152 v2:172.21.15.52:6800/3036033923 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+54 (secure 0 0 0) 0x7f3b1c0fdca0 con 0x7f3b080a1d80 2024-03-04T09:29:05.181 INFO:teuthology.orchestra.run.smithi052.stdout:/volumes/qa/sv_0/366945d4-d137-4e51-a697-7b343b46c017 2024-03-04T09:29:05.182 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:05.070+0000 7f3b2118a640 1 -- 172.21.15.52:0/3965727234 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f3b080a1d80 msgr2=0x7f3b080a4240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:29:05.182 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:05.070+0000 7f3b2118a640 1 --2- 172.21.15.52:0/3965727234 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f3b080a1d80 0x7f3b080a4240 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f3b0c000c00 tx=0x7f3b0c005fd0 comp rx=0 tx=0).stop 2024-03-04T09:29:05.182 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:05.070+0000 7f3b2118a640 1 -- 172.21.15.52:0/3965727234 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3b1c1020a0 msgr2=0x7f3b1c19ef10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:29:05.182 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:05.070+0000 7f3b2118a640 1 --2- 172.21.15.52:0/3965727234 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3b1c1020a0 0x7f3b1c19ef10 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f3b14009fd0 tx=0x7f3b1400b260 comp rx=0 tx=0).stop 2024-03-04T09:29:05.182 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:05.070+0000 7f3b2118a640 1 -- 172.21.15.52:0/3965727234 shutdown_connections 2024-03-04T09:29:05.182 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:05.070+0000 7f3b2118a640 1 --2- 172.21.15.52:0/3965727234 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f3b080a1d80 0x7f3b080a4240 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:29:05.183 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:05.070+0000 7f3b2118a640 1 --2- 172.21.15.52:0/3965727234 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3b1c1020a0 0x7f3b1c19ef10 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:29:05.183 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:05.070+0000 7f3b2118a640 1 --2- 172.21.15.52:0/3965727234 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3b1c1016e0 0x7f3b1c197ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:29:05.183 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:05.070+0000 7f3b2118a640 1 --2- 172.21.15.52:0/3965727234 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3b1c1004f0 0x7f3b1c197960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:29:05.183 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:05.070+0000 7f3b2118a640 1 -- 172.21.15.52:0/3965727234 >> 172.21.15.52:0/3965727234 conn(0x7f3b1c0fba80 msgr2=0x7f3b1c0fd590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:29:05.183 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:05.070+0000 7f3b2118a640 1 -- 172.21.15.52:0/3965727234 shutdown_connections 2024-03-04T09:29:05.183 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:05.070+0000 7f3b2118a640 1 -- 172.21.15.52:0/3965727234 wait complete. 2024-03-04T09:29:05.183 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_1 --group_name qa 2024-03-04T09:29:05.298 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:05.298+0000 7f9c8b9c7640 1 -- 172.21.15.52:0/1518991037 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9c84107310 msgr2=0x7f9c84109700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:29:05.298 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:05.298+0000 7f9c8b9c7640 1 --2- 172.21.15.52:0/1518991037 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9c84107310 0x7f9c84109700 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f9c74009ca0 tx=0x7f9c7402c0f0 comp rx=0 tx=0).stop 2024-03-04T09:29:05.298 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:05.298+0000 7f9c8b9c7640 1 -- 172.21.15.52:0/1518991037 shutdown_connections 2024-03-04T09:29:05.298 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:05.298+0000 7f9c8b9c7640 1 --2- 172.21.15.52:0/1518991037 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9c84107310 0x7f9c84109700 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:29:05.298 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:05.298+0000 7f9c8b9c7640 1 --2- 172.21.15.52:0/1518991037 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9c841049e0 0x7f9c84106dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:29:05.299 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:05.298+0000 7f9c8b9c7640 1 --2- 172.21.15.52:0/1518991037 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9c841020b0 0x7f9c841044a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:29:05.299 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:05.298+0000 7f9c8b9c7640 1 -- 172.21.15.52:0/1518991037 >> 172.21.15.52:0/1518991037 conn(0x7f9c840fba60 msgr2=0x7f9c840fde80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:29:05.299 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:05.299+0000 7f9c8b9c7640 1 -- 172.21.15.52:0/1518991037 shutdown_connections 2024-03-04T09:29:05.299 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:05.299+0000 7f9c8b9c7640 1 -- 172.21.15.52:0/1518991037 wait complete. 2024-03-04T09:29:05.299 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:05.299+0000 7f9c8b9c7640 1 Processor -- start 2024-03-04T09:29:05.299 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:05.299+0000 7f9c8b9c7640 1 -- start start 2024-03-04T09:29:05.299 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:05.299+0000 7f9c8b9c7640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9c841020b0 0x7f9c84101a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:29:05.299 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:05.299+0000 7f9c8b9c7640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9c841049e0 0x7f9c84100060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:29:05.299 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:05.299+0000 7f9c8b9c7640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9c84107310 0x7f9c841005a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:29:05.299 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:05.299+0000 7f9c8b9c7640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c8410d260 con 0x7f9c841020b0 2024-03-04T09:29:05.299 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:05.299+0000 7f9c8b9c7640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c8410d0e0 con 0x7f9c841049e0 2024-03-04T09:29:05.300 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:05.299+0000 7f9c8b9c7640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c8410d3e0 con 0x7f9c84107310 2024-03-04T09:29:05.300 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:05.299+0000 7f9c89763640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9c841020b0 0x7f9c84101a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:29:05.300 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:05.299+0000 7f9c89763640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9c841020b0 0x7f9c84101a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:46970/0 (socket says 172.21.15.52:46970) 2024-03-04T09:29:05.300 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:05.299+0000 7f9c89763640 1 -- 172.21.15.52:0/3413796356 learned_addr learned my addr 172.21.15.52:0/3413796356 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-04T09:29:05.300 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:05.300+0000 7f9c88f62640 1 --2- 172.21.15.52:0/3413796356 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9c841049e0 0x7f9c84100060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:29:05.300 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:05.300+0000 7f9c89f64640 1 --2- 172.21.15.52:0/3413796356 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9c84107310 0x7f9c841005a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:29:05.300 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:05.300+0000 7f9c89763640 1 -- 172.21.15.52:0/3413796356 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9c84107310 msgr2=0x7f9c841005a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:29:05.301 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:05.300+0000 7f9c89763640 1 --2- 172.21.15.52:0/3413796356 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9c84107310 0x7f9c841005a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:29:05.301 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:05.300+0000 7f9c89763640 1 -- 172.21.15.52:0/3413796356 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9c841049e0 msgr2=0x7f9c84100060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:29:05.301 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:05.300+0000 7f9c89763640 1 --2- 172.21.15.52:0/3413796356 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9c841049e0 0x7f9c84100060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:29:05.301 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:05.300+0000 7f9c89763640 1 -- 172.21.15.52:0/3413796356 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c84100e30 con 0x7f9c841020b0 2024-03-04T09:29:05.301 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:05.300+0000 7f9c89f64640 1 --2- 172.21.15.52:0/3413796356 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9c84107310 0x7f9c841005a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T09:29:05.301 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:05.300+0000 7f9c89763640 1 --2- 172.21.15.52:0/3413796356 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9c841020b0 0x7f9c84101a20 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f9c7c00ea60 tx=0x7f9c7c00ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:29:05.301 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:05.300+0000 7f9c827fc640 1 -- 172.21.15.52:0/3413796356 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c7c00cde0 con 0x7f9c841020b0 2024-03-04T09:29:05.301 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:05.300+0000 7f9c8b9c7640 1 -- 172.21.15.52:0/3413796356 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c841a1a00 con 0x7f9c841020b0 2024-03-04T09:29:05.302 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:05.300+0000 7f9c827fc640 1 -- 172.21.15.52:0/3413796356 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9c7c0035f0 con 0x7f9c841020b0 2024-03-04T09:29:05.302 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:05.300+0000 7f9c827fc640 1 -- 172.21.15.52:0/3413796356 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c7c010670 con 0x7f9c841020b0 2024-03-04T09:29:05.302 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:05.300+0000 7f9c8b9c7640 1 -- 172.21.15.52:0/3413796356 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c841a1f50 con 0x7f9c841020b0 2024-03-04T09:29:05.303 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:05.301+0000 7f9c8b9c7640 1 -- 172.21.15.52:0/3413796356 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9c840fd550 con 0x7f9c841020b0 2024-03-04T09:29:05.303 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:05.302+0000 7f9c827fc640 1 -- 172.21.15.52:0/3413796356 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 16) v1 ==== 136756+0+0 (secure 0 0 0) 0x7f9c7c0037a0 con 0x7f9c841020b0 2024-03-04T09:29:05.303 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:05.303+0000 7f9c827fc640 1 --2- 172.21.15.52:0/3413796356 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f9c6c0a1d80 0x7f9c6c0a4240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:29:05.303 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:05.303+0000 7f9c88f62640 1 --2- 172.21.15.52:0/3413796356 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f9c6c0a1d80 0x7f9c6c0a4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:29:05.303 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:05.303+0000 7f9c827fc640 1 -- 172.21.15.52:0/3413796356 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8802+0+0 (secure 0 0 0) 0x7f9c7c0ce4c0 con 0x7f9c841020b0 2024-03-04T09:29:05.303 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:05.303+0000 7f9c88f62640 1 --2- 172.21.15.52:0/3413796356 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f9c6c0a1d80 0x7f9c6c0a4240 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f9c84101790 tx=0x7f9c78005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:29:05.304 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:05.306+0000 7f9c827fc640 1 -- 172.21.15.52:0/3413796356 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f9c7c09b740 con 0x7f9c841020b0 2024-03-04T09:29:05.335 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:29:04 smithi052 ceph-mon[32473]: pgmap v301: 129 pgs: 129 active+clean; 581 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s wr, 0 op/s 2024-03-04T09:29:05.335 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:29:04 smithi052 ceph-mon[32473]: from='client.? 172.21.15.52:0/1676378979' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-03-04T09:29:05.335 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:29:04 smithi052 ceph-mon[32473]: mds.? [v2:172.21.15.106:6836/2901755066,v1:172.21.15.106:6837/2901755066] up:active 2024-03-04T09:29:05.336 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:29:04 smithi052 ceph-mon[32473]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 9 up:standby 2024-03-04T09:29:05.336 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:29:04 smithi052 ceph-mon[32473]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-03-04T09:29:05.336 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:29:04 smithi052 ceph-mon[32473]: from='client.14889 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T09:29:05.336 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:29:04 smithi191 ceph-mon[36816]: pgmap v301: 129 pgs: 129 active+clean; 581 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s wr, 0 op/s 2024-03-04T09:29:05.336 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:29:04 smithi191 ceph-mon[36816]: from='client.? 172.21.15.52:0/1676378979' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-03-04T09:29:05.336 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:29:04 smithi191 ceph-mon[36816]: mds.? [v2:172.21.15.106:6836/2901755066,v1:172.21.15.106:6837/2901755066] up:active 2024-03-04T09:29:05.337 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:29:04 smithi191 ceph-mon[36816]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 9 up:standby 2024-03-04T09:29:05.337 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:29:04 smithi191 ceph-mon[36816]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-03-04T09:29:05.337 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:29:04 smithi191 ceph-mon[36816]: from='client.14889 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T09:29:05.381 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:29:04 smithi106 ceph-mon[36484]: pgmap v301: 129 pgs: 129 active+clean; 581 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s wr, 0 op/s 2024-03-04T09:29:05.381 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:29:04 smithi106 ceph-mon[36484]: from='client.? 172.21.15.52:0/1676378979' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-03-04T09:29:05.381 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:29:04 smithi106 ceph-mon[36484]: mds.? [v2:172.21.15.106:6836/2901755066,v1:172.21.15.106:6837/2901755066] up:active 2024-03-04T09:29:05.381 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:29:04 smithi106 ceph-mon[36484]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 9 up:standby 2024-03-04T09:29:05.381 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:29:04 smithi106 ceph-mon[36484]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-03-04T09:29:05.381 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:29:04 smithi106 ceph-mon[36484]: from='client.14889 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T09:29:05.515 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:05.518+0000 7f9c8b9c7640 1 -- 172.21.15.52:0/3413796356 --> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}) v1 -- 0x7f9c84061a10 con 0x7f9c6c0a1d80 2024-03-04T09:29:05.534 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:05.538+0000 7f9c827fc640 1 -- 172.21.15.52:0/3413796356 <== mgr.14152 v2:172.21.15.52:6800/3036033923 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f9c84061a10 con 0x7f9c6c0a1d80 2024-03-04T09:29:05.537 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:05.540+0000 7f9c8b9c7640 1 -- 172.21.15.52:0/3413796356 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f9c6c0a1d80 msgr2=0x7f9c6c0a4240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:29:05.537 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:05.540+0000 7f9c8b9c7640 1 --2- 172.21.15.52:0/3413796356 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f9c6c0a1d80 0x7f9c6c0a4240 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f9c84101790 tx=0x7f9c78005f90 comp rx=0 tx=0).stop 2024-03-04T09:29:05.537 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:05.540+0000 7f9c8b9c7640 1 -- 172.21.15.52:0/3413796356 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9c841020b0 msgr2=0x7f9c84101a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:29:05.537 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:05.540+0000 7f9c8b9c7640 1 --2- 172.21.15.52:0/3413796356 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9c841020b0 0x7f9c84101a20 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f9c7c00ea60 tx=0x7f9c7c00ef20 comp rx=0 tx=0).stop 2024-03-04T09:29:05.537 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:05.540+0000 7f9c8b9c7640 1 -- 172.21.15.52:0/3413796356 shutdown_connections 2024-03-04T09:29:05.538 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:05.540+0000 7f9c8b9c7640 1 --2- 172.21.15.52:0/3413796356 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f9c6c0a1d80 0x7f9c6c0a4240 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:29:05.538 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:05.540+0000 7f9c8b9c7640 1 --2- 172.21.15.52:0/3413796356 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9c84107310 0x7f9c841005a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:29:05.538 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:05.540+0000 7f9c8b9c7640 1 --2- 172.21.15.52:0/3413796356 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9c841049e0 0x7f9c84100060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:29:05.538 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:05.540+0000 7f9c8b9c7640 1 --2- 172.21.15.52:0/3413796356 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9c841020b0 0x7f9c84101a20 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:29:05.538 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:05.540+0000 7f9c8b9c7640 1 -- 172.21.15.52:0/3413796356 >> 172.21.15.52:0/3413796356 conn(0x7f9c840fba60 msgr2=0x7f9c841029f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:29:05.538 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:05.540+0000 7f9c8b9c7640 1 -- 172.21.15.52:0/3413796356 shutdown_connections 2024-03-04T09:29:05.538 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:05.540+0000 7f9c8b9c7640 1 -- 172.21.15.52:0/3413796356 wait complete. 2024-03-04T09:29:05.548 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs --group_name qa sv_1 2024-03-04T09:29:05.660 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:05.658+0000 7f23fb9f7640 1 -- 172.21.15.52:0/2937629629 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f23f4103240 msgr2=0x7f23f4109ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:29:05.660 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:05.658+0000 7f23fb9f7640 1 --2- 172.21.15.52:0/2937629629 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f23f4103240 0x7f23f4109ad0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f23ec009cd0 tx=0x7f23ec02c0f0 comp rx=0 tx=0).stop 2024-03-04T09:29:05.660 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:05.658+0000 7f23fb9f7640 1 -- 172.21.15.52:0/2937629629 shutdown_connections 2024-03-04T09:29:05.660 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:05.658+0000 7f23fb9f7640 1 --2- 172.21.15.52:0/2937629629 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f23f4103240 0x7f23f4109ad0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:29:05.660 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:05.658+0000 7f23fb9f7640 1 --2- 172.21.15.52:0/2937629629 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f23f4102880 0x7f23f4102d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:29:05.661 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:05.658+0000 7f23fb9f7640 1 --2- 172.21.15.52:0/2937629629 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f23f40ff8b0 0x7f23f40ffcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:29:05.661 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:05.658+0000 7f23fb9f7640 1 -- 172.21.15.52:0/2937629629 >> 172.21.15.52:0/2937629629 conn(0x7f23f40fb1c0 msgr2=0x7f23f40fd620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:29:05.661 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:05.658+0000 7f23fb9f7640 1 -- 172.21.15.52:0/2937629629 shutdown_connections 2024-03-04T09:29:05.661 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:05.658+0000 7f23fb9f7640 1 -- 172.21.15.52:0/2937629629 wait complete. 2024-03-04T09:29:05.661 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:05.659+0000 7f23fb9f7640 1 Processor -- start 2024-03-04T09:29:05.661 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:05.659+0000 7f23fb9f7640 1 -- start start 2024-03-04T09:29:05.662 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:05.659+0000 7f23fb9f7640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f23f40ff8b0 0x7f23f4197890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:29:05.662 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:05.659+0000 7f23fb9f7640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f23f4102880 0x7f23f4197dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:29:05.662 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:05.659+0000 7f23fb9f7640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f23f4103240 0x7f23f419ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:29:05.662 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:05.659+0000 7f23fb9f7640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23f410d280 con 0x7f23f40ff8b0 2024-03-04T09:29:05.662 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:05.659+0000 7f23fb9f7640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23f410d100 con 0x7f23f4103240 2024-03-04T09:29:05.662 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:05.659+0000 7f23fb9f7640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23f410d400 con 0x7f23f4102880 2024-03-04T09:29:05.662 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:05.659+0000 7f23f9793640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f23f40ff8b0 0x7f23f4197890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:29:05.662 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:05.659+0000 7f23f9793640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f23f40ff8b0 0x7f23f4197890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:46982/0 (socket says 172.21.15.52:46982) 2024-03-04T09:29:05.662 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:05.659+0000 7f23f9793640 1 -- 172.21.15.52:0/262048791 learned_addr learned my addr 172.21.15.52:0/262048791 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-04T09:29:05.662 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:05.659+0000 7f23f8f92640 1 --2- 172.21.15.52:0/262048791 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f23f4102880 0x7f23f4197dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:29:05.662 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:05.659+0000 7f23f9f94640 1 --2- 172.21.15.52:0/262048791 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f23f4103240 0x7f23f419ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:29:05.662 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:05.659+0000 7f23f9793640 1 -- 172.21.15.52:0/262048791 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f23f4102880 msgr2=0x7f23f4197dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:29:05.663 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:05.659+0000 7f23f9793640 1 --2- 172.21.15.52:0/262048791 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f23f4102880 0x7f23f4197dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:29:05.663 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:05.659+0000 7f23f9793640 1 -- 172.21.15.52:0/262048791 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f23f4103240 msgr2=0x7f23f419ee40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:29:05.663 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:05.659+0000 7f23f9793640 1 --2- 172.21.15.52:0/262048791 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f23f4103240 0x7f23f419ee40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:29:05.663 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:05.659+0000 7f23f9793640 1 -- 172.21.15.52:0/262048791 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23f419f4b0 con 0x7f23f40ff8b0 2024-03-04T09:29:05.663 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:05.660+0000 7f23f9793640 1 --2- 172.21.15.52:0/262048791 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f23f40ff8b0 0x7f23f4197890 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f23f000e9c0 tx=0x7f23f000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:29:05.664 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:05.660+0000 7f23ea7fc640 1 -- 172.21.15.52:0/262048791 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23f000cde0 con 0x7f23f40ff8b0 2024-03-04T09:29:05.664 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:05.660+0000 7f23ea7fc640 1 -- 172.21.15.52:0/262048791 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f23f0010910 con 0x7f23f40ff8b0 2024-03-04T09:29:05.664 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:05.660+0000 7f23ea7fc640 1 -- 172.21.15.52:0/262048791 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23f0018a00 con 0x7f23f40ff8b0 2024-03-04T09:29:05.664 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:05.660+0000 7f23fb9f7640 1 -- 172.21.15.52:0/262048791 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23f419f7a0 con 0x7f23f40ff8b0 2024-03-04T09:29:05.664 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:05.660+0000 7f23fb9f7640 1 -- 172.21.15.52:0/262048791 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23f419fd40 con 0x7f23f40ff8b0 2024-03-04T09:29:05.664 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:05.661+0000 7f23fb9f7640 1 -- 172.21.15.52:0/262048791 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f23f4108950 con 0x7f23f40ff8b0 2024-03-04T09:29:05.665 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:05.662+0000 7f23ea7fc640 1 -- 172.21.15.52:0/262048791 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 16) v1 ==== 136756+0+0 (secure 0 0 0) 0x7f23f001f020 con 0x7f23f40ff8b0 2024-03-04T09:29:05.665 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:05.663+0000 7f23ea7fc640 1 --2- 172.21.15.52:0/262048791 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f23e00a1e00 0x7f23e00a42c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:29:05.665 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:05.663+0000 7f23f8f92640 1 --2- 172.21.15.52:0/262048791 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f23e00a1e00 0x7f23e00a42c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:29:05.665 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:05.663+0000 7f23ea7fc640 1 -- 172.21.15.52:0/262048791 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8802+0+0 (secure 0 0 0) 0x7f23f00ce790 con 0x7f23f40ff8b0 2024-03-04T09:29:05.665 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:05.663+0000 7f23f8f92640 1 --2- 172.21.15.52:0/262048791 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f23e00a1e00 0x7f23e00a42c0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f23f4198da0 tx=0x7f23e4005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:29:05.666 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:05.666+0000 7f23ea7fc640 1 -- 172.21.15.52:0/262048791 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f23f009b920 con 0x7f23f40ff8b0 2024-03-04T09:29:05.874 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:05.878+0000 7f23fb9f7640 1 -- 172.21.15.52:0/262048791 --> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7f23f4061aa0 con 0x7f23e00a1e00 2024-03-04T09:29:05.878 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:05.881+0000 7f23ea7fc640 1 -- 172.21.15.52:0/262048791 <== mgr.14152 v2:172.21.15.52:6800/3036033923 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+54 (secure 0 0 0) 0x7f23f4061aa0 con 0x7f23e00a1e00 2024-03-04T09:29:05.878 INFO:teuthology.orchestra.run.smithi052.stdout:/volumes/qa/sv_1/9260c598-0246-4b15-9bc1-639e0bc67ba7 2024-03-04T09:29:05.880 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:05.883+0000 7f23fb9f7640 1 -- 172.21.15.52:0/262048791 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f23e00a1e00 msgr2=0x7f23e00a42c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:29:05.880 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:05.883+0000 7f23fb9f7640 1 --2- 172.21.15.52:0/262048791 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f23e00a1e00 0x7f23e00a42c0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f23f4198da0 tx=0x7f23e4005f90 comp rx=0 tx=0).stop 2024-03-04T09:29:05.880 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:05.883+0000 7f23fb9f7640 1 -- 172.21.15.52:0/262048791 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f23f40ff8b0 msgr2=0x7f23f4197890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:29:05.880 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:05.883+0000 7f23fb9f7640 1 --2- 172.21.15.52:0/262048791 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f23f40ff8b0 0x7f23f4197890 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f23f000e9c0 tx=0x7f23f000ee80 comp rx=0 tx=0).stop 2024-03-04T09:29:05.880 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:05.883+0000 7f23fb9f7640 1 -- 172.21.15.52:0/262048791 shutdown_connections 2024-03-04T09:29:05.880 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:05.883+0000 7f23fb9f7640 1 --2- 172.21.15.52:0/262048791 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f23e00a1e00 0x7f23e00a42c0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:29:05.881 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:05.883+0000 7f23fb9f7640 1 --2- 172.21.15.52:0/262048791 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f23f4103240 0x7f23f419ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:29:05.881 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:05.883+0000 7f23fb9f7640 1 --2- 172.21.15.52:0/262048791 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f23f4102880 0x7f23f4197dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:29:05.881 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:05.883+0000 7f23fb9f7640 1 --2- 172.21.15.52:0/262048791 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f23f40ff8b0 0x7f23f4197890 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:29:05.881 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:05.883+0000 7f23fb9f7640 1 -- 172.21.15.52:0/262048791 >> 172.21.15.52:0/262048791 conn(0x7f23f40fb1c0 msgr2=0x7f23f4105b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:29:05.881 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:05.883+0000 7f23fb9f7640 1 -- 172.21.15.52:0/262048791 shutdown_connections 2024-03-04T09:29:05.881 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:05.883+0000 7f23fb9f7640 1 -- 172.21.15.52:0/262048791 wait complete. 2024-03-04T09:29:05.891 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T09:29:06.002 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:06.002+0000 7f0a78d2c640 1 -- 172.21.15.52:0/3130811017 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0a741027d0 msgr2=0x7f0a74102c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:29:06.002 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:06.002+0000 7f0a78d2c640 1 --2- 172.21.15.52:0/3130811017 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0a741027d0 0x7f0a74102c50 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f0a5c009900 tx=0x7f0a5c02bc30 comp rx=0 tx=0).stop 2024-03-04T09:29:06.002 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:06.003+0000 7f0a78d2c640 1 -- 172.21.15.52:0/3130811017 shutdown_connections 2024-03-04T09:29:06.002 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:06.003+0000 7f0a78d2c640 1 --2- 172.21.15.52:0/3130811017 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0a74103190 0x7f0a74109a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:29:06.002 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:06.003+0000 7f0a78d2c640 1 --2- 172.21.15.52:0/3130811017 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0a741027d0 0x7f0a74102c50 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:29:06.003 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:06.003+0000 7f0a78d2c640 1 --2- 172.21.15.52:0/3130811017 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0a741015e0 0x7f0a74101a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:29:06.003 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:06.003+0000 7f0a78d2c640 1 -- 172.21.15.52:0/3130811017 >> 172.21.15.52:0/3130811017 conn(0x7f0a740fcbb0 msgr2=0x7f0a740fefd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:29:06.003 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:06.003+0000 7f0a78d2c640 1 -- 172.21.15.52:0/3130811017 shutdown_connections 2024-03-04T09:29:06.003 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:06.003+0000 7f0a78d2c640 1 -- 172.21.15.52:0/3130811017 wait complete. 2024-03-04T09:29:06.003 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:06.003+0000 7f0a78d2c640 1 Processor -- start 2024-03-04T09:29:06.003 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:06.003+0000 7f0a78d2c640 1 -- start start 2024-03-04T09:29:06.003 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:06.004+0000 7f0a78d2c640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0a741015e0 0x7f0a741978e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:29:06.003 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:06.004+0000 7f0a78d2c640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0a741027d0 0x7f0a74197e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:29:06.003 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:06.004+0000 7f0a78d2c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0a74103190 0x7f0a7419ee90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:29:06.003 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:06.004+0000 7f0a78d2c640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a7410d260 con 0x7f0a741027d0 2024-03-04T09:29:06.003 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:06.004+0000 7f0a78d2c640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a7410d0e0 con 0x7f0a74103190 2024-03-04T09:29:06.003 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:06.004+0000 7f0a78d2c640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a7410d3e0 con 0x7f0a741015e0 2024-03-04T09:29:06.003 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:06.004+0000 7f0a71d9b640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0a741027d0 0x7f0a74197e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:29:06.003 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:06.004+0000 7f0a71d9b640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0a741027d0 0x7f0a74197e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:47000/0 (socket says 172.21.15.52:47000) 2024-03-04T09:29:06.004 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:06.004+0000 7f0a71d9b640 1 -- 172.21.15.52:0/2501347692 learned_addr learned my addr 172.21.15.52:0/2501347692 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-04T09:29:06.004 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:06.004+0000 7f0a7259c640 1 --2- 172.21.15.52:0/2501347692 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0a741015e0 0x7f0a741978e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:29:06.004 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:06.004+0000 7f0a72d9d640 1 --2- 172.21.15.52:0/2501347692 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0a74103190 0x7f0a7419ee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:29:06.004 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:06.004+0000 7f0a71d9b640 1 -- 172.21.15.52:0/2501347692 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0a741015e0 msgr2=0x7f0a741978e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:29:06.004 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:06.004+0000 7f0a71d9b640 1 --2- 172.21.15.52:0/2501347692 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0a741015e0 0x7f0a741978e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:29:06.004 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:06.004+0000 7f0a71d9b640 1 -- 172.21.15.52:0/2501347692 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0a74103190 msgr2=0x7f0a7419ee90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:29:06.004 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:06.004+0000 7f0a71d9b640 1 --2- 172.21.15.52:0/2501347692 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0a74103190 0x7f0a7419ee90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:29:06.004 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:06.004+0000 7f0a71d9b640 1 -- 172.21.15.52:0/2501347692 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a7419f500 con 0x7f0a741027d0 2024-03-04T09:29:06.004 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:06.004+0000 7f0a71d9b640 1 --2- 172.21.15.52:0/2501347692 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0a741027d0 0x7f0a74197e20 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f0a5c009230 tx=0x7f0a5c02c650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:29:06.004 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:06.005+0000 7f0a6b7fe640 1 -- 172.21.15.52:0/2501347692 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a5c041070 con 0x7f0a741027d0 2024-03-04T09:29:06.004 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:06.005+0000 7f0a6b7fe640 1 -- 172.21.15.52:0/2501347692 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0a5c004450 con 0x7f0a741027d0 2024-03-04T09:29:06.004 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:06.005+0000 7f0a78d2c640 1 -- 172.21.15.52:0/2501347692 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a7419f790 con 0x7f0a741027d0 2024-03-04T09:29:06.004 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:06.005+0000 7f0a6b7fe640 1 -- 172.21.15.52:0/2501347692 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a5c003e20 con 0x7f0a741027d0 2024-03-04T09:29:06.005 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:06.005+0000 7f0a78d2c640 1 -- 172.21.15.52:0/2501347692 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a7419fcd0 con 0x7f0a741027d0 2024-03-04T09:29:06.005 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:06.006+0000 7f0a78d2c640 1 -- 172.21.15.52:0/2501347692 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a74108980 con 0x7f0a741027d0 2024-03-04T09:29:06.005 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:06.007+0000 7f0a6b7fe640 1 -- 172.21.15.52:0/2501347692 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 16) v1 ==== 136756+0+0 (secure 0 0 0) 0x7f0a5c044070 con 0x7f0a741027d0 2024-03-04T09:29:06.005 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:06.007+0000 7f0a6b7fe640 1 --2- 172.21.15.52:0/2501347692 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f0a600a1e00 0x7f0a600a42c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:29:06.006 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:06.007+0000 7f0a6b7fe640 1 -- 172.21.15.52:0/2501347692 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8802+0+0 (secure 0 0 0) 0x7f0a5c0efe70 con 0x7f0a741027d0 2024-03-04T09:29:06.006 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:06.007+0000 7f0a7259c640 1 --2- 172.21.15.52:0/2501347692 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f0a600a1e00 0x7f0a600a42c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:29:06.006 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:06.008+0000 7f0a7259c640 1 --2- 172.21.15.52:0/2501347692 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f0a600a1e00 0x7f0a600a42c0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f0a74102630 tx=0x7f0a6c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:29:06.008 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:06.011+0000 7f0a6b7fe640 1 -- 172.21.15.52:0/2501347692 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f0a5c0bca10 con 0x7f0a741027d0 2024-03-04T09:29:06.230 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:06.233+0000 7f0a78d2c640 1 -- 172.21.15.52:0/2501347692 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0a7402d240 con 0x7f0a741027d0 2024-03-04T09:29:06.230 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:06.234+0000 7f0a6b7fe640 1 -- 172.21.15.52:0/2501347692 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+11406 (secure 0 0 0) 0x7f0a5c0c0ab0 con 0x7f0a741027d0 2024-03-04T09:29:06.231 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-03-04T09:29:06.231 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6835/1007961019","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6834","nonce":1007961019},{"type":"v1","addr":"172.21.15.52:6835","nonce":1007961019}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6837/3753881283","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6836","nonce":3753881283},{"type":"v1","addr":"172.21.15.52:6837","nonce":3753881283}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14628,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6839/3467897808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6838","nonce":3467897808},{"type":"v1","addr":"172.21.15.52:6839","nonce":3467897808}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14640,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6841/2804292291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6840","nonce":2804292291},{"type":"v1","addr":"172.21.15.52:6841","nonce":2804292291}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24407,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6833/1818480323","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6832","nonce":1818480323},{"type":"v1","addr":"172.21.15.106:6833","nonce":1818480323}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24412,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6833/765768204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":765768204},{"type":"v1","addr":"172.21.15.191:6833","nonce":765768204}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-04T09:28:37.264970+0000","modified":"2024-03-04T09:29:04.001848+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24506,"mds_1":24490,"mds_2":24473},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24440":{"gid":24440,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.106:6835/1541069897","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6834","nonce":1541069897},{"type":"v1","addr":"172.21.15.106:6835","nonce":1541069897}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24451":{"gid":24451,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.191:6835/105048699","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":105048699},{"type":"v1","addr":"172.21.15.191:6835","nonce":105048699}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24463":{"gid":24463,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.191:6837/4071064225","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6836","nonce":4071064225},{"type":"v1","addr":"172.21.15.191:6837","nonce":4071064225}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24473":{"gid":24473,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.106:6837/2901755066","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6836","nonce":2901755066},{"type":"v1","addr":"172.21.15.106:6837","nonce":2901755066}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24490":{"gid":24490,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.191:6839/2194324891","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6838","nonce":2194324891},{"type":"v1","addr":"172.21.15.191:6839","nonce":2194324891}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24506":{"gid":24506,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.106:6839/1995853093","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6838","nonce":1995853093},{"type":"v1","addr":"172.21.15.106:6839","nonce":1995853093}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24506,"qdb_cluster":[24473,24490,24506]},"id":1}]} 2024-03-04T09:29:06.231 INFO:teuthology.orchestra.run.smithi052.stderr:dumped fsmap epoch 24 2024-03-04T09:29:06.233 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:06.236+0000 7f0a78d2c640 1 -- 172.21.15.52:0/2501347692 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f0a600a1e00 msgr2=0x7f0a600a42c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:29:06.233 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:06.236+0000 7f0a78d2c640 1 --2- 172.21.15.52:0/2501347692 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f0a600a1e00 0x7f0a600a42c0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f0a74102630 tx=0x7f0a6c005f90 comp rx=0 tx=0).stop 2024-03-04T09:29:06.233 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:06.236+0000 7f0a78d2c640 1 -- 172.21.15.52:0/2501347692 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0a741027d0 msgr2=0x7f0a74197e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:29:06.233 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:06.236+0000 7f0a78d2c640 1 --2- 172.21.15.52:0/2501347692 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0a741027d0 0x7f0a74197e20 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f0a5c009230 tx=0x7f0a5c02c650 comp rx=0 tx=0).stop 2024-03-04T09:29:06.233 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:06.236+0000 7f0a78d2c640 1 -- 172.21.15.52:0/2501347692 shutdown_connections 2024-03-04T09:29:06.233 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:06.236+0000 7f0a78d2c640 1 --2- 172.21.15.52:0/2501347692 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f0a600a1e00 0x7f0a600a42c0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:29:06.233 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:06.236+0000 7f0a78d2c640 1 --2- 172.21.15.52:0/2501347692 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0a74103190 0x7f0a7419ee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:29:06.233 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:06.236+0000 7f0a78d2c640 1 --2- 172.21.15.52:0/2501347692 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0a741027d0 0x7f0a74197e20 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:29:06.233 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:06.236+0000 7f0a78d2c640 1 --2- 172.21.15.52:0/2501347692 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0a741015e0 0x7f0a741978e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:29:06.234 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:06.236+0000 7f0a78d2c640 1 -- 172.21.15.52:0/2501347692 >> 172.21.15.52:0/2501347692 conn(0x7f0a740fcbb0 msgr2=0x7f0a74105b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:29:06.234 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:06.236+0000 7f0a78d2c640 1 -- 172.21.15.52:0/2501347692 shutdown_connections 2024-03-04T09:29:06.234 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:06.236+0000 7f0a78d2c640 1 -- 172.21.15.52:0/2501347692 wait complete. 2024-03-04T09:29:06.242 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:29:06 smithi052 ceph-mon[32473]: from='client.14901 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T09:29:06.243 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:29:06 smithi052 ceph-mon[32473]: from='client.14907 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T09:29:06.244 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-04T09:29:06.336 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:29:06 smithi191 ceph-mon[36816]: from='client.14901 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T09:29:06.336 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:29:06 smithi191 ceph-mon[36816]: from='client.14907 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T09:29:06.354 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:06.350+0000 7fa51c8fb640 1 -- 172.21.15.52:0/1742698584 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa5181027e0 msgr2=0x7fa518102c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:29:06.354 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:06.350+0000 7fa51c8fb640 1 --2- 172.21.15.52:0/1742698584 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa5181027e0 0x7fa518102c60 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fa500009900 tx=0x7fa50002bc30 comp rx=0 tx=0).stop 2024-03-04T09:29:06.354 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:06.350+0000 7fa51c8fb640 1 -- 172.21.15.52:0/1742698584 shutdown_connections 2024-03-04T09:29:06.355 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:06.350+0000 7fa51c8fb640 1 --2- 172.21.15.52:0/1742698584 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa5181031a0 0x7fa518109a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:29:06.355 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:06.350+0000 7fa51c8fb640 1 --2- 172.21.15.52:0/1742698584 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa5181027e0 0x7fa518102c60 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:29:06.355 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:06.350+0000 7fa51c8fb640 1 --2- 172.21.15.52:0/1742698584 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa5181015f0 0x7fa518101a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:29:06.355 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:06.350+0000 7fa51c8fb640 1 -- 172.21.15.52:0/1742698584 >> 172.21.15.52:0/1742698584 conn(0x7fa5180fcb80 msgr2=0x7fa5180fefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:29:06.355 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:06.350+0000 7fa51c8fb640 1 -- 172.21.15.52:0/1742698584 shutdown_connections 2024-03-04T09:29:06.355 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:06.350+0000 7fa51c8fb640 1 -- 172.21.15.52:0/1742698584 wait complete. 2024-03-04T09:29:06.355 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:06.351+0000 7fa51c8fb640 1 Processor -- start 2024-03-04T09:29:06.355 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:06.351+0000 7fa51c8fb640 1 -- start start 2024-03-04T09:29:06.355 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:06.351+0000 7fa51c8fb640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa5181015f0 0x7fa5181978e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:29:06.355 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:06.351+0000 7fa51c8fb640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa5181027e0 0x7fa518197e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:29:06.356 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:06.351+0000 7fa51c8fb640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa5181031a0 0x7fa51819ee90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:29:06.356 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:06.351+0000 7fa51c8fb640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa51810d270 con 0x7fa5181015f0 2024-03-04T09:29:06.356 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:06.351+0000 7fa51c8fb640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa51810d0f0 con 0x7fa5181027e0 2024-03-04T09:29:06.356 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:06.351+0000 7fa51c8fb640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa51810d3f0 con 0x7fa5181031a0 2024-03-04T09:29:06.356 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:06.351+0000 7fa51659c640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa5181015f0 0x7fa5181978e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:29:06.356 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:06.351+0000 7fa50dd9b640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa5181027e0 0x7fa518197e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:29:06.356 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:06.351+0000 7fa51659c640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa5181015f0 0x7fa5181978e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:47020/0 (socket says 172.21.15.52:47020) 2024-03-04T09:29:06.356 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:06.351+0000 7fa50dd9b640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa5181027e0 0x7fa518197e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.52:35550/0 (socket says 172.21.15.52:35550) 2024-03-04T09:29:06.356 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:06.351+0000 7fa51659c640 1 -- 172.21.15.52:0/3689523684 learned_addr learned my addr 172.21.15.52:0/3689523684 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-04T09:29:06.356 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:06.351+0000 7fa516d9d640 1 --2- 172.21.15.52:0/3689523684 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa5181031a0 0x7fa51819ee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:29:06.356 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:06.352+0000 7fa51659c640 1 -- 172.21.15.52:0/3689523684 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa5181031a0 msgr2=0x7fa51819ee90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:29:06.356 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:06.352+0000 7fa51659c640 1 --2- 172.21.15.52:0/3689523684 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa5181031a0 0x7fa51819ee90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:29:06.356 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:06.352+0000 7fa51659c640 1 -- 172.21.15.52:0/3689523684 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa5181027e0 msgr2=0x7fa518197e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:29:06.357 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:06.352+0000 7fa51659c640 1 --2- 172.21.15.52:0/3689523684 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa5181027e0 0x7fa518197e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:29:06.357 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:06.352+0000 7fa51659c640 1 -- 172.21.15.52:0/3689523684 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa51819f500 con 0x7fa5181015f0 2024-03-04T09:29:06.357 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:06.352+0000 7fa50dd9b640 1 --2- 172.21.15.52:0/3689523684 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa5181027e0 0x7fa518197e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-04T09:29:06.357 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:06.352+0000 7fa51659c640 1 --2- 172.21.15.52:0/3689523684 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa5181015f0 0x7fa5181978e0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fa50800e9c0 tx=0x7fa50800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:29:06.357 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:06.352+0000 7fa50ffff640 1 -- 172.21.15.52:0/3689523684 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa50800cd80 con 0x7fa5181015f0 2024-03-04T09:29:06.357 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:06.352+0000 7fa51c8fb640 1 -- 172.21.15.52:0/3689523684 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa51819f7f0 con 0x7fa5181015f0 2024-03-04T09:29:06.357 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:06.352+0000 7fa50ffff640 1 -- 172.21.15.52:0/3689523684 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa5080035f0 con 0x7fa5181015f0 2024-03-04T09:29:06.357 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:06.352+0000 7fa50ffff640 1 -- 172.21.15.52:0/3689523684 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa508010690 con 0x7fa5181015f0 2024-03-04T09:29:06.357 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:06.352+0000 7fa51c8fb640 1 -- 172.21.15.52:0/3689523684 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa51819fd10 con 0x7fa5181015f0 2024-03-04T09:29:06.357 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:06.353+0000 7fa51c8fb640 1 -- 172.21.15.52:0/3689523684 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa518108910 con 0x7fa5181015f0 2024-03-04T09:29:06.357 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:06.354+0000 7fa50ffff640 1 -- 172.21.15.52:0/3689523684 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 16) v1 ==== 136756+0+0 (secure 0 0 0) 0x7fa5080037a0 con 0x7fa5181015f0 2024-03-04T09:29:06.357 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:06.355+0000 7fa50ffff640 1 --2- 172.21.15.52:0/3689523684 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7fa5040a1d80 0x7fa5040a4240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:29:06.357 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:06.355+0000 7fa50dd9b640 1 --2- 172.21.15.52:0/3689523684 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7fa5040a1d80 0x7fa5040a4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:29:06.357 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:06.355+0000 7fa50ffff640 1 -- 172.21.15.52:0/3689523684 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8802+0+0 (secure 0 0 0) 0x7fa5080ce3e0 con 0x7fa5181015f0 2024-03-04T09:29:06.358 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:06.355+0000 7fa50dd9b640 1 --2- 172.21.15.52:0/3689523684 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7fa5040a1d80 0x7fa5040a4240 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fa50002c7b0 tx=0x7fa500005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:29:06.358 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:06.359+0000 7fa50ffff640 1 -- 172.21.15.52:0/3689523684 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fa50809b570 con 0x7fa5181015f0 2024-03-04T09:29:06.381 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:29:06 smithi106 ceph-mon[36484]: from='client.14901 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T09:29:06.381 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:29:06 smithi106 ceph-mon[36484]: from='client.14907 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T09:29:06.541 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:06.544+0000 7fa51c8fb640 1 -- 172.21.15.52:0/3689523684 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa5180621d0 con 0x7fa5181015f0 2024-03-04T09:29:06.542 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:06.545+0000 7fa50ffff640 1 -- 172.21.15.52:0/3689523684 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) v1 ==== 74+0+24164 (secure 0 0 0) 0x7fa50809ae70 con 0x7fa5181015f0 2024-03-04T09:29:06.543 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-03-04T09:29:06.543 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":89,"fsid":"4e059f08-da08-11ee-95c6-87774f69a715","created":"2024-03-04T09:20:46.104963+0000","modified":"2024-03-04T09:28:42.295274+0000","last_up_change":"2024-03-04T09:27:49.261365+0000","last_in_change":"2024-03-04T09:27:30.024964+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-04T09:24:01.742461+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-04T09:28:35.835600+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-04T09:28:36.453099+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-04T09:28:40.168831+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"583172be-602f-4bf4-bfcb-42663cf5b277","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6802","nonce":1371221397},{"type":"v1","addr":"172.21.15.52:6803","nonce":1371221397}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6804","nonce":1371221397},{"type":"v1","addr":"172.21.15.52:6805","nonce":1371221397}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6808","nonce":1371221397},{"type":"v1","addr":"172.21.15.52:6809","nonce":1371221397}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6806","nonce":1371221397},{"type":"v1","addr":"172.21.15.52:6807","nonce":1371221397}]},"public_addr":"172.21.15.52:6803/1371221397","cluster_addr":"172.21.15.52:6805/1371221397","heartbeat_back_addr":"172.21.15.52:6809/1371221397","heartbeat_front_addr":"172.21.15.52:6807/1371221397","state":["exists","up"]},{"osd":1,"uuid":"5ae0081c-a2f6-46eb-ace0-0368208377b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":2540865537},{"type":"v1","addr":"172.21.15.106:6801","nonce":2540865537}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":2540865537},{"type":"v1","addr":"172.21.15.106:6803","nonce":2540865537}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":2540865537},{"type":"v1","addr":"172.21.15.106:6807","nonce":2540865537}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":2540865537},{"type":"v1","addr":"172.21.15.106:6805","nonce":2540865537}]},"public_addr":"172.21.15.106:6801/2540865537","cluster_addr":"172.21.15.106:6803/2540865537","heartbeat_back_addr":"172.21.15.106:6807/2540865537","heartbeat_front_addr":"172.21.15.106:6805/2540865537","state":["exists","up"]},{"osd":2,"uuid":"1948bac7-2bb2-4218-8b93-4e4743a0de04","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":1377083164},{"type":"v1","addr":"172.21.15.191:6801","nonce":1377083164}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":1377083164},{"type":"v1","addr":"172.21.15.191:6803","nonce":1377083164}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":1377083164},{"type":"v1","addr":"172.21.15.191:6807","nonce":1377083164}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":1377083164},{"type":"v1","addr":"172.21.15.191:6805","nonce":1377083164}]},"public_addr":"172.21.15.191:6801/1377083164","cluster_addr":"172.21.15.191:6803/1377083164","heartbeat_back_addr":"172.21.15.191:6807/1377083164","heartbeat_front_addr":"172.21.15.191:6805/1377083164","state":["exists","up"]},{"osd":3,"uuid":"16c9e9fe-7167-4992-bf12-1a7398e0498a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6810","nonce":2388571810},{"type":"v1","addr":"172.21.15.52:6811","nonce":2388571810}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6812","nonce":2388571810},{"type":"v1","addr":"172.21.15.52:6813","nonce":2388571810}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6816","nonce":2388571810},{"type":"v1","addr":"172.21.15.52:6817","nonce":2388571810}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6814","nonce":2388571810},{"type":"v1","addr":"172.21.15.52:6815","nonce":2388571810}]},"public_addr":"172.21.15.52:6811/2388571810","cluster_addr":"172.21.15.52:6813/2388571810","heartbeat_back_addr":"172.21.15.52:6817/2388571810","heartbeat_front_addr":"172.21.15.52:6815/2388571810","state":["exists","up"]},{"osd":4,"uuid":"c8ef48c7-7c78-44d3-a9e5-99ec750abd3d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6808","nonce":3465868015},{"type":"v1","addr":"172.21.15.106:6809","nonce":3465868015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6810","nonce":3465868015},{"type":"v1","addr":"172.21.15.106:6811","nonce":3465868015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6814","nonce":3465868015},{"type":"v1","addr":"172.21.15.106:6815","nonce":3465868015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6812","nonce":3465868015},{"type":"v1","addr":"172.21.15.106:6813","nonce":3465868015}]},"public_addr":"172.21.15.106:6809/3465868015","cluster_addr":"172.21.15.106:6811/3465868015","heartbeat_back_addr":"172.21.15.106:6815/3465868015","heartbeat_front_addr":"172.21.15.106:6813/3465868015","state":["exists","up"]},{"osd":5,"uuid":"9e41824a-c5a2-4f88-be50-2ba1dceb3220","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":1485092643},{"type":"v1","addr":"172.21.15.191:6809","nonce":1485092643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":1485092643},{"type":"v1","addr":"172.21.15.191:6811","nonce":1485092643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":1485092643},{"type":"v1","addr":"172.21.15.191:6815","nonce":1485092643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":1485092643},{"type":"v1","addr":"172.21.15.191:6813","nonce":1485092643}]},"public_addr":"172.21.15.191:6809/1485092643","cluster_addr":"172.21.15.191:6811/1485092643","heartbeat_back_addr":"172.21.15.191:6815/1485092643","heartbeat_front_addr":"172.21.15.191:6813/1485092643","state":["exists","up"]},{"osd":6,"uuid":"b35dccee-ec16-452e-ac69-3ac2fb7b27fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6818","nonce":1989797856},{"type":"v1","addr":"172.21.15.52:6819","nonce":1989797856}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6820","nonce":1989797856},{"type":"v1","addr":"172.21.15.52:6821","nonce":1989797856}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6824","nonce":1989797856},{"type":"v1","addr":"172.21.15.52:6825","nonce":1989797856}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6822","nonce":1989797856},{"type":"v1","addr":"172.21.15.52:6823","nonce":1989797856}]},"public_addr":"172.21.15.52:6819/1989797856","cluster_addr":"172.21.15.52:6821/1989797856","heartbeat_back_addr":"172.21.15.52:6825/1989797856","heartbeat_front_addr":"172.21.15.52:6823/1989797856","state":["exists","up"]},{"osd":7,"uuid":"71303e3d-14ec-4cbd-b19c-67645a1de3c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6816","nonce":1450870329},{"type":"v1","addr":"172.21.15.106:6817","nonce":1450870329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":1450870329},{"type":"v1","addr":"172.21.15.106:6819","nonce":1450870329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":1450870329},{"type":"v1","addr":"172.21.15.106:6823","nonce":1450870329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6820","nonce":1450870329},{"type":"v1","addr":"172.21.15.106:6821","nonce":1450870329}]},"public_addr":"172.21.15.106:6817/1450870329","cluster_addr":"172.21.15.106:6819/1450870329","heartbeat_back_addr":"172.21.15.106:6823/1450870329","heartbeat_front_addr":"172.21.15.106:6821/1450870329","state":["exists","up"]},{"osd":8,"uuid":"6f5786c4-4402-435a-bdad-adff46fe9473","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":1114119072},{"type":"v1","addr":"172.21.15.191:6817","nonce":1114119072}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6818","nonce":1114119072},{"type":"v1","addr":"172.21.15.191:6819","nonce":1114119072}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":1114119072},{"type":"v1","addr":"172.21.15.191:6823","nonce":1114119072}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":1114119072},{"type":"v1","addr":"172.21.15.191:6821","nonce":1114119072}]},"public_addr":"172.21.15.191:6817/1114119072","cluster_addr":"172.21.15.191:6819/1114119072","heartbeat_back_addr":"172.21.15.191:6823/1114119072","heartbeat_front_addr":"172.21.15.191:6821/1114119072","state":["exists","up"]},{"osd":9,"uuid":"062bbaa5-1227-4684-b859-450f5594a90c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6826","nonce":3485821914},{"type":"v1","addr":"172.21.15.52:6827","nonce":3485821914}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6828","nonce":3485821914},{"type":"v1","addr":"172.21.15.52:6829","nonce":3485821914}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6832","nonce":3485821914},{"type":"v1","addr":"172.21.15.52:6833","nonce":3485821914}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6830","nonce":3485821914},{"type":"v1","addr":"172.21.15.52:6831","nonce":3485821914}]},"public_addr":"172.21.15.52:6827/3485821914","cluster_addr":"172.21.15.52:6829/3485821914","heartbeat_back_addr":"172.21.15.52:6833/3485821914","heartbeat_front_addr":"172.21.15.52:6831/3485821914","state":["exists","up"]},{"osd":10,"uuid":"1a285a5a-f793-48da-a8eb-5326d9528cd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":3815586576},{"type":"v1","addr":"172.21.15.106:6825","nonce":3815586576}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6826","nonce":3815586576},{"type":"v1","addr":"172.21.15.106:6827","nonce":3815586576}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":3815586576},{"type":"v1","addr":"172.21.15.106:6831","nonce":3815586576}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":3815586576},{"type":"v1","addr":"172.21.15.106:6829","nonce":3815586576}]},"public_addr":"172.21.15.106:6825/3815586576","cluster_addr":"172.21.15.106:6827/3815586576","heartbeat_back_addr":"172.21.15.106:6831/3815586576","heartbeat_front_addr":"172.21.15.106:6829/3815586576","state":["exists","up"]},{"osd":11,"uuid":"1f8b9a43-57fa-41f4-9969-3b803a74bf45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":2009632843},{"type":"v1","addr":"172.21.15.191:6825","nonce":2009632843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":2009632843},{"type":"v1","addr":"172.21.15.191:6827","nonce":2009632843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":2009632843},{"type":"v1","addr":"172.21.15.191:6831","nonce":2009632843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":2009632843},{"type":"v1","addr":"172.21.15.191:6829","nonce":2009632843}]},"public_addr":"172.21.15.191:6825/2009632843","cluster_addr":"172.21.15.191:6827/2009632843","heartbeat_back_addr":"172.21.15.191:6831/2009632843","heartbeat_front_addr":"172.21.15.191:6829/2009632843","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T09:23:10.894763+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T09:23:34.420055+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T09:23:58.288324+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T09:24:23.673457+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T09:24:48.208561+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T09:25:13.405473+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T09:25:38.670915+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T09:26:04.451257+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T09:26:30.281380+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T09:26:56.543143+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T09:27:22.412930+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T09:27:48.003866+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.52:0/4082645718":"2024-03-05T09:21:39.401935+0000","172.21.15.52:0/1271949988":"2024-03-05T09:21:39.401935+0000","172.21.15.52:6801/2379361333":"2024-03-05T09:21:12.760938+0000","172.21.15.52:0/497075075":"2024-03-05T09:21:12.760938+0000","172.21.15.52:6800/2379361333":"2024-03-05T09:21:12.760938+0000","172.21.15.52:0/568242577":"2024-03-05T09:21:12.760938+0000","172.21.15.52:6801/2425730636":"2024-03-05T09:21:39.401935+0000","172.21.15.52:6800/2425730636":"2024-03-05T09:21:39.401935+0000","172.21.15.52:0/2743084243":"2024-03-05T09:21:39.401935+0000","172.21.15.52:0/1035878034":"2024-03-05T09:21:12.760938+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-04T09:29:06.544 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:06.547+0000 7fa51c8fb640 1 -- 172.21.15.52:0/3689523684 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7fa5040a1d80 msgr2=0x7fa5040a4240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:29:06.544 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:06.547+0000 7fa51c8fb640 1 --2- 172.21.15.52:0/3689523684 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7fa5040a1d80 0x7fa5040a4240 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fa50002c7b0 tx=0x7fa500005fd0 comp rx=0 tx=0).stop 2024-03-04T09:29:06.545 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:06.547+0000 7fa51c8fb640 1 -- 172.21.15.52:0/3689523684 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa5181015f0 msgr2=0x7fa5181978e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:29:06.545 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:06.547+0000 7fa51c8fb640 1 --2- 172.21.15.52:0/3689523684 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa5181015f0 0x7fa5181978e0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fa50800e9c0 tx=0x7fa50800ee80 comp rx=0 tx=0).stop 2024-03-04T09:29:06.545 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:06.547+0000 7fa51c8fb640 1 -- 172.21.15.52:0/3689523684 shutdown_connections 2024-03-04T09:29:06.545 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:06.547+0000 7fa51c8fb640 1 --2- 172.21.15.52:0/3689523684 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7fa5040a1d80 0x7fa5040a4240 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:29:06.545 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:06.547+0000 7fa51c8fb640 1 --2- 172.21.15.52:0/3689523684 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa5181031a0 0x7fa51819ee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:29:06.545 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:06.547+0000 7fa51c8fb640 1 --2- 172.21.15.52:0/3689523684 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa5181027e0 0x7fa518197e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:29:06.545 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:06.547+0000 7fa51c8fb640 1 --2- 172.21.15.52:0/3689523684 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa5181015f0 0x7fa5181978e0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:29:06.545 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:06.547+0000 7fa51c8fb640 1 -- 172.21.15.52:0/3689523684 >> 172.21.15.52:0/3689523684 conn(0x7fa5180fcb80 msgr2=0x7fa518105ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:29:06.545 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:06.547+0000 7fa51c8fb640 1 -- 172.21.15.52:0/3689523684 shutdown_connections 2024-03-04T09:29:06.545 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:06.548+0000 7fa51c8fb640 1 -- 172.21.15.52:0/3689523684 wait complete. 2024-03-04T09:29:06.560 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-03-04T09:29:06.561 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-04T09:29:06.669 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:06.666+0000 7fd05c1f6640 1 -- 172.21.15.52:0/1633996934 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd054100500 msgr2=0x7fd054100920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:29:06.669 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:06.666+0000 7fd05c1f6640 1 --2- 172.21.15.52:0/1633996934 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd054100500 0x7fd054100920 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fd050009900 tx=0x7fd05002bc30 comp rx=0 tx=0).stop 2024-03-04T09:29:06.670 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:06.666+0000 7fd05c1f6640 1 -- 172.21.15.52:0/1633996934 shutdown_connections 2024-03-04T09:29:06.670 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:06.666+0000 7fd05c1f6640 1 --2- 172.21.15.52:0/1633996934 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd0541020b0 0x7fd054108940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:29:06.670 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:06.666+0000 7fd05c1f6640 1 --2- 172.21.15.52:0/1633996934 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd0541016f0 0x7fd054101b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:29:06.670 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:06.666+0000 7fd05c1f6640 1 --2- 172.21.15.52:0/1633996934 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd054100500 0x7fd054100920 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:29:06.670 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:06.666+0000 7fd05c1f6640 1 -- 172.21.15.52:0/1633996934 >> 172.21.15.52:0/1633996934 conn(0x7fd0540fbab0 msgr2=0x7fd0540fded0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:29:06.670 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:06.666+0000 7fd05c1f6640 1 -- 172.21.15.52:0/1633996934 shutdown_connections 2024-03-04T09:29:06.670 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:06.666+0000 7fd05c1f6640 1 -- 172.21.15.52:0/1633996934 wait complete. 2024-03-04T09:29:06.670 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:06.667+0000 7fd05c1f6640 1 Processor -- start 2024-03-04T09:29:06.670 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:06.667+0000 7fd05c1f6640 1 -- start start 2024-03-04T09:29:06.670 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:06.667+0000 7fd05c1f6640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd054100500 0x7fd054197970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:29:06.670 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:06.667+0000 7fd05c1f6640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd0541016f0 0x7fd054197eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:29:06.670 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:06.667+0000 7fd05c1f6640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd0541020b0 0x7fd05419eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:29:06.670 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:06.667+0000 7fd05c1f6640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd05410c190 con 0x7fd0541020b0 2024-03-04T09:29:06.670 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:06.667+0000 7fd05c1f6640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd05410c010 con 0x7fd054100500 2024-03-04T09:29:06.671 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:06.667+0000 7fd05c1f6640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd05410c310 con 0x7fd0541016f0 2024-03-04T09:29:06.671 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:06.667+0000 7fd05a793640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd0541020b0 0x7fd05419eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:29:06.671 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:06.667+0000 7fd05a793640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd0541020b0 0x7fd05419eed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:47034/0 (socket says 172.21.15.52:47034) 2024-03-04T09:29:06.671 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:06.667+0000 7fd05a793640 1 -- 172.21.15.52:0/3154885204 learned_addr learned my addr 172.21.15.52:0/3154885204 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-04T09:29:06.671 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:06.667+0000 7fd059791640 1 --2- 172.21.15.52:0/3154885204 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd0541016f0 0x7fd054197eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:29:06.671 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:06.667+0000 7fd059f92640 1 --2- 172.21.15.52:0/3154885204 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd054100500 0x7fd054197970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:29:06.671 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:06.667+0000 7fd05a793640 1 -- 172.21.15.52:0/3154885204 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd0541016f0 msgr2=0x7fd054197eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:29:06.671 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:06.667+0000 7fd05a793640 1 --2- 172.21.15.52:0/3154885204 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd0541016f0 0x7fd054197eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:29:06.671 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:06.667+0000 7fd05a793640 1 -- 172.21.15.52:0/3154885204 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd054100500 msgr2=0x7fd054197970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:29:06.671 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:06.667+0000 7fd05a793640 1 --2- 172.21.15.52:0/3154885204 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd054100500 0x7fd054197970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:29:06.671 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:06.667+0000 7fd05a793640 1 -- 172.21.15.52:0/3154885204 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd05419f540 con 0x7fd0541020b0 2024-03-04T09:29:06.671 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:06.667+0000 7fd05a793640 1 --2- 172.21.15.52:0/3154885204 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd0541020b0 0x7fd05419eed0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fd04c00eb10 tx=0x7fd04c00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:29:06.671 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:06.668+0000 7fd04affd640 1 -- 172.21.15.52:0/3154885204 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd04c004b20 con 0x7fd0541020b0 2024-03-04T09:29:06.672 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:06.668+0000 7fd04affd640 1 -- 172.21.15.52:0/3154885204 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd04c004cc0 con 0x7fd0541020b0 2024-03-04T09:29:06.672 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:06.668+0000 7fd05c1f6640 1 -- 172.21.15.52:0/3154885204 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd05419f830 con 0x7fd0541020b0 2024-03-04T09:29:06.672 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:06.668+0000 7fd04affd640 1 -- 172.21.15.52:0/3154885204 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd04c010760 con 0x7fd0541020b0 2024-03-04T09:29:06.672 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:06.668+0000 7fd05c1f6640 1 -- 172.21.15.52:0/3154885204 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd05419fd80 con 0x7fd0541020b0 2024-03-04T09:29:06.672 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:06.669+0000 7fd05c1f6640 1 -- 172.21.15.52:0/3154885204 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd05419ffe0 con 0x7fd0541020b0 2024-03-04T09:29:06.672 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:06.670+0000 7fd04affd640 1 -- 172.21.15.52:0/3154885204 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 16) v1 ==== 136756+0+0 (secure 0 0 0) 0x7fd04c005020 con 0x7fd0541020b0 2024-03-04T09:29:06.672 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:06.670+0000 7fd04affd640 1 --2- 172.21.15.52:0/3154885204 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7fd0400a1e00 0x7fd0400a42c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:29:06.672 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:06.670+0000 7fd059f92640 1 --2- 172.21.15.52:0/3154885204 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7fd0400a1e00 0x7fd0400a42c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:29:06.672 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:06.671+0000 7fd04affd640 1 -- 172.21.15.52:0/3154885204 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8802+0+0 (secure 0 0 0) 0x7fd04c0ce4e0 con 0x7fd0541020b0 2024-03-04T09:29:06.672 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:06.671+0000 7fd059f92640 1 --2- 172.21.15.52:0/3154885204 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7fd0400a1e00 0x7fd0400a42c0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fd050000c00 tx=0x7fd050005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:29:06.673 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:06.675+0000 7fd04affd640 1 -- 172.21.15.52:0/3154885204 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fd04c09b6d0 con 0x7fd0541020b0 2024-03-04T09:29:06.852 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:06.855+0000 7fd05c1f6640 1 -- 172.21.15.52:0/3154885204 --> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fd0541a01b0 con 0x7fd0400a1e00 2024-03-04T09:29:06.862 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:06.865+0000 7fd04affd640 1 -- 172.21.15.52:0/3154885204 <== mgr.14152 v2:172.21.15.52:6800/3036033923 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+340274 (secure 0 0 0) 0x7fd0541a01b0 con 0x7fd0400a1e00 2024-03-04T09:29:06.867 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-03-04T09:29:06.869 INFO:teuthology.orchestra.run.smithi052.stderr:dumped all 2024-03-04T09:29:06.871 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:06.869+0000 7fd05c1f6640 1 -- 172.21.15.52:0/3154885204 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7fd0400a1e00 msgr2=0x7fd0400a42c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:29:06.871 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:06.869+0000 7fd05c1f6640 1 --2- 172.21.15.52:0/3154885204 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7fd0400a1e00 0x7fd0400a42c0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fd050000c00 tx=0x7fd050005fd0 comp rx=0 tx=0).stop 2024-03-04T09:29:06.871 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:06.869+0000 7fd05c1f6640 1 -- 172.21.15.52:0/3154885204 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd0541020b0 msgr2=0x7fd05419eed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:29:06.871 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:06.869+0000 7fd05c1f6640 1 --2- 172.21.15.52:0/3154885204 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd0541020b0 0x7fd05419eed0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fd04c00eb10 tx=0x7fd04c00c2d0 comp rx=0 tx=0).stop 2024-03-04T09:29:06.871 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:06.869+0000 7fd05c1f6640 1 -- 172.21.15.52:0/3154885204 shutdown_connections 2024-03-04T09:29:06.871 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:06.869+0000 7fd05c1f6640 1 --2- 172.21.15.52:0/3154885204 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7fd0400a1e00 0x7fd0400a42c0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:29:06.871 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:06.869+0000 7fd05c1f6640 1 --2- 172.21.15.52:0/3154885204 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd0541020b0 0x7fd05419eed0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:29:06.871 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:06.869+0000 7fd05c1f6640 1 --2- 172.21.15.52:0/3154885204 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd0541016f0 0x7fd054197eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:29:06.871 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:06.869+0000 7fd05c1f6640 1 --2- 172.21.15.52:0/3154885204 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd054100500 0x7fd054197970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:29:06.871 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:06.869+0000 7fd05c1f6640 1 -- 172.21.15.52:0/3154885204 >> 172.21.15.52:0/3154885204 conn(0x7fd0540fbab0 msgr2=0x7fd0540fd590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:29:06.872 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:06.869+0000 7fd05c1f6640 1 -- 172.21.15.52:0/3154885204 shutdown_connections 2024-03-04T09:29:06.872 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:06.869+0000 7fd05c1f6640 1 -- 172.21.15.52:0/3154885204 wait complete. 2024-03-04T09:29:06.877 INFO:teuthology.orchestra.run.smithi052.stdout:{"pg_ready":true,"pg_map":{"version":302,"stamp":"2024-03-04T09:29:05.488318+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":614487,"num_objects":34,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":0,"num_read":145,"num_read_kb":147,"num_write":263,"num_write_kb":2981,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":22,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":207,"ondisk_log_size":207,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":366784,"kb_used_data":13440,"kb_used_omap":18,"kb_used_meta":353197,"kb_avail":1124575040,"statfs":{"total":1151940427776,"available":1151564840960,"internally_reserved":0,"allocated":13762560,"data_stored":7526461,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19140,"internal_metadata":361674044},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19620,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":9,"num_read_kb":28,"num_write":35,"num_write_kb":26,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":8,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003333"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-04T09:28:42.972654+0000","last_change":"2024-03-04T09:28:37.914945+0000","last_active":"2024-03-04T09:28:42.972654+0000","last_peered":"2024-03-04T09:28:42.972654+0000","last_clean":"2024-03-04T09:28:42.972654+0000","last_became_active":"2024-03-04T09:28:37.914647+0000","last_became_peered":"2024-03-04T09:28:37.914647+0000","last_unstale":"2024-03-04T09:28:42.972654+0000","last_undegraded":"2024-03-04T09:28:42.972654+0000","last_fullsized":"2024-03-04T09:28:42.972654+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_clean_scrub_stamp":"2024-03-04T09:28:36.887656+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T13:46:41.262758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-04T09:28:42.972542+0000","last_change":"2024-03-04T09:28:37.918650+0000","last_active":"2024-03-04T09:28:42.972542+0000","last_peered":"2024-03-04T09:28:42.972542+0000","last_clean":"2024-03-04T09:28:42.972542+0000","last_became_active":"2024-03-04T09:28:37.918314+0000","last_became_peered":"2024-03-04T09:28:37.918314+0000","last_unstale":"2024-03-04T09:28:42.972542+0000","last_undegraded":"2024-03-04T09:28:42.972542+0000","last_fullsized":"2024-03-04T09:28:42.972542+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_clean_scrub_stamp":"2024-03-04T09:28:36.887656+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T18:36:41.948549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-04T09:28:42.972499+0000","last_change":"2024-03-04T09:28:37.920140+0000","last_active":"2024-03-04T09:28:42.972499+0000","last_peered":"2024-03-04T09:28:42.972499+0000","last_clean":"2024-03-04T09:28:42.972499+0000","last_became_active":"2024-03-04T09:28:37.917379+0000","last_became_peered":"2024-03-04T09:28:37.917379+0000","last_unstale":"2024-03-04T09:28:42.972499+0000","last_undegraded":"2024-03-04T09:28:42.972499+0000","last_fullsized":"2024-03-04T09:28:42.972499+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_clean_scrub_stamp":"2024-03-04T09:28:36.887656+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T17:23:33.144752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-04T09:28:42.972605+0000","last_change":"2024-03-04T09:28:37.916999+0000","last_active":"2024-03-04T09:28:42.972605+0000","last_peered":"2024-03-04T09:28:42.972605+0000","last_clean":"2024-03-04T09:28:42.972605+0000","last_became_active":"2024-03-04T09:28:37.916597+0000","last_became_peered":"2024-03-04T09:28:37.916597+0000","last_unstale":"2024-03-04T09:28:42.972605+0000","last_undegraded":"2024-03-04T09:28:42.972605+0000","last_fullsized":"2024-03-04T09:28:42.972605+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_clean_scrub_stamp":"2024-03-04T09:28:36.887656+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T19:23:20.556283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-04T09:28:42.971678+0000","last_change":"2024-03-04T09:28:37.904298+0000","last_active":"2024-03-04T09:28:42.971678+0000","last_peered":"2024-03-04T09:28:42.971678+0000","last_clean":"2024-03-04T09:28:42.971678+0000","last_became_active":"2024-03-04T09:28:37.903966+0000","last_became_peered":"2024-03-04T09:28:37.903966+0000","last_unstale":"2024-03-04T09:28:42.971678+0000","last_undegraded":"2024-03-04T09:28:42.971678+0000","last_fullsized":"2024-03-04T09:28:42.971678+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_clean_scrub_stamp":"2024-03-04T09:28:36.887656+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T20:14:21.615521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-04T09:28:42.971741+0000","last_change":"2024-03-04T09:28:37.917800+0000","last_active":"2024-03-04T09:28:42.971741+0000","last_peered":"2024-03-04T09:28:42.971741+0000","last_clean":"2024-03-04T09:28:42.971741+0000","last_became_active":"2024-03-04T09:28:37.917500+0000","last_became_peered":"2024-03-04T09:28:37.917500+0000","last_unstale":"2024-03-04T09:28:42.971741+0000","last_undegraded":"2024-03-04T09:28:42.971741+0000","last_fullsized":"2024-03-04T09:28:42.971741+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_clean_scrub_stamp":"2024-03-04T09:28:36.887656+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T20:06:31.330705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-04T09:28:42.933649+0000","last_change":"2024-03-04T09:28:37.917708+0000","last_active":"2024-03-04T09:28:42.933649+0000","last_peered":"2024-03-04T09:28:42.933649+0000","last_clean":"2024-03-04T09:28:42.933649+0000","last_became_active":"2024-03-04T09:28:37.917323+0000","last_became_peered":"2024-03-04T09:28:37.917323+0000","last_unstale":"2024-03-04T09:28:42.933649+0000","last_undegraded":"2024-03-04T09:28:42.933649+0000","last_fullsized":"2024-03-04T09:28:42.933649+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_clean_scrub_stamp":"2024-03-04T09:28:36.887656+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T11:10:46.345702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-04T09:28:42.971142+0000","last_change":"2024-03-04T09:28:37.925427+0000","last_active":"2024-03-04T09:28:42.971142+0000","last_peered":"2024-03-04T09:28:42.971142+0000","last_clean":"2024-03-04T09:28:42.971142+0000","last_became_active":"2024-03-04T09:28:37.925159+0000","last_became_peered":"2024-03-04T09:28:37.925159+0000","last_unstale":"2024-03-04T09:28:42.971142+0000","last_undegraded":"2024-03-04T09:28:42.971142+0000","last_fullsized":"2024-03-04T09:28:42.971142+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_clean_scrub_stamp":"2024-03-04T09:28:36.887656+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T17:29:51.564928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-04T09:28:42.971251+0000","last_change":"2024-03-04T09:28:37.928279+0000","last_active":"2024-03-04T09:28:42.971251+0000","last_peered":"2024-03-04T09:28:42.971251+0000","last_clean":"2024-03-04T09:28:42.971251+0000","last_became_active":"2024-03-04T09:28:37.927934+0000","last_became_peered":"2024-03-04T09:28:37.927934+0000","last_unstale":"2024-03-04T09:28:42.971251+0000","last_undegraded":"2024-03-04T09:28:42.971251+0000","last_fullsized":"2024-03-04T09:28:42.971251+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_clean_scrub_stamp":"2024-03-04T09:28:36.887656+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T21:23:06.592526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-04T09:28:42.972629+0000","last_change":"2024-03-04T09:28:37.918990+0000","last_active":"2024-03-04T09:28:42.972629+0000","last_peered":"2024-03-04T09:28:42.972629+0000","last_clean":"2024-03-04T09:28:42.972629+0000","last_became_active":"2024-03-04T09:28:37.918599+0000","last_became_peered":"2024-03-04T09:28:37.918599+0000","last_unstale":"2024-03-04T09:28:42.972629+0000","last_undegraded":"2024-03-04T09:28:42.972629+0000","last_fullsized":"2024-03-04T09:28:42.972629+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_clean_scrub_stamp":"2024-03-04T09:28:36.887656+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T20:39:09.603428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-04T09:28:42.934117+0000","last_change":"2024-03-04T09:28:37.914234+0000","last_active":"2024-03-04T09:28:42.934117+0000","last_peered":"2024-03-04T09:28:42.934117+0000","last_clean":"2024-03-04T09:28:42.934117+0000","last_became_active":"2024-03-04T09:28:37.913897+0000","last_became_peered":"2024-03-04T09:28:37.913897+0000","last_unstale":"2024-03-04T09:28:42.934117+0000","last_undegraded":"2024-03-04T09:28:42.934117+0000","last_fullsized":"2024-03-04T09:28:42.934117+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_clean_scrub_stamp":"2024-03-04T09:28:36.887656+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T15:43:53.634817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-04T09:28:42.972516+0000","last_change":"2024-03-04T09:28:37.919935+0000","last_active":"2024-03-04T09:28:42.972516+0000","last_peered":"2024-03-04T09:28:42.972516+0000","last_clean":"2024-03-04T09:28:42.972516+0000","last_became_active":"2024-03-04T09:28:37.919447+0000","last_became_peered":"2024-03-04T09:28:37.919447+0000","last_unstale":"2024-03-04T09:28:42.972516+0000","last_undegraded":"2024-03-04T09:28:42.972516+0000","last_fullsized":"2024-03-04T09:28:42.972516+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_clean_scrub_stamp":"2024-03-04T09:28:36.887656+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T09:44:03.678767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-04T09:28:42.973524+0000","last_change":"2024-03-04T09:28:37.900570+0000","last_active":"2024-03-04T09:28:42.973524+0000","last_peered":"2024-03-04T09:28:42.973524+0000","last_clean":"2024-03-04T09:28:42.973524+0000","last_became_active":"2024-03-04T09:28:37.900186+0000","last_became_peered":"2024-03-04T09:28:37.900186+0000","last_unstale":"2024-03-04T09:28:42.973524+0000","last_undegraded":"2024-03-04T09:28:42.973524+0000","last_fullsized":"2024-03-04T09:28:42.973524+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_clean_scrub_stamp":"2024-03-04T09:28:36.887656+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T21:17:12.915989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-04T09:28:42.971487+0000","last_change":"2024-03-04T09:28:37.920447+0000","last_active":"2024-03-04T09:28:42.971487+0000","last_peered":"2024-03-04T09:28:42.971487+0000","last_clean":"2024-03-04T09:28:42.971487+0000","last_became_active":"2024-03-04T09:28:37.919580+0000","last_became_peered":"2024-03-04T09:28:37.919580+0000","last_unstale":"2024-03-04T09:28:42.971487+0000","last_undegraded":"2024-03-04T09:28:42.971487+0000","last_fullsized":"2024-03-04T09:28:42.971487+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_clean_scrub_stamp":"2024-03-04T09:28:36.887656+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T13:25:56.434285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-04T09:28:42.971892+0000","last_change":"2024-03-04T09:28:37.913735+0000","last_active":"2024-03-04T09:28:42.971892+0000","last_peered":"2024-03-04T09:28:42.971892+0000","last_clean":"2024-03-04T09:28:42.971892+0000","last_became_active":"2024-03-04T09:28:37.913388+0000","last_became_peered":"2024-03-04T09:28:37.913388+0000","last_unstale":"2024-03-04T09:28:42.971892+0000","last_undegraded":"2024-03-04T09:28:42.971892+0000","last_fullsized":"2024-03-04T09:28:42.971892+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_clean_scrub_stamp":"2024-03-04T09:28:36.887656+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T16:04:28.468313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-04T09:28:42.970900+0000","last_change":"2024-03-04T09:28:37.925031+0000","last_active":"2024-03-04T09:28:42.970900+0000","last_peered":"2024-03-04T09:28:42.970900+0000","last_clean":"2024-03-04T09:28:42.970900+0000","last_became_active":"2024-03-04T09:28:37.923561+0000","last_became_peered":"2024-03-04T09:28:37.923561+0000","last_unstale":"2024-03-04T09:28:42.970900+0000","last_undegraded":"2024-03-04T09:28:42.970900+0000","last_fullsized":"2024-03-04T09:28:42.970900+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_clean_scrub_stamp":"2024-03-04T09:28:36.887656+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T16:29:04.266384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-04T09:28:42.972454+0000","last_change":"2024-03-04T09:28:37.912029+0000","last_active":"2024-03-04T09:28:42.972454+0000","last_peered":"2024-03-04T09:28:42.972454+0000","last_clean":"2024-03-04T09:28:42.972454+0000","last_became_active":"2024-03-04T09:28:37.911726+0000","last_became_peered":"2024-03-04T09:28:37.911726+0000","last_unstale":"2024-03-04T09:28:42.972454+0000","last_undegraded":"2024-03-04T09:28:42.972454+0000","last_fullsized":"2024-03-04T09:28:42.972454+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_clean_scrub_stamp":"2024-03-04T09:28:36.887656+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T12:32:03.385375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-04T09:28:42.932750+0000","last_change":"2024-03-04T09:28:37.914448+0000","last_active":"2024-03-04T09:28:42.932750+0000","last_peered":"2024-03-04T09:28:42.932750+0000","last_clean":"2024-03-04T09:28:42.932750+0000","last_became_active":"2024-03-04T09:28:37.914036+0000","last_became_peered":"2024-03-04T09:28:37.914036+0000","last_unstale":"2024-03-04T09:28:42.932750+0000","last_undegraded":"2024-03-04T09:28:42.932750+0000","last_fullsized":"2024-03-04T09:28:42.932750+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_clean_scrub_stamp":"2024-03-04T09:28:36.887656+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T20:57:10.348039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-04T09:28:42.932814+0000","last_change":"2024-03-04T09:28:37.907640+0000","last_active":"2024-03-04T09:28:42.932814+0000","last_peered":"2024-03-04T09:28:42.932814+0000","last_clean":"2024-03-04T09:28:42.932814+0000","last_became_active":"2024-03-04T09:28:37.907350+0000","last_became_peered":"2024-03-04T09:28:37.907350+0000","last_unstale":"2024-03-04T09:28:42.932814+0000","last_undegraded":"2024-03-04T09:28:42.932814+0000","last_fullsized":"2024-03-04T09:28:42.932814+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_clean_scrub_stamp":"2024-03-04T09:28:36.887656+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T10:17:37.576254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-04T09:28:42.932424+0000","last_change":"2024-03-04T09:28:41.950312+0000","last_active":"2024-03-04T09:28:42.932424+0000","last_peered":"2024-03-04T09:28:42.932424+0000","last_clean":"2024-03-04T09:28:42.932424+0000","last_became_active":"2024-03-04T09:28:41.949882+0000","last_became_peered":"2024-03-04T09:28:41.949882+0000","last_unstale":"2024-03-04T09:28:42.932424+0000","last_undegraded":"2024-03-04T09:28:42.932424+0000","last_fullsized":"2024-03-04T09:28:42.932424+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:40.917115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:40.917115+0000","last_clean_scrub_stamp":"2024-03-04T09:28:40.917115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T14:20:48.740518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"89'9","reported_seq":32,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-04T09:29:03.002641+0000","last_change":"2024-03-04T09:28:36.904805+0000","last_active":"2024-03-04T09:29:03.002641+0000","last_peered":"2024-03-04T09:29:03.002641+0000","last_clean":"2024-03-04T09:29:03.002641+0000","last_became_active":"2024-03-04T09:28:36.904469+0000","last_became_peered":"2024-03-04T09:28:36.904469+0000","last_unstale":"2024-03-04T09:29:03.002641+0000","last_undegraded":"2024-03-04T09:29:03.002641+0000","last_fullsized":"2024-03-04T09:29:03.002641+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:35.885776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:35.885776+0000","last_clean_scrub_stamp":"2024-03-04T09:28:35.885776+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T18:35:35.992042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-04T09:28:42.972245+0000","last_change":"2024-03-04T09:28:37.920102+0000","last_active":"2024-03-04T09:28:42.972245+0000","last_peered":"2024-03-04T09:28:42.972245+0000","last_clean":"2024-03-04T09:28:42.972245+0000","last_became_active":"2024-03-04T09:28:37.919749+0000","last_became_peered":"2024-03-04T09:28:37.919749+0000","last_unstale":"2024-03-04T09:28:42.972245+0000","last_undegraded":"2024-03-04T09:28:42.972245+0000","last_fullsized":"2024-03-04T09:28:42.972245+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_clean_scrub_stamp":"2024-03-04T09:28:36.887656+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T19:16:36.269782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-04T09:28:42.971614+0000","last_change":"2024-03-04T09:28:41.950477+0000","last_active":"2024-03-04T09:28:42.971614+0000","last_peered":"2024-03-04T09:28:42.971614+0000","last_clean":"2024-03-04T09:28:42.971614+0000","last_became_active":"2024-03-04T09:28:41.950087+0000","last_became_peered":"2024-03-04T09:28:41.950087+0000","last_unstale":"2024-03-04T09:28:42.971614+0000","last_undegraded":"2024-03-04T09:28:42.971614+0000","last_fullsized":"2024-03-04T09:28:42.971614+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:40.917115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:40.917115+0000","last_clean_scrub_stamp":"2024-03-04T09:28:40.917115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T14:48:17.327956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"89'2","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-04T09:28:42.972371+0000","last_change":"2024-03-04T09:28:36.913923+0000","last_active":"2024-03-04T09:28:42.972371+0000","last_peered":"2024-03-04T09:28:42.972371+0000","last_clean":"2024-03-04T09:28:42.972371+0000","last_became_active":"2024-03-04T09:28:36.911076+0000","last_became_peered":"2024-03-04T09:28:36.911076+0000","last_unstale":"2024-03-04T09:28:42.972371+0000","last_undegraded":"2024-03-04T09:28:42.972371+0000","last_fullsized":"2024-03-04T09:28:42.972371+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:35.885776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:35.885776+0000","last_clean_scrub_stamp":"2024-03-04T09:28:35.885776+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T16:13:23.004827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-04T09:28:42.972616+0000","last_change":"2024-03-04T09:28:37.912747+0000","last_active":"2024-03-04T09:28:42.972616+0000","last_peered":"2024-03-04T09:28:42.972616+0000","last_clean":"2024-03-04T09:28:42.972616+0000","last_became_active":"2024-03-04T09:28:37.912436+0000","last_became_peered":"2024-03-04T09:28:37.912436+0000","last_unstale":"2024-03-04T09:28:42.972616+0000","last_undegraded":"2024-03-04T09:28:42.972616+0000","last_fullsized":"2024-03-04T09:28:42.972616+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_clean_scrub_stamp":"2024-03-04T09:28:36.887656+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T14:52:47.318456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-04T09:28:42.972214+0000","last_change":"2024-03-04T09:28:41.948878+0000","last_active":"2024-03-04T09:28:42.972214+0000","last_peered":"2024-03-04T09:28:42.972214+0000","last_clean":"2024-03-04T09:28:42.972214+0000","last_became_active":"2024-03-04T09:28:41.948233+0000","last_became_peered":"2024-03-04T09:28:41.948233+0000","last_unstale":"2024-03-04T09:28:42.972214+0000","last_undegraded":"2024-03-04T09:28:42.972214+0000","last_fullsized":"2024-03-04T09:28:42.972214+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:40.917115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:40.917115+0000","last_clean_scrub_stamp":"2024-03-04T09:28:40.917115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T18:12:02.571086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"84'2","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-04T09:28:42.972126+0000","last_change":"2024-03-04T09:28:36.910279+0000","last_active":"2024-03-04T09:28:42.972126+0000","last_peered":"2024-03-04T09:28:42.972126+0000","last_clean":"2024-03-04T09:28:42.972126+0000","last_became_active":"2024-03-04T09:28:36.909214+0000","last_became_peered":"2024-03-04T09:28:36.909214+0000","last_unstale":"2024-03-04T09:28:42.972126+0000","last_undegraded":"2024-03-04T09:28:42.972126+0000","last_fullsized":"2024-03-04T09:28:42.972126+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:35.885776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:35.885776+0000","last_clean_scrub_stamp":"2024-03-04T09:28:35.885776+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T19:22:45.808418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-04T09:28:42.971105+0000","last_change":"2024-03-04T09:28:37.914593+0000","last_active":"2024-03-04T09:28:42.971105+0000","last_peered":"2024-03-04T09:28:42.971105+0000","last_clean":"2024-03-04T09:28:42.971105+0000","last_became_active":"2024-03-04T09:28:37.914217+0000","last_became_peered":"2024-03-04T09:28:37.914217+0000","last_unstale":"2024-03-04T09:28:42.971105+0000","last_undegraded":"2024-03-04T09:28:42.971105+0000","last_fullsized":"2024-03-04T09:28:42.971105+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_clean_scrub_stamp":"2024-03-04T09:28:36.887656+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T19:32:39.893906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-04T09:28:42.972082+0000","last_change":"2024-03-04T09:28:41.951685+0000","last_active":"2024-03-04T09:28:42.972082+0000","last_peered":"2024-03-04T09:28:42.972082+0000","last_clean":"2024-03-04T09:28:42.972082+0000","last_became_active":"2024-03-04T09:28:41.951178+0000","last_became_peered":"2024-03-04T09:28:41.951178+0000","last_unstale":"2024-03-04T09:28:42.972082+0000","last_undegraded":"2024-03-04T09:28:42.972082+0000","last_fullsized":"2024-03-04T09:28:42.972082+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:40.917115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:40.917115+0000","last_clean_scrub_stamp":"2024-03-04T09:28:40.917115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T10:23:19.028344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"84'2","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-04T09:28:42.971982+0000","last_change":"2024-03-04T09:28:36.908066+0000","last_active":"2024-03-04T09:28:42.971982+0000","last_peered":"2024-03-04T09:28:42.971982+0000","last_clean":"2024-03-04T09:28:42.971982+0000","last_became_active":"2024-03-04T09:28:36.907761+0000","last_became_peered":"2024-03-04T09:28:36.907761+0000","last_unstale":"2024-03-04T09:28:42.971982+0000","last_undegraded":"2024-03-04T09:28:42.971982+0000","last_fullsized":"2024-03-04T09:28:42.971982+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:35.885776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:35.885776+0000","last_clean_scrub_stamp":"2024-03-04T09:28:35.885776+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T20:53:11.559719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-04T09:28:42.973318+0000","last_change":"2024-03-04T09:28:37.917667+0000","last_active":"2024-03-04T09:28:42.973318+0000","last_peered":"2024-03-04T09:28:42.973318+0000","last_clean":"2024-03-04T09:28:42.973318+0000","last_became_active":"2024-03-04T09:28:37.917320+0000","last_became_peered":"2024-03-04T09:28:37.917320+0000","last_unstale":"2024-03-04T09:28:42.973318+0000","last_undegraded":"2024-03-04T09:28:42.973318+0000","last_fullsized":"2024-03-04T09:28:42.973318+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_clean_scrub_stamp":"2024-03-04T09:28:36.887656+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T12:17:07.162334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-04T09:28:42.932737+0000","last_change":"2024-03-04T09:28:41.946078+0000","last_active":"2024-03-04T09:28:42.932737+0000","last_peered":"2024-03-04T09:28:42.932737+0000","last_clean":"2024-03-04T09:28:42.932737+0000","last_became_active":"2024-03-04T09:28:41.945639+0000","last_became_peered":"2024-03-04T09:28:41.945639+0000","last_unstale":"2024-03-04T09:28:42.932737+0000","last_undegraded":"2024-03-04T09:28:42.932737+0000","last_fullsized":"2024-03-04T09:28:42.932737+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:40.917115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:40.917115+0000","last_clean_scrub_stamp":"2024-03-04T09:28:40.917115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T16:31:41.939728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"84'1","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-04T09:28:42.973125+0000","last_change":"2024-03-04T09:28:36.905596+0000","last_active":"2024-03-04T09:28:42.973125+0000","last_peered":"2024-03-04T09:28:42.973125+0000","last_clean":"2024-03-04T09:28:42.973125+0000","last_became_active":"2024-03-04T09:28:36.905270+0000","last_became_peered":"2024-03-04T09:28:36.905270+0000","last_unstale":"2024-03-04T09:28:42.973125+0000","last_undegraded":"2024-03-04T09:28:42.973125+0000","last_fullsized":"2024-03-04T09:28:42.973125+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:35.885776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:35.885776+0000","last_clean_scrub_stamp":"2024-03-04T09:28:35.885776+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T09:43:32.986704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-04T09:28:42.972489+0000","last_change":"2024-03-04T09:28:37.906282+0000","last_active":"2024-03-04T09:28:42.972489+0000","last_peered":"2024-03-04T09:28:42.972489+0000","last_clean":"2024-03-04T09:28:42.972489+0000","last_became_active":"2024-03-04T09:28:37.905964+0000","last_became_peered":"2024-03-04T09:28:37.905964+0000","last_unstale":"2024-03-04T09:28:42.972489+0000","last_undegraded":"2024-03-04T09:28:42.972489+0000","last_fullsized":"2024-03-04T09:28:42.972489+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_clean_scrub_stamp":"2024-03-04T09:28:36.887656+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T11:13:16.287959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-04T09:28:42.932641+0000","last_change":"2024-03-04T09:28:41.949196+0000","last_active":"2024-03-04T09:28:42.932641+0000","last_peered":"2024-03-04T09:28:42.932641+0000","last_clean":"2024-03-04T09:28:42.932641+0000","last_became_active":"2024-03-04T09:28:41.948783+0000","last_became_peered":"2024-03-04T09:28:41.948783+0000","last_unstale":"2024-03-04T09:28:42.932641+0000","last_undegraded":"2024-03-04T09:28:42.932641+0000","last_fullsized":"2024-03-04T09:28:42.932641+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:40.917115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:40.917115+0000","last_clean_scrub_stamp":"2024-03-04T09:28:40.917115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T13:42:05.888482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"89'3","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-04T09:29:03.000480+0000","last_change":"2024-03-04T09:28:36.906900+0000","last_active":"2024-03-04T09:29:03.000480+0000","last_peered":"2024-03-04T09:29:03.000480+0000","last_clean":"2024-03-04T09:29:03.000480+0000","last_became_active":"2024-03-04T09:28:36.906556+0000","last_became_peered":"2024-03-04T09:28:36.906556+0000","last_unstale":"2024-03-04T09:29:03.000480+0000","last_undegraded":"2024-03-04T09:29:03.000480+0000","last_fullsized":"2024-03-04T09:29:03.000480+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:35.885776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:35.885776+0000","last_clean_scrub_stamp":"2024-03-04T09:28:35.885776+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T18:13:56.535621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-04T09:28:42.973318+0000","last_change":"2024-03-04T09:28:37.912123+0000","last_active":"2024-03-04T09:28:42.973318+0000","last_peered":"2024-03-04T09:28:42.973318+0000","last_clean":"2024-03-04T09:28:42.973318+0000","last_became_active":"2024-03-04T09:28:37.911810+0000","last_became_peered":"2024-03-04T09:28:37.911810+0000","last_unstale":"2024-03-04T09:28:42.973318+0000","last_undegraded":"2024-03-04T09:28:42.973318+0000","last_fullsized":"2024-03-04T09:28:42.973318+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_clean_scrub_stamp":"2024-03-04T09:28:36.887656+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T18:14:25.229989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-04T09:28:42.972521+0000","last_change":"2024-03-04T09:28:41.942142+0000","last_active":"2024-03-04T09:28:42.972521+0000","last_peered":"2024-03-04T09:28:42.972521+0000","last_clean":"2024-03-04T09:28:42.972521+0000","last_became_active":"2024-03-04T09:28:41.941740+0000","last_became_peered":"2024-03-04T09:28:41.941740+0000","last_unstale":"2024-03-04T09:28:42.972521+0000","last_undegraded":"2024-03-04T09:28:42.972521+0000","last_fullsized":"2024-03-04T09:28:42.972521+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:40.917115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:40.917115+0000","last_clean_scrub_stamp":"2024-03-04T09:28:40.917115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T19:40:25.163087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-04T09:28:42.973184+0000","last_change":"2024-03-04T09:28:36.902206+0000","last_active":"2024-03-04T09:28:42.973184+0000","last_peered":"2024-03-04T09:28:42.973184+0000","last_clean":"2024-03-04T09:28:42.973184+0000","last_became_active":"2024-03-04T09:28:36.901847+0000","last_became_peered":"2024-03-04T09:28:36.901847+0000","last_unstale":"2024-03-04T09:28:42.973184+0000","last_undegraded":"2024-03-04T09:28:42.973184+0000","last_fullsized":"2024-03-04T09:28:42.973184+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:35.885776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:35.885776+0000","last_clean_scrub_stamp":"2024-03-04T09:28:35.885776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T12:32:05.755876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-04T09:28:42.971350+0000","last_change":"2024-03-04T09:28:37.906175+0000","last_active":"2024-03-04T09:28:42.971350+0000","last_peered":"2024-03-04T09:28:42.971350+0000","last_clean":"2024-03-04T09:28:42.971350+0000","last_became_active":"2024-03-04T09:28:37.905845+0000","last_became_peered":"2024-03-04T09:28:37.905845+0000","last_unstale":"2024-03-04T09:28:42.971350+0000","last_undegraded":"2024-03-04T09:28:42.971350+0000","last_fullsized":"2024-03-04T09:28:42.971350+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_clean_scrub_stamp":"2024-03-04T09:28:36.887656+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T11:50:23.569240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-04T09:28:42.972534+0000","last_change":"2024-03-04T09:28:41.949290+0000","last_active":"2024-03-04T09:28:42.972534+0000","last_peered":"2024-03-04T09:28:42.972534+0000","last_clean":"2024-03-04T09:28:42.972534+0000","last_became_active":"2024-03-04T09:28:41.948866+0000","last_became_peered":"2024-03-04T09:28:41.948866+0000","last_unstale":"2024-03-04T09:28:42.972534+0000","last_undegraded":"2024-03-04T09:28:42.972534+0000","last_fullsized":"2024-03-04T09:28:42.972534+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:40.917115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:40.917115+0000","last_clean_scrub_stamp":"2024-03-04T09:28:40.917115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T15:30:11.980139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-04T09:28:42.972143+0000","last_change":"2024-03-04T09:28:36.910071+0000","last_active":"2024-03-04T09:28:42.972143+0000","last_peered":"2024-03-04T09:28:42.972143+0000","last_clean":"2024-03-04T09:28:42.972143+0000","last_became_active":"2024-03-04T09:28:36.908025+0000","last_became_peered":"2024-03-04T09:28:36.908025+0000","last_unstale":"2024-03-04T09:28:42.972143+0000","last_undegraded":"2024-03-04T09:28:42.972143+0000","last_fullsized":"2024-03-04T09:28:42.972143+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:35.885776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:35.885776+0000","last_clean_scrub_stamp":"2024-03-04T09:28:35.885776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T10:45:42.310473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-04T09:28:42.932089+0000","last_change":"2024-03-04T09:28:37.915792+0000","last_active":"2024-03-04T09:28:42.932089+0000","last_peered":"2024-03-04T09:28:42.932089+0000","last_clean":"2024-03-04T09:28:42.932089+0000","last_became_active":"2024-03-04T09:28:37.914914+0000","last_became_peered":"2024-03-04T09:28:37.914914+0000","last_unstale":"2024-03-04T09:28:42.932089+0000","last_undegraded":"2024-03-04T09:28:42.932089+0000","last_fullsized":"2024-03-04T09:28:42.932089+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_clean_scrub_stamp":"2024-03-04T09:28:36.887656+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T14:13:57.828616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-04T09:28:42.971714+0000","last_change":"2024-03-04T09:28:41.947909+0000","last_active":"2024-03-04T09:28:42.971714+0000","last_peered":"2024-03-04T09:28:42.971714+0000","last_clean":"2024-03-04T09:28:42.971714+0000","last_became_active":"2024-03-04T09:28:41.947553+0000","last_became_peered":"2024-03-04T09:28:41.947553+0000","last_unstale":"2024-03-04T09:28:42.971714+0000","last_undegraded":"2024-03-04T09:28:42.971714+0000","last_fullsized":"2024-03-04T09:28:42.971714+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:40.917115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:40.917115+0000","last_clean_scrub_stamp":"2024-03-04T09:28:40.917115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T14:19:52.302321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"89'3","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-04T09:29:04.009368+0000","last_change":"2024-03-04T09:28:36.911823+0000","last_active":"2024-03-04T09:29:04.009368+0000","last_peered":"2024-03-04T09:29:04.009368+0000","last_clean":"2024-03-04T09:29:04.009368+0000","last_became_active":"2024-03-04T09:28:36.911502+0000","last_became_peered":"2024-03-04T09:28:36.911502+0000","last_unstale":"2024-03-04T09:29:04.009368+0000","last_undegraded":"2024-03-04T09:29:04.009368+0000","last_fullsized":"2024-03-04T09:29:04.009368+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:35.885776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:35.885776+0000","last_clean_scrub_stamp":"2024-03-04T09:28:35.885776+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T15:31:04.806604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-04T09:28:42.971241+0000","last_change":"2024-03-04T09:28:37.914093+0000","last_active":"2024-03-04T09:28:42.971241+0000","last_peered":"2024-03-04T09:28:42.971241+0000","last_clean":"2024-03-04T09:28:42.971241+0000","last_became_active":"2024-03-04T09:28:37.913661+0000","last_became_peered":"2024-03-04T09:28:37.913661+0000","last_unstale":"2024-03-04T09:28:42.971241+0000","last_undegraded":"2024-03-04T09:28:42.971241+0000","last_fullsized":"2024-03-04T09:28:42.971241+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_clean_scrub_stamp":"2024-03-04T09:28:36.887656+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T18:17:42.826342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-04T09:28:42.932023+0000","last_change":"2024-03-04T09:28:41.947371+0000","last_active":"2024-03-04T09:28:42.932023+0000","last_peered":"2024-03-04T09:28:42.932023+0000","last_clean":"2024-03-04T09:28:42.932023+0000","last_became_active":"2024-03-04T09:28:41.946944+0000","last_became_peered":"2024-03-04T09:28:41.946944+0000","last_unstale":"2024-03-04T09:28:42.932023+0000","last_undegraded":"2024-03-04T09:28:42.932023+0000","last_fullsized":"2024-03-04T09:28:42.932023+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:40.917115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:40.917115+0000","last_clean_scrub_stamp":"2024-03-04T09:28:40.917115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T11:01:34.348927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-04T09:28:42.934060+0000","last_change":"2024-03-04T09:28:36.907927+0000","last_active":"2024-03-04T09:28:42.934060+0000","last_peered":"2024-03-04T09:28:42.934060+0000","last_clean":"2024-03-04T09:28:42.934060+0000","last_became_active":"2024-03-04T09:28:36.907507+0000","last_became_peered":"2024-03-04T09:28:36.907507+0000","last_unstale":"2024-03-04T09:28:42.934060+0000","last_undegraded":"2024-03-04T09:28:42.934060+0000","last_fullsized":"2024-03-04T09:28:42.934060+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:35.885776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:35.885776+0000","last_clean_scrub_stamp":"2024-03-04T09:28:35.885776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T21:11:24.942564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-04T09:28:42.972674+0000","last_change":"2024-03-04T09:28:37.913816+0000","last_active":"2024-03-04T09:28:42.972674+0000","last_peered":"2024-03-04T09:28:42.972674+0000","last_clean":"2024-03-04T09:28:42.972674+0000","last_became_active":"2024-03-04T09:28:37.913481+0000","last_became_peered":"2024-03-04T09:28:37.913481+0000","last_unstale":"2024-03-04T09:28:42.972674+0000","last_undegraded":"2024-03-04T09:28:42.972674+0000","last_fullsized":"2024-03-04T09:28:42.972674+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_clean_scrub_stamp":"2024-03-04T09:28:36.887656+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T14:33:07.371224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-04T09:28:42.972234+0000","last_change":"2024-03-04T09:28:41.948963+0000","last_active":"2024-03-04T09:28:42.972234+0000","last_peered":"2024-03-04T09:28:42.972234+0000","last_clean":"2024-03-04T09:28:42.972234+0000","last_became_active":"2024-03-04T09:28:41.948469+0000","last_became_peered":"2024-03-04T09:28:41.948469+0000","last_unstale":"2024-03-04T09:28:42.972234+0000","last_undegraded":"2024-03-04T09:28:42.972234+0000","last_fullsized":"2024-03-04T09:28:42.972234+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:40.917115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:40.917115+0000","last_clean_scrub_stamp":"2024-03-04T09:28:40.917115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T21:26:52.238364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"84'3","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-04T09:28:42.973197+0000","last_change":"2024-03-04T09:28:36.913162+0000","last_active":"2024-03-04T09:28:42.973197+0000","last_peered":"2024-03-04T09:28:42.973197+0000","last_clean":"2024-03-04T09:28:42.973197+0000","last_became_active":"2024-03-04T09:28:36.912834+0000","last_became_peered":"2024-03-04T09:28:36.912834+0000","last_unstale":"2024-03-04T09:28:42.973197+0000","last_undegraded":"2024-03-04T09:28:42.973197+0000","last_fullsized":"2024-03-04T09:28:42.973197+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:35.885776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:35.885776+0000","last_clean_scrub_stamp":"2024-03-04T09:28:35.885776+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T17:59:07.210048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-04T09:28:42.971265+0000","last_change":"2024-03-04T09:28:37.914801+0000","last_active":"2024-03-04T09:28:42.971265+0000","last_peered":"2024-03-04T09:28:42.971265+0000","last_clean":"2024-03-04T09:28:42.971265+0000","last_became_active":"2024-03-04T09:28:37.914517+0000","last_became_peered":"2024-03-04T09:28:37.914517+0000","last_unstale":"2024-03-04T09:28:42.971265+0000","last_undegraded":"2024-03-04T09:28:42.971265+0000","last_fullsized":"2024-03-04T09:28:42.971265+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_clean_scrub_stamp":"2024-03-04T09:28:36.887656+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T18:15:50.967364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-04T09:28:42.932300+0000","last_change":"2024-03-04T09:28:41.957821+0000","last_active":"2024-03-04T09:28:42.932300+0000","last_peered":"2024-03-04T09:28:42.932300+0000","last_clean":"2024-03-04T09:28:42.932300+0000","last_became_active":"2024-03-04T09:28:41.957392+0000","last_became_peered":"2024-03-04T09:28:41.957392+0000","last_unstale":"2024-03-04T09:28:42.932300+0000","last_undegraded":"2024-03-04T09:28:42.932300+0000","last_fullsized":"2024-03-04T09:28:42.932300+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:40.917115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:40.917115+0000","last_clean_scrub_stamp":"2024-03-04T09:28:40.917115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T19:44:24.294299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"89'19","reported_seq":49,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-04T09:29:05.012668+0000","last_change":"2024-03-04T09:28:36.909243+0000","last_active":"2024-03-04T09:29:05.012668+0000","last_peered":"2024-03-04T09:29:05.012668+0000","last_clean":"2024-03-04T09:29:05.012668+0000","last_became_active":"2024-03-04T09:28:36.908941+0000","last_became_peered":"2024-03-04T09:28:36.908941+0000","last_unstale":"2024-03-04T09:29:05.012668+0000","last_undegraded":"2024-03-04T09:29:05.012668+0000","last_fullsized":"2024-03-04T09:29:05.012668+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:35.885776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:35.885776+0000","last_clean_scrub_stamp":"2024-03-04T09:28:35.885776+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T16:46:26.014209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23141,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":7,"num_read_kb":26,"num_write":20,"num_write_kb":30,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-04T09:28:42.971950+0000","last_change":"2024-03-04T09:28:37.910465+0000","last_active":"2024-03-04T09:28:42.971950+0000","last_peered":"2024-03-04T09:28:42.971950+0000","last_clean":"2024-03-04T09:28:42.971950+0000","last_became_active":"2024-03-04T09:28:37.910129+0000","last_became_peered":"2024-03-04T09:28:37.910129+0000","last_unstale":"2024-03-04T09:28:42.971950+0000","last_undegraded":"2024-03-04T09:28:42.971950+0000","last_fullsized":"2024-03-04T09:28:42.971950+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_clean_scrub_stamp":"2024-03-04T09:28:36.887656+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T21:25:48.272477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-04T09:28:42.973636+0000","last_change":"2024-03-04T09:28:41.948465+0000","last_active":"2024-03-04T09:28:42.973636+0000","last_peered":"2024-03-04T09:28:42.973636+0000","last_clean":"2024-03-04T09:28:42.973636+0000","last_became_active":"2024-03-04T09:28:41.946995+0000","last_became_peered":"2024-03-04T09:28:41.946995+0000","last_unstale":"2024-03-04T09:28:42.973636+0000","last_undegraded":"2024-03-04T09:28:42.973636+0000","last_fullsized":"2024-03-04T09:28:42.973636+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:40.917115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:40.917115+0000","last_clean_scrub_stamp":"2024-03-04T09:28:40.917115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T12:35:10.579279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"84'2","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-04T09:28:42.932539+0000","last_change":"2024-03-04T09:28:36.905641+0000","last_active":"2024-03-04T09:28:42.932539+0000","last_peered":"2024-03-04T09:28:42.932539+0000","last_clean":"2024-03-04T09:28:42.932539+0000","last_became_active":"2024-03-04T09:28:36.905309+0000","last_became_peered":"2024-03-04T09:28:36.905309+0000","last_unstale":"2024-03-04T09:28:42.932539+0000","last_undegraded":"2024-03-04T09:28:42.932539+0000","last_fullsized":"2024-03-04T09:28:42.932539+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:35.885776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:35.885776+0000","last_clean_scrub_stamp":"2024-03-04T09:28:35.885776+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T12:08:40.200654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-04T09:28:42.973519+0000","last_change":"2024-03-04T09:28:37.907744+0000","last_active":"2024-03-04T09:28:42.973519+0000","last_peered":"2024-03-04T09:28:42.973519+0000","last_clean":"2024-03-04T09:28:42.973519+0000","last_became_active":"2024-03-04T09:28:37.907393+0000","last_became_peered":"2024-03-04T09:28:37.907393+0000","last_unstale":"2024-03-04T09:28:42.973519+0000","last_undegraded":"2024-03-04T09:28:42.973519+0000","last_fullsized":"2024-03-04T09:28:42.973519+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_clean_scrub_stamp":"2024-03-04T09:28:36.887656+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T20:17:11.822846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-04T09:28:42.971775+0000","last_change":"2024-03-04T09:28:41.949824+0000","last_active":"2024-03-04T09:28:42.971775+0000","last_peered":"2024-03-04T09:28:42.971775+0000","last_clean":"2024-03-04T09:28:42.971775+0000","last_became_active":"2024-03-04T09:28:41.949252+0000","last_became_peered":"2024-03-04T09:28:41.949252+0000","last_unstale":"2024-03-04T09:28:42.971775+0000","last_undegraded":"2024-03-04T09:28:42.971775+0000","last_fullsized":"2024-03-04T09:28:42.971775+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:40.917115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:40.917115+0000","last_clean_scrub_stamp":"2024-03-04T09:28:40.917115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T21:22:19.045780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-04T09:28:42.973178+0000","last_change":"2024-03-04T09:28:36.908718+0000","last_active":"2024-03-04T09:28:42.973178+0000","last_peered":"2024-03-04T09:28:42.973178+0000","last_clean":"2024-03-04T09:28:42.973178+0000","last_became_active":"2024-03-04T09:28:36.908102+0000","last_became_peered":"2024-03-04T09:28:36.908102+0000","last_unstale":"2024-03-04T09:28:42.973178+0000","last_undegraded":"2024-03-04T09:28:42.973178+0000","last_fullsized":"2024-03-04T09:28:42.973178+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:35.885776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:35.885776+0000","last_clean_scrub_stamp":"2024-03-04T09:28:35.885776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T20:53:39.603281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-04T09:28:42.933919+0000","last_change":"2024-03-04T09:28:37.912741+0000","last_active":"2024-03-04T09:28:42.933919+0000","last_peered":"2024-03-04T09:28:42.933919+0000","last_clean":"2024-03-04T09:28:42.933919+0000","last_became_active":"2024-03-04T09:28:37.912430+0000","last_became_peered":"2024-03-04T09:28:37.912430+0000","last_unstale":"2024-03-04T09:28:42.933919+0000","last_undegraded":"2024-03-04T09:28:42.933919+0000","last_fullsized":"2024-03-04T09:28:42.933919+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_clean_scrub_stamp":"2024-03-04T09:28:36.887656+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T20:33:52.234238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-04T09:28:42.932697+0000","last_change":"2024-03-04T09:28:41.947303+0000","last_active":"2024-03-04T09:28:42.932697+0000","last_peered":"2024-03-04T09:28:42.932697+0000","last_clean":"2024-03-04T09:28:42.932697+0000","last_became_active":"2024-03-04T09:28:41.946838+0000","last_became_peered":"2024-03-04T09:28:41.946838+0000","last_unstale":"2024-03-04T09:28:42.932697+0000","last_undegraded":"2024-03-04T09:28:42.932697+0000","last_fullsized":"2024-03-04T09:28:42.932697+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:40.917115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:40.917115+0000","last_clean_scrub_stamp":"2024-03-04T09:28:40.917115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T20:08:29.575678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-04T09:28:42.971918+0000","last_change":"2024-03-04T09:28:36.905097+0000","last_active":"2024-03-04T09:28:42.971918+0000","last_peered":"2024-03-04T09:28:42.971918+0000","last_clean":"2024-03-04T09:28:42.971918+0000","last_became_active":"2024-03-04T09:28:36.904709+0000","last_became_peered":"2024-03-04T09:28:36.904709+0000","last_unstale":"2024-03-04T09:28:42.971918+0000","last_undegraded":"2024-03-04T09:28:42.971918+0000","last_fullsized":"2024-03-04T09:28:42.971918+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:35.885776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:35.885776+0000","last_clean_scrub_stamp":"2024-03-04T09:28:35.885776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T13:58:23.977258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-04T09:28:42.933763+0000","last_change":"2024-03-04T09:28:37.918023+0000","last_active":"2024-03-04T09:28:42.933763+0000","last_peered":"2024-03-04T09:28:42.933763+0000","last_clean":"2024-03-04T09:28:42.933763+0000","last_became_active":"2024-03-04T09:28:37.917671+0000","last_became_peered":"2024-03-04T09:28:37.917671+0000","last_unstale":"2024-03-04T09:28:42.933763+0000","last_undegraded":"2024-03-04T09:28:42.933763+0000","last_fullsized":"2024-03-04T09:28:42.933763+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_clean_scrub_stamp":"2024-03-04T09:28:36.887656+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T09:51:15.618328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-04T09:28:42.974003+0000","last_change":"2024-03-04T09:28:41.958903+0000","last_active":"2024-03-04T09:28:42.974003+0000","last_peered":"2024-03-04T09:28:42.974003+0000","last_clean":"2024-03-04T09:28:42.974003+0000","last_became_active":"2024-03-04T09:28:41.958256+0000","last_became_peered":"2024-03-04T09:28:41.958256+0000","last_unstale":"2024-03-04T09:28:42.974003+0000","last_undegraded":"2024-03-04T09:28:42.974003+0000","last_fullsized":"2024-03-04T09:28:42.974003+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:40.917115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:40.917115+0000","last_clean_scrub_stamp":"2024-03-04T09:28:40.917115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T18:49:48.022650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"84'8","reported_seq":31,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-04T09:28:42.971902+0000","last_change":"2024-03-04T09:28:36.911924+0000","last_active":"2024-03-04T09:28:42.971902+0000","last_peered":"2024-03-04T09:28:42.971902+0000","last_clean":"2024-03-04T09:28:42.971902+0000","last_became_active":"2024-03-04T09:28:36.911623+0000","last_became_peered":"2024-03-04T09:28:36.911623+0000","last_unstale":"2024-03-04T09:28:42.971902+0000","last_undegraded":"2024-03-04T09:28:42.971902+0000","last_fullsized":"2024-03-04T09:28:42.971902+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:35.885776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:35.885776+0000","last_clean_scrub_stamp":"2024-03-04T09:28:35.885776+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T13:17:32.606767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-04T09:28:42.972962+0000","last_change":"2024-03-04T09:28:37.916578+0000","last_active":"2024-03-04T09:28:42.972962+0000","last_peered":"2024-03-04T09:28:42.972962+0000","last_clean":"2024-03-04T09:28:42.972962+0000","last_became_active":"2024-03-04T09:28:37.913830+0000","last_became_peered":"2024-03-04T09:28:37.913830+0000","last_unstale":"2024-03-04T09:28:42.972962+0000","last_undegraded":"2024-03-04T09:28:42.972962+0000","last_fullsized":"2024-03-04T09:28:42.972962+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_clean_scrub_stamp":"2024-03-04T09:28:36.887656+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T09:29:15.882496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-04T09:28:42.972986+0000","last_change":"2024-03-04T09:28:41.951674+0000","last_active":"2024-03-04T09:28:42.972986+0000","last_peered":"2024-03-04T09:28:42.972986+0000","last_clean":"2024-03-04T09:28:42.972986+0000","last_became_active":"2024-03-04T09:28:41.951175+0000","last_became_peered":"2024-03-04T09:28:41.951175+0000","last_unstale":"2024-03-04T09:28:42.972986+0000","last_undegraded":"2024-03-04T09:28:42.972986+0000","last_fullsized":"2024-03-04T09:28:42.972986+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:40.917115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:40.917115+0000","last_clean_scrub_stamp":"2024-03-04T09:28:40.917115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T16:32:00.482396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"84'1","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-04T09:28:42.972464+0000","last_change":"2024-03-04T09:28:36.909779+0000","last_active":"2024-03-04T09:28:42.972464+0000","last_peered":"2024-03-04T09:28:42.972464+0000","last_clean":"2024-03-04T09:28:42.972464+0000","last_became_active":"2024-03-04T09:28:36.909466+0000","last_became_peered":"2024-03-04T09:28:36.909466+0000","last_unstale":"2024-03-04T09:28:42.972464+0000","last_undegraded":"2024-03-04T09:28:42.972464+0000","last_fullsized":"2024-03-04T09:28:42.972464+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:35.885776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:35.885776+0000","last_clean_scrub_stamp":"2024-03-04T09:28:35.885776+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T17:16:28.183292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-04T09:28:42.972320+0000","last_change":"2024-03-04T09:28:37.916371+0000","last_active":"2024-03-04T09:28:42.972320+0000","last_peered":"2024-03-04T09:28:42.972320+0000","last_clean":"2024-03-04T09:28:42.972320+0000","last_became_active":"2024-03-04T09:28:37.916068+0000","last_became_peered":"2024-03-04T09:28:37.916068+0000","last_unstale":"2024-03-04T09:28:42.972320+0000","last_undegraded":"2024-03-04T09:28:42.972320+0000","last_fullsized":"2024-03-04T09:28:42.972320+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_clean_scrub_stamp":"2024-03-04T09:28:36.887656+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T13:18:18.850165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-04T09:28:42.972568+0000","last_change":"2024-03-04T09:28:41.955301+0000","last_active":"2024-03-04T09:28:42.972568+0000","last_peered":"2024-03-04T09:28:42.972568+0000","last_clean":"2024-03-04T09:28:42.972568+0000","last_became_active":"2024-03-04T09:28:41.954774+0000","last_became_peered":"2024-03-04T09:28:41.954774+0000","last_unstale":"2024-03-04T09:28:42.972568+0000","last_undegraded":"2024-03-04T09:28:42.972568+0000","last_fullsized":"2024-03-04T09:28:42.972568+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:40.917115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:40.917115+0000","last_clean_scrub_stamp":"2024-03-04T09:28:40.917115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T16:56:55.463344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-04T09:28:42.932424+0000","last_change":"2024-03-04T09:28:36.901659+0000","last_active":"2024-03-04T09:28:42.932424+0000","last_peered":"2024-03-04T09:28:42.932424+0000","last_clean":"2024-03-04T09:28:42.932424+0000","last_became_active":"2024-03-04T09:28:36.901235+0000","last_became_peered":"2024-03-04T09:28:36.901235+0000","last_unstale":"2024-03-04T09:28:42.932424+0000","last_undegraded":"2024-03-04T09:28:42.932424+0000","last_fullsized":"2024-03-04T09:28:42.932424+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:35.885776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:35.885776+0000","last_clean_scrub_stamp":"2024-03-04T09:28:35.885776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T12:59:50.237172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-04T09:28:42.972456+0000","last_change":"2024-03-04T09:28:37.907905+0000","last_active":"2024-03-04T09:28:42.972456+0000","last_peered":"2024-03-04T09:28:42.972456+0000","last_clean":"2024-03-04T09:28:42.972456+0000","last_became_active":"2024-03-04T09:28:37.907376+0000","last_became_peered":"2024-03-04T09:28:37.907376+0000","last_unstale":"2024-03-04T09:28:42.972456+0000","last_undegraded":"2024-03-04T09:28:42.972456+0000","last_fullsized":"2024-03-04T09:28:42.972456+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_clean_scrub_stamp":"2024-03-04T09:28:36.887656+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T14:25:31.784659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-04T09:28:42.973802+0000","last_change":"2024-03-04T09:28:41.947629+0000","last_active":"2024-03-04T09:28:42.973802+0000","last_peered":"2024-03-04T09:28:42.973802+0000","last_clean":"2024-03-04T09:28:42.973802+0000","last_became_active":"2024-03-04T09:28:41.946251+0000","last_became_peered":"2024-03-04T09:28:41.946251+0000","last_unstale":"2024-03-04T09:28:42.973802+0000","last_undegraded":"2024-03-04T09:28:42.973802+0000","last_fullsized":"2024-03-04T09:28:42.973802+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:40.917115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:40.917115+0000","last_clean_scrub_stamp":"2024-03-04T09:28:40.917115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T18:26:57.770657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-04T09:28:42.933780+0000","last_change":"2024-03-04T09:28:36.905323+0000","last_active":"2024-03-04T09:28:42.933780+0000","last_peered":"2024-03-04T09:28:42.933780+0000","last_clean":"2024-03-04T09:28:42.933780+0000","last_became_active":"2024-03-04T09:28:36.904517+0000","last_became_peered":"2024-03-04T09:28:36.904517+0000","last_unstale":"2024-03-04T09:28:42.933780+0000","last_undegraded":"2024-03-04T09:28:42.933780+0000","last_fullsized":"2024-03-04T09:28:42.933780+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:35.885776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:35.885776+0000","last_clean_scrub_stamp":"2024-03-04T09:28:35.885776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T18:52:11.615107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-04T09:28:42.971819+0000","last_change":"2024-03-04T09:28:37.921000+0000","last_active":"2024-03-04T09:28:42.971819+0000","last_peered":"2024-03-04T09:28:42.971819+0000","last_clean":"2024-03-04T09:28:42.971819+0000","last_became_active":"2024-03-04T09:28:37.920620+0000","last_became_peered":"2024-03-04T09:28:37.920620+0000","last_unstale":"2024-03-04T09:28:42.971819+0000","last_undegraded":"2024-03-04T09:28:42.971819+0000","last_fullsized":"2024-03-04T09:28:42.971819+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_clean_scrub_stamp":"2024-03-04T09:28:36.887656+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T10:41:29.187701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-04T09:28:42.973811+0000","last_change":"2024-03-04T09:28:41.949132+0000","last_active":"2024-03-04T09:28:42.973811+0000","last_peered":"2024-03-04T09:28:42.973811+0000","last_clean":"2024-03-04T09:28:42.973811+0000","last_became_active":"2024-03-04T09:28:41.948618+0000","last_became_peered":"2024-03-04T09:28:41.948618+0000","last_unstale":"2024-03-04T09:28:42.973811+0000","last_undegraded":"2024-03-04T09:28:42.973811+0000","last_fullsized":"2024-03-04T09:28:42.973811+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:40.917115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:40.917115+0000","last_clean_scrub_stamp":"2024-03-04T09:28:40.917115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T19:45:45.001345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-04T09:28:42.972419+0000","last_change":"2024-03-04T09:28:37.910475+0000","last_active":"2024-03-04T09:28:42.972419+0000","last_peered":"2024-03-04T09:28:42.972419+0000","last_clean":"2024-03-04T09:28:42.972419+0000","last_became_active":"2024-03-04T09:28:37.910184+0000","last_became_peered":"2024-03-04T09:28:37.910184+0000","last_unstale":"2024-03-04T09:28:42.972419+0000","last_undegraded":"2024-03-04T09:28:42.972419+0000","last_fullsized":"2024-03-04T09:28:42.972419+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_clean_scrub_stamp":"2024-03-04T09:28:36.887656+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T10:16:56.244331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-04T09:28:42.973116+0000","last_change":"2024-03-04T09:28:36.913489+0000","last_active":"2024-03-04T09:28:42.973116+0000","last_peered":"2024-03-04T09:28:42.973116+0000","last_clean":"2024-03-04T09:28:42.973116+0000","last_became_active":"2024-03-04T09:28:36.913162+0000","last_became_peered":"2024-03-04T09:28:36.913162+0000","last_unstale":"2024-03-04T09:28:42.973116+0000","last_undegraded":"2024-03-04T09:28:42.973116+0000","last_fullsized":"2024-03-04T09:28:42.973116+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:35.885776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:35.885776+0000","last_clean_scrub_stamp":"2024-03-04T09:28:35.885776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T12:20:35.870058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-04T09:28:42.934143+0000","last_change":"2024-03-04T09:28:41.945717+0000","last_active":"2024-03-04T09:28:42.934143+0000","last_peered":"2024-03-04T09:28:42.934143+0000","last_clean":"2024-03-04T09:28:42.934143+0000","last_became_active":"2024-03-04T09:28:41.945300+0000","last_became_peered":"2024-03-04T09:28:41.945300+0000","last_unstale":"2024-03-04T09:28:42.934143+0000","last_undegraded":"2024-03-04T09:28:42.934143+0000","last_fullsized":"2024-03-04T09:28:42.934143+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:40.917115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:40.917115+0000","last_clean_scrub_stamp":"2024-03-04T09:28:40.917115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T11:05:30.407232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-04T09:28:42.971603+0000","last_change":"2024-03-04T09:28:37.921030+0000","last_active":"2024-03-04T09:28:42.971603+0000","last_peered":"2024-03-04T09:28:42.971603+0000","last_clean":"2024-03-04T09:28:42.971603+0000","last_became_active":"2024-03-04T09:28:37.920655+0000","last_became_peered":"2024-03-04T09:28:37.920655+0000","last_unstale":"2024-03-04T09:28:42.971603+0000","last_undegraded":"2024-03-04T09:28:42.971603+0000","last_fullsized":"2024-03-04T09:28:42.971603+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_clean_scrub_stamp":"2024-03-04T09:28:36.887656+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T19:37:34.063523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-04T09:28:42.972144+0000","last_change":"2024-03-04T09:28:36.902902+0000","last_active":"2024-03-04T09:28:42.972144+0000","last_peered":"2024-03-04T09:28:42.972144+0000","last_clean":"2024-03-04T09:28:42.972144+0000","last_became_active":"2024-03-04T09:28:36.902574+0000","last_became_peered":"2024-03-04T09:28:36.902574+0000","last_unstale":"2024-03-04T09:28:42.972144+0000","last_undegraded":"2024-03-04T09:28:42.972144+0000","last_fullsized":"2024-03-04T09:28:42.972144+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:35.885776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:35.885776+0000","last_clean_scrub_stamp":"2024-03-04T09:28:35.885776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T14:11:17.544497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":357,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-04T09:28:42.933649+0000","last_change":"2024-03-04T09:27:28.478476+0000","last_active":"2024-03-04T09:28:42.933649+0000","last_peered":"2024-03-04T09:28:42.933649+0000","last_clean":"2024-03-04T09:28:42.933649+0000","last_became_active":"2024-03-04T09:27:28.478041+0000","last_became_peered":"2024-03-04T09:27:28.478041+0000","last_unstale":"2024-03-04T09:28:42.933649+0000","last_undegraded":"2024-03-04T09:28:42.933649+0000","last_fullsized":"2024-03-04T09:28:42.933649+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:24:01.742551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:24:01.742551+0000","last_clean_scrub_stamp":"2024-03-04T09:24:01.742551+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T12:51:11.585296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-04T09:28:42.973248+0000","last_change":"2024-03-04T09:28:41.958895+0000","last_active":"2024-03-04T09:28:42.973248+0000","last_peered":"2024-03-04T09:28:42.973248+0000","last_clean":"2024-03-04T09:28:42.973248+0000","last_became_active":"2024-03-04T09:28:41.958477+0000","last_became_peered":"2024-03-04T09:28:41.958477+0000","last_unstale":"2024-03-04T09:28:42.973248+0000","last_undegraded":"2024-03-04T09:28:42.973248+0000","last_fullsized":"2024-03-04T09:28:42.973248+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:40.917115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:40.917115+0000","last_clean_scrub_stamp":"2024-03-04T09:28:40.917115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T17:48:37.149539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-04T09:28:42.973063+0000","last_change":"2024-03-04T09:28:37.911872+0000","last_active":"2024-03-04T09:28:42.973063+0000","last_peered":"2024-03-04T09:28:42.973063+0000","last_clean":"2024-03-04T09:28:42.973063+0000","last_became_active":"2024-03-04T09:28:37.911583+0000","last_became_peered":"2024-03-04T09:28:37.911583+0000","last_unstale":"2024-03-04T09:28:42.973063+0000","last_undegraded":"2024-03-04T09:28:42.973063+0000","last_fullsized":"2024-03-04T09:28:42.973063+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_clean_scrub_stamp":"2024-03-04T09:28:36.887656+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T20:12:15.606046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-04T09:28:42.933658+0000","last_change":"2024-03-04T09:28:36.905156+0000","last_active":"2024-03-04T09:28:42.933658+0000","last_peered":"2024-03-04T09:28:42.933658+0000","last_clean":"2024-03-04T09:28:42.933658+0000","last_became_active":"2024-03-04T09:28:36.904702+0000","last_became_peered":"2024-03-04T09:28:36.904702+0000","last_unstale":"2024-03-04T09:28:42.933658+0000","last_undegraded":"2024-03-04T09:28:42.933658+0000","last_fullsized":"2024-03-04T09:28:42.933658+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:35.885776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:35.885776+0000","last_clean_scrub_stamp":"2024-03-04T09:28:35.885776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T19:38:16.485245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-04T09:28:42.971722+0000","last_change":"2024-03-04T09:28:41.954087+0000","last_active":"2024-03-04T09:28:42.971722+0000","last_peered":"2024-03-04T09:28:42.971722+0000","last_clean":"2024-03-04T09:28:42.971722+0000","last_became_active":"2024-03-04T09:28:41.953681+0000","last_became_peered":"2024-03-04T09:28:41.953681+0000","last_unstale":"2024-03-04T09:28:42.971722+0000","last_undegraded":"2024-03-04T09:28:42.971722+0000","last_fullsized":"2024-03-04T09:28:42.971722+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:40.917115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:40.917115+0000","last_clean_scrub_stamp":"2024-03-04T09:28:40.917115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T19:45:18.525709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-04T09:28:42.972047+0000","last_change":"2024-03-04T09:28:36.912400+0000","last_active":"2024-03-04T09:28:42.972047+0000","last_peered":"2024-03-04T09:28:42.972047+0000","last_clean":"2024-03-04T09:28:42.972047+0000","last_became_active":"2024-03-04T09:28:36.912096+0000","last_became_peered":"2024-03-04T09:28:36.912096+0000","last_unstale":"2024-03-04T09:28:42.972047+0000","last_undegraded":"2024-03-04T09:28:42.972047+0000","last_fullsized":"2024-03-04T09:28:42.972047+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:35.885776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:35.885776+0000","last_clean_scrub_stamp":"2024-03-04T09:28:35.885776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T16:45:54.750427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-04T09:28:42.973855+0000","last_change":"2024-03-04T09:28:37.902851+0000","last_active":"2024-03-04T09:28:42.973855+0000","last_peered":"2024-03-04T09:28:42.973855+0000","last_clean":"2024-03-04T09:28:42.973855+0000","last_became_active":"2024-03-04T09:28:37.902537+0000","last_became_peered":"2024-03-04T09:28:37.902537+0000","last_unstale":"2024-03-04T09:28:42.973855+0000","last_undegraded":"2024-03-04T09:28:42.973855+0000","last_fullsized":"2024-03-04T09:28:42.973855+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_clean_scrub_stamp":"2024-03-04T09:28:36.887656+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T14:30:50.350161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-04T09:28:42.971436+0000","last_change":"2024-03-04T09:28:41.948551+0000","last_active":"2024-03-04T09:28:42.971436+0000","last_peered":"2024-03-04T09:28:42.971436+0000","last_clean":"2024-03-04T09:28:42.971436+0000","last_became_active":"2024-03-04T09:28:41.946636+0000","last_became_peered":"2024-03-04T09:28:41.946636+0000","last_unstale":"2024-03-04T09:28:42.971436+0000","last_undegraded":"2024-03-04T09:28:42.971436+0000","last_fullsized":"2024-03-04T09:28:42.971436+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:40.917115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:40.917115+0000","last_clean_scrub_stamp":"2024-03-04T09:28:40.917115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T14:00:07.624296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"89'3","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-04T09:29:03.002306+0000","last_change":"2024-03-04T09:28:36.910605+0000","last_active":"2024-03-04T09:29:03.002306+0000","last_peered":"2024-03-04T09:29:03.002306+0000","last_clean":"2024-03-04T09:29:03.002306+0000","last_became_active":"2024-03-04T09:28:36.910253+0000","last_became_peered":"2024-03-04T09:28:36.910253+0000","last_unstale":"2024-03-04T09:29:03.002306+0000","last_undegraded":"2024-03-04T09:29:03.002306+0000","last_fullsized":"2024-03-04T09:29:03.002306+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:35.885776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:35.885776+0000","last_clean_scrub_stamp":"2024-03-04T09:28:35.885776+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T18:08:30.941310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-04T09:28:42.972134+0000","last_change":"2024-03-04T09:28:37.907342+0000","last_active":"2024-03-04T09:28:42.972134+0000","last_peered":"2024-03-04T09:28:42.972134+0000","last_clean":"2024-03-04T09:28:42.972134+0000","last_became_active":"2024-03-04T09:28:37.906974+0000","last_became_peered":"2024-03-04T09:28:37.906974+0000","last_unstale":"2024-03-04T09:28:42.972134+0000","last_undegraded":"2024-03-04T09:28:42.972134+0000","last_fullsized":"2024-03-04T09:28:42.972134+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_clean_scrub_stamp":"2024-03-04T09:28:36.887656+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T11:18:43.099544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-04T09:28:42.932170+0000","last_change":"2024-03-04T09:28:41.945713+0000","last_active":"2024-03-04T09:28:42.932170+0000","last_peered":"2024-03-04T09:28:42.932170+0000","last_clean":"2024-03-04T09:28:42.932170+0000","last_became_active":"2024-03-04T09:28:41.945245+0000","last_became_peered":"2024-03-04T09:28:41.945245+0000","last_unstale":"2024-03-04T09:28:42.932170+0000","last_undegraded":"2024-03-04T09:28:42.932170+0000","last_fullsized":"2024-03-04T09:28:42.932170+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:40.917115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:40.917115+0000","last_clean_scrub_stamp":"2024-03-04T09:28:40.917115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T11:06:39.595269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"84'2","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-04T09:28:42.972930+0000","last_change":"2024-03-04T09:28:36.902020+0000","last_active":"2024-03-04T09:28:42.972930+0000","last_peered":"2024-03-04T09:28:42.972930+0000","last_clean":"2024-03-04T09:28:42.972930+0000","last_became_active":"2024-03-04T09:28:36.901666+0000","last_became_peered":"2024-03-04T09:28:36.901666+0000","last_unstale":"2024-03-04T09:28:42.972930+0000","last_undegraded":"2024-03-04T09:28:42.972930+0000","last_fullsized":"2024-03-04T09:28:42.972930+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:35.885776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:35.885776+0000","last_clean_scrub_stamp":"2024-03-04T09:28:35.885776+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T12:48:17.270200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-04T09:28:42.972604+0000","last_change":"2024-03-04T09:28:37.912889+0000","last_active":"2024-03-04T09:28:42.972604+0000","last_peered":"2024-03-04T09:28:42.972604+0000","last_clean":"2024-03-04T09:28:42.972604+0000","last_became_active":"2024-03-04T09:28:37.912542+0000","last_became_peered":"2024-03-04T09:28:37.912542+0000","last_unstale":"2024-03-04T09:28:42.972604+0000","last_undegraded":"2024-03-04T09:28:42.972604+0000","last_fullsized":"2024-03-04T09:28:42.972604+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_clean_scrub_stamp":"2024-03-04T09:28:36.887656+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T20:51:12.540364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-04T09:28:42.972605+0000","last_change":"2024-03-04T09:28:41.949009+0000","last_active":"2024-03-04T09:28:42.972605+0000","last_peered":"2024-03-04T09:28:42.972605+0000","last_clean":"2024-03-04T09:28:42.972605+0000","last_became_active":"2024-03-04T09:28:41.948178+0000","last_became_peered":"2024-03-04T09:28:41.948178+0000","last_unstale":"2024-03-04T09:28:42.972605+0000","last_undegraded":"2024-03-04T09:28:42.972605+0000","last_fullsized":"2024-03-04T09:28:42.972605+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:40.917115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:40.917115+0000","last_clean_scrub_stamp":"2024-03-04T09:28:40.917115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T10:54:54.360229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"89'2","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-04T09:29:01.998640+0000","last_change":"2024-03-04T09:28:36.901847+0000","last_active":"2024-03-04T09:29:01.998640+0000","last_peered":"2024-03-04T09:29:01.998640+0000","last_clean":"2024-03-04T09:29:01.998640+0000","last_became_active":"2024-03-04T09:28:36.901461+0000","last_became_peered":"2024-03-04T09:28:36.901461+0000","last_unstale":"2024-03-04T09:29:01.998640+0000","last_undegraded":"2024-03-04T09:29:01.998640+0000","last_fullsized":"2024-03-04T09:29:01.998640+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:35.885776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:35.885776+0000","last_clean_scrub_stamp":"2024-03-04T09:28:35.885776+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T14:36:33.041447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-04T09:28:42.971633+0000","last_change":"2024-03-04T09:28:37.907958+0000","last_active":"2024-03-04T09:28:42.971633+0000","last_peered":"2024-03-04T09:28:42.971633+0000","last_clean":"2024-03-04T09:28:42.971633+0000","last_became_active":"2024-03-04T09:28:37.907653+0000","last_became_peered":"2024-03-04T09:28:37.907653+0000","last_unstale":"2024-03-04T09:28:42.971633+0000","last_undegraded":"2024-03-04T09:28:42.971633+0000","last_fullsized":"2024-03-04T09:28:42.971633+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_clean_scrub_stamp":"2024-03-04T09:28:36.887656+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T20:42:44.266275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-04T09:28:42.932881+0000","last_change":"2024-03-04T09:28:41.957479+0000","last_active":"2024-03-04T09:28:42.932881+0000","last_peered":"2024-03-04T09:28:42.932881+0000","last_clean":"2024-03-04T09:28:42.932881+0000","last_became_active":"2024-03-04T09:28:41.954950+0000","last_became_peered":"2024-03-04T09:28:41.954950+0000","last_unstale":"2024-03-04T09:28:42.932881+0000","last_undegraded":"2024-03-04T09:28:42.932881+0000","last_fullsized":"2024-03-04T09:28:42.932881+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:40.917115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:40.917115+0000","last_clean_scrub_stamp":"2024-03-04T09:28:40.917115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T14:53:16.636230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"89'3","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-04T09:29:01.998148+0000","last_change":"2024-03-04T09:28:36.908649+0000","last_active":"2024-03-04T09:29:01.998148+0000","last_peered":"2024-03-04T09:29:01.998148+0000","last_clean":"2024-03-04T09:29:01.998148+0000","last_became_active":"2024-03-04T09:28:36.908323+0000","last_became_peered":"2024-03-04T09:28:36.908323+0000","last_unstale":"2024-03-04T09:29:01.998148+0000","last_undegraded":"2024-03-04T09:29:01.998148+0000","last_fullsized":"2024-03-04T09:29:01.998148+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:35.885776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:35.885776+0000","last_clean_scrub_stamp":"2024-03-04T09:28:35.885776+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T16:49:40.367952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-04T09:28:42.971427+0000","last_change":"2024-03-04T09:28:37.905673+0000","last_active":"2024-03-04T09:28:42.971427+0000","last_peered":"2024-03-04T09:28:42.971427+0000","last_clean":"2024-03-04T09:28:42.971427+0000","last_became_active":"2024-03-04T09:28:37.905368+0000","last_became_peered":"2024-03-04T09:28:37.905368+0000","last_unstale":"2024-03-04T09:28:42.971427+0000","last_undegraded":"2024-03-04T09:28:42.971427+0000","last_fullsized":"2024-03-04T09:28:42.971427+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_clean_scrub_stamp":"2024-03-04T09:28:36.887656+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T17:25:35.058092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-04T09:28:42.972120+0000","last_change":"2024-03-04T09:28:41.948688+0000","last_active":"2024-03-04T09:28:42.972120+0000","last_peered":"2024-03-04T09:28:42.972120+0000","last_clean":"2024-03-04T09:28:42.972120+0000","last_became_active":"2024-03-04T09:28:41.948111+0000","last_became_peered":"2024-03-04T09:28:41.948111+0000","last_unstale":"2024-03-04T09:28:42.972120+0000","last_undegraded":"2024-03-04T09:28:42.972120+0000","last_fullsized":"2024-03-04T09:28:42.972120+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:40.917115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:40.917115+0000","last_clean_scrub_stamp":"2024-03-04T09:28:40.917115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T12:27:37.121328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-04T09:28:42.932178+0000","last_change":"2024-03-04T09:28:36.908438+0000","last_active":"2024-03-04T09:28:42.932178+0000","last_peered":"2024-03-04T09:28:42.932178+0000","last_clean":"2024-03-04T09:28:42.932178+0000","last_became_active":"2024-03-04T09:28:36.906529+0000","last_became_peered":"2024-03-04T09:28:36.906529+0000","last_unstale":"2024-03-04T09:28:42.932178+0000","last_undegraded":"2024-03-04T09:28:42.932178+0000","last_fullsized":"2024-03-04T09:28:42.932178+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:35.885776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:35.885776+0000","last_clean_scrub_stamp":"2024-03-04T09:28:35.885776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T10:12:54.833556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-04T09:28:42.971906+0000","last_change":"2024-03-04T09:28:37.927048+0000","last_active":"2024-03-04T09:28:42.971906+0000","last_peered":"2024-03-04T09:28:42.971906+0000","last_clean":"2024-03-04T09:28:42.971906+0000","last_became_active":"2024-03-04T09:28:37.925381+0000","last_became_peered":"2024-03-04T09:28:37.925381+0000","last_unstale":"2024-03-04T09:28:42.971906+0000","last_undegraded":"2024-03-04T09:28:42.971906+0000","last_fullsized":"2024-03-04T09:28:42.971906+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_clean_scrub_stamp":"2024-03-04T09:28:36.887656+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T20:19:48.554102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-04T09:28:42.972670+0000","last_change":"2024-03-04T09:28:41.945805+0000","last_active":"2024-03-04T09:28:42.972670+0000","last_peered":"2024-03-04T09:28:42.972670+0000","last_clean":"2024-03-04T09:28:42.972670+0000","last_became_active":"2024-03-04T09:28:41.945414+0000","last_became_peered":"2024-03-04T09:28:41.945414+0000","last_unstale":"2024-03-04T09:28:42.972670+0000","last_undegraded":"2024-03-04T09:28:42.972670+0000","last_fullsized":"2024-03-04T09:28:42.972670+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:40.917115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:40.917115+0000","last_clean_scrub_stamp":"2024-03-04T09:28:40.917115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T09:45:59.612507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"84'2","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-04T09:28:42.971811+0000","last_change":"2024-03-04T09:28:36.909808+0000","last_active":"2024-03-04T09:28:42.971811+0000","last_peered":"2024-03-04T09:28:42.971811+0000","last_clean":"2024-03-04T09:28:42.971811+0000","last_became_active":"2024-03-04T09:28:36.909486+0000","last_became_peered":"2024-03-04T09:28:36.909486+0000","last_unstale":"2024-03-04T09:28:42.971811+0000","last_undegraded":"2024-03-04T09:28:42.971811+0000","last_fullsized":"2024-03-04T09:28:42.971811+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:35.885776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:35.885776+0000","last_clean_scrub_stamp":"2024-03-04T09:28:35.885776+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T19:38:15.524941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-04T09:28:42.933892+0000","last_change":"2024-03-04T09:28:37.910807+0000","last_active":"2024-03-04T09:28:42.933892+0000","last_peered":"2024-03-04T09:28:42.933892+0000","last_clean":"2024-03-04T09:28:42.933892+0000","last_became_active":"2024-03-04T09:28:37.910504+0000","last_became_peered":"2024-03-04T09:28:37.910504+0000","last_unstale":"2024-03-04T09:28:42.933892+0000","last_undegraded":"2024-03-04T09:28:42.933892+0000","last_fullsized":"2024-03-04T09:28:42.933892+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_clean_scrub_stamp":"2024-03-04T09:28:36.887656+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T20:03:35.484747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-04T09:28:42.970986+0000","last_change":"2024-03-04T09:28:41.949798+0000","last_active":"2024-03-04T09:28:42.970986+0000","last_peered":"2024-03-04T09:28:42.970986+0000","last_clean":"2024-03-04T09:28:42.970986+0000","last_became_active":"2024-03-04T09:28:41.949358+0000","last_became_peered":"2024-03-04T09:28:41.949358+0000","last_unstale":"2024-03-04T09:28:42.970986+0000","last_undegraded":"2024-03-04T09:28:42.970986+0000","last_fullsized":"2024-03-04T09:28:42.970986+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:40.917115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:40.917115+0000","last_clean_scrub_stamp":"2024-03-04T09:28:40.917115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T13:30:26.519090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-04T09:28:42.932771+0000","last_change":"2024-03-04T09:28:36.912458+0000","last_active":"2024-03-04T09:28:42.932771+0000","last_peered":"2024-03-04T09:28:42.932771+0000","last_clean":"2024-03-04T09:28:42.932771+0000","last_became_active":"2024-03-04T09:28:36.912132+0000","last_became_peered":"2024-03-04T09:28:36.912132+0000","last_unstale":"2024-03-04T09:28:42.932771+0000","last_undegraded":"2024-03-04T09:28:42.932771+0000","last_fullsized":"2024-03-04T09:28:42.932771+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:35.885776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:35.885776+0000","last_clean_scrub_stamp":"2024-03-04T09:28:35.885776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T13:51:29.713381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-04T09:28:42.934196+0000","last_change":"2024-03-04T09:28:37.915750+0000","last_active":"2024-03-04T09:28:42.934196+0000","last_peered":"2024-03-04T09:28:42.934196+0000","last_clean":"2024-03-04T09:28:42.934196+0000","last_became_active":"2024-03-04T09:28:37.915424+0000","last_became_peered":"2024-03-04T09:28:37.915424+0000","last_unstale":"2024-03-04T09:28:42.934196+0000","last_undegraded":"2024-03-04T09:28:42.934196+0000","last_fullsized":"2024-03-04T09:28:42.934196+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_clean_scrub_stamp":"2024-03-04T09:28:36.887656+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T19:15:39.555122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-04T09:28:42.971982+0000","last_change":"2024-03-04T09:28:41.949321+0000","last_active":"2024-03-04T09:28:42.971982+0000","last_peered":"2024-03-04T09:28:42.971982+0000","last_clean":"2024-03-04T09:28:42.971982+0000","last_became_active":"2024-03-04T09:28:41.948302+0000","last_became_peered":"2024-03-04T09:28:41.948302+0000","last_unstale":"2024-03-04T09:28:42.971982+0000","last_undegraded":"2024-03-04T09:28:42.971982+0000","last_fullsized":"2024-03-04T09:28:42.971982+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:40.917115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:40.917115+0000","last_clean_scrub_stamp":"2024-03-04T09:28:40.917115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T10:18:44.988651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"84'1","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-04T09:28:42.973358+0000","last_change":"2024-03-04T09:28:36.909585+0000","last_active":"2024-03-04T09:28:42.973358+0000","last_peered":"2024-03-04T09:28:42.973358+0000","last_clean":"2024-03-04T09:28:42.973358+0000","last_became_active":"2024-03-04T09:28:36.909286+0000","last_became_peered":"2024-03-04T09:28:36.909286+0000","last_unstale":"2024-03-04T09:28:42.973358+0000","last_undegraded":"2024-03-04T09:28:42.973358+0000","last_fullsized":"2024-03-04T09:28:42.973358+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:35.885776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:35.885776+0000","last_clean_scrub_stamp":"2024-03-04T09:28:35.885776+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T09:29:45.236355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-04T09:28:42.933788+0000","last_change":"2024-03-04T09:28:37.924411+0000","last_active":"2024-03-04T09:28:42.933788+0000","last_peered":"2024-03-04T09:28:42.933788+0000","last_clean":"2024-03-04T09:28:42.933788+0000","last_became_active":"2024-03-04T09:28:37.924147+0000","last_became_peered":"2024-03-04T09:28:37.924147+0000","last_unstale":"2024-03-04T09:28:42.933788+0000","last_undegraded":"2024-03-04T09:28:42.933788+0000","last_fullsized":"2024-03-04T09:28:42.933788+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_clean_scrub_stamp":"2024-03-04T09:28:36.887656+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T15:05:16.573937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-04T09:28:42.971552+0000","last_change":"2024-03-04T09:28:41.943710+0000","last_active":"2024-03-04T09:28:42.971552+0000","last_peered":"2024-03-04T09:28:42.971552+0000","last_clean":"2024-03-04T09:28:42.971552+0000","last_became_active":"2024-03-04T09:28:41.943258+0000","last_became_peered":"2024-03-04T09:28:41.943258+0000","last_unstale":"2024-03-04T09:28:42.971552+0000","last_undegraded":"2024-03-04T09:28:42.971552+0000","last_fullsized":"2024-03-04T09:28:42.971552+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:40.917115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:40.917115+0000","last_clean_scrub_stamp":"2024-03-04T09:28:40.917115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T15:32:52.597594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"84'2","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-04T09:28:42.973945+0000","last_change":"2024-03-04T09:28:36.909499+0000","last_active":"2024-03-04T09:28:42.973945+0000","last_peered":"2024-03-04T09:28:42.973945+0000","last_clean":"2024-03-04T09:28:42.973945+0000","last_became_active":"2024-03-04T09:28:36.909168+0000","last_became_peered":"2024-03-04T09:28:36.909168+0000","last_unstale":"2024-03-04T09:28:42.973945+0000","last_undegraded":"2024-03-04T09:28:42.973945+0000","last_fullsized":"2024-03-04T09:28:42.973945+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:35.885776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:35.885776+0000","last_clean_scrub_stamp":"2024-03-04T09:28:35.885776+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T17:54:23.936630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-04T09:28:42.934178+0000","last_change":"2024-03-04T09:28:37.916569+0000","last_active":"2024-03-04T09:28:42.934178+0000","last_peered":"2024-03-04T09:28:42.934178+0000","last_clean":"2024-03-04T09:28:42.934178+0000","last_became_active":"2024-03-04T09:28:37.916264+0000","last_became_peered":"2024-03-04T09:28:37.916264+0000","last_unstale":"2024-03-04T09:28:42.934178+0000","last_undegraded":"2024-03-04T09:28:42.934178+0000","last_fullsized":"2024-03-04T09:28:42.934178+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_clean_scrub_stamp":"2024-03-04T09:28:36.887656+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T16:59:51.981066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-04T09:28:42.971509+0000","last_change":"2024-03-04T09:28:37.910027+0000","last_active":"2024-03-04T09:28:42.971509+0000","last_peered":"2024-03-04T09:28:42.971509+0000","last_clean":"2024-03-04T09:28:42.971509+0000","last_became_active":"2024-03-04T09:28:37.909715+0000","last_became_peered":"2024-03-04T09:28:37.909715+0000","last_unstale":"2024-03-04T09:28:42.971509+0000","last_undegraded":"2024-03-04T09:28:42.971509+0000","last_fullsized":"2024-03-04T09:28:42.971509+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_clean_scrub_stamp":"2024-03-04T09:28:36.887656+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T20:09:12.150573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-04T09:28:42.932253+0000","last_change":"2024-03-04T09:28:37.912211+0000","last_active":"2024-03-04T09:28:42.932253+0000","last_peered":"2024-03-04T09:28:42.932253+0000","last_clean":"2024-03-04T09:28:42.932253+0000","last_became_active":"2024-03-04T09:28:37.911904+0000","last_became_peered":"2024-03-04T09:28:37.911904+0000","last_unstale":"2024-03-04T09:28:42.932253+0000","last_undegraded":"2024-03-04T09:28:42.932253+0000","last_fullsized":"2024-03-04T09:28:42.932253+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_clean_scrub_stamp":"2024-03-04T09:28:36.887656+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T20:02:04.538449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-04T09:28:42.932147+0000","last_change":"2024-03-04T09:28:37.906956+0000","last_active":"2024-03-04T09:28:42.932147+0000","last_peered":"2024-03-04T09:28:42.932147+0000","last_clean":"2024-03-04T09:28:42.932147+0000","last_became_active":"2024-03-04T09:28:37.906642+0000","last_became_peered":"2024-03-04T09:28:37.906642+0000","last_unstale":"2024-03-04T09:28:42.932147+0000","last_undegraded":"2024-03-04T09:28:42.932147+0000","last_fullsized":"2024-03-04T09:28:42.932147+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_clean_scrub_stamp":"2024-03-04T09:28:36.887656+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T20:58:18.387119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-04T09:28:42.972180+0000","last_change":"2024-03-04T09:28:37.916105+0000","last_active":"2024-03-04T09:28:42.972180+0000","last_peered":"2024-03-04T09:28:42.972180+0000","last_clean":"2024-03-04T09:28:42.972180+0000","last_became_active":"2024-03-04T09:28:37.913893+0000","last_became_peered":"2024-03-04T09:28:37.913893+0000","last_unstale":"2024-03-04T09:28:42.972180+0000","last_undegraded":"2024-03-04T09:28:42.972180+0000","last_fullsized":"2024-03-04T09:28:42.972180+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_clean_scrub_stamp":"2024-03-04T09:28:36.887656+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T12:57:46.128587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-04T09:28:42.973726+0000","last_change":"2024-03-04T09:28:37.912702+0000","last_active":"2024-03-04T09:28:42.973726+0000","last_peered":"2024-03-04T09:28:42.973726+0000","last_clean":"2024-03-04T09:28:42.973726+0000","last_became_active":"2024-03-04T09:28:37.912397+0000","last_became_peered":"2024-03-04T09:28:37.912397+0000","last_unstale":"2024-03-04T09:28:42.973726+0000","last_undegraded":"2024-03-04T09:28:42.973726+0000","last_fullsized":"2024-03-04T09:28:42.973726+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_clean_scrub_stamp":"2024-03-04T09:28:36.887656+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T16:36:59.575111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-04T09:28:42.972243+0000","last_change":"2024-03-04T09:28:37.909801+0000","last_active":"2024-03-04T09:28:42.972243+0000","last_peered":"2024-03-04T09:28:42.972243+0000","last_clean":"2024-03-04T09:28:42.972243+0000","last_became_active":"2024-03-04T09:28:37.909505+0000","last_became_peered":"2024-03-04T09:28:37.909505+0000","last_unstale":"2024-03-04T09:28:42.972243+0000","last_undegraded":"2024-03-04T09:28:42.972243+0000","last_fullsized":"2024-03-04T09:28:42.972243+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_clean_scrub_stamp":"2024-03-04T09:28:36.887656+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T11:56:44.276188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-04T09:28:42.932875+0000","last_change":"2024-03-04T09:28:37.911076+0000","last_active":"2024-03-04T09:28:42.932875+0000","last_peered":"2024-03-04T09:28:42.932875+0000","last_clean":"2024-03-04T09:28:42.932875+0000","last_became_active":"2024-03-04T09:28:37.910786+0000","last_became_peered":"2024-03-04T09:28:37.910786+0000","last_unstale":"2024-03-04T09:28:42.932875+0000","last_undegraded":"2024-03-04T09:28:42.932875+0000","last_fullsized":"2024-03-04T09:28:42.932875+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_clean_scrub_stamp":"2024-03-04T09:28:36.887656+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T11:42:41.932235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-04T09:28:42.972340+0000","last_change":"2024-03-04T09:28:37.914661+0000","last_active":"2024-03-04T09:28:42.972340+0000","last_peered":"2024-03-04T09:28:42.972340+0000","last_clean":"2024-03-04T09:28:42.972340+0000","last_became_active":"2024-03-04T09:28:37.913851+0000","last_became_peered":"2024-03-04T09:28:37.913851+0000","last_unstale":"2024-03-04T09:28:42.972340+0000","last_undegraded":"2024-03-04T09:28:42.972340+0000","last_fullsized":"2024-03-04T09:28:42.972340+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_clean_scrub_stamp":"2024-03-04T09:28:36.887656+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T10:45:43.670006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-04T09:28:42.972285+0000","last_change":"2024-03-04T09:28:37.904746+0000","last_active":"2024-03-04T09:28:42.972285+0000","last_peered":"2024-03-04T09:28:42.972285+0000","last_clean":"2024-03-04T09:28:42.972285+0000","last_became_active":"2024-03-04T09:28:37.904466+0000","last_became_peered":"2024-03-04T09:28:37.904466+0000","last_unstale":"2024-03-04T09:28:42.972285+0000","last_undegraded":"2024-03-04T09:28:42.972285+0000","last_fullsized":"2024-03-04T09:28:42.972285+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_clean_scrub_stamp":"2024-03-04T09:28:36.887656+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T13:59:24.810820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-04T09:28:42.971972+0000","last_change":"2024-03-04T09:28:37.913252+0000","last_active":"2024-03-04T09:28:42.971972+0000","last_peered":"2024-03-04T09:28:42.971972+0000","last_clean":"2024-03-04T09:28:42.971972+0000","last_became_active":"2024-03-04T09:28:37.910819+0000","last_became_peered":"2024-03-04T09:28:37.910819+0000","last_unstale":"2024-03-04T09:28:42.971972+0000","last_undegraded":"2024-03-04T09:28:42.971972+0000","last_fullsized":"2024-03-04T09:28:42.971972+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_clean_scrub_stamp":"2024-03-04T09:28:36.887656+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T13:39:47.886505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-04T09:28:42.971241+0000","last_change":"2024-03-04T09:28:37.915716+0000","last_active":"2024-03-04T09:28:42.971241+0000","last_peered":"2024-03-04T09:28:42.971241+0000","last_clean":"2024-03-04T09:28:42.971241+0000","last_became_active":"2024-03-04T09:28:37.915398+0000","last_became_peered":"2024-03-04T09:28:37.915398+0000","last_unstale":"2024-03-04T09:28:42.971241+0000","last_undegraded":"2024-03-04T09:28:42.971241+0000","last_fullsized":"2024-03-04T09:28:42.971241+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_clean_scrub_stamp":"2024-03-04T09:28:36.887656+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T20:10:53.793715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-04T09:28:42.973586+0000","last_change":"2024-03-04T09:28:37.906671+0000","last_active":"2024-03-04T09:28:42.973586+0000","last_peered":"2024-03-04T09:28:42.973586+0000","last_clean":"2024-03-04T09:28:42.973586+0000","last_became_active":"2024-03-04T09:28:37.906344+0000","last_became_peered":"2024-03-04T09:28:37.906344+0000","last_unstale":"2024-03-04T09:28:42.973586+0000","last_undegraded":"2024-03-04T09:28:42.973586+0000","last_fullsized":"2024-03-04T09:28:42.973586+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_clean_scrub_stamp":"2024-03-04T09:28:36.887656+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T16:02:18.204424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-04T09:28:42.971914+0000","last_change":"2024-03-04T09:28:37.916895+0000","last_active":"2024-03-04T09:28:42.971914+0000","last_peered":"2024-03-04T09:28:42.971914+0000","last_clean":"2024-03-04T09:28:42.971914+0000","last_became_active":"2024-03-04T09:28:37.916585+0000","last_became_peered":"2024-03-04T09:28:37.916585+0000","last_unstale":"2024-03-04T09:28:42.971914+0000","last_undegraded":"2024-03-04T09:28:42.971914+0000","last_fullsized":"2024-03-04T09:28:42.971914+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_clean_scrub_stamp":"2024-03-04T09:28:36.887656+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T12:13:50.941390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":24119,"num_objects":32,"num_object_clones":0,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":0,"num_read":9,"num_read_kb":28,"num_write":82,"num_write_kb":43,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":22,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":192512,"data_stored":37380,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":70,"ondisk_log_size":70,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383698,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27984,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717168,"statfs":{"total":95995035648,"available":95966380032,"internally_reserved":0,"allocated":991232,"data_stored":481893,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":72,"seq":309237645334,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36928,"kb_used_data":1592,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708224,"statfs":{"total":95995035648,"available":95957221376,"internally_reserved":0,"allocated":1630208,"data_stored":1079994,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":67,"seq":287762808860,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36316,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708836,"statfs":{"total":95995035648,"available":95957848064,"internally_reserved":0,"allocated":1003520,"data_stored":478033,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":36174273},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":8,"up_from":59,"seq":253403070496,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27992,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717160,"statfs":{"total":95995035648,"available":95966371840,"internally_reserved":0,"allocated":991232,"data_stored":477819,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332133,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28564,"kb_used_data":1544,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716588,"statfs":{"total":95995035648,"available":95965786112,"internally_reserved":0,"allocated":1581056,"data_stored":1068165,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495659,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27984,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717168,"statfs":{"total":95995035648,"available":95966380032,"internally_reserved":0,"allocated":983040,"data_stored":477215,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574915,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27988,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717164,"statfs":{"total":95995035648,"available":95966375936,"internally_reserved":0,"allocated":995328,"data_stored":481057,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738441,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28004,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717148,"statfs":{"total":95995035648,"available":95966359552,"internally_reserved":0,"allocated":1011712,"data_stored":478693,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411391,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27992,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717160,"statfs":{"total":95995035648,"available":95966371840,"internally_reserved":0,"allocated":999424,"data_stored":481091,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182457,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36312,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708840,"statfs":{"total":95995035648,"available":95957852160,"internally_reserved":0,"allocated":999424,"data_stored":477499,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920820,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27988,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717164,"statfs":{"total":95995035648,"available":95966375936,"internally_reserved":0,"allocated":987136,"data_stored":477237,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757296,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32732,"kb_used_data":1552,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712420,"statfs":{"total":95995035648,"available":95961518080,"internally_reserved":0,"allocated":1589248,"data_stored":1067765,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":3876,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":25110,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-04T09:29:06.884 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-04T09:29:07.026 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:07.026+0000 7f8e88e66640 1 -- 172.21.15.52:0/2753469522 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8e841027f0 msgr2=0x7f8e84102c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:29:07.026 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:07.026+0000 7f8e88e66640 1 --2- 172.21.15.52:0/2753469522 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8e841027f0 0x7f8e84102c70 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f8e6c009900 tx=0x7f8e6c02bc30 comp rx=0 tx=0).stop 2024-03-04T09:29:07.026 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:07.027+0000 7f8e88e66640 1 -- 172.21.15.52:0/2753469522 shutdown_connections 2024-03-04T09:29:07.026 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:07.027+0000 7f8e88e66640 1 --2- 172.21.15.52:0/2753469522 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8e841031b0 0x7f8e84109a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:29:07.026 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:07.027+0000 7f8e88e66640 1 --2- 172.21.15.52:0/2753469522 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8e841027f0 0x7f8e84102c70 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:29:07.026 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:07.027+0000 7f8e88e66640 1 --2- 172.21.15.52:0/2753469522 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8e84101600 0x7f8e84101a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:29:07.026 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:07.027+0000 7f8e88e66640 1 -- 172.21.15.52:0/2753469522 >> 172.21.15.52:0/2753469522 conn(0x7f8e840fcbb0 msgr2=0x7f8e840fefd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:29:07.026 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:07.027+0000 7f8e88e66640 1 -- 172.21.15.52:0/2753469522 shutdown_connections 2024-03-04T09:29:07.026 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:07.027+0000 7f8e88e66640 1 -- 172.21.15.52:0/2753469522 wait complete. 2024-03-04T09:29:07.026 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:07.027+0000 7f8e88e66640 1 Processor -- start 2024-03-04T09:29:07.026 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:07.027+0000 7f8e88e66640 1 -- start start 2024-03-04T09:29:07.027 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:07.027+0000 7f8e88e66640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8e84101600 0x7f8e841978f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:29:07.027 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:07.027+0000 7f8e88e66640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8e841027f0 0x7f8e84197e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:29:07.027 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:07.027+0000 7f8e88e66640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8e841031b0 0x7f8e8419eea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:29:07.027 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:07.028+0000 7f8e88e66640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e8410d280 con 0x7f8e84101600 2024-03-04T09:29:07.027 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:07.028+0000 7f8e88e66640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e8410d100 con 0x7f8e841027f0 2024-03-04T09:29:07.027 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:07.028+0000 7f8e88e66640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e8410d400 con 0x7f8e841031b0 2024-03-04T09:29:07.027 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:07.028+0000 7f8e8259c640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8e84101600 0x7f8e841978f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:29:07.027 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:07.028+0000 7f8e8259c640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8e84101600 0x7f8e841978f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:47052/0 (socket says 172.21.15.52:47052) 2024-03-04T09:29:07.027 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:07.028+0000 7f8e8259c640 1 -- 172.21.15.52:0/1651330987 learned_addr learned my addr 172.21.15.52:0/1651330987 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-04T09:29:07.027 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:07.028+0000 7f8e81d9b640 1 --2- 172.21.15.52:0/1651330987 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8e841027f0 0x7f8e84197e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:29:07.027 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:07.028+0000 7f8e82d9d640 1 --2- 172.21.15.52:0/1651330987 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8e841031b0 0x7f8e8419eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:29:07.027 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:07.028+0000 7f8e8259c640 1 -- 172.21.15.52:0/1651330987 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8e841031b0 msgr2=0x7f8e8419eea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:29:07.027 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:07.028+0000 7f8e8259c640 1 --2- 172.21.15.52:0/1651330987 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8e841031b0 0x7f8e8419eea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:29:07.027 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:07.028+0000 7f8e8259c640 1 -- 172.21.15.52:0/1651330987 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8e841027f0 msgr2=0x7f8e84197e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:29:07.028 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:07.028+0000 7f8e8259c640 1 --2- 172.21.15.52:0/1651330987 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8e841027f0 0x7f8e84197e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:29:07.028 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:07.028+0000 7f8e8259c640 1 -- 172.21.15.52:0/1651330987 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e8419f510 con 0x7f8e84101600 2024-03-04T09:29:07.028 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:07.028+0000 7f8e8259c640 1 --2- 172.21.15.52:0/1651330987 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8e84101600 0x7f8e841978f0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f8e7c00d950 tx=0x7f8e7c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:29:07.028 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:07.028+0000 7f8e7b7fe640 1 -- 172.21.15.52:0/1651330987 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e7c017070 con 0x7f8e84101600 2024-03-04T09:29:07.028 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:07.028+0000 7f8e7b7fe640 1 -- 172.21.15.52:0/1651330987 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8e7c009c30 con 0x7f8e84101600 2024-03-04T09:29:07.028 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:07.028+0000 7f8e7b7fe640 1 -- 172.21.15.52:0/1651330987 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e7c00f600 con 0x7f8e84101600 2024-03-04T09:29:07.028 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:07.028+0000 7f8e88e66640 1 -- 172.21.15.52:0/1651330987 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e8419f800 con 0x7f8e84101600 2024-03-04T09:29:07.028 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:07.028+0000 7f8e88e66640 1 -- 172.21.15.52:0/1651330987 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e8419fd20 con 0x7f8e84101600 2024-03-04T09:29:07.029 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:07.029+0000 7f8e88e66640 1 -- 172.21.15.52:0/1651330987 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8e84108920 con 0x7f8e84101600 2024-03-04T09:29:07.029 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:07.030+0000 7f8e7b7fe640 1 -- 172.21.15.52:0/1651330987 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 17) v1 ==== 136807+0+0 (secure 0 0 0) 0x7f8e7c01a070 con 0x7f8e84101600 2024-03-04T09:29:07.029 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:07.031+0000 7f8e7b7fe640 1 --2- 172.21.15.52:0/1651330987 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f8e700a1e90 0x7f8e700a4350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:29:07.030 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:07.031+0000 7f8e81d9b640 1 --2- 172.21.15.52:0/1651330987 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f8e700a1e90 0x7f8e700a4350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:29:07.030 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:07.031+0000 7f8e7b7fe640 1 -- 172.21.15.52:0/1651330987 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8802+0+0 (secure 0 0 0) 0x7f8e7c0cdac0 con 0x7f8e84101600 2024-03-04T09:29:07.030 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:07.031+0000 7f8e81d9b640 1 --2- 172.21.15.52:0/1651330987 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f8e700a1e90 0x7f8e700a4350 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f8e6c02c7b0 tx=0x7f8e6c005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:29:07.032 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:07.035+0000 7f8e7b7fe640 1 -- 172.21.15.52:0/1651330987 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f8e7c009de0 con 0x7f8e84101600 2024-03-04T09:29:07.212 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:07.215+0000 7f8e88e66640 1 -- 172.21.15.52:0/1651330987 --> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f8e84061a10 con 0x7f8e700a1e90 2024-03-04T09:29:07.223 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:07.224+0000 7f8e7b7fe640 1 -- 172.21.15.52:0/1651330987 <== mgr.14152 v2:172.21.15.52:6800/3036033923 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+340274 (secure 0 0 0) 0x7f8e84061a10 con 0x7f8e700a1e90 2024-03-04T09:29:07.226 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-03-04T09:29:07.228 INFO:teuthology.orchestra.run.smithi052.stderr:dumped all 2024-03-04T09:29:07.230 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:07.228+0000 7f8e88e66640 1 -- 172.21.15.52:0/1651330987 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f8e700a1e90 msgr2=0x7f8e700a4350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:29:07.230 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:07.228+0000 7f8e88e66640 1 --2- 172.21.15.52:0/1651330987 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f8e700a1e90 0x7f8e700a4350 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f8e6c02c7b0 tx=0x7f8e6c005fd0 comp rx=0 tx=0).stop 2024-03-04T09:29:07.230 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:07.228+0000 7f8e88e66640 1 -- 172.21.15.52:0/1651330987 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8e84101600 msgr2=0x7f8e841978f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:29:07.230 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:07.228+0000 7f8e88e66640 1 --2- 172.21.15.52:0/1651330987 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8e84101600 0x7f8e841978f0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f8e7c00d950 tx=0x7f8e7c00de10 comp rx=0 tx=0).stop 2024-03-04T09:29:07.230 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:07.228+0000 7f8e88e66640 1 -- 172.21.15.52:0/1651330987 shutdown_connections 2024-03-04T09:29:07.230 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:07.228+0000 7f8e88e66640 1 --2- 172.21.15.52:0/1651330987 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f8e700a1e90 0x7f8e700a4350 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:29:07.230 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:07.228+0000 7f8e88e66640 1 --2- 172.21.15.52:0/1651330987 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8e841031b0 0x7f8e8419eea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:29:07.230 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:07.228+0000 7f8e88e66640 1 --2- 172.21.15.52:0/1651330987 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8e841027f0 0x7f8e84197e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:29:07.230 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:07.228+0000 7f8e88e66640 1 --2- 172.21.15.52:0/1651330987 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8e84101600 0x7f8e841978f0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:29:07.231 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:07.228+0000 7f8e88e66640 1 -- 172.21.15.52:0/1651330987 >> 172.21.15.52:0/1651330987 conn(0x7f8e840fcbb0 msgr2=0x7f8e84105af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:29:07.231 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:07.228+0000 7f8e88e66640 1 -- 172.21.15.52:0/1651330987 shutdown_connections 2024-03-04T09:29:07.231 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:07.228+0000 7f8e88e66640 1 -- 172.21.15.52:0/1651330987 wait complete. 2024-03-04T09:29:07.234 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:29:07 smithi052 ceph-mon[32473]: pgmap v302: 129 pgs: 129 active+clean; 600 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 2.2 KiB/s wr, 3 op/s 2024-03-04T09:29:07.235 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:29:07 smithi052 ceph-mon[32473]: from='client.14913 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T09:29:07.235 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:29:07 smithi052 ceph-mon[32473]: from='client.14919 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T09:29:07.235 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:29:07 smithi052 ceph-mon[32473]: from='client.? 172.21.15.52:0/2501347692' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-04T09:29:07.235 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:29:07 smithi052 ceph-mon[32473]: from='client.? 172.21.15.52:0/3689523684' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-04T09:29:07.236 INFO:teuthology.orchestra.run.smithi052.stdout:{"pg_ready":true,"pg_map":{"version":302,"stamp":"2024-03-04T09:29:05.488318+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":614487,"num_objects":34,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":0,"num_read":145,"num_read_kb":147,"num_write":263,"num_write_kb":2981,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":22,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":207,"ondisk_log_size":207,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":366784,"kb_used_data":13440,"kb_used_omap":18,"kb_used_meta":353197,"kb_avail":1124575040,"statfs":{"total":1151940427776,"available":1151564840960,"internally_reserved":0,"allocated":13762560,"data_stored":7526461,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19140,"internal_metadata":361674044},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19620,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":9,"num_read_kb":28,"num_write":35,"num_write_kb":26,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":8,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003333"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-04T09:28:42.972654+0000","last_change":"2024-03-04T09:28:37.914945+0000","last_active":"2024-03-04T09:28:42.972654+0000","last_peered":"2024-03-04T09:28:42.972654+0000","last_clean":"2024-03-04T09:28:42.972654+0000","last_became_active":"2024-03-04T09:28:37.914647+0000","last_became_peered":"2024-03-04T09:28:37.914647+0000","last_unstale":"2024-03-04T09:28:42.972654+0000","last_undegraded":"2024-03-04T09:28:42.972654+0000","last_fullsized":"2024-03-04T09:28:42.972654+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_clean_scrub_stamp":"2024-03-04T09:28:36.887656+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T13:46:41.262758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-04T09:28:42.972542+0000","last_change":"2024-03-04T09:28:37.918650+0000","last_active":"2024-03-04T09:28:42.972542+0000","last_peered":"2024-03-04T09:28:42.972542+0000","last_clean":"2024-03-04T09:28:42.972542+0000","last_became_active":"2024-03-04T09:28:37.918314+0000","last_became_peered":"2024-03-04T09:28:37.918314+0000","last_unstale":"2024-03-04T09:28:42.972542+0000","last_undegraded":"2024-03-04T09:28:42.972542+0000","last_fullsized":"2024-03-04T09:28:42.972542+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_clean_scrub_stamp":"2024-03-04T09:28:36.887656+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T18:36:41.948549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-04T09:28:42.972499+0000","last_change":"2024-03-04T09:28:37.920140+0000","last_active":"2024-03-04T09:28:42.972499+0000","last_peered":"2024-03-04T09:28:42.972499+0000","last_clean":"2024-03-04T09:28:42.972499+0000","last_became_active":"2024-03-04T09:28:37.917379+0000","last_became_peered":"2024-03-04T09:28:37.917379+0000","last_unstale":"2024-03-04T09:28:42.972499+0000","last_undegraded":"2024-03-04T09:28:42.972499+0000","last_fullsized":"2024-03-04T09:28:42.972499+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_clean_scrub_stamp":"2024-03-04T09:28:36.887656+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T17:23:33.144752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-04T09:28:42.972605+0000","last_change":"2024-03-04T09:28:37.916999+0000","last_active":"2024-03-04T09:28:42.972605+0000","last_peered":"2024-03-04T09:28:42.972605+0000","last_clean":"2024-03-04T09:28:42.972605+0000","last_became_active":"2024-03-04T09:28:37.916597+0000","last_became_peered":"2024-03-04T09:28:37.916597+0000","last_unstale":"2024-03-04T09:28:42.972605+0000","last_undegraded":"2024-03-04T09:28:42.972605+0000","last_fullsized":"2024-03-04T09:28:42.972605+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_clean_scrub_stamp":"2024-03-04T09:28:36.887656+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T19:23:20.556283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-04T09:28:42.971678+0000","last_change":"2024-03-04T09:28:37.904298+0000","last_active":"2024-03-04T09:28:42.971678+0000","last_peered":"2024-03-04T09:28:42.971678+0000","last_clean":"2024-03-04T09:28:42.971678+0000","last_became_active":"2024-03-04T09:28:37.903966+0000","last_became_peered":"2024-03-04T09:28:37.903966+0000","last_unstale":"2024-03-04T09:28:42.971678+0000","last_undegraded":"2024-03-04T09:28:42.971678+0000","last_fullsized":"2024-03-04T09:28:42.971678+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_clean_scrub_stamp":"2024-03-04T09:28:36.887656+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T20:14:21.615521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-04T09:28:42.971741+0000","last_change":"2024-03-04T09:28:37.917800+0000","last_active":"2024-03-04T09:28:42.971741+0000","last_peered":"2024-03-04T09:28:42.971741+0000","last_clean":"2024-03-04T09:28:42.971741+0000","last_became_active":"2024-03-04T09:28:37.917500+0000","last_became_peered":"2024-03-04T09:28:37.917500+0000","last_unstale":"2024-03-04T09:28:42.971741+0000","last_undegraded":"2024-03-04T09:28:42.971741+0000","last_fullsized":"2024-03-04T09:28:42.971741+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_clean_scrub_stamp":"2024-03-04T09:28:36.887656+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T20:06:31.330705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-04T09:28:42.933649+0000","last_change":"2024-03-04T09:28:37.917708+0000","last_active":"2024-03-04T09:28:42.933649+0000","last_peered":"2024-03-04T09:28:42.933649+0000","last_clean":"2024-03-04T09:28:42.933649+0000","last_became_active":"2024-03-04T09:28:37.917323+0000","last_became_peered":"2024-03-04T09:28:37.917323+0000","last_unstale":"2024-03-04T09:28:42.933649+0000","last_undegraded":"2024-03-04T09:28:42.933649+0000","last_fullsized":"2024-03-04T09:28:42.933649+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_clean_scrub_stamp":"2024-03-04T09:28:36.887656+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T11:10:46.345702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-04T09:28:42.971142+0000","last_change":"2024-03-04T09:28:37.925427+0000","last_active":"2024-03-04T09:28:42.971142+0000","last_peered":"2024-03-04T09:28:42.971142+0000","last_clean":"2024-03-04T09:28:42.971142+0000","last_became_active":"2024-03-04T09:28:37.925159+0000","last_became_peered":"2024-03-04T09:28:37.925159+0000","last_unstale":"2024-03-04T09:28:42.971142+0000","last_undegraded":"2024-03-04T09:28:42.971142+0000","last_fullsized":"2024-03-04T09:28:42.971142+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_clean_scrub_stamp":"2024-03-04T09:28:36.887656+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T17:29:51.564928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-04T09:28:42.971251+0000","last_change":"2024-03-04T09:28:37.928279+0000","last_active":"2024-03-04T09:28:42.971251+0000","last_peered":"2024-03-04T09:28:42.971251+0000","last_clean":"2024-03-04T09:28:42.971251+0000","last_became_active":"2024-03-04T09:28:37.927934+0000","last_became_peered":"2024-03-04T09:28:37.927934+0000","last_unstale":"2024-03-04T09:28:42.971251+0000","last_undegraded":"2024-03-04T09:28:42.971251+0000","last_fullsized":"2024-03-04T09:28:42.971251+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_clean_scrub_stamp":"2024-03-04T09:28:36.887656+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T21:23:06.592526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-04T09:28:42.972629+0000","last_change":"2024-03-04T09:28:37.918990+0000","last_active":"2024-03-04T09:28:42.972629+0000","last_peered":"2024-03-04T09:28:42.972629+0000","last_clean":"2024-03-04T09:28:42.972629+0000","last_became_active":"2024-03-04T09:28:37.918599+0000","last_became_peered":"2024-03-04T09:28:37.918599+0000","last_unstale":"2024-03-04T09:28:42.972629+0000","last_undegraded":"2024-03-04T09:28:42.972629+0000","last_fullsized":"2024-03-04T09:28:42.972629+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_clean_scrub_stamp":"2024-03-04T09:28:36.887656+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T20:39:09.603428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-04T09:28:42.934117+0000","last_change":"2024-03-04T09:28:37.914234+0000","last_active":"2024-03-04T09:28:42.934117+0000","last_peered":"2024-03-04T09:28:42.934117+0000","last_clean":"2024-03-04T09:28:42.934117+0000","last_became_active":"2024-03-04T09:28:37.913897+0000","last_became_peered":"2024-03-04T09:28:37.913897+0000","last_unstale":"2024-03-04T09:28:42.934117+0000","last_undegraded":"2024-03-04T09:28:42.934117+0000","last_fullsized":"2024-03-04T09:28:42.934117+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_clean_scrub_stamp":"2024-03-04T09:28:36.887656+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T15:43:53.634817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-04T09:28:42.972516+0000","last_change":"2024-03-04T09:28:37.919935+0000","last_active":"2024-03-04T09:28:42.972516+0000","last_peered":"2024-03-04T09:28:42.972516+0000","last_clean":"2024-03-04T09:28:42.972516+0000","last_became_active":"2024-03-04T09:28:37.919447+0000","last_became_peered":"2024-03-04T09:28:37.919447+0000","last_unstale":"2024-03-04T09:28:42.972516+0000","last_undegraded":"2024-03-04T09:28:42.972516+0000","last_fullsized":"2024-03-04T09:28:42.972516+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_clean_scrub_stamp":"2024-03-04T09:28:36.887656+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T09:44:03.678767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-04T09:28:42.973524+0000","last_change":"2024-03-04T09:28:37.900570+0000","last_active":"2024-03-04T09:28:42.973524+0000","last_peered":"2024-03-04T09:28:42.973524+0000","last_clean":"2024-03-04T09:28:42.973524+0000","last_became_active":"2024-03-04T09:28:37.900186+0000","last_became_peered":"2024-03-04T09:28:37.900186+0000","last_unstale":"2024-03-04T09:28:42.973524+0000","last_undegraded":"2024-03-04T09:28:42.973524+0000","last_fullsized":"2024-03-04T09:28:42.973524+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_clean_scrub_stamp":"2024-03-04T09:28:36.887656+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T21:17:12.915989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-04T09:28:42.971487+0000","last_change":"2024-03-04T09:28:37.920447+0000","last_active":"2024-03-04T09:28:42.971487+0000","last_peered":"2024-03-04T09:28:42.971487+0000","last_clean":"2024-03-04T09:28:42.971487+0000","last_became_active":"2024-03-04T09:28:37.919580+0000","last_became_peered":"2024-03-04T09:28:37.919580+0000","last_unstale":"2024-03-04T09:28:42.971487+0000","last_undegraded":"2024-03-04T09:28:42.971487+0000","last_fullsized":"2024-03-04T09:28:42.971487+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_clean_scrub_stamp":"2024-03-04T09:28:36.887656+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T13:25:56.434285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-04T09:28:42.971892+0000","last_change":"2024-03-04T09:28:37.913735+0000","last_active":"2024-03-04T09:28:42.971892+0000","last_peered":"2024-03-04T09:28:42.971892+0000","last_clean":"2024-03-04T09:28:42.971892+0000","last_became_active":"2024-03-04T09:28:37.913388+0000","last_became_peered":"2024-03-04T09:28:37.913388+0000","last_unstale":"2024-03-04T09:28:42.971892+0000","last_undegraded":"2024-03-04T09:28:42.971892+0000","last_fullsized":"2024-03-04T09:28:42.971892+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_clean_scrub_stamp":"2024-03-04T09:28:36.887656+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T16:04:28.468313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-04T09:28:42.970900+0000","last_change":"2024-03-04T09:28:37.925031+0000","last_active":"2024-03-04T09:28:42.970900+0000","last_peered":"2024-03-04T09:28:42.970900+0000","last_clean":"2024-03-04T09:28:42.970900+0000","last_became_active":"2024-03-04T09:28:37.923561+0000","last_became_peered":"2024-03-04T09:28:37.923561+0000","last_unstale":"2024-03-04T09:28:42.970900+0000","last_undegraded":"2024-03-04T09:28:42.970900+0000","last_fullsized":"2024-03-04T09:28:42.970900+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_clean_scrub_stamp":"2024-03-04T09:28:36.887656+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T16:29:04.266384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-04T09:28:42.972454+0000","last_change":"2024-03-04T09:28:37.912029+0000","last_active":"2024-03-04T09:28:42.972454+0000","last_peered":"2024-03-04T09:28:42.972454+0000","last_clean":"2024-03-04T09:28:42.972454+0000","last_became_active":"2024-03-04T09:28:37.911726+0000","last_became_peered":"2024-03-04T09:28:37.911726+0000","last_unstale":"2024-03-04T09:28:42.972454+0000","last_undegraded":"2024-03-04T09:28:42.972454+0000","last_fullsized":"2024-03-04T09:28:42.972454+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_clean_scrub_stamp":"2024-03-04T09:28:36.887656+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T12:32:03.385375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-04T09:28:42.932750+0000","last_change":"2024-03-04T09:28:37.914448+0000","last_active":"2024-03-04T09:28:42.932750+0000","last_peered":"2024-03-04T09:28:42.932750+0000","last_clean":"2024-03-04T09:28:42.932750+0000","last_became_active":"2024-03-04T09:28:37.914036+0000","last_became_peered":"2024-03-04T09:28:37.914036+0000","last_unstale":"2024-03-04T09:28:42.932750+0000","last_undegraded":"2024-03-04T09:28:42.932750+0000","last_fullsized":"2024-03-04T09:28:42.932750+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_clean_scrub_stamp":"2024-03-04T09:28:36.887656+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T20:57:10.348039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-04T09:28:42.932814+0000","last_change":"2024-03-04T09:28:37.907640+0000","last_active":"2024-03-04T09:28:42.932814+0000","last_peered":"2024-03-04T09:28:42.932814+0000","last_clean":"2024-03-04T09:28:42.932814+0000","last_became_active":"2024-03-04T09:28:37.907350+0000","last_became_peered":"2024-03-04T09:28:37.907350+0000","last_unstale":"2024-03-04T09:28:42.932814+0000","last_undegraded":"2024-03-04T09:28:42.932814+0000","last_fullsized":"2024-03-04T09:28:42.932814+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_clean_scrub_stamp":"2024-03-04T09:28:36.887656+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T10:17:37.576254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-04T09:28:42.932424+0000","last_change":"2024-03-04T09:28:41.950312+0000","last_active":"2024-03-04T09:28:42.932424+0000","last_peered":"2024-03-04T09:28:42.932424+0000","last_clean":"2024-03-04T09:28:42.932424+0000","last_became_active":"2024-03-04T09:28:41.949882+0000","last_became_peered":"2024-03-04T09:28:41.949882+0000","last_unstale":"2024-03-04T09:28:42.932424+0000","last_undegraded":"2024-03-04T09:28:42.932424+0000","last_fullsized":"2024-03-04T09:28:42.932424+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:40.917115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:40.917115+0000","last_clean_scrub_stamp":"2024-03-04T09:28:40.917115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T14:20:48.740518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"89'9","reported_seq":32,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-04T09:29:03.002641+0000","last_change":"2024-03-04T09:28:36.904805+0000","last_active":"2024-03-04T09:29:03.002641+0000","last_peered":"2024-03-04T09:29:03.002641+0000","last_clean":"2024-03-04T09:29:03.002641+0000","last_became_active":"2024-03-04T09:28:36.904469+0000","last_became_peered":"2024-03-04T09:28:36.904469+0000","last_unstale":"2024-03-04T09:29:03.002641+0000","last_undegraded":"2024-03-04T09:29:03.002641+0000","last_fullsized":"2024-03-04T09:29:03.002641+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:35.885776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:35.885776+0000","last_clean_scrub_stamp":"2024-03-04T09:28:35.885776+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T18:35:35.992042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-04T09:28:42.972245+0000","last_change":"2024-03-04T09:28:37.920102+0000","last_active":"2024-03-04T09:28:42.972245+0000","last_peered":"2024-03-04T09:28:42.972245+0000","last_clean":"2024-03-04T09:28:42.972245+0000","last_became_active":"2024-03-04T09:28:37.919749+0000","last_became_peered":"2024-03-04T09:28:37.919749+0000","last_unstale":"2024-03-04T09:28:42.972245+0000","last_undegraded":"2024-03-04T09:28:42.972245+0000","last_fullsized":"2024-03-04T09:28:42.972245+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_clean_scrub_stamp":"2024-03-04T09:28:36.887656+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T19:16:36.269782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-04T09:28:42.971614+0000","last_change":"2024-03-04T09:28:41.950477+0000","last_active":"2024-03-04T09:28:42.971614+0000","last_peered":"2024-03-04T09:28:42.971614+0000","last_clean":"2024-03-04T09:28:42.971614+0000","last_became_active":"2024-03-04T09:28:41.950087+0000","last_became_peered":"2024-03-04T09:28:41.950087+0000","last_unstale":"2024-03-04T09:28:42.971614+0000","last_undegraded":"2024-03-04T09:28:42.971614+0000","last_fullsized":"2024-03-04T09:28:42.971614+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:40.917115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:40.917115+0000","last_clean_scrub_stamp":"2024-03-04T09:28:40.917115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T14:48:17.327956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"89'2","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-04T09:28:42.972371+0000","last_change":"2024-03-04T09:28:36.913923+0000","last_active":"2024-03-04T09:28:42.972371+0000","last_peered":"2024-03-04T09:28:42.972371+0000","last_clean":"2024-03-04T09:28:42.972371+0000","last_became_active":"2024-03-04T09:28:36.911076+0000","last_became_peered":"2024-03-04T09:28:36.911076+0000","last_unstale":"2024-03-04T09:28:42.972371+0000","last_undegraded":"2024-03-04T09:28:42.972371+0000","last_fullsized":"2024-03-04T09:28:42.972371+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:35.885776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:35.885776+0000","last_clean_scrub_stamp":"2024-03-04T09:28:35.885776+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T16:13:23.004827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-04T09:28:42.972616+0000","last_change":"2024-03-04T09:28:37.912747+0000","last_active":"2024-03-04T09:28:42.972616+0000","last_peered":"2024-03-04T09:28:42.972616+0000","last_clean":"2024-03-04T09:28:42.972616+0000","last_became_active":"2024-03-04T09:28:37.912436+0000","last_became_peered":"2024-03-04T09:28:37.912436+0000","last_unstale":"2024-03-04T09:28:42.972616+0000","last_undegraded":"2024-03-04T09:28:42.972616+0000","last_fullsized":"2024-03-04T09:28:42.972616+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_clean_scrub_stamp":"2024-03-04T09:28:36.887656+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T14:52:47.318456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-04T09:28:42.972214+0000","last_change":"2024-03-04T09:28:41.948878+0000","last_active":"2024-03-04T09:28:42.972214+0000","last_peered":"2024-03-04T09:28:42.972214+0000","last_clean":"2024-03-04T09:28:42.972214+0000","last_became_active":"2024-03-04T09:28:41.948233+0000","last_became_peered":"2024-03-04T09:28:41.948233+0000","last_unstale":"2024-03-04T09:28:42.972214+0000","last_undegraded":"2024-03-04T09:28:42.972214+0000","last_fullsized":"2024-03-04T09:28:42.972214+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:40.917115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:40.917115+0000","last_clean_scrub_stamp":"2024-03-04T09:28:40.917115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T18:12:02.571086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"84'2","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-04T09:28:42.972126+0000","last_change":"2024-03-04T09:28:36.910279+0000","last_active":"2024-03-04T09:28:42.972126+0000","last_peered":"2024-03-04T09:28:42.972126+0000","last_clean":"2024-03-04T09:28:42.972126+0000","last_became_active":"2024-03-04T09:28:36.909214+0000","last_became_peered":"2024-03-04T09:28:36.909214+0000","last_unstale":"2024-03-04T09:28:42.972126+0000","last_undegraded":"2024-03-04T09:28:42.972126+0000","last_fullsized":"2024-03-04T09:28:42.972126+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:35.885776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:35.885776+0000","last_clean_scrub_stamp":"2024-03-04T09:28:35.885776+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T19:22:45.808418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-04T09:28:42.971105+0000","last_change":"2024-03-04T09:28:37.914593+0000","last_active":"2024-03-04T09:28:42.971105+0000","last_peered":"2024-03-04T09:28:42.971105+0000","last_clean":"2024-03-04T09:28:42.971105+0000","last_became_active":"2024-03-04T09:28:37.914217+0000","last_became_peered":"2024-03-04T09:28:37.914217+0000","last_unstale":"2024-03-04T09:28:42.971105+0000","last_undegraded":"2024-03-04T09:28:42.971105+0000","last_fullsized":"2024-03-04T09:28:42.971105+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_clean_scrub_stamp":"2024-03-04T09:28:36.887656+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T19:32:39.893906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-04T09:28:42.972082+0000","last_change":"2024-03-04T09:28:41.951685+0000","last_active":"2024-03-04T09:28:42.972082+0000","last_peered":"2024-03-04T09:28:42.972082+0000","last_clean":"2024-03-04T09:28:42.972082+0000","last_became_active":"2024-03-04T09:28:41.951178+0000","last_became_peered":"2024-03-04T09:28:41.951178+0000","last_unstale":"2024-03-04T09:28:42.972082+0000","last_undegraded":"2024-03-04T09:28:42.972082+0000","last_fullsized":"2024-03-04T09:28:42.972082+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:40.917115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:40.917115+0000","last_clean_scrub_stamp":"2024-03-04T09:28:40.917115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T10:23:19.028344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"84'2","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-04T09:28:42.971982+0000","last_change":"2024-03-04T09:28:36.908066+0000","last_active":"2024-03-04T09:28:42.971982+0000","last_peered":"2024-03-04T09:28:42.971982+0000","last_clean":"2024-03-04T09:28:42.971982+0000","last_became_active":"2024-03-04T09:28:36.907761+0000","last_became_peered":"2024-03-04T09:28:36.907761+0000","last_unstale":"2024-03-04T09:28:42.971982+0000","last_undegraded":"2024-03-04T09:28:42.971982+0000","last_fullsized":"2024-03-04T09:28:42.971982+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:35.885776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:35.885776+0000","last_clean_scrub_stamp":"2024-03-04T09:28:35.885776+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T20:53:11.559719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-04T09:28:42.973318+0000","last_change":"2024-03-04T09:28:37.917667+0000","last_active":"2024-03-04T09:28:42.973318+0000","last_peered":"2024-03-04T09:28:42.973318+0000","last_clean":"2024-03-04T09:28:42.973318+0000","last_became_active":"2024-03-04T09:28:37.917320+0000","last_became_peered":"2024-03-04T09:28:37.917320+0000","last_unstale":"2024-03-04T09:28:42.973318+0000","last_undegraded":"2024-03-04T09:28:42.973318+0000","last_fullsized":"2024-03-04T09:28:42.973318+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_clean_scrub_stamp":"2024-03-04T09:28:36.887656+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T12:17:07.162334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-04T09:28:42.932737+0000","last_change":"2024-03-04T09:28:41.946078+0000","last_active":"2024-03-04T09:28:42.932737+0000","last_peered":"2024-03-04T09:28:42.932737+0000","last_clean":"2024-03-04T09:28:42.932737+0000","last_became_active":"2024-03-04T09:28:41.945639+0000","last_became_peered":"2024-03-04T09:28:41.945639+0000","last_unstale":"2024-03-04T09:28:42.932737+0000","last_undegraded":"2024-03-04T09:28:42.932737+0000","last_fullsized":"2024-03-04T09:28:42.932737+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:40.917115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:40.917115+0000","last_clean_scrub_stamp":"2024-03-04T09:28:40.917115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T16:31:41.939728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"84'1","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-04T09:28:42.973125+0000","last_change":"2024-03-04T09:28:36.905596+0000","last_active":"2024-03-04T09:28:42.973125+0000","last_peered":"2024-03-04T09:28:42.973125+0000","last_clean":"2024-03-04T09:28:42.973125+0000","last_became_active":"2024-03-04T09:28:36.905270+0000","last_became_peered":"2024-03-04T09:28:36.905270+0000","last_unstale":"2024-03-04T09:28:42.973125+0000","last_undegraded":"2024-03-04T09:28:42.973125+0000","last_fullsized":"2024-03-04T09:28:42.973125+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:35.885776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:35.885776+0000","last_clean_scrub_stamp":"2024-03-04T09:28:35.885776+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T09:43:32.986704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-04T09:28:42.972489+0000","last_change":"2024-03-04T09:28:37.906282+0000","last_active":"2024-03-04T09:28:42.972489+0000","last_peered":"2024-03-04T09:28:42.972489+0000","last_clean":"2024-03-04T09:28:42.972489+0000","last_became_active":"2024-03-04T09:28:37.905964+0000","last_became_peered":"2024-03-04T09:28:37.905964+0000","last_unstale":"2024-03-04T09:28:42.972489+0000","last_undegraded":"2024-03-04T09:28:42.972489+0000","last_fullsized":"2024-03-04T09:28:42.972489+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_clean_scrub_stamp":"2024-03-04T09:28:36.887656+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T11:13:16.287959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-04T09:28:42.932641+0000","last_change":"2024-03-04T09:28:41.949196+0000","last_active":"2024-03-04T09:28:42.932641+0000","last_peered":"2024-03-04T09:28:42.932641+0000","last_clean":"2024-03-04T09:28:42.932641+0000","last_became_active":"2024-03-04T09:28:41.948783+0000","last_became_peered":"2024-03-04T09:28:41.948783+0000","last_unstale":"2024-03-04T09:28:42.932641+0000","last_undegraded":"2024-03-04T09:28:42.932641+0000","last_fullsized":"2024-03-04T09:28:42.932641+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:40.917115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:40.917115+0000","last_clean_scrub_stamp":"2024-03-04T09:28:40.917115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T13:42:05.888482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"89'3","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-04T09:29:03.000480+0000","last_change":"2024-03-04T09:28:36.906900+0000","last_active":"2024-03-04T09:29:03.000480+0000","last_peered":"2024-03-04T09:29:03.000480+0000","last_clean":"2024-03-04T09:29:03.000480+0000","last_became_active":"2024-03-04T09:28:36.906556+0000","last_became_peered":"2024-03-04T09:28:36.906556+0000","last_unstale":"2024-03-04T09:29:03.000480+0000","last_undegraded":"2024-03-04T09:29:03.000480+0000","last_fullsized":"2024-03-04T09:29:03.000480+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:35.885776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:35.885776+0000","last_clean_scrub_stamp":"2024-03-04T09:28:35.885776+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T18:13:56.535621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-04T09:28:42.973318+0000","last_change":"2024-03-04T09:28:37.912123+0000","last_active":"2024-03-04T09:28:42.973318+0000","last_peered":"2024-03-04T09:28:42.973318+0000","last_clean":"2024-03-04T09:28:42.973318+0000","last_became_active":"2024-03-04T09:28:37.911810+0000","last_became_peered":"2024-03-04T09:28:37.911810+0000","last_unstale":"2024-03-04T09:28:42.973318+0000","last_undegraded":"2024-03-04T09:28:42.973318+0000","last_fullsized":"2024-03-04T09:28:42.973318+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_clean_scrub_stamp":"2024-03-04T09:28:36.887656+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T18:14:25.229989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-04T09:28:42.972521+0000","last_change":"2024-03-04T09:28:41.942142+0000","last_active":"2024-03-04T09:28:42.972521+0000","last_peered":"2024-03-04T09:28:42.972521+0000","last_clean":"2024-03-04T09:28:42.972521+0000","last_became_active":"2024-03-04T09:28:41.941740+0000","last_became_peered":"2024-03-04T09:28:41.941740+0000","last_unstale":"2024-03-04T09:28:42.972521+0000","last_undegraded":"2024-03-04T09:28:42.972521+0000","last_fullsized":"2024-03-04T09:28:42.972521+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:40.917115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:40.917115+0000","last_clean_scrub_stamp":"2024-03-04T09:28:40.917115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T19:40:25.163087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-04T09:28:42.973184+0000","last_change":"2024-03-04T09:28:36.902206+0000","last_active":"2024-03-04T09:28:42.973184+0000","last_peered":"2024-03-04T09:28:42.973184+0000","last_clean":"2024-03-04T09:28:42.973184+0000","last_became_active":"2024-03-04T09:28:36.901847+0000","last_became_peered":"2024-03-04T09:28:36.901847+0000","last_unstale":"2024-03-04T09:28:42.973184+0000","last_undegraded":"2024-03-04T09:28:42.973184+0000","last_fullsized":"2024-03-04T09:28:42.973184+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:35.885776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:35.885776+0000","last_clean_scrub_stamp":"2024-03-04T09:28:35.885776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T12:32:05.755876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-04T09:28:42.971350+0000","last_change":"2024-03-04T09:28:37.906175+0000","last_active":"2024-03-04T09:28:42.971350+0000","last_peered":"2024-03-04T09:28:42.971350+0000","last_clean":"2024-03-04T09:28:42.971350+0000","last_became_active":"2024-03-04T09:28:37.905845+0000","last_became_peered":"2024-03-04T09:28:37.905845+0000","last_unstale":"2024-03-04T09:28:42.971350+0000","last_undegraded":"2024-03-04T09:28:42.971350+0000","last_fullsized":"2024-03-04T09:28:42.971350+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_clean_scrub_stamp":"2024-03-04T09:28:36.887656+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T11:50:23.569240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-04T09:28:42.972534+0000","last_change":"2024-03-04T09:28:41.949290+0000","last_active":"2024-03-04T09:28:42.972534+0000","last_peered":"2024-03-04T09:28:42.972534+0000","last_clean":"2024-03-04T09:28:42.972534+0000","last_became_active":"2024-03-04T09:28:41.948866+0000","last_became_peered":"2024-03-04T09:28:41.948866+0000","last_unstale":"2024-03-04T09:28:42.972534+0000","last_undegraded":"2024-03-04T09:28:42.972534+0000","last_fullsized":"2024-03-04T09:28:42.972534+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:40.917115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:40.917115+0000","last_clean_scrub_stamp":"2024-03-04T09:28:40.917115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T15:30:11.980139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-04T09:28:42.972143+0000","last_change":"2024-03-04T09:28:36.910071+0000","last_active":"2024-03-04T09:28:42.972143+0000","last_peered":"2024-03-04T09:28:42.972143+0000","last_clean":"2024-03-04T09:28:42.972143+0000","last_became_active":"2024-03-04T09:28:36.908025+0000","last_became_peered":"2024-03-04T09:28:36.908025+0000","last_unstale":"2024-03-04T09:28:42.972143+0000","last_undegraded":"2024-03-04T09:28:42.972143+0000","last_fullsized":"2024-03-04T09:28:42.972143+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:35.885776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:35.885776+0000","last_clean_scrub_stamp":"2024-03-04T09:28:35.885776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T10:45:42.310473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-04T09:28:42.932089+0000","last_change":"2024-03-04T09:28:37.915792+0000","last_active":"2024-03-04T09:28:42.932089+0000","last_peered":"2024-03-04T09:28:42.932089+0000","last_clean":"2024-03-04T09:28:42.932089+0000","last_became_active":"2024-03-04T09:28:37.914914+0000","last_became_peered":"2024-03-04T09:28:37.914914+0000","last_unstale":"2024-03-04T09:28:42.932089+0000","last_undegraded":"2024-03-04T09:28:42.932089+0000","last_fullsized":"2024-03-04T09:28:42.932089+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_clean_scrub_stamp":"2024-03-04T09:28:36.887656+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T14:13:57.828616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-04T09:28:42.971714+0000","last_change":"2024-03-04T09:28:41.947909+0000","last_active":"2024-03-04T09:28:42.971714+0000","last_peered":"2024-03-04T09:28:42.971714+0000","last_clean":"2024-03-04T09:28:42.971714+0000","last_became_active":"2024-03-04T09:28:41.947553+0000","last_became_peered":"2024-03-04T09:28:41.947553+0000","last_unstale":"2024-03-04T09:28:42.971714+0000","last_undegraded":"2024-03-04T09:28:42.971714+0000","last_fullsized":"2024-03-04T09:28:42.971714+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:40.917115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:40.917115+0000","last_clean_scrub_stamp":"2024-03-04T09:28:40.917115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T14:19:52.302321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"89'3","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-04T09:29:04.009368+0000","last_change":"2024-03-04T09:28:36.911823+0000","last_active":"2024-03-04T09:29:04.009368+0000","last_peered":"2024-03-04T09:29:04.009368+0000","last_clean":"2024-03-04T09:29:04.009368+0000","last_became_active":"2024-03-04T09:28:36.911502+0000","last_became_peered":"2024-03-04T09:28:36.911502+0000","last_unstale":"2024-03-04T09:29:04.009368+0000","last_undegraded":"2024-03-04T09:29:04.009368+0000","last_fullsized":"2024-03-04T09:29:04.009368+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:35.885776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:35.885776+0000","last_clean_scrub_stamp":"2024-03-04T09:28:35.885776+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T15:31:04.806604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-04T09:28:42.971241+0000","last_change":"2024-03-04T09:28:37.914093+0000","last_active":"2024-03-04T09:28:42.971241+0000","last_peered":"2024-03-04T09:28:42.971241+0000","last_clean":"2024-03-04T09:28:42.971241+0000","last_became_active":"2024-03-04T09:28:37.913661+0000","last_became_peered":"2024-03-04T09:28:37.913661+0000","last_unstale":"2024-03-04T09:28:42.971241+0000","last_undegraded":"2024-03-04T09:28:42.971241+0000","last_fullsized":"2024-03-04T09:28:42.971241+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_clean_scrub_stamp":"2024-03-04T09:28:36.887656+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T18:17:42.826342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-04T09:28:42.932023+0000","last_change":"2024-03-04T09:28:41.947371+0000","last_active":"2024-03-04T09:28:42.932023+0000","last_peered":"2024-03-04T09:28:42.932023+0000","last_clean":"2024-03-04T09:28:42.932023+0000","last_became_active":"2024-03-04T09:28:41.946944+0000","last_became_peered":"2024-03-04T09:28:41.946944+0000","last_unstale":"2024-03-04T09:28:42.932023+0000","last_undegraded":"2024-03-04T09:28:42.932023+0000","last_fullsized":"2024-03-04T09:28:42.932023+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:40.917115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:40.917115+0000","last_clean_scrub_stamp":"2024-03-04T09:28:40.917115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T11:01:34.348927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-04T09:28:42.934060+0000","last_change":"2024-03-04T09:28:36.907927+0000","last_active":"2024-03-04T09:28:42.934060+0000","last_peered":"2024-03-04T09:28:42.934060+0000","last_clean":"2024-03-04T09:28:42.934060+0000","last_became_active":"2024-03-04T09:28:36.907507+0000","last_became_peered":"2024-03-04T09:28:36.907507+0000","last_unstale":"2024-03-04T09:28:42.934060+0000","last_undegraded":"2024-03-04T09:28:42.934060+0000","last_fullsized":"2024-03-04T09:28:42.934060+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:35.885776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:35.885776+0000","last_clean_scrub_stamp":"2024-03-04T09:28:35.885776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T21:11:24.942564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-04T09:28:42.972674+0000","last_change":"2024-03-04T09:28:37.913816+0000","last_active":"2024-03-04T09:28:42.972674+0000","last_peered":"2024-03-04T09:28:42.972674+0000","last_clean":"2024-03-04T09:28:42.972674+0000","last_became_active":"2024-03-04T09:28:37.913481+0000","last_became_peered":"2024-03-04T09:28:37.913481+0000","last_unstale":"2024-03-04T09:28:42.972674+0000","last_undegraded":"2024-03-04T09:28:42.972674+0000","last_fullsized":"2024-03-04T09:28:42.972674+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_clean_scrub_stamp":"2024-03-04T09:28:36.887656+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T14:33:07.371224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-04T09:28:42.972234+0000","last_change":"2024-03-04T09:28:41.948963+0000","last_active":"2024-03-04T09:28:42.972234+0000","last_peered":"2024-03-04T09:28:42.972234+0000","last_clean":"2024-03-04T09:28:42.972234+0000","last_became_active":"2024-03-04T09:28:41.948469+0000","last_became_peered":"2024-03-04T09:28:41.948469+0000","last_unstale":"2024-03-04T09:28:42.972234+0000","last_undegraded":"2024-03-04T09:28:42.972234+0000","last_fullsized":"2024-03-04T09:28:42.972234+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:40.917115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:40.917115+0000","last_clean_scrub_stamp":"2024-03-04T09:28:40.917115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T21:26:52.238364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"84'3","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-04T09:28:42.973197+0000","last_change":"2024-03-04T09:28:36.913162+0000","last_active":"2024-03-04T09:28:42.973197+0000","last_peered":"2024-03-04T09:28:42.973197+0000","last_clean":"2024-03-04T09:28:42.973197+0000","last_became_active":"2024-03-04T09:28:36.912834+0000","last_became_peered":"2024-03-04T09:28:36.912834+0000","last_unstale":"2024-03-04T09:28:42.973197+0000","last_undegraded":"2024-03-04T09:28:42.973197+0000","last_fullsized":"2024-03-04T09:28:42.973197+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:35.885776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:35.885776+0000","last_clean_scrub_stamp":"2024-03-04T09:28:35.885776+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T17:59:07.210048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-04T09:28:42.971265+0000","last_change":"2024-03-04T09:28:37.914801+0000","last_active":"2024-03-04T09:28:42.971265+0000","last_peered":"2024-03-04T09:28:42.971265+0000","last_clean":"2024-03-04T09:28:42.971265+0000","last_became_active":"2024-03-04T09:28:37.914517+0000","last_became_peered":"2024-03-04T09:28:37.914517+0000","last_unstale":"2024-03-04T09:28:42.971265+0000","last_undegraded":"2024-03-04T09:28:42.971265+0000","last_fullsized":"2024-03-04T09:28:42.971265+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_clean_scrub_stamp":"2024-03-04T09:28:36.887656+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T18:15:50.967364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-04T09:28:42.932300+0000","last_change":"2024-03-04T09:28:41.957821+0000","last_active":"2024-03-04T09:28:42.932300+0000","last_peered":"2024-03-04T09:28:42.932300+0000","last_clean":"2024-03-04T09:28:42.932300+0000","last_became_active":"2024-03-04T09:28:41.957392+0000","last_became_peered":"2024-03-04T09:28:41.957392+0000","last_unstale":"2024-03-04T09:28:42.932300+0000","last_undegraded":"2024-03-04T09:28:42.932300+0000","last_fullsized":"2024-03-04T09:28:42.932300+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:40.917115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:40.917115+0000","last_clean_scrub_stamp":"2024-03-04T09:28:40.917115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T19:44:24.294299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"89'19","reported_seq":49,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-04T09:29:05.012668+0000","last_change":"2024-03-04T09:28:36.909243+0000","last_active":"2024-03-04T09:29:05.012668+0000","last_peered":"2024-03-04T09:29:05.012668+0000","last_clean":"2024-03-04T09:29:05.012668+0000","last_became_active":"2024-03-04T09:28:36.908941+0000","last_became_peered":"2024-03-04T09:28:36.908941+0000","last_unstale":"2024-03-04T09:29:05.012668+0000","last_undegraded":"2024-03-04T09:29:05.012668+0000","last_fullsized":"2024-03-04T09:29:05.012668+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:35.885776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:35.885776+0000","last_clean_scrub_stamp":"2024-03-04T09:28:35.885776+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T16:46:26.014209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23141,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":7,"num_read_kb":26,"num_write":20,"num_write_kb":30,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-04T09:28:42.971950+0000","last_change":"2024-03-04T09:28:37.910465+0000","last_active":"2024-03-04T09:28:42.971950+0000","last_peered":"2024-03-04T09:28:42.971950+0000","last_clean":"2024-03-04T09:28:42.971950+0000","last_became_active":"2024-03-04T09:28:37.910129+0000","last_became_peered":"2024-03-04T09:28:37.910129+0000","last_unstale":"2024-03-04T09:28:42.971950+0000","last_undegraded":"2024-03-04T09:28:42.971950+0000","last_fullsized":"2024-03-04T09:28:42.971950+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_clean_scrub_stamp":"2024-03-04T09:28:36.887656+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T21:25:48.272477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-04T09:28:42.973636+0000","last_change":"2024-03-04T09:28:41.948465+0000","last_active":"2024-03-04T09:28:42.973636+0000","last_peered":"2024-03-04T09:28:42.973636+0000","last_clean":"2024-03-04T09:28:42.973636+0000","last_became_active":"2024-03-04T09:28:41.946995+0000","last_became_peered":"2024-03-04T09:28:41.946995+0000","last_unstale":"2024-03-04T09:28:42.973636+0000","last_undegraded":"2024-03-04T09:28:42.973636+0000","last_fullsized":"2024-03-04T09:28:42.973636+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:40.917115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:40.917115+0000","last_clean_scrub_stamp":"2024-03-04T09:28:40.917115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T12:35:10.579279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"84'2","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-04T09:28:42.932539+0000","last_change":"2024-03-04T09:28:36.905641+0000","last_active":"2024-03-04T09:28:42.932539+0000","last_peered":"2024-03-04T09:28:42.932539+0000","last_clean":"2024-03-04T09:28:42.932539+0000","last_became_active":"2024-03-04T09:28:36.905309+0000","last_became_peered":"2024-03-04T09:28:36.905309+0000","last_unstale":"2024-03-04T09:28:42.932539+0000","last_undegraded":"2024-03-04T09:28:42.932539+0000","last_fullsized":"2024-03-04T09:28:42.932539+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:35.885776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:35.885776+0000","last_clean_scrub_stamp":"2024-03-04T09:28:35.885776+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T12:08:40.200654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-04T09:28:42.973519+0000","last_change":"2024-03-04T09:28:37.907744+0000","last_active":"2024-03-04T09:28:42.973519+0000","last_peered":"2024-03-04T09:28:42.973519+0000","last_clean":"2024-03-04T09:28:42.973519+0000","last_became_active":"2024-03-04T09:28:37.907393+0000","last_became_peered":"2024-03-04T09:28:37.907393+0000","last_unstale":"2024-03-04T09:28:42.973519+0000","last_undegraded":"2024-03-04T09:28:42.973519+0000","last_fullsized":"2024-03-04T09:28:42.973519+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_clean_scrub_stamp":"2024-03-04T09:28:36.887656+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T20:17:11.822846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-04T09:28:42.971775+0000","last_change":"2024-03-04T09:28:41.949824+0000","last_active":"2024-03-04T09:28:42.971775+0000","last_peered":"2024-03-04T09:28:42.971775+0000","last_clean":"2024-03-04T09:28:42.971775+0000","last_became_active":"2024-03-04T09:28:41.949252+0000","last_became_peered":"2024-03-04T09:28:41.949252+0000","last_unstale":"2024-03-04T09:28:42.971775+0000","last_undegraded":"2024-03-04T09:28:42.971775+0000","last_fullsized":"2024-03-04T09:28:42.971775+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:40.917115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:40.917115+0000","last_clean_scrub_stamp":"2024-03-04T09:28:40.917115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T21:22:19.045780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-04T09:28:42.973178+0000","last_change":"2024-03-04T09:28:36.908718+0000","last_active":"2024-03-04T09:28:42.973178+0000","last_peered":"2024-03-04T09:28:42.973178+0000","last_clean":"2024-03-04T09:28:42.973178+0000","last_became_active":"2024-03-04T09:28:36.908102+0000","last_became_peered":"2024-03-04T09:28:36.908102+0000","last_unstale":"2024-03-04T09:28:42.973178+0000","last_undegraded":"2024-03-04T09:28:42.973178+0000","last_fullsized":"2024-03-04T09:28:42.973178+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:35.885776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:35.885776+0000","last_clean_scrub_stamp":"2024-03-04T09:28:35.885776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T20:53:39.603281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-04T09:28:42.933919+0000","last_change":"2024-03-04T09:28:37.912741+0000","last_active":"2024-03-04T09:28:42.933919+0000","last_peered":"2024-03-04T09:28:42.933919+0000","last_clean":"2024-03-04T09:28:42.933919+0000","last_became_active":"2024-03-04T09:28:37.912430+0000","last_became_peered":"2024-03-04T09:28:37.912430+0000","last_unstale":"2024-03-04T09:28:42.933919+0000","last_undegraded":"2024-03-04T09:28:42.933919+0000","last_fullsized":"2024-03-04T09:28:42.933919+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_clean_scrub_stamp":"2024-03-04T09:28:36.887656+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T20:33:52.234238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-04T09:28:42.932697+0000","last_change":"2024-03-04T09:28:41.947303+0000","last_active":"2024-03-04T09:28:42.932697+0000","last_peered":"2024-03-04T09:28:42.932697+0000","last_clean":"2024-03-04T09:28:42.932697+0000","last_became_active":"2024-03-04T09:28:41.946838+0000","last_became_peered":"2024-03-04T09:28:41.946838+0000","last_unstale":"2024-03-04T09:28:42.932697+0000","last_undegraded":"2024-03-04T09:28:42.932697+0000","last_fullsized":"2024-03-04T09:28:42.932697+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:40.917115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:40.917115+0000","last_clean_scrub_stamp":"2024-03-04T09:28:40.917115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T20:08:29.575678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-04T09:28:42.971918+0000","last_change":"2024-03-04T09:28:36.905097+0000","last_active":"2024-03-04T09:28:42.971918+0000","last_peered":"2024-03-04T09:28:42.971918+0000","last_clean":"2024-03-04T09:28:42.971918+0000","last_became_active":"2024-03-04T09:28:36.904709+0000","last_became_peered":"2024-03-04T09:28:36.904709+0000","last_unstale":"2024-03-04T09:28:42.971918+0000","last_undegraded":"2024-03-04T09:28:42.971918+0000","last_fullsized":"2024-03-04T09:28:42.971918+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:35.885776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:35.885776+0000","last_clean_scrub_stamp":"2024-03-04T09:28:35.885776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T13:58:23.977258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-04T09:28:42.933763+0000","last_change":"2024-03-04T09:28:37.918023+0000","last_active":"2024-03-04T09:28:42.933763+0000","last_peered":"2024-03-04T09:28:42.933763+0000","last_clean":"2024-03-04T09:28:42.933763+0000","last_became_active":"2024-03-04T09:28:37.917671+0000","last_became_peered":"2024-03-04T09:28:37.917671+0000","last_unstale":"2024-03-04T09:28:42.933763+0000","last_undegraded":"2024-03-04T09:28:42.933763+0000","last_fullsized":"2024-03-04T09:28:42.933763+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_clean_scrub_stamp":"2024-03-04T09:28:36.887656+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T09:51:15.618328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-04T09:28:42.974003+0000","last_change":"2024-03-04T09:28:41.958903+0000","last_active":"2024-03-04T09:28:42.974003+0000","last_peered":"2024-03-04T09:28:42.974003+0000","last_clean":"2024-03-04T09:28:42.974003+0000","last_became_active":"2024-03-04T09:28:41.958256+0000","last_became_peered":"2024-03-04T09:28:41.958256+0000","last_unstale":"2024-03-04T09:28:42.974003+0000","last_undegraded":"2024-03-04T09:28:42.974003+0000","last_fullsized":"2024-03-04T09:28:42.974003+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:40.917115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:40.917115+0000","last_clean_scrub_stamp":"2024-03-04T09:28:40.917115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T18:49:48.022650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"84'8","reported_seq":31,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-04T09:28:42.971902+0000","last_change":"2024-03-04T09:28:36.911924+0000","last_active":"2024-03-04T09:28:42.971902+0000","last_peered":"2024-03-04T09:28:42.971902+0000","last_clean":"2024-03-04T09:28:42.971902+0000","last_became_active":"2024-03-04T09:28:36.911623+0000","last_became_peered":"2024-03-04T09:28:36.911623+0000","last_unstale":"2024-03-04T09:28:42.971902+0000","last_undegraded":"2024-03-04T09:28:42.971902+0000","last_fullsized":"2024-03-04T09:28:42.971902+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:35.885776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:35.885776+0000","last_clean_scrub_stamp":"2024-03-04T09:28:35.885776+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T13:17:32.606767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-04T09:28:42.972962+0000","last_change":"2024-03-04T09:28:37.916578+0000","last_active":"2024-03-04T09:28:42.972962+0000","last_peered":"2024-03-04T09:28:42.972962+0000","last_clean":"2024-03-04T09:28:42.972962+0000","last_became_active":"2024-03-04T09:28:37.913830+0000","last_became_peered":"2024-03-04T09:28:37.913830+0000","last_unstale":"2024-03-04T09:28:42.972962+0000","last_undegraded":"2024-03-04T09:28:42.972962+0000","last_fullsized":"2024-03-04T09:28:42.972962+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_clean_scrub_stamp":"2024-03-04T09:28:36.887656+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T09:29:15.882496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-04T09:28:42.972986+0000","last_change":"2024-03-04T09:28:41.951674+0000","last_active":"2024-03-04T09:28:42.972986+0000","last_peered":"2024-03-04T09:28:42.972986+0000","last_clean":"2024-03-04T09:28:42.972986+0000","last_became_active":"2024-03-04T09:28:41.951175+0000","last_became_peered":"2024-03-04T09:28:41.951175+0000","last_unstale":"2024-03-04T09:28:42.972986+0000","last_undegraded":"2024-03-04T09:28:42.972986+0000","last_fullsized":"2024-03-04T09:28:42.972986+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:40.917115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:40.917115+0000","last_clean_scrub_stamp":"2024-03-04T09:28:40.917115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T16:32:00.482396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"84'1","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-04T09:28:42.972464+0000","last_change":"2024-03-04T09:28:36.909779+0000","last_active":"2024-03-04T09:28:42.972464+0000","last_peered":"2024-03-04T09:28:42.972464+0000","last_clean":"2024-03-04T09:28:42.972464+0000","last_became_active":"2024-03-04T09:28:36.909466+0000","last_became_peered":"2024-03-04T09:28:36.909466+0000","last_unstale":"2024-03-04T09:28:42.972464+0000","last_undegraded":"2024-03-04T09:28:42.972464+0000","last_fullsized":"2024-03-04T09:28:42.972464+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:35.885776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:35.885776+0000","last_clean_scrub_stamp":"2024-03-04T09:28:35.885776+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T17:16:28.183292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-04T09:28:42.972320+0000","last_change":"2024-03-04T09:28:37.916371+0000","last_active":"2024-03-04T09:28:42.972320+0000","last_peered":"2024-03-04T09:28:42.972320+0000","last_clean":"2024-03-04T09:28:42.972320+0000","last_became_active":"2024-03-04T09:28:37.916068+0000","last_became_peered":"2024-03-04T09:28:37.916068+0000","last_unstale":"2024-03-04T09:28:42.972320+0000","last_undegraded":"2024-03-04T09:28:42.972320+0000","last_fullsized":"2024-03-04T09:28:42.972320+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_clean_scrub_stamp":"2024-03-04T09:28:36.887656+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T13:18:18.850165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-04T09:28:42.972568+0000","last_change":"2024-03-04T09:28:41.955301+0000","last_active":"2024-03-04T09:28:42.972568+0000","last_peered":"2024-03-04T09:28:42.972568+0000","last_clean":"2024-03-04T09:28:42.972568+0000","last_became_active":"2024-03-04T09:28:41.954774+0000","last_became_peered":"2024-03-04T09:28:41.954774+0000","last_unstale":"2024-03-04T09:28:42.972568+0000","last_undegraded":"2024-03-04T09:28:42.972568+0000","last_fullsized":"2024-03-04T09:28:42.972568+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:40.917115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:40.917115+0000","last_clean_scrub_stamp":"2024-03-04T09:28:40.917115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T16:56:55.463344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-04T09:28:42.932424+0000","last_change":"2024-03-04T09:28:36.901659+0000","last_active":"2024-03-04T09:28:42.932424+0000","last_peered":"2024-03-04T09:28:42.932424+0000","last_clean":"2024-03-04T09:28:42.932424+0000","last_became_active":"2024-03-04T09:28:36.901235+0000","last_became_peered":"2024-03-04T09:28:36.901235+0000","last_unstale":"2024-03-04T09:28:42.932424+0000","last_undegraded":"2024-03-04T09:28:42.932424+0000","last_fullsized":"2024-03-04T09:28:42.932424+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:35.885776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:35.885776+0000","last_clean_scrub_stamp":"2024-03-04T09:28:35.885776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T12:59:50.237172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-04T09:28:42.972456+0000","last_change":"2024-03-04T09:28:37.907905+0000","last_active":"2024-03-04T09:28:42.972456+0000","last_peered":"2024-03-04T09:28:42.972456+0000","last_clean":"2024-03-04T09:28:42.972456+0000","last_became_active":"2024-03-04T09:28:37.907376+0000","last_became_peered":"2024-03-04T09:28:37.907376+0000","last_unstale":"2024-03-04T09:28:42.972456+0000","last_undegraded":"2024-03-04T09:28:42.972456+0000","last_fullsized":"2024-03-04T09:28:42.972456+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_clean_scrub_stamp":"2024-03-04T09:28:36.887656+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T14:25:31.784659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-04T09:28:42.973802+0000","last_change":"2024-03-04T09:28:41.947629+0000","last_active":"2024-03-04T09:28:42.973802+0000","last_peered":"2024-03-04T09:28:42.973802+0000","last_clean":"2024-03-04T09:28:42.973802+0000","last_became_active":"2024-03-04T09:28:41.946251+0000","last_became_peered":"2024-03-04T09:28:41.946251+0000","last_unstale":"2024-03-04T09:28:42.973802+0000","last_undegraded":"2024-03-04T09:28:42.973802+0000","last_fullsized":"2024-03-04T09:28:42.973802+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:40.917115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:40.917115+0000","last_clean_scrub_stamp":"2024-03-04T09:28:40.917115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T18:26:57.770657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-04T09:28:42.933780+0000","last_change":"2024-03-04T09:28:36.905323+0000","last_active":"2024-03-04T09:28:42.933780+0000","last_peered":"2024-03-04T09:28:42.933780+0000","last_clean":"2024-03-04T09:28:42.933780+0000","last_became_active":"2024-03-04T09:28:36.904517+0000","last_became_peered":"2024-03-04T09:28:36.904517+0000","last_unstale":"2024-03-04T09:28:42.933780+0000","last_undegraded":"2024-03-04T09:28:42.933780+0000","last_fullsized":"2024-03-04T09:28:42.933780+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:35.885776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:35.885776+0000","last_clean_scrub_stamp":"2024-03-04T09:28:35.885776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T18:52:11.615107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-04T09:28:42.971819+0000","last_change":"2024-03-04T09:28:37.921000+0000","last_active":"2024-03-04T09:28:42.971819+0000","last_peered":"2024-03-04T09:28:42.971819+0000","last_clean":"2024-03-04T09:28:42.971819+0000","last_became_active":"2024-03-04T09:28:37.920620+0000","last_became_peered":"2024-03-04T09:28:37.920620+0000","last_unstale":"2024-03-04T09:28:42.971819+0000","last_undegraded":"2024-03-04T09:28:42.971819+0000","last_fullsized":"2024-03-04T09:28:42.971819+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_clean_scrub_stamp":"2024-03-04T09:28:36.887656+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T10:41:29.187701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-04T09:28:42.973811+0000","last_change":"2024-03-04T09:28:41.949132+0000","last_active":"2024-03-04T09:28:42.973811+0000","last_peered":"2024-03-04T09:28:42.973811+0000","last_clean":"2024-03-04T09:28:42.973811+0000","last_became_active":"2024-03-04T09:28:41.948618+0000","last_became_peered":"2024-03-04T09:28:41.948618+0000","last_unstale":"2024-03-04T09:28:42.973811+0000","last_undegraded":"2024-03-04T09:28:42.973811+0000","last_fullsized":"2024-03-04T09:28:42.973811+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:40.917115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:40.917115+0000","last_clean_scrub_stamp":"2024-03-04T09:28:40.917115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T19:45:45.001345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-04T09:28:42.972419+0000","last_change":"2024-03-04T09:28:37.910475+0000","last_active":"2024-03-04T09:28:42.972419+0000","last_peered":"2024-03-04T09:28:42.972419+0000","last_clean":"2024-03-04T09:28:42.972419+0000","last_became_active":"2024-03-04T09:28:37.910184+0000","last_became_peered":"2024-03-04T09:28:37.910184+0000","last_unstale":"2024-03-04T09:28:42.972419+0000","last_undegraded":"2024-03-04T09:28:42.972419+0000","last_fullsized":"2024-03-04T09:28:42.972419+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_clean_scrub_stamp":"2024-03-04T09:28:36.887656+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T10:16:56.244331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-04T09:28:42.973116+0000","last_change":"2024-03-04T09:28:36.913489+0000","last_active":"2024-03-04T09:28:42.973116+0000","last_peered":"2024-03-04T09:28:42.973116+0000","last_clean":"2024-03-04T09:28:42.973116+0000","last_became_active":"2024-03-04T09:28:36.913162+0000","last_became_peered":"2024-03-04T09:28:36.913162+0000","last_unstale":"2024-03-04T09:28:42.973116+0000","last_undegraded":"2024-03-04T09:28:42.973116+0000","last_fullsized":"2024-03-04T09:28:42.973116+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:35.885776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:35.885776+0000","last_clean_scrub_stamp":"2024-03-04T09:28:35.885776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T12:20:35.870058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-04T09:28:42.934143+0000","last_change":"2024-03-04T09:28:41.945717+0000","last_active":"2024-03-04T09:28:42.934143+0000","last_peered":"2024-03-04T09:28:42.934143+0000","last_clean":"2024-03-04T09:28:42.934143+0000","last_became_active":"2024-03-04T09:28:41.945300+0000","last_became_peered":"2024-03-04T09:28:41.945300+0000","last_unstale":"2024-03-04T09:28:42.934143+0000","last_undegraded":"2024-03-04T09:28:42.934143+0000","last_fullsized":"2024-03-04T09:28:42.934143+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:40.917115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:40.917115+0000","last_clean_scrub_stamp":"2024-03-04T09:28:40.917115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T11:05:30.407232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-04T09:28:42.971603+0000","last_change":"2024-03-04T09:28:37.921030+0000","last_active":"2024-03-04T09:28:42.971603+0000","last_peered":"2024-03-04T09:28:42.971603+0000","last_clean":"2024-03-04T09:28:42.971603+0000","last_became_active":"2024-03-04T09:28:37.920655+0000","last_became_peered":"2024-03-04T09:28:37.920655+0000","last_unstale":"2024-03-04T09:28:42.971603+0000","last_undegraded":"2024-03-04T09:28:42.971603+0000","last_fullsized":"2024-03-04T09:28:42.971603+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_clean_scrub_stamp":"2024-03-04T09:28:36.887656+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T19:37:34.063523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-04T09:28:42.972144+0000","last_change":"2024-03-04T09:28:36.902902+0000","last_active":"2024-03-04T09:28:42.972144+0000","last_peered":"2024-03-04T09:28:42.972144+0000","last_clean":"2024-03-04T09:28:42.972144+0000","last_became_active":"2024-03-04T09:28:36.902574+0000","last_became_peered":"2024-03-04T09:28:36.902574+0000","last_unstale":"2024-03-04T09:28:42.972144+0000","last_undegraded":"2024-03-04T09:28:42.972144+0000","last_fullsized":"2024-03-04T09:28:42.972144+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:35.885776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:35.885776+0000","last_clean_scrub_stamp":"2024-03-04T09:28:35.885776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T14:11:17.544497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":357,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-04T09:28:42.933649+0000","last_change":"2024-03-04T09:27:28.478476+0000","last_active":"2024-03-04T09:28:42.933649+0000","last_peered":"2024-03-04T09:28:42.933649+0000","last_clean":"2024-03-04T09:28:42.933649+0000","last_became_active":"2024-03-04T09:27:28.478041+0000","last_became_peered":"2024-03-04T09:27:28.478041+0000","last_unstale":"2024-03-04T09:28:42.933649+0000","last_undegraded":"2024-03-04T09:28:42.933649+0000","last_fullsized":"2024-03-04T09:28:42.933649+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:24:01.742551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:24:01.742551+0000","last_clean_scrub_stamp":"2024-03-04T09:24:01.742551+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T12:51:11.585296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-04T09:28:42.973248+0000","last_change":"2024-03-04T09:28:41.958895+0000","last_active":"2024-03-04T09:28:42.973248+0000","last_peered":"2024-03-04T09:28:42.973248+0000","last_clean":"2024-03-04T09:28:42.973248+0000","last_became_active":"2024-03-04T09:28:41.958477+0000","last_became_peered":"2024-03-04T09:28:41.958477+0000","last_unstale":"2024-03-04T09:28:42.973248+0000","last_undegraded":"2024-03-04T09:28:42.973248+0000","last_fullsized":"2024-03-04T09:28:42.973248+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:40.917115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:40.917115+0000","last_clean_scrub_stamp":"2024-03-04T09:28:40.917115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T17:48:37.149539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-04T09:28:42.973063+0000","last_change":"2024-03-04T09:28:37.911872+0000","last_active":"2024-03-04T09:28:42.973063+0000","last_peered":"2024-03-04T09:28:42.973063+0000","last_clean":"2024-03-04T09:28:42.973063+0000","last_became_active":"2024-03-04T09:28:37.911583+0000","last_became_peered":"2024-03-04T09:28:37.911583+0000","last_unstale":"2024-03-04T09:28:42.973063+0000","last_undegraded":"2024-03-04T09:28:42.973063+0000","last_fullsized":"2024-03-04T09:28:42.973063+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_clean_scrub_stamp":"2024-03-04T09:28:36.887656+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T20:12:15.606046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-04T09:28:42.933658+0000","last_change":"2024-03-04T09:28:36.905156+0000","last_active":"2024-03-04T09:28:42.933658+0000","last_peered":"2024-03-04T09:28:42.933658+0000","last_clean":"2024-03-04T09:28:42.933658+0000","last_became_active":"2024-03-04T09:28:36.904702+0000","last_became_peered":"2024-03-04T09:28:36.904702+0000","last_unstale":"2024-03-04T09:28:42.933658+0000","last_undegraded":"2024-03-04T09:28:42.933658+0000","last_fullsized":"2024-03-04T09:28:42.933658+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:35.885776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:35.885776+0000","last_clean_scrub_stamp":"2024-03-04T09:28:35.885776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T19:38:16.485245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-04T09:28:42.971722+0000","last_change":"2024-03-04T09:28:41.954087+0000","last_active":"2024-03-04T09:28:42.971722+0000","last_peered":"2024-03-04T09:28:42.971722+0000","last_clean":"2024-03-04T09:28:42.971722+0000","last_became_active":"2024-03-04T09:28:41.953681+0000","last_became_peered":"2024-03-04T09:28:41.953681+0000","last_unstale":"2024-03-04T09:28:42.971722+0000","last_undegraded":"2024-03-04T09:28:42.971722+0000","last_fullsized":"2024-03-04T09:28:42.971722+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:40.917115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:40.917115+0000","last_clean_scrub_stamp":"2024-03-04T09:28:40.917115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T19:45:18.525709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-04T09:28:42.972047+0000","last_change":"2024-03-04T09:28:36.912400+0000","last_active":"2024-03-04T09:28:42.972047+0000","last_peered":"2024-03-04T09:28:42.972047+0000","last_clean":"2024-03-04T09:28:42.972047+0000","last_became_active":"2024-03-04T09:28:36.912096+0000","last_became_peered":"2024-03-04T09:28:36.912096+0000","last_unstale":"2024-03-04T09:28:42.972047+0000","last_undegraded":"2024-03-04T09:28:42.972047+0000","last_fullsized":"2024-03-04T09:28:42.972047+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:35.885776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:35.885776+0000","last_clean_scrub_stamp":"2024-03-04T09:28:35.885776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T16:45:54.750427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-04T09:28:42.973855+0000","last_change":"2024-03-04T09:28:37.902851+0000","last_active":"2024-03-04T09:28:42.973855+0000","last_peered":"2024-03-04T09:28:42.973855+0000","last_clean":"2024-03-04T09:28:42.973855+0000","last_became_active":"2024-03-04T09:28:37.902537+0000","last_became_peered":"2024-03-04T09:28:37.902537+0000","last_unstale":"2024-03-04T09:28:42.973855+0000","last_undegraded":"2024-03-04T09:28:42.973855+0000","last_fullsized":"2024-03-04T09:28:42.973855+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_clean_scrub_stamp":"2024-03-04T09:28:36.887656+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T14:30:50.350161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-04T09:28:42.971436+0000","last_change":"2024-03-04T09:28:41.948551+0000","last_active":"2024-03-04T09:28:42.971436+0000","last_peered":"2024-03-04T09:28:42.971436+0000","last_clean":"2024-03-04T09:28:42.971436+0000","last_became_active":"2024-03-04T09:28:41.946636+0000","last_became_peered":"2024-03-04T09:28:41.946636+0000","last_unstale":"2024-03-04T09:28:42.971436+0000","last_undegraded":"2024-03-04T09:28:42.971436+0000","last_fullsized":"2024-03-04T09:28:42.971436+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:40.917115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:40.917115+0000","last_clean_scrub_stamp":"2024-03-04T09:28:40.917115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T14:00:07.624296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"89'3","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-04T09:29:03.002306+0000","last_change":"2024-03-04T09:28:36.910605+0000","last_active":"2024-03-04T09:29:03.002306+0000","last_peered":"2024-03-04T09:29:03.002306+0000","last_clean":"2024-03-04T09:29:03.002306+0000","last_became_active":"2024-03-04T09:28:36.910253+0000","last_became_peered":"2024-03-04T09:28:36.910253+0000","last_unstale":"2024-03-04T09:29:03.002306+0000","last_undegraded":"2024-03-04T09:29:03.002306+0000","last_fullsized":"2024-03-04T09:29:03.002306+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:35.885776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:35.885776+0000","last_clean_scrub_stamp":"2024-03-04T09:28:35.885776+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T18:08:30.941310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-04T09:28:42.972134+0000","last_change":"2024-03-04T09:28:37.907342+0000","last_active":"2024-03-04T09:28:42.972134+0000","last_peered":"2024-03-04T09:28:42.972134+0000","last_clean":"2024-03-04T09:28:42.972134+0000","last_became_active":"2024-03-04T09:28:37.906974+0000","last_became_peered":"2024-03-04T09:28:37.906974+0000","last_unstale":"2024-03-04T09:28:42.972134+0000","last_undegraded":"2024-03-04T09:28:42.972134+0000","last_fullsized":"2024-03-04T09:28:42.972134+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_clean_scrub_stamp":"2024-03-04T09:28:36.887656+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T11:18:43.099544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-04T09:28:42.932170+0000","last_change":"2024-03-04T09:28:41.945713+0000","last_active":"2024-03-04T09:28:42.932170+0000","last_peered":"2024-03-04T09:28:42.932170+0000","last_clean":"2024-03-04T09:28:42.932170+0000","last_became_active":"2024-03-04T09:28:41.945245+0000","last_became_peered":"2024-03-04T09:28:41.945245+0000","last_unstale":"2024-03-04T09:28:42.932170+0000","last_undegraded":"2024-03-04T09:28:42.932170+0000","last_fullsized":"2024-03-04T09:28:42.932170+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:40.917115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:40.917115+0000","last_clean_scrub_stamp":"2024-03-04T09:28:40.917115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T11:06:39.595269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"84'2","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-04T09:28:42.972930+0000","last_change":"2024-03-04T09:28:36.902020+0000","last_active":"2024-03-04T09:28:42.972930+0000","last_peered":"2024-03-04T09:28:42.972930+0000","last_clean":"2024-03-04T09:28:42.972930+0000","last_became_active":"2024-03-04T09:28:36.901666+0000","last_became_peered":"2024-03-04T09:28:36.901666+0000","last_unstale":"2024-03-04T09:28:42.972930+0000","last_undegraded":"2024-03-04T09:28:42.972930+0000","last_fullsized":"2024-03-04T09:28:42.972930+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:35.885776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:35.885776+0000","last_clean_scrub_stamp":"2024-03-04T09:28:35.885776+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T12:48:17.270200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-04T09:28:42.972604+0000","last_change":"2024-03-04T09:28:37.912889+0000","last_active":"2024-03-04T09:28:42.972604+0000","last_peered":"2024-03-04T09:28:42.972604+0000","last_clean":"2024-03-04T09:28:42.972604+0000","last_became_active":"2024-03-04T09:28:37.912542+0000","last_became_peered":"2024-03-04T09:28:37.912542+0000","last_unstale":"2024-03-04T09:28:42.972604+0000","last_undegraded":"2024-03-04T09:28:42.972604+0000","last_fullsized":"2024-03-04T09:28:42.972604+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_clean_scrub_stamp":"2024-03-04T09:28:36.887656+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T20:51:12.540364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-04T09:28:42.972605+0000","last_change":"2024-03-04T09:28:41.949009+0000","last_active":"2024-03-04T09:28:42.972605+0000","last_peered":"2024-03-04T09:28:42.972605+0000","last_clean":"2024-03-04T09:28:42.972605+0000","last_became_active":"2024-03-04T09:28:41.948178+0000","last_became_peered":"2024-03-04T09:28:41.948178+0000","last_unstale":"2024-03-04T09:28:42.972605+0000","last_undegraded":"2024-03-04T09:28:42.972605+0000","last_fullsized":"2024-03-04T09:28:42.972605+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:40.917115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:40.917115+0000","last_clean_scrub_stamp":"2024-03-04T09:28:40.917115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T10:54:54.360229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"89'2","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-04T09:29:01.998640+0000","last_change":"2024-03-04T09:28:36.901847+0000","last_active":"2024-03-04T09:29:01.998640+0000","last_peered":"2024-03-04T09:29:01.998640+0000","last_clean":"2024-03-04T09:29:01.998640+0000","last_became_active":"2024-03-04T09:28:36.901461+0000","last_became_peered":"2024-03-04T09:28:36.901461+0000","last_unstale":"2024-03-04T09:29:01.998640+0000","last_undegraded":"2024-03-04T09:29:01.998640+0000","last_fullsized":"2024-03-04T09:29:01.998640+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:35.885776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:35.885776+0000","last_clean_scrub_stamp":"2024-03-04T09:28:35.885776+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T14:36:33.041447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-04T09:28:42.971633+0000","last_change":"2024-03-04T09:28:37.907958+0000","last_active":"2024-03-04T09:28:42.971633+0000","last_peered":"2024-03-04T09:28:42.971633+0000","last_clean":"2024-03-04T09:28:42.971633+0000","last_became_active":"2024-03-04T09:28:37.907653+0000","last_became_peered":"2024-03-04T09:28:37.907653+0000","last_unstale":"2024-03-04T09:28:42.971633+0000","last_undegraded":"2024-03-04T09:28:42.971633+0000","last_fullsized":"2024-03-04T09:28:42.971633+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_clean_scrub_stamp":"2024-03-04T09:28:36.887656+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T20:42:44.266275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-04T09:28:42.932881+0000","last_change":"2024-03-04T09:28:41.957479+0000","last_active":"2024-03-04T09:28:42.932881+0000","last_peered":"2024-03-04T09:28:42.932881+0000","last_clean":"2024-03-04T09:28:42.932881+0000","last_became_active":"2024-03-04T09:28:41.954950+0000","last_became_peered":"2024-03-04T09:28:41.954950+0000","last_unstale":"2024-03-04T09:28:42.932881+0000","last_undegraded":"2024-03-04T09:28:42.932881+0000","last_fullsized":"2024-03-04T09:28:42.932881+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:40.917115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:40.917115+0000","last_clean_scrub_stamp":"2024-03-04T09:28:40.917115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T14:53:16.636230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"89'3","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-04T09:29:01.998148+0000","last_change":"2024-03-04T09:28:36.908649+0000","last_active":"2024-03-04T09:29:01.998148+0000","last_peered":"2024-03-04T09:29:01.998148+0000","last_clean":"2024-03-04T09:29:01.998148+0000","last_became_active":"2024-03-04T09:28:36.908323+0000","last_became_peered":"2024-03-04T09:28:36.908323+0000","last_unstale":"2024-03-04T09:29:01.998148+0000","last_undegraded":"2024-03-04T09:29:01.998148+0000","last_fullsized":"2024-03-04T09:29:01.998148+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:35.885776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:35.885776+0000","last_clean_scrub_stamp":"2024-03-04T09:28:35.885776+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T16:49:40.367952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-04T09:28:42.971427+0000","last_change":"2024-03-04T09:28:37.905673+0000","last_active":"2024-03-04T09:28:42.971427+0000","last_peered":"2024-03-04T09:28:42.971427+0000","last_clean":"2024-03-04T09:28:42.971427+0000","last_became_active":"2024-03-04T09:28:37.905368+0000","last_became_peered":"2024-03-04T09:28:37.905368+0000","last_unstale":"2024-03-04T09:28:42.971427+0000","last_undegraded":"2024-03-04T09:28:42.971427+0000","last_fullsized":"2024-03-04T09:28:42.971427+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_clean_scrub_stamp":"2024-03-04T09:28:36.887656+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T17:25:35.058092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-04T09:28:42.972120+0000","last_change":"2024-03-04T09:28:41.948688+0000","last_active":"2024-03-04T09:28:42.972120+0000","last_peered":"2024-03-04T09:28:42.972120+0000","last_clean":"2024-03-04T09:28:42.972120+0000","last_became_active":"2024-03-04T09:28:41.948111+0000","last_became_peered":"2024-03-04T09:28:41.948111+0000","last_unstale":"2024-03-04T09:28:42.972120+0000","last_undegraded":"2024-03-04T09:28:42.972120+0000","last_fullsized":"2024-03-04T09:28:42.972120+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:40.917115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:40.917115+0000","last_clean_scrub_stamp":"2024-03-04T09:28:40.917115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T12:27:37.121328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-04T09:28:42.932178+0000","last_change":"2024-03-04T09:28:36.908438+0000","last_active":"2024-03-04T09:28:42.932178+0000","last_peered":"2024-03-04T09:28:42.932178+0000","last_clean":"2024-03-04T09:28:42.932178+0000","last_became_active":"2024-03-04T09:28:36.906529+0000","last_became_peered":"2024-03-04T09:28:36.906529+0000","last_unstale":"2024-03-04T09:28:42.932178+0000","last_undegraded":"2024-03-04T09:28:42.932178+0000","last_fullsized":"2024-03-04T09:28:42.932178+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:35.885776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:35.885776+0000","last_clean_scrub_stamp":"2024-03-04T09:28:35.885776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T10:12:54.833556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-04T09:28:42.971906+0000","last_change":"2024-03-04T09:28:37.927048+0000","last_active":"2024-03-04T09:28:42.971906+0000","last_peered":"2024-03-04T09:28:42.971906+0000","last_clean":"2024-03-04T09:28:42.971906+0000","last_became_active":"2024-03-04T09:28:37.925381+0000","last_became_peered":"2024-03-04T09:28:37.925381+0000","last_unstale":"2024-03-04T09:28:42.971906+0000","last_undegraded":"2024-03-04T09:28:42.971906+0000","last_fullsized":"2024-03-04T09:28:42.971906+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_clean_scrub_stamp":"2024-03-04T09:28:36.887656+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T20:19:48.554102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-04T09:28:42.972670+0000","last_change":"2024-03-04T09:28:41.945805+0000","last_active":"2024-03-04T09:28:42.972670+0000","last_peered":"2024-03-04T09:28:42.972670+0000","last_clean":"2024-03-04T09:28:42.972670+0000","last_became_active":"2024-03-04T09:28:41.945414+0000","last_became_peered":"2024-03-04T09:28:41.945414+0000","last_unstale":"2024-03-04T09:28:42.972670+0000","last_undegraded":"2024-03-04T09:28:42.972670+0000","last_fullsized":"2024-03-04T09:28:42.972670+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:40.917115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:40.917115+0000","last_clean_scrub_stamp":"2024-03-04T09:28:40.917115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T09:45:59.612507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"84'2","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-04T09:28:42.971811+0000","last_change":"2024-03-04T09:28:36.909808+0000","last_active":"2024-03-04T09:28:42.971811+0000","last_peered":"2024-03-04T09:28:42.971811+0000","last_clean":"2024-03-04T09:28:42.971811+0000","last_became_active":"2024-03-04T09:28:36.909486+0000","last_became_peered":"2024-03-04T09:28:36.909486+0000","last_unstale":"2024-03-04T09:28:42.971811+0000","last_undegraded":"2024-03-04T09:28:42.971811+0000","last_fullsized":"2024-03-04T09:28:42.971811+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:35.885776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:35.885776+0000","last_clean_scrub_stamp":"2024-03-04T09:28:35.885776+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T19:38:15.524941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-04T09:28:42.933892+0000","last_change":"2024-03-04T09:28:37.910807+0000","last_active":"2024-03-04T09:28:42.933892+0000","last_peered":"2024-03-04T09:28:42.933892+0000","last_clean":"2024-03-04T09:28:42.933892+0000","last_became_active":"2024-03-04T09:28:37.910504+0000","last_became_peered":"2024-03-04T09:28:37.910504+0000","last_unstale":"2024-03-04T09:28:42.933892+0000","last_undegraded":"2024-03-04T09:28:42.933892+0000","last_fullsized":"2024-03-04T09:28:42.933892+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_clean_scrub_stamp":"2024-03-04T09:28:36.887656+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T20:03:35.484747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-04T09:28:42.970986+0000","last_change":"2024-03-04T09:28:41.949798+0000","last_active":"2024-03-04T09:28:42.970986+0000","last_peered":"2024-03-04T09:28:42.970986+0000","last_clean":"2024-03-04T09:28:42.970986+0000","last_became_active":"2024-03-04T09:28:41.949358+0000","last_became_peered":"2024-03-04T09:28:41.949358+0000","last_unstale":"2024-03-04T09:28:42.970986+0000","last_undegraded":"2024-03-04T09:28:42.970986+0000","last_fullsized":"2024-03-04T09:28:42.970986+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:40.917115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:40.917115+0000","last_clean_scrub_stamp":"2024-03-04T09:28:40.917115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T13:30:26.519090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-04T09:28:42.932771+0000","last_change":"2024-03-04T09:28:36.912458+0000","last_active":"2024-03-04T09:28:42.932771+0000","last_peered":"2024-03-04T09:28:42.932771+0000","last_clean":"2024-03-04T09:28:42.932771+0000","last_became_active":"2024-03-04T09:28:36.912132+0000","last_became_peered":"2024-03-04T09:28:36.912132+0000","last_unstale":"2024-03-04T09:28:42.932771+0000","last_undegraded":"2024-03-04T09:28:42.932771+0000","last_fullsized":"2024-03-04T09:28:42.932771+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:35.885776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:35.885776+0000","last_clean_scrub_stamp":"2024-03-04T09:28:35.885776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T13:51:29.713381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-04T09:28:42.934196+0000","last_change":"2024-03-04T09:28:37.915750+0000","last_active":"2024-03-04T09:28:42.934196+0000","last_peered":"2024-03-04T09:28:42.934196+0000","last_clean":"2024-03-04T09:28:42.934196+0000","last_became_active":"2024-03-04T09:28:37.915424+0000","last_became_peered":"2024-03-04T09:28:37.915424+0000","last_unstale":"2024-03-04T09:28:42.934196+0000","last_undegraded":"2024-03-04T09:28:42.934196+0000","last_fullsized":"2024-03-04T09:28:42.934196+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_clean_scrub_stamp":"2024-03-04T09:28:36.887656+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T19:15:39.555122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-04T09:28:42.971982+0000","last_change":"2024-03-04T09:28:41.949321+0000","last_active":"2024-03-04T09:28:42.971982+0000","last_peered":"2024-03-04T09:28:42.971982+0000","last_clean":"2024-03-04T09:28:42.971982+0000","last_became_active":"2024-03-04T09:28:41.948302+0000","last_became_peered":"2024-03-04T09:28:41.948302+0000","last_unstale":"2024-03-04T09:28:42.971982+0000","last_undegraded":"2024-03-04T09:28:42.971982+0000","last_fullsized":"2024-03-04T09:28:42.971982+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:40.917115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:40.917115+0000","last_clean_scrub_stamp":"2024-03-04T09:28:40.917115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T10:18:44.988651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"84'1","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-04T09:28:42.973358+0000","last_change":"2024-03-04T09:28:36.909585+0000","last_active":"2024-03-04T09:28:42.973358+0000","last_peered":"2024-03-04T09:28:42.973358+0000","last_clean":"2024-03-04T09:28:42.973358+0000","last_became_active":"2024-03-04T09:28:36.909286+0000","last_became_peered":"2024-03-04T09:28:36.909286+0000","last_unstale":"2024-03-04T09:28:42.973358+0000","last_undegraded":"2024-03-04T09:28:42.973358+0000","last_fullsized":"2024-03-04T09:28:42.973358+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:35.885776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:35.885776+0000","last_clean_scrub_stamp":"2024-03-04T09:28:35.885776+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T09:29:45.236355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-04T09:28:42.933788+0000","last_change":"2024-03-04T09:28:37.924411+0000","last_active":"2024-03-04T09:28:42.933788+0000","last_peered":"2024-03-04T09:28:42.933788+0000","last_clean":"2024-03-04T09:28:42.933788+0000","last_became_active":"2024-03-04T09:28:37.924147+0000","last_became_peered":"2024-03-04T09:28:37.924147+0000","last_unstale":"2024-03-04T09:28:42.933788+0000","last_undegraded":"2024-03-04T09:28:42.933788+0000","last_fullsized":"2024-03-04T09:28:42.933788+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_clean_scrub_stamp":"2024-03-04T09:28:36.887656+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T15:05:16.573937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-04T09:28:42.971552+0000","last_change":"2024-03-04T09:28:41.943710+0000","last_active":"2024-03-04T09:28:42.971552+0000","last_peered":"2024-03-04T09:28:42.971552+0000","last_clean":"2024-03-04T09:28:42.971552+0000","last_became_active":"2024-03-04T09:28:41.943258+0000","last_became_peered":"2024-03-04T09:28:41.943258+0000","last_unstale":"2024-03-04T09:28:42.971552+0000","last_undegraded":"2024-03-04T09:28:42.971552+0000","last_fullsized":"2024-03-04T09:28:42.971552+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:40.917115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:40.917115+0000","last_clean_scrub_stamp":"2024-03-04T09:28:40.917115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T15:32:52.597594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"84'2","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-04T09:28:42.973945+0000","last_change":"2024-03-04T09:28:36.909499+0000","last_active":"2024-03-04T09:28:42.973945+0000","last_peered":"2024-03-04T09:28:42.973945+0000","last_clean":"2024-03-04T09:28:42.973945+0000","last_became_active":"2024-03-04T09:28:36.909168+0000","last_became_peered":"2024-03-04T09:28:36.909168+0000","last_unstale":"2024-03-04T09:28:42.973945+0000","last_undegraded":"2024-03-04T09:28:42.973945+0000","last_fullsized":"2024-03-04T09:28:42.973945+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:35.885776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:35.885776+0000","last_clean_scrub_stamp":"2024-03-04T09:28:35.885776+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T17:54:23.936630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-04T09:28:42.934178+0000","last_change":"2024-03-04T09:28:37.916569+0000","last_active":"2024-03-04T09:28:42.934178+0000","last_peered":"2024-03-04T09:28:42.934178+0000","last_clean":"2024-03-04T09:28:42.934178+0000","last_became_active":"2024-03-04T09:28:37.916264+0000","last_became_peered":"2024-03-04T09:28:37.916264+0000","last_unstale":"2024-03-04T09:28:42.934178+0000","last_undegraded":"2024-03-04T09:28:42.934178+0000","last_fullsized":"2024-03-04T09:28:42.934178+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_clean_scrub_stamp":"2024-03-04T09:28:36.887656+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T16:59:51.981066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-04T09:28:42.971509+0000","last_change":"2024-03-04T09:28:37.910027+0000","last_active":"2024-03-04T09:28:42.971509+0000","last_peered":"2024-03-04T09:28:42.971509+0000","last_clean":"2024-03-04T09:28:42.971509+0000","last_became_active":"2024-03-04T09:28:37.909715+0000","last_became_peered":"2024-03-04T09:28:37.909715+0000","last_unstale":"2024-03-04T09:28:42.971509+0000","last_undegraded":"2024-03-04T09:28:42.971509+0000","last_fullsized":"2024-03-04T09:28:42.971509+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_clean_scrub_stamp":"2024-03-04T09:28:36.887656+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T20:09:12.150573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-04T09:28:42.932253+0000","last_change":"2024-03-04T09:28:37.912211+0000","last_active":"2024-03-04T09:28:42.932253+0000","last_peered":"2024-03-04T09:28:42.932253+0000","last_clean":"2024-03-04T09:28:42.932253+0000","last_became_active":"2024-03-04T09:28:37.911904+0000","last_became_peered":"2024-03-04T09:28:37.911904+0000","last_unstale":"2024-03-04T09:28:42.932253+0000","last_undegraded":"2024-03-04T09:28:42.932253+0000","last_fullsized":"2024-03-04T09:28:42.932253+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_clean_scrub_stamp":"2024-03-04T09:28:36.887656+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T20:02:04.538449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-04T09:28:42.932147+0000","last_change":"2024-03-04T09:28:37.906956+0000","last_active":"2024-03-04T09:28:42.932147+0000","last_peered":"2024-03-04T09:28:42.932147+0000","last_clean":"2024-03-04T09:28:42.932147+0000","last_became_active":"2024-03-04T09:28:37.906642+0000","last_became_peered":"2024-03-04T09:28:37.906642+0000","last_unstale":"2024-03-04T09:28:42.932147+0000","last_undegraded":"2024-03-04T09:28:42.932147+0000","last_fullsized":"2024-03-04T09:28:42.932147+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_clean_scrub_stamp":"2024-03-04T09:28:36.887656+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T20:58:18.387119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-04T09:28:42.972180+0000","last_change":"2024-03-04T09:28:37.916105+0000","last_active":"2024-03-04T09:28:42.972180+0000","last_peered":"2024-03-04T09:28:42.972180+0000","last_clean":"2024-03-04T09:28:42.972180+0000","last_became_active":"2024-03-04T09:28:37.913893+0000","last_became_peered":"2024-03-04T09:28:37.913893+0000","last_unstale":"2024-03-04T09:28:42.972180+0000","last_undegraded":"2024-03-04T09:28:42.972180+0000","last_fullsized":"2024-03-04T09:28:42.972180+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_clean_scrub_stamp":"2024-03-04T09:28:36.887656+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T12:57:46.128587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-04T09:28:42.973726+0000","last_change":"2024-03-04T09:28:37.912702+0000","last_active":"2024-03-04T09:28:42.973726+0000","last_peered":"2024-03-04T09:28:42.973726+0000","last_clean":"2024-03-04T09:28:42.973726+0000","last_became_active":"2024-03-04T09:28:37.912397+0000","last_became_peered":"2024-03-04T09:28:37.912397+0000","last_unstale":"2024-03-04T09:28:42.973726+0000","last_undegraded":"2024-03-04T09:28:42.973726+0000","last_fullsized":"2024-03-04T09:28:42.973726+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_clean_scrub_stamp":"2024-03-04T09:28:36.887656+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T16:36:59.575111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-04T09:28:42.972243+0000","last_change":"2024-03-04T09:28:37.909801+0000","last_active":"2024-03-04T09:28:42.972243+0000","last_peered":"2024-03-04T09:28:42.972243+0000","last_clean":"2024-03-04T09:28:42.972243+0000","last_became_active":"2024-03-04T09:28:37.909505+0000","last_became_peered":"2024-03-04T09:28:37.909505+0000","last_unstale":"2024-03-04T09:28:42.972243+0000","last_undegraded":"2024-03-04T09:28:42.972243+0000","last_fullsized":"2024-03-04T09:28:42.972243+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_clean_scrub_stamp":"2024-03-04T09:28:36.887656+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T11:56:44.276188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-04T09:28:42.932875+0000","last_change":"2024-03-04T09:28:37.911076+0000","last_active":"2024-03-04T09:28:42.932875+0000","last_peered":"2024-03-04T09:28:42.932875+0000","last_clean":"2024-03-04T09:28:42.932875+0000","last_became_active":"2024-03-04T09:28:37.910786+0000","last_became_peered":"2024-03-04T09:28:37.910786+0000","last_unstale":"2024-03-04T09:28:42.932875+0000","last_undegraded":"2024-03-04T09:28:42.932875+0000","last_fullsized":"2024-03-04T09:28:42.932875+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_clean_scrub_stamp":"2024-03-04T09:28:36.887656+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T11:42:41.932235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-04T09:28:42.972340+0000","last_change":"2024-03-04T09:28:37.914661+0000","last_active":"2024-03-04T09:28:42.972340+0000","last_peered":"2024-03-04T09:28:42.972340+0000","last_clean":"2024-03-04T09:28:42.972340+0000","last_became_active":"2024-03-04T09:28:37.913851+0000","last_became_peered":"2024-03-04T09:28:37.913851+0000","last_unstale":"2024-03-04T09:28:42.972340+0000","last_undegraded":"2024-03-04T09:28:42.972340+0000","last_fullsized":"2024-03-04T09:28:42.972340+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_clean_scrub_stamp":"2024-03-04T09:28:36.887656+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T10:45:43.670006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-04T09:28:42.972285+0000","last_change":"2024-03-04T09:28:37.904746+0000","last_active":"2024-03-04T09:28:42.972285+0000","last_peered":"2024-03-04T09:28:42.972285+0000","last_clean":"2024-03-04T09:28:42.972285+0000","last_became_active":"2024-03-04T09:28:37.904466+0000","last_became_peered":"2024-03-04T09:28:37.904466+0000","last_unstale":"2024-03-04T09:28:42.972285+0000","last_undegraded":"2024-03-04T09:28:42.972285+0000","last_fullsized":"2024-03-04T09:28:42.972285+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_clean_scrub_stamp":"2024-03-04T09:28:36.887656+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T13:59:24.810820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-04T09:28:42.971972+0000","last_change":"2024-03-04T09:28:37.913252+0000","last_active":"2024-03-04T09:28:42.971972+0000","last_peered":"2024-03-04T09:28:42.971972+0000","last_clean":"2024-03-04T09:28:42.971972+0000","last_became_active":"2024-03-04T09:28:37.910819+0000","last_became_peered":"2024-03-04T09:28:37.910819+0000","last_unstale":"2024-03-04T09:28:42.971972+0000","last_undegraded":"2024-03-04T09:28:42.971972+0000","last_fullsized":"2024-03-04T09:28:42.971972+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_clean_scrub_stamp":"2024-03-04T09:28:36.887656+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T13:39:47.886505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-04T09:28:42.971241+0000","last_change":"2024-03-04T09:28:37.915716+0000","last_active":"2024-03-04T09:28:42.971241+0000","last_peered":"2024-03-04T09:28:42.971241+0000","last_clean":"2024-03-04T09:28:42.971241+0000","last_became_active":"2024-03-04T09:28:37.915398+0000","last_became_peered":"2024-03-04T09:28:37.915398+0000","last_unstale":"2024-03-04T09:28:42.971241+0000","last_undegraded":"2024-03-04T09:28:42.971241+0000","last_fullsized":"2024-03-04T09:28:42.971241+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_clean_scrub_stamp":"2024-03-04T09:28:36.887656+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T20:10:53.793715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-04T09:28:42.973586+0000","last_change":"2024-03-04T09:28:37.906671+0000","last_active":"2024-03-04T09:28:42.973586+0000","last_peered":"2024-03-04T09:28:42.973586+0000","last_clean":"2024-03-04T09:28:42.973586+0000","last_became_active":"2024-03-04T09:28:37.906344+0000","last_became_peered":"2024-03-04T09:28:37.906344+0000","last_unstale":"2024-03-04T09:28:42.973586+0000","last_undegraded":"2024-03-04T09:28:42.973586+0000","last_fullsized":"2024-03-04T09:28:42.973586+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_clean_scrub_stamp":"2024-03-04T09:28:36.887656+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T16:02:18.204424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-04T09:28:42.971914+0000","last_change":"2024-03-04T09:28:37.916895+0000","last_active":"2024-03-04T09:28:42.971914+0000","last_peered":"2024-03-04T09:28:42.971914+0000","last_clean":"2024-03-04T09:28:42.971914+0000","last_became_active":"2024-03-04T09:28:37.916585+0000","last_became_peered":"2024-03-04T09:28:37.916585+0000","last_unstale":"2024-03-04T09:28:42.971914+0000","last_undegraded":"2024-03-04T09:28:42.971914+0000","last_fullsized":"2024-03-04T09:28:42.971914+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_clean_scrub_stamp":"2024-03-04T09:28:36.887656+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T12:13:50.941390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":24119,"num_objects":32,"num_object_clones":0,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":0,"num_read":9,"num_read_kb":28,"num_write":82,"num_write_kb":43,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":22,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":192512,"data_stored":37380,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":70,"ondisk_log_size":70,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383698,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27984,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717168,"statfs":{"total":95995035648,"available":95966380032,"internally_reserved":0,"allocated":991232,"data_stored":481893,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":72,"seq":309237645334,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36928,"kb_used_data":1592,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708224,"statfs":{"total":95995035648,"available":95957221376,"internally_reserved":0,"allocated":1630208,"data_stored":1079994,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":67,"seq":287762808860,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36316,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708836,"statfs":{"total":95995035648,"available":95957848064,"internally_reserved":0,"allocated":1003520,"data_stored":478033,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":36174273},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":8,"up_from":59,"seq":253403070496,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27992,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717160,"statfs":{"total":95995035648,"available":95966371840,"internally_reserved":0,"allocated":991232,"data_stored":477819,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332133,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28564,"kb_used_data":1544,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716588,"statfs":{"total":95995035648,"available":95965786112,"internally_reserved":0,"allocated":1581056,"data_stored":1068165,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495659,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27984,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717168,"statfs":{"total":95995035648,"available":95966380032,"internally_reserved":0,"allocated":983040,"data_stored":477215,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574915,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27988,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717164,"statfs":{"total":95995035648,"available":95966375936,"internally_reserved":0,"allocated":995328,"data_stored":481057,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738441,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28004,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717148,"statfs":{"total":95995035648,"available":95966359552,"internally_reserved":0,"allocated":1011712,"data_stored":478693,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411391,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27992,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717160,"statfs":{"total":95995035648,"available":95966371840,"internally_reserved":0,"allocated":999424,"data_stored":481091,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182457,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36312,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708840,"statfs":{"total":95995035648,"available":95957852160,"internally_reserved":0,"allocated":999424,"data_stored":477499,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920820,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27988,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717164,"statfs":{"total":95995035648,"available":95966375936,"internally_reserved":0,"allocated":987136,"data_stored":477237,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757296,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32732,"kb_used_data":1552,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712420,"statfs":{"total":95995035648,"available":95961518080,"internally_reserved":0,"allocated":1589248,"data_stored":1067765,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":3876,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":25110,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-04T09:29:07.241 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-03-04T09:29:07.242 INFO:tasks.cephadm:Setting up client nodes... 2024-03-04T09:29:07.242 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4e059f08-da08-11ee-95c6-87774f69a715 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-03-04T09:29:07.336 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:29:07 smithi191 ceph-mon[36816]: pgmap v302: 129 pgs: 129 active+clean; 600 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 2.2 KiB/s wr, 3 op/s 2024-03-04T09:29:07.336 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:29:07 smithi191 ceph-mon[36816]: from='client.14913 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T09:29:07.336 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:29:07 smithi191 ceph-mon[36816]: from='client.14919 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T09:29:07.336 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:29:07 smithi191 ceph-mon[36816]: from='client.? 172.21.15.52:0/2501347692' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-04T09:29:07.336 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:29:07 smithi191 ceph-mon[36816]: from='client.? 172.21.15.52:0/3689523684' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-04T09:29:07.381 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:29:07 smithi106 ceph-mon[36484]: pgmap v302: 129 pgs: 129 active+clean; 600 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 2.2 KiB/s wr, 3 op/s 2024-03-04T09:29:07.381 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:29:07 smithi106 ceph-mon[36484]: from='client.14913 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T09:29:07.381 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:29:07 smithi106 ceph-mon[36484]: from='client.14919 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T09:29:07.381 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:29:07 smithi106 ceph-mon[36484]: from='client.? 172.21.15.52:0/2501347692' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-04T09:29:07.381 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:29:07 smithi106 ceph-mon[36484]: from='client.? 172.21.15.52:0/3689523684' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-04T09:29:07.521 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/4e059f08-da08-11ee-95c6-87774f69a715/mon.a/config 2024-03-04T09:29:08.335 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:29:08 smithi052 ceph-mon[32473]: from='client.14937 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-04T09:29:08.335 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:29:08 smithi052 ceph-mon[32473]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-03-04T09:29:08.335 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:29:08 smithi052 ceph-mon[32473]: from='client.14943 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-04T09:29:08.336 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:29:08 smithi191 ceph-mon[36816]: from='client.14937 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-04T09:29:08.336 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:29:08 smithi191 ceph-mon[36816]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-03-04T09:29:08.336 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:29:08 smithi191 ceph-mon[36816]: from='client.14943 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-04T09:29:08.381 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:29:08 smithi106 ceph-mon[36484]: from='client.14937 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-04T09:29:08.381 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:29:08 smithi106 ceph-mon[36484]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-03-04T09:29:08.381 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:29:08 smithi106 ceph-mon[36484]: from='client.14943 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-04T09:29:09.075 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:29:09 smithi052 ceph-mon[32473]: pgmap v303: 129 pgs: 129 active+clean; 603 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 4.1 KiB/s wr, 10 op/s 2024-03-04T09:29:09.337 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:29:09 smithi191 ceph-mon[36816]: pgmap v303: 129 pgs: 129 active+clean; 603 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 4.1 KiB/s wr, 10 op/s 2024-03-04T09:29:09.382 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:29:09 smithi106 ceph-mon[36484]: pgmap v303: 129 pgs: 129 active+clean; 603 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 4.1 KiB/s wr, 10 op/s 2024-03-04T09:29:09.488 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:09.488+0000 7f40fed9e700 1 -- 172.21.15.52:0/403709666 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f40f8101fc0 msgr2=0x7f40f8102440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:29:09.488 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:09.488+0000 7f40fed9e700 1 --2- 172.21.15.52:0/403709666 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f40f8101fc0 0x7f40f8102440 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f40ec009990 tx=0x7f40ec009ca0 comp rx=0 tx=0).stop 2024-03-04T09:29:09.488 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:09.488+0000 7f40fed9e700 1 -- 172.21.15.52:0/403709666 shutdown_connections 2024-03-04T09:29:09.488 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:09.488+0000 7f40fed9e700 1 --2- 172.21.15.52:0/403709666 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f40f8102980 0x7f40f81092a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:29:09.488 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:09.488+0000 7f40fed9e700 1 --2- 172.21.15.52:0/403709666 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f40f8100e60 0x7f40f8101280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:29:09.488 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:09.488+0000 7f40fed9e700 1 --2- 172.21.15.52:0/403709666 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f40f8101fc0 0x7f40f8102440 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:29:09.488 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:09.488+0000 7f40fed9e700 1 -- 172.21.15.52:0/403709666 >> 172.21.15.52:0/403709666 conn(0x7f40f80fc400 msgr2=0x7f40f80fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:29:09.488 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:09.488+0000 7f40fed9e700 1 -- 172.21.15.52:0/403709666 shutdown_connections 2024-03-04T09:29:09.488 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:09.488+0000 7f40fed9e700 1 -- 172.21.15.52:0/403709666 wait complete. 2024-03-04T09:29:09.489 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:09.489+0000 7f40fed9e700 1 Processor -- start 2024-03-04T09:29:09.489 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:09.489+0000 7f40fed9e700 1 -- start start 2024-03-04T09:29:09.489 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:09.489+0000 7f40fed9e700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f40f8100e60 0x7f40f81980f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:29:09.489 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:09.489+0000 7f40fed9e700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f40f8101fc0 0x7f40f8198630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:29:09.489 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:09.489+0000 7f40fed9e700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f40f8102980 0x7f40f819f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:29:09.489 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:09.489+0000 7f40fed9e700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40f810cab0 con 0x7f40f8101fc0 2024-03-04T09:29:09.489 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:09.489+0000 7f40fed9e700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40f810c930 con 0x7f40f8102980 2024-03-04T09:29:09.489 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:09.489+0000 7f40fed9e700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40f810cc30 con 0x7f40f8100e60 2024-03-04T09:29:09.489 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:09.489+0000 7f40fd59b700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f40f8101fc0 0x7f40f8198630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:29:09.489 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:09.489+0000 7f40fd59b700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f40f8101fc0 0x7f40f8198630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:47074/0 (socket says 172.21.15.52:47074) 2024-03-04T09:29:09.489 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:09.489+0000 7f40fd59b700 1 -- 172.21.15.52:0/3067272504 learned_addr learned my addr 172.21.15.52:0/3067272504 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-04T09:29:09.489 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:09.489+0000 7f40fe59d700 1 --2- 172.21.15.52:0/3067272504 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f40f8102980 0x7f40f819f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:29:09.489 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:09.490+0000 7f40fdd9c700 1 --2- 172.21.15.52:0/3067272504 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f40f8100e60 0x7f40f81980f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:29:09.489 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:09.490+0000 7f40fe59d700 1 -- 172.21.15.52:0/3067272504 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f40f8100e60 msgr2=0x7f40f81980f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:29:09.490 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:09.490+0000 7f40fe59d700 1 --2- 172.21.15.52:0/3067272504 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f40f8100e60 0x7f40f81980f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:29:09.490 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:09.490+0000 7f40fe59d700 1 -- 172.21.15.52:0/3067272504 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f40f8101fc0 msgr2=0x7f40f8198630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:29:09.490 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:09.490+0000 7f40fe59d700 1 --2- 172.21.15.52:0/3067272504 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f40f8101fc0 0x7f40f8198630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:29:09.490 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:09.490+0000 7f40fe59d700 1 -- 172.21.15.52:0/3067272504 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40f819fd50 con 0x7f40f8102980 2024-03-04T09:29:09.490 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:09.490+0000 7f40fd59b700 1 --2- 172.21.15.52:0/3067272504 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f40f8101fc0 0x7f40f8198630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-04T09:29:09.490 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:09.490+0000 7f40fe59d700 1 --2- 172.21.15.52:0/3067272504 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f40f8102980 0x7f40f819f6b0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f40f000efc0 tx=0x7f40f000c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:29:09.490 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:09.490+0000 7f40eaffd700 1 -- 172.21.15.52:0/3067272504 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40f00092e0 con 0x7f40f8102980 2024-03-04T09:29:09.490 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:09.490+0000 7f40eaffd700 1 -- 172.21.15.52:0/3067272504 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f40f0004bb0 con 0x7f40f8102980 2024-03-04T09:29:09.490 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:09.490+0000 7f40fed9e700 1 -- 172.21.15.52:0/3067272504 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40f819ffe0 con 0x7f40f8102980 2024-03-04T09:29:09.490 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:09.490+0000 7f40eaffd700 1 -- 172.21.15.52:0/3067272504 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40f0004280 con 0x7f40f8102980 2024-03-04T09:29:09.490 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:09.490+0000 7f40fed9e700 1 -- 172.21.15.52:0/3067272504 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40f81a0440 con 0x7f40f8102980 2024-03-04T09:29:09.490 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:09.491+0000 7f40fed9e700 1 -- 172.21.15.52:0/3067272504 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f40f80636f0 con 0x7f40f8102980 2024-03-04T09:29:09.491 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:09.492+0000 7f40eaffd700 1 -- 172.21.15.52:0/3067272504 <== mon.2 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 17) v1 ==== 136807+0+0 (secure 0 0 0) 0x7f40f001b030 con 0x7f40f8102980 2024-03-04T09:29:09.491 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:09.493+0000 7f40eaffd700 1 --2- 172.21.15.52:0/3067272504 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f40e40a1f50 0x7f40e40a4410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:29:09.491 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:09.493+0000 7f40fdd9c700 1 --2- 172.21.15.52:0/3067272504 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f40e40a1f50 0x7f40e40a4410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:29:09.491 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:09.493+0000 7f40eaffd700 1 -- 172.21.15.52:0/3067272504 <== mon.2 v2:172.21.15.106:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8802+0+0 (secure 0 0 0) 0x7f40f00cde60 con 0x7f40f8102980 2024-03-04T09:29:09.491 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:09.493+0000 7f40fdd9c700 1 --2- 172.21.15.52:0/3067272504 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f40e40a1f50 0x7f40e40a4410 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f40f4009c60 tx=0x7f40f4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:29:09.493 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:09.496+0000 7f40eaffd700 1 -- 172.21.15.52:0/3067272504 <== mon.2 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f40f0022020 con 0x7f40f8102980 2024-03-04T09:29:09.780 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:09.782+0000 7f40fed9e700 1 -- 172.21.15.52:0/3067272504 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) v1 -- 0x7f40f81a06c0 con 0x7f40f8102980 2024-03-04T09:29:09.785 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:09.788+0000 7f40eaffd700 1 -- 172.21.15.52:0/3067272504 <== mon.2 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v32) v1 ==== 170+0+59 (secure 0 0 0) 0x7f40f0010040 con 0x7f40f8102980 2024-03-04T09:29:09.785 INFO:teuthology.orchestra.run.smithi052.stdout:[client.0] 2024-03-04T09:29:09.785 INFO:teuthology.orchestra.run.smithi052.stdout: key = AQBllOVl79nELhAAeASS4gs7KsjuCOdVXkA48A== 2024-03-04T09:29:09.787 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:09.790+0000 7f40fed9e700 1 -- 172.21.15.52:0/3067272504 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f40e40a1f50 msgr2=0x7f40e40a4410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:29:09.787 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:09.790+0000 7f40fed9e700 1 --2- 172.21.15.52:0/3067272504 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f40e40a1f50 0x7f40e40a4410 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f40f4009c60 tx=0x7f40f4009260 comp rx=0 tx=0).stop 2024-03-04T09:29:09.787 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:09.790+0000 7f40fed9e700 1 -- 172.21.15.52:0/3067272504 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f40f8102980 msgr2=0x7f40f819f6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:29:09.787 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:09.790+0000 7f40fed9e700 1 --2- 172.21.15.52:0/3067272504 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f40f8102980 0x7f40f819f6b0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f40f000efc0 tx=0x7f40f000c3a0 comp rx=0 tx=0).stop 2024-03-04T09:29:09.788 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:09.790+0000 7f40fed9e700 1 -- 172.21.15.52:0/3067272504 shutdown_connections 2024-03-04T09:29:09.788 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:09.790+0000 7f40fed9e700 1 --2- 172.21.15.52:0/3067272504 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f40e40a1f50 0x7f40e40a4410 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:29:09.788 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:09.790+0000 7f40fed9e700 1 --2- 172.21.15.52:0/3067272504 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f40f8100e60 0x7f40f81980f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:29:09.788 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:09.790+0000 7f40fed9e700 1 --2- 172.21.15.52:0/3067272504 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f40f8102980 0x7f40f819f6b0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:29:09.788 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:09.790+0000 7f40fed9e700 1 --2- 172.21.15.52:0/3067272504 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f40f8101fc0 0x7f40f8198630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:29:09.788 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:09.790+0000 7f40fed9e700 1 -- 172.21.15.52:0/3067272504 >> 172.21.15.52:0/3067272504 conn(0x7f40f80fc400 msgr2=0x7f40f80fdd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:29:09.788 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:09.790+0000 7f40fed9e700 1 -- 172.21.15.52:0/3067272504 shutdown_connections 2024-03-04T09:29:09.788 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:09.790+0000 7f40fed9e700 1 -- 172.21.15.52:0/3067272504 wait complete. 2024-03-04T09:29:10.085 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:29:10 smithi052 ceph-mon[32473]: from='client.? 172.21.15.52:0/3067272504' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-03-04T09:29:10.086 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:29:10 smithi052 ceph-mon[32473]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-03-04T09:29:10.086 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:29:10 smithi052 ceph-mon[32473]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-03-04T09:29:10.131 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:29:10 smithi106 ceph-mon[36484]: from='client.? 172.21.15.52:0/3067272504' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-03-04T09:29:10.131 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:29:10 smithi106 ceph-mon[36484]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-03-04T09:29:10.131 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:29:10 smithi106 ceph-mon[36484]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-03-04T09:29:10.336 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:29:10 smithi191 ceph-mon[36816]: from='client.? 172.21.15.52:0/3067272504' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-03-04T09:29:10.337 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:29:10 smithi191 ceph-mon[36816]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-03-04T09:29:10.337 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:29:10 smithi191 ceph-mon[36816]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-03-04T09:29:10.554 DEBUG:teuthology.orchestra.run.smithi052:> set -ex 2024-03-04T09:29:10.554 DEBUG:teuthology.orchestra.run.smithi052:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-03-04T09:29:10.554 DEBUG:teuthology.orchestra.run.smithi052:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-03-04T09:29:10.607 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-03-04T09:29:10.607 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-03-04T09:29:10.608 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd shell --fsid 4e059f08-da08-11ee-95c6-87774f69a715 -- ceph mgr dump --format=json 2024-03-04T09:29:10.899 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/4e059f08-da08-11ee-95c6-87774f69a715/mon.a/config 2024-03-04T09:29:11.085 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:29:11 smithi052 ceph-mon[32473]: pgmap v304: 129 pgs: 129 active+clean; 603 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 4.2 KiB/s wr, 10 op/s 2024-03-04T09:29:11.336 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:29:11 smithi191 ceph-mon[36816]: pgmap v304: 129 pgs: 129 active+clean; 603 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 4.2 KiB/s wr, 10 op/s 2024-03-04T09:29:11.381 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:29:11 smithi106 ceph-mon[36484]: pgmap v304: 129 pgs: 129 active+clean; 603 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 4.2 KiB/s wr, 10 op/s 2024-03-04T09:29:12.618 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:12.620+0000 7f689a776700 1 -- 172.21.15.52:0/552891099 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6894108220 msgr2=0x7f6894108600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:29:12.618 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:12.620+0000 7f689a776700 1 --2- 172.21.15.52:0/552891099 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6894108220 0x7f6894108600 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f6884009990 tx=0x7f6884009ca0 comp rx=0 tx=0).stop 2024-03-04T09:29:12.618 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:12.620+0000 7f689a776700 1 -- 172.21.15.52:0/552891099 shutdown_connections 2024-03-04T09:29:12.618 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:12.620+0000 7f689a776700 1 --2- 172.21.15.52:0/552891099 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6894102b00 0x7f689410ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:29:12.618 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:12.620+0000 7f689a776700 1 --2- 172.21.15.52:0/552891099 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6894108220 0x7f6894108600 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:29:12.618 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:12.620+0000 7f689a776700 1 --2- 172.21.15.52:0/552891099 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6894102140 0x7f68941025c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:29:12.618 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:12.620+0000 7f689a776700 1 -- 172.21.15.52:0/552891099 >> 172.21.15.52:0/552891099 conn(0x7f6894079180 msgr2=0x7f6894079590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:29:12.618 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:12.620+0000 7f689a776700 1 -- 172.21.15.52:0/552891099 shutdown_connections 2024-03-04T09:29:12.618 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:12.620+0000 7f689a776700 1 -- 172.21.15.52:0/552891099 wait complete. 2024-03-04T09:29:12.618 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:12.621+0000 7f689a776700 1 Processor -- start 2024-03-04T09:29:12.619 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:12.621+0000 7f689a776700 1 -- start start 2024-03-04T09:29:12.620 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:12.621+0000 7f689a776700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6894102140 0x7f689419e5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:29:12.620 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:12.621+0000 7f689a776700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6894102b00 0x7f689419eae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:29:12.620 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:12.621+0000 7f689a776700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6894108220 0x7f68941986c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:29:12.620 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:12.621+0000 7f689a776700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6894112380 con 0x7f6894102b00 2024-03-04T09:29:12.621 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:12.621+0000 7f689a776700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6894112200 con 0x7f6894108220 2024-03-04T09:29:12.621 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:12.621+0000 7f689a776700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6894112500 con 0x7f6894102140 2024-03-04T09:29:12.621 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:12.621+0000 7f68937fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6894102b00 0x7f689419eae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:29:12.621 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:12.621+0000 7f68937fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6894102b00 0x7f689419eae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:47088/0 (socket says 172.21.15.52:47088) 2024-03-04T09:29:12.621 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:12.621+0000 7f68937fe700 1 -- 172.21.15.52:0/3670959569 learned_addr learned my addr 172.21.15.52:0/3670959569 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-04T09:29:12.621 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:12.622+0000 7f6893fff700 1 --2- 172.21.15.52:0/3670959569 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6894102140 0x7f689419e5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:29:12.621 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:12.622+0000 7f6898d13700 1 --2- 172.21.15.52:0/3670959569 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6894108220 0x7f68941986c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:29:12.621 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:12.622+0000 7f68937fe700 1 -- 172.21.15.52:0/3670959569 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6894102140 msgr2=0x7f689419e5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:29:12.621 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:12.622+0000 7f68937fe700 1 --2- 172.21.15.52:0/3670959569 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6894102140 0x7f689419e5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:29:12.621 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:12.622+0000 7f68937fe700 1 -- 172.21.15.52:0/3670959569 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6894108220 msgr2=0x7f68941986c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:29:12.621 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:12.622+0000 7f68937fe700 1 --2- 172.21.15.52:0/3670959569 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6894108220 0x7f68941986c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:29:12.621 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:12.622+0000 7f68937fe700 1 -- 172.21.15.52:0/3670959569 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6894198fa0 con 0x7f6894102b00 2024-03-04T09:29:12.621 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:12.622+0000 7f68937fe700 1 --2- 172.21.15.52:0/3670959569 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6894102b00 0x7f689419eae0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f688800e9a0 tx=0x7f688800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:29:12.621 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:12.622+0000 7f68917fa700 1 -- 172.21.15.52:0/3670959569 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f688800ca10 con 0x7f6894102b00 2024-03-04T09:29:12.622 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:12.622+0000 7f68917fa700 1 -- 172.21.15.52:0/3670959569 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f688800cbb0 con 0x7f6894102b00 2024-03-04T09:29:12.622 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:12.622+0000 7f689a776700 1 -- 172.21.15.52:0/3670959569 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6894199290 con 0x7f6894102b00 2024-03-04T09:29:12.622 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:12.622+0000 7f68917fa700 1 -- 172.21.15.52:0/3670959569 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6888018680 con 0x7f6894102b00 2024-03-04T09:29:12.622 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:12.622+0000 7f689a776700 1 -- 172.21.15.52:0/3670959569 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68941a5520 con 0x7f6894102b00 2024-03-04T09:29:12.622 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:12.623+0000 7f689a776700 1 -- 172.21.15.52:0/3670959569 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6894103800 con 0x7f6894102b00 2024-03-04T09:29:12.623 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:12.624+0000 7f68917fa700 1 -- 172.21.15.52:0/3670959569 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 17) v1 ==== 136807+0+0 (secure 0 0 0) 0x7f6888010430 con 0x7f6894102b00 2024-03-04T09:29:12.623 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:12.625+0000 7f68917fa700 1 --2- 172.21.15.52:0/3670959569 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f687c0a1f20 0x7f687c0a43e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:29:12.623 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:12.625+0000 7f6893fff700 1 --2- 172.21.15.52:0/3670959569 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f687c0a1f20 0x7f687c0a43e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:29:12.623 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:12.625+0000 7f68917fa700 1 -- 172.21.15.52:0/3670959569 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8802+0+0 (secure 0 0 0) 0x7f68880ce830 con 0x7f6894102b00 2024-03-04T09:29:12.623 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:12.626+0000 7f6893fff700 1 --2- 172.21.15.52:0/3670959569 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f687c0a1f20 0x7f687c0a43e0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f688400b2e0 tx=0x7f6884005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:29:12.625 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:12.629+0000 7f68917fa700 1 -- 172.21.15.52:0/3670959569 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f688809b860 con 0x7f6894102b00 2024-03-04T09:29:12.886 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:12.889+0000 7f689a776700 1 -- 172.21.15.52:0/3670959569 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7f6894199ba0 con 0x7f6894102b00 2024-03-04T09:29:12.891 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:12.894+0000 7f68917fa700 1 -- 172.21.15.52:0/3670959569 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v17) v1 ==== 74+0+261466 (secure 0 0 0) 0x7f688809b160 con 0x7f6894102b00 2024-03-04T09:29:12.899 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-03-04T09:29:12.901 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:12.897+0000 7f689a776700 1 -- 172.21.15.52:0/3670959569 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f687c0a1f20 msgr2=0x7f687c0a43e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:29:12.901 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:12.897+0000 7f689a776700 1 --2- 172.21.15.52:0/3670959569 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f687c0a1f20 0x7f687c0a43e0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f688400b2e0 tx=0x7f6884005960 comp rx=0 tx=0).stop 2024-03-04T09:29:12.901 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:12.897+0000 7f689a776700 1 -- 172.21.15.52:0/3670959569 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6894102b00 msgr2=0x7f689419eae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:29:12.901 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:12.897+0000 7f689a776700 1 --2- 172.21.15.52:0/3670959569 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6894102b00 0x7f689419eae0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f688800e9a0 tx=0x7f688800ecb0 comp rx=0 tx=0).stop 2024-03-04T09:29:12.901 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:12.897+0000 7f689a776700 1 -- 172.21.15.52:0/3670959569 shutdown_connections 2024-03-04T09:29:12.901 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:12.897+0000 7f689a776700 1 --2- 172.21.15.52:0/3670959569 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f687c0a1f20 0x7f687c0a43e0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:29:12.901 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:12.897+0000 7f689a776700 1 --2- 172.21.15.52:0/3670959569 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6894102140 0x7f689419e5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:29:12.901 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:12.897+0000 7f689a776700 1 --2- 172.21.15.52:0/3670959569 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6894108220 0x7f68941986c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:29:12.901 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:12.897+0000 7f689a776700 1 --2- 172.21.15.52:0/3670959569 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6894102b00 0x7f689419eae0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:29:12.901 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:12.897+0000 7f689a776700 1 -- 172.21.15.52:0/3670959569 >> 172.21.15.52:0/3670959569 conn(0x7f6894079180 msgr2=0x7f6894100e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:29:12.901 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:12.897+0000 7f689a776700 1 -- 172.21.15.52:0/3670959569 shutdown_connections 2024-03-04T09:29:12.901 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:12.897+0000 7f689a776700 1 -- 172.21.15.52:0/3670959569 wait complete. 2024-03-04T09:29:13.334 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:29:13 smithi052 ceph-mon[32473]: pgmap v305: 129 pgs: 129 active+clean; 625 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 6.2 KiB/s wr, 13 op/s 2024-03-04T09:29:13.335 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:29:13 smithi052 ceph-mon[32473]: from='client.? 172.21.15.52:0/3670959569' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-03-04T09:29:13.336 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:29:13 smithi191 ceph-mon[36816]: pgmap v305: 129 pgs: 129 active+clean; 625 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 6.2 KiB/s wr, 13 op/s 2024-03-04T09:29:13.336 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:29:13 smithi191 ceph-mon[36816]: from='client.? 172.21.15.52:0/3670959569' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-03-04T09:29:13.381 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:29:13 smithi106 ceph-mon[36484]: pgmap v305: 129 pgs: 129 active+clean; 625 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 6.2 KiB/s wr, 13 op/s 2024-03-04T09:29:13.381 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:29:13 smithi106 ceph-mon[36484]: from='client.? 172.21.15.52:0/3670959569' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-03-04T09:29:13.784 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":17,"flags":0,"active_gid":14152,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6800","nonce":3036033923},{"type":"v1","addr":"172.21.15.52:6801","nonce":3036033923}]},"active_addr":"172.21.15.52:6801/3036033923","active_change":"2024-03-04T09:21:39.402059+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":24109,"name":"z","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:latest","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":24113,"name":"y","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:latest","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:latest","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.52:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.52:0","nonce":2466768}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.52:0","nonce":2491451071}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.52:0","nonce":2080168931}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.52:0","nonce":2935914357}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.52:0","nonce":899140431}]}]} 2024-03-04T09:29:13.789 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-03-04T09:29:13.789 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-03-04T09:29:13.789 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd shell --fsid 4e059f08-da08-11ee-95c6-87774f69a715 -- ceph osd dump --format=json 2024-03-04T09:29:14.042 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/4e059f08-da08-11ee-95c6-87774f69a715/mon.a/config 2024-03-04T09:29:15.287 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:29:15 smithi052 ceph-mon[32473]: pgmap v306: 129 pgs: 129 active+clean; 625 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 6.2 KiB/s wr, 13 op/s 2024-03-04T09:29:15.336 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:29:15 smithi191 ceph-mon[36816]: pgmap v306: 129 pgs: 129 active+clean; 625 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 6.2 KiB/s wr, 13 op/s 2024-03-04T09:29:15.381 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:29:15 smithi106 ceph-mon[36484]: pgmap v306: 129 pgs: 129 active+clean; 625 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 6.2 KiB/s wr, 13 op/s 2024-03-04T09:29:15.855 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:15.858+0000 7f6aac70f700 1 -- 172.21.15.52:0/336968634 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6aa41080f0 msgr2=0x7f6aa41084d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:29:15.855 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:15.858+0000 7f6aac70f700 1 --2- 172.21.15.52:0/336968634 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6aa41080f0 0x7f6aa41084d0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f6a940099c0 tx=0x7f6a94009cd0 comp rx=0 tx=0).stop 2024-03-04T09:29:15.855 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:15.858+0000 7f6aac70f700 1 -- 172.21.15.52:0/336968634 shutdown_connections 2024-03-04T09:29:15.855 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:15.858+0000 7f6aac70f700 1 --2- 172.21.15.52:0/336968634 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6aa4102ad0 0x7f6aa410ef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:29:15.856 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:15.858+0000 7f6aac70f700 1 --2- 172.21.15.52:0/336968634 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6aa41080f0 0x7f6aa41084d0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:29:15.856 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:15.858+0000 7f6aac70f700 1 --2- 172.21.15.52:0/336968634 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6aa4102110 0x7f6aa4102590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:29:15.856 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:15.858+0000 7f6aac70f700 1 -- 172.21.15.52:0/336968634 >> 172.21.15.52:0/336968634 conn(0x7f6aa40791f0 msgr2=0x7f6aa4079600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:29:15.856 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:15.858+0000 7f6aac70f700 1 -- 172.21.15.52:0/336968634 shutdown_connections 2024-03-04T09:29:15.856 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:15.858+0000 7f6aac70f700 1 -- 172.21.15.52:0/336968634 wait complete. 2024-03-04T09:29:15.857 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:15.859+0000 7f6aac70f700 1 Processor -- start 2024-03-04T09:29:15.857 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:15.859+0000 7f6aac70f700 1 -- start start 2024-03-04T09:29:15.857 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:15.859+0000 7f6aac70f700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6aa4102110 0x7f6aa419e3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:29:15.857 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:15.859+0000 7f6aac70f700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6aa4102ad0 0x7f6aa419e920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:29:15.857 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:15.859+0000 7f6aac70f700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6aa41080f0 0x7f6aa41985a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:29:15.857 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:15.859+0000 7f6aac70f700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6aa41123a0 con 0x7f6aa41080f0 2024-03-04T09:29:15.857 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:15.859+0000 7f6aac70f700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6aa4112220 con 0x7f6aa4102110 2024-03-04T09:29:15.857 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:15.859+0000 7f6aac70f700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6aa4112520 con 0x7f6aa4102ad0 2024-03-04T09:29:15.858 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:15.859+0000 7f6aaa4ab700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6aa4102110 0x7f6aa419e3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:29:15.858 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:15.859+0000 7f6aaacac700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6aa41080f0 0x7f6aa41985a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:29:15.858 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:15.859+0000 7f6aaa4ab700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6aa4102110 0x7f6aa419e3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.52:46828/0 (socket says 172.21.15.52:46828) 2024-03-04T09:29:15.858 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:15.859+0000 7f6aaa4ab700 1 -- 172.21.15.52:0/3666048820 learned_addr learned my addr 172.21.15.52:0/3666048820 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-04T09:29:15.858 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:15.859+0000 7f6aa9caa700 1 --2- 172.21.15.52:0/3666048820 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6aa4102ad0 0x7f6aa419e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:29:15.858 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:15.859+0000 7f6aaa4ab700 1 -- 172.21.15.52:0/3666048820 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6aa4102ad0 msgr2=0x7f6aa419e920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:29:15.858 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:15.859+0000 7f6aaa4ab700 1 --2- 172.21.15.52:0/3666048820 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6aa4102ad0 0x7f6aa419e920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:29:15.858 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:15.859+0000 7f6aaa4ab700 1 -- 172.21.15.52:0/3666048820 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6aa41080f0 msgr2=0x7f6aa41985a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:29:15.858 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:15.859+0000 7f6aaa4ab700 1 --2- 172.21.15.52:0/3666048820 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6aa41080f0 0x7f6aa41985a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:29:15.858 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:15.859+0000 7f6aaa4ab700 1 -- 172.21.15.52:0/3666048820 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6aa4198d00 con 0x7f6aa4102110 2024-03-04T09:29:15.858 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:15.859+0000 7f6aaacac700 1 --2- 172.21.15.52:0/3666048820 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6aa41080f0 0x7f6aa41985a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-04T09:29:15.858 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:15.860+0000 7f6aaa4ab700 1 --2- 172.21.15.52:0/3666048820 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6aa4102110 0x7f6aa419e3e0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f6a94006010 tx=0x7f6a9400f690 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:29:15.858 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:15.860+0000 7f6a9b7fe700 1 -- 172.21.15.52:0/3666048820 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a94003680 con 0x7f6aa4102110 2024-03-04T09:29:15.858 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:15.860+0000 7f6aac70f700 1 -- 172.21.15.52:0/3666048820 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6aa4198f90 con 0x7f6aa4102110 2024-03-04T09:29:15.859 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:15.860+0000 7f6a9b7fe700 1 -- 172.21.15.52:0/3666048820 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6a940050d0 con 0x7f6aa4102110 2024-03-04T09:29:15.859 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:15.860+0000 7f6a9b7fe700 1 -- 172.21.15.52:0/3666048820 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a94003db0 con 0x7f6aa4102110 2024-03-04T09:29:15.859 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:15.860+0000 7f6aac70f700 1 -- 172.21.15.52:0/3666048820 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6aa41a52b0 con 0x7f6aa4102110 2024-03-04T09:29:15.859 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:15.861+0000 7f6aac70f700 1 -- 172.21.15.52:0/3666048820 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6aa4068d80 con 0x7f6aa4102110 2024-03-04T09:29:15.859 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:15.862+0000 7f6a9b7fe700 1 -- 172.21.15.52:0/3666048820 <== mon.2 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 17) v1 ==== 136807+0+0 (secure 0 0 0) 0x7f6a94005280 con 0x7f6aa4102110 2024-03-04T09:29:15.860 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:15.862+0000 7f6a9b7fe700 1 --2- 172.21.15.52:0/3666048820 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f6a900a2040 0x7f6a900a4500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:29:15.860 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:15.862+0000 7f6aa9caa700 1 --2- 172.21.15.52:0/3666048820 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f6a900a2040 0x7f6a900a4500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:29:15.860 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:15.862+0000 7f6a9b7fe700 1 -- 172.21.15.52:0/3666048820 <== mon.2 v2:172.21.15.106:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8802+0+0 (secure 0 0 0) 0x7f6a940ceea0 con 0x7f6aa4102110 2024-03-04T09:29:15.860 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:15.863+0000 7f6aa9caa700 1 --2- 172.21.15.52:0/3666048820 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f6a900a2040 0x7f6a900a4500 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f6aa4079780 tx=0x7f6aa0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:29:15.862 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:15.866+0000 7f6a9b7fe700 1 -- 172.21.15.52:0/3666048820 <== mon.2 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f6a94013070 con 0x7f6aa4102110 2024-03-04T09:29:16.068 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:16.071+0000 7f6aac70f700 1 -- 172.21.15.52:0/3666048820 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6aa402d9b0 con 0x7f6aa4102110 2024-03-04T09:29:16.069 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:16.073+0000 7f6a9b7fe700 1 -- 172.21.15.52:0/3666048820 <== mon.2 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) v1 ==== 74+0+24164 (secure 0 0 0) 0x7f6a9409bf60 con 0x7f6aa4102110 2024-03-04T09:29:16.070 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-03-04T09:29:16.070 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":89,"fsid":"4e059f08-da08-11ee-95c6-87774f69a715","created":"2024-03-04T09:20:46.104963+0000","modified":"2024-03-04T09:28:42.295274+0000","last_up_change":"2024-03-04T09:27:49.261365+0000","last_in_change":"2024-03-04T09:27:30.024964+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-04T09:24:01.742461+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-04T09:28:35.835600+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-04T09:28:36.453099+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-04T09:28:40.168831+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"583172be-602f-4bf4-bfcb-42663cf5b277","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6802","nonce":1371221397},{"type":"v1","addr":"172.21.15.52:6803","nonce":1371221397}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6804","nonce":1371221397},{"type":"v1","addr":"172.21.15.52:6805","nonce":1371221397}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6808","nonce":1371221397},{"type":"v1","addr":"172.21.15.52:6809","nonce":1371221397}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6806","nonce":1371221397},{"type":"v1","addr":"172.21.15.52:6807","nonce":1371221397}]},"public_addr":"172.21.15.52:6803/1371221397","cluster_addr":"172.21.15.52:6805/1371221397","heartbeat_back_addr":"172.21.15.52:6809/1371221397","heartbeat_front_addr":"172.21.15.52:6807/1371221397","state":["exists","up"]},{"osd":1,"uuid":"5ae0081c-a2f6-46eb-ace0-0368208377b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":2540865537},{"type":"v1","addr":"172.21.15.106:6801","nonce":2540865537}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":2540865537},{"type":"v1","addr":"172.21.15.106:6803","nonce":2540865537}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":2540865537},{"type":"v1","addr":"172.21.15.106:6807","nonce":2540865537}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":2540865537},{"type":"v1","addr":"172.21.15.106:6805","nonce":2540865537}]},"public_addr":"172.21.15.106:6801/2540865537","cluster_addr":"172.21.15.106:6803/2540865537","heartbeat_back_addr":"172.21.15.106:6807/2540865537","heartbeat_front_addr":"172.21.15.106:6805/2540865537","state":["exists","up"]},{"osd":2,"uuid":"1948bac7-2bb2-4218-8b93-4e4743a0de04","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":1377083164},{"type":"v1","addr":"172.21.15.191:6801","nonce":1377083164}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":1377083164},{"type":"v1","addr":"172.21.15.191:6803","nonce":1377083164}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":1377083164},{"type":"v1","addr":"172.21.15.191:6807","nonce":1377083164}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":1377083164},{"type":"v1","addr":"172.21.15.191:6805","nonce":1377083164}]},"public_addr":"172.21.15.191:6801/1377083164","cluster_addr":"172.21.15.191:6803/1377083164","heartbeat_back_addr":"172.21.15.191:6807/1377083164","heartbeat_front_addr":"172.21.15.191:6805/1377083164","state":["exists","up"]},{"osd":3,"uuid":"16c9e9fe-7167-4992-bf12-1a7398e0498a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6810","nonce":2388571810},{"type":"v1","addr":"172.21.15.52:6811","nonce":2388571810}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6812","nonce":2388571810},{"type":"v1","addr":"172.21.15.52:6813","nonce":2388571810}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6816","nonce":2388571810},{"type":"v1","addr":"172.21.15.52:6817","nonce":2388571810}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6814","nonce":2388571810},{"type":"v1","addr":"172.21.15.52:6815","nonce":2388571810}]},"public_addr":"172.21.15.52:6811/2388571810","cluster_addr":"172.21.15.52:6813/2388571810","heartbeat_back_addr":"172.21.15.52:6817/2388571810","heartbeat_front_addr":"172.21.15.52:6815/2388571810","state":["exists","up"]},{"osd":4,"uuid":"c8ef48c7-7c78-44d3-a9e5-99ec750abd3d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6808","nonce":3465868015},{"type":"v1","addr":"172.21.15.106:6809","nonce":3465868015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6810","nonce":3465868015},{"type":"v1","addr":"172.21.15.106:6811","nonce":3465868015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6814","nonce":3465868015},{"type":"v1","addr":"172.21.15.106:6815","nonce":3465868015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6812","nonce":3465868015},{"type":"v1","addr":"172.21.15.106:6813","nonce":3465868015}]},"public_addr":"172.21.15.106:6809/3465868015","cluster_addr":"172.21.15.106:6811/3465868015","heartbeat_back_addr":"172.21.15.106:6815/3465868015","heartbeat_front_addr":"172.21.15.106:6813/3465868015","state":["exists","up"]},{"osd":5,"uuid":"9e41824a-c5a2-4f88-be50-2ba1dceb3220","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":1485092643},{"type":"v1","addr":"172.21.15.191:6809","nonce":1485092643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":1485092643},{"type":"v1","addr":"172.21.15.191:6811","nonce":1485092643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":1485092643},{"type":"v1","addr":"172.21.15.191:6815","nonce":1485092643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":1485092643},{"type":"v1","addr":"172.21.15.191:6813","nonce":1485092643}]},"public_addr":"172.21.15.191:6809/1485092643","cluster_addr":"172.21.15.191:6811/1485092643","heartbeat_back_addr":"172.21.15.191:6815/1485092643","heartbeat_front_addr":"172.21.15.191:6813/1485092643","state":["exists","up"]},{"osd":6,"uuid":"b35dccee-ec16-452e-ac69-3ac2fb7b27fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6818","nonce":1989797856},{"type":"v1","addr":"172.21.15.52:6819","nonce":1989797856}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6820","nonce":1989797856},{"type":"v1","addr":"172.21.15.52:6821","nonce":1989797856}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6824","nonce":1989797856},{"type":"v1","addr":"172.21.15.52:6825","nonce":1989797856}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6822","nonce":1989797856},{"type":"v1","addr":"172.21.15.52:6823","nonce":1989797856}]},"public_addr":"172.21.15.52:6819/1989797856","cluster_addr":"172.21.15.52:6821/1989797856","heartbeat_back_addr":"172.21.15.52:6825/1989797856","heartbeat_front_addr":"172.21.15.52:6823/1989797856","state":["exists","up"]},{"osd":7,"uuid":"71303e3d-14ec-4cbd-b19c-67645a1de3c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6816","nonce":1450870329},{"type":"v1","addr":"172.21.15.106:6817","nonce":1450870329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":1450870329},{"type":"v1","addr":"172.21.15.106:6819","nonce":1450870329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":1450870329},{"type":"v1","addr":"172.21.15.106:6823","nonce":1450870329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6820","nonce":1450870329},{"type":"v1","addr":"172.21.15.106:6821","nonce":1450870329}]},"public_addr":"172.21.15.106:6817/1450870329","cluster_addr":"172.21.15.106:6819/1450870329","heartbeat_back_addr":"172.21.15.106:6823/1450870329","heartbeat_front_addr":"172.21.15.106:6821/1450870329","state":["exists","up"]},{"osd":8,"uuid":"6f5786c4-4402-435a-bdad-adff46fe9473","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":1114119072},{"type":"v1","addr":"172.21.15.191:6817","nonce":1114119072}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6818","nonce":1114119072},{"type":"v1","addr":"172.21.15.191:6819","nonce":1114119072}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":1114119072},{"type":"v1","addr":"172.21.15.191:6823","nonce":1114119072}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":1114119072},{"type":"v1","addr":"172.21.15.191:6821","nonce":1114119072}]},"public_addr":"172.21.15.191:6817/1114119072","cluster_addr":"172.21.15.191:6819/1114119072","heartbeat_back_addr":"172.21.15.191:6823/1114119072","heartbeat_front_addr":"172.21.15.191:6821/1114119072","state":["exists","up"]},{"osd":9,"uuid":"062bbaa5-1227-4684-b859-450f5594a90c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6826","nonce":3485821914},{"type":"v1","addr":"172.21.15.52:6827","nonce":3485821914}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6828","nonce":3485821914},{"type":"v1","addr":"172.21.15.52:6829","nonce":3485821914}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6832","nonce":3485821914},{"type":"v1","addr":"172.21.15.52:6833","nonce":3485821914}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6830","nonce":3485821914},{"type":"v1","addr":"172.21.15.52:6831","nonce":3485821914}]},"public_addr":"172.21.15.52:6827/3485821914","cluster_addr":"172.21.15.52:6829/3485821914","heartbeat_back_addr":"172.21.15.52:6833/3485821914","heartbeat_front_addr":"172.21.15.52:6831/3485821914","state":["exists","up"]},{"osd":10,"uuid":"1a285a5a-f793-48da-a8eb-5326d9528cd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":3815586576},{"type":"v1","addr":"172.21.15.106:6825","nonce":3815586576}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6826","nonce":3815586576},{"type":"v1","addr":"172.21.15.106:6827","nonce":3815586576}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":3815586576},{"type":"v1","addr":"172.21.15.106:6831","nonce":3815586576}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":3815586576},{"type":"v1","addr":"172.21.15.106:6829","nonce":3815586576}]},"public_addr":"172.21.15.106:6825/3815586576","cluster_addr":"172.21.15.106:6827/3815586576","heartbeat_back_addr":"172.21.15.106:6831/3815586576","heartbeat_front_addr":"172.21.15.106:6829/3815586576","state":["exists","up"]},{"osd":11,"uuid":"1f8b9a43-57fa-41f4-9969-3b803a74bf45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":2009632843},{"type":"v1","addr":"172.21.15.191:6825","nonce":2009632843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":2009632843},{"type":"v1","addr":"172.21.15.191:6827","nonce":2009632843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":2009632843},{"type":"v1","addr":"172.21.15.191:6831","nonce":2009632843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":2009632843},{"type":"v1","addr":"172.21.15.191:6829","nonce":2009632843}]},"public_addr":"172.21.15.191:6825/2009632843","cluster_addr":"172.21.15.191:6827/2009632843","heartbeat_back_addr":"172.21.15.191:6831/2009632843","heartbeat_front_addr":"172.21.15.191:6829/2009632843","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T09:23:10.894763+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T09:23:34.420055+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T09:23:58.288324+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T09:24:23.673457+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T09:24:48.208561+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T09:25:13.405473+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T09:25:38.670915+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T09:26:04.451257+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T09:26:30.281380+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T09:26:56.543143+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T09:27:22.412930+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T09:27:48.003866+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.52:0/4082645718":"2024-03-05T09:21:39.401935+0000","172.21.15.52:0/1271949988":"2024-03-05T09:21:39.401935+0000","172.21.15.52:6801/2379361333":"2024-03-05T09:21:12.760938+0000","172.21.15.52:0/497075075":"2024-03-05T09:21:12.760938+0000","172.21.15.52:6800/2379361333":"2024-03-05T09:21:12.760938+0000","172.21.15.52:0/568242577":"2024-03-05T09:21:12.760938+0000","172.21.15.52:6801/2425730636":"2024-03-05T09:21:39.401935+0000","172.21.15.52:6800/2425730636":"2024-03-05T09:21:39.401935+0000","172.21.15.52:0/2743084243":"2024-03-05T09:21:39.401935+0000","172.21.15.52:0/1035878034":"2024-03-05T09:21:12.760938+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-04T09:29:16.072 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:16.075+0000 7f6aac70f700 1 -- 172.21.15.52:0/3666048820 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f6a900a2040 msgr2=0x7f6a900a4500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:29:16.072 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:16.075+0000 7f6aac70f700 1 --2- 172.21.15.52:0/3666048820 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f6a900a2040 0x7f6a900a4500 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f6aa4079780 tx=0x7f6aa0009260 comp rx=0 tx=0).stop 2024-03-04T09:29:16.072 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:16.075+0000 7f6aac70f700 1 -- 172.21.15.52:0/3666048820 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6aa4102110 msgr2=0x7f6aa419e3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:29:16.072 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:16.075+0000 7f6aac70f700 1 --2- 172.21.15.52:0/3666048820 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6aa4102110 0x7f6aa419e3e0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f6a94006010 tx=0x7f6a9400f690 comp rx=0 tx=0).stop 2024-03-04T09:29:16.072 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:16.075+0000 7f6aac70f700 1 -- 172.21.15.52:0/3666048820 shutdown_connections 2024-03-04T09:29:16.072 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:16.075+0000 7f6aac70f700 1 --2- 172.21.15.52:0/3666048820 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f6a900a2040 0x7f6a900a4500 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:29:16.073 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:16.075+0000 7f6aac70f700 1 --2- 172.21.15.52:0/3666048820 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6aa41080f0 0x7f6aa41985a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:29:16.073 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:16.075+0000 7f6aac70f700 1 --2- 172.21.15.52:0/3666048820 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6aa4102110 0x7f6aa419e3e0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:29:16.073 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:16.075+0000 7f6aac70f700 1 --2- 172.21.15.52:0/3666048820 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6aa4102ad0 0x7f6aa419e920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:29:16.073 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:16.075+0000 7f6aac70f700 1 -- 172.21.15.52:0/3666048820 >> 172.21.15.52:0/3666048820 conn(0x7f6aa40791f0 msgr2=0x7f6aa4100cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:29:16.073 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:16.075+0000 7f6aac70f700 1 -- 172.21.15.52:0/3666048820 shutdown_connections 2024-03-04T09:29:16.073 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:16.075+0000 7f6aac70f700 1 -- 172.21.15.52:0/3666048820 wait complete. 2024-03-04T09:29:16.852 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-03-04T09:29:16.853 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd shell --fsid 4e059f08-da08-11ee-95c6-87774f69a715 -- ceph osd dump --format=json 2024-03-04T09:29:17.084 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:29:17 smithi052 ceph-mon[32473]: pgmap v307: 129 pgs: 129 active+clean; 640 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 7.8 KiB/s wr, 15 op/s 2024-03-04T09:29:17.085 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:29:17 smithi052 ceph-mon[32473]: from='client.? 172.21.15.52:0/3666048820' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-04T09:29:17.105 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/4e059f08-da08-11ee-95c6-87774f69a715/mon.a/config 2024-03-04T09:29:17.336 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:29:17 smithi191 ceph-mon[36816]: pgmap v307: 129 pgs: 129 active+clean; 640 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 7.8 KiB/s wr, 15 op/s 2024-03-04T09:29:17.336 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:29:17 smithi191 ceph-mon[36816]: from='client.? 172.21.15.52:0/3666048820' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-04T09:29:17.381 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:29:17 smithi106 ceph-mon[36484]: pgmap v307: 129 pgs: 129 active+clean; 640 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 7.8 KiB/s wr, 15 op/s 2024-03-04T09:29:17.381 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:29:17 smithi106 ceph-mon[36484]: from='client.? 172.21.15.52:0/3666048820' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-04T09:29:19.066 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:19.067+0000 7f4ffec09700 1 -- 172.21.15.52:0/2863510295 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4ff8102d20 msgr2=0x7f4ff8106ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:29:19.066 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:19.067+0000 7f4ffec09700 1 --2- 172.21.15.52:0/2863510295 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4ff8102d20 0x7f4ff8106ce0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f4fe40099c0 tx=0x7f4fe4009cd0 comp rx=0 tx=0).stop 2024-03-04T09:29:19.066 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:19.067+0000 7f4ffec09700 1 -- 172.21.15.52:0/2863510295 shutdown_connections 2024-03-04T09:29:19.067 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:19.067+0000 7f4ffec09700 1 --2- 172.21.15.52:0/2863510295 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4ff8102370 0x7f4ff8102750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:29:19.067 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:19.067+0000 7f4ffec09700 1 --2- 172.21.15.52:0/2863510295 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4ff81073c0 0x7f4ff810ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:29:19.067 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:19.067+0000 7f4ffec09700 1 --2- 172.21.15.52:0/2863510295 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4ff8102d20 0x7f4ff8106ce0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:29:19.067 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:19.067+0000 7f4ffec09700 1 -- 172.21.15.52:0/2863510295 >> 172.21.15.52:0/2863510295 conn(0x7f4ff8079220 msgr2=0x7f4ff8079630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:29:19.067 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:19.067+0000 7f4ffec09700 1 -- 172.21.15.52:0/2863510295 shutdown_connections 2024-03-04T09:29:19.067 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:19.067+0000 7f4ffec09700 1 -- 172.21.15.52:0/2863510295 wait complete. 2024-03-04T09:29:19.067 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:19.068+0000 7f4ffec09700 1 Processor -- start 2024-03-04T09:29:19.067 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:19.068+0000 7f4ffec09700 1 -- start start 2024-03-04T09:29:19.067 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:19.068+0000 7f4ffec09700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4ff8102370 0x7f4ff80743c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:29:19.067 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:19.068+0000 7f4ffec09700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4ff8102d20 0x7f4ff8074900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:29:19.067 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:19.068+0000 7f4ffec09700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4ff81073c0 0x7f4ff8074e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:29:19.068 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:19.068+0000 7f4ffec09700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ff8112380 con 0x7f4ff8102d20 2024-03-04T09:29:19.068 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:19.068+0000 7f4ffec09700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ff8112200 con 0x7f4ff8102370 2024-03-04T09:29:19.068 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:19.068+0000 7f4ffec09700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ff8112500 con 0x7f4ff81073c0 2024-03-04T09:29:19.068 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:19.068+0000 7f4ff7fff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4ff8102d20 0x7f4ff8074900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:29:19.068 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:19.068+0000 7f4ff7fff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4ff8102d20 0x7f4ff8074900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:42508/0 (socket says 172.21.15.52:42508) 2024-03-04T09:29:19.068 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:19.068+0000 7f4ff7fff700 1 -- 172.21.15.52:0/4284242295 learned_addr learned my addr 172.21.15.52:0/4284242295 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-04T09:29:19.068 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:19.068+0000 7f4ffc9a5700 1 --2- 172.21.15.52:0/4284242295 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4ff8102370 0x7f4ff80743c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:29:19.068 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:19.068+0000 7f4ffd1a6700 1 --2- 172.21.15.52:0/4284242295 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4ff81073c0 0x7f4ff8074e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:29:19.069 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:19.069+0000 7f4ff7fff700 1 -- 172.21.15.52:0/4284242295 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4ff81073c0 msgr2=0x7f4ff8074e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:29:19.069 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:19.069+0000 7f4ff7fff700 1 --2- 172.21.15.52:0/4284242295 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4ff81073c0 0x7f4ff8074e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:29:19.069 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:19.069+0000 7f4ff7fff700 1 -- 172.21.15.52:0/4284242295 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4ff8102370 msgr2=0x7f4ff80743c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:29:19.069 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:19.069+0000 7f4ff7fff700 1 --2- 172.21.15.52:0/4284242295 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4ff8102370 0x7f4ff80743c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:29:19.069 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:19.069+0000 7f4ff7fff700 1 -- 172.21.15.52:0/4284242295 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ff81a6da0 con 0x7f4ff8102d20 2024-03-04T09:29:19.069 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:19.069+0000 7f4ffc9a5700 1 --2- 172.21.15.52:0/4284242295 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4ff8102370 0x7f4ff80743c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-04T09:29:19.069 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:19.069+0000 7f4ff7fff700 1 --2- 172.21.15.52:0/4284242295 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4ff8102d20 0x7f4ff8074900 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f4fe40059d0 tx=0x7f4fe40038b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:29:19.069 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:19.069+0000 7f4ff5ffb700 1 -- 172.21.15.52:0/4284242295 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fe4027070 con 0x7f4ff8102d20 2024-03-04T09:29:19.069 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:19.069+0000 7f4ff5ffb700 1 -- 172.21.15.52:0/4284242295 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4fe4003e40 con 0x7f4ff8102d20 2024-03-04T09:29:19.069 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:19.069+0000 7f4ffec09700 1 -- 172.21.15.52:0/4284242295 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ff81a7030 con 0x7f4ff8102d20 2024-03-04T09:29:19.070 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:19.069+0000 7f4ff5ffb700 1 -- 172.21.15.52:0/4284242295 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fe4022c70 con 0x7f4ff8102d20 2024-03-04T09:29:19.070 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:19.069+0000 7f4ffec09700 1 -- 172.21.15.52:0/4284242295 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ff81a7480 con 0x7f4ff8102d20 2024-03-04T09:29:19.071 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:19.070+0000 7f4ffec09700 1 -- 172.21.15.52:0/4284242295 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ff8068d80 con 0x7f4ff8102d20 2024-03-04T09:29:19.071 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:19.072+0000 7f4ff5ffb700 1 -- 172.21.15.52:0/4284242295 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 17) v1 ==== 136807+0+0 (secure 0 0 0) 0x7f4fe402a070 con 0x7f4ff8102d20 2024-03-04T09:29:19.071 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:19.073+0000 7f4ff5ffb700 1 --2- 172.21.15.52:0/4284242295 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f4fe80a1ea0 0x7f4fe80a4360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:29:19.071 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:19.073+0000 7f4ffc9a5700 1 --2- 172.21.15.52:0/4284242295 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f4fe80a1ea0 0x7f4fe80a4360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:29:19.071 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:19.073+0000 7f4ff5ffb700 1 -- 172.21.15.52:0/4284242295 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8802+0+0 (secure 0 0 0) 0x7f4fe40cf860 con 0x7f4ff8102d20 2024-03-04T09:29:19.071 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:19.073+0000 7f4ffc9a5700 1 --2- 172.21.15.52:0/4284242295 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f4fe80a1ea0 0x7f4fe80a4360 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f4fec007fd0 tx=0x7f4fec0073d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:29:19.073 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:19.076+0000 7f4ff5ffb700 1 -- 172.21.15.52:0/4284242295 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f4fe4031020 con 0x7f4ff8102d20 2024-03-04T09:29:19.275 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:19.278+0000 7f4ffec09700 1 -- 172.21.15.52:0/4284242295 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4ff802d9b0 con 0x7f4ff8102d20 2024-03-04T09:29:19.276 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:29:19 smithi052 ceph-mon[32473]: pgmap v308: 129 pgs: 129 active+clean; 640 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 5.8 KiB/s wr, 13 op/s 2024-03-04T09:29:19.277 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:19.280+0000 7f4ff5ffb700 1 -- 172.21.15.52:0/4284242295 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) v1 ==== 74+0+24164 (secure 0 0 0) 0x7f4fe4013070 con 0x7f4ff8102d20 2024-03-04T09:29:19.277 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-03-04T09:29:19.277 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":89,"fsid":"4e059f08-da08-11ee-95c6-87774f69a715","created":"2024-03-04T09:20:46.104963+0000","modified":"2024-03-04T09:28:42.295274+0000","last_up_change":"2024-03-04T09:27:49.261365+0000","last_in_change":"2024-03-04T09:27:30.024964+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-04T09:24:01.742461+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-04T09:28:35.835600+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-04T09:28:36.453099+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-04T09:28:40.168831+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"583172be-602f-4bf4-bfcb-42663cf5b277","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6802","nonce":1371221397},{"type":"v1","addr":"172.21.15.52:6803","nonce":1371221397}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6804","nonce":1371221397},{"type":"v1","addr":"172.21.15.52:6805","nonce":1371221397}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6808","nonce":1371221397},{"type":"v1","addr":"172.21.15.52:6809","nonce":1371221397}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6806","nonce":1371221397},{"type":"v1","addr":"172.21.15.52:6807","nonce":1371221397}]},"public_addr":"172.21.15.52:6803/1371221397","cluster_addr":"172.21.15.52:6805/1371221397","heartbeat_back_addr":"172.21.15.52:6809/1371221397","heartbeat_front_addr":"172.21.15.52:6807/1371221397","state":["exists","up"]},{"osd":1,"uuid":"5ae0081c-a2f6-46eb-ace0-0368208377b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":2540865537},{"type":"v1","addr":"172.21.15.106:6801","nonce":2540865537}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":2540865537},{"type":"v1","addr":"172.21.15.106:6803","nonce":2540865537}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":2540865537},{"type":"v1","addr":"172.21.15.106:6807","nonce":2540865537}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":2540865537},{"type":"v1","addr":"172.21.15.106:6805","nonce":2540865537}]},"public_addr":"172.21.15.106:6801/2540865537","cluster_addr":"172.21.15.106:6803/2540865537","heartbeat_back_addr":"172.21.15.106:6807/2540865537","heartbeat_front_addr":"172.21.15.106:6805/2540865537","state":["exists","up"]},{"osd":2,"uuid":"1948bac7-2bb2-4218-8b93-4e4743a0de04","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":1377083164},{"type":"v1","addr":"172.21.15.191:6801","nonce":1377083164}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":1377083164},{"type":"v1","addr":"172.21.15.191:6803","nonce":1377083164}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":1377083164},{"type":"v1","addr":"172.21.15.191:6807","nonce":1377083164}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":1377083164},{"type":"v1","addr":"172.21.15.191:6805","nonce":1377083164}]},"public_addr":"172.21.15.191:6801/1377083164","cluster_addr":"172.21.15.191:6803/1377083164","heartbeat_back_addr":"172.21.15.191:6807/1377083164","heartbeat_front_addr":"172.21.15.191:6805/1377083164","state":["exists","up"]},{"osd":3,"uuid":"16c9e9fe-7167-4992-bf12-1a7398e0498a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6810","nonce":2388571810},{"type":"v1","addr":"172.21.15.52:6811","nonce":2388571810}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6812","nonce":2388571810},{"type":"v1","addr":"172.21.15.52:6813","nonce":2388571810}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6816","nonce":2388571810},{"type":"v1","addr":"172.21.15.52:6817","nonce":2388571810}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6814","nonce":2388571810},{"type":"v1","addr":"172.21.15.52:6815","nonce":2388571810}]},"public_addr":"172.21.15.52:6811/2388571810","cluster_addr":"172.21.15.52:6813/2388571810","heartbeat_back_addr":"172.21.15.52:6817/2388571810","heartbeat_front_addr":"172.21.15.52:6815/2388571810","state":["exists","up"]},{"osd":4,"uuid":"c8ef48c7-7c78-44d3-a9e5-99ec750abd3d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6808","nonce":3465868015},{"type":"v1","addr":"172.21.15.106:6809","nonce":3465868015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6810","nonce":3465868015},{"type":"v1","addr":"172.21.15.106:6811","nonce":3465868015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6814","nonce":3465868015},{"type":"v1","addr":"172.21.15.106:6815","nonce":3465868015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6812","nonce":3465868015},{"type":"v1","addr":"172.21.15.106:6813","nonce":3465868015}]},"public_addr":"172.21.15.106:6809/3465868015","cluster_addr":"172.21.15.106:6811/3465868015","heartbeat_back_addr":"172.21.15.106:6815/3465868015","heartbeat_front_addr":"172.21.15.106:6813/3465868015","state":["exists","up"]},{"osd":5,"uuid":"9e41824a-c5a2-4f88-be50-2ba1dceb3220","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":1485092643},{"type":"v1","addr":"172.21.15.191:6809","nonce":1485092643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":1485092643},{"type":"v1","addr":"172.21.15.191:6811","nonce":1485092643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":1485092643},{"type":"v1","addr":"172.21.15.191:6815","nonce":1485092643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":1485092643},{"type":"v1","addr":"172.21.15.191:6813","nonce":1485092643}]},"public_addr":"172.21.15.191:6809/1485092643","cluster_addr":"172.21.15.191:6811/1485092643","heartbeat_back_addr":"172.21.15.191:6815/1485092643","heartbeat_front_addr":"172.21.15.191:6813/1485092643","state":["exists","up"]},{"osd":6,"uuid":"b35dccee-ec16-452e-ac69-3ac2fb7b27fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6818","nonce":1989797856},{"type":"v1","addr":"172.21.15.52:6819","nonce":1989797856}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6820","nonce":1989797856},{"type":"v1","addr":"172.21.15.52:6821","nonce":1989797856}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6824","nonce":1989797856},{"type":"v1","addr":"172.21.15.52:6825","nonce":1989797856}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6822","nonce":1989797856},{"type":"v1","addr":"172.21.15.52:6823","nonce":1989797856}]},"public_addr":"172.21.15.52:6819/1989797856","cluster_addr":"172.21.15.52:6821/1989797856","heartbeat_back_addr":"172.21.15.52:6825/1989797856","heartbeat_front_addr":"172.21.15.52:6823/1989797856","state":["exists","up"]},{"osd":7,"uuid":"71303e3d-14ec-4cbd-b19c-67645a1de3c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6816","nonce":1450870329},{"type":"v1","addr":"172.21.15.106:6817","nonce":1450870329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":1450870329},{"type":"v1","addr":"172.21.15.106:6819","nonce":1450870329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":1450870329},{"type":"v1","addr":"172.21.15.106:6823","nonce":1450870329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6820","nonce":1450870329},{"type":"v1","addr":"172.21.15.106:6821","nonce":1450870329}]},"public_addr":"172.21.15.106:6817/1450870329","cluster_addr":"172.21.15.106:6819/1450870329","heartbeat_back_addr":"172.21.15.106:6823/1450870329","heartbeat_front_addr":"172.21.15.106:6821/1450870329","state":["exists","up"]},{"osd":8,"uuid":"6f5786c4-4402-435a-bdad-adff46fe9473","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":1114119072},{"type":"v1","addr":"172.21.15.191:6817","nonce":1114119072}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6818","nonce":1114119072},{"type":"v1","addr":"172.21.15.191:6819","nonce":1114119072}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":1114119072},{"type":"v1","addr":"172.21.15.191:6823","nonce":1114119072}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":1114119072},{"type":"v1","addr":"172.21.15.191:6821","nonce":1114119072}]},"public_addr":"172.21.15.191:6817/1114119072","cluster_addr":"172.21.15.191:6819/1114119072","heartbeat_back_addr":"172.21.15.191:6823/1114119072","heartbeat_front_addr":"172.21.15.191:6821/1114119072","state":["exists","up"]},{"osd":9,"uuid":"062bbaa5-1227-4684-b859-450f5594a90c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6826","nonce":3485821914},{"type":"v1","addr":"172.21.15.52:6827","nonce":3485821914}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6828","nonce":3485821914},{"type":"v1","addr":"172.21.15.52:6829","nonce":3485821914}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6832","nonce":3485821914},{"type":"v1","addr":"172.21.15.52:6833","nonce":3485821914}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6830","nonce":3485821914},{"type":"v1","addr":"172.21.15.52:6831","nonce":3485821914}]},"public_addr":"172.21.15.52:6827/3485821914","cluster_addr":"172.21.15.52:6829/3485821914","heartbeat_back_addr":"172.21.15.52:6833/3485821914","heartbeat_front_addr":"172.21.15.52:6831/3485821914","state":["exists","up"]},{"osd":10,"uuid":"1a285a5a-f793-48da-a8eb-5326d9528cd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":3815586576},{"type":"v1","addr":"172.21.15.106:6825","nonce":3815586576}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6826","nonce":3815586576},{"type":"v1","addr":"172.21.15.106:6827","nonce":3815586576}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":3815586576},{"type":"v1","addr":"172.21.15.106:6831","nonce":3815586576}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":3815586576},{"type":"v1","addr":"172.21.15.106:6829","nonce":3815586576}]},"public_addr":"172.21.15.106:6825/3815586576","cluster_addr":"172.21.15.106:6827/3815586576","heartbeat_back_addr":"172.21.15.106:6831/3815586576","heartbeat_front_addr":"172.21.15.106:6829/3815586576","state":["exists","up"]},{"osd":11,"uuid":"1f8b9a43-57fa-41f4-9969-3b803a74bf45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":2009632843},{"type":"v1","addr":"172.21.15.191:6825","nonce":2009632843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":2009632843},{"type":"v1","addr":"172.21.15.191:6827","nonce":2009632843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":2009632843},{"type":"v1","addr":"172.21.15.191:6831","nonce":2009632843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":2009632843},{"type":"v1","addr":"172.21.15.191:6829","nonce":2009632843}]},"public_addr":"172.21.15.191:6825/2009632843","cluster_addr":"172.21.15.191:6827/2009632843","heartbeat_back_addr":"172.21.15.191:6831/2009632843","heartbeat_front_addr":"172.21.15.191:6829/2009632843","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T09:23:10.894763+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T09:23:34.420055+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T09:23:58.288324+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T09:24:23.673457+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T09:24:48.208561+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T09:25:13.405473+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T09:25:38.670915+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T09:26:04.451257+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T09:26:30.281380+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T09:26:56.543143+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T09:27:22.412930+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T09:27:48.003866+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.52:0/4082645718":"2024-03-05T09:21:39.401935+0000","172.21.15.52:0/1271949988":"2024-03-05T09:21:39.401935+0000","172.21.15.52:6801/2379361333":"2024-03-05T09:21:12.760938+0000","172.21.15.52:0/497075075":"2024-03-05T09:21:12.760938+0000","172.21.15.52:6800/2379361333":"2024-03-05T09:21:12.760938+0000","172.21.15.52:0/568242577":"2024-03-05T09:21:12.760938+0000","172.21.15.52:6801/2425730636":"2024-03-05T09:21:39.401935+0000","172.21.15.52:6800/2425730636":"2024-03-05T09:21:39.401935+0000","172.21.15.52:0/2743084243":"2024-03-05T09:21:39.401935+0000","172.21.15.52:0/1035878034":"2024-03-05T09:21:12.760938+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-04T09:29:19.279 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:19.282+0000 7f4ffec09700 1 -- 172.21.15.52:0/4284242295 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f4fe80a1ea0 msgr2=0x7f4fe80a4360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:29:19.279 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:19.282+0000 7f4ffec09700 1 --2- 172.21.15.52:0/4284242295 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f4fe80a1ea0 0x7f4fe80a4360 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f4fec007fd0 tx=0x7f4fec0073d0 comp rx=0 tx=0).stop 2024-03-04T09:29:19.279 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:19.282+0000 7f4ffec09700 1 -- 172.21.15.52:0/4284242295 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4ff8102d20 msgr2=0x7f4ff8074900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:29:19.279 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:19.282+0000 7f4ffec09700 1 --2- 172.21.15.52:0/4284242295 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4ff8102d20 0x7f4ff8074900 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f4fe40059d0 tx=0x7f4fe40038b0 comp rx=0 tx=0).stop 2024-03-04T09:29:19.279 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:19.282+0000 7f4ffec09700 1 -- 172.21.15.52:0/4284242295 shutdown_connections 2024-03-04T09:29:19.279 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:19.282+0000 7f4ffec09700 1 --2- 172.21.15.52:0/4284242295 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f4fe80a1ea0 0x7f4fe80a4360 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:29:19.280 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:19.282+0000 7f4ffec09700 1 --2- 172.21.15.52:0/4284242295 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4ff81073c0 0x7f4ff8074e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:29:19.280 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:19.282+0000 7f4ffec09700 1 --2- 172.21.15.52:0/4284242295 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4ff8102370 0x7f4ff80743c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:29:19.280 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:19.282+0000 7f4ffec09700 1 --2- 172.21.15.52:0/4284242295 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4ff8102d20 0x7f4ff8074900 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:29:19.280 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:19.282+0000 7f4ffec09700 1 -- 172.21.15.52:0/4284242295 >> 172.21.15.52:0/4284242295 conn(0x7f4ff8079220 msgr2=0x7f4ff81001c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:29:19.280 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:19.282+0000 7f4ffec09700 1 -- 172.21.15.52:0/4284242295 shutdown_connections 2024-03-04T09:29:19.280 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:19.282+0000 7f4ffec09700 1 -- 172.21.15.52:0/4284242295 wait complete. 2024-03-04T09:29:19.336 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:29:19 smithi191 ceph-mon[36816]: pgmap v308: 129 pgs: 129 active+clean; 640 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 5.8 KiB/s wr, 13 op/s 2024-03-04T09:29:19.381 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:29:19 smithi106 ceph-mon[36484]: pgmap v308: 129 pgs: 129 active+clean; 640 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 5.8 KiB/s wr, 13 op/s 2024-03-04T09:29:20.153 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd shell --fsid 4e059f08-da08-11ee-95c6-87774f69a715 -- ceph tell osd.0 flush_pg_stats 2024-03-04T09:29:20.154 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd shell --fsid 4e059f08-da08-11ee-95c6-87774f69a715 -- ceph tell osd.1 flush_pg_stats 2024-03-04T09:29:20.154 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd shell --fsid 4e059f08-da08-11ee-95c6-87774f69a715 -- ceph tell osd.2 flush_pg_stats 2024-03-04T09:29:20.154 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd shell --fsid 4e059f08-da08-11ee-95c6-87774f69a715 -- ceph tell osd.3 flush_pg_stats 2024-03-04T09:29:20.155 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd shell --fsid 4e059f08-da08-11ee-95c6-87774f69a715 -- ceph tell osd.4 flush_pg_stats 2024-03-04T09:29:20.155 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd shell --fsid 4e059f08-da08-11ee-95c6-87774f69a715 -- ceph tell osd.5 flush_pg_stats 2024-03-04T09:29:20.156 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd shell --fsid 4e059f08-da08-11ee-95c6-87774f69a715 -- ceph tell osd.6 flush_pg_stats 2024-03-04T09:29:20.156 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd shell --fsid 4e059f08-da08-11ee-95c6-87774f69a715 -- ceph tell osd.7 flush_pg_stats 2024-03-04T09:29:20.156 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd shell --fsid 4e059f08-da08-11ee-95c6-87774f69a715 -- ceph tell osd.8 flush_pg_stats 2024-03-04T09:29:20.157 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd shell --fsid 4e059f08-da08-11ee-95c6-87774f69a715 -- ceph tell osd.9 flush_pg_stats 2024-03-04T09:29:20.157 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd shell --fsid 4e059f08-da08-11ee-95c6-87774f69a715 -- ceph tell osd.10 flush_pg_stats 2024-03-04T09:29:20.157 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd shell --fsid 4e059f08-da08-11ee-95c6-87774f69a715 -- ceph tell osd.11 flush_pg_stats 2024-03-04T09:29:20.293 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:29:20 smithi052 ceph-mon[32473]: from='client.? 172.21.15.52:0/4284242295' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-04T09:29:20.336 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:29:20 smithi191 ceph-mon[36816]: from='client.? 172.21.15.52:0/4284242295' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-04T09:29:20.381 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:29:20 smithi106 ceph-mon[36484]: from='client.? 172.21.15.52:0/4284242295' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-04T09:29:20.900 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/4e059f08-da08-11ee-95c6-87774f69a715/mon.a/config 2024-03-04T09:29:20.901 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/4e059f08-da08-11ee-95c6-87774f69a715/mon.a/config 2024-03-04T09:29:20.901 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/4e059f08-da08-11ee-95c6-87774f69a715/mon.a/config 2024-03-04T09:29:20.901 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/4e059f08-da08-11ee-95c6-87774f69a715/mon.a/config 2024-03-04T09:29:20.902 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/4e059f08-da08-11ee-95c6-87774f69a715/mon.a/config 2024-03-04T09:29:20.902 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/4e059f08-da08-11ee-95c6-87774f69a715/mon.a/config 2024-03-04T09:29:20.910 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/4e059f08-da08-11ee-95c6-87774f69a715/mon.a/config 2024-03-04T09:29:20.974 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/4e059f08-da08-11ee-95c6-87774f69a715/mon.a/config 2024-03-04T09:29:21.328 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:29:21 smithi052 ceph-mon[32473]: pgmap v309: 129 pgs: 129 active+clean; 640 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 3.7 KiB/s wr, 7 op/s 2024-03-04T09:29:21.336 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:29:21 smithi191 ceph-mon[36816]: pgmap v309: 129 pgs: 129 active+clean; 640 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 3.7 KiB/s wr, 7 op/s 2024-03-04T09:29:21.381 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:29:21 smithi106 ceph-mon[36484]: pgmap v309: 129 pgs: 129 active+clean; 640 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 3.7 KiB/s wr, 7 op/s 2024-03-04T09:29:22.311 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/4e059f08-da08-11ee-95c6-87774f69a715/mon.a/config 2024-03-04T09:29:22.312 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/4e059f08-da08-11ee-95c6-87774f69a715/mon.a/config 2024-03-04T09:29:22.312 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/4e059f08-da08-11ee-95c6-87774f69a715/mon.a/config 2024-03-04T09:29:22.312 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/4e059f08-da08-11ee-95c6-87774f69a715/mon.a/config 2024-03-04T09:29:23.334 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:29:23 smithi052 ceph-mon[32473]: pgmap v310: 129 pgs: 129 active+clean; 640 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 3.6 KiB/s wr, 9 op/s 2024-03-04T09:29:23.336 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:29:23 smithi191 ceph-mon[36816]: pgmap v310: 129 pgs: 129 active+clean; 640 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 3.6 KiB/s wr, 9 op/s 2024-03-04T09:29:23.381 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:29:23 smithi106 ceph-mon[36484]: pgmap v310: 129 pgs: 129 active+clean; 640 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 3.6 KiB/s wr, 9 op/s 2024-03-04T09:29:25.301 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:29:25 smithi052 ceph-mon[32473]: pgmap v311: 129 pgs: 129 active+clean; 640 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.6 KiB/s wr, 6 op/s 2024-03-04T09:29:25.336 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:29:25 smithi191 ceph-mon[36816]: pgmap v311: 129 pgs: 129 active+clean; 640 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.6 KiB/s wr, 6 op/s 2024-03-04T09:29:25.381 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:29:25 smithi106 ceph-mon[36484]: pgmap v311: 129 pgs: 129 active+clean; 640 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.6 KiB/s wr, 6 op/s 2024-03-04T09:29:27.224 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:29:27 smithi052 ceph-mon[32473]: pgmap v312: 129 pgs: 129 active+clean; 641 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.7 KiB/s wr, 7 op/s 2024-03-04T09:29:27.336 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:29:27 smithi191 ceph-mon[36816]: pgmap v312: 129 pgs: 129 active+clean; 641 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.7 KiB/s wr, 7 op/s 2024-03-04T09:29:27.381 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:29:27 smithi106 ceph-mon[36484]: pgmap v312: 129 pgs: 129 active+clean; 641 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.7 KiB/s wr, 7 op/s 2024-03-04T09:29:29.335 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:29:29 smithi052 ceph-mon[32473]: pgmap v313: 129 pgs: 129 active+clean; 641 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 170 B/s wr, 6 op/s 2024-03-04T09:29:29.336 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:29:29 smithi191 ceph-mon[36816]: pgmap v313: 129 pgs: 129 active+clean; 641 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 170 B/s wr, 6 op/s 2024-03-04T09:29:29.381 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:29:29 smithi106 ceph-mon[36484]: pgmap v313: 129 pgs: 129 active+clean; 641 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 170 B/s wr, 6 op/s 2024-03-04T09:29:29.642 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:29.645+0000 7fb283239700 1 -- 172.21.15.52:0/1376150975 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb27c1073d0 msgr2=0x7fb27c10efa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:29:29.642 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:29.645+0000 7fb283239700 1 --2- 172.21.15.52:0/1376150975 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb27c1073d0 0x7fb27c10efa0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fb27400b0a0 tx=0x7fb27400b3b0 comp rx=0 tx=0).stop 2024-03-04T09:29:29.642 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:29.645+0000 7fb283239700 1 -- 172.21.15.52:0/1376150975 shutdown_connections 2024-03-04T09:29:29.642 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:29.645+0000 7fb283239700 1 --2- 172.21.15.52:0/1376150975 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb27c1023a0 0x7fb27c102780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:29:29.643 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:29.645+0000 7fb283239700 1 --2- 172.21.15.52:0/1376150975 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb27c1073d0 0x7fb27c10efa0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:29:29.643 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:29.645+0000 7fb283239700 1 --2- 172.21.15.52:0/1376150975 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb27c102d50 0x7fb27c106cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:29:29.643 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:29.645+0000 7fb283239700 1 -- 172.21.15.52:0/1376150975 >> 172.21.15.52:0/1376150975 conn(0x7fb27c079150 msgr2=0x7fb27c079560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:29:29.643 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:29.645+0000 7fb283239700 1 -- 172.21.15.52:0/1376150975 shutdown_connections 2024-03-04T09:29:29.643 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:29.645+0000 7fb283239700 1 -- 172.21.15.52:0/1376150975 wait complete. 2024-03-04T09:29:29.644 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:29.646+0000 7fb283239700 1 Processor -- start 2024-03-04T09:29:29.644 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:29.646+0000 7fb283239700 1 -- start start 2024-03-04T09:29:29.645 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:29.646+0000 7fb283239700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb27c1023a0 0x7fb27c19e5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:29:29.645 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:29.646+0000 7fb283239700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb27c102d50 0x7fb27c19eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:29:29.645 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:29.646+0000 7fb283239700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb27c1073d0 0x7fb27c198680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:29:29.645 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:29.646+0000 7fb283239700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb27c112400 con 0x7fb27c1073d0 2024-03-04T09:29:29.645 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:29.646+0000 7fb283239700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb27c112280 con 0x7fb27c102d50 2024-03-04T09:29:29.645 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:29.646+0000 7fb283239700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb27c112580 con 0x7fb27c1023a0 2024-03-04T09:29:29.645 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:29.646+0000 7fb27bfff700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb27c102d50 0x7fb27c19eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:29:29.645 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:29.646+0000 7fb280fd5700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb27c1023a0 0x7fb27c19e5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:29:29.645 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:29.646+0000 7fb2817d6700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb27c1073d0 0x7fb27c198680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:29:29.645 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:29.646+0000 7fb27bfff700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb27c102d50 0x7fb27c19eaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.52:55540/0 (socket says 172.21.15.52:55540) 2024-03-04T09:29:29.645 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:29.646+0000 7fb27bfff700 1 -- 172.21.15.52:0/639586988 learned_addr learned my addr 172.21.15.52:0/639586988 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-04T09:29:29.645 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:29.647+0000 7fb27bfff700 1 -- 172.21.15.52:0/639586988 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb27c1023a0 msgr2=0x7fb27c19e5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:29:29.645 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:29.647+0000 7fb27bfff700 1 --2- 172.21.15.52:0/639586988 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb27c1023a0 0x7fb27c19e5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:29:29.645 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:29.647+0000 7fb27bfff700 1 -- 172.21.15.52:0/639586988 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb27c1073d0 msgr2=0x7fb27c198680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:29:29.646 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:29.647+0000 7fb27bfff700 1 --2- 172.21.15.52:0/639586988 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb27c1073d0 0x7fb27c198680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:29:29.646 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:29.647+0000 7fb27bfff700 1 -- 172.21.15.52:0/639586988 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb27c198f60 con 0x7fb27c102d50 2024-03-04T09:29:29.646 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:29.647+0000 7fb280fd5700 1 --2- 172.21.15.52:0/639586988 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb27c1023a0 0x7fb27c19e5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-04T09:29:29.646 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:29.647+0000 7fb2817d6700 1 --2- 172.21.15.52:0/639586988 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb27c1073d0 0x7fb27c198680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-04T09:29:29.646 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:29.647+0000 7fb27bfff700 1 --2- 172.21.15.52:0/639586988 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb27c102d50 0x7fb27c19eaf0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fb26800e970 tx=0x7fb26800ec80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:29:29.646 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:29.647+0000 7fb279ffb700 1 -- 172.21.15.52:0/639586988 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb268004a30 con 0x7fb27c102d50 2024-03-04T09:29:29.646 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:29.647+0000 7fb283239700 1 -- 172.21.15.52:0/639586988 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb27c199250 con 0x7fb27c102d50 2024-03-04T09:29:29.646 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:29.647+0000 7fb279ffb700 1 -- 172.21.15.52:0/639586988 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb26800ccc0 con 0x7fb27c102d50 2024-03-04T09:29:29.646 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:29.647+0000 7fb279ffb700 1 -- 172.21.15.52:0/639586988 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb268005650 con 0x7fb27c102d50 2024-03-04T09:29:29.646 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:29.647+0000 7fb283239700 1 -- 172.21.15.52:0/639586988 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb27c1a5530 con 0x7fb27c102d50 2024-03-04T09:29:29.647 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:29.648+0000 7fb283239700 1 -- 172.21.15.52:0/639586988 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fb27c062810 con 0x7fb27c102d50 2024-03-04T09:29:29.648 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:29.649+0000 7fb279ffb700 1 -- 172.21.15.52:0/639586988 <== mon.2 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 17) v1 ==== 136807+0+0 (secure 0 0 0) 0x7fb268004bd0 con 0x7fb27c102d50 2024-03-04T09:29:29.648 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:29.650+0000 7fb279ffb700 1 --2- 172.21.15.52:0/639586988 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7fb26c0a1ea0 0x7fb26c0a4360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:29:29.648 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:29.650+0000 7fb280fd5700 1 --2- 172.21.15.52:0/639586988 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7fb26c0a1ea0 0x7fb26c0a4360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:29:29.648 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:29.650+0000 7fb279ffb700 1 -- 172.21.15.52:0/639586988 <== mon.2 v2:172.21.15.106:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8802+0+0 (secure 0 0 0) 0x7fb2680ce7d0 con 0x7fb27c102d50 2024-03-04T09:29:29.648 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:29.650+0000 7fb279ffb700 1 --2- 172.21.15.52:0/639586988 >> [v2:172.21.15.106:6800/2540865537,v1:172.21.15.106:6801/2540865537] conn(0x7fb26c0acb60 0x7fb26c0aefa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:29:29.648 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:29.650+0000 7fb279ffb700 1 -- 172.21.15.52:0/639586988 --> [v2:172.21.15.106:6800/2540865537,v1:172.21.15.106:6801/2540865537] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fb268015e80 con 0x7fb26c0acb60 2024-03-04T09:29:29.648 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:29.650+0000 7fb279ffb700 1 -- 172.21.15.52:0/639586988 <== mon.2 v2:172.21.15.106:3300/0 6 ==== mon_get_version_reply(handle=1 version=89) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb2680ceb80 con 0x7fb27c102d50 2024-03-04T09:29:29.648 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:29.650+0000 7fb2817d6700 1 --2- 172.21.15.52:0/639586988 >> [v2:172.21.15.106:6800/2540865537,v1:172.21.15.106:6801/2540865537] conn(0x7fb26c0acb60 0x7fb26c0aefa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:29:29.648 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:29.651+0000 7fb280fd5700 1 --2- 172.21.15.52:0/639586988 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7fb26c0a1ea0 0x7fb26c0a4360 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fb270011490 tx=0x7fb270009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:29:29.648 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:29.651+0000 7fb2817d6700 1 --2- 172.21.15.52:0/639586988 >> [v2:172.21.15.106:6800/2540865537,v1:172.21.15.106:6801/2540865537] conn(0x7fb26c0acb60 0x7fb26c0aefa0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:29:29.649 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:29.652+0000 7fb279ffb700 1 -- 172.21.15.52:0/639586988 <== osd.1 v2:172.21.15.106:6800/2540865537 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7fb268015e80 con 0x7fb26c0acb60 2024-03-04T09:29:29.665 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:29.668+0000 7fb283239700 1 -- 172.21.15.52:0/639586988 --> [v2:172.21.15.106:6800/2540865537,v1:172.21.15.106:6801/2540865537] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fb27c062c10 con 0x7fb26c0acb60 2024-03-04T09:29:29.665 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:29.669+0000 7fb279ffb700 1 -- 172.21.15.52:0/639586988 <== osd.1 v2:172.21.15.106:6800/2540865537 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fb27c062c10 con 0x7fb26c0acb60 2024-03-04T09:29:29.666 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:29.669+0000 7fb283239700 1 -- 172.21.15.52:0/639586988 >> [v2:172.21.15.106:6800/2540865537,v1:172.21.15.106:6801/2540865537] conn(0x7fb26c0acb60 msgr2=0x7fb26c0aefa0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:29:29.666 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:29.669+0000 7fb283239700 1 --2- 172.21.15.52:0/639586988 >> [v2:172.21.15.106:6800/2540865537,v1:172.21.15.106:6801/2540865537] conn(0x7fb26c0acb60 0x7fb26c0aefa0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:29:29.666 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:29.669+0000 7fb283239700 1 -- 172.21.15.52:0/639586988 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7fb26c0a1ea0 msgr2=0x7fb26c0a4360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:29:29.667 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:29.669+0000 7fb283239700 1 --2- 172.21.15.52:0/639586988 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7fb26c0a1ea0 0x7fb26c0a4360 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fb270011490 tx=0x7fb270009260 comp rx=0 tx=0).stop 2024-03-04T09:29:29.667 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:29.669+0000 7fb283239700 1 -- 172.21.15.52:0/639586988 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb27c102d50 msgr2=0x7fb27c19eaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:29:29.667 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:29.669+0000 7fb283239700 1 --2- 172.21.15.52:0/639586988 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb27c102d50 0x7fb27c19eaf0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fb26800e970 tx=0x7fb26800ec80 comp rx=0 tx=0).stop 2024-03-04T09:29:29.667 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:29.669+0000 7fb2817d6700 1 -- 172.21.15.52:0/639586988 reap_dead start 2024-03-04T09:29:29.667 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:29.669+0000 7fb283239700 1 -- 172.21.15.52:0/639586988 shutdown_connections 2024-03-04T09:29:29.667 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:29.669+0000 7fb283239700 1 -- 172.21.15.52:0/639586988 >> 172.21.15.52:0/639586988 conn(0x7fb27c079150 msgr2=0x7fb27c1001f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:29:29.667 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:29.669+0000 7fb283239700 1 -- 172.21.15.52:0/639586988 shutdown_connections 2024-03-04T09:29:29.667 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:29.669+0000 7fb283239700 1 -- 172.21.15.52:0/639586988 wait complete. 2024-03-04T09:29:30.888 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:27.661+0000 7f4c2f49f700 1 -- 172.21.15.52:0/2723522776 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4c2806a860 msgr2=0x7f4c2806ac40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:29:30.888 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:27.661+0000 7f4c2f49f700 1 --2- 172.21.15.52:0/2723522776 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4c2806a860 0x7f4c2806ac40 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f4c240099c0 tx=0x7f4c24009cd0 comp rx=0 tx=0).stop 2024-03-04T09:29:30.888 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:27.661+0000 7f4c2f49f700 1 -- 172.21.15.52:0/2723522776 shutdown_connections 2024-03-04T09:29:30.888 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:27.661+0000 7f4c2f49f700 1 --2- 172.21.15.52:0/2723522776 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4c28107440 0x7f4c2810efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:29:30.888 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:27.661+0000 7f4c2f49f700 1 --2- 172.21.15.52:0/2723522776 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4c2806a860 0x7f4c2806ac40 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:29:30.889 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:27.661+0000 7f4c2f49f700 1 --2- 172.21.15.52:0/2723522776 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4c2806b210 0x7f4c28106d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:29:30.889 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:27.661+0000 7f4c2f49f700 1 -- 172.21.15.52:0/2723522776 >> 172.21.15.52:0/2723522776 conn(0x7f4c28077010 msgr2=0x7f4c28077420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:29:30.889 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:27.662+0000 7f4c2f49f700 1 -- 172.21.15.52:0/2723522776 shutdown_connections 2024-03-04T09:29:30.889 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:27.662+0000 7f4c2f49f700 1 -- 172.21.15.52:0/2723522776 wait complete. 2024-03-04T09:29:30.889 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:27.662+0000 7f4c2f49f700 1 Processor -- start 2024-03-04T09:29:30.889 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:27.662+0000 7f4c2f49f700 1 -- start start 2024-03-04T09:29:30.889 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:27.662+0000 7f4c2f49f700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4c2806a860 0x7f4c2819e630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:29:30.889 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:27.662+0000 7f4c2f49f700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4c2806b210 0x7f4c2819eb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:29:30.889 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:27.662+0000 7f4c2f49f700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4c28107440 0x7f4c28198700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:29:30.889 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:27.662+0000 7f4c2f49f700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c28112400 con 0x7f4c28107440 2024-03-04T09:29:30.889 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:27.662+0000 7f4c2f49f700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c28112280 con 0x7f4c2806a860 2024-03-04T09:29:30.889 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:27.662+0000 7f4c2f49f700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c28112580 con 0x7f4c2806b210 2024-03-04T09:29:30.890 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:27.662+0000 7f4c2da3c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4c28107440 0x7f4c28198700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:29:30.890 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:27.662+0000 7f4c2da3c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4c28107440 0x7f4c28198700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:54100/0 (socket says 172.21.15.52:54100) 2024-03-04T09:29:30.890 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:27.662+0000 7f4c2da3c700 1 -- 172.21.15.52:0/330241471 learned_addr learned my addr 172.21.15.52:0/330241471 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-04T09:29:30.890 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:27.663+0000 7f4c2d23b700 1 --2- 172.21.15.52:0/330241471 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4c2806a860 0x7f4c2819e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:29:30.890 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:27.663+0000 7f4c2ca3a700 1 --2- 172.21.15.52:0/330241471 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4c2806b210 0x7f4c2819eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:29:30.890 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:27.663+0000 7f4c2da3c700 1 -- 172.21.15.52:0/330241471 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4c2806b210 msgr2=0x7f4c2819eb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:29:30.890 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:27.663+0000 7f4c2da3c700 1 --2- 172.21.15.52:0/330241471 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4c2806b210 0x7f4c2819eb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:29:30.890 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:27.663+0000 7f4c2da3c700 1 -- 172.21.15.52:0/330241471 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4c2806a860 msgr2=0x7f4c2819e630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:29:30.890 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:27.663+0000 7f4c2da3c700 1 --2- 172.21.15.52:0/330241471 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4c2806a860 0x7f4c2819e630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:29:30.890 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:27.663+0000 7f4c2da3c700 1 -- 172.21.15.52:0/330241471 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c28198fe0 con 0x7f4c28107440 2024-03-04T09:29:30.891 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:27.663+0000 7f4c2da3c700 1 --2- 172.21.15.52:0/330241471 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4c28107440 0x7f4c28198700 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f4c2000ef30 tx=0x7f4c2000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:29:30.891 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:27.663+0000 7f4c1e7fc700 1 -- 172.21.15.52:0/330241471 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c200092e0 con 0x7f4c28107440 2024-03-04T09:29:30.891 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:27.663+0000 7f4c2f49f700 1 -- 172.21.15.52:0/330241471 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c28199270 con 0x7f4c28107440 2024-03-04T09:29:30.891 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:27.663+0000 7f4c1e7fc700 1 -- 172.21.15.52:0/330241471 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4c20004bb0 con 0x7f4c28107440 2024-03-04T09:29:30.891 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:27.663+0000 7f4c1e7fc700 1 -- 172.21.15.52:0/330241471 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c20004280 con 0x7f4c28107440 2024-03-04T09:29:30.891 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:27.663+0000 7f4c2f49f700 1 -- 172.21.15.52:0/330241471 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c281a53e0 con 0x7f4c28107440 2024-03-04T09:29:30.891 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:27.664+0000 7f4c2f49f700 1 -- 172.21.15.52:0/330241471 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f4c28062750 con 0x7f4c28107440 2024-03-04T09:29:30.891 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:27.665+0000 7f4c1e7fc700 1 -- 172.21.15.52:0/330241471 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 17) v1 ==== 136807+0+0 (secure 0 0 0) 0x7f4c2001b030 con 0x7f4c28107440 2024-03-04T09:29:30.891 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:27.666+0000 7f4c1e7fc700 1 --2- 172.21.15.52:0/330241471 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f4c140a1ea0 0x7f4c140a4360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:29:30.891 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:27.666+0000 7f4c2d23b700 1 --2- 172.21.15.52:0/330241471 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f4c140a1ea0 0x7f4c140a4360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:29:30.891 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:27.666+0000 7f4c1e7fc700 1 -- 172.21.15.52:0/330241471 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8802+0+0 (secure 0 0 0) 0x7f4c200ce550 con 0x7f4c28107440 2024-03-04T09:29:30.891 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:27.666+0000 7f4c1e7fc700 1 --2- 172.21.15.52:0/330241471 >> [v2:172.21.15.52:6810/2388571810,v1:172.21.15.52:6811/2388571810] conn(0x7f4c140acb60 0x7f4c140aefa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:29:30.892 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:27.666+0000 7f4c1e7fc700 1 -- 172.21.15.52:0/330241471 --> [v2:172.21.15.52:6810/2388571810,v1:172.21.15.52:6811/2388571810] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f4c2001fe50 con 0x7f4c140acb60 2024-03-04T09:29:30.892 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:27.666+0000 7f4c1e7fc700 1 -- 172.21.15.52:0/330241471 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_get_version_reply(handle=1 version=89) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4c200ce9a0 con 0x7f4c28107440 2024-03-04T09:29:30.892 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:27.666+0000 7f4c2ca3a700 1 --2- 172.21.15.52:0/330241471 >> [v2:172.21.15.52:6810/2388571810,v1:172.21.15.52:6811/2388571810] conn(0x7f4c140acb60 0x7f4c140aefa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:29:30.892 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:27.666+0000 7f4c2d23b700 1 --2- 172.21.15.52:0/330241471 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f4c140a1ea0 0x7f4c140a4360 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f4c24000c00 tx=0x7f4c24005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:29:30.892 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:27.667+0000 7f4c2ca3a700 1 --2- 172.21.15.52:0/330241471 >> [v2:172.21.15.52:6810/2388571810,v1:172.21.15.52:6811/2388571810] conn(0x7f4c140acb60 0x7f4c140aefa0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:29:30.892 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:27.667+0000 7f4c1e7fc700 1 -- 172.21.15.52:0/330241471 <== osd.3 v2:172.21.15.52:6810/2388571810 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f4c2001fe50 con 0x7f4c140acb60 2024-03-04T09:29:30.892 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:27.683+0000 7f4c2f49f700 1 -- 172.21.15.52:0/330241471 --> [v2:172.21.15.52:6810/2388571810,v1:172.21.15.52:6811/2388571810] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f4c28062bd0 con 0x7f4c140acb60 2024-03-04T09:29:30.892 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:27.684+0000 7f4c1e7fc700 1 -- 172.21.15.52:0/330241471 <== osd.3 v2:172.21.15.52:6810/2388571810 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f4c28062bd0 con 0x7f4c140acb60 2024-03-04T09:29:30.892 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:27.684+0000 7f4c2f49f700 1 -- 172.21.15.52:0/330241471 >> [v2:172.21.15.52:6810/2388571810,v1:172.21.15.52:6811/2388571810] conn(0x7f4c140acb60 msgr2=0x7f4c140aefa0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:29:30.892 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:27.684+0000 7f4c2f49f700 1 --2- 172.21.15.52:0/330241471 >> [v2:172.21.15.52:6810/2388571810,v1:172.21.15.52:6811/2388571810] conn(0x7f4c140acb60 0x7f4c140aefa0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:29:30.892 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:27.684+0000 7f4c2f49f700 1 -- 172.21.15.52:0/330241471 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f4c140a1ea0 msgr2=0x7f4c140a4360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:29:30.892 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:27.684+0000 7f4c2f49f700 1 --2- 172.21.15.52:0/330241471 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f4c140a1ea0 0x7f4c140a4360 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f4c24000c00 tx=0x7f4c24005fb0 comp rx=0 tx=0).stop 2024-03-04T09:29:30.892 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:27.684+0000 7f4c2f49f700 1 -- 172.21.15.52:0/330241471 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4c28107440 msgr2=0x7f4c28198700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:29:30.893 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:27.684+0000 7f4c2f49f700 1 --2- 172.21.15.52:0/330241471 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4c28107440 0x7f4c28198700 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f4c2000ef30 tx=0x7f4c2000c3a0 comp rx=0 tx=0).stop 2024-03-04T09:29:30.893 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:27.684+0000 7f4c2da3c700 1 -- 172.21.15.52:0/330241471 reap_dead start 2024-03-04T09:29:30.893 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:27.684+0000 7f4c2f49f700 1 -- 172.21.15.52:0/330241471 shutdown_connections 2024-03-04T09:29:30.893 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:27.684+0000 7f4c2f49f700 1 -- 172.21.15.52:0/330241471 >> 172.21.15.52:0/330241471 conn(0x7f4c28077010 msgr2=0x7f4c280ff2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:29:30.893 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:27.684+0000 7f4c2f49f700 1 -- 172.21.15.52:0/330241471 shutdown_connections 2024-03-04T09:29:30.893 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:27.684+0000 7f4c2f49f700 1 -- 172.21.15.52:0/330241471 wait complete. 2024-03-04T09:29:31.335 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:29:31 smithi052 ceph-mon[32473]: pgmap v314: 129 pgs: 129 active+clean; 641 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 5 op/s 2024-03-04T09:29:31.336 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:29:31 smithi191 ceph-mon[36816]: pgmap v314: 129 pgs: 129 active+clean; 641 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 5 op/s 2024-03-04T09:29:31.381 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:29:31 smithi106 ceph-mon[36484]: pgmap v314: 129 pgs: 129 active+clean; 641 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 5 op/s 2024-03-04T09:29:32.443 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:32.446+0000 7ff1a78ab700 1 -- 172.21.15.52:0/226724450 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff1a0102ad0 msgr2=0x7ff1a010ef10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:29:32.443 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:32.446+0000 7ff1a78ab700 1 --2- 172.21.15.52:0/226724450 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff1a0102ad0 0x7ff1a010ef10 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7ff19800b0d0 tx=0x7ff19800b3e0 comp rx=0 tx=0).stop 2024-03-04T09:29:32.444 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:32.446+0000 7ff1a78ab700 1 -- 172.21.15.52:0/226724450 shutdown_connections 2024-03-04T09:29:32.444 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:32.446+0000 7ff1a78ab700 1 --2- 172.21.15.52:0/226724450 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff1a0102ad0 0x7ff1a010ef10 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:29:32.444 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:32.446+0000 7ff1a78ab700 1 --2- 172.21.15.52:0/226724450 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff1a01080f0 0x7ff1a01084d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:29:32.444 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:32.446+0000 7ff1a78ab700 1 --2- 172.21.15.52:0/226724450 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff1a0102110 0x7ff1a0102590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:29:32.444 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:32.446+0000 7ff1a78ab700 1 -- 172.21.15.52:0/226724450 >> 172.21.15.52:0/226724450 conn(0x7ff1a00791f0 msgr2=0x7ff1a0079600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:29:32.444 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:32.446+0000 7ff1a78ab700 1 -- 172.21.15.52:0/226724450 shutdown_connections 2024-03-04T09:29:32.444 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:32.446+0000 7ff1a78ab700 1 -- 172.21.15.52:0/226724450 wait complete. 2024-03-04T09:29:32.447 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:32.447+0000 7ff1a78ab700 1 Processor -- start 2024-03-04T09:29:32.447 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:32.447+0000 7ff1a78ab700 1 -- start start 2024-03-04T09:29:32.447 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:32.447+0000 7ff1a78ab700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff1a0102110 0x7ff1a0198750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:29:32.447 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:32.447+0000 7ff1a78ab700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff1a0102ad0 0x7ff1a0198c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:29:32.448 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:32.447+0000 7ff1a78ab700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff1a01080f0 0x7ff1a019d020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:29:32.448 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:32.447+0000 7ff1a78ab700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1a01123a0 con 0x7ff1a0102ad0 2024-03-04T09:29:32.448 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:32.447+0000 7ff1a78ab700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1a0112220 con 0x7ff1a0102110 2024-03-04T09:29:32.448 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:32.447+0000 7ff1a78ab700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1a0112520 con 0x7ff1a01080f0 2024-03-04T09:29:32.448 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:32.447+0000 7ff1a4e46700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff1a0102ad0 0x7ff1a0198c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:29:32.448 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:32.447+0000 7ff1a4e46700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff1a0102ad0 0x7ff1a0198c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:54128/0 (socket says 172.21.15.52:54128) 2024-03-04T09:29:32.448 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:32.447+0000 7ff1a5647700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff1a0102110 0x7ff1a0198750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:29:32.448 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:32.447+0000 7ff1a4e46700 1 -- 172.21.15.52:0/1150382263 learned_addr learned my addr 172.21.15.52:0/1150382263 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-04T09:29:32.449 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:32.447+0000 7ff1a5e48700 1 --2- 172.21.15.52:0/1150382263 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff1a01080f0 0x7ff1a019d020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:29:32.449 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:32.447+0000 7ff1a4e46700 1 -- 172.21.15.52:0/1150382263 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff1a01080f0 msgr2=0x7ff1a019d020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:29:32.449 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:32.447+0000 7ff1a4e46700 1 --2- 172.21.15.52:0/1150382263 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff1a01080f0 0x7ff1a019d020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:29:32.449 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:32.447+0000 7ff1a4e46700 1 -- 172.21.15.52:0/1150382263 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff1a0102110 msgr2=0x7ff1a0198750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:29:32.449 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:32.447+0000 7ff1a4e46700 1 --2- 172.21.15.52:0/1150382263 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff1a0102110 0x7ff1a0198750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:29:32.449 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:32.447+0000 7ff1a4e46700 1 -- 172.21.15.52:0/1150382263 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1a019d760 con 0x7ff1a0102ad0 2024-03-04T09:29:32.449 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:32.447+0000 7ff1a5647700 1 --2- 172.21.15.52:0/1150382263 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff1a0102110 0x7ff1a0198750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-04T09:29:32.449 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:32.448+0000 7ff1a4e46700 1 --2- 172.21.15.52:0/1150382263 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff1a0102ad0 0x7ff1a0198c90 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7ff19000e9c0 tx=0x7ff19000ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:29:32.450 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:32.448+0000 7ff1967fc700 1 -- 172.21.15.52:0/1150382263 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff19000cbd0 con 0x7ff1a0102ad0 2024-03-04T09:29:32.450 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:32.448+0000 7ff1967fc700 1 -- 172.21.15.52:0/1150382263 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff19000ee60 con 0x7ff1a0102ad0 2024-03-04T09:29:32.450 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:32.448+0000 7ff1a78ab700 1 -- 172.21.15.52:0/1150382263 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1a019da50 con 0x7ff1a0102ad0 2024-03-04T09:29:32.450 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:32.448+0000 7ff1967fc700 1 -- 172.21.15.52:0/1150382263 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff190005630 con 0x7ff1a0102ad0 2024-03-04T09:29:32.450 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:32.448+0000 7ff1a78ab700 1 -- 172.21.15.52:0/1150382263 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1a01a52d0 con 0x7ff1a0102ad0 2024-03-04T09:29:32.450 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:32.449+0000 7ff1a78ab700 1 -- 172.21.15.52:0/1150382263 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7ff1a0062810 con 0x7ff1a0102ad0 2024-03-04T09:29:32.450 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:32.450+0000 7ff1967fc700 1 -- 172.21.15.52:0/1150382263 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 17) v1 ==== 136807+0+0 (secure 0 0 0) 0x7ff190004750 con 0x7ff1a0102ad0 2024-03-04T09:29:32.451 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:32.450+0000 7ff1967fc700 1 --2- 172.21.15.52:0/1150382263 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7ff18c0a1f20 0x7ff18c0a43e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:29:32.452 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:32.451+0000 7ff1a5647700 1 --2- 172.21.15.52:0/1150382263 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7ff18c0a1f20 0x7ff18c0a43e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:29:32.452 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:32.451+0000 7ff1967fc700 1 -- 172.21.15.52:0/1150382263 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8802+0+0 (secure 0 0 0) 0x7ff1900ce810 con 0x7ff1a0102ad0 2024-03-04T09:29:32.452 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:32.451+0000 7ff1967fc700 1 --2- 172.21.15.52:0/1150382263 >> [v2:172.21.15.52:6802/1371221397,v1:172.21.15.52:6803/1371221397] conn(0x7ff18c0acbe0 0x7ff18c0af020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:29:32.452 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:32.451+0000 7ff1967fc700 1 -- 172.21.15.52:0/1150382263 --> [v2:172.21.15.52:6802/1371221397,v1:172.21.15.52:6803/1371221397] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff190011e50 con 0x7ff18c0acbe0 2024-03-04T09:29:32.452 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:32.451+0000 7ff1967fc700 1 -- 172.21.15.52:0/1150382263 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_get_version_reply(handle=1 version=89) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff1900cebc0 con 0x7ff1a0102ad0 2024-03-04T09:29:32.452 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:32.451+0000 7ff1a5e48700 1 --2- 172.21.15.52:0/1150382263 >> [v2:172.21.15.52:6802/1371221397,v1:172.21.15.52:6803/1371221397] conn(0x7ff18c0acbe0 0x7ff18c0af020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:29:32.452 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:32.451+0000 7ff1a5647700 1 --2- 172.21.15.52:0/1150382263 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7ff18c0a1f20 0x7ff18c0a43e0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7ff19c011490 tx=0x7ff19c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:29:32.452 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:32.451+0000 7ff1a5e48700 1 --2- 172.21.15.52:0/1150382263 >> [v2:172.21.15.52:6802/1371221397,v1:172.21.15.52:6803/1371221397] conn(0x7ff18c0acbe0 0x7ff18c0af020 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:29:32.452 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:32.452+0000 7ff1967fc700 1 -- 172.21.15.52:0/1150382263 <== osd.0 v2:172.21.15.52:6802/1371221397 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7ff190011e50 con 0x7ff18c0acbe0 2024-03-04T09:29:32.465 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:32.469+0000 7ff1a78ab700 1 -- 172.21.15.52:0/1150382263 --> [v2:172.21.15.52:6802/1371221397,v1:172.21.15.52:6803/1371221397] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7ff1a0062c90 con 0x7ff18c0acbe0 2024-03-04T09:29:32.466 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:32.469+0000 7ff1967fc700 1 -- 172.21.15.52:0/1150382263 <== osd.0 v2:172.21.15.52:6802/1371221397 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7ff1a0062c90 con 0x7ff18c0acbe0 2024-03-04T09:29:32.467 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:32.469+0000 7ff1a78ab700 1 -- 172.21.15.52:0/1150382263 >> [v2:172.21.15.52:6802/1371221397,v1:172.21.15.52:6803/1371221397] conn(0x7ff18c0acbe0 msgr2=0x7ff18c0af020 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:29:32.467 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:32.469+0000 7ff1a78ab700 1 --2- 172.21.15.52:0/1150382263 >> [v2:172.21.15.52:6802/1371221397,v1:172.21.15.52:6803/1371221397] conn(0x7ff18c0acbe0 0x7ff18c0af020 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:29:32.467 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:32.469+0000 7ff1a78ab700 1 -- 172.21.15.52:0/1150382263 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7ff18c0a1f20 msgr2=0x7ff18c0a43e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:29:32.467 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:32.470+0000 7ff1a78ab700 1 --2- 172.21.15.52:0/1150382263 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7ff18c0a1f20 0x7ff18c0a43e0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7ff19c011490 tx=0x7ff19c009260 comp rx=0 tx=0).stop 2024-03-04T09:29:32.467 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:32.470+0000 7ff1a78ab700 1 -- 172.21.15.52:0/1150382263 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff1a0102ad0 msgr2=0x7ff1a0198c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:29:32.467 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:32.470+0000 7ff1a78ab700 1 --2- 172.21.15.52:0/1150382263 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff1a0102ad0 0x7ff1a0198c90 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7ff19000e9c0 tx=0x7ff19000ecd0 comp rx=0 tx=0).stop 2024-03-04T09:29:32.467 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:32.470+0000 7ff1a5e48700 1 -- 172.21.15.52:0/1150382263 reap_dead start 2024-03-04T09:29:32.468 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:32.470+0000 7ff1a78ab700 1 -- 172.21.15.52:0/1150382263 shutdown_connections 2024-03-04T09:29:32.468 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:32.470+0000 7ff1a78ab700 1 -- 172.21.15.52:0/1150382263 >> 172.21.15.52:0/1150382263 conn(0x7ff1a00791f0 msgr2=0x7ff1a0100cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:29:32.468 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:32.470+0000 7ff1a78ab700 1 -- 172.21.15.52:0/1150382263 shutdown_connections 2024-03-04T09:29:32.468 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:32.470+0000 7ff1a78ab700 1 -- 172.21.15.52:0/1150382263 wait complete. 2024-03-04T09:29:33.335 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:29:33 smithi052 ceph-mon[32473]: pgmap v315: 129 pgs: 129 active+clean; 641 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 170 B/s wr, 7 op/s 2024-03-04T09:29:33.336 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:29:33 smithi191 ceph-mon[36816]: pgmap v315: 129 pgs: 129 active+clean; 641 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 170 B/s wr, 7 op/s 2024-03-04T09:29:33.383 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:29:33 smithi106 ceph-mon[36484]: pgmap v315: 129 pgs: 129 active+clean; 641 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 170 B/s wr, 7 op/s 2024-03-04T09:29:35.012 INFO:teuthology.orchestra.run.smithi052.stdout:55834574921 2024-03-04T09:29:35.012 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd shell --fsid 4e059f08-da08-11ee-95c6-87774f69a715 -- ceph osd last-stat-seq osd.1 2024-03-04T09:29:35.308 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:29:35 smithi052 ceph-mon[32473]: pgmap v316: 129 pgs: 129 active+clean; 641 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 5 op/s 2024-03-04T09:29:35.336 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:29:35 smithi191 ceph-mon[36816]: pgmap v316: 129 pgs: 129 active+clean; 641 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 5 op/s 2024-03-04T09:29:35.381 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:29:35 smithi106 ceph-mon[36484]: pgmap v316: 129 pgs: 129 active+clean; 641 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 5 op/s 2024-03-04T09:29:37.044 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:37.042+0000 7fbafa7a4700 1 -- 172.21.15.52:0/1313076520 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbaf4108220 msgr2=0x7fbaf4108600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:29:37.045 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:37.042+0000 7fbafa7a4700 1 --2- 172.21.15.52:0/1313076520 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbaf4108220 0x7fbaf4108600 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fbae4009990 tx=0x7fbae4009ca0 comp rx=0 tx=0).stop 2024-03-04T09:29:37.045 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:37.042+0000 7fbafa7a4700 1 -- 172.21.15.52:0/1313076520 shutdown_connections 2024-03-04T09:29:37.045 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:37.042+0000 7fbafa7a4700 1 --2- 172.21.15.52:0/1313076520 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbaf4102b00 0x7fbaf410ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:29:37.045 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:37.042+0000 7fbafa7a4700 1 --2- 172.21.15.52:0/1313076520 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbaf4108220 0x7fbaf4108600 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:29:37.045 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:37.042+0000 7fbafa7a4700 1 --2- 172.21.15.52:0/1313076520 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbaf4102140 0x7fbaf41025c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:29:37.045 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:37.042+0000 7fbafa7a4700 1 -- 172.21.15.52:0/1313076520 >> 172.21.15.52:0/1313076520 conn(0x7fbaf4079180 msgr2=0x7fbaf4079590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:29:37.045 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:37.042+0000 7fbafa7a4700 1 -- 172.21.15.52:0/1313076520 shutdown_connections 2024-03-04T09:29:37.045 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:37.042+0000 7fbafa7a4700 1 -- 172.21.15.52:0/1313076520 wait complete. 2024-03-04T09:29:37.045 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:37.043+0000 7fbafa7a4700 1 Processor -- start 2024-03-04T09:29:37.046 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:37.043+0000 7fbafa7a4700 1 -- start start 2024-03-04T09:29:37.046 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:37.043+0000 7fbafa7a4700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbaf4102140 0x7fbaf40743c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:29:37.046 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:37.043+0000 7fbafa7a4700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbaf4102b00 0x7fbaf4074900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:29:37.046 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:37.043+0000 7fbafa7a4700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbaf4108220 0x7fbaf4074e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:29:37.046 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:37.043+0000 7fbafa7a4700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbaf41123a0 con 0x7fbaf4102140 2024-03-04T09:29:37.046 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:37.043+0000 7fbafa7a4700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbaf4112220 con 0x7fbaf4108220 2024-03-04T09:29:37.046 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:37.043+0000 7fbafa7a4700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbaf4112520 con 0x7fbaf4102b00 2024-03-04T09:29:37.046 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:37.043+0000 7fbaf3fff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbaf4102140 0x7fbaf40743c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:29:37.046 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:37.043+0000 7fbaf3fff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbaf4102140 0x7fbaf40743c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:56966/0 (socket says 172.21.15.52:56966) 2024-03-04T09:29:37.047 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:37.043+0000 7fbaf3fff700 1 -- 172.21.15.52:0/3638947615 learned_addr learned my addr 172.21.15.52:0/3638947615 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-04T09:29:37.047 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:37.043+0000 7fbaf37fe700 1 --2- 172.21.15.52:0/3638947615 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbaf4102b00 0x7fbaf4074900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:29:37.047 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:37.043+0000 7fbaf8d41700 1 --2- 172.21.15.52:0/3638947615 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbaf4108220 0x7fbaf4074e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:29:37.047 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:37.044+0000 7fbaf3fff700 1 -- 172.21.15.52:0/3638947615 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbaf4102b00 msgr2=0x7fbaf4074900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:29:37.047 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:37.044+0000 7fbaf3fff700 1 --2- 172.21.15.52:0/3638947615 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbaf4102b00 0x7fbaf4074900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:29:37.047 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:37.044+0000 7fbaf3fff700 1 -- 172.21.15.52:0/3638947615 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbaf4108220 msgr2=0x7fbaf4074e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:29:37.047 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:37.044+0000 7fbaf3fff700 1 --2- 172.21.15.52:0/3638947615 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbaf4108220 0x7fbaf4074e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:29:37.047 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:37.044+0000 7fbaf3fff700 1 -- 172.21.15.52:0/3638947615 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbaf41a6db0 con 0x7fbaf4102140 2024-03-04T09:29:37.047 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:37.044+0000 7fbaf3fff700 1 --2- 172.21.15.52:0/3638947615 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbaf4102140 0x7fbaf40743c0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fbae4000c00 tx=0x7fbae400bab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:29:37.047 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:37.044+0000 7fbaf17fa700 1 -- 172.21.15.52:0/3638947615 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbae4027070 con 0x7fbaf4102140 2024-03-04T09:29:37.047 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:37.044+0000 7fbafa7a4700 1 -- 172.21.15.52:0/3638947615 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbaf41a7040 con 0x7fbaf4102140 2024-03-04T09:29:37.047 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:37.044+0000 7fbaf17fa700 1 -- 172.21.15.52:0/3638947615 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fbae4003d70 con 0x7fbaf4102140 2024-03-04T09:29:37.048 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:37.044+0000 7fbaf17fa700 1 -- 172.21.15.52:0/3638947615 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbae400f460 con 0x7fbaf4102140 2024-03-04T09:29:37.048 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:37.044+0000 7fbafa7a4700 1 -- 172.21.15.52:0/3638947615 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbaf41a7520 con 0x7fbaf4102140 2024-03-04T09:29:37.048 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:37.045+0000 7fbafa7a4700 1 -- 172.21.15.52:0/3638947615 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fbaf40627d0 con 0x7fbaf4102140 2024-03-04T09:29:37.048 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:37.046+0000 7fbaf17fa700 1 -- 172.21.15.52:0/3638947615 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 17) v1 ==== 136807+0+0 (secure 0 0 0) 0x7fbae402a070 con 0x7fbaf4102140 2024-03-04T09:29:37.048 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:37.046+0000 7fbaf17fa700 1 --2- 172.21.15.52:0/3638947615 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7fbadc0a1ea0 0x7fbadc0a4360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:29:37.048 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:37.047+0000 7fbaf17fa700 1 -- 172.21.15.52:0/3638947615 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8802+0+0 (secure 0 0 0) 0x7fbae40cf5b0 con 0x7fbaf4102140 2024-03-04T09:29:37.048 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:37.047+0000 7fbaf37fe700 1 --2- 172.21.15.52:0/3638947615 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7fbadc0a1ea0 0x7fbadc0a4360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:29:37.048 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:37.047+0000 7fbaf17fa700 1 --2- 172.21.15.52:0/3638947615 >> [v2:172.21.15.52:6818/1989797856,v1:172.21.15.52:6819/1989797856] conn(0x7fbadc0acb60 0x7fbadc0aefa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:29:37.048 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:37.047+0000 7fbaf17fa700 1 -- 172.21.15.52:0/3638947615 --> [v2:172.21.15.52:6818/1989797856,v1:172.21.15.52:6819/1989797856] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fbae4001d30 con 0x7fbadc0acb60 2024-03-04T09:29:37.048 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:37.047+0000 7fbaf17fa700 1 -- 172.21.15.52:0/3638947615 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_get_version_reply(handle=1 version=89) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbae40cf960 con 0x7fbaf4102140 2024-03-04T09:29:37.048 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:37.047+0000 7fbaf8d41700 1 --2- 172.21.15.52:0/3638947615 >> [v2:172.21.15.52:6818/1989797856,v1:172.21.15.52:6819/1989797856] conn(0x7fbadc0acb60 0x7fbadc0aefa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:29:37.048 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:37.047+0000 7fbaf37fe700 1 --2- 172.21.15.52:0/3638947615 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7fbadc0a1ea0 0x7fbadc0a4360 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fbaf40797f0 tx=0x7fbae8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:29:37.049 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:37.047+0000 7fbaf8d41700 1 --2- 172.21.15.52:0/3638947615 >> [v2:172.21.15.52:6818/1989797856,v1:172.21.15.52:6819/1989797856] conn(0x7fbadc0acb60 0x7fbadc0aefa0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:29:37.053 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:37.048+0000 7fbaf17fa700 1 -- 172.21.15.52:0/3638947615 <== osd.6 v2:172.21.15.52:6818/1989797856 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7fbae4001d30 con 0x7fbadc0acb60 2024-03-04T09:29:37.062 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:37.065+0000 7fbafa7a4700 1 -- 172.21.15.52:0/3638947615 --> [v2:172.21.15.52:6818/1989797856,v1:172.21.15.52:6819/1989797856] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fbaf4062bd0 con 0x7fbadc0acb60 2024-03-04T09:29:37.062 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:37.065+0000 7fbaf17fa700 1 -- 172.21.15.52:0/3638947615 <== osd.6 v2:172.21.15.52:6818/1989797856 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7fbaf4062bd0 con 0x7fbadc0acb60 2024-03-04T09:29:37.064 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:37.066+0000 7fbafa7a4700 1 -- 172.21.15.52:0/3638947615 >> [v2:172.21.15.52:6818/1989797856,v1:172.21.15.52:6819/1989797856] conn(0x7fbadc0acb60 msgr2=0x7fbadc0aefa0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:29:37.064 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:37.066+0000 7fbafa7a4700 1 --2- 172.21.15.52:0/3638947615 >> [v2:172.21.15.52:6818/1989797856,v1:172.21.15.52:6819/1989797856] conn(0x7fbadc0acb60 0x7fbadc0aefa0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:29:37.064 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:37.066+0000 7fbafa7a4700 1 -- 172.21.15.52:0/3638947615 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7fbadc0a1ea0 msgr2=0x7fbadc0a4360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:29:37.064 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:37.066+0000 7fbafa7a4700 1 --2- 172.21.15.52:0/3638947615 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7fbadc0a1ea0 0x7fbadc0a4360 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fbaf40797f0 tx=0x7fbae8009260 comp rx=0 tx=0).stop 2024-03-04T09:29:37.064 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:37.066+0000 7fbafa7a4700 1 -- 172.21.15.52:0/3638947615 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbaf4102140 msgr2=0x7fbaf40743c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:29:37.064 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:37.066+0000 7fbafa7a4700 1 --2- 172.21.15.52:0/3638947615 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbaf4102140 0x7fbaf40743c0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fbae4000c00 tx=0x7fbae400bab0 comp rx=0 tx=0).stop 2024-03-04T09:29:37.064 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:37.066+0000 7fbaf8d41700 1 -- 172.21.15.52:0/3638947615 reap_dead start 2024-03-04T09:29:37.064 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:37.066+0000 7fbafa7a4700 1 -- 172.21.15.52:0/3638947615 shutdown_connections 2024-03-04T09:29:37.064 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:37.066+0000 7fbafa7a4700 1 -- 172.21.15.52:0/3638947615 >> 172.21.15.52:0/3638947615 conn(0x7fbaf4079180 msgr2=0x7fbaf4100e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:29:37.064 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:37.066+0000 7fbafa7a4700 1 -- 172.21.15.52:0/3638947615 shutdown_connections 2024-03-04T09:29:37.065 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:37.066+0000 7fbafa7a4700 1 -- 172.21.15.52:0/3638947615 wait complete. 2024-03-04T09:29:37.319 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:29:37 smithi052 ceph-mon[32473]: pgmap v317: 129 pgs: 129 active+clean; 641 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 170 B/s wr, 6 op/s 2024-03-04T09:29:37.336 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:29:37 smithi191 ceph-mon[36816]: pgmap v317: 129 pgs: 129 active+clean; 641 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 170 B/s wr, 6 op/s 2024-03-04T09:29:37.381 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:29:37 smithi106 ceph-mon[36484]: pgmap v317: 129 pgs: 129 active+clean; 641 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 170 B/s wr, 6 op/s 2024-03-04T09:29:37.509 INFO:teuthology.orchestra.run.smithi052.stdout:34359738447 2024-03-04T09:29:37.510 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd shell --fsid 4e059f08-da08-11ee-95c6-87774f69a715 -- ceph osd last-stat-seq osd.0 2024-03-04T09:29:37.730 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:37.732+0000 7f9075825700 1 -- 172.21.15.52:0/828170943 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f907006aa20 msgr2=0x7f907006ae00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:29:37.730 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:37.732+0000 7f9075825700 1 --2- 172.21.15.52:0/828170943 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f907006aa20 0x7f907006ae00 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f90600099c0 tx=0x7f9060009cd0 comp rx=0 tx=0).stop 2024-03-04T09:29:37.730 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:37.732+0000 7f9075825700 1 -- 172.21.15.52:0/828170943 shutdown_connections 2024-03-04T09:29:37.730 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:37.732+0000 7f9075825700 1 --2- 172.21.15.52:0/828170943 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9070107470 0x7f907010eff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:29:37.730 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:37.732+0000 7f9075825700 1 --2- 172.21.15.52:0/828170943 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f907006aa20 0x7f907006ae00 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:29:37.730 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:37.732+0000 7f9075825700 1 --2- 172.21.15.52:0/828170943 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f907006b3d0 0x7f9070106d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:29:37.730 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:37.732+0000 7f9075825700 1 -- 172.21.15.52:0/828170943 >> 172.21.15.52:0/828170943 conn(0x7f9070077010 msgr2=0x7f9070077420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:29:37.730 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:37.732+0000 7f9075825700 1 -- 172.21.15.52:0/828170943 shutdown_connections 2024-03-04T09:29:37.730 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:37.732+0000 7f9075825700 1 -- 172.21.15.52:0/828170943 wait complete. 2024-03-04T09:29:37.731 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:37.733+0000 7f9075825700 1 Processor -- start 2024-03-04T09:29:37.733 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:37.733+0000 7f9075825700 1 -- start start 2024-03-04T09:29:37.733 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:37.733+0000 7f9075825700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f907006aa20 0x7f90701010f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:29:37.733 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:37.733+0000 7f9075825700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f907006b3d0 0x7f9070101630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:29:37.733 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:37.733+0000 7f9075825700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9070107470 0x7f9070101b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:29:37.733 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:37.733+0000 7f9075825700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9070112400 con 0x7f907006b3d0 2024-03-04T09:29:37.733 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:37.733+0000 7f9075825700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9070112280 con 0x7f9070107470 2024-03-04T09:29:37.733 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:37.733+0000 7f9075825700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9070112580 con 0x7f907006aa20 2024-03-04T09:29:37.733 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:37.734+0000 7f906e7fc700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f907006b3d0 0x7f9070101630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:29:37.733 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:37.734+0000 7f906effd700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f907006aa20 0x7f90701010f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:29:37.733 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:37.734+0000 7f906e7fc700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f907006b3d0 0x7f9070101630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:56988/0 (socket says 172.21.15.52:56988) 2024-03-04T09:29:37.734 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:37.734+0000 7f906e7fc700 1 -- 172.21.15.52:0/3665472957 learned_addr learned my addr 172.21.15.52:0/3665472957 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-04T09:29:37.734 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:37.734+0000 7f906f7fe700 1 --2- 172.21.15.52:0/3665472957 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9070107470 0x7f9070101b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:29:37.734 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:37.734+0000 7f906effd700 1 -- 172.21.15.52:0/3665472957 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9070107470 msgr2=0x7f9070101b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:29:37.735 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:37.734+0000 7f906effd700 1 --2- 172.21.15.52:0/3665472957 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9070107470 0x7f9070101b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:29:37.735 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:37.734+0000 7f906effd700 1 -- 172.21.15.52:0/3665472957 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f907006b3d0 msgr2=0x7f9070101630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:29:37.735 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:37.734+0000 7f906effd700 1 --2- 172.21.15.52:0/3665472957 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f907006b3d0 0x7f9070101630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:29:37.735 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:37.734+0000 7f906effd700 1 -- 172.21.15.52:0/3665472957 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90701a6da0 con 0x7f907006aa20 2024-03-04T09:29:37.735 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:37.734+0000 7f906f7fe700 1 --2- 172.21.15.52:0/3665472957 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9070107470 0x7f9070101b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-04T09:29:37.735 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:37.734+0000 7f906e7fc700 1 --2- 172.21.15.52:0/3665472957 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f907006b3d0 0x7f9070101630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-04T09:29:37.735 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:37.734+0000 7f906effd700 1 --2- 172.21.15.52:0/3665472957 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f907006aa20 0x7f90701010f0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f90600098e0 tx=0x7f906000fa00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:29:37.735 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:37.734+0000 7f9074823700 1 -- 172.21.15.52:0/3665472957 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9060027070 con 0x7f907006aa20 2024-03-04T09:29:37.736 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:37.734+0000 7f9074823700 1 -- 172.21.15.52:0/3665472957 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9060003680 con 0x7f907006aa20 2024-03-04T09:29:37.736 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:37.734+0000 7f9075825700 1 -- 172.21.15.52:0/3665472957 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90701a7030 con 0x7f907006aa20 2024-03-04T09:29:37.736 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:37.734+0000 7f9074823700 1 -- 172.21.15.52:0/3665472957 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90600173f0 con 0x7f907006aa20 2024-03-04T09:29:37.736 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:37.734+0000 7f9075825700 1 -- 172.21.15.52:0/3665472957 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90701a7480 con 0x7f907006aa20 2024-03-04T09:29:37.736 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:37.735+0000 7f9075825700 1 -- 172.21.15.52:0/3665472957 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f90700627d0 con 0x7f907006aa20 2024-03-04T09:29:37.737 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:37.736+0000 7f9074823700 1 -- 172.21.15.52:0/3665472957 <== mon.1 v2:172.21.15.191:3300/0 4 ==== mgrmap(e 17) v1 ==== 136807+0+0 (secure 0 0 0) 0x7f906002a070 con 0x7f907006aa20 2024-03-04T09:29:37.737 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:37.737+0000 7f9074823700 1 --2- 172.21.15.52:0/3665472957 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f90580a1ea0 0x7f90580a4360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:29:37.738 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:37.737+0000 7f906e7fc700 1 --2- 172.21.15.52:0/3665472957 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f90580a1ea0 0x7f90580a4360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:29:37.738 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:37.737+0000 7f9074823700 1 -- 172.21.15.52:0/3665472957 <== mon.1 v2:172.21.15.191:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8802+0+0 (secure 0 0 0) 0x7f90600cef60 con 0x7f907006aa20 2024-03-04T09:29:37.738 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:37.737+0000 7f9074823700 1 --2- 172.21.15.52:0/3665472957 >> [v2:172.21.15.106:6824/3815586576,v1:172.21.15.106:6825/3815586576] conn(0x7f90580acb60 0x7f90580aefa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:29:37.738 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:37.737+0000 7f9074823700 1 -- 172.21.15.52:0/3665472957 --> [v2:172.21.15.106:6824/3815586576,v1:172.21.15.106:6825/3815586576] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f90580af670 con 0x7f90580acb60 2024-03-04T09:29:37.738 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:37.737+0000 7f9074823700 1 -- 172.21.15.52:0/3665472957 <== mon.1 v2:172.21.15.191:3300/0 6 ==== mon_get_version_reply(handle=1 version=89) v2 ==== 24+0+0 (secure 0 0 0) 0x7f90600cf3b0 con 0x7f907006aa20 2024-03-04T09:29:37.738 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:37.737+0000 7f906e7fc700 1 --2- 172.21.15.52:0/3665472957 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f90580a1ea0 0x7f90580a4360 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f9070100bb0 tx=0x7f9064009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:29:37.738 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:37.737+0000 7f906f7fe700 1 --2- 172.21.15.52:0/3665472957 >> [v2:172.21.15.106:6824/3815586576,v1:172.21.15.106:6825/3815586576] conn(0x7f90580acb60 0x7f90580aefa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:29:37.738 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:37.738+0000 7f906f7fe700 1 --2- 172.21.15.52:0/3665472957 >> [v2:172.21.15.106:6824/3815586576,v1:172.21.15.106:6825/3815586576] conn(0x7f90580acb60 0x7f90580aefa0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:29:37.738 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:37.739+0000 7f9074823700 1 -- 172.21.15.52:0/3665472957 <== osd.10 v2:172.21.15.106:6824/3815586576 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f90580af670 con 0x7f90580acb60 2024-03-04T09:29:37.752 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:37.755+0000 7f9075825700 1 -- 172.21.15.52:0/3665472957 --> [v2:172.21.15.106:6824/3815586576,v1:172.21.15.106:6825/3815586576] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f9070062c50 con 0x7f90580acb60 2024-03-04T09:29:37.753 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:37.755+0000 7f9074823700 1 -- 172.21.15.52:0/3665472957 <== osd.10 v2:172.21.15.106:6824/3815586576 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f9070062c50 con 0x7f90580acb60 2024-03-04T09:29:37.755 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:37.756+0000 7f9075825700 1 -- 172.21.15.52:0/3665472957 >> [v2:172.21.15.106:6824/3815586576,v1:172.21.15.106:6825/3815586576] conn(0x7f90580acb60 msgr2=0x7f90580aefa0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:29:37.755 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:37.756+0000 7f9075825700 1 --2- 172.21.15.52:0/3665472957 >> [v2:172.21.15.106:6824/3815586576,v1:172.21.15.106:6825/3815586576] conn(0x7f90580acb60 0x7f90580aefa0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:29:37.755 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:37.756+0000 7f9075825700 1 -- 172.21.15.52:0/3665472957 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f90580a1ea0 msgr2=0x7f90580a4360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:29:37.755 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:37.756+0000 7f9075825700 1 --2- 172.21.15.52:0/3665472957 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f90580a1ea0 0x7f90580a4360 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f9070100bb0 tx=0x7f9064009260 comp rx=0 tx=0).stop 2024-03-04T09:29:37.755 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:37.756+0000 7f9075825700 1 -- 172.21.15.52:0/3665472957 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f907006aa20 msgr2=0x7f90701010f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:29:37.755 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:37.756+0000 7f9075825700 1 --2- 172.21.15.52:0/3665472957 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f907006aa20 0x7f90701010f0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f90600098e0 tx=0x7f906000fa00 comp rx=0 tx=0).stop 2024-03-04T09:29:37.755 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:37.756+0000 7f906f7fe700 1 -- 172.21.15.52:0/3665472957 reap_dead start 2024-03-04T09:29:37.755 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:37.756+0000 7f9075825700 1 -- 172.21.15.52:0/3665472957 shutdown_connections 2024-03-04T09:29:37.755 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:37.756+0000 7f9075825700 1 -- 172.21.15.52:0/3665472957 >> 172.21.15.52:0/3665472957 conn(0x7f9070077010 msgr2=0x7f90700ff1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:29:37.756 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:37.756+0000 7f9075825700 1 -- 172.21.15.52:0/3665472957 shutdown_connections 2024-03-04T09:29:37.756 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:37.756+0000 7f9075825700 1 -- 172.21.15.52:0/3665472957 wait complete. 2024-03-04T09:29:38.785 INFO:teuthology.orchestra.run.smithi052.stdout:107374182462 2024-03-04T09:29:38.786 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd shell --fsid 4e059f08-da08-11ee-95c6-87774f69a715 -- ceph osd last-stat-seq osd.3 2024-03-04T09:29:39.007 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:39.009+0000 7f142f27b700 1 -- 172.21.15.52:0/165951619 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f142806aca0 msgr2=0x7f142806b120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:29:39.007 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:39.009+0000 7f142f27b700 1 --2- 172.21.15.52:0/165951619 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f142806aca0 0x7f142806b120 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f1418009990 tx=0x7f1418009ca0 comp rx=0 tx=0).stop 2024-03-04T09:29:39.007 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:39.009+0000 7f142f27b700 1 -- 172.21.15.52:0/165951619 shutdown_connections 2024-03-04T09:29:39.007 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:39.009+0000 7f142f27b700 1 --2- 172.21.15.52:0/165951619 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f142810e910 0x7f1428110d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:29:39.007 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:39.009+0000 7f142f27b700 1 --2- 172.21.15.52:0/165951619 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1428102fb0 0x7f142806a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:29:39.007 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:39.009+0000 7f142f27b700 1 --2- 172.21.15.52:0/165951619 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f142806aca0 0x7f142806b120 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:29:39.007 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:39.009+0000 7f142f27b700 1 -- 172.21.15.52:0/165951619 >> 172.21.15.52:0/165951619 conn(0x7f1428077e80 msgr2=0x7f1428078290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:29:39.007 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:39.010+0000 7f142f27b700 1 -- 172.21.15.52:0/165951619 shutdown_connections 2024-03-04T09:29:39.008 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:39.010+0000 7f142f27b700 1 -- 172.21.15.52:0/165951619 wait complete. 2024-03-04T09:29:39.009 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:39.010+0000 7f142f27b700 1 Processor -- start 2024-03-04T09:29:39.009 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:39.011+0000 7f142f27b700 1 -- start start 2024-03-04T09:29:39.009 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:39.011+0000 7f142f27b700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1428102fb0 0x7f14281a08f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:29:39.009 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:39.011+0000 7f142f27b700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f142810e910 0x7f14281a0e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:29:39.009 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:39.011+0000 7f142f27b700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f142819aab0 0x7f142819af30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:29:39.009 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:39.011+0000 7f142f27b700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14281145c0 con 0x7f142810e910 2024-03-04T09:29:39.009 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:39.011+0000 7f142f27b700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1428114440 con 0x7f1428102fb0 2024-03-04T09:29:39.010 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:39.011+0000 7f142f27b700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1428114740 con 0x7f142819aab0 2024-03-04T09:29:39.010 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:39.011+0000 7f142c816700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f142810e910 0x7f14281a0e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:29:39.010 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:39.011+0000 7f142d017700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1428102fb0 0x7f14281a08f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:29:39.010 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:39.011+0000 7f142c816700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f142810e910 0x7f14281a0e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:57000/0 (socket says 172.21.15.52:57000) 2024-03-04T09:29:39.010 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:39.011+0000 7f142d017700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1428102fb0 0x7f14281a08f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.52:40290/0 (socket says 172.21.15.52:40290) 2024-03-04T09:29:39.010 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:39.011+0000 7f142c816700 1 -- 172.21.15.52:0/2289087895 learned_addr learned my addr 172.21.15.52:0/2289087895 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-04T09:29:39.010 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:39.011+0000 7f142d818700 1 --2- 172.21.15.52:0/2289087895 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f142819aab0 0x7f142819af30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:29:39.010 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:39.011+0000 7f142d017700 1 -- 172.21.15.52:0/2289087895 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f142819aab0 msgr2=0x7f142819af30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:29:39.010 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:39.011+0000 7f142d017700 1 --2- 172.21.15.52:0/2289087895 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f142819aab0 0x7f142819af30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:29:39.011 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:39.011+0000 7f142d017700 1 -- 172.21.15.52:0/2289087895 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f142810e910 msgr2=0x7f14281a0e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:29:39.011 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:39.011+0000 7f142d017700 1 --2- 172.21.15.52:0/2289087895 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f142810e910 0x7f14281a0e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:29:39.011 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:39.011+0000 7f142d017700 1 -- 172.21.15.52:0/2289087895 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f142819b630 con 0x7f1428102fb0 2024-03-04T09:29:39.011 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:39.012+0000 7f142d017700 1 --2- 172.21.15.52:0/2289087895 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1428102fb0 0x7f14281a08f0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f142400e9f0 tx=0x7f142400ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:29:39.011 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:39.012+0000 7f141e7fc700 1 -- 172.21.15.52:0/2289087895 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f142400c9c0 con 0x7f1428102fb0 2024-03-04T09:29:39.011 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:39.012+0000 7f141e7fc700 1 -- 172.21.15.52:0/2289087895 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f142400cb60 con 0x7f1428102fb0 2024-03-04T09:29:39.011 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:39.012+0000 7f142f27b700 1 -- 172.21.15.52:0/2289087895 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f142819b8c0 con 0x7f1428102fb0 2024-03-04T09:29:39.011 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:39.012+0000 7f142f27b700 1 -- 172.21.15.52:0/2289087895 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14281037f0 con 0x7f1428102fb0 2024-03-04T09:29:39.011 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:39.012+0000 7f141e7fc700 1 -- 172.21.15.52:0/2289087895 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1424018680 con 0x7f1428102fb0 2024-03-04T09:29:39.011 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:39.013+0000 7f142f27b700 1 -- 172.21.15.52:0/2289087895 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f140c001030 con 0x7f1428102fb0 2024-03-04T09:29:39.013 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:39.014+0000 7f141e7fc700 1 -- 172.21.15.52:0/2289087895 <== mon.2 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 17) v1 ==== 136807+0+0 (secure 0 0 0) 0x7f1424018820 con 0x7f1428102fb0 2024-03-04T09:29:39.013 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:39.015+0000 7f141e7fc700 1 --2- 172.21.15.52:0/2289087895 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f14140a1ea0 0x7f14140a4360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:29:39.013 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:39.015+0000 7f142c816700 1 --2- 172.21.15.52:0/2289087895 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f14140a1ea0 0x7f14140a4360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:29:39.013 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:39.015+0000 7f141e7fc700 1 -- 172.21.15.52:0/2289087895 <== mon.2 v2:172.21.15.106:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8802+0+0 (secure 0 0 0) 0x7f14240cee10 con 0x7f1428102fb0 2024-03-04T09:29:39.013 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:39.015+0000 7f141e7fc700 1 --2- 172.21.15.52:0/2289087895 >> [v2:172.21.15.52:6826/3485821914,v1:172.21.15.52:6827/3485821914] conn(0x7f14140acb60 0x7f14140aefa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:29:39.013 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:39.015+0000 7f141e7fc700 1 -- 172.21.15.52:0/2289087895 --> [v2:172.21.15.52:6826/3485821914,v1:172.21.15.52:6827/3485821914] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f1424017e70 con 0x7f14140acb60 2024-03-04T09:29:39.013 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:39.015+0000 7f141e7fc700 1 -- 172.21.15.52:0/2289087895 <== mon.2 v2:172.21.15.106:3300/0 6 ==== mon_get_version_reply(handle=1 version=89) v2 ==== 24+0+0 (secure 0 0 0) 0x7f14240cf1c0 con 0x7f1428102fb0 2024-03-04T09:29:39.013 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:39.015+0000 7f142d818700 1 --2- 172.21.15.52:0/2289087895 >> [v2:172.21.15.52:6826/3485821914,v1:172.21.15.52:6827/3485821914] conn(0x7f14140acb60 0x7f14140aefa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:29:39.013 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:39.015+0000 7f142c816700 1 --2- 172.21.15.52:0/2289087895 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f14140a1ea0 0x7f14140a4360 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f1428078730 tx=0x7f1418005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:29:39.013 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:39.016+0000 7f142d818700 1 --2- 172.21.15.52:0/2289087895 >> [v2:172.21.15.52:6826/3485821914,v1:172.21.15.52:6827/3485821914] conn(0x7f14140acb60 0x7f14140aefa0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:29:39.014 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:39.016+0000 7f141e7fc700 1 -- 172.21.15.52:0/2289087895 <== osd.9 v2:172.21.15.52:6826/3485821914 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f1424017e70 con 0x7f14140acb60 2024-03-04T09:29:39.030 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:39.033+0000 7f142f27b700 1 -- 172.21.15.52:0/2289087895 --> [v2:172.21.15.52:6826/3485821914,v1:172.21.15.52:6827/3485821914] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f140c002e60 con 0x7f14140acb60 2024-03-04T09:29:39.030 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:39.033+0000 7f141e7fc700 1 -- 172.21.15.52:0/2289087895 <== osd.9 v2:172.21.15.52:6826/3485821914 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f140c002e60 con 0x7f14140acb60 2024-03-04T09:29:39.031 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:39.033+0000 7f142f27b700 1 -- 172.21.15.52:0/2289087895 >> [v2:172.21.15.52:6826/3485821914,v1:172.21.15.52:6827/3485821914] conn(0x7f14140acb60 msgr2=0x7f14140aefa0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:29:39.031 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:39.033+0000 7f142f27b700 1 --2- 172.21.15.52:0/2289087895 >> [v2:172.21.15.52:6826/3485821914,v1:172.21.15.52:6827/3485821914] conn(0x7f14140acb60 0x7f14140aefa0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:29:39.031 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:39.034+0000 7f142f27b700 1 -- 172.21.15.52:0/2289087895 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f14140a1ea0 msgr2=0x7f14140a4360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:29:39.031 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:39.034+0000 7f142f27b700 1 --2- 172.21.15.52:0/2289087895 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f14140a1ea0 0x7f14140a4360 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f1428078730 tx=0x7f1418005960 comp rx=0 tx=0).stop 2024-03-04T09:29:39.031 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:39.034+0000 7f142f27b700 1 -- 172.21.15.52:0/2289087895 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1428102fb0 msgr2=0x7f14281a08f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:29:39.031 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:39.034+0000 7f142f27b700 1 --2- 172.21.15.52:0/2289087895 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1428102fb0 0x7f14281a08f0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f142400e9f0 tx=0x7f142400ed00 comp rx=0 tx=0).stop 2024-03-04T09:29:39.031 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:39.034+0000 7f142d818700 1 -- 172.21.15.52:0/2289087895 reap_dead start 2024-03-04T09:29:39.032 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:39.034+0000 7f142f27b700 1 -- 172.21.15.52:0/2289087895 shutdown_connections 2024-03-04T09:29:39.032 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:39.034+0000 7f142f27b700 1 -- 172.21.15.52:0/2289087895 >> 172.21.15.52:0/2289087895 conn(0x7f1428077e80 msgr2=0x7f1428100290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:29:39.032 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:39.034+0000 7f142f27b700 1 -- 172.21.15.52:0/2289087895 shutdown_connections 2024-03-04T09:29:39.032 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:39.034+0000 7f142f27b700 1 -- 172.21.15.52:0/2289087895 wait complete. 2024-03-04T09:29:39.270 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:29:39 smithi052 ceph-mon[32473]: pgmap v318: 129 pgs: 129 active+clean; 641 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-04T09:29:39.270 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:29:39 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T09:29:39.336 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:29:39 smithi191 ceph-mon[36816]: pgmap v318: 129 pgs: 129 active+clean; 641 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-04T09:29:39.337 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:29:39 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T09:29:39.381 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:29:39 smithi106 ceph-mon[36484]: pgmap v318: 129 pgs: 129 active+clean; 641 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-03-04T09:29:39.381 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:29:39 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T09:29:40.264 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:29:40 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 4]}]: dispatch 2024-03-04T09:29:40.264 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:29:40 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.13", "id": [4, 9]}]: dispatch 2024-03-04T09:29:40.264 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:29:40 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.20", "id": [5, 9]}]: dispatch 2024-03-04T09:29:40.264 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:29:40 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [4, 6]}]: dispatch 2024-03-04T09:29:40.265 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:29:40 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.23", "id": [4, 8]}]: dispatch 2024-03-04T09:29:40.265 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:29:40 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.38", "id": [3, 9]}]: dispatch 2024-03-04T09:29:40.265 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:29:40 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.9", "id": [5, 8]}]: dispatch 2024-03-04T09:29:40.265 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:29:40 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.17", "id": [9, 8]}]: dispatch 2024-03-04T09:29:40.265 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:29:40 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [9, 10]}]: dispatch 2024-03-04T09:29:40.265 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:29:40 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [9, 8]}]: dispatch 2024-03-04T09:29:40.336 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:29:40 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 4]}]: dispatch 2024-03-04T09:29:40.336 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:29:40 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.13", "id": [4, 9]}]: dispatch 2024-03-04T09:29:40.336 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:29:40 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.20", "id": [5, 9]}]: dispatch 2024-03-04T09:29:40.337 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:29:40 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [4, 6]}]: dispatch 2024-03-04T09:29:40.337 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:29:40 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.23", "id": [4, 8]}]: dispatch 2024-03-04T09:29:40.337 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:29:40 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.38", "id": [3, 9]}]: dispatch 2024-03-04T09:29:40.337 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:29:40 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.9", "id": [5, 8]}]: dispatch 2024-03-04T09:29:40.337 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:29:40 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.17", "id": [9, 8]}]: dispatch 2024-03-04T09:29:40.337 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:29:40 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [9, 10]}]: dispatch 2024-03-04T09:29:40.337 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:29:40 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [9, 8]}]: dispatch 2024-03-04T09:29:40.381 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:29:40 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 4]}]: dispatch 2024-03-04T09:29:40.382 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:29:40 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.13", "id": [4, 9]}]: dispatch 2024-03-04T09:29:40.382 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:29:40 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.20", "id": [5, 9]}]: dispatch 2024-03-04T09:29:40.382 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:29:40 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [4, 6]}]: dispatch 2024-03-04T09:29:40.382 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:29:40 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.23", "id": [4, 8]}]: dispatch 2024-03-04T09:29:40.382 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:29:40 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.38", "id": [3, 9]}]: dispatch 2024-03-04T09:29:40.382 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:29:40 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.9", "id": [5, 8]}]: dispatch 2024-03-04T09:29:40.382 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:29:40 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.17", "id": [9, 8]}]: dispatch 2024-03-04T09:29:40.382 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:29:40 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [9, 10]}]: dispatch 2024-03-04T09:29:40.382 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:29:40 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [9, 8]}]: dispatch 2024-03-04T09:29:40.383 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:40.379+0000 7ffb3d100700 1 -- 172.21.15.52:0/2867271924 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ffb38100a00 msgr2=0x7ffb38100e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:29:40.383 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:40.379+0000 7ffb3d100700 1 --2- 172.21.15.52:0/2867271924 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ffb38100a00 0x7ffb38100e80 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7ffb28009990 tx=0x7ffb28009ca0 comp rx=0 tx=0).stop 2024-03-04T09:29:40.383 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:40.379+0000 7ffb3d100700 1 -- 172.21.15.52:0/2867271924 shutdown_connections 2024-03-04T09:29:40.383 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:40.379+0000 7ffb3d100700 1 --2- 172.21.15.52:0/2867271924 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ffb381013c0 0x7ffb3810d600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:29:40.383 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:40.379+0000 7ffb3d100700 1 --2- 172.21.15.52:0/2867271924 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ffb381067e0 0x7ffb38106bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:29:40.383 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:40.379+0000 7ffb3d100700 1 --2- 172.21.15.52:0/2867271924 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ffb38100a00 0x7ffb38100e80 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:29:40.383 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:40.379+0000 7ffb3d100700 1 -- 172.21.15.52:0/2867271924 >> 172.21.15.52:0/2867271924 conn(0x7ffb38076300 msgr2=0x7ffb38076710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:29:40.383 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:40.379+0000 7ffb3d100700 1 -- 172.21.15.52:0/2867271924 shutdown_connections 2024-03-04T09:29:40.383 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:40.379+0000 7ffb3d100700 1 -- 172.21.15.52:0/2867271924 wait complete. 2024-03-04T09:29:40.383 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:40.380+0000 7ffb3d100700 1 Processor -- start 2024-03-04T09:29:40.384 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:40.380+0000 7ffb3d100700 1 -- start start 2024-03-04T09:29:40.384 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:40.380+0000 7ffb3d100700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ffb38100a00 0x7ffb381987a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:29:40.384 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:40.380+0000 7ffb3d100700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ffb381013c0 0x7ffb38198ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:29:40.384 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:40.380+0000 7ffb3d100700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ffb381067e0 0x7ffb3819d020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:29:40.384 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:40.380+0000 7ffb3d100700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb38110a70 con 0x7ffb381013c0 2024-03-04T09:29:40.384 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:40.380+0000 7ffb3d100700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb381108f0 con 0x7ffb38100a00 2024-03-04T09:29:40.384 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:40.380+0000 7ffb3d100700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb38110bf0 con 0x7ffb381067e0 2024-03-04T09:29:40.384 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:40.380+0000 7ffb3659c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ffb381013c0 0x7ffb38198ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:29:40.384 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:40.380+0000 7ffb3659c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ffb381013c0 0x7ffb38198ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:57016/0 (socket says 172.21.15.52:57016) 2024-03-04T09:29:40.384 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:40.380+0000 7ffb3659c700 1 -- 172.21.15.52:0/1862827529 learned_addr learned my addr 172.21.15.52:0/1862827529 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-04T09:29:40.384 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:40.381+0000 7ffb3759e700 1 --2- 172.21.15.52:0/1862827529 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ffb381067e0 0x7ffb3819d020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:29:40.384 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:40.380+0000 7ffb36d9d700 1 --2- 172.21.15.52:0/1862827529 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ffb38100a00 0x7ffb381987a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:29:40.384 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:40.381+0000 7ffb3659c700 1 -- 172.21.15.52:0/1862827529 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ffb381067e0 msgr2=0x7ffb3819d020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:29:40.385 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:40.381+0000 7ffb3659c700 1 --2- 172.21.15.52:0/1862827529 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ffb381067e0 0x7ffb3819d020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:29:40.385 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:40.381+0000 7ffb3659c700 1 -- 172.21.15.52:0/1862827529 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ffb38100a00 msgr2=0x7ffb381987a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:29:40.385 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:40.381+0000 7ffb3659c700 1 --2- 172.21.15.52:0/1862827529 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ffb38100a00 0x7ffb381987a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:29:40.385 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:40.381+0000 7ffb3659c700 1 -- 172.21.15.52:0/1862827529 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb3819d760 con 0x7ffb381013c0 2024-03-04T09:29:40.385 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:40.381+0000 7ffb3659c700 1 --2- 172.21.15.52:0/1862827529 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ffb381013c0 0x7ffb38198ce0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7ffb280038b0 tx=0x7ffb28009f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:29:40.385 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:40.381+0000 7ffb2ffff700 1 -- 172.21.15.52:0/1862827529 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb28027070 con 0x7ffb381013c0 2024-03-04T09:29:40.385 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:40.381+0000 7ffb2ffff700 1 -- 172.21.15.52:0/1862827529 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ffb28004050 con 0x7ffb381013c0 2024-03-04T09:29:40.385 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:40.381+0000 7ffb3d100700 1 -- 172.21.15.52:0/1862827529 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb3819d9f0 con 0x7ffb381013c0 2024-03-04T09:29:40.385 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:40.381+0000 7ffb2ffff700 1 -- 172.21.15.52:0/1862827529 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb28022c00 con 0x7ffb381013c0 2024-03-04T09:29:40.385 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:40.381+0000 7ffb3d100700 1 -- 172.21.15.52:0/1862827529 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb381a52d0 con 0x7ffb381013c0 2024-03-04T09:29:40.385 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:40.382+0000 7ffb3d100700 1 -- 172.21.15.52:0/1862827529 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7ffb38062850 con 0x7ffb381013c0 2024-03-04T09:29:40.385 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:40.383+0000 7ffb2ffff700 1 -- 172.21.15.52:0/1862827529 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 17) v1 ==== 136807+0+0 (secure 0 0 0) 0x7ffb2802a070 con 0x7ffb381013c0 2024-03-04T09:29:40.385 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:40.384+0000 7ffb2ffff700 1 --2- 172.21.15.52:0/1862827529 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7ffb240a1ea0 0x7ffb240a4360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:29:40.385 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:40.384+0000 7ffb36d9d700 1 --2- 172.21.15.52:0/1862827529 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7ffb240a1ea0 0x7ffb240a4360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:29:40.386 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:40.384+0000 7ffb2ffff700 1 -- 172.21.15.52:0/1862827529 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9092+0+0 (secure 0 0 0) 0x7ffb280cfc30 con 0x7ffb381013c0 2024-03-04T09:29:40.386 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:40.384+0000 7ffb2ffff700 1 --2- 172.21.15.52:0/1862827529 >> [v2:172.21.15.191:6816/1114119072,v1:172.21.15.191:6817/1114119072] conn(0x7ffb240ad000 0x7ffb240af4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:29:40.386 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:40.384+0000 7ffb2ffff700 1 -- 172.21.15.52:0/1862827529 --> [v2:172.21.15.191:6816/1114119072,v1:172.21.15.191:6817/1114119072] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7ffb240afb70 con 0x7ffb240ad000 2024-03-04T09:29:40.386 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:40.384+0000 7ffb2ffff700 1 -- 172.21.15.52:0/1862827529 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) v2 ==== 24+0+0 (secure 0 0 0) 0x7ffb280d0080 con 0x7ffb381013c0 2024-03-04T09:29:40.386 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:40.384+0000 7ffb36d9d700 1 --2- 172.21.15.52:0/1862827529 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7ffb240a1ea0 0x7ffb240a4360 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7ffb20005950 tx=0x7ffb2000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:29:40.386 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:40.384+0000 7ffb3759e700 1 --2- 172.21.15.52:0/1862827529 >> [v2:172.21.15.191:6816/1114119072,v1:172.21.15.191:6817/1114119072] conn(0x7ffb240ad000 0x7ffb240af4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:29:40.386 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:40.385+0000 7ffb3759e700 1 --2- 172.21.15.52:0/1862827529 >> [v2:172.21.15.191:6816/1114119072,v1:172.21.15.191:6817/1114119072] conn(0x7ffb240ad000 0x7ffb240af4a0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:29:40.386 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:40.385+0000 7ffb2ffff700 1 -- 172.21.15.52:0/1862827529 <== osd.8 v2:172.21.15.191:6816/1114119072 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7ffb240afb70 con 0x7ffb240ad000 2024-03-04T09:29:40.398 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:40.402+0000 7ffb3d100700 1 -- 172.21.15.52:0/1862827529 --> [v2:172.21.15.191:6816/1114119072,v1:172.21.15.191:6817/1114119072] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7ffb38062cd0 con 0x7ffb240ad000 2024-03-04T09:29:40.399 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:40.402+0000 7ffb2ffff700 1 -- 172.21.15.52:0/1862827529 <== osd.8 v2:172.21.15.191:6816/1114119072 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7ffb38062cd0 con 0x7ffb240ad000 2024-03-04T09:29:40.400 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:40.403+0000 7ffb3d100700 1 -- 172.21.15.52:0/1862827529 >> [v2:172.21.15.191:6816/1114119072,v1:172.21.15.191:6817/1114119072] conn(0x7ffb240ad000 msgr2=0x7ffb240af4a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:29:40.400 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:40.403+0000 7ffb3d100700 1 --2- 172.21.15.52:0/1862827529 >> [v2:172.21.15.191:6816/1114119072,v1:172.21.15.191:6817/1114119072] conn(0x7ffb240ad000 0x7ffb240af4a0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:29:40.400 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:40.403+0000 7ffb3d100700 1 -- 172.21.15.52:0/1862827529 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7ffb240a1ea0 msgr2=0x7ffb240a4360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:29:40.400 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:40.403+0000 7ffb3d100700 1 --2- 172.21.15.52:0/1862827529 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7ffb240a1ea0 0x7ffb240a4360 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7ffb20005950 tx=0x7ffb2000b410 comp rx=0 tx=0).stop 2024-03-04T09:29:40.400 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:40.403+0000 7ffb3d100700 1 -- 172.21.15.52:0/1862827529 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ffb381013c0 msgr2=0x7ffb38198ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:29:40.400 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:40.403+0000 7ffb3d100700 1 --2- 172.21.15.52:0/1862827529 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ffb381013c0 0x7ffb38198ce0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7ffb280038b0 tx=0x7ffb28009f50 comp rx=0 tx=0).stop 2024-03-04T09:29:40.400 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:40.403+0000 7ffb3759e700 1 -- 172.21.15.52:0/1862827529 reap_dead start 2024-03-04T09:29:40.400 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:40.403+0000 7ffb3d100700 1 -- 172.21.15.52:0/1862827529 shutdown_connections 2024-03-04T09:29:40.400 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:40.403+0000 7ffb3d100700 1 -- 172.21.15.52:0/1862827529 >> 172.21.15.52:0/1862827529 conn(0x7ffb38076300 msgr2=0x7ffb381075b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:29:40.400 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:40.403+0000 7ffb3d100700 1 -- 172.21.15.52:0/1862827529 shutdown_connections 2024-03-04T09:29:40.401 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:40.403+0000 7ffb3d100700 1 -- 172.21.15.52:0/1862827529 wait complete. 2024-03-04T09:29:40.517 INFO:teuthology.orchestra.run.smithi052.stdout:287762808868 2024-03-04T09:29:40.517 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd shell --fsid 4e059f08-da08-11ee-95c6-87774f69a715 -- ceph osd last-stat-seq osd.9 2024-03-04T09:29:40.818 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:40.793+0000 7f2d3ee3c700 1 -- 172.21.15.52:0/4141457177 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2d38101960 msgr2=0x7f2d3810de40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:29:40.818 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:40.793+0000 7f2d3ee3c700 1 --2- 172.21.15.52:0/4141457177 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2d38101960 0x7f2d3810de40 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f2d3000b0a0 tx=0x7f2d3000b3b0 comp rx=0 tx=0).stop 2024-03-04T09:29:40.818 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:40.793+0000 7f2d3ee3c700 1 -- 172.21.15.52:0/4141457177 shutdown_connections 2024-03-04T09:29:40.818 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:40.793+0000 7f2d3ee3c700 1 --2- 172.21.15.52:0/4141457177 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2d38107000 0x7f2d381073e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:29:40.818 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:40.793+0000 7f2d3ee3c700 1 --2- 172.21.15.52:0/4141457177 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2d38101960 0x7f2d3810de40 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:29:40.819 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:40.793+0000 7f2d3ee3c700 1 --2- 172.21.15.52:0/4141457177 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2d38100fc0 0x7f2d38101420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:29:40.819 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:40.793+0000 7f2d3ee3c700 1 -- 172.21.15.52:0/4141457177 >> 172.21.15.52:0/4141457177 conn(0x7f2d38077ef0 msgr2=0x7f2d38078300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:29:40.819 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:40.793+0000 7f2d3ee3c700 1 -- 172.21.15.52:0/4141457177 shutdown_connections 2024-03-04T09:29:40.819 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:40.793+0000 7f2d3ee3c700 1 -- 172.21.15.52:0/4141457177 wait complete. 2024-03-04T09:29:40.819 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:40.794+0000 7f2d3ee3c700 1 Processor -- start 2024-03-04T09:29:40.819 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:40.794+0000 7f2d3ee3c700 1 -- start start 2024-03-04T09:29:40.819 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:40.794+0000 7f2d3ee3c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2d38100fc0 0x7f2d38199660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:29:40.819 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:40.794+0000 7f2d3ee3c700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2d38101960 0x7f2d38199ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:29:40.819 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:40.794+0000 7f2d3ee3c700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2d38107000 0x7f2d381a46e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:29:40.819 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:40.794+0000 7f2d3ee3c700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d381112a0 con 0x7f2d38100fc0 2024-03-04T09:29:40.819 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:40.794+0000 7f2d3ee3c700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d38111120 con 0x7f2d38107000 2024-03-04T09:29:40.819 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:40.794+0000 7f2d3ee3c700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d38111420 con 0x7f2d38101960 2024-03-04T09:29:40.819 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:40.794+0000 7f2d3cbd8700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2d38100fc0 0x7f2d38199660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:29:40.820 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:40.794+0000 7f2d3cbd8700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2d38100fc0 0x7f2d38199660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:57026/0 (socket says 172.21.15.52:57026) 2024-03-04T09:29:40.820 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:40.794+0000 7f2d3cbd8700 1 -- 172.21.15.52:0/3353502872 learned_addr learned my addr 172.21.15.52:0/3353502872 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-04T09:29:40.820 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:40.795+0000 7f2d37fff700 1 --2- 172.21.15.52:0/3353502872 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2d38101960 0x7f2d38199ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:29:40.820 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:40.795+0000 7f2d3d3d9700 1 --2- 172.21.15.52:0/3353502872 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2d38107000 0x7f2d381a46e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:29:40.820 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:40.795+0000 7f2d3cbd8700 1 -- 172.21.15.52:0/3353502872 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2d38101960 msgr2=0x7f2d38199ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:29:40.820 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:40.795+0000 7f2d3cbd8700 1 --2- 172.21.15.52:0/3353502872 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2d38101960 0x7f2d38199ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:29:40.820 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:40.795+0000 7f2d3cbd8700 1 -- 172.21.15.52:0/3353502872 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2d38107000 msgr2=0x7f2d381a46e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:29:40.820 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:40.795+0000 7f2d3cbd8700 1 --2- 172.21.15.52:0/3353502872 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2d38107000 0x7f2d381a46e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:29:40.820 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:40.795+0000 7f2d3cbd8700 1 -- 172.21.15.52:0/3353502872 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d381a4c20 con 0x7f2d38100fc0 2024-03-04T09:29:40.820 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:40.795+0000 7f2d3d3d9700 1 --2- 172.21.15.52:0/3353502872 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2d38107000 0x7f2d381a46e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T09:29:40.820 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:40.795+0000 7f2d3cbd8700 1 --2- 172.21.15.52:0/3353502872 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2d38100fc0 0x7f2d38199660 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f2d2c00e9a0 tx=0x7f2d2c00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:29:40.820 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:40.795+0000 7f2d35ffb700 1 -- 172.21.15.52:0/3353502872 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d2c00cbd0 con 0x7f2d38100fc0 2024-03-04T09:29:40.821 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:40.795+0000 7f2d35ffb700 1 -- 172.21.15.52:0/3353502872 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2d2c00cd70 con 0x7f2d38100fc0 2024-03-04T09:29:40.821 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:40.795+0000 7f2d35ffb700 1 -- 172.21.15.52:0/3353502872 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d2c018750 con 0x7f2d38100fc0 2024-03-04T09:29:40.821 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:40.795+0000 7f2d3ee3c700 1 -- 172.21.15.52:0/3353502872 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d381a4e50 con 0x7f2d38100fc0 2024-03-04T09:29:40.821 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:40.795+0000 7f2d3ee3c700 1 -- 172.21.15.52:0/3353502872 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d381a52b0 con 0x7f2d38100fc0 2024-03-04T09:29:40.821 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:40.796+0000 7f2d3ee3c700 1 -- 172.21.15.52:0/3353502872 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f2d38062750 con 0x7f2d38100fc0 2024-03-04T09:29:40.821 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:40.797+0000 7f2d35ffb700 1 -- 172.21.15.52:0/3353502872 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 17) v1 ==== 136807+0+0 (secure 0 0 0) 0x7f2d2c010430 con 0x7f2d38100fc0 2024-03-04T09:29:40.822 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:40.798+0000 7f2d35ffb700 1 --2- 172.21.15.52:0/3353502872 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f2d280a1f20 0x7f2d280a43e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:29:40.822 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:40.798+0000 7f2d37fff700 1 --2- 172.21.15.52:0/3353502872 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f2d280a1f20 0x7f2d280a43e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:29:40.822 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:40.798+0000 7f2d35ffb700 1 -- 172.21.15.52:0/3353502872 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9092+0+0 (secure 0 0 0) 0x7f2d2c0cec60 con 0x7f2d38100fc0 2024-03-04T09:29:40.822 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:40.798+0000 7f2d35ffb700 1 --2- 172.21.15.52:0/3353502872 >> [v2:172.21.15.191:6800/1377083164,v1:172.21.15.191:6801/1377083164] conn(0x7f2d280ad080 0x7f2d280af520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:29:40.822 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:40.798+0000 7f2d35ffb700 1 -- 172.21.15.52:0/3353502872 --> [v2:172.21.15.191:6800/1377083164,v1:172.21.15.191:6801/1377083164] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f2d2c00ae50 con 0x7f2d280ad080 2024-03-04T09:29:40.822 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:40.798+0000 7f2d35ffb700 1 -- 172.21.15.52:0/3353502872 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2d2c0cf010 con 0x7f2d38100fc0 2024-03-04T09:29:40.822 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:40.798+0000 7f2d37fff700 1 --2- 172.21.15.52:0/3353502872 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f2d280a1f20 0x7f2d280a43e0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f2d38078510 tx=0x7f2d2400a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:29:40.822 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:40.799+0000 7f2d3d3d9700 1 --2- 172.21.15.52:0/3353502872 >> [v2:172.21.15.191:6800/1377083164,v1:172.21.15.191:6801/1377083164] conn(0x7f2d280ad080 0x7f2d280af520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:29:40.822 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:40.799+0000 7f2d3d3d9700 1 --2- 172.21.15.52:0/3353502872 >> [v2:172.21.15.191:6800/1377083164,v1:172.21.15.191:6801/1377083164] conn(0x7f2d280ad080 0x7f2d280af520 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:29:40.822 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:40.800+0000 7f2d35ffb700 1 -- 172.21.15.52:0/3353502872 <== osd.2 v2:172.21.15.191:6800/1377083164 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f2d2c00ae50 con 0x7f2d280ad080 2024-03-04T09:29:40.822 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:40.816+0000 7f2d3ee3c700 1 -- 172.21.15.52:0/3353502872 --> [v2:172.21.15.191:6800/1377083164,v1:172.21.15.191:6801/1377083164] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f2d38062ba0 con 0x7f2d280ad080 2024-03-04T09:29:40.822 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:40.816+0000 7f2d35ffb700 1 -- 172.21.15.52:0/3353502872 <== osd.2 v2:172.21.15.191:6800/1377083164 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f2d38062ba0 con 0x7f2d280ad080 2024-03-04T09:29:40.822 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:40.817+0000 7f2d3ee3c700 1 -- 172.21.15.52:0/3353502872 >> [v2:172.21.15.191:6800/1377083164,v1:172.21.15.191:6801/1377083164] conn(0x7f2d280ad080 msgr2=0x7f2d280af520 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:29:40.822 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:40.817+0000 7f2d3ee3c700 1 --2- 172.21.15.52:0/3353502872 >> [v2:172.21.15.191:6800/1377083164,v1:172.21.15.191:6801/1377083164] conn(0x7f2d280ad080 0x7f2d280af520 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:29:40.823 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:40.817+0000 7f2d3ee3c700 1 -- 172.21.15.52:0/3353502872 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f2d280a1f20 msgr2=0x7f2d280a43e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:29:40.823 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:40.817+0000 7f2d3ee3c700 1 --2- 172.21.15.52:0/3353502872 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f2d280a1f20 0x7f2d280a43e0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f2d38078510 tx=0x7f2d2400a380 comp rx=0 tx=0).stop 2024-03-04T09:29:40.823 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:40.817+0000 7f2d3ee3c700 1 -- 172.21.15.52:0/3353502872 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2d38100fc0 msgr2=0x7f2d38199660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:29:40.823 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:40.817+0000 7f2d3ee3c700 1 --2- 172.21.15.52:0/3353502872 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2d38100fc0 0x7f2d38199660 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f2d2c00e9a0 tx=0x7f2d2c00ecb0 comp rx=0 tx=0).stop 2024-03-04T09:29:40.823 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:40.817+0000 7f2d3d3d9700 1 -- 172.21.15.52:0/3353502872 reap_dead start 2024-03-04T09:29:40.823 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:40.817+0000 7f2d3ee3c700 1 -- 172.21.15.52:0/3353502872 shutdown_connections 2024-03-04T09:29:40.823 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:40.817+0000 7f2d3ee3c700 1 -- 172.21.15.52:0/3353502872 >> 172.21.15.52:0/3353502872 conn(0x7f2d38077ef0 msgr2=0x7f2d380ffb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:29:40.823 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:40.817+0000 7f2d3ee3c700 1 -- 172.21.15.52:0/3353502872 shutdown_connections 2024-03-04T09:29:40.823 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:40.817+0000 7f2d3ee3c700 1 -- 172.21.15.52:0/3353502872 wait complete. 2024-03-04T09:29:41.279 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:41.060+0000 7f57ee85b700 1 -- 172.21.15.52:0/270673548 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f57e8102b00 msgr2=0x7f57e810ef70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:29:41.279 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:41.060+0000 7f57ee85b700 1 --2- 172.21.15.52:0/270673548 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f57e8102b00 0x7f57e810ef70 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f57e000b0d0 tx=0x7f57e000b3e0 comp rx=0 tx=0).stop 2024-03-04T09:29:41.279 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:41.060+0000 7f57ee85b700 1 -- 172.21.15.52:0/270673548 shutdown_connections 2024-03-04T09:29:41.279 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:41.060+0000 7f57ee85b700 1 --2- 172.21.15.52:0/270673548 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f57e8102b00 0x7f57e810ef70 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:29:41.279 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:41.060+0000 7f57ee85b700 1 --2- 172.21.15.52:0/270673548 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f57e8108150 0x7f57e8108530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:29:41.279 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:41.060+0000 7f57ee85b700 1 --2- 172.21.15.52:0/270673548 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f57e8102140 0x7f57e81025c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:29:41.280 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:41.060+0000 7f57ee85b700 1 -- 172.21.15.52:0/270673548 >> 172.21.15.52:0/270673548 conn(0x7f57e8079220 msgr2=0x7f57e8079630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:29:41.280 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:41.060+0000 7f57ee85b700 1 -- 172.21.15.52:0/270673548 shutdown_connections 2024-03-04T09:29:41.280 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:41.060+0000 7f57ee85b700 1 -- 172.21.15.52:0/270673548 wait complete. 2024-03-04T09:29:41.280 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:41.061+0000 7f57ee85b700 1 Processor -- start 2024-03-04T09:29:41.280 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:41.061+0000 7f57ee85b700 1 -- start start 2024-03-04T09:29:41.280 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:41.061+0000 7f57ee85b700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f57e8102140 0x7f57e819e410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:29:41.280 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:41.061+0000 7f57ee85b700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f57e8102b00 0x7f57e819e950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:29:41.280 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:41.061+0000 7f57ee85b700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f57e8108150 0x7f57e81985d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:29:41.280 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:41.061+0000 7f57ee85b700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57e8112450 con 0x7f57e8102140 2024-03-04T09:29:41.280 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:41.061+0000 7f57ee85b700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57e81122d0 con 0x7f57e8102b00 2024-03-04T09:29:41.280 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:41.061+0000 7f57ee85b700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57e81125d0 con 0x7f57e8108150 2024-03-04T09:29:41.280 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:41.061+0000 7f57e7fff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f57e8102140 0x7f57e819e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:29:41.280 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:41.062+0000 7f57e7fff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f57e8102140 0x7f57e819e410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:57044/0 (socket says 172.21.15.52:57044) 2024-03-04T09:29:41.280 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:41.062+0000 7f57e7fff700 1 -- 172.21.15.52:0/4028127306 learned_addr learned my addr 172.21.15.52:0/4028127306 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-04T09:29:41.281 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:41.062+0000 7f57ecdf8700 1 --2- 172.21.15.52:0/4028127306 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f57e8108150 0x7f57e81985d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:29:41.281 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:41.062+0000 7f57e7fff700 1 -- 172.21.15.52:0/4028127306 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f57e8108150 msgr2=0x7f57e81985d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:29:41.281 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:41.062+0000 7f57e7fff700 1 --2- 172.21.15.52:0/4028127306 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f57e8108150 0x7f57e81985d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:29:41.281 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:41.062+0000 7f57e7fff700 1 -- 172.21.15.52:0/4028127306 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f57e8102b00 msgr2=0x7f57e819e950 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-04T09:29:41.281 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:41.062+0000 7f57e7fff700 1 --2- 172.21.15.52:0/4028127306 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f57e8102b00 0x7f57e819e950 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:29:41.281 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:41.062+0000 7f57e7fff700 1 -- 172.21.15.52:0/4028127306 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57e8198d30 con 0x7f57e8102140 2024-03-04T09:29:41.281 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:41.062+0000 7f57e7fff700 1 --2- 172.21.15.52:0/4028127306 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f57e8102140 0x7f57e819e410 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f57d800e9a0 tx=0x7f57d800ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:29:41.282 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:41.064+0000 7f57e57fa700 1 -- 172.21.15.52:0/4028127306 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57d8020070 con 0x7f57e8102140 2024-03-04T09:29:41.282 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:41.064+0000 7f57e57fa700 1 -- 172.21.15.52:0/4028127306 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f57d800cca0 con 0x7f57e8102140 2024-03-04T09:29:41.282 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:41.064+0000 7f57e57fa700 1 -- 172.21.15.52:0/4028127306 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57d801b9a0 con 0x7f57e8102140 2024-03-04T09:29:41.282 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:41.064+0000 7f57ee85b700 1 -- 172.21.15.52:0/4028127306 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57e8199020 con 0x7f57e8102140 2024-03-04T09:29:41.282 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:41.064+0000 7f57ee85b700 1 -- 172.21.15.52:0/4028127306 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57e80742f0 con 0x7f57e8102140 2024-03-04T09:29:41.282 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:41.065+0000 7f57ee85b700 1 -- 172.21.15.52:0/4028127306 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f57e81014c0 con 0x7f57e8102140 2024-03-04T09:29:41.282 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:41.068+0000 7f57e57fa700 1 -- 172.21.15.52:0/4028127306 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 17) v1 ==== 136807+0+0 (secure 0 0 0) 0x7f57d8023070 con 0x7f57e8102140 2024-03-04T09:29:41.282 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:41.069+0000 7f57e57fa700 1 --2- 172.21.15.52:0/4028127306 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f57d00a1ff0 0x7f57d00a44b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:29:41.282 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:41.070+0000 7f57e77fe700 1 --2- 172.21.15.52:0/4028127306 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f57d00a1ff0 0x7f57d00a44b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:29:41.282 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:41.070+0000 7f57e77fe700 1 --2- 172.21.15.52:0/4028127306 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f57d00a1ff0 0x7f57d00a44b0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f57e80796d0 tx=0x7f57dc009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:29:41.282 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:41.071+0000 7f57e57fa700 1 -- 172.21.15.52:0/4028127306 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9092+0+0 (secure 0 0 0) 0x7f57d809d050 con 0x7f57e8102140 2024-03-04T09:29:41.282 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:41.072+0000 7f57e57fa700 1 --2- 172.21.15.52:0/4028127306 >> [v2:172.21.15.191:6824/2009632843,v1:172.21.15.191:6825/2009632843] conn(0x7f57d00ad0d0 0x7f57d00af570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:29:41.282 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:41.072+0000 7f57e57fa700 1 -- 172.21.15.52:0/4028127306 --> [v2:172.21.15.191:6824/2009632843,v1:172.21.15.191:6825/2009632843] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f57d801fec0 con 0x7f57d00ad0d0 2024-03-04T09:29:41.283 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:41.072+0000 7f57e57fa700 1 -- 172.21.15.52:0/4028127306 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_get_version_reply(handle=1 version=91) v2 ==== 24+0+0 (secure 0 0 0) 0x7f57d809b250 con 0x7f57e8102140 2024-03-04T09:29:41.283 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:41.072+0000 7f57ecdf8700 1 --2- 172.21.15.52:0/4028127306 >> [v2:172.21.15.191:6824/2009632843,v1:172.21.15.191:6825/2009632843] conn(0x7f57d00ad0d0 0x7f57d00af570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:29:41.283 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:41.073+0000 7f57ecdf8700 1 --2- 172.21.15.52:0/4028127306 >> [v2:172.21.15.191:6824/2009632843,v1:172.21.15.191:6825/2009632843] conn(0x7f57d00ad0d0 0x7f57d00af570 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:29:41.283 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:41.073+0000 7f57e57fa700 1 -- 172.21.15.52:0/4028127306 <== osd.11 v2:172.21.15.191:6824/2009632843 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f57d801fec0 con 0x7f57d00ad0d0 2024-03-04T09:29:41.283 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:41.090+0000 7f57ee85b700 1 -- 172.21.15.52:0/4028127306 --> [v2:172.21.15.191:6824/2009632843,v1:172.21.15.191:6825/2009632843] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f57e8103800 con 0x7f57d00ad0d0 2024-03-04T09:29:41.283 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:41.091+0000 7f57e57fa700 1 -- 172.21.15.52:0/4028127306 <== osd.11 v2:172.21.15.191:6824/2009632843 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f57e8103800 con 0x7f57d00ad0d0 2024-03-04T09:29:41.283 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:41.091+0000 7f57ee85b700 1 -- 172.21.15.52:0/4028127306 >> [v2:172.21.15.191:6824/2009632843,v1:172.21.15.191:6825/2009632843] conn(0x7f57d00ad0d0 msgr2=0x7f57d00af570 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:29:41.283 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:41.091+0000 7f57ee85b700 1 --2- 172.21.15.52:0/4028127306 >> [v2:172.21.15.191:6824/2009632843,v1:172.21.15.191:6825/2009632843] conn(0x7f57d00ad0d0 0x7f57d00af570 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:29:41.283 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:41.091+0000 7f57ee85b700 1 -- 172.21.15.52:0/4028127306 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f57d00a1ff0 msgr2=0x7f57d00a44b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:29:41.283 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:41.091+0000 7f57ee85b700 1 --2- 172.21.15.52:0/4028127306 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f57d00a1ff0 0x7f57d00a44b0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f57e80796d0 tx=0x7f57dc009260 comp rx=0 tx=0).stop 2024-03-04T09:29:41.283 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:41.091+0000 7f57ee85b700 1 -- 172.21.15.52:0/4028127306 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f57e8102140 msgr2=0x7f57e819e410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:29:41.283 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:41.091+0000 7f57ee85b700 1 --2- 172.21.15.52:0/4028127306 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f57e8102140 0x7f57e819e410 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f57d800e9a0 tx=0x7f57d800ed60 comp rx=0 tx=0).stop 2024-03-04T09:29:41.283 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:41.091+0000 7f57ecdf8700 1 -- 172.21.15.52:0/4028127306 reap_dead start 2024-03-04T09:29:41.283 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:41.091+0000 7f57ee85b700 1 -- 172.21.15.52:0/4028127306 shutdown_connections 2024-03-04T09:29:41.284 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:41.091+0000 7f57ee85b700 1 -- 172.21.15.52:0/4028127306 >> 172.21.15.52:0/4028127306 conn(0x7f57e8079220 msgr2=0x7f57e8100aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:29:41.284 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:41.091+0000 7f57ee85b700 1 -- 172.21.15.52:0/4028127306 shutdown_connections 2024-03-04T09:29:41.284 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:41.091+0000 7f57ee85b700 1 -- 172.21.15.52:0/4028127306 wait complete. 2024-03-04T09:29:41.335 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:29:41 smithi052 ceph-mon[32473]: pgmap v319: 129 pgs: 129 active+clean; 641 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-04T09:29:41.335 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:29:41 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 4]}]': finished 2024-03-04T09:29:41.335 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:29:41 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.13", "id": [4, 9]}]': finished 2024-03-04T09:29:41.335 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:29:41 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.20", "id": [5, 9]}]': finished 2024-03-04T09:29:41.335 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:29:41 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [4, 6]}]': finished 2024-03-04T09:29:41.335 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:29:41 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.23", "id": [4, 8]}]': finished 2024-03-04T09:29:41.336 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:29:41 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.38", "id": [3, 9]}]': finished 2024-03-04T09:29:41.336 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:29:41 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.9", "id": [5, 8]}]': finished 2024-03-04T09:29:41.336 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:29:41 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.17", "id": [9, 8]}]': finished 2024-03-04T09:29:41.336 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:29:41 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [9, 10]}]': finished 2024-03-04T09:29:41.336 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:29:41 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [9, 8]}]': finished 2024-03-04T09:29:41.336 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:29:41 smithi052 ceph-mon[32473]: osdmap e90: 12 total, 12 up, 12 in 2024-03-04T09:29:41.336 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:29:41 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:29:41.336 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:29:41 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:29:41.336 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:29:41 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:29:41.336 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:29:41 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:29:41.337 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:29:41 smithi191 ceph-mon[36816]: pgmap v319: 129 pgs: 129 active+clean; 641 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-04T09:29:41.337 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:29:41 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 4]}]': finished 2024-03-04T09:29:41.337 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:29:41 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.13", "id": [4, 9]}]': finished 2024-03-04T09:29:41.337 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:29:41 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.20", "id": [5, 9]}]': finished 2024-03-04T09:29:41.337 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:29:41 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [4, 6]}]': finished 2024-03-04T09:29:41.337 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:29:41 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.23", "id": [4, 8]}]': finished 2024-03-04T09:29:41.337 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:29:41 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.38", "id": [3, 9]}]': finished 2024-03-04T09:29:41.337 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:29:41 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.9", "id": [5, 8]}]': finished 2024-03-04T09:29:41.337 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:29:41 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.17", "id": [9, 8]}]': finished 2024-03-04T09:29:41.337 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:29:41 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [9, 10]}]': finished 2024-03-04T09:29:41.337 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:29:41 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [9, 8]}]': finished 2024-03-04T09:29:41.337 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:29:41 smithi191 ceph-mon[36816]: osdmap e90: 12 total, 12 up, 12 in 2024-03-04T09:29:41.338 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:29:41 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:29:41.338 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:29:41 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:29:41.338 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:29:41 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:29:41.338 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:29:41 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:29:41.381 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:29:41 smithi106 ceph-mon[36484]: pgmap v319: 129 pgs: 129 active+clean; 641 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-04T09:29:41.381 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:29:41 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 4]}]': finished 2024-03-04T09:29:41.381 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:29:41 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.13", "id": [4, 9]}]': finished 2024-03-04T09:29:41.381 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:29:41 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.20", "id": [5, 9]}]': finished 2024-03-04T09:29:41.381 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:29:41 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [4, 6]}]': finished 2024-03-04T09:29:41.381 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:29:41 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.23", "id": [4, 8]}]': finished 2024-03-04T09:29:41.382 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:29:41 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.38", "id": [3, 9]}]': finished 2024-03-04T09:29:41.382 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:29:41 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.9", "id": [5, 8]}]': finished 2024-03-04T09:29:41.382 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:29:41 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.17", "id": [9, 8]}]': finished 2024-03-04T09:29:41.382 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:29:41 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [9, 10]}]': finished 2024-03-04T09:29:41.382 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:29:41 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [9, 8]}]': finished 2024-03-04T09:29:41.382 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:29:41 smithi106 ceph-mon[36484]: osdmap e90: 12 total, 12 up, 12 in 2024-03-04T09:29:41.382 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:29:41 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:29:41.382 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:29:41 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:29:41.382 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:29:41 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:29:41.383 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:29:41 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:29:42.335 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:29:42 smithi052 ceph-mon[32473]: osdmap e91: 12 total, 12 up, 12 in 2024-03-04T09:29:42.336 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:29:42 smithi191 ceph-mon[36816]: osdmap e91: 12 total, 12 up, 12 in 2024-03-04T09:29:42.363 INFO:teuthology.orchestra.run.smithi052.stdout:77309411399 2024-03-04T09:29:42.364 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd shell --fsid 4e059f08-da08-11ee-95c6-87774f69a715 -- ceph osd last-stat-seq osd.2 2024-03-04T09:29:42.381 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:29:42 smithi106 ceph-mon[36484]: osdmap e91: 12 total, 12 up, 12 in 2024-03-04T09:29:42.956 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:42.958+0000 7fa16aaf5700 1 -- 172.21.15.52:0/3095281507 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa164100fa0 msgr2=0x7fa164101420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:29:42.956 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:42.958+0000 7fa16aaf5700 1 --2- 172.21.15.52:0/3095281507 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa164100fa0 0x7fa164101420 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fa14c009990 tx=0x7fa14c009ca0 comp rx=0 tx=0).stop 2024-03-04T09:29:42.956 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:42.958+0000 7fa16aaf5700 1 -- 172.21.15.52:0/3095281507 shutdown_connections 2024-03-04T09:29:42.956 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:42.958+0000 7fa16aaf5700 1 --2- 172.21.15.52:0/3095281507 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa164107050 0x7fa164107430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:29:42.956 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:42.958+0000 7fa16aaf5700 1 --2- 172.21.15.52:0/3095281507 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa164101960 0x7fa16410de00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:29:42.957 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:42.958+0000 7fa16aaf5700 1 --2- 172.21.15.52:0/3095281507 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa164100fa0 0x7fa164101420 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:29:42.957 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:42.958+0000 7fa16aaf5700 1 -- 172.21.15.52:0/3095281507 >> 172.21.15.52:0/3095281507 conn(0x7fa164077fd0 msgr2=0x7fa1640783e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:29:42.957 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:42.958+0000 7fa16aaf5700 1 -- 172.21.15.52:0/3095281507 shutdown_connections 2024-03-04T09:29:42.957 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:42.958+0000 7fa16aaf5700 1 -- 172.21.15.52:0/3095281507 wait complete. 2024-03-04T09:29:42.957 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:42.959+0000 7fa16aaf5700 1 Processor -- start 2024-03-04T09:29:42.958 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:42.959+0000 7fa16aaf5700 1 -- start start 2024-03-04T09:29:42.958 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:42.959+0000 7fa16aaf5700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa164100fa0 0x7fa1641976f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:29:42.958 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:42.959+0000 7fa16aaf5700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa164101960 0x7fa164197c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:29:42.958 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:42.959+0000 7fa16aaf5700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa164107050 0x7fa16419bfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:29:42.958 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:42.959+0000 7fa16aaf5700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa164111280 con 0x7fa164107050 2024-03-04T09:29:42.958 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:42.959+0000 7fa16aaf5700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa164111100 con 0x7fa164100fa0 2024-03-04T09:29:42.958 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:42.959+0000 7fa16aaf5700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa164111400 con 0x7fa164101960 2024-03-04T09:29:42.958 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:42.959+0000 7fa169092700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa164107050 0x7fa16419bfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:29:42.958 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:42.959+0000 7fa169092700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa164107050 0x7fa16419bfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:57058/0 (socket says 172.21.15.52:57058) 2024-03-04T09:29:42.958 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:42.959+0000 7fa169092700 1 -- 172.21.15.52:0/149044934 learned_addr learned my addr 172.21.15.52:0/149044934 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-04T09:29:42.959 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:42.959+0000 7fa168891700 1 --2- 172.21.15.52:0/149044934 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa164100fa0 0x7fa1641976f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:29:42.960 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:42.959+0000 7fa163fff700 1 --2- 172.21.15.52:0/149044934 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa164101960 0x7fa164197c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:29:42.960 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:42.960+0000 7fa169092700 1 -- 172.21.15.52:0/149044934 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa164101960 msgr2=0x7fa164197c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:29:42.960 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:42.960+0000 7fa169092700 1 --2- 172.21.15.52:0/149044934 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa164101960 0x7fa164197c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:29:42.960 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:42.960+0000 7fa169092700 1 -- 172.21.15.52:0/149044934 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa164100fa0 msgr2=0x7fa1641976f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:29:42.960 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:42.960+0000 7fa169092700 1 --2- 172.21.15.52:0/149044934 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa164100fa0 0x7fa1641976f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:29:42.960 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:42.960+0000 7fa169092700 1 -- 172.21.15.52:0/149044934 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa16419c700 con 0x7fa164107050 2024-03-04T09:29:42.960 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:42.960+0000 7fa168891700 1 --2- 172.21.15.52:0/149044934 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa164100fa0 0x7fa1641976f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-04T09:29:42.960 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:42.960+0000 7fa169092700 1 --2- 172.21.15.52:0/149044934 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa164107050 0x7fa16419bfc0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fa15c00e940 tx=0x7fa15c00ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:29:42.960 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:42.960+0000 7fa161ffb700 1 -- 172.21.15.52:0/149044934 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa15c004a30 con 0x7fa164107050 2024-03-04T09:29:42.960 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:42.960+0000 7fa161ffb700 1 -- 172.21.15.52:0/149044934 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa15c005500 con 0x7fa164107050 2024-03-04T09:29:42.960 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:42.960+0000 7fa16aaf5700 1 -- 172.21.15.52:0/149044934 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa16419c9f0 con 0x7fa164107050 2024-03-04T09:29:42.960 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:42.960+0000 7fa161ffb700 1 -- 172.21.15.52:0/149044934 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa15c00ce20 con 0x7fa164107050 2024-03-04T09:29:42.960 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:42.960+0000 7fa16aaf5700 1 -- 172.21.15.52:0/149044934 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1641a4270 con 0x7fa164107050 2024-03-04T09:29:42.961 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:42.961+0000 7fa16aaf5700 1 -- 172.21.15.52:0/149044934 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fa164062810 con 0x7fa164107050 2024-03-04T09:29:42.961 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:42.962+0000 7fa161ffb700 1 -- 172.21.15.52:0/149044934 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 17) v1 ==== 136807+0+0 (secure 0 0 0) 0x7fa15c005000 con 0x7fa164107050 2024-03-04T09:29:42.962 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:42.963+0000 7fa161ffb700 1 --2- 172.21.15.52:0/149044934 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7fa1500a1f20 0x7fa1500a43e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:29:42.962 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:42.963+0000 7fa168891700 1 --2- 172.21.15.52:0/149044934 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7fa1500a1f20 0x7fa1500a43e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:29:42.962 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:42.963+0000 7fa161ffb700 1 -- 172.21.15.52:0/149044934 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9092+0+0 (secure 0 0 0) 0x7fa15c0ce7b0 con 0x7fa164107050 2024-03-04T09:29:42.962 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:42.963+0000 7fa161ffb700 1 --2- 172.21.15.52:0/149044934 >> [v2:172.21.15.191:6808/1485092643,v1:172.21.15.191:6809/1485092643] conn(0x7fa1500ad080 0x7fa1500af520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:29:42.962 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:42.963+0000 7fa161ffb700 1 -- 172.21.15.52:0/149044934 --> [v2:172.21.15.191:6808/1485092643,v1:172.21.15.191:6809/1485092643] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa15c00be50 con 0x7fa1500ad080 2024-03-04T09:29:42.962 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:42.963+0000 7fa161ffb700 1 -- 172.21.15.52:0/149044934 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_get_version_reply(handle=1 version=91) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa15c0cec00 con 0x7fa164107050 2024-03-04T09:29:42.962 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:42.963+0000 7fa163fff700 1 --2- 172.21.15.52:0/149044934 >> [v2:172.21.15.191:6808/1485092643,v1:172.21.15.191:6809/1485092643] conn(0x7fa1500ad080 0x7fa1500af520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:29:42.962 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:42.963+0000 7fa168891700 1 --2- 172.21.15.52:0/149044934 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7fa1500a1f20 0x7fa1500a43e0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fa154003e30 tx=0x7fa15400b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:29:42.962 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:42.964+0000 7fa163fff700 1 --2- 172.21.15.52:0/149044934 >> [v2:172.21.15.191:6808/1485092643,v1:172.21.15.191:6809/1485092643] conn(0x7fa1500ad080 0x7fa1500af520 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:29:42.962 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:42.965+0000 7fa161ffb700 1 -- 172.21.15.52:0/149044934 <== osd.5 v2:172.21.15.191:6808/1485092643 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7fa15c00be50 con 0x7fa1500ad080 2024-03-04T09:29:42.980 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:42.983+0000 7fa16aaf5700 1 -- 172.21.15.52:0/149044934 --> [v2:172.21.15.191:6808/1485092643,v1:172.21.15.191:6809/1485092643] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fa164062c90 con 0x7fa1500ad080 2024-03-04T09:29:42.980 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:42.983+0000 7fa161ffb700 1 -- 172.21.15.52:0/149044934 <== osd.5 v2:172.21.15.191:6808/1485092643 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7fa164062c90 con 0x7fa1500ad080 2024-03-04T09:29:42.981 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:42.983+0000 7fa16aaf5700 1 -- 172.21.15.52:0/149044934 >> [v2:172.21.15.191:6808/1485092643,v1:172.21.15.191:6809/1485092643] conn(0x7fa1500ad080 msgr2=0x7fa1500af520 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:29:42.981 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:42.983+0000 7fa16aaf5700 1 --2- 172.21.15.52:0/149044934 >> [v2:172.21.15.191:6808/1485092643,v1:172.21.15.191:6809/1485092643] conn(0x7fa1500ad080 0x7fa1500af520 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:29:42.981 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:42.984+0000 7fa16aaf5700 1 -- 172.21.15.52:0/149044934 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7fa1500a1f20 msgr2=0x7fa1500a43e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:29:42.981 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:42.984+0000 7fa16aaf5700 1 --2- 172.21.15.52:0/149044934 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7fa1500a1f20 0x7fa1500a43e0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fa154003e30 tx=0x7fa15400b040 comp rx=0 tx=0).stop 2024-03-04T09:29:42.981 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:42.984+0000 7fa16aaf5700 1 -- 172.21.15.52:0/149044934 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa164107050 msgr2=0x7fa16419bfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:29:42.981 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:42.984+0000 7fa16aaf5700 1 --2- 172.21.15.52:0/149044934 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa164107050 0x7fa16419bfc0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fa15c00e940 tx=0x7fa15c00ec50 comp rx=0 tx=0).stop 2024-03-04T09:29:42.981 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:42.984+0000 7fa169092700 1 -- 172.21.15.52:0/149044934 reap_dead start 2024-03-04T09:29:42.981 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:42.984+0000 7fa16aaf5700 1 -- 172.21.15.52:0/149044934 shutdown_connections 2024-03-04T09:29:42.982 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:42.984+0000 7fa16aaf5700 1 -- 172.21.15.52:0/149044934 >> 172.21.15.52:0/149044934 conn(0x7fa164077fd0 msgr2=0x7fa1640ffc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:29:42.983 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:42.984+0000 7fa16aaf5700 1 -- 172.21.15.52:0/149044934 shutdown_connections 2024-03-04T09:29:42.983 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:42.984+0000 7fa16aaf5700 1 -- 172.21.15.52:0/149044934 wait complete. 2024-03-04T09:29:43.254 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:29:43 smithi052 ceph-mon[32473]: pgmap v322: 129 pgs: 2 activating, 4 peering, 123 active+clean; 641 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-04T09:29:43.254 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:29:43 smithi052 ceph-mon[32473]: Health check failed: Reduced data availability: 2 pgs inactive, 2 pgs peering (PG_AVAILABILITY) 2024-03-04T09:29:43.336 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:29:43 smithi191 ceph-mon[36816]: pgmap v322: 129 pgs: 2 activating, 4 peering, 123 active+clean; 641 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-04T09:29:43.336 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:29:43 smithi191 ceph-mon[36816]: Health check failed: Reduced data availability: 2 pgs inactive, 2 pgs peering (PG_AVAILABILITY) 2024-03-04T09:29:43.347 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:43.350+0000 7f3db5b38700 1 -- 172.21.15.52:0/1087066377 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3db0101c00 msgr2=0x7f3db0105c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:29:43.347 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:43.350+0000 7f3db5b38700 1 --2- 172.21.15.52:0/1087066377 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3db0101c00 0x7f3db0105c00 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f3d98009990 tx=0x7f3d98009ca0 comp rx=0 tx=0).stop 2024-03-04T09:29:43.348 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:43.350+0000 7f3db5b38700 1 -- 172.21.15.52:0/1087066377 shutdown_connections 2024-03-04T09:29:43.348 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:43.350+0000 7f3db5b38700 1 --2- 172.21.15.52:0/1087066377 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3db0101250 0x7f3db0101630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:29:43.348 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:43.350+0000 7f3db5b38700 1 --2- 172.21.15.52:0/1087066377 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3db01062e0 0x7f3db010deb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:29:43.348 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:43.350+0000 7f3db5b38700 1 --2- 172.21.15.52:0/1087066377 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3db0101c00 0x7f3db0105c00 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:29:43.348 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:43.350+0000 7f3db5b38700 1 -- 172.21.15.52:0/1087066377 >> 172.21.15.52:0/1087066377 conn(0x7f3db0076fe0 msgr2=0x7f3db00773f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:29:43.348 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:43.350+0000 7f3db5b38700 1 -- 172.21.15.52:0/1087066377 shutdown_connections 2024-03-04T09:29:43.348 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:43.350+0000 7f3db5b38700 1 -- 172.21.15.52:0/1087066377 wait complete. 2024-03-04T09:29:43.350 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:43.351+0000 7f3db5b38700 1 Processor -- start 2024-03-04T09:29:43.350 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:43.351+0000 7f3db5b38700 1 -- start start 2024-03-04T09:29:43.350 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:43.351+0000 7f3db5b38700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3db0101250 0x7f3db0102ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:29:43.350 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:43.351+0000 7f3db5b38700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3db0101c00 0x7f3db01031e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:29:43.350 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:43.351+0000 7f3db5b38700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3db01062e0 0x7f3db0103720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:29:43.350 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:43.351+0000 7f3db5b38700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3db01112a0 con 0x7f3db0101250 2024-03-04T09:29:43.350 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:43.351+0000 7f3db5b38700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3db0111120 con 0x7f3db01062e0 2024-03-04T09:29:43.350 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:43.351+0000 7f3db5b38700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3db0111420 con 0x7f3db0101c00 2024-03-04T09:29:43.350 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:43.351+0000 7f3daf7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3db0101250 0x7f3db0102ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:29:43.350 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:43.351+0000 7f3daf7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3db0101250 0x7f3db0102ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:57080/0 (socket says 172.21.15.52:57080) 2024-03-04T09:29:43.350 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:43.351+0000 7f3daf7fe700 1 -- 172.21.15.52:0/3235915395 learned_addr learned my addr 172.21.15.52:0/3235915395 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-04T09:29:43.351 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:43.351+0000 7f3daffff700 1 --2- 172.21.15.52:0/3235915395 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3db01062e0 0x7f3db0103720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:29:43.351 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:43.351+0000 7f3da6ffd700 1 --2- 172.21.15.52:0/3235915395 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3db0101c00 0x7f3db01031e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:29:43.351 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:43.352+0000 7f3daf7fe700 1 -- 172.21.15.52:0/3235915395 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3db0101c00 msgr2=0x7f3db01031e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:29:43.351 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:43.352+0000 7f3daf7fe700 1 --2- 172.21.15.52:0/3235915395 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3db0101c00 0x7f3db01031e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:29:43.351 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:43.352+0000 7f3daf7fe700 1 -- 172.21.15.52:0/3235915395 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3db01062e0 msgr2=0x7f3db0103720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:29:43.351 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:43.352+0000 7f3daf7fe700 1 --2- 172.21.15.52:0/3235915395 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3db01062e0 0x7f3db0103720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:29:43.351 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:43.352+0000 7f3daf7fe700 1 -- 172.21.15.52:0/3235915395 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3db01a6dd0 con 0x7f3db0101250 2024-03-04T09:29:43.351 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:43.352+0000 7f3daffff700 1 --2- 172.21.15.52:0/3235915395 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3db01062e0 0x7f3db0103720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-04T09:29:43.351 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:43.352+0000 7f3daf7fe700 1 --2- 172.21.15.52:0/3235915395 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3db0101250 0x7f3db0102ca0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f3da000e9c0 tx=0x7f3da000ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:29:43.351 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:43.352+0000 7f3dad7fa700 1 -- 172.21.15.52:0/3235915395 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3da000cbd0 con 0x7f3db0101250 2024-03-04T09:29:43.351 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:43.352+0000 7f3db5b38700 1 -- 172.21.15.52:0/3235915395 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3db01a70c0 con 0x7f3db0101250 2024-03-04T09:29:43.351 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:43.352+0000 7f3dad7fa700 1 -- 172.21.15.52:0/3235915395 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3da000ee60 con 0x7f3db0101250 2024-03-04T09:29:43.351 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:43.352+0000 7f3dad7fa700 1 -- 172.21.15.52:0/3235915395 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3da0005630 con 0x7f3db0101250 2024-03-04T09:29:43.351 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:43.352+0000 7f3db5b38700 1 -- 172.21.15.52:0/3235915395 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3db01a7580 con 0x7f3db0101250 2024-03-04T09:29:43.352 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:43.353+0000 7f3db5b38700 1 -- 172.21.15.52:0/3235915395 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f3db00627d0 con 0x7f3db0101250 2024-03-04T09:29:43.352 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:43.354+0000 7f3dad7fa700 1 -- 172.21.15.52:0/3235915395 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 17) v1 ==== 136807+0+0 (secure 0 0 0) 0x7f3da0004750 con 0x7f3db0101250 2024-03-04T09:29:43.352 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:43.355+0000 7f3dad7fa700 1 --2- 172.21.15.52:0/3235915395 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f3d9c0a1ea0 0x7f3d9c0a4360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:29:43.353 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:43.355+0000 7f3da6ffd700 1 --2- 172.21.15.52:0/3235915395 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f3d9c0a1ea0 0x7f3d9c0a4360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:29:43.353 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:43.355+0000 7f3dad7fa700 1 -- 172.21.15.52:0/3235915395 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9092+0+0 (secure 0 0 0) 0x7f3da00ce8a0 con 0x7f3db0101250 2024-03-04T09:29:43.353 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:43.355+0000 7f3dad7fa700 1 --2- 172.21.15.52:0/3235915395 >> [v2:172.21.15.106:6808/3465868015,v1:172.21.15.106:6809/3465868015] conn(0x7f3d9c0ad000 0x7f3d9c0af4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:29:43.353 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:43.355+0000 7f3dad7fa700 1 -- 172.21.15.52:0/3235915395 --> [v2:172.21.15.106:6808/3465868015,v1:172.21.15.106:6809/3465868015] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f3da000be50 con 0x7f3d9c0ad000 2024-03-04T09:29:43.353 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:43.355+0000 7f3dad7fa700 1 -- 172.21.15.52:0/3235915395 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_get_version_reply(handle=1 version=91) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3da00cecf0 con 0x7f3db0101250 2024-03-04T09:29:43.353 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:43.355+0000 7f3daffff700 1 --2- 172.21.15.52:0/3235915395 >> [v2:172.21.15.106:6808/3465868015,v1:172.21.15.106:6809/3465868015] conn(0x7f3d9c0ad000 0x7f3d9c0af4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:29:43.353 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:43.355+0000 7f3da6ffd700 1 --2- 172.21.15.52:0/3235915395 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f3d9c0a1ea0 0x7f3d9c0a4360 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f3d98006010 tx=0x7f3d98005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:29:43.353 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:43.356+0000 7f3daffff700 1 --2- 172.21.15.52:0/3235915395 >> [v2:172.21.15.106:6808/3465868015,v1:172.21.15.106:6809/3465868015] conn(0x7f3d9c0ad000 0x7f3d9c0af4a0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:29:43.354 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:43.357+0000 7f3dad7fa700 1 -- 172.21.15.52:0/3235915395 <== osd.4 v2:172.21.15.106:6808/3465868015 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f3da000be50 con 0x7f3d9c0ad000 2024-03-04T09:29:43.370 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:43.373+0000 7f3db5b38700 1 -- 172.21.15.52:0/3235915395 --> [v2:172.21.15.106:6808/3465868015,v1:172.21.15.106:6809/3465868015] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f3db0062c00 con 0x7f3d9c0ad000 2024-03-04T09:29:43.370 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:43.373+0000 7f3dad7fa700 1 -- 172.21.15.52:0/3235915395 <== osd.4 v2:172.21.15.106:6808/3465868015 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f3db0062c00 con 0x7f3d9c0ad000 2024-03-04T09:29:43.371 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:43.374+0000 7f3db5b38700 1 -- 172.21.15.52:0/3235915395 >> [v2:172.21.15.106:6808/3465868015,v1:172.21.15.106:6809/3465868015] conn(0x7f3d9c0ad000 msgr2=0x7f3d9c0af4a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:29:43.371 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:43.374+0000 7f3db5b38700 1 --2- 172.21.15.52:0/3235915395 >> [v2:172.21.15.106:6808/3465868015,v1:172.21.15.106:6809/3465868015] conn(0x7f3d9c0ad000 0x7f3d9c0af4a0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:29:43.371 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:43.374+0000 7f3db5b38700 1 -- 172.21.15.52:0/3235915395 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f3d9c0a1ea0 msgr2=0x7f3d9c0a4360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:29:43.371 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:43.374+0000 7f3db5b38700 1 --2- 172.21.15.52:0/3235915395 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f3d9c0a1ea0 0x7f3d9c0a4360 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f3d98006010 tx=0x7f3d98005960 comp rx=0 tx=0).stop 2024-03-04T09:29:43.371 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:43.374+0000 7f3db5b38700 1 -- 172.21.15.52:0/3235915395 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3db0101250 msgr2=0x7f3db0102ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:29:43.371 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:43.374+0000 7f3db5b38700 1 --2- 172.21.15.52:0/3235915395 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3db0101250 0x7f3db0102ca0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f3da000e9c0 tx=0x7f3da000ecd0 comp rx=0 tx=0).stop 2024-03-04T09:29:43.372 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:43.374+0000 7f3daffff700 1 -- 172.21.15.52:0/3235915395 reap_dead start 2024-03-04T09:29:43.372 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:43.374+0000 7f3db5b38700 1 -- 172.21.15.52:0/3235915395 shutdown_connections 2024-03-04T09:29:43.372 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:43.374+0000 7f3db5b38700 1 -- 172.21.15.52:0/3235915395 >> 172.21.15.52:0/3235915395 conn(0x7f3db0076fe0 msgr2=0x7f3db006aba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:29:43.372 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:43.374+0000 7f3db5b38700 1 -- 172.21.15.52:0/3235915395 shutdown_connections 2024-03-04T09:29:43.372 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:43.374+0000 7f3db5b38700 1 -- 172.21.15.52:0/3235915395 wait complete. 2024-03-04T09:29:43.382 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:29:43 smithi106 ceph-mon[36484]: pgmap v322: 129 pgs: 2 activating, 4 peering, 123 active+clean; 641 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-04T09:29:43.382 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:29:43 smithi106 ceph-mon[36484]: Health check failed: Reduced data availability: 2 pgs inactive, 2 pgs peering (PG_AVAILABILITY) 2024-03-04T09:29:44.021 INFO:teuthology.orchestra.run.smithi052.stdout:309237645341 2024-03-04T09:29:44.022 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd shell --fsid 4e059f08-da08-11ee-95c6-87774f69a715 -- ceph osd last-stat-seq osd.10 2024-03-04T09:29:44.505 INFO:teuthology.orchestra.run.smithi052.stdout:253403070504 2024-03-04T09:29:44.506 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd shell --fsid 4e059f08-da08-11ee-95c6-87774f69a715 -- ceph osd last-stat-seq osd.8 2024-03-04T09:29:45.117 INFO:teuthology.orchestra.run.smithi052.stdout:163208757305 2024-03-04T09:29:45.118 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd shell --fsid 4e059f08-da08-11ee-95c6-87774f69a715 -- ceph osd last-stat-seq osd.5 2024-03-04T09:29:45.335 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:29:45 smithi052 ceph-mon[32473]: pgmap v323: 129 pgs: 1 activating, 2 peering, 126 active+clean; 641 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-04T09:29:45.336 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:29:45 smithi191 ceph-mon[36816]: pgmap v323: 129 pgs: 1 activating, 2 peering, 126 active+clean; 641 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-04T09:29:45.381 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:29:45 smithi106 ceph-mon[36484]: pgmap v323: 129 pgs: 1 activating, 2 peering, 126 active+clean; 641 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-04T09:29:45.662 INFO:teuthology.orchestra.run.smithi052.stdout:343597383706 2024-03-04T09:29:45.662 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd shell --fsid 4e059f08-da08-11ee-95c6-87774f69a715 -- ceph osd last-stat-seq osd.11 2024-03-04T09:29:46.230 INFO:teuthology.orchestra.run.smithi052.stdout:197568495667 2024-03-04T09:29:46.230 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd shell --fsid 4e059f08-da08-11ee-95c6-87774f69a715 -- ceph osd last-stat-seq osd.6 2024-03-04T09:29:46.778 INFO:teuthology.orchestra.run.smithi052.stdout:141733920829 2024-03-04T09:29:46.779 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd shell --fsid 4e059f08-da08-11ee-95c6-87774f69a715 -- ceph osd last-stat-seq osd.4 2024-03-04T09:29:47.239 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:47.241+0000 7f825d1bd700 1 -- 172.21.15.52:0/3092215079 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8258101070 msgr2=0x7f825810efd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:29:47.239 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:47.242+0000 7f825d1bd700 1 --2- 172.21.15.52:0/3092215079 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8258101070 0x7f825810efd0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f825000b0a0 tx=0x7f825000b3b0 comp rx=0 tx=0).stop 2024-03-04T09:29:47.239 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:47.242+0000 7f825d1bd700 1 -- 172.21.15.52:0/3092215079 shutdown_connections 2024-03-04T09:29:47.239 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:47.242+0000 7f825d1bd700 1 --2- 172.21.15.52:0/3092215079 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8258101070 0x7f825810efd0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:29:47.239 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:47.242+0000 7f825d1bd700 1 --2- 172.21.15.52:0/3092215079 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8258107130 0x7f8258107510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:29:47.239 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:47.242+0000 7f825d1bd700 1 --2- 172.21.15.52:0/3092215079 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f82581006b0 0x7f8258100b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:29:47.239 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:47.242+0000 7f825d1bd700 1 -- 172.21.15.52:0/3092215079 >> 172.21.15.52:0/3092215079 conn(0x7f82580775c0 msgr2=0x7f82580fe9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:29:47.239 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:47.242+0000 7f825d1bd700 1 -- 172.21.15.52:0/3092215079 shutdown_connections 2024-03-04T09:29:47.239 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:47.242+0000 7f825d1bd700 1 -- 172.21.15.52:0/3092215079 wait complete. 2024-03-04T09:29:47.240 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:47.243+0000 7f825d1bd700 1 Processor -- start 2024-03-04T09:29:47.241 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:47.243+0000 7f825d1bd700 1 -- start start 2024-03-04T09:29:47.241 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:47.243+0000 7f825d1bd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f82581006b0 0x7f825819e620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:29:47.241 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:47.243+0000 7f825d1bd700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8258101070 0x7f825819eb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:29:47.241 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:47.243+0000 7f825d1bd700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8258107130 0x7f82581987f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:29:47.241 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:47.243+0000 7f825d1bd700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8258112510 con 0x7f82581006b0 2024-03-04T09:29:47.241 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:47.243+0000 7f825d1bd700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8258112390 con 0x7f8258101070 2024-03-04T09:29:47.241 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:47.243+0000 7f825d1bd700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8258112690 con 0x7f8258107130 2024-03-04T09:29:47.241 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:47.243+0000 7f8256d9d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f82581006b0 0x7f825819e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:29:47.241 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:47.243+0000 7f8256d9d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f82581006b0 0x7f825819e620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:56918/0 (socket says 172.21.15.52:56918) 2024-03-04T09:29:47.241 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:47.243+0000 7f8256d9d700 1 -- 172.21.15.52:0/2092615380 learned_addr learned my addr 172.21.15.52:0/2092615380 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-04T09:29:47.242 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:47.243+0000 7f825659c700 1 --2- 172.21.15.52:0/2092615380 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8258101070 0x7f825819eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:29:47.242 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:47.243+0000 7f825759e700 1 --2- 172.21.15.52:0/2092615380 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8258107130 0x7f82581987f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:29:47.242 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:47.244+0000 7f8256d9d700 1 -- 172.21.15.52:0/2092615380 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8258107130 msgr2=0x7f82581987f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:29:47.242 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:47.244+0000 7f8256d9d700 1 --2- 172.21.15.52:0/2092615380 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8258107130 0x7f82581987f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:29:47.242 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:47.244+0000 7f8256d9d700 1 -- 172.21.15.52:0/2092615380 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8258101070 msgr2=0x7f825819eb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:29:47.243 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:47.244+0000 7f8256d9d700 1 --2- 172.21.15.52:0/2092615380 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8258101070 0x7f825819eb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:29:47.243 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:47.244+0000 7f8256d9d700 1 -- 172.21.15.52:0/2092615380 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8258198fd0 con 0x7f82581006b0 2024-03-04T09:29:47.243 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:47.244+0000 7f8256d9d700 1 --2- 172.21.15.52:0/2092615380 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f82581006b0 0x7f825819e620 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f824800e9a0 tx=0x7f824800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:29:47.243 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:47.244+0000 7f823ffff700 1 -- 172.21.15.52:0/2092615380 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f824800ca10 con 0x7f82581006b0 2024-03-04T09:29:47.243 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:47.244+0000 7f823ffff700 1 -- 172.21.15.52:0/2092615380 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f824800cbb0 con 0x7f82581006b0 2024-03-04T09:29:47.243 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:47.244+0000 7f825d1bd700 1 -- 172.21.15.52:0/2092615380 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82581992c0 con 0x7f82581006b0 2024-03-04T09:29:47.244 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:47.244+0000 7f823ffff700 1 -- 172.21.15.52:0/2092615380 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8248018680 con 0x7f82581006b0 2024-03-04T09:29:47.244 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:47.244+0000 7f825d1bd700 1 -- 172.21.15.52:0/2092615380 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82581017c0 con 0x7f82581006b0 2024-03-04T09:29:47.244 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:47.245+0000 7f825d1bd700 1 -- 172.21.15.52:0/2092615380 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f8258062750 con 0x7f82581006b0 2024-03-04T09:29:47.244 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:47.247+0000 7f823ffff700 1 -- 172.21.15.52:0/2092615380 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 17) v1 ==== 136807+0+0 (secure 0 0 0) 0x7f8248010430 con 0x7f82581006b0 2024-03-04T09:29:47.245 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:47.247+0000 7f823ffff700 1 --2- 172.21.15.52:0/2092615380 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f82400a1ea0 0x7f82400a4360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:29:47.245 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:47.247+0000 7f825659c700 1 --2- 172.21.15.52:0/2092615380 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f82400a1ea0 0x7f82400a4360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:29:47.246 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:47.247+0000 7f823ffff700 1 -- 172.21.15.52:0/2092615380 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9092+0+0 (secure 0 0 0) 0x7f82480cec80 con 0x7f82581006b0 2024-03-04T09:29:47.246 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:47.247+0000 7f823ffff700 1 --2- 172.21.15.52:0/2092615380 >> [v2:172.21.15.106:6816/1450870329,v1:172.21.15.106:6817/1450870329] conn(0x7f82400ad000 0x7f82400af4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:29:47.246 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:47.247+0000 7f823ffff700 1 -- 172.21.15.52:0/2092615380 --> [v2:172.21.15.106:6816/1450870329,v1:172.21.15.106:6817/1450870329] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f8248019ec0 con 0x7f82400ad000 2024-03-04T09:29:47.246 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:47.247+0000 7f823ffff700 1 -- 172.21.15.52:0/2092615380 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_get_version_reply(handle=1 version=91) v2 ==== 24+0+0 (secure 0 0 0) 0x7f82480cf0d0 con 0x7f82581006b0 2024-03-04T09:29:47.246 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:47.248+0000 7f825759e700 1 --2- 172.21.15.52:0/2092615380 >> [v2:172.21.15.106:6816/1450870329,v1:172.21.15.106:6817/1450870329] conn(0x7f82400ad000 0x7f82400af4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:29:47.246 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:47.248+0000 7f825659c700 1 --2- 172.21.15.52:0/2092615380 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f82400a1ea0 0x7f82400a4360 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f8258106990 tx=0x7f824c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:29:47.246 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:47.248+0000 7f825759e700 1 --2- 172.21.15.52:0/2092615380 >> [v2:172.21.15.106:6816/1450870329,v1:172.21.15.106:6817/1450870329] conn(0x7f82400ad000 0x7f82400af4a0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:29:47.247 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:47.249+0000 7f823ffff700 1 -- 172.21.15.52:0/2092615380 <== osd.7 v2:172.21.15.106:6816/1450870329 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f8248019ec0 con 0x7f82400ad000 2024-03-04T09:29:47.261 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/4e059f08-da08-11ee-95c6-87774f69a715/mon.a/config 2024-03-04T09:29:47.264 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/4e059f08-da08-11ee-95c6-87774f69a715/mon.a/config 2024-03-04T09:29:47.269 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/4e059f08-da08-11ee-95c6-87774f69a715/mon.a/config 2024-03-04T09:29:47.280 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:47.283+0000 7f825d1bd700 1 -- 172.21.15.52:0/2092615380 --> [v2:172.21.15.106:6816/1450870329,v1:172.21.15.106:6817/1450870329] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f8258062b50 con 0x7f82400ad000 2024-03-04T09:29:47.280 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:47.283+0000 7f823ffff700 1 -- 172.21.15.52:0/2092615380 <== osd.7 v2:172.21.15.106:6816/1450870329 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f8258062b50 con 0x7f82400ad000 2024-03-04T09:29:47.284 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:47.287+0000 7f825d1bd700 1 -- 172.21.15.52:0/2092615380 >> [v2:172.21.15.106:6816/1450870329,v1:172.21.15.106:6817/1450870329] conn(0x7f82400ad000 msgr2=0x7f82400af4a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:29:47.284 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:47.287+0000 7f825d1bd700 1 --2- 172.21.15.52:0/2092615380 >> [v2:172.21.15.106:6816/1450870329,v1:172.21.15.106:6817/1450870329] conn(0x7f82400ad000 0x7f82400af4a0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:29:47.285 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:47.287+0000 7f825d1bd700 1 -- 172.21.15.52:0/2092615380 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f82400a1ea0 msgr2=0x7f82400a4360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:29:47.285 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:47.287+0000 7f825d1bd700 1 --2- 172.21.15.52:0/2092615380 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f82400a1ea0 0x7f82400a4360 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f8258106990 tx=0x7f824c009260 comp rx=0 tx=0).stop 2024-03-04T09:29:47.285 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:47.288+0000 7f825d1bd700 1 -- 172.21.15.52:0/2092615380 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f82581006b0 msgr2=0x7f825819e620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:29:47.285 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:47.288+0000 7f825d1bd700 1 --2- 172.21.15.52:0/2092615380 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f82581006b0 0x7f825819e620 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f824800e9a0 tx=0x7f824800ecb0 comp rx=0 tx=0).stop 2024-03-04T09:29:47.287 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:47.290+0000 7f825759e700 1 -- 172.21.15.52:0/2092615380 reap_dead start 2024-03-04T09:29:47.288 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:47.291+0000 7f825d1bd700 1 -- 172.21.15.52:0/2092615380 shutdown_connections 2024-03-04T09:29:47.288 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:47.291+0000 7f825d1bd700 1 -- 172.21.15.52:0/2092615380 >> 172.21.15.52:0/2092615380 conn(0x7f82580775c0 msgr2=0x7f8258105040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:29:47.290 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:47.293+0000 7f825d1bd700 1 -- 172.21.15.52:0/2092615380 shutdown_connections 2024-03-04T09:29:47.295 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:47.298+0000 7f825d1bd700 1 -- 172.21.15.52:0/2092615380 wait complete. 2024-03-04T09:29:47.334 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:29:47 smithi052 ceph-mon[32473]: pgmap v324: 129 pgs: 1 activating, 2 peering, 126 active+clean; 641 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-04T09:29:47.336 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:29:47 smithi191 ceph-mon[36816]: pgmap v324: 129 pgs: 1 activating, 2 peering, 126 active+clean; 641 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-04T09:29:47.381 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:29:47 smithi106 ceph-mon[36484]: pgmap v324: 129 pgs: 1 activating, 2 peering, 126 active+clean; 641 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-04T09:29:48.381 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:29:48 smithi106 ceph-mon[36484]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 2 pgs peering) 2024-03-04T09:29:48.381 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:29:48 smithi106 ceph-mon[36484]: Cluster is now healthy 2024-03-04T09:29:48.445 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:29:48 smithi052 ceph-mon[32473]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 2 pgs peering) 2024-03-04T09:29:48.445 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:29:48 smithi052 ceph-mon[32473]: Cluster is now healthy 2024-03-04T09:29:48.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:29:48 smithi191 ceph-mon[36816]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 2 pgs peering) 2024-03-04T09:29:48.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:29:48 smithi191 ceph-mon[36816]: Cluster is now healthy 2024-03-04T09:29:49.334 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:29:49 smithi052 ceph-mon[32473]: pgmap v325: 129 pgs: 129 active+clean; 641 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s; 0 B/s, 0 objects/s recovering 2024-03-04T09:29:49.381 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:29:49 smithi106 ceph-mon[36484]: pgmap v325: 129 pgs: 129 active+clean; 641 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s; 0 B/s, 0 objects/s recovering 2024-03-04T09:29:49.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:29:49 smithi191 ceph-mon[36816]: pgmap v325: 129 pgs: 129 active+clean; 641 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s; 0 B/s, 0 objects/s recovering 2024-03-04T09:29:49.702 INFO:teuthology.orchestra.run.smithi052.stdout:219043332143 2024-03-04T09:29:49.702 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd shell --fsid 4e059f08-da08-11ee-95c6-87774f69a715 -- ceph osd last-stat-seq osd.7 2024-03-04T09:29:49.885 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/4e059f08-da08-11ee-95c6-87774f69a715/mon.a/config 2024-03-04T09:29:49.885 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/4e059f08-da08-11ee-95c6-87774f69a715/mon.a/config 2024-03-04T09:29:51.162 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:29:51 smithi052 ceph-mon[32473]: pgmap v326: 129 pgs: 129 active+clean; 641 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s; 0 B/s, 0 objects/s recovering 2024-03-04T09:29:51.381 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:29:51 smithi106 ceph-mon[36484]: pgmap v326: 129 pgs: 129 active+clean; 641 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s; 0 B/s, 0 objects/s recovering 2024-03-04T09:29:51.418 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:51.418+0000 7fdbaed29700 1 -- 172.21.15.52:0/3974768680 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdba8102110 msgr2=0x7fdba8102590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:29:51.418 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:51.418+0000 7fdbaed29700 1 --2- 172.21.15.52:0/3974768680 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdba8102110 0x7fdba8102590 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fdb940099c0 tx=0x7fdb94009cd0 comp rx=0 tx=0).stop 2024-03-04T09:29:51.418 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:51.418+0000 7fdbaed29700 1 -- 172.21.15.52:0/3974768680 shutdown_connections 2024-03-04T09:29:51.418 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:51.418+0000 7fdbaed29700 1 --2- 172.21.15.52:0/3974768680 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdba81080f0 0x7fdba81084d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:29:51.419 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:51.418+0000 7fdbaed29700 1 --2- 172.21.15.52:0/3974768680 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdba8102ad0 0x7fdba810ef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:29:51.419 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:51.418+0000 7fdbaed29700 1 --2- 172.21.15.52:0/3974768680 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdba8102110 0x7fdba8102590 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:29:51.419 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:51.418+0000 7fdbaed29700 1 -- 172.21.15.52:0/3974768680 >> 172.21.15.52:0/3974768680 conn(0x7fdba80791f0 msgr2=0x7fdba8079600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:29:51.419 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:51.418+0000 7fdbaed29700 1 -- 172.21.15.52:0/3974768680 shutdown_connections 2024-03-04T09:29:51.419 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:51.418+0000 7fdbaed29700 1 -- 172.21.15.52:0/3974768680 wait complete. 2024-03-04T09:29:51.419 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:51.418+0000 7fdbaed29700 1 Processor -- start 2024-03-04T09:29:51.419 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:51.418+0000 7fdbaed29700 1 -- start start 2024-03-04T09:29:51.419 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:51.419+0000 7fdbaed29700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdba8102110 0x7fdba819e430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:29:51.419 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:51.419+0000 7fdbaed29700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdba8102ad0 0x7fdba819e970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:29:51.420 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:51.419+0000 7fdbaed29700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdba81080f0 0x7fdba81985f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:29:51.420 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:51.419+0000 7fdbaed29700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdba8112390 con 0x7fdba81080f0 2024-03-04T09:29:51.420 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:51.419+0000 7fdbaed29700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdba8112210 con 0x7fdba8102110 2024-03-04T09:29:51.420 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:51.419+0000 7fdbaed29700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdba8112510 con 0x7fdba8102ad0 2024-03-04T09:29:51.420 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:51.419+0000 7fdbad2c6700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdba81080f0 0x7fdba81985f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:29:51.420 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:51.419+0000 7fdbad2c6700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdba81080f0 0x7fdba81985f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:56924/0 (socket says 172.21.15.52:56924) 2024-03-04T09:29:51.420 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:51.419+0000 7fdbad2c6700 1 -- 172.21.15.52:0/2318650195 learned_addr learned my addr 172.21.15.52:0/2318650195 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-04T09:29:51.420 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:51.419+0000 7fdba7fff700 1 --2- 172.21.15.52:0/2318650195 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdba8102ad0 0x7fdba819e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:29:51.420 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:51.419+0000 7fdbacac5700 1 --2- 172.21.15.52:0/2318650195 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdba8102110 0x7fdba819e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:29:51.420 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:51.419+0000 7fdbad2c6700 1 -- 172.21.15.52:0/2318650195 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdba8102ad0 msgr2=0x7fdba819e970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:29:51.421 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:51.419+0000 7fdbad2c6700 1 --2- 172.21.15.52:0/2318650195 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdba8102ad0 0x7fdba819e970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:29:51.421 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:51.419+0000 7fdbad2c6700 1 -- 172.21.15.52:0/2318650195 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdba8102110 msgr2=0x7fdba819e430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:29:51.421 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:51.419+0000 7fdbad2c6700 1 --2- 172.21.15.52:0/2318650195 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdba8102110 0x7fdba819e430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:29:51.421 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:51.419+0000 7fdbad2c6700 1 -- 172.21.15.52:0/2318650195 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdba8198d50 con 0x7fdba81080f0 2024-03-04T09:29:51.421 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:51.419+0000 7fdba7fff700 1 --2- 172.21.15.52:0/2318650195 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdba8102ad0 0x7fdba819e970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-04T09:29:51.421 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:51.419+0000 7fdbad2c6700 1 --2- 172.21.15.52:0/2318650195 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdba81080f0 0x7fdba81985f0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fdba000ef30 tx=0x7fdba000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:29:51.421 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:51.419+0000 7fdba5ffb700 1 -- 172.21.15.52:0/2318650195 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdba00092e0 con 0x7fdba81080f0 2024-03-04T09:29:51.421 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:51.419+0000 7fdba5ffb700 1 -- 172.21.15.52:0/2318650195 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fdba0004bb0 con 0x7fdba81080f0 2024-03-04T09:29:51.421 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:51.419+0000 7fdbaed29700 1 -- 172.21.15.52:0/2318650195 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdba8198fe0 con 0x7fdba81080f0 2024-03-04T09:29:51.421 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:51.419+0000 7fdba5ffb700 1 -- 172.21.15.52:0/2318650195 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdba0004280 con 0x7fdba81080f0 2024-03-04T09:29:51.421 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:51.420+0000 7fdbaed29700 1 -- 172.21.15.52:0/2318650195 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdba81a5300 con 0x7fdba81080f0 2024-03-04T09:29:51.421 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:51.421+0000 7fdbaed29700 1 -- 172.21.15.52:0/2318650195 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdba8068d80 con 0x7fdba81080f0 2024-03-04T09:29:51.422 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:51.422+0000 7fdba5ffb700 1 -- 172.21.15.52:0/2318650195 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 17) v1 ==== 136807+0+0 (secure 0 0 0) 0x7fdba001b030 con 0x7fdba81080f0 2024-03-04T09:29:51.422 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:51.422+0000 7fdba5ffb700 1 --2- 172.21.15.52:0/2318650195 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7fdb980a1ea0 0x7fdb980a4360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:29:51.422 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:51.422+0000 7fdbacac5700 1 --2- 172.21.15.52:0/2318650195 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7fdb980a1ea0 0x7fdb980a4360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:29:51.423 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:51.422+0000 7fdba5ffb700 1 -- 172.21.15.52:0/2318650195 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9092+0+0 (secure 0 0 0) 0x7fdba00cddd0 con 0x7fdba81080f0 2024-03-04T09:29:51.423 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:51.423+0000 7fdbacac5700 1 --2- 172.21.15.52:0/2318650195 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7fdb980a1ea0 0x7fdb980a4360 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fdb9c009c60 tx=0x7fdb9c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:29:51.423 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:51.427+0000 7fdba5ffb700 1 -- 172.21.15.52:0/2318650195 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fdba0022020 con 0x7fdba81080f0 2024-03-04T09:29:51.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:29:51 smithi191 ceph-mon[36816]: pgmap v326: 129 pgs: 129 active+clean; 641 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s; 0 B/s, 0 objects/s recovering 2024-03-04T09:29:51.634 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:51.637+0000 7fdbaed29700 1 -- 172.21.15.52:0/2318650195 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fdba802d9b0 con 0x7fdba81080f0 2024-03-04T09:29:51.634 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:51.637+0000 7fdba5ffb700 1 -- 172.21.15.52:0/2318650195 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fdba0010040 con 0x7fdba81080f0 2024-03-04T09:29:51.635 INFO:teuthology.orchestra.run.smithi052.stdout:107374182466 2024-03-04T09:29:51.637 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:51.639+0000 7fdbaed29700 1 -- 172.21.15.52:0/2318650195 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7fdb980a1ea0 msgr2=0x7fdb980a4360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:29:51.637 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:51.639+0000 7fdbaed29700 1 --2- 172.21.15.52:0/2318650195 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7fdb980a1ea0 0x7fdb980a4360 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fdb9c009c60 tx=0x7fdb9c009260 comp rx=0 tx=0).stop 2024-03-04T09:29:51.637 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:51.639+0000 7fdbaed29700 1 -- 172.21.15.52:0/2318650195 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdba81080f0 msgr2=0x7fdba81985f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:29:51.637 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:51.639+0000 7fdbaed29700 1 --2- 172.21.15.52:0/2318650195 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdba81080f0 0x7fdba81985f0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fdba000ef30 tx=0x7fdba000c3a0 comp rx=0 tx=0).stop 2024-03-04T09:29:51.637 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:51.639+0000 7fdbaed29700 1 -- 172.21.15.52:0/2318650195 shutdown_connections 2024-03-04T09:29:51.637 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:51.639+0000 7fdbaed29700 1 --2- 172.21.15.52:0/2318650195 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7fdb980a1ea0 0x7fdb980a4360 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:29:51.637 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:51.639+0000 7fdbaed29700 1 --2- 172.21.15.52:0/2318650195 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdba81080f0 0x7fdba81985f0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:29:51.637 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:51.639+0000 7fdbaed29700 1 --2- 172.21.15.52:0/2318650195 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdba8102110 0x7fdba819e430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:29:51.637 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:51.639+0000 7fdbaed29700 1 --2- 172.21.15.52:0/2318650195 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdba8102ad0 0x7fdba819e970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:29:51.638 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:51.639+0000 7fdbaed29700 1 -- 172.21.15.52:0/2318650195 >> 172.21.15.52:0/2318650195 conn(0x7fdba80791f0 msgr2=0x7fdba8100cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:29:51.638 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:51.639+0000 7fdbaed29700 1 -- 172.21.15.52:0/2318650195 shutdown_connections 2024-03-04T09:29:51.638 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:51.639+0000 7fdbaed29700 1 -- 172.21.15.52:0/2318650195 wait complete. 2024-03-04T09:29:52.585 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:29:52 smithi052 ceph-mon[32473]: from='client.? 172.21.15.52:0/2318650195' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-03-04T09:29:52.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:29:52 smithi191 ceph-mon[36816]: from='client.? 172.21.15.52:0/2318650195' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-03-04T09:29:52.631 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:29:52 smithi106 ceph-mon[36484]: from='client.? 172.21.15.52:0/2318650195' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-03-04T09:29:53.414 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:29:53 smithi052 ceph-mon[32473]: pgmap v327: 129 pgs: 129 active+clean; 641 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s; 0 B/s, 0 objects/s recovering 2024-03-04T09:29:53.487 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:29:53 smithi191 ceph-mon[36816]: pgmap v327: 129 pgs: 129 active+clean; 641 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s; 0 B/s, 0 objects/s recovering 2024-03-04T09:29:53.631 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:29:53 smithi106 ceph-mon[36484]: pgmap v327: 129 pgs: 129 active+clean; 641 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s; 0 B/s, 0 objects/s recovering 2024-03-04T09:29:53.977 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/4e059f08-da08-11ee-95c6-87774f69a715/mon.a/config 2024-03-04T09:29:53.978 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/4e059f08-da08-11ee-95c6-87774f69a715/mon.a/config 2024-03-04T09:29:53.980 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/4e059f08-da08-11ee-95c6-87774f69a715/mon.a/config 2024-03-04T09:29:53.980 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/4e059f08-da08-11ee-95c6-87774f69a715/mon.a/config 2024-03-04T09:29:53.980 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/4e059f08-da08-11ee-95c6-87774f69a715/mon.a/config 2024-03-04T09:29:54.220 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:54.222+0000 7fee00142700 1 -- 172.21.15.52:0/1217492989 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fedf8107130 msgr2=0x7fedf8107510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:29:54.220 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:54.222+0000 7fee00142700 1 --2- 172.21.15.52:0/1217492989 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fedf8107130 0x7fedf8107510 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fedf4009990 tx=0x7fedf4009ca0 comp rx=0 tx=0).stop 2024-03-04T09:29:54.220 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:54.222+0000 7fee00142700 1 -- 172.21.15.52:0/1217492989 shutdown_connections 2024-03-04T09:29:54.220 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:54.222+0000 7fee00142700 1 --2- 172.21.15.52:0/1217492989 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fedf8101070 0x7fedf810efd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:29:54.220 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:54.222+0000 7fee00142700 1 --2- 172.21.15.52:0/1217492989 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fedf8107130 0x7fedf8107510 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:29:54.220 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:54.222+0000 7fee00142700 1 --2- 172.21.15.52:0/1217492989 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fedf81006b0 0x7fedf8100b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:29:54.220 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:54.222+0000 7fee00142700 1 -- 172.21.15.52:0/1217492989 >> 172.21.15.52:0/1217492989 conn(0x7fedf80775c0 msgr2=0x7fedf80fe9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:29:54.220 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:54.222+0000 7fee00142700 1 -- 172.21.15.52:0/1217492989 shutdown_connections 2024-03-04T09:29:54.220 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:54.222+0000 7fee00142700 1 -- 172.21.15.52:0/1217492989 wait complete. 2024-03-04T09:29:54.220 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:54.222+0000 7fee00142700 1 Processor -- start 2024-03-04T09:29:54.220 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:54.222+0000 7fee00142700 1 -- start start 2024-03-04T09:29:54.220 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:54.222+0000 7fee00142700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fedf81006b0 0x7fedf8101890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:29:54.221 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:54.222+0000 7fee00142700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fedf8101070 0x7fedf8101dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:29:54.221 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:54.223+0000 7fee00142700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fedf8107130 0x7fedf8102310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:29:54.221 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:54.223+0000 7fee00142700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedf81124f0 con 0x7fedf81006b0 2024-03-04T09:29:54.221 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:54.223+0000 7fee00142700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedf8112370 con 0x7fedf8101070 2024-03-04T09:29:54.221 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:54.223+0000 7fee00142700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedf8112670 con 0x7fedf8107130 2024-03-04T09:29:54.221 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:54.223+0000 7fedfdede700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fedf81006b0 0x7fedf8101890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:29:54.221 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:54.223+0000 7fedfdede700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fedf81006b0 0x7fedf8101890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:56936/0 (socket says 172.21.15.52:56936) 2024-03-04T09:29:54.221 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:54.223+0000 7fedfdede700 1 -- 172.21.15.52:0/3834218877 learned_addr learned my addr 172.21.15.52:0/3834218877 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-04T09:29:54.221 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:54.223+0000 7fedfd6dd700 1 --2- 172.21.15.52:0/3834218877 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fedf8101070 0x7fedf8101dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:29:54.221 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:54.223+0000 7fedfe6df700 1 --2- 172.21.15.52:0/3834218877 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fedf8107130 0x7fedf8102310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:29:54.221 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:54.223+0000 7fedfdede700 1 -- 172.21.15.52:0/3834218877 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fedf8107130 msgr2=0x7fedf8102310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:29:54.221 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:54.223+0000 7fedfdede700 1 --2- 172.21.15.52:0/3834218877 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fedf8107130 0x7fedf8102310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:29:54.221 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:54.223+0000 7fedfdede700 1 -- 172.21.15.52:0/3834218877 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fedf8101070 msgr2=0x7fedf8101dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:29:54.221 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:54.223+0000 7fedfdede700 1 --2- 172.21.15.52:0/3834218877 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fedf8101070 0x7fedf8101dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:29:54.222 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:54.223+0000 7fedfdede700 1 -- 172.21.15.52:0/3834218877 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fedf81a6e80 con 0x7fedf81006b0 2024-03-04T09:29:54.223 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:54.223+0000 7fedfdede700 1 --2- 172.21.15.52:0/3834218877 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fedf81006b0 0x7fedf8101890 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fedf4000c00 tx=0x7fedf400bab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:29:54.223 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:54.223+0000 7fedeeffd700 1 -- 172.21.15.52:0/3834218877 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedf4027070 con 0x7fedf81006b0 2024-03-04T09:29:54.223 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:54.223+0000 7fee00142700 1 -- 172.21.15.52:0/3834218877 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fedf81a7110 con 0x7fedf81006b0 2024-03-04T09:29:54.223 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:54.223+0000 7fedeeffd700 1 -- 172.21.15.52:0/3834218877 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fedf4003d70 con 0x7fedf81006b0 2024-03-04T09:29:54.223 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:54.223+0000 7fedeeffd700 1 -- 172.21.15.52:0/3834218877 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedf400f460 con 0x7fedf81006b0 2024-03-04T09:29:54.223 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:54.223+0000 7fee00142700 1 -- 172.21.15.52:0/3834218877 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fedf81a75c0 con 0x7fedf81006b0 2024-03-04T09:29:54.223 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:54.224+0000 7fee00142700 1 -- 172.21.15.52:0/3834218877 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fedf810a700 con 0x7fedf81006b0 2024-03-04T09:29:54.223 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:54.225+0000 7fedeeffd700 1 -- 172.21.15.52:0/3834218877 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 17) v1 ==== 136807+0+0 (secure 0 0 0) 0x7fedf402a070 con 0x7fedf81006b0 2024-03-04T09:29:54.223 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:54.226+0000 7fedeeffd700 1 --2- 172.21.15.52:0/3834218877 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7fede40a1f20 0x7fede40a43e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:29:54.223 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:54.226+0000 7fedfd6dd700 1 --2- 172.21.15.52:0/3834218877 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7fede40a1f20 0x7fede40a43e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:29:54.223 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:54.226+0000 7fedeeffd700 1 -- 172.21.15.52:0/3834218877 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9092+0+0 (secure 0 0 0) 0x7fedf40cf030 con 0x7fedf81006b0 2024-03-04T09:29:54.224 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:54.226+0000 7fedfd6dd700 1 --2- 172.21.15.52:0/3834218877 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7fede40a1f20 0x7fede40a43e0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fedf80ffc20 tx=0x7fede8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:29:54.227 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:54.230+0000 7fedeeffd700 1 -- 172.21.15.52:0/3834218877 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fedf4013070 con 0x7fedf81006b0 2024-03-04T09:29:54.438 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:54.440+0000 7fee00142700 1 -- 172.21.15.52:0/3834218877 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fedf81039b0 con 0x7fedf81006b0 2024-03-04T09:29:54.438 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:54.441+0000 7fedeeffd700 1 -- 172.21.15.52:0/3834218877 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fedf409c030 con 0x7fedf81006b0 2024-03-04T09:29:54.438 INFO:teuthology.orchestra.run.smithi052.stdout:55834574926 2024-03-04T09:29:54.440 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:54.443+0000 7fee00142700 1 -- 172.21.15.52:0/3834218877 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7fede40a1f20 msgr2=0x7fede40a43e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:29:54.440 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:54.443+0000 7fee00142700 1 --2- 172.21.15.52:0/3834218877 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7fede40a1f20 0x7fede40a43e0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fedf80ffc20 tx=0x7fede8009260 comp rx=0 tx=0).stop 2024-03-04T09:29:54.440 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:54.443+0000 7fee00142700 1 -- 172.21.15.52:0/3834218877 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fedf81006b0 msgr2=0x7fedf8101890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:29:54.440 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:54.443+0000 7fee00142700 1 --2- 172.21.15.52:0/3834218877 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fedf81006b0 0x7fedf8101890 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fedf4000c00 tx=0x7fedf400bab0 comp rx=0 tx=0).stop 2024-03-04T09:29:54.440 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:54.443+0000 7fee00142700 1 -- 172.21.15.52:0/3834218877 shutdown_connections 2024-03-04T09:29:54.440 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:54.443+0000 7fee00142700 1 --2- 172.21.15.52:0/3834218877 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7fede40a1f20 0x7fede40a43e0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:29:54.440 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:54.443+0000 7fee00142700 1 --2- 172.21.15.52:0/3834218877 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fedf8107130 0x7fedf8102310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:29:54.440 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:54.443+0000 7fee00142700 1 --2- 172.21.15.52:0/3834218877 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fedf81006b0 0x7fedf8101890 secure :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fedf4000c00 tx=0x7fedf400bab0 comp rx=0 tx=0).stop 2024-03-04T09:29:54.440 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:54.443+0000 7fee00142700 1 --2- 172.21.15.52:0/3834218877 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fedf8101070 0x7fedf8101dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:29:54.441 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:54.443+0000 7fee00142700 1 -- 172.21.15.52:0/3834218877 >> 172.21.15.52:0/3834218877 conn(0x7fedf80775c0 msgr2=0x7fedf81051d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:29:54.441 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:54.444+0000 7fee00142700 1 -- 172.21.15.52:0/3834218877 shutdown_connections 2024-03-04T09:29:54.441 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:54.444+0000 7fee00142700 1 -- 172.21.15.52:0/3834218877 wait complete. 2024-03-04T09:29:55.334 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:29:55 smithi052 ceph-mon[32473]: pgmap v328: 129 pgs: 129 active+clean; 641 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s; 0 B/s, 0 objects/s recovering 2024-03-04T09:29:55.335 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:29:55 smithi052 ceph-mon[32473]: from='client.? 172.21.15.52:0/3834218877' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-03-04T09:29:55.587 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:29:55 smithi191 ceph-mon[36816]: pgmap v328: 129 pgs: 129 active+clean; 641 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s; 0 B/s, 0 objects/s recovering 2024-03-04T09:29:55.587 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:29:55 smithi191 ceph-mon[36816]: from='client.? 172.21.15.52:0/3834218877' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-03-04T09:29:55.631 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:29:55 smithi106 ceph-mon[36484]: pgmap v328: 129 pgs: 129 active+clean; 641 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s; 0 B/s, 0 objects/s recovering 2024-03-04T09:29:55.631 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:29:55 smithi106 ceph-mon[36484]: from='client.? 172.21.15.52:0/3834218877' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-03-04T09:29:55.763 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:55.766+0000 7f71a3cdb700 1 -- 172.21.15.52:0/905256383 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f719c107000 msgr2=0x7f719c1073e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:29:55.763 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:55.766+0000 7f71a3cdb700 1 --2- 172.21.15.52:0/905256383 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f719c107000 0x7f719c1073e0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f719800b0a0 tx=0x7f719800b3b0 comp rx=0 tx=0).stop 2024-03-04T09:29:55.764 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:55.766+0000 7f71a3cdb700 1 -- 172.21.15.52:0/905256383 shutdown_connections 2024-03-04T09:29:55.764 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:55.766+0000 7f71a3cdb700 1 --2- 172.21.15.52:0/905256383 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f719c101960 0x7f719c10de40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:29:55.764 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:55.766+0000 7f71a3cdb700 1 --2- 172.21.15.52:0/905256383 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f719c107000 0x7f719c1073e0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:29:55.764 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:55.766+0000 7f71a3cdb700 1 --2- 172.21.15.52:0/905256383 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f719c100fc0 0x7f719c101420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:29:55.764 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:55.766+0000 7f71a3cdb700 1 -- 172.21.15.52:0/905256383 >> 172.21.15.52:0/905256383 conn(0x7f719c077ef0 msgr2=0x7f719c078300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:29:55.764 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:55.766+0000 7f71a3cdb700 1 -- 172.21.15.52:0/905256383 shutdown_connections 2024-03-04T09:29:55.764 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:55.766+0000 7f71a3cdb700 1 -- 172.21.15.52:0/905256383 wait complete. 2024-03-04T09:29:55.766 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:55.767+0000 7f71a3cdb700 1 Processor -- start 2024-03-04T09:29:55.766 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:55.767+0000 7f71a3cdb700 1 -- start start 2024-03-04T09:29:55.766 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:55.767+0000 7f71a3cdb700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f719c100fc0 0x7f719c1996a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:29:55.767 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:55.767+0000 7f71a3cdb700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f719c101960 0x7f719c199be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:29:55.767 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:55.767+0000 7f71a3cdb700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f719c107000 0x7f719c1a4720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:29:55.767 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:55.767+0000 7f71a3cdb700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f719c111260 con 0x7f719c101960 2024-03-04T09:29:55.767 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:55.767+0000 7f71a3cdb700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f719c1110e0 con 0x7f719c107000 2024-03-04T09:29:55.767 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:55.767+0000 7f71a3cdb700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f719c1113e0 con 0x7f719c100fc0 2024-03-04T09:29:55.767 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:55.767+0000 7f71a1276700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f719c101960 0x7f719c199be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:29:55.767 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:55.767+0000 7f71a1276700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f719c101960 0x7f719c199be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:42122/0 (socket says 172.21.15.52:42122) 2024-03-04T09:29:55.767 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:55.767+0000 7f71a1276700 1 -- 172.21.15.52:0/2411530114 learned_addr learned my addr 172.21.15.52:0/2411530114 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-04T09:29:55.767 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:55.767+0000 7f71a2278700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f719c107000 0x7f719c1a4720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:29:55.767 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:55.767+0000 7f71a1a77700 1 --2- 172.21.15.52:0/2411530114 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f719c100fc0 0x7f719c1996a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:29:55.767 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:55.768+0000 7f71a1276700 1 -- 172.21.15.52:0/2411530114 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f719c100fc0 msgr2=0x7f719c1996a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:29:55.767 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:55.768+0000 7f71a1276700 1 --2- 172.21.15.52:0/2411530114 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f719c100fc0 0x7f719c1996a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:29:55.768 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:55.768+0000 7f71a1276700 1 -- 172.21.15.52:0/2411530114 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f719c107000 msgr2=0x7f719c1a4720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:29:55.768 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:55.768+0000 7f71a1276700 1 --2- 172.21.15.52:0/2411530114 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f719c107000 0x7f719c1a4720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:29:55.768 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:55.768+0000 7f71a1276700 1 -- 172.21.15.52:0/2411530114 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f719c1a4c60 con 0x7f719c101960 2024-03-04T09:29:55.768 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:55.768+0000 7f71a2278700 1 --2- 172.21.15.52:0/2411530114 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f719c107000 0x7f719c1a4720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-04T09:29:55.768 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:55.768+0000 7f71a1276700 1 --2- 172.21.15.52:0/2411530114 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f719c101960 0x7f719c199be0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f718c00e9a0 tx=0x7f718c00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:29:55.768 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:55.768+0000 7f7192ffd700 1 -- 172.21.15.52:0/2411530114 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f718c00ca10 con 0x7f719c101960 2024-03-04T09:29:55.768 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:55.768+0000 7f71a3cdb700 1 -- 172.21.15.52:0/2411530114 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f719c1a4e90 con 0x7f719c101960 2024-03-04T09:29:55.768 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:55.768+0000 7f7192ffd700 1 -- 172.21.15.52:0/2411530114 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f718c00cbb0 con 0x7f719c101960 2024-03-04T09:29:55.768 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:55.768+0000 7f7192ffd700 1 -- 172.21.15.52:0/2411530114 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f718c018680 con 0x7f719c101960 2024-03-04T09:29:55.768 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:55.768+0000 7f71a3cdb700 1 -- 172.21.15.52:0/2411530114 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f719c1a5390 con 0x7f719c101960 2024-03-04T09:29:55.768 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:55.769+0000 7f71a3cdb700 1 -- 172.21.15.52:0/2411530114 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f719c068d80 con 0x7f719c101960 2024-03-04T09:29:55.769 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:55.770+0000 7f7192ffd700 1 -- 172.21.15.52:0/2411530114 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 17) v1 ==== 136807+0+0 (secure 0 0 0) 0x7f718c010430 con 0x7f719c101960 2024-03-04T09:29:55.769 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:55.771+0000 7f7192ffd700 1 --2- 172.21.15.52:0/2411530114 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f71880a1ea0 0x7f71880a4360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:29:55.769 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:55.771+0000 7f71a1a77700 1 --2- 172.21.15.52:0/2411530114 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f71880a1ea0 0x7f71880a4360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:29:55.769 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:55.771+0000 7f7192ffd700 1 -- 172.21.15.52:0/2411530114 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9092+0+0 (secure 0 0 0) 0x7f718c0ce970 con 0x7f719c101960 2024-03-04T09:29:55.769 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:55.771+0000 7f71a1a77700 1 --2- 172.21.15.52:0/2411530114 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f71880a1ea0 0x7f71880a4360 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f7198009e60 tx=0x7f719800b700 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:29:55.771 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:55.774+0000 7f7192ffd700 1 -- 172.21.15.52:0/2411530114 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f718c09b880 con 0x7f719c101960 2024-03-04T09:29:55.978 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:55.980+0000 7f71a3cdb700 1 -- 172.21.15.52:0/2411530114 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7f719c02d9b0 con 0x7f719c101960 2024-03-04T09:29:55.978 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:55.981+0000 7f7192ffd700 1 -- 172.21.15.52:0/2411530114 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f718c09b180 con 0x7f719c101960 2024-03-04T09:29:55.978 INFO:teuthology.orchestra.run.smithi052.stdout:163208757307 2024-03-04T09:29:55.980 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:55.983+0000 7f71a3cdb700 1 -- 172.21.15.52:0/2411530114 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f71880a1ea0 msgr2=0x7f71880a4360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:29:55.980 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:55.983+0000 7f71a3cdb700 1 --2- 172.21.15.52:0/2411530114 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f71880a1ea0 0x7f71880a4360 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f7198009e60 tx=0x7f719800b700 comp rx=0 tx=0).stop 2024-03-04T09:29:55.980 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:55.983+0000 7f71a3cdb700 1 -- 172.21.15.52:0/2411530114 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f719c101960 msgr2=0x7f719c199be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:29:55.980 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:55.983+0000 7f71a3cdb700 1 --2- 172.21.15.52:0/2411530114 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f719c101960 0x7f719c199be0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f718c00e9a0 tx=0x7f718c00ecb0 comp rx=0 tx=0).stop 2024-03-04T09:29:55.981 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:55.983+0000 7f71a3cdb700 1 -- 172.21.15.52:0/2411530114 shutdown_connections 2024-03-04T09:29:55.981 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:55.983+0000 7f71a3cdb700 1 --2- 172.21.15.52:0/2411530114 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f71880a1ea0 0x7f71880a4360 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:29:55.981 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:55.983+0000 7f71a3cdb700 1 --2- 172.21.15.52:0/2411530114 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f719c100fc0 0x7f719c1996a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:29:55.981 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:55.983+0000 7f71a3cdb700 1 --2- 172.21.15.52:0/2411530114 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f719c107000 0x7f719c1a4720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:29:55.981 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:55.983+0000 7f71a3cdb700 1 --2- 172.21.15.52:0/2411530114 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f719c101960 0x7f719c199be0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:29:55.981 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:55.983+0000 7f71a3cdb700 1 -- 172.21.15.52:0/2411530114 >> 172.21.15.52:0/2411530114 conn(0x7f719c077ef0 msgr2=0x7f719c0ffdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:29:55.981 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:55.983+0000 7f71a3cdb700 1 -- 172.21.15.52:0/2411530114 shutdown_connections 2024-03-04T09:29:55.981 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:55.983+0000 7f71a3cdb700 1 -- 172.21.15.52:0/2411530114 wait complete. 2024-03-04T09:29:56.226 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:29:56 smithi052 ceph-mon[32473]: from='client.? 172.21.15.52:0/2411530114' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-03-04T09:29:56.576 INFO:tasks.cephadm.ceph_manager.ceph:need seq 107374182462 got 107374182466 for osd.3 2024-03-04T09:29:56.576 DEBUG:teuthology.parallel:result is None 2024-03-04T09:29:56.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:29:56 smithi191 ceph-mon[36816]: from='client.? 172.21.15.52:0/2411530114' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-03-04T09:29:56.631 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:29:56 smithi106 ceph-mon[36484]: from='client.? 172.21.15.52:0/2411530114' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-03-04T09:29:57.406 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:29:57 smithi052 ceph-mon[32473]: pgmap v329: 129 pgs: 129 active+clean; 641 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 5 op/s; 0 B/s, 0 objects/s recovering 2024-03-04T09:29:57.522 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:57.525+0000 7f70472e3700 1 -- 172.21.15.52:0/3239690525 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f70401013c0 msgr2=0x7f704010d600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:29:57.522 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:57.525+0000 7f70472e3700 1 --2- 172.21.15.52:0/3239690525 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f70401013c0 0x7f704010d600 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f703800aaa0 tx=0x7f703800adb0 comp rx=0 tx=0).stop 2024-03-04T09:29:57.522 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:57.525+0000 7f70472e3700 1 -- 172.21.15.52:0/3239690525 shutdown_connections 2024-03-04T09:29:57.522 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:57.525+0000 7f70472e3700 1 --2- 172.21.15.52:0/3239690525 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f70401013c0 0x7f704010d600 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:29:57.522 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:57.525+0000 7f70472e3700 1 --2- 172.21.15.52:0/3239690525 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f70401067e0 0x7f7040106bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:29:57.523 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:57.525+0000 7f70472e3700 1 --2- 172.21.15.52:0/3239690525 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7040100a00 0x7f7040100e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:29:57.523 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:57.525+0000 7f70472e3700 1 -- 172.21.15.52:0/3239690525 >> 172.21.15.52:0/3239690525 conn(0x7f7040076300 msgr2=0x7f7040076710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:29:57.523 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:57.525+0000 7f70472e3700 1 -- 172.21.15.52:0/3239690525 shutdown_connections 2024-03-04T09:29:57.523 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:57.525+0000 7f70472e3700 1 -- 172.21.15.52:0/3239690525 wait complete. 2024-03-04T09:29:57.525 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:57.526+0000 7f70472e3700 1 Processor -- start 2024-03-04T09:29:57.525 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:57.526+0000 7f70472e3700 1 -- start start 2024-03-04T09:29:57.525 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:57.526+0000 7f70472e3700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7040100a00 0x7f704019e430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:29:57.525 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:57.526+0000 7f70472e3700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f70401013c0 0x7f704019e970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:29:57.525 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:57.526+0000 7f70472e3700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f70401067e0 0x7f70401985f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:29:57.525 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:57.526+0000 7f70472e3700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70401101e0 con 0x7f7040100a00 2024-03-04T09:29:57.525 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:57.526+0000 7f70472e3700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7040110060 con 0x7f70401067e0 2024-03-04T09:29:57.525 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:57.526+0000 7f70472e3700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7040110360 con 0x7f70401013c0 2024-03-04T09:29:57.525 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:57.526+0000 7f704507f700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7040100a00 0x7f704019e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:29:57.525 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:57.526+0000 7f704507f700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7040100a00 0x7f704019e430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:42132/0 (socket says 172.21.15.52:42132) 2024-03-04T09:29:57.526 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:57.526+0000 7f704507f700 1 -- 172.21.15.52:0/3528665262 learned_addr learned my addr 172.21.15.52:0/3528665262 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-04T09:29:57.526 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:57.526+0000 7f7045880700 1 --2- 172.21.15.52:0/3528665262 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f70401067e0 0x7f70401985f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:29:57.526 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:57.526+0000 7f704487e700 1 --2- 172.21.15.52:0/3528665262 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f70401013c0 0x7f704019e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:29:57.526 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:57.526+0000 7f704507f700 1 -- 172.21.15.52:0/3528665262 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f70401013c0 msgr2=0x7f704019e970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:29:57.526 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:57.526+0000 7f704507f700 1 --2- 172.21.15.52:0/3528665262 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f70401013c0 0x7f704019e970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:29:57.526 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:57.526+0000 7f704507f700 1 -- 172.21.15.52:0/3528665262 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f70401067e0 msgr2=0x7f70401985f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:29:57.526 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:57.526+0000 7f704507f700 1 --2- 172.21.15.52:0/3528665262 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f70401067e0 0x7f70401985f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:29:57.526 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:57.526+0000 7f704507f700 1 -- 172.21.15.52:0/3528665262 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7040198d50 con 0x7f7040100a00 2024-03-04T09:29:57.526 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:57.527+0000 7f704507f700 1 --2- 172.21.15.52:0/3528665262 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7040100a00 0x7f704019e430 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f703000ef10 tx=0x7f703000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:29:57.526 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:57.527+0000 7f703e7fc700 1 -- 172.21.15.52:0/3528665262 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f703000c9c0 con 0x7f7040100a00 2024-03-04T09:29:57.527 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:57.527+0000 7f703e7fc700 1 -- 172.21.15.52:0/3528665262 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7030019070 con 0x7f7040100a00 2024-03-04T09:29:57.527 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:57.527+0000 7f70472e3700 1 -- 172.21.15.52:0/3528665262 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7040198fe0 con 0x7f7040100a00 2024-03-04T09:29:57.527 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:57.527+0000 7f703e7fc700 1 -- 172.21.15.52:0/3528665262 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7030007e60 con 0x7f7040100a00 2024-03-04T09:29:57.527 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:57.527+0000 7f70472e3700 1 -- 172.21.15.52:0/3528665262 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70401a5300 con 0x7f7040100a00 2024-03-04T09:29:57.527 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:57.528+0000 7f70472e3700 1 -- 172.21.15.52:0/3528665262 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7040068d80 con 0x7f7040100a00 2024-03-04T09:29:57.528 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:57.529+0000 7f703e7fc700 1 -- 172.21.15.52:0/3528665262 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 17) v1 ==== 136807+0+0 (secure 0 0 0) 0x7f7030012050 con 0x7f7040100a00 2024-03-04T09:29:57.528 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:57.530+0000 7f703e7fc700 1 --2- 172.21.15.52:0/3528665262 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f70280a1ea0 0x7f70280a4360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:29:57.528 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:57.530+0000 7f704487e700 1 --2- 172.21.15.52:0/3528665262 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f70280a1ea0 0x7f70280a4360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:29:57.528 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:57.530+0000 7f703e7fc700 1 -- 172.21.15.52:0/3528665262 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9092+0+0 (secure 0 0 0) 0x7f70300ce1f0 con 0x7f7040100a00 2024-03-04T09:29:57.528 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:57.530+0000 7f704487e700 1 --2- 172.21.15.52:0/3528665262 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f70280a1ea0 0x7f70280a4360 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f7040076a20 tx=0x7f7034009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:29:57.532 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:57.535+0000 7f703e7fc700 1 -- 172.21.15.52:0/3528665262 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f7030010040 con 0x7f7040100a00 2024-03-04T09:29:57.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:29:57 smithi191 ceph-mon[36816]: pgmap v329: 129 pgs: 129 active+clean; 641 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 5 op/s; 0 B/s, 0 objects/s recovering 2024-03-04T09:29:57.631 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:29:57 smithi106 ceph-mon[36484]: pgmap v329: 129 pgs: 129 active+clean; 641 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 5 op/s; 0 B/s, 0 objects/s recovering 2024-03-04T09:29:57.736 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:57.739+0000 7f70472e3700 1 -- 172.21.15.52:0/3528665262 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f704002d9b0 con 0x7f7040100a00 2024-03-04T09:29:57.737 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:57.740+0000 7f703e7fc700 1 -- 172.21.15.52:0/3528665262 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f703009b100 con 0x7f7040100a00 2024-03-04T09:29:57.737 INFO:teuthology.orchestra.run.smithi052.stdout:77309411402 2024-03-04T09:29:57.739 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:57.742+0000 7f70472e3700 1 -- 172.21.15.52:0/3528665262 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f70280a1ea0 msgr2=0x7f70280a4360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:29:57.739 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:57.742+0000 7f70472e3700 1 --2- 172.21.15.52:0/3528665262 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f70280a1ea0 0x7f70280a4360 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f7040076a20 tx=0x7f7034009260 comp rx=0 tx=0).stop 2024-03-04T09:29:57.739 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:57.742+0000 7f70472e3700 1 -- 172.21.15.52:0/3528665262 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7040100a00 msgr2=0x7f704019e430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:29:57.739 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:57.742+0000 7f70472e3700 1 --2- 172.21.15.52:0/3528665262 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7040100a00 0x7f704019e430 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f703000ef10 tx=0x7f703000c3a0 comp rx=0 tx=0).stop 2024-03-04T09:29:57.739 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:57.742+0000 7f70472e3700 1 -- 172.21.15.52:0/3528665262 shutdown_connections 2024-03-04T09:29:57.739 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:57.742+0000 7f70472e3700 1 --2- 172.21.15.52:0/3528665262 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f70280a1ea0 0x7f70280a4360 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:29:57.740 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:57.742+0000 7f70472e3700 1 --2- 172.21.15.52:0/3528665262 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f70401067e0 0x7f70401985f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:29:57.740 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:57.742+0000 7f70472e3700 1 --2- 172.21.15.52:0/3528665262 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7040100a00 0x7f704019e430 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:29:57.740 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:57.742+0000 7f70472e3700 1 --2- 172.21.15.52:0/3528665262 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f70401013c0 0x7f704019e970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:29:57.740 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:57.742+0000 7f70472e3700 1 -- 172.21.15.52:0/3528665262 >> 172.21.15.52:0/3528665262 conn(0x7f7040076300 msgr2=0x7f70400febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:29:57.740 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:57.742+0000 7f70472e3700 1 -- 172.21.15.52:0/3528665262 shutdown_connections 2024-03-04T09:29:57.740 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:29:57.742+0000 7f70472e3700 1 -- 172.21.15.52:0/3528665262 wait complete. 2024-03-04T09:29:58.430 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:29:58 smithi052 ceph-mon[32473]: from='client.? 172.21.15.52:0/3528665262' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-03-04T09:29:58.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:29:58 smithi191 ceph-mon[36816]: from='client.? 172.21.15.52:0/3528665262' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-03-04T09:29:58.631 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:29:58 smithi106 ceph-mon[36484]: from='client.? 172.21.15.52:0/3528665262' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-03-04T09:29:58.638 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/4e059f08-da08-11ee-95c6-87774f69a715/mon.a/config 2024-03-04T09:29:59.513 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:29:59 smithi052 ceph-mon[32473]: pgmap v330: 129 pgs: 129 active+clean; 641 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s; 0 B/s, 0 objects/s recovering 2024-03-04T09:29:59.532 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:29:59 smithi106 ceph-mon[36484]: pgmap v330: 129 pgs: 129 active+clean; 641 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s; 0 B/s, 0 objects/s recovering 2024-03-04T09:29:59.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:29:59 smithi191 ceph-mon[36816]: pgmap v330: 129 pgs: 129 active+clean; 641 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s; 0 B/s, 0 objects/s recovering 2024-03-04T09:30:00.243 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:30:00 smithi052 ceph-mon[32473]: overall HEALTH_OK 2024-03-04T09:30:00.277 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574921 got 55834574926 for osd.1 2024-03-04T09:30:00.277 DEBUG:teuthology.parallel:result is None 2024-03-04T09:30:00.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:30:00 smithi191 ceph-mon[36816]: overall HEALTH_OK 2024-03-04T09:30:00.631 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:30:00 smithi106 ceph-mon[36484]: overall HEALTH_OK 2024-03-04T09:30:01.246 INFO:tasks.cephadm.ceph_manager.ceph:need seq 163208757305 got 163208757307 for osd.5 2024-03-04T09:30:01.246 DEBUG:teuthology.parallel:result is None 2024-03-04T09:30:01.452 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:30:01 smithi052 ceph-mon[32473]: pgmap v331: 129 pgs: 129 active+clean; 641 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-04T09:30:01.453 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:01.456+0000 7f9d2661f700 1 -- 172.21.15.52:0/3479432269 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9d20102ad0 msgr2=0x7f9d2010ef10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:30:01.453 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:01.456+0000 7f9d2661f700 1 --2- 172.21.15.52:0/3479432269 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9d20102ad0 0x7f9d2010ef10 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f9d1800b0d0 tx=0x7f9d1800b3e0 comp rx=0 tx=0).stop 2024-03-04T09:30:01.453 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:01.456+0000 7f9d2661f700 1 -- 172.21.15.52:0/3479432269 shutdown_connections 2024-03-04T09:30:01.453 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:01.456+0000 7f9d2661f700 1 --2- 172.21.15.52:0/3479432269 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9d20102ad0 0x7f9d2010ef10 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:01.453 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:01.456+0000 7f9d2661f700 1 --2- 172.21.15.52:0/3479432269 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9d201080f0 0x7f9d201084d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:01.453 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:01.456+0000 7f9d2661f700 1 --2- 172.21.15.52:0/3479432269 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9d20102110 0x7f9d20102590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:01.453 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:01.456+0000 7f9d2661f700 1 -- 172.21.15.52:0/3479432269 >> 172.21.15.52:0/3479432269 conn(0x7f9d200791f0 msgr2=0x7f9d20079600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:30:01.453 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:01.456+0000 7f9d2661f700 1 -- 172.21.15.52:0/3479432269 shutdown_connections 2024-03-04T09:30:01.453 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:01.456+0000 7f9d2661f700 1 -- 172.21.15.52:0/3479432269 wait complete. 2024-03-04T09:30:01.455 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:01.456+0000 7f9d2661f700 1 Processor -- start 2024-03-04T09:30:01.455 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:01.456+0000 7f9d2661f700 1 -- start start 2024-03-04T09:30:01.455 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:01.457+0000 7f9d2661f700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9d20102110 0x7f9d200743c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:30:01.455 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:01.457+0000 7f9d2661f700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9d20102ad0 0x7f9d20074900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:30:01.455 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:01.457+0000 7f9d2661f700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9d201080f0 0x7f9d20074e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:30:01.455 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:01.457+0000 7f9d2661f700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d201123c0 con 0x7f9d20102ad0 2024-03-04T09:30:01.455 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:01.457+0000 7f9d2661f700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d20112240 con 0x7f9d20102110 2024-03-04T09:30:01.455 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:01.457+0000 7f9d2661f700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d20112540 con 0x7f9d201080f0 2024-03-04T09:30:01.456 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:01.457+0000 7f9d1f7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9d20102ad0 0x7f9d20074900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:30:01.456 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:01.457+0000 7f9d1ffff700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9d20102110 0x7f9d200743c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:30:01.456 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:01.457+0000 7f9d1f7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9d20102ad0 0x7f9d20074900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:42150/0 (socket says 172.21.15.52:42150) 2024-03-04T09:30:01.456 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:01.457+0000 7f9d1f7fe700 1 -- 172.21.15.52:0/3020917994 learned_addr learned my addr 172.21.15.52:0/3020917994 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-04T09:30:01.456 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:01.457+0000 7f9d24bbc700 1 --2- 172.21.15.52:0/3020917994 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9d201080f0 0x7f9d20074e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:30:01.456 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:01.457+0000 7f9d1f7fe700 1 -- 172.21.15.52:0/3020917994 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9d201080f0 msgr2=0x7f9d20074e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:30:01.456 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:01.457+0000 7f9d1f7fe700 1 --2- 172.21.15.52:0/3020917994 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9d201080f0 0x7f9d20074e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:01.456 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:01.457+0000 7f9d1f7fe700 1 -- 172.21.15.52:0/3020917994 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9d20102110 msgr2=0x7f9d200743c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:30:01.456 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:01.457+0000 7f9d1f7fe700 1 --2- 172.21.15.52:0/3020917994 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9d20102110 0x7f9d200743c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:01.456 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:01.457+0000 7f9d1f7fe700 1 -- 172.21.15.52:0/3020917994 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d201a6cc0 con 0x7f9d20102ad0 2024-03-04T09:30:01.456 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:01.457+0000 7f9d24bbc700 1 --2- 172.21.15.52:0/3020917994 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9d201080f0 0x7f9d20074e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-04T09:30:01.456 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:01.457+0000 7f9d1ffff700 1 --2- 172.21.15.52:0/3020917994 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9d20102110 0x7f9d200743c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-04T09:30:01.456 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:01.458+0000 7f9d1f7fe700 1 --2- 172.21.15.52:0/3020917994 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9d20102ad0 0x7f9d20074900 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f9d1400e9a0 tx=0x7f9d1400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:30:01.456 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:01.458+0000 7f9d1d7fa700 1 -- 172.21.15.52:0/3020917994 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d1400ca10 con 0x7f9d20102ad0 2024-03-04T09:30:01.457 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:01.458+0000 7f9d2661f700 1 -- 172.21.15.52:0/3020917994 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d201a6fb0 con 0x7f9d20102ad0 2024-03-04T09:30:01.457 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:01.458+0000 7f9d1d7fa700 1 -- 172.21.15.52:0/3020917994 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9d1400cbb0 con 0x7f9d20102ad0 2024-03-04T09:30:01.457 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:01.458+0000 7f9d1d7fa700 1 -- 172.21.15.52:0/3020917994 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d14018680 con 0x7f9d20102ad0 2024-03-04T09:30:01.457 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:01.458+0000 7f9d2661f700 1 -- 172.21.15.52:0/3020917994 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d201a74b0 con 0x7f9d20102ad0 2024-03-04T09:30:01.458 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:01.459+0000 7f9d2661f700 1 -- 172.21.15.52:0/3020917994 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d20068d80 con 0x7f9d20102ad0 2024-03-04T09:30:01.458 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:01.460+0000 7f9d1d7fa700 1 -- 172.21.15.52:0/3020917994 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 17) v1 ==== 136807+0+0 (secure 0 0 0) 0x7f9d14010430 con 0x7f9d20102ad0 2024-03-04T09:30:01.458 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:01.460+0000 7f9d1d7fa700 1 --2- 172.21.15.52:0/3020917994 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f9d080a1ea0 0x7f9d080a4360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:30:01.458 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:01.461+0000 7f9d1ffff700 1 --2- 172.21.15.52:0/3020917994 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f9d080a1ea0 0x7f9d080a4360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:30:01.458 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:01.461+0000 7f9d1d7fa700 1 -- 172.21.15.52:0/3020917994 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9092+0+0 (secure 0 0 0) 0x7f9d140ce4b0 con 0x7f9d20102ad0 2024-03-04T09:30:01.458 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:01.461+0000 7f9d1ffff700 1 --2- 172.21.15.52:0/3020917994 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f9d080a1ea0 0x7f9d080a4360 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f9d10009680 tx=0x7f9d10009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:30:01.462 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:01.465+0000 7f9d1d7fa700 1 -- 172.21.15.52:0/3020917994 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f9d1409b420 con 0x7f9d20102ad0 2024-03-04T09:30:01.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:30:01 smithi191 ceph-mon[36816]: pgmap v331: 129 pgs: 129 active+clean; 641 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-04T09:30:01.633 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:30:01 smithi106 ceph-mon[36484]: pgmap v331: 129 pgs: 129 active+clean; 641 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-04T09:30:01.669 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:01.672+0000 7f9d2661f700 1 -- 172.21.15.52:0/3020917994 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) v1 -- 0x7f9d2002dc10 con 0x7f9d20102ad0 2024-03-04T09:30:01.669 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:01.672+0000 7f9d1d7fa700 1 -- 172.21.15.52:0/3020917994 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) v1 ==== 75+0+13 (secure 0 0 0) 0x7f9d1409ad20 con 0x7f9d20102ad0 2024-03-04T09:30:01.669 INFO:teuthology.orchestra.run.smithi052.stdout:309237645345 2024-03-04T09:30:01.671 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:01.674+0000 7f9d2661f700 1 -- 172.21.15.52:0/3020917994 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f9d080a1ea0 msgr2=0x7f9d080a4360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:30:01.671 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:01.674+0000 7f9d2661f700 1 --2- 172.21.15.52:0/3020917994 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f9d080a1ea0 0x7f9d080a4360 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f9d10009680 tx=0x7f9d10009260 comp rx=0 tx=0).stop 2024-03-04T09:30:01.671 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:01.674+0000 7f9d2661f700 1 -- 172.21.15.52:0/3020917994 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9d20102ad0 msgr2=0x7f9d20074900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:30:01.671 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:01.674+0000 7f9d2661f700 1 --2- 172.21.15.52:0/3020917994 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9d20102ad0 0x7f9d20074900 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f9d1400e9a0 tx=0x7f9d1400ecb0 comp rx=0 tx=0).stop 2024-03-04T09:30:01.671 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:01.674+0000 7f9d2661f700 1 -- 172.21.15.52:0/3020917994 shutdown_connections 2024-03-04T09:30:01.672 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:01.674+0000 7f9d2661f700 1 --2- 172.21.15.52:0/3020917994 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f9d080a1ea0 0x7f9d080a4360 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:01.672 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:01.674+0000 7f9d2661f700 1 --2- 172.21.15.52:0/3020917994 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9d201080f0 0x7f9d20074e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:01.672 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:01.674+0000 7f9d2661f700 1 --2- 172.21.15.52:0/3020917994 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9d20102110 0x7f9d200743c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:01.672 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:01.674+0000 7f9d2661f700 1 --2- 172.21.15.52:0/3020917994 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9d20102ad0 0x7f9d20074900 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:01.672 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:01.674+0000 7f9d2661f700 1 -- 172.21.15.52:0/3020917994 >> 172.21.15.52:0/3020917994 conn(0x7f9d200791f0 msgr2=0x7f9d20100cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:30:01.672 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:01.674+0000 7f9d2661f700 1 -- 172.21.15.52:0/3020917994 shutdown_connections 2024-03-04T09:30:01.672 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:01.674+0000 7f9d2661f700 1 -- 172.21.15.52:0/3020917994 wait complete. 2024-03-04T09:30:01.920 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411399 got 77309411402 for osd.2 2024-03-04T09:30:01.920 DEBUG:teuthology.parallel:result is None 2024-03-04T09:30:02.504 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:30:02 smithi052 ceph-mon[32473]: from='client.? 172.21.15.52:0/3020917994' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-03-04T09:30:02.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:30:02 smithi191 ceph-mon[36816]: from='client.? 172.21.15.52:0/3020917994' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-03-04T09:30:02.631 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:30:02 smithi106 ceph-mon[36484]: from='client.? 172.21.15.52:0/3020917994' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-03-04T09:30:02.811 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/4e059f08-da08-11ee-95c6-87774f69a715/mon.a/config 2024-03-04T09:30:03.462 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:30:03 smithi052 ceph-mon[32473]: pgmap v332: 129 pgs: 129 active+clean; 641 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-04T09:30:03.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:30:03 smithi191 ceph-mon[36816]: pgmap v332: 129 pgs: 129 active+clean; 641 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-04T09:30:03.631 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:30:03 smithi106 ceph-mon[36484]: pgmap v332: 129 pgs: 129 active+clean; 641 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-04T09:30:05.507 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:30:05 smithi052 ceph-mon[32473]: pgmap v333: 129 pgs: 129 active+clean; 641 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-04T09:30:05.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:30:05 smithi191 ceph-mon[36816]: pgmap v333: 129 pgs: 129 active+clean; 641 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-04T09:30:05.631 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:30:05 smithi106 ceph-mon[36484]: pgmap v333: 129 pgs: 129 active+clean; 641 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-04T09:30:05.707 INFO:tasks.cephadm.ceph_manager.ceph:need seq 309237645341 got 309237645345 for osd.10 2024-03-04T09:30:05.707 DEBUG:teuthology.parallel:result is None 2024-03-04T09:30:05.984 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:05.986+0000 7ff708a94700 1 -- 172.21.15.52:0/1096148418 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff7041073d0 msgr2=0x7ff70410efa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:30:05.984 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:05.986+0000 7ff708a94700 1 --2- 172.21.15.52:0/1096148418 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff7041073d0 0x7ff70410efa0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7ff6fc009f90 tx=0x7ff6fc00b330 comp rx=0 tx=0).stop 2024-03-04T09:30:05.984 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:05.987+0000 7ff708a94700 1 -- 172.21.15.52:0/1096148418 shutdown_connections 2024-03-04T09:30:05.984 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:05.987+0000 7ff708a94700 1 --2- 172.21.15.52:0/1096148418 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff7041023a0 0x7ff704102780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:05.984 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:05.987+0000 7ff708a94700 1 --2- 172.21.15.52:0/1096148418 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff7041073d0 0x7ff70410efa0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:05.984 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:05.987+0000 7ff708a94700 1 --2- 172.21.15.52:0/1096148418 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff704102d50 0x7ff704106cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:05.984 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:05.987+0000 7ff708a94700 1 -- 172.21.15.52:0/1096148418 >> 172.21.15.52:0/1096148418 conn(0x7ff704079150 msgr2=0x7ff704079560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:30:05.984 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:05.987+0000 7ff708a94700 1 -- 172.21.15.52:0/1096148418 shutdown_connections 2024-03-04T09:30:05.985 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:05.987+0000 7ff708a94700 1 -- 172.21.15.52:0/1096148418 wait complete. 2024-03-04T09:30:05.986 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:05.987+0000 7ff708a94700 1 Processor -- start 2024-03-04T09:30:05.986 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:05.987+0000 7ff708a94700 1 -- start start 2024-03-04T09:30:05.986 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:05.988+0000 7ff708a94700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff7041023a0 0x7ff7040743c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:30:05.986 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:05.988+0000 7ff708a94700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff704102d50 0x7ff704074900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:30:05.986 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:05.988+0000 7ff708a94700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff7041073d0 0x7ff704074e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:30:05.986 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:05.988+0000 7ff708a94700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7041123b0 con 0x7ff704102d50 2024-03-04T09:30:05.986 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:05.988+0000 7ff708a94700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff704112230 con 0x7ff7041023a0 2024-03-04T09:30:05.986 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:05.988+0000 7ff708a94700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff704112530 con 0x7ff7041073d0 2024-03-04T09:30:05.986 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:05.988+0000 7ff701d9b700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff704102d50 0x7ff704074900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:30:05.986 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:05.988+0000 7ff701d9b700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff704102d50 0x7ff704074900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:47070/0 (socket says 172.21.15.52:47070) 2024-03-04T09:30:05.987 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:05.988+0000 7ff702d9d700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff7041073d0 0x7ff704074e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:30:05.987 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:05.988+0000 7ff701d9b700 1 -- 172.21.15.52:0/2721352930 learned_addr learned my addr 172.21.15.52:0/2721352930 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-04T09:30:05.987 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:05.988+0000 7ff701d9b700 1 -- 172.21.15.52:0/2721352930 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff7041073d0 msgr2=0x7ff704074e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:30:05.987 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:05.988+0000 7ff701d9b700 1 --2- 172.21.15.52:0/2721352930 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff7041073d0 0x7ff704074e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:05.987 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:05.988+0000 7ff701d9b700 1 -- 172.21.15.52:0/2721352930 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff7041023a0 msgr2=0x7ff7040743c0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-03-04T09:30:05.987 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:05.988+0000 7ff701d9b700 1 --2- 172.21.15.52:0/2721352930 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff7041023a0 0x7ff7040743c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:05.987 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:05.988+0000 7ff701d9b700 1 -- 172.21.15.52:0/2721352930 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7041a6cd0 con 0x7ff704102d50 2024-03-04T09:30:05.987 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:05.988+0000 7ff702d9d700 1 --2- 172.21.15.52:0/2721352930 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff7041073d0 0x7ff704074e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-04T09:30:05.987 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:05.988+0000 7ff701d9b700 1 --2- 172.21.15.52:0/2721352930 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff704102d50 0x7ff704074900 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7ff6f800ec70 tx=0x7ff6f800ef80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:30:05.987 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:05.989+0000 7ff6f37fe700 1 -- 172.21.15.52:0/2721352930 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6f800cd40 con 0x7ff704102d50 2024-03-04T09:30:05.987 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:05.989+0000 7ff708a94700 1 -- 172.21.15.52:0/2721352930 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7041a6fc0 con 0x7ff704102d50 2024-03-04T09:30:05.987 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:05.989+0000 7ff6f37fe700 1 -- 172.21.15.52:0/2721352930 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff6f8004500 con 0x7ff704102d50 2024-03-04T09:30:05.987 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:05.989+0000 7ff6f37fe700 1 -- 172.21.15.52:0/2721352930 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6f8010690 con 0x7ff704102d50 2024-03-04T09:30:05.988 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:05.989+0000 7ff708a94700 1 -- 172.21.15.52:0/2721352930 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7041a7470 con 0x7ff704102d50 2024-03-04T09:30:05.988 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:05.990+0000 7ff708a94700 1 -- 172.21.15.52:0/2721352930 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff704068d80 con 0x7ff704102d50 2024-03-04T09:30:05.988 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:05.991+0000 7ff6f37fe700 1 -- 172.21.15.52:0/2721352930 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 17) v1 ==== 136807+0+0 (secure 0 0 0) 0x7ff6f8003ee0 con 0x7ff704102d50 2024-03-04T09:30:05.989 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:05.992+0000 7ff6f37fe700 1 --2- 172.21.15.52:0/2721352930 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7ff6ec0a1ea0 0x7ff6ec0a4360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:30:05.989 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:05.992+0000 7ff6f37fe700 1 -- 172.21.15.52:0/2721352930 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9092+0+0 (secure 0 0 0) 0x7ff6f80ce570 con 0x7ff704102d50 2024-03-04T09:30:05.989 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:05.992+0000 7ff70259c700 1 --2- 172.21.15.52:0/2721352930 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7ff6ec0a1ea0 0x7ff6ec0a4360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:30:05.989 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:05.993+0000 7ff70259c700 1 --2- 172.21.15.52:0/2721352930 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7ff6ec0a1ea0 0x7ff6ec0a4360 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7ff6f4009af0 tx=0x7ff6f4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:30:05.994 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:05.998+0000 7ff6f37fe700 1 -- 172.21.15.52:0/2721352930 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7ff6f809b4e0 con 0x7ff704102d50 2024-03-04T09:30:06.207 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:06.210+0000 7ff708a94700 1 -- 172.21.15.52:0/2721352930 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7ff70402da50 con 0x7ff704102d50 2024-03-04T09:30:06.207 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:06.211+0000 7ff6f37fe700 1 -- 172.21.15.52:0/2721352930 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7ff6f809ade0 con 0x7ff704102d50 2024-03-04T09:30:06.208 INFO:teuthology.orchestra.run.smithi052.stdout:197568495672 2024-03-04T09:30:06.210 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:06.213+0000 7ff708a94700 1 -- 172.21.15.52:0/2721352930 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7ff6ec0a1ea0 msgr2=0x7ff6ec0a4360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:30:06.210 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:06.213+0000 7ff708a94700 1 --2- 172.21.15.52:0/2721352930 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7ff6ec0a1ea0 0x7ff6ec0a4360 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7ff6f4009af0 tx=0x7ff6f4009260 comp rx=0 tx=0).stop 2024-03-04T09:30:06.210 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:06.213+0000 7ff708a94700 1 -- 172.21.15.52:0/2721352930 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff704102d50 msgr2=0x7ff704074900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:30:06.210 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:06.213+0000 7ff708a94700 1 --2- 172.21.15.52:0/2721352930 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff704102d50 0x7ff704074900 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7ff6f800ec70 tx=0x7ff6f800ef80 comp rx=0 tx=0).stop 2024-03-04T09:30:06.210 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:06.213+0000 7ff708a94700 1 -- 172.21.15.52:0/2721352930 shutdown_connections 2024-03-04T09:30:06.210 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:06.213+0000 7ff708a94700 1 --2- 172.21.15.52:0/2721352930 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7ff6ec0a1ea0 0x7ff6ec0a4360 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:06.210 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:06.213+0000 7ff708a94700 1 --2- 172.21.15.52:0/2721352930 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff7041073d0 0x7ff704074e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:06.210 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:06.213+0000 7ff708a94700 1 --2- 172.21.15.52:0/2721352930 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff7041023a0 0x7ff7040743c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:06.211 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:06.213+0000 7ff708a94700 1 --2- 172.21.15.52:0/2721352930 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff704102d50 0x7ff704074900 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:06.211 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:06.213+0000 7ff708a94700 1 -- 172.21.15.52:0/2721352930 >> 172.21.15.52:0/2721352930 conn(0x7ff704079150 msgr2=0x7ff7041001f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:30:06.211 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:06.213+0000 7ff708a94700 1 -- 172.21.15.52:0/2721352930 shutdown_connections 2024-03-04T09:30:06.211 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:06.213+0000 7ff708a94700 1 -- 172.21.15.52:0/2721352930 wait complete. 2024-03-04T09:30:06.479 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:30:06 smithi052 ceph-mon[32473]: from='client.? 172.21.15.52:0/2417378449' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-03-04T09:30:06.558 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:06.482+0000 7f8b64cfa700 1 -- 172.21.15.52:0/372421012 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8b60107050 msgr2=0x7f8b60107430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:30:06.558 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:06.482+0000 7f8b64cfa700 1 --2- 172.21.15.52:0/372421012 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8b60107050 0x7f8b60107430 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f8b50009990 tx=0x7f8b50009ca0 comp rx=0 tx=0).stop 2024-03-04T09:30:06.558 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:06.482+0000 7f8b64cfa700 1 -- 172.21.15.52:0/372421012 shutdown_connections 2024-03-04T09:30:06.558 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:06.482+0000 7f8b64cfa700 1 --2- 172.21.15.52:0/372421012 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8b60101960 0x7f8b6010de00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:06.558 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:06.482+0000 7f8b64cfa700 1 --2- 172.21.15.52:0/372421012 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8b60107050 0x7f8b60107430 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:06.558 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:06.482+0000 7f8b64cfa700 1 --2- 172.21.15.52:0/372421012 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8b60100fa0 0x7f8b60101420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:06.558 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:06.482+0000 7f8b64cfa700 1 -- 172.21.15.52:0/372421012 >> 172.21.15.52:0/372421012 conn(0x7f8b60077fd0 msgr2=0x7f8b600783e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:30:06.558 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:06.482+0000 7f8b64cfa700 1 -- 172.21.15.52:0/372421012 shutdown_connections 2024-03-04T09:30:06.559 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:06.482+0000 7f8b64cfa700 1 -- 172.21.15.52:0/372421012 wait complete. 2024-03-04T09:30:06.559 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:06.483+0000 7f8b64cfa700 1 Processor -- start 2024-03-04T09:30:06.559 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:06.483+0000 7f8b64cfa700 1 -- start start 2024-03-04T09:30:06.559 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:06.483+0000 7f8b64cfa700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8b60100fa0 0x7f8b60102c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:30:06.559 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:06.483+0000 7f8b64cfa700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8b60101960 0x7f8b60103190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:30:06.559 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:06.483+0000 7f8b64cfa700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8b60107050 0x7f8b601036d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:30:06.559 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:06.483+0000 7f8b64cfa700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b60111290 con 0x7f8b60100fa0 2024-03-04T09:30:06.559 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:06.483+0000 7f8b64cfa700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b60111110 con 0x7f8b60101960 2024-03-04T09:30:06.559 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:06.483+0000 7f8b64cfa700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b60111410 con 0x7f8b60107050 2024-03-04T09:30:06.559 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:06.484+0000 7f8b5e59c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8b60100fa0 0x7f8b60102c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:30:06.559 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:06.484+0000 7f8b5e59c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8b60100fa0 0x7f8b60102c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:47090/0 (socket says 172.21.15.52:47090) 2024-03-04T09:30:06.559 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:06.484+0000 7f8b5e59c700 1 -- 172.21.15.52:0/2643183870 learned_addr learned my addr 172.21.15.52:0/2643183870 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-04T09:30:06.559 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:06.484+0000 7f8b5dd9b700 1 --2- 172.21.15.52:0/2643183870 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8b60101960 0x7f8b60103190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:30:06.560 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:06.484+0000 7f8b5ed9d700 1 --2- 172.21.15.52:0/2643183870 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8b60107050 0x7f8b601036d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:30:06.560 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:06.484+0000 7f8b5e59c700 1 -- 172.21.15.52:0/2643183870 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8b60107050 msgr2=0x7f8b601036d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:30:06.560 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:06.484+0000 7f8b5e59c700 1 --2- 172.21.15.52:0/2643183870 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8b60107050 0x7f8b601036d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:06.560 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:06.484+0000 7f8b5e59c700 1 -- 172.21.15.52:0/2643183870 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8b60101960 msgr2=0x7f8b60103190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:30:06.560 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:06.484+0000 7f8b5e59c700 1 --2- 172.21.15.52:0/2643183870 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8b60101960 0x7f8b60103190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:06.560 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:06.484+0000 7f8b5e59c700 1 -- 172.21.15.52:0/2643183870 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b601a6dc0 con 0x7f8b60100fa0 2024-03-04T09:30:06.560 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:06.484+0000 7f8b5dd9b700 1 --2- 172.21.15.52:0/2643183870 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8b60101960 0x7f8b60103190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-04T09:30:06.560 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:06.484+0000 7f8b5e59c700 1 --2- 172.21.15.52:0/2643183870 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8b60100fa0 0x7f8b60102c50 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f8b50000c00 tx=0x7f8b5000bab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:30:06.560 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:06.484+0000 7f8b4f7fe700 1 -- 172.21.15.52:0/2643183870 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b50027070 con 0x7f8b60100fa0 2024-03-04T09:30:06.560 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:06.484+0000 7f8b4f7fe700 1 -- 172.21.15.52:0/2643183870 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8b50003d70 con 0x7f8b60100fa0 2024-03-04T09:30:06.560 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:06.484+0000 7f8b64cfa700 1 -- 172.21.15.52:0/2643183870 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b601a7050 con 0x7f8b60100fa0 2024-03-04T09:30:06.560 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:06.484+0000 7f8b4f7fe700 1 -- 172.21.15.52:0/2643183870 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b5000f460 con 0x7f8b60100fa0 2024-03-04T09:30:06.560 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:06.484+0000 7f8b64cfa700 1 -- 172.21.15.52:0/2643183870 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b601a7530 con 0x7f8b60100fa0 2024-03-04T09:30:06.561 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:06.485+0000 7f8b64cfa700 1 -- 172.21.15.52:0/2643183870 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b60068d80 con 0x7f8b60100fa0 2024-03-04T09:30:06.561 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:06.486+0000 7f8b4f7fe700 1 -- 172.21.15.52:0/2643183870 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 17) v1 ==== 136807+0+0 (secure 0 0 0) 0x7f8b5002a070 con 0x7f8b60100fa0 2024-03-04T09:30:06.561 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:06.487+0000 7f8b4f7fe700 1 --2- 172.21.15.52:0/2643183870 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f8b480a1ea0 0x7f8b480a4360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:30:06.561 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:06.487+0000 7f8b5dd9b700 1 --2- 172.21.15.52:0/2643183870 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f8b480a1ea0 0x7f8b480a4360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:30:06.561 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:06.487+0000 7f8b4f7fe700 1 -- 172.21.15.52:0/2643183870 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9092+0+0 (secure 0 0 0) 0x7f8b500cf460 con 0x7f8b60100fa0 2024-03-04T09:30:06.561 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:06.487+0000 7f8b5dd9b700 1 --2- 172.21.15.52:0/2643183870 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f8b480a1ea0 0x7f8b480a4360 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f8b600785f0 tx=0x7f8b54009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:30:06.561 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:06.490+0000 7f8b4f7fe700 1 -- 172.21.15.52:0/2643183870 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f8b50013070 con 0x7f8b60100fa0 2024-03-04T09:30:06.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:30:06 smithi191 ceph-mon[36816]: from='client.? 172.21.15.52:0/2417378449' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-03-04T09:30:06.631 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:30:06 smithi106 ceph-mon[36484]: from='client.? 172.21.15.52:0/2417378449' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-03-04T09:30:06.693 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:06.696+0000 7f8b64cfa700 1 -- 172.21.15.52:0/2643183870 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) v1 -- 0x7f8b601a7790 con 0x7f8b60100fa0 2024-03-04T09:30:06.693 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:06.696+0000 7f8b4f7fe700 1 -- 172.21.15.52:0/2643183870 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f8b5009c400 con 0x7f8b60100fa0 2024-03-04T09:30:06.693 INFO:teuthology.orchestra.run.smithi052.stdout:253403070509 2024-03-04T09:30:06.695 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:06.698+0000 7f8b64cfa700 1 -- 172.21.15.52:0/2643183870 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f8b480a1ea0 msgr2=0x7f8b480a4360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:30:06.695 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:06.698+0000 7f8b64cfa700 1 --2- 172.21.15.52:0/2643183870 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f8b480a1ea0 0x7f8b480a4360 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f8b600785f0 tx=0x7f8b54009260 comp rx=0 tx=0).stop 2024-03-04T09:30:06.695 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:06.698+0000 7f8b64cfa700 1 -- 172.21.15.52:0/2643183870 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8b60100fa0 msgr2=0x7f8b60102c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:30:06.695 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:06.698+0000 7f8b64cfa700 1 --2- 172.21.15.52:0/2643183870 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8b60100fa0 0x7f8b60102c50 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f8b50000c00 tx=0x7f8b5000bab0 comp rx=0 tx=0).stop 2024-03-04T09:30:06.695 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:06.698+0000 7f8b64cfa700 1 -- 172.21.15.52:0/2643183870 shutdown_connections 2024-03-04T09:30:06.695 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:06.698+0000 7f8b64cfa700 1 --2- 172.21.15.52:0/2643183870 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f8b480a1ea0 0x7f8b480a4360 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:06.696 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:06.698+0000 7f8b64cfa700 1 --2- 172.21.15.52:0/2643183870 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8b60107050 0x7f8b601036d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:06.696 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:06.698+0000 7f8b64cfa700 1 --2- 172.21.15.52:0/2643183870 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8b60100fa0 0x7f8b60102c50 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:06.696 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:06.698+0000 7f8b64cfa700 1 --2- 172.21.15.52:0/2643183870 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8b60101960 0x7f8b60103190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:06.696 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:06.698+0000 7f8b64cfa700 1 -- 172.21.15.52:0/2643183870 >> 172.21.15.52:0/2643183870 conn(0x7f8b60077fd0 msgr2=0x7f8b60100340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:30:06.696 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:06.698+0000 7f8b64cfa700 1 -- 172.21.15.52:0/2643183870 shutdown_connections 2024-03-04T09:30:06.696 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:06.698+0000 7f8b64cfa700 1 -- 172.21.15.52:0/2643183870 wait complete. 2024-03-04T09:30:06.715 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:05.884+0000 7f3795ffc700 1 -- 172.21.15.52:0/3726318035 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3790102320 msgr2=0x7f3790102700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:30:06.715 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:05.884+0000 7f3795ffc700 1 --2- 172.21.15.52:0/3726318035 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3790102320 0x7f3790102700 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f37800099c0 tx=0x7f3780009cd0 comp rx=0 tx=0).stop 2024-03-04T09:30:06.715 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:05.884+0000 7f3795ffc700 1 -- 172.21.15.52:0/3726318035 shutdown_connections 2024-03-04T09:30:06.715 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:05.884+0000 7f3795ffc700 1 --2- 172.21.15.52:0/3726318035 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3790107400 0x7f379010efd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:06.715 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:05.884+0000 7f3795ffc700 1 --2- 172.21.15.52:0/3726318035 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3790102320 0x7f3790102700 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:06.715 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:05.884+0000 7f3795ffc700 1 --2- 172.21.15.52:0/3726318035 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3790102cd0 0x7f3790106d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:06.715 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:05.884+0000 7f3795ffc700 1 -- 172.21.15.52:0/3726318035 >> 172.21.15.52:0/3726318035 conn(0x7f37900fdb90 msgr2=0x7f37900fffb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:30:06.715 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:05.884+0000 7f3795ffc700 1 -- 172.21.15.52:0/3726318035 shutdown_connections 2024-03-04T09:30:06.716 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:05.884+0000 7f3795ffc700 1 -- 172.21.15.52:0/3726318035 wait complete. 2024-03-04T09:30:06.716 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:05.885+0000 7f3795ffc700 1 Processor -- start 2024-03-04T09:30:06.716 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:05.885+0000 7f3795ffc700 1 -- start start 2024-03-04T09:30:06.716 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:05.885+0000 7f3795ffc700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3790102320 0x7f37900743c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:30:06.716 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:05.885+0000 7f3795ffc700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3790102cd0 0x7f3790074900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:30:06.716 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:05.885+0000 7f3795ffc700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3790107400 0x7f3790074e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:30:06.716 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:05.885+0000 7f3795ffc700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37901123e0 con 0x7f3790107400 2024-03-04T09:30:06.716 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:05.885+0000 7f3795ffc700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3790112260 con 0x7f3790102320 2024-03-04T09:30:06.716 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:05.885+0000 7f3795ffc700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3790112560 con 0x7f3790102cd0 2024-03-04T09:30:06.716 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:05.885+0000 7f378ffff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3790107400 0x7f3790074e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:30:06.716 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:05.885+0000 7f378ffff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3790107400 0x7f3790074e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:47044/0 (socket says 172.21.15.52:47044) 2024-03-04T09:30:06.716 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:05.885+0000 7f378f7fe700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3790102320 0x7f37900743c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:30:06.716 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:05.885+0000 7f378ffff700 1 -- 172.21.15.52:0/2417378449 learned_addr learned my addr 172.21.15.52:0/2417378449 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-04T09:30:06.717 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:05.885+0000 7f378effd700 1 --2- 172.21.15.52:0/2417378449 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3790102cd0 0x7f3790074900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:30:06.717 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:05.885+0000 7f378ffff700 1 -- 172.21.15.52:0/2417378449 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3790102cd0 msgr2=0x7f3790074900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:30:06.717 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:05.885+0000 7f378ffff700 1 --2- 172.21.15.52:0/2417378449 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3790102cd0 0x7f3790074900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:06.717 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:05.885+0000 7f378ffff700 1 -- 172.21.15.52:0/2417378449 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3790102320 msgr2=0x7f37900743c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:30:06.717 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:05.885+0000 7f378ffff700 1 --2- 172.21.15.52:0/2417378449 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3790102320 0x7f37900743c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:06.717 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:05.885+0000 7f378ffff700 1 -- 172.21.15.52:0/2417378449 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37901a6de0 con 0x7f3790107400 2024-03-04T09:30:06.717 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:05.886+0000 7f378f7fe700 1 --2- 172.21.15.52:0/2417378449 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3790102320 0x7f37900743c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-04T09:30:06.717 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:05.886+0000 7f378ffff700 1 --2- 172.21.15.52:0/2417378449 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3790107400 0x7f3790074e40 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f378800ef30 tx=0x7f378800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:30:06.717 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:05.886+0000 7f378cff9700 1 -- 172.21.15.52:0/2417378449 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37880092e0 con 0x7f3790107400 2024-03-04T09:30:06.717 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:05.886+0000 7f378cff9700 1 -- 172.21.15.52:0/2417378449 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3788004bb0 con 0x7f3790107400 2024-03-04T09:30:06.717 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:05.886+0000 7f378cff9700 1 -- 172.21.15.52:0/2417378449 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3788004280 con 0x7f3790107400 2024-03-04T09:30:06.717 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:05.886+0000 7f3795ffc700 1 -- 172.21.15.52:0/2417378449 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37901a7070 con 0x7f3790107400 2024-03-04T09:30:06.717 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:05.886+0000 7f3795ffc700 1 -- 172.21.15.52:0/2417378449 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37901a7560 con 0x7f3790107400 2024-03-04T09:30:06.717 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:05.887+0000 7f3795ffc700 1 -- 172.21.15.52:0/2417378449 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3790068d80 con 0x7f3790107400 2024-03-04T09:30:06.717 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:05.888+0000 7f378cff9700 1 -- 172.21.15.52:0/2417378449 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 17) v1 ==== 136807+0+0 (secure 0 0 0) 0x7f378801b030 con 0x7f3790107400 2024-03-04T09:30:06.718 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:05.889+0000 7f378cff9700 1 --2- 172.21.15.52:0/2417378449 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f37780a1ea0 0x7f37780a4360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:30:06.718 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:05.889+0000 7f378f7fe700 1 --2- 172.21.15.52:0/2417378449 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f37780a1ea0 0x7f37780a4360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:30:06.718 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:05.889+0000 7f378cff9700 1 -- 172.21.15.52:0/2417378449 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9092+0+0 (secure 0 0 0) 0x7f37880ce740 con 0x7f3790107400 2024-03-04T09:30:06.718 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:05.889+0000 7f378f7fe700 1 --2- 172.21.15.52:0/2417378449 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f37780a1ea0 0x7f37780a4360 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f3780000c00 tx=0x7f3780005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:30:06.718 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:05.892+0000 7f378cff9700 1 -- 172.21.15.52:0/2417378449 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f3788022020 con 0x7f3790107400 2024-03-04T09:30:06.718 INFO:teuthology.orchestra.run.smithi052.stdout:141733920833 2024-03-04T09:30:06.718 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:06.107+0000 7f3795ffc700 1 -- 172.21.15.52:0/2417378449 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f379002db70 con 0x7f3790107400 2024-03-04T09:30:06.719 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:06.107+0000 7f378cff9700 1 -- 172.21.15.52:0/2417378449 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f3788010040 con 0x7f3790107400 2024-03-04T09:30:06.719 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:06.109+0000 7f3795ffc700 1 -- 172.21.15.52:0/2417378449 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f37780a1ea0 msgr2=0x7f37780a4360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:30:06.719 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:06.109+0000 7f3795ffc700 1 --2- 172.21.15.52:0/2417378449 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f37780a1ea0 0x7f37780a4360 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f3780000c00 tx=0x7f3780005fb0 comp rx=0 tx=0).stop 2024-03-04T09:30:06.719 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:06.109+0000 7f3795ffc700 1 -- 172.21.15.52:0/2417378449 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3790107400 msgr2=0x7f3790074e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:30:06.719 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:06.109+0000 7f3795ffc700 1 --2- 172.21.15.52:0/2417378449 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3790107400 0x7f3790074e40 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f378800ef30 tx=0x7f378800c3a0 comp rx=0 tx=0).stop 2024-03-04T09:30:06.719 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:06.109+0000 7f3795ffc700 1 -- 172.21.15.52:0/2417378449 shutdown_connections 2024-03-04T09:30:06.719 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:06.109+0000 7f3795ffc700 1 --2- 172.21.15.52:0/2417378449 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f37780a1ea0 0x7f37780a4360 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:06.719 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:06.109+0000 7f3795ffc700 1 --2- 172.21.15.52:0/2417378449 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3790107400 0x7f3790074e40 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:06.719 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:06.109+0000 7f3795ffc700 1 --2- 172.21.15.52:0/2417378449 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3790102320 0x7f37900743c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:06.719 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:06.109+0000 7f3795ffc700 1 --2- 172.21.15.52:0/2417378449 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3790102cd0 0x7f3790074900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:06.719 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:06.109+0000 7f3795ffc700 1 -- 172.21.15.52:0/2417378449 >> 172.21.15.52:0/2417378449 conn(0x7f37900fdb90 msgr2=0x7f37900ffec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:30:06.719 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:06.109+0000 7f3795ffc700 1 -- 172.21.15.52:0/2417378449 shutdown_connections 2024-03-04T09:30:06.719 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:06.109+0000 7f3795ffc700 1 -- 172.21.15.52:0/2417378449 wait complete. 2024-03-04T09:30:07.295 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:30:07 smithi052 ceph-mon[32473]: pgmap v334: 129 pgs: 129 active+clean; 641 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-04T09:30:07.295 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:30:07 smithi052 ceph-mon[32473]: from='client.? 172.21.15.52:0/2721352930' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-03-04T09:30:07.295 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:30:07 smithi052 ceph-mon[32473]: from='client.? 172.21.15.52:0/2643183870' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-03-04T09:30:07.324 INFO:tasks.cephadm.ceph_manager.ceph:need seq 141733920829 got 141733920833 for osd.4 2024-03-04T09:30:07.325 DEBUG:teuthology.parallel:result is None 2024-03-04T09:30:07.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:30:07 smithi191 ceph-mon[36816]: pgmap v334: 129 pgs: 129 active+clean; 641 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-04T09:30:07.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:30:07 smithi191 ceph-mon[36816]: from='client.? 172.21.15.52:0/2721352930' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-03-04T09:30:07.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:30:07 smithi191 ceph-mon[36816]: from='client.? 172.21.15.52:0/2643183870' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-03-04T09:30:07.631 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:30:07 smithi106 ceph-mon[36484]: pgmap v334: 129 pgs: 129 active+clean; 641 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-04T09:30:07.631 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:30:07 smithi106 ceph-mon[36484]: from='client.? 172.21.15.52:0/2721352930' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-03-04T09:30:07.631 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:30:07 smithi106 ceph-mon[36484]: from='client.? 172.21.15.52:0/2643183870' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-03-04T09:30:08.576 INFO:tasks.cephadm.ceph_manager.ceph:need seq 253403070504 got 253403070509 for osd.8 2024-03-04T09:30:08.576 DEBUG:teuthology.parallel:result is None 2024-03-04T09:30:09.227 INFO:tasks.cephadm.ceph_manager.ceph:need seq 197568495667 got 197568495672 for osd.6 2024-03-04T09:30:09.228 DEBUG:teuthology.parallel:result is None 2024-03-04T09:30:09.497 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:30:09 smithi052 ceph-mon[32473]: pgmap v335: 129 pgs: 129 active+clean; 641 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-03-04T09:30:09.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:30:09 smithi191 ceph-mon[36816]: pgmap v335: 129 pgs: 129 active+clean; 641 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-03-04T09:30:09.631 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:30:09 smithi106 ceph-mon[36484]: pgmap v335: 129 pgs: 129 active+clean; 641 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-03-04T09:30:10.591 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:10.594+0000 7f4b8690d700 1 -- 172.21.15.52:0/3083104202 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4b80102300 msgr2=0x7f4b801026e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:30:10.591 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:10.594+0000 7f4b8690d700 1 --2- 172.21.15.52:0/3083104202 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4b80102300 0x7f4b801026e0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f4b700099c0 tx=0x7f4b70009cd0 comp rx=0 tx=0).stop 2024-03-04T09:30:10.591 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:10.594+0000 7f4b8690d700 1 -- 172.21.15.52:0/3083104202 shutdown_connections 2024-03-04T09:30:10.592 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:10.594+0000 7f4b8690d700 1 --2- 172.21.15.52:0/3083104202 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4b801073e0 0x7f4b8010efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:10.592 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:10.594+0000 7f4b8690d700 1 --2- 172.21.15.52:0/3083104202 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4b80102300 0x7f4b801026e0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:10.592 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:10.594+0000 7f4b8690d700 1 --2- 172.21.15.52:0/3083104202 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4b80102cb0 0x7f4b80106d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:10.592 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:10.594+0000 7f4b8690d700 1 -- 172.21.15.52:0/3083104202 >> 172.21.15.52:0/3083104202 conn(0x7f4b800fdb70 msgr2=0x7f4b800fff90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:30:10.592 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:10.594+0000 7f4b8690d700 1 -- 172.21.15.52:0/3083104202 shutdown_connections 2024-03-04T09:30:10.592 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:10.594+0000 7f4b8690d700 1 -- 172.21.15.52:0/3083104202 wait complete. 2024-03-04T09:30:10.594 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:10.595+0000 7f4b8690d700 1 Processor -- start 2024-03-04T09:30:10.594 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:10.595+0000 7f4b8690d700 1 -- start start 2024-03-04T09:30:10.594 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:10.595+0000 7f4b8690d700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4b80102300 0x7f4b800743c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:30:10.594 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:10.595+0000 7f4b8690d700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4b80102cb0 0x7f4b80074900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:30:10.594 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:10.595+0000 7f4b8690d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4b801073e0 0x7f4b80074e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:30:10.594 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:10.595+0000 7f4b8690d700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b801123c0 con 0x7f4b801073e0 2024-03-04T09:30:10.594 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:10.595+0000 7f4b8690d700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b80112240 con 0x7f4b80102300 2024-03-04T09:30:10.594 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:10.595+0000 7f4b8690d700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b80112540 con 0x7f4b80102cb0 2024-03-04T09:30:10.594 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:10.595+0000 7f4b84eaa700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4b801073e0 0x7f4b80074e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:30:10.594 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:10.595+0000 7f4b84eaa700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4b801073e0 0x7f4b80074e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:47110/0 (socket says 172.21.15.52:47110) 2024-03-04T09:30:10.594 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:10.595+0000 7f4b84eaa700 1 -- 172.21.15.52:0/3439006652 learned_addr learned my addr 172.21.15.52:0/3439006652 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-04T09:30:10.594 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:10.595+0000 7f4b7ffff700 1 --2- 172.21.15.52:0/3439006652 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4b80102300 0x7f4b800743c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:30:10.594 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:10.596+0000 7f4b7f7fe700 1 --2- 172.21.15.52:0/3439006652 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4b80102cb0 0x7f4b80074900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:30:10.595 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:10.596+0000 7f4b84eaa700 1 -- 172.21.15.52:0/3439006652 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4b80102cb0 msgr2=0x7f4b80074900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:30:10.595 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:10.596+0000 7f4b84eaa700 1 --2- 172.21.15.52:0/3439006652 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4b80102cb0 0x7f4b80074900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:10.595 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:10.596+0000 7f4b84eaa700 1 -- 172.21.15.52:0/3439006652 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4b80102300 msgr2=0x7f4b800743c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:30:10.595 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:10.596+0000 7f4b84eaa700 1 --2- 172.21.15.52:0/3439006652 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4b80102300 0x7f4b800743c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:10.595 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:10.596+0000 7f4b84eaa700 1 -- 172.21.15.52:0/3439006652 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b801a6d80 con 0x7f4b801073e0 2024-03-04T09:30:10.595 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:10.596+0000 7f4b7ffff700 1 --2- 172.21.15.52:0/3439006652 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4b80102300 0x7f4b800743c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-04T09:30:10.595 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:10.596+0000 7f4b84eaa700 1 --2- 172.21.15.52:0/3439006652 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4b801073e0 0x7f4b80074e40 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f4b7800ef30 tx=0x7f4b7800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:30:10.595 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:10.596+0000 7f4b7d7fa700 1 -- 172.21.15.52:0/3439006652 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b780092e0 con 0x7f4b801073e0 2024-03-04T09:30:10.595 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:10.596+0000 7f4b8690d700 1 -- 172.21.15.52:0/3439006652 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b801a7010 con 0x7f4b801073e0 2024-03-04T09:30:10.595 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:10.596+0000 7f4b7d7fa700 1 -- 172.21.15.52:0/3439006652 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4b78004bb0 con 0x7f4b801073e0 2024-03-04T09:30:10.595 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:10.596+0000 7f4b7d7fa700 1 -- 172.21.15.52:0/3439006652 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b78004280 con 0x7f4b801073e0 2024-03-04T09:30:10.595 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:10.596+0000 7f4b8690d700 1 -- 172.21.15.52:0/3439006652 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b801a7500 con 0x7f4b801073e0 2024-03-04T09:30:10.596 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:10.597+0000 7f4b8690d700 1 -- 172.21.15.52:0/3439006652 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4b80068d80 con 0x7f4b801073e0 2024-03-04T09:30:10.596 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:10.599+0000 7f4b7d7fa700 1 -- 172.21.15.52:0/3439006652 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 17) v1 ==== 136807+0+0 (secure 0 0 0) 0x7f4b7801b030 con 0x7f4b801073e0 2024-03-04T09:30:10.596 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:10.599+0000 7f4b7d7fa700 1 --2- 172.21.15.52:0/3439006652 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f4b680a1ea0 0x7f4b680a4360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:30:10.596 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:10.600+0000 7f4b7ffff700 1 --2- 172.21.15.52:0/3439006652 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f4b680a1ea0 0x7f4b680a4360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:30:10.597 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:10.600+0000 7f4b7d7fa700 1 -- 172.21.15.52:0/3439006652 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9092+0+0 (secure 0 0 0) 0x7f4b780ce700 con 0x7f4b801073e0 2024-03-04T09:30:10.597 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:10.600+0000 7f4b7ffff700 1 --2- 172.21.15.52:0/3439006652 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f4b680a1ea0 0x7f4b680a4360 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f4b70000c00 tx=0x7f4b70005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:30:10.600 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:10.603+0000 7f4b7d7fa700 1 -- 172.21.15.52:0/3439006652 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f4b78022020 con 0x7f4b801073e0 2024-03-04T09:30:10.805 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:10.808+0000 7f4b8690d700 1 -- 172.21.15.52:0/3439006652 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f4b8002d9b0 con 0x7f4b801073e0 2024-03-04T09:30:10.805 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:10.808+0000 7f4b7d7fa700 1 -- 172.21.15.52:0/3439006652 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f4b78010040 con 0x7f4b801073e0 2024-03-04T09:30:10.805 INFO:teuthology.orchestra.run.smithi052.stdout:34359738455 2024-03-04T09:30:10.809 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:10.811+0000 7f4b8690d700 1 -- 172.21.15.52:0/3439006652 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f4b680a1ea0 msgr2=0x7f4b680a4360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:30:10.809 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:10.811+0000 7f4b8690d700 1 --2- 172.21.15.52:0/3439006652 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f4b680a1ea0 0x7f4b680a4360 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f4b70000c00 tx=0x7f4b70005fb0 comp rx=0 tx=0).stop 2024-03-04T09:30:10.809 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:10.811+0000 7f4b8690d700 1 -- 172.21.15.52:0/3439006652 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4b801073e0 msgr2=0x7f4b80074e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:30:10.809 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:10.811+0000 7f4b8690d700 1 --2- 172.21.15.52:0/3439006652 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4b801073e0 0x7f4b80074e40 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f4b7800ef30 tx=0x7f4b7800c3a0 comp rx=0 tx=0).stop 2024-03-04T09:30:10.809 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:10.811+0000 7f4b8690d700 1 -- 172.21.15.52:0/3439006652 shutdown_connections 2024-03-04T09:30:10.809 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:10.811+0000 7f4b8690d700 1 --2- 172.21.15.52:0/3439006652 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f4b680a1ea0 0x7f4b680a4360 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:10.809 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:10.811+0000 7f4b8690d700 1 --2- 172.21.15.52:0/3439006652 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4b801073e0 0x7f4b80074e40 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:10.809 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:10.811+0000 7f4b8690d700 1 --2- 172.21.15.52:0/3439006652 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4b80102300 0x7f4b800743c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:10.809 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:10.811+0000 7f4b8690d700 1 --2- 172.21.15.52:0/3439006652 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4b80102cb0 0x7f4b80074900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:10.809 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:10.811+0000 7f4b8690d700 1 -- 172.21.15.52:0/3439006652 >> 172.21.15.52:0/3439006652 conn(0x7f4b800fdb70 msgr2=0x7f4b800ff890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:30:10.810 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:10.811+0000 7f4b8690d700 1 -- 172.21.15.52:0/3439006652 shutdown_connections 2024-03-04T09:30:10.810 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:10.811+0000 7f4b8690d700 1 -- 172.21.15.52:0/3439006652 wait complete. 2024-03-04T09:30:11.462 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:30:11 smithi052 ceph-mon[32473]: pgmap v336: 129 pgs: 129 active+clean; 641 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-04T09:30:11.463 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:30:11 smithi052 ceph-mon[32473]: from='client.? 172.21.15.52:0/3439006652' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-03-04T09:30:11.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:30:11 smithi191 ceph-mon[36816]: pgmap v336: 129 pgs: 129 active+clean; 641 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-04T09:30:11.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:30:11 smithi191 ceph-mon[36816]: from='client.? 172.21.15.52:0/3439006652' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-03-04T09:30:11.631 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:30:11 smithi106 ceph-mon[36484]: pgmap v336: 129 pgs: 129 active+clean; 641 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-04T09:30:11.631 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:30:11 smithi106 ceph-mon[36484]: from='client.? 172.21.15.52:0/3439006652' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-03-04T09:30:11.664 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:11.664+0000 7f3b66da6700 1 -- 172.21.15.52:0/2078935169 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3b601081b0 msgr2=0x7f3b60108590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:30:11.664 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:11.664+0000 7f3b66da6700 1 --2- 172.21.15.52:0/2078935169 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3b601081b0 0x7f3b60108590 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f3b54009990 tx=0x7f3b54009ca0 comp rx=0 tx=0).stop 2024-03-04T09:30:11.664 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:11.664+0000 7f3b66da6700 1 -- 172.21.15.52:0/2078935169 shutdown_connections 2024-03-04T09:30:11.665 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:11.664+0000 7f3b66da6700 1 --2- 172.21.15.52:0/2078935169 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3b6006b120 0x7f3b6010ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:11.665 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:11.664+0000 7f3b66da6700 1 --2- 172.21.15.52:0/2078935169 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3b601081b0 0x7f3b60108590 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:11.665 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:11.664+0000 7f3b66da6700 1 --2- 172.21.15.52:0/2078935169 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3b6006a760 0x7f3b6006abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:11.665 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:11.664+0000 7f3b66da6700 1 -- 172.21.15.52:0/2078935169 >> 172.21.15.52:0/2078935169 conn(0x7f3b60077e80 msgr2=0x7f3b60078290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:30:11.665 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:11.664+0000 7f3b66da6700 1 -- 172.21.15.52:0/2078935169 shutdown_connections 2024-03-04T09:30:11.665 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:11.664+0000 7f3b66da6700 1 -- 172.21.15.52:0/2078935169 wait complete. 2024-03-04T09:30:11.665 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:11.665+0000 7f3b66da6700 1 Processor -- start 2024-03-04T09:30:11.665 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:11.665+0000 7f3b66da6700 1 -- start start 2024-03-04T09:30:11.665 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:11.665+0000 7f3b66da6700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3b6006a760 0x7f3b60101370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:30:11.665 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:11.665+0000 7f3b66da6700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3b6006b120 0x7f3b601018b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:30:11.665 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:11.665+0000 7f3b66da6700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3b601081b0 0x7f3b60101df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:30:11.665 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:11.665+0000 7f3b66da6700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b601123d0 con 0x7f3b601081b0 2024-03-04T09:30:11.665 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:11.665+0000 7f3b66da6700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b60112250 con 0x7f3b6006a760 2024-03-04T09:30:11.666 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:11.665+0000 7f3b66da6700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b60112550 con 0x7f3b6006b120 2024-03-04T09:30:11.666 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:11.665+0000 7f3b65343700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3b601081b0 0x7f3b60101df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:30:11.666 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:11.665+0000 7f3b65343700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3b601081b0 0x7f3b60101df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:47124/0 (socket says 172.21.15.52:47124) 2024-03-04T09:30:11.666 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:11.665+0000 7f3b64b42700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3b6006a760 0x7f3b60101370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:30:11.666 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:11.665+0000 7f3b65343700 1 -- 172.21.15.52:0/3519248180 learned_addr learned my addr 172.21.15.52:0/3519248180 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-04T09:30:11.666 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:11.665+0000 7f3b5ffff700 1 --2- 172.21.15.52:0/3519248180 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3b6006b120 0x7f3b601018b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:30:11.666 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:11.665+0000 7f3b65343700 1 -- 172.21.15.52:0/3519248180 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3b6006b120 msgr2=0x7f3b601018b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:30:11.666 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:11.665+0000 7f3b65343700 1 --2- 172.21.15.52:0/3519248180 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3b6006b120 0x7f3b601018b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:11.666 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:11.665+0000 7f3b65343700 1 -- 172.21.15.52:0/3519248180 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3b6006a760 msgr2=0x7f3b60101370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:30:11.666 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:11.665+0000 7f3b65343700 1 --2- 172.21.15.52:0/3519248180 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3b6006a760 0x7f3b60101370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:11.666 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:11.665+0000 7f3b65343700 1 -- 172.21.15.52:0/3519248180 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b601a6d50 con 0x7f3b601081b0 2024-03-04T09:30:11.666 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:11.666+0000 7f3b65343700 1 --2- 172.21.15.52:0/3519248180 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3b601081b0 0x7f3b60101df0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f3b5800efc0 tx=0x7f3b5800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:30:11.666 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:11.666+0000 7f3b5dffb700 1 -- 172.21.15.52:0/3519248180 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b580092e0 con 0x7f3b601081b0 2024-03-04T09:30:11.666 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:11.666+0000 7f3b5dffb700 1 -- 172.21.15.52:0/3519248180 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3b58004bb0 con 0x7f3b601081b0 2024-03-04T09:30:11.667 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:11.666+0000 7f3b66da6700 1 -- 172.21.15.52:0/3519248180 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b601a6fe0 con 0x7f3b601081b0 2024-03-04T09:30:11.667 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:11.666+0000 7f3b5dffb700 1 -- 172.21.15.52:0/3519248180 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b58004020 con 0x7f3b601081b0 2024-03-04T09:30:11.667 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:11.666+0000 7f3b66da6700 1 -- 172.21.15.52:0/3519248180 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b601a74a0 con 0x7f3b601081b0 2024-03-04T09:30:11.667 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:11.667+0000 7f3b66da6700 1 -- 172.21.15.52:0/3519248180 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3b60068d80 con 0x7f3b601081b0 2024-03-04T09:30:11.668 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:11.668+0000 7f3b5dffb700 1 -- 172.21.15.52:0/3519248180 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) v1 ==== 136756+0+0 (secure 0 0 0) 0x7f3b5801b030 con 0x7f3b601081b0 2024-03-04T09:30:11.668 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:11.669+0000 7f3b5dffb700 1 --2- 172.21.15.52:0/3519248180 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f3b500a1e10 0x7f3b500a42d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:30:11.668 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:11.669+0000 7f3b64b42700 1 --2- 172.21.15.52:0/3519248180 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f3b500a1e10 0x7f3b500a42d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:30:11.668 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:11.669+0000 7f3b5dffb700 1 -- 172.21.15.52:0/3519248180 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9092+0+0 (secure 0 0 0) 0x7f3b580cd9a0 con 0x7f3b601081b0 2024-03-04T09:30:11.668 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:11.669+0000 7f3b64b42700 1 --2- 172.21.15.52:0/3519248180 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f3b500a1e10 0x7f3b500a42d0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f3b5400b2e0 tx=0x7f3b54005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:30:11.670 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:11.673+0000 7f3b5dffb700 1 -- 172.21.15.52:0/3519248180 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f3b58022020 con 0x7f3b601081b0 2024-03-04T09:30:11.876 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:11.879+0000 7f3b66da6700 1 -- 172.21.15.52:0/3519248180 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) v1 -- 0x7f3b6002dc10 con 0x7f3b601081b0 2024-03-04T09:30:11.876 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:11.879+0000 7f3b5dffb700 1 -- 172.21.15.52:0/3519248180 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f3b58010040 con 0x7f3b601081b0 2024-03-04T09:30:11.877 INFO:teuthology.orchestra.run.smithi052.stdout:287762808874 2024-03-04T09:30:11.878 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:11.881+0000 7f3b66da6700 1 -- 172.21.15.52:0/3519248180 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f3b500a1e10 msgr2=0x7f3b500a42d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:30:11.879 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:11.881+0000 7f3b66da6700 1 --2- 172.21.15.52:0/3519248180 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f3b500a1e10 0x7f3b500a42d0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f3b5400b2e0 tx=0x7f3b54005960 comp rx=0 tx=0).stop 2024-03-04T09:30:11.879 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:11.881+0000 7f3b66da6700 1 -- 172.21.15.52:0/3519248180 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3b601081b0 msgr2=0x7f3b60101df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:30:11.879 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:11.881+0000 7f3b66da6700 1 --2- 172.21.15.52:0/3519248180 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3b601081b0 0x7f3b60101df0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f3b5800efc0 tx=0x7f3b5800c3a0 comp rx=0 tx=0).stop 2024-03-04T09:30:11.879 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:11.882+0000 7f3b66da6700 1 -- 172.21.15.52:0/3519248180 shutdown_connections 2024-03-04T09:30:11.879 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:11.882+0000 7f3b66da6700 1 --2- 172.21.15.52:0/3519248180 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f3b500a1e10 0x7f3b500a42d0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:11.879 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:11.882+0000 7f3b66da6700 1 --2- 172.21.15.52:0/3519248180 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3b601081b0 0x7f3b60101df0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:11.879 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:11.882+0000 7f3b66da6700 1 --2- 172.21.15.52:0/3519248180 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3b6006a760 0x7f3b60101370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:11.879 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:11.882+0000 7f3b66da6700 1 --2- 172.21.15.52:0/3519248180 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3b6006b120 0x7f3b601018b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:11.879 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:11.882+0000 7f3b66da6700 1 -- 172.21.15.52:0/3519248180 >> 172.21.15.52:0/3519248180 conn(0x7f3b60077e80 msgr2=0x7f3b600fffc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:30:11.879 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:11.882+0000 7f3b66da6700 1 -- 172.21.15.52:0/3519248180 shutdown_connections 2024-03-04T09:30:11.879 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:11.882+0000 7f3b66da6700 1 -- 172.21.15.52:0/3519248180 wait complete. 2024-03-04T09:30:12.446 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:30:12 smithi052 ceph-mon[32473]: mgrmap e18: x(active, since 8m), standbys: z, y 2024-03-04T09:30:12.447 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:30:12 smithi052 ceph-mon[32473]: from='client.? 172.21.15.52:0/3519248180' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-03-04T09:30:12.467 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:12.469+0000 7f866cdae700 1 -- 172.21.15.52:0/3163503066 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8668102110 msgr2=0x7f8668102590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:30:12.467 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:12.469+0000 7f866cdae700 1 --2- 172.21.15.52:0/3163503066 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8668102110 0x7f8668102590 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f865c0099c0 tx=0x7f865c009cd0 comp rx=0 tx=0).stop 2024-03-04T09:30:12.467 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:12.469+0000 7f866cdae700 1 -- 172.21.15.52:0/3163503066 shutdown_connections 2024-03-04T09:30:12.467 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:12.469+0000 7f866cdae700 1 --2- 172.21.15.52:0/3163503066 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f86681080f0 0x7f86681084d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:12.467 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:12.469+0000 7f866cdae700 1 --2- 172.21.15.52:0/3163503066 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8668102ad0 0x7f866810ef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:12.467 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:12.469+0000 7f866cdae700 1 --2- 172.21.15.52:0/3163503066 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8668102110 0x7f8668102590 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:12.468 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:12.469+0000 7f866cdae700 1 -- 172.21.15.52:0/3163503066 >> 172.21.15.52:0/3163503066 conn(0x7f86680791f0 msgr2=0x7f8668079600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:30:12.468 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:12.470+0000 7f866cdae700 1 -- 172.21.15.52:0/3163503066 shutdown_connections 2024-03-04T09:30:12.468 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:12.470+0000 7f866cdae700 1 -- 172.21.15.52:0/3163503066 wait complete. 2024-03-04T09:30:12.468 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:12.470+0000 7f866cdae700 1 Processor -- start 2024-03-04T09:30:12.468 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:12.470+0000 7f866cdae700 1 -- start start 2024-03-04T09:30:12.469 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:12.470+0000 7f866cdae700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8668102110 0x7f86680743c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:30:12.470 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:12.470+0000 7f866cdae700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8668102ad0 0x7f8668074900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:30:12.470 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:12.471+0000 7f866cdae700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f86681080f0 0x7f8668074e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:30:12.470 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:12.471+0000 7f866cdae700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8668112390 con 0x7f8668102110 2024-03-04T09:30:12.470 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:12.471+0000 7f866cdae700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8668112210 con 0x7f86681080f0 2024-03-04T09:30:12.470 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:12.471+0000 7f866cdae700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8668112510 con 0x7f8668102ad0 2024-03-04T09:30:12.470 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:12.471+0000 7f866659c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8668102110 0x7f86680743c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:30:12.470 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:12.471+0000 7f866659c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8668102110 0x7f86680743c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:47140/0 (socket says 172.21.15.52:47140) 2024-03-04T09:30:12.470 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:12.471+0000 7f866659c700 1 -- 172.21.15.52:0/2396217310 learned_addr learned my addr 172.21.15.52:0/2396217310 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-04T09:30:12.470 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:12.471+0000 7f8665d9b700 1 --2- 172.21.15.52:0/2396217310 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8668102ad0 0x7f8668074900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:30:12.470 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:12.471+0000 7f8666d9d700 1 --2- 172.21.15.52:0/2396217310 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f86681080f0 0x7f8668074e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:30:12.470 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:12.471+0000 7f866659c700 1 -- 172.21.15.52:0/2396217310 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8668102ad0 msgr2=0x7f8668074900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:30:12.470 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:12.471+0000 7f866659c700 1 --2- 172.21.15.52:0/2396217310 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8668102ad0 0x7f8668074900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:12.470 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:12.471+0000 7f866659c700 1 -- 172.21.15.52:0/2396217310 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f86681080f0 msgr2=0x7f8668074e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:30:12.471 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:12.471+0000 7f866659c700 1 --2- 172.21.15.52:0/2396217310 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f86681080f0 0x7f8668074e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:12.471 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:12.471+0000 7f866659c700 1 -- 172.21.15.52:0/2396217310 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86681a6d20 con 0x7f8668102110 2024-03-04T09:30:12.471 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:12.471+0000 7f8665d9b700 1 --2- 172.21.15.52:0/2396217310 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8668102ad0 0x7f8668074900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-04T09:30:12.471 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:12.471+0000 7f866659c700 1 --2- 172.21.15.52:0/2396217310 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8668102110 0x7f86680743c0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f865800e9c0 tx=0x7f865800ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:30:12.471 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:12.471+0000 7f86577fe700 1 -- 172.21.15.52:0/2396217310 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f865800cbd0 con 0x7f8668102110 2024-03-04T09:30:12.471 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:12.471+0000 7f86577fe700 1 -- 172.21.15.52:0/2396217310 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f865800ee60 con 0x7f8668102110 2024-03-04T09:30:12.471 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:12.471+0000 7f86577fe700 1 -- 172.21.15.52:0/2396217310 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8658005630 con 0x7f8668102110 2024-03-04T09:30:12.471 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:12.471+0000 7f866cdae700 1 -- 172.21.15.52:0/2396217310 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86681a6fb0 con 0x7f8668102110 2024-03-04T09:30:12.471 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:12.472+0000 7f866cdae700 1 -- 172.21.15.52:0/2396217310 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86681a7460 con 0x7f8668102110 2024-03-04T09:30:12.471 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:12.472+0000 7f866cdae700 1 -- 172.21.15.52:0/2396217310 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8668068d80 con 0x7f8668102110 2024-03-04T09:30:12.472 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:12.474+0000 7f86577fe700 1 -- 172.21.15.52:0/2396217310 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) v1 ==== 136756+0+0 (secure 0 0 0) 0x7f8658004750 con 0x7f8668102110 2024-03-04T09:30:12.472 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:12.475+0000 7f86577fe700 1 --2- 172.21.15.52:0/2396217310 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f86500a1e90 0x7f86500a4350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:30:12.472 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:12.475+0000 7f8665d9b700 1 --2- 172.21.15.52:0/2396217310 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f86500a1e90 0x7f86500a4350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:30:12.472 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:12.475+0000 7f86577fe700 1 -- 172.21.15.52:0/2396217310 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9092+0+0 (secure 0 0 0) 0x7f86580ce410 con 0x7f8668102110 2024-03-04T09:30:12.472 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:12.475+0000 7f8665d9b700 1 --2- 172.21.15.52:0/2396217310 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f86500a1e90 0x7f86500a4350 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f8668075e00 tx=0x7f865c005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:30:12.475 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:12.479+0000 7f86577fe700 1 -- 172.21.15.52:0/2396217310 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f8658053280 con 0x7f8668102110 2024-03-04T09:30:12.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:30:12 smithi191 ceph-mon[36816]: mgrmap e18: x(active, since 8m), standbys: z, y 2024-03-04T09:30:12.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:30:12 smithi191 ceph-mon[36816]: from='client.? 172.21.15.52:0/3519248180' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-03-04T09:30:12.631 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:30:12 smithi106 ceph-mon[36484]: mgrmap e18: x(active, since 8m), standbys: z, y 2024-03-04T09:30:12.631 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:30:12 smithi106 ceph-mon[36484]: from='client.? 172.21.15.52:0/3519248180' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-03-04T09:30:12.683 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:12.685+0000 7f866cdae700 1 -- 172.21.15.52:0/2396217310 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) v1 -- 0x7f866802dc10 con 0x7f8668102110 2024-03-04T09:30:12.683 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:12.686+0000 7f86577fe700 1 -- 172.21.15.52:0/2396217310 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) v1 ==== 75+0+13 (secure 0 0 0) 0x7f865809b4e0 con 0x7f8668102110 2024-03-04T09:30:12.683 INFO:teuthology.orchestra.run.smithi052.stdout:343597383712 2024-03-04T09:30:12.685 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:12.688+0000 7f866cdae700 1 -- 172.21.15.52:0/2396217310 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f86500a1e90 msgr2=0x7f86500a4350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:30:12.685 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:12.688+0000 7f866cdae700 1 --2- 172.21.15.52:0/2396217310 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f86500a1e90 0x7f86500a4350 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f8668075e00 tx=0x7f865c005fb0 comp rx=0 tx=0).stop 2024-03-04T09:30:12.685 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:12.688+0000 7f866cdae700 1 -- 172.21.15.52:0/2396217310 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8668102110 msgr2=0x7f86680743c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:30:12.685 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:12.688+0000 7f866cdae700 1 --2- 172.21.15.52:0/2396217310 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8668102110 0x7f86680743c0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f865800e9c0 tx=0x7f865800ecd0 comp rx=0 tx=0).stop 2024-03-04T09:30:12.685 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:12.688+0000 7f866cdae700 1 -- 172.21.15.52:0/2396217310 shutdown_connections 2024-03-04T09:30:12.686 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:12.688+0000 7f866cdae700 1 --2- 172.21.15.52:0/2396217310 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f86500a1e90 0x7f86500a4350 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:12.686 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:12.688+0000 7f866cdae700 1 --2- 172.21.15.52:0/2396217310 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f86681080f0 0x7f8668074e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:12.686 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:12.688+0000 7f866cdae700 1 --2- 172.21.15.52:0/2396217310 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8668102110 0x7f86680743c0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:12.686 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:12.688+0000 7f866cdae700 1 --2- 172.21.15.52:0/2396217310 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8668102ad0 0x7f8668074900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:12.686 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:12.688+0000 7f866cdae700 1 -- 172.21.15.52:0/2396217310 >> 172.21.15.52:0/2396217310 conn(0x7f86680791f0 msgr2=0x7f8668100d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:30:12.686 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:12.688+0000 7f866cdae700 1 -- 172.21.15.52:0/2396217310 shutdown_connections 2024-03-04T09:30:12.686 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:12.688+0000 7f866cdae700 1 -- 172.21.15.52:0/2396217310 wait complete. 2024-03-04T09:30:12.793 INFO:tasks.cephadm.ceph_manager.ceph:need seq 287762808868 got 287762808874 for osd.9 2024-03-04T09:30:12.793 DEBUG:teuthology.parallel:result is None 2024-03-04T09:30:13.335 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:30:13 smithi052 ceph-mon[32473]: pgmap v337: 129 pgs: 129 active+clean; 641 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 426 B/s wr, 8 op/s 2024-03-04T09:30:13.335 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:30:13 smithi052 ceph-mon[32473]: from='client.? 172.21.15.52:0/2396217310' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-03-04T09:30:13.400 INFO:tasks.cephadm.ceph_manager.ceph:need seq 343597383706 got 343597383712 for osd.11 2024-03-04T09:30:13.400 DEBUG:teuthology.parallel:result is None 2024-03-04T09:30:13.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:30:13 smithi191 ceph-mon[36816]: pgmap v337: 129 pgs: 129 active+clean; 641 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 426 B/s wr, 8 op/s 2024-03-04T09:30:13.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:30:13 smithi191 ceph-mon[36816]: from='client.? 172.21.15.52:0/2396217310' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-03-04T09:30:13.631 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:30:13 smithi106 ceph-mon[36484]: pgmap v337: 129 pgs: 129 active+clean; 641 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 426 B/s wr, 8 op/s 2024-03-04T09:30:13.631 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:30:13 smithi106 ceph-mon[36484]: from='client.? 172.21.15.52:0/2396217310' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-03-04T09:30:13.873 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738447 got 34359738455 for osd.0 2024-03-04T09:30:13.873 DEBUG:teuthology.parallel:result is None 2024-03-04T09:30:14.973 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:14.973+0000 7f6d3bde7700 1 -- 172.21.15.52:0/4267638095 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6d34101c40 msgr2=0x7f6d34105c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:30:14.974 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:14.973+0000 7f6d3bde7700 1 --2- 172.21.15.52:0/4267638095 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6d34101c40 0x7f6d34105c90 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f6d240099c0 tx=0x7f6d24009cd0 comp rx=0 tx=0).stop 2024-03-04T09:30:14.974 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:14.973+0000 7f6d3bde7700 1 -- 172.21.15.52:0/4267638095 shutdown_connections 2024-03-04T09:30:14.974 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:14.973+0000 7f6d3bde7700 1 --2- 172.21.15.52:0/4267638095 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6d34106370 0x7f6d3410df40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:14.974 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:14.973+0000 7f6d3bde7700 1 --2- 172.21.15.52:0/4267638095 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6d34101290 0x7f6d34101670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:14.974 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:14.973+0000 7f6d3bde7700 1 --2- 172.21.15.52:0/4267638095 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6d34101c40 0x7f6d34105c90 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:14.974 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:14.973+0000 7f6d3bde7700 1 -- 172.21.15.52:0/4267638095 >> 172.21.15.52:0/4267638095 conn(0x7f6d340780f0 msgr2=0x7f6d34078500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:30:14.974 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:14.974+0000 7f6d3bde7700 1 -- 172.21.15.52:0/4267638095 shutdown_connections 2024-03-04T09:30:14.975 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:14.974+0000 7f6d3bde7700 1 -- 172.21.15.52:0/4267638095 wait complete. 2024-03-04T09:30:14.975 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:14.974+0000 7f6d3bde7700 1 Processor -- start 2024-03-04T09:30:14.975 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:14.974+0000 7f6d3bde7700 1 -- start start 2024-03-04T09:30:14.975 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:14.974+0000 7f6d3bde7700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6d34101290 0x7f6d3419e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:30:14.975 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:14.974+0000 7f6d3bde7700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6d34101c40 0x7f6d3419ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:30:14.975 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:14.974+0000 7f6d3bde7700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6d34106370 0x7f6d34198770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:30:14.975 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:14.974+0000 7f6d3bde7700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d34111330 con 0x7f6d34106370 2024-03-04T09:30:14.975 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:14.974+0000 7f6d3bde7700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d341111b0 con 0x7f6d34101290 2024-03-04T09:30:14.975 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:14.974+0000 7f6d3bde7700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d341114b0 con 0x7f6d34101c40 2024-03-04T09:30:14.976 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:14.975+0000 7f6d3a384700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6d34106370 0x7f6d34198770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:30:14.976 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:14.975+0000 7f6d3a384700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6d34106370 0x7f6d34198770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:47662/0 (socket says 172.21.15.52:47662) 2024-03-04T09:30:14.980 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:14.975+0000 7f6d39b83700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6d34101290 0x7f6d3419e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:30:14.981 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:14.975+0000 7f6d3a384700 1 -- 172.21.15.52:0/3687248653 learned_addr learned my addr 172.21.15.52:0/3687248653 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-04T09:30:14.981 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:14.975+0000 7f6d39382700 1 --2- 172.21.15.52:0/3687248653 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6d34101c40 0x7f6d3419ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:30:14.981 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:14.975+0000 7f6d3a384700 1 -- 172.21.15.52:0/3687248653 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6d34101c40 msgr2=0x7f6d3419ebe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:30:14.981 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:14.975+0000 7f6d3a384700 1 --2- 172.21.15.52:0/3687248653 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6d34101c40 0x7f6d3419ebe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:14.981 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:14.975+0000 7f6d3a384700 1 -- 172.21.15.52:0/3687248653 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6d34101290 msgr2=0x7f6d3419e6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:30:14.981 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:14.975+0000 7f6d3a384700 1 --2- 172.21.15.52:0/3687248653 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6d34101290 0x7f6d3419e6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:14.981 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:14.975+0000 7f6d3a384700 1 -- 172.21.15.52:0/3687248653 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d34199050 con 0x7f6d34106370 2024-03-04T09:30:14.981 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:14.975+0000 7f6d39b83700 1 --2- 172.21.15.52:0/3687248653 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6d34101290 0x7f6d3419e6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-04T09:30:14.981 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:14.975+0000 7f6d39382700 1 --2- 172.21.15.52:0/3687248653 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6d34101c40 0x7f6d3419ebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-04T09:30:14.981 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:14.975+0000 7f6d3a384700 1 --2- 172.21.15.52:0/3687248653 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6d34106370 0x7f6d34198770 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f6d2c00ef30 tx=0x7f6d2c00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:30:14.981 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:14.975+0000 7f6d2affd700 1 -- 172.21.15.52:0/3687248653 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d2c0092e0 con 0x7f6d34106370 2024-03-04T09:30:14.981 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:14.975+0000 7f6d3bde7700 1 -- 172.21.15.52:0/3687248653 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d341992e0 con 0x7f6d34106370 2024-03-04T09:30:14.981 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:14.975+0000 7f6d2affd700 1 -- 172.21.15.52:0/3687248653 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6d2c004bb0 con 0x7f6d34106370 2024-03-04T09:30:14.982 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:14.975+0000 7f6d2affd700 1 -- 172.21.15.52:0/3687248653 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d2c004280 con 0x7f6d34106370 2024-03-04T09:30:14.982 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:14.975+0000 7f6d3bde7700 1 -- 172.21.15.52:0/3687248653 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d341a5620 con 0x7f6d34106370 2024-03-04T09:30:14.982 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:14.976+0000 7f6d3bde7700 1 -- 172.21.15.52:0/3687248653 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6d34102770 con 0x7f6d34106370 2024-03-04T09:30:14.983 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:14.978+0000 7f6d2affd700 1 -- 172.21.15.52:0/3687248653 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) v1 ==== 136756+0+0 (secure 0 0 0) 0x7f6d2c01b030 con 0x7f6d34106370 2024-03-04T09:30:14.983 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:14.978+0000 7f6d2affd700 1 --2- 172.21.15.52:0/3687248653 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f6d200a1e10 0x7f6d200a42d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:30:14.983 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:14.978+0000 7f6d39b83700 1 --2- 172.21.15.52:0/3687248653 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f6d200a1e10 0x7f6d200a42d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:30:14.983 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:14.979+0000 7f6d2affd700 1 -- 172.21.15.52:0/3687248653 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9092+0+0 (secure 0 0 0) 0x7f6d2c0cdbc0 con 0x7f6d34106370 2024-03-04T09:30:14.984 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:14.979+0000 7f6d39b83700 1 --2- 172.21.15.52:0/3687248653 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f6d200a1e10 0x7f6d200a42d0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f6d30011490 tx=0x7f6d30009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:30:14.984 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:14.986+0000 7f6d2affd700 1 -- 172.21.15.52:0/3687248653 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f6d2c022020 con 0x7f6d34106370 2024-03-04T09:30:15.196 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:15.198+0000 7f6d3bde7700 1 -- 172.21.15.52:0/3687248653 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7f6d34199620 con 0x7f6d34106370 2024-03-04T09:30:15.196 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:15.198+0000 7f6d2affd700 1 -- 172.21.15.52:0/3687248653 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f6d2c010040 con 0x7f6d34106370 2024-03-04T09:30:15.196 INFO:teuthology.orchestra.run.smithi052.stdout:219043332148 2024-03-04T09:30:15.198 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:15.200+0000 7f6d3bde7700 1 -- 172.21.15.52:0/3687248653 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f6d200a1e10 msgr2=0x7f6d200a42d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:30:15.198 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:15.200+0000 7f6d3bde7700 1 --2- 172.21.15.52:0/3687248653 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f6d200a1e10 0x7f6d200a42d0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f6d30011490 tx=0x7f6d30009260 comp rx=0 tx=0).stop 2024-03-04T09:30:15.198 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:15.200+0000 7f6d3bde7700 1 -- 172.21.15.52:0/3687248653 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6d34106370 msgr2=0x7f6d34198770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:30:15.198 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:15.200+0000 7f6d3bde7700 1 --2- 172.21.15.52:0/3687248653 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6d34106370 0x7f6d34198770 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f6d2c00ef30 tx=0x7f6d2c00c3a0 comp rx=0 tx=0).stop 2024-03-04T09:30:15.198 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:15.200+0000 7f6d3bde7700 1 -- 172.21.15.52:0/3687248653 shutdown_connections 2024-03-04T09:30:15.198 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:15.200+0000 7f6d3bde7700 1 --2- 172.21.15.52:0/3687248653 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f6d200a1e10 0x7f6d200a42d0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:15.198 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:15.200+0000 7f6d3bde7700 1 --2- 172.21.15.52:0/3687248653 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6d34106370 0x7f6d34198770 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:15.198 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:15.200+0000 7f6d3bde7700 1 --2- 172.21.15.52:0/3687248653 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6d34101290 0x7f6d3419e6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:15.198 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:15.200+0000 7f6d3bde7700 1 --2- 172.21.15.52:0/3687248653 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6d34101c40 0x7f6d3419ebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:15.199 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:15.200+0000 7f6d3bde7700 1 -- 172.21.15.52:0/3687248653 >> 172.21.15.52:0/3687248653 conn(0x7f6d340780f0 msgr2=0x7f6d3406ab00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:30:15.199 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:15.201+0000 7f6d3bde7700 1 -- 172.21.15.52:0/3687248653 shutdown_connections 2024-03-04T09:30:15.199 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:15.201+0000 7f6d3bde7700 1 -- 172.21.15.52:0/3687248653 wait complete. 2024-03-04T09:30:15.335 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:30:15 smithi052 ceph-mon[32473]: pgmap v338: 129 pgs: 129 active+clean; 641 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 426 B/s wr, 5 op/s 2024-03-04T09:30:15.335 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:30:15 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:30:15.335 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:30:15 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:30:15.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:30:15 smithi191 ceph-mon[36816]: pgmap v338: 129 pgs: 129 active+clean; 641 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 426 B/s wr, 5 op/s 2024-03-04T09:30:15.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:30:15 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:30:15.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:30:15 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:30:15.631 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:30:15 smithi106 ceph-mon[36484]: pgmap v338: 129 pgs: 129 active+clean; 641 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 426 B/s wr, 5 op/s 2024-03-04T09:30:15.631 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:30:15 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:30:15.631 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:30:15 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:30:16.037 INFO:tasks.cephadm.ceph_manager.ceph:need seq 219043332143 got 219043332148 for osd.7 2024-03-04T09:30:16.038 DEBUG:teuthology.parallel:result is None 2024-03-04T09:30:16.038 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-03-04T09:30:16.038 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd shell --fsid 4e059f08-da08-11ee-95c6-87774f69a715 -- ceph pg dump --format=json 2024-03-04T09:30:16.301 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/4e059f08-da08-11ee-95c6-87774f69a715/mon.a/config 2024-03-04T09:30:16.334 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:30:16 smithi052 ceph-mon[32473]: from='client.? 172.21.15.52:0/3687248653' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-03-04T09:30:16.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:30:16 smithi191 ceph-mon[36816]: from='client.? 172.21.15.52:0/3687248653' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-03-04T09:30:16.631 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:30:16 smithi106 ceph-mon[36484]: from='client.? 172.21.15.52:0/3687248653' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-03-04T09:30:17.451 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:30:17 smithi052 ceph-mon[32473]: pgmap v339: 129 pgs: 129 active+clean; 641 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 426 B/s wr, 6 op/s 2024-03-04T09:30:17.451 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:30:17 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T09:30:17.451 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:30:17 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T09:30:17.451 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:30:17 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:30:17.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:30:17 smithi191 ceph-mon[36816]: pgmap v339: 129 pgs: 129 active+clean; 641 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 426 B/s wr, 6 op/s 2024-03-04T09:30:17.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:30:17 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T09:30:17.587 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:30:17 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T09:30:17.587 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:30:17 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:30:17.631 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:30:17 smithi106 ceph-mon[36484]: pgmap v339: 129 pgs: 129 active+clean; 641 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 426 B/s wr, 6 op/s 2024-03-04T09:30:17.631 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:30:17 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T09:30:17.631 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:30:17 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T09:30:17.631 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:30:17 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' 2024-03-04T09:30:18.160 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:18.142+0000 7ff1f3b91700 1 -- 172.21.15.52:0/2207166742 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff1ec102300 msgr2=0x7ff1ec1026e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:30:18.160 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:18.142+0000 7ff1f3b91700 1 --2- 172.21.15.52:0/2207166742 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff1ec102300 0x7ff1ec1026e0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7ff1dc0099c0 tx=0x7ff1dc009cd0 comp rx=0 tx=0).stop 2024-03-04T09:30:18.160 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:18.142+0000 7ff1f3b91700 1 -- 172.21.15.52:0/2207166742 shutdown_connections 2024-03-04T09:30:18.160 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:18.142+0000 7ff1f3b91700 1 --2- 172.21.15.52:0/2207166742 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff1ec1073e0 0x7ff1ec10efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:18.160 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:18.142+0000 7ff1f3b91700 1 --2- 172.21.15.52:0/2207166742 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff1ec102300 0x7ff1ec1026e0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:18.160 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:18.142+0000 7ff1f3b91700 1 --2- 172.21.15.52:0/2207166742 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff1ec102cb0 0x7ff1ec106d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:18.161 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:18.142+0000 7ff1f3b91700 1 -- 172.21.15.52:0/2207166742 >> 172.21.15.52:0/2207166742 conn(0x7ff1ec0fdb70 msgr2=0x7ff1ec0fff90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:30:18.161 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:18.142+0000 7ff1f3b91700 1 -- 172.21.15.52:0/2207166742 shutdown_connections 2024-03-04T09:30:18.161 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:18.142+0000 7ff1f3b91700 1 -- 172.21.15.52:0/2207166742 wait complete. 2024-03-04T09:30:18.161 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:18.143+0000 7ff1f3b91700 1 Processor -- start 2024-03-04T09:30:18.161 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:18.143+0000 7ff1f3b91700 1 -- start start 2024-03-04T09:30:18.161 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:18.143+0000 7ff1f3b91700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff1ec102300 0x7ff1ec19e610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:30:18.161 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:18.143+0000 7ff1f3b91700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff1ec102cb0 0x7ff1ec19eb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:30:18.161 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:18.143+0000 7ff1f3b91700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff1ec1073e0 0x7ff1ec1986e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:30:18.161 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:18.143+0000 7ff1f3b91700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1ec112440 con 0x7ff1ec102cb0 2024-03-04T09:30:18.161 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:18.143+0000 7ff1f3b91700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1ec1122c0 con 0x7ff1ec102300 2024-03-04T09:30:18.161 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:18.143+0000 7ff1f3b91700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1ec1125c0 con 0x7ff1ec1073e0 2024-03-04T09:30:18.161 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:18.143+0000 7ff1f112c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff1ec102cb0 0x7ff1ec19eb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:30:18.161 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:18.143+0000 7ff1f192d700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff1ec102300 0x7ff1ec19e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:30:18.162 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:18.143+0000 7ff1f112c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff1ec102cb0 0x7ff1ec19eb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:47668/0 (socket says 172.21.15.52:47668) 2024-03-04T09:30:18.162 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:18.143+0000 7ff1f112c700 1 -- 172.21.15.52:0/1441001379 learned_addr learned my addr 172.21.15.52:0/1441001379 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-04T09:30:18.162 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:18.143+0000 7ff1f212e700 1 --2- 172.21.15.52:0/1441001379 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff1ec1073e0 0x7ff1ec1986e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:30:18.162 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:18.143+0000 7ff1f112c700 1 -- 172.21.15.52:0/1441001379 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff1ec1073e0 msgr2=0x7ff1ec1986e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:30:18.162 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:18.143+0000 7ff1f112c700 1 --2- 172.21.15.52:0/1441001379 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff1ec1073e0 0x7ff1ec1986e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:18.162 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:18.144+0000 7ff1f112c700 1 -- 172.21.15.52:0/1441001379 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff1ec102300 msgr2=0x7ff1ec19e610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:30:18.162 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:18.144+0000 7ff1f112c700 1 --2- 172.21.15.52:0/1441001379 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff1ec102300 0x7ff1ec19e610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:18.162 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:18.144+0000 7ff1f112c700 1 -- 172.21.15.52:0/1441001379 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1ec198fc0 con 0x7ff1ec102cb0 2024-03-04T09:30:18.162 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:18.144+0000 7ff1f192d700 1 --2- 172.21.15.52:0/1441001379 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff1ec102300 0x7ff1ec19e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-04T09:30:18.162 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:18.144+0000 7ff1f112c700 1 --2- 172.21.15.52:0/1441001379 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff1ec102cb0 0x7ff1ec19eb50 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7ff1e800e970 tx=0x7ff1e800ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:30:18.162 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:18.144+0000 7ff1e2ffd700 1 -- 172.21.15.52:0/1441001379 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1e8004a30 con 0x7ff1ec102cb0 2024-03-04T09:30:18.162 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:18.144+0000 7ff1e2ffd700 1 -- 172.21.15.52:0/1441001379 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff1e800ccc0 con 0x7ff1ec102cb0 2024-03-04T09:30:18.162 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:18.144+0000 7ff1f3b91700 1 -- 172.21.15.52:0/1441001379 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1ec1992b0 con 0x7ff1ec102cb0 2024-03-04T09:30:18.162 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:18.144+0000 7ff1e2ffd700 1 -- 172.21.15.52:0/1441001379 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1e8005650 con 0x7ff1ec102cb0 2024-03-04T09:30:18.163 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:18.144+0000 7ff1f3b91700 1 -- 172.21.15.52:0/1441001379 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1ec1a5590 con 0x7ff1ec102cb0 2024-03-04T09:30:18.163 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:18.145+0000 7ff1f3b91700 1 -- 172.21.15.52:0/1441001379 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff1ec1037e0 con 0x7ff1ec102cb0 2024-03-04T09:30:18.163 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:18.146+0000 7ff1e2ffd700 1 -- 172.21.15.52:0/1441001379 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) v1 ==== 136756+0+0 (secure 0 0 0) 0x7ff1e8004bd0 con 0x7ff1ec102cb0 2024-03-04T09:30:18.163 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:18.147+0000 7ff1e2ffd700 1 --2- 172.21.15.52:0/1441001379 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7ff1d80a1e10 0x7ff1d80a42d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:30:18.163 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:18.147+0000 7ff1f192d700 1 --2- 172.21.15.52:0/1441001379 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7ff1d80a1e10 0x7ff1d80a42d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:30:18.163 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:18.147+0000 7ff1e2ffd700 1 -- 172.21.15.52:0/1441001379 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9092+0+0 (secure 0 0 0) 0x7ff1e80ce500 con 0x7ff1ec102cb0 2024-03-04T09:30:18.163 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:18.147+0000 7ff1f192d700 1 --2- 172.21.15.52:0/1441001379 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7ff1d80a1e10 0x7ff1d80a42d0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7ff1dc000c00 tx=0x7ff1dc005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:30:18.163 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:18.150+0000 7ff1e2ffd700 1 -- 172.21.15.52:0/1441001379 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7ff1e8053280 con 0x7ff1ec102cb0 2024-03-04T09:30:18.348 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:18.351+0000 7ff1f3b91700 1 -- 172.21.15.52:0/1441001379 --> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7ff1ec062500 con 0x7ff1d80a1e10 2024-03-04T09:30:18.357 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:18.361+0000 7ff1e2ffd700 1 -- 172.21.15.52:0/1441001379 <== mgr.14152 v2:172.21.15.52:6800/3036033923 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+340499 (secure 0 0 0) 0x7ff1ec062500 con 0x7ff1d80a1e10 2024-03-04T09:30:18.363 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-03-04T09:30:18.366 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:18.364+0000 7ff1f3b91700 1 -- 172.21.15.52:0/1441001379 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7ff1d80a1e10 msgr2=0x7ff1d80a42d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:30:18.366 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:18.364+0000 7ff1f3b91700 1 --2- 172.21.15.52:0/1441001379 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7ff1d80a1e10 0x7ff1d80a42d0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7ff1dc000c00 tx=0x7ff1dc005fb0 comp rx=0 tx=0).stop 2024-03-04T09:30:18.366 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:18.364+0000 7ff1f3b91700 1 -- 172.21.15.52:0/1441001379 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff1ec102cb0 msgr2=0x7ff1ec19eb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:30:18.366 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:18.364+0000 7ff1f3b91700 1 --2- 172.21.15.52:0/1441001379 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff1ec102cb0 0x7ff1ec19eb50 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7ff1e800e970 tx=0x7ff1e800ec80 comp rx=0 tx=0).stop 2024-03-04T09:30:18.366 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:18.364+0000 7ff1f3b91700 1 -- 172.21.15.52:0/1441001379 shutdown_connections 2024-03-04T09:30:18.367 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:18.364+0000 7ff1f3b91700 1 --2- 172.21.15.52:0/1441001379 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7ff1d80a1e10 0x7ff1d80a42d0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:18.367 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:18.364+0000 7ff1f3b91700 1 --2- 172.21.15.52:0/1441001379 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff1ec1073e0 0x7ff1ec1986e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:18.367 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:18.364+0000 7ff1f3b91700 1 --2- 172.21.15.52:0/1441001379 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff1ec102300 0x7ff1ec19e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:18.367 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:18.364+0000 7ff1f3b91700 1 --2- 172.21.15.52:0/1441001379 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff1ec102cb0 0x7ff1ec19eb50 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:18.367 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:18.364+0000 7ff1f3b91700 1 -- 172.21.15.52:0/1441001379 >> 172.21.15.52:0/1441001379 conn(0x7ff1ec0fdb70 msgr2=0x7ff1ec0ffe50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:30:18.367 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:18.364+0000 7ff1f3b91700 1 -- 172.21.15.52:0/1441001379 shutdown_connections 2024-03-04T09:30:18.367 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:18.364+0000 7ff1f3b91700 1 -- 172.21.15.52:0/1441001379 wait complete. 2024-03-04T09:30:18.367 INFO:teuthology.orchestra.run.smithi052.stderr:dumped all 2024-03-04T09:30:19.044 INFO:teuthology.orchestra.run.smithi052.stdout:{"pg_ready":true,"pg_map":{"version":340,"stamp":"2024-03-04T09:30:17.510926+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":656768,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":598,"num_read_kb":420,"num_write":349,"num_write_kb":3058,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":783,"ondisk_log_size":783,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":371556,"kb_used_data":14084,"kb_used_omap":18,"kb_used_meta":357357,"kb_avail":1124570268,"statfs":{"total":1151940427776,"available":1151559954432,"internally_reserved":0,"allocated":14422016,"data_stored":7956206,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19140,"internal_metadata":365933884},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":390,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":77,"num_read_kb":37,"num_write":7,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004334"},"pg_stats":[{"pgid":"3.3f","version":"91'8","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T09:30:09.527473+0000","last_change":"2024-03-04T09:28:37.914945+0000","last_active":"2024-03-04T09:30:09.527473+0000","last_peered":"2024-03-04T09:30:09.527473+0000","last_clean":"2024-03-04T09:30:09.527473+0000","last_became_active":"2024-03-04T09:28:37.914647+0000","last_became_peered":"2024-03-04T09:28:37.914647+0000","last_unstale":"2024-03-04T09:30:09.527473+0000","last_undegraded":"2024-03-04T09:30:09.527473+0000","last_fullsized":"2024-03-04T09:30:09.527473+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_clean_scrub_stamp":"2024-03-04T09:28:36.887656+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T13:46:41.262758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"91'4","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T09:30:09.520902+0000","last_change":"2024-03-04T09:28:37.918650+0000","last_active":"2024-03-04T09:30:09.520902+0000","last_peered":"2024-03-04T09:30:09.520902+0000","last_clean":"2024-03-04T09:30:09.520902+0000","last_became_active":"2024-03-04T09:28:37.918314+0000","last_became_peered":"2024-03-04T09:28:37.918314+0000","last_unstale":"2024-03-04T09:30:09.520902+0000","last_undegraded":"2024-03-04T09:30:09.520902+0000","last_fullsized":"2024-03-04T09:30:09.520902+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_clean_scrub_stamp":"2024-03-04T09:28:36.887656+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T18:36:41.948549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"91'7","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T09:30:09.527792+0000","last_change":"2024-03-04T09:28:37.920140+0000","last_active":"2024-03-04T09:30:09.527792+0000","last_peered":"2024-03-04T09:30:09.527792+0000","last_clean":"2024-03-04T09:30:09.527792+0000","last_became_active":"2024-03-04T09:28:37.917379+0000","last_became_peered":"2024-03-04T09:28:37.917379+0000","last_unstale":"2024-03-04T09:30:09.527792+0000","last_undegraded":"2024-03-04T09:30:09.527792+0000","last_fullsized":"2024-03-04T09:30:09.527792+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_clean_scrub_stamp":"2024-03-04T09:28:36.887656+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T17:23:33.144752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"91'8","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T09:30:09.511151+0000","last_change":"2024-03-04T09:28:37.916999+0000","last_active":"2024-03-04T09:30:09.511151+0000","last_peered":"2024-03-04T09:30:09.511151+0000","last_clean":"2024-03-04T09:30:09.511151+0000","last_became_active":"2024-03-04T09:28:37.916597+0000","last_became_peered":"2024-03-04T09:28:37.916597+0000","last_unstale":"2024-03-04T09:30:09.511151+0000","last_undegraded":"2024-03-04T09:30:09.511151+0000","last_fullsized":"2024-03-04T09:30:09.511151+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_clean_scrub_stamp":"2024-03-04T09:28:36.887656+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T19:23:20.556283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"91'4","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T09:30:09.505887+0000","last_change":"2024-03-04T09:28:37.904298+0000","last_active":"2024-03-04T09:30:09.505887+0000","last_peered":"2024-03-04T09:30:09.505887+0000","last_clean":"2024-03-04T09:30:09.505887+0000","last_became_active":"2024-03-04T09:28:37.903966+0000","last_became_peered":"2024-03-04T09:28:37.903966+0000","last_unstale":"2024-03-04T09:30:09.505887+0000","last_undegraded":"2024-03-04T09:30:09.505887+0000","last_fullsized":"2024-03-04T09:30:09.505887+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_clean_scrub_stamp":"2024-03-04T09:28:36.887656+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T20:14:21.615521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"91'6","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T09:30:09.526067+0000","last_change":"2024-03-04T09:28:37.917800+0000","last_active":"2024-03-04T09:30:09.526067+0000","last_peered":"2024-03-04T09:30:09.526067+0000","last_clean":"2024-03-04T09:30:09.526067+0000","last_became_active":"2024-03-04T09:28:37.917500+0000","last_became_peered":"2024-03-04T09:28:37.917500+0000","last_unstale":"2024-03-04T09:30:09.526067+0000","last_undegraded":"2024-03-04T09:30:09.526067+0000","last_fullsized":"2024-03-04T09:30:09.526067+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_clean_scrub_stamp":"2024-03-04T09:28:36.887656+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T20:06:31.330705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"91'5","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T09:30:09.502432+0000","last_change":"2024-03-04T09:28:37.917708+0000","last_active":"2024-03-04T09:30:09.502432+0000","last_peered":"2024-03-04T09:30:09.502432+0000","last_clean":"2024-03-04T09:30:09.502432+0000","last_became_active":"2024-03-04T09:28:37.917323+0000","last_became_peered":"2024-03-04T09:28:37.917323+0000","last_unstale":"2024-03-04T09:30:09.502432+0000","last_undegraded":"2024-03-04T09:30:09.502432+0000","last_fullsized":"2024-03-04T09:30:09.502432+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_clean_scrub_stamp":"2024-03-04T09:28:36.887656+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T11:10:46.345702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"91'9","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T09:30:09.524295+0000","last_change":"2024-03-04T09:29:41.411289+0000","last_active":"2024-03-04T09:30:09.524295+0000","last_peered":"2024-03-04T09:30:09.524295+0000","last_clean":"2024-03-04T09:30:09.524295+0000","last_became_active":"2024-03-04T09:29:41.410905+0000","last_became_peered":"2024-03-04T09:29:41.410905+0000","last_unstale":"2024-03-04T09:30:09.524295+0000","last_undegraded":"2024-03-04T09:30:09.524295+0000","last_fullsized":"2024-03-04T09:30:09.524295+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_clean_scrub_stamp":"2024-03-04T09:28:36.887656+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T17:55:14.687909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,10],"acting":[9,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.37","version":"91'11","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T09:30:09.529366+0000","last_change":"2024-03-04T09:28:37.928279+0000","last_active":"2024-03-04T09:30:09.529366+0000","last_peered":"2024-03-04T09:30:09.529366+0000","last_clean":"2024-03-04T09:30:09.529366+0000","last_became_active":"2024-03-04T09:28:37.927934+0000","last_became_peered":"2024-03-04T09:28:37.927934+0000","last_unstale":"2024-03-04T09:30:09.529366+0000","last_undegraded":"2024-03-04T09:30:09.529366+0000","last_fullsized":"2024-03-04T09:30:09.529366+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_clean_scrub_stamp":"2024-03-04T09:28:36.887656+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T21:23:06.592526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"91'16","reported_seq":41,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T09:30:09.525051+0000","last_change":"2024-03-04T09:28:37.918990+0000","last_active":"2024-03-04T09:30:09.525051+0000","last_peered":"2024-03-04T09:30:09.525051+0000","last_clean":"2024-03-04T09:30:09.525051+0000","last_became_active":"2024-03-04T09:28:37.918599+0000","last_became_peered":"2024-03-04T09:28:37.918599+0000","last_unstale":"2024-03-04T09:30:09.525051+0000","last_undegraded":"2024-03-04T09:30:09.525051+0000","last_fullsized":"2024-03-04T09:30:09.525051+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_clean_scrub_stamp":"2024-03-04T09:28:36.887656+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T20:39:09.603428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"91'6","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T09:30:09.517158+0000","last_change":"2024-03-04T09:28:37.914234+0000","last_active":"2024-03-04T09:30:09.517158+0000","last_peered":"2024-03-04T09:30:09.517158+0000","last_clean":"2024-03-04T09:30:09.517158+0000","last_became_active":"2024-03-04T09:28:37.913897+0000","last_became_peered":"2024-03-04T09:28:37.913897+0000","last_unstale":"2024-03-04T09:30:09.517158+0000","last_undegraded":"2024-03-04T09:30:09.517158+0000","last_fullsized":"2024-03-04T09:30:09.517158+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_clean_scrub_stamp":"2024-03-04T09:28:36.887656+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T15:43:53.634817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"91'7","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T09:30:09.509698+0000","last_change":"2024-03-04T09:28:37.919935+0000","last_active":"2024-03-04T09:30:09.509698+0000","last_peered":"2024-03-04T09:30:09.509698+0000","last_clean":"2024-03-04T09:30:09.509698+0000","last_became_active":"2024-03-04T09:28:37.919447+0000","last_became_peered":"2024-03-04T09:28:37.919447+0000","last_unstale":"2024-03-04T09:30:09.509698+0000","last_undegraded":"2024-03-04T09:30:09.509698+0000","last_fullsized":"2024-03-04T09:30:09.509698+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_clean_scrub_stamp":"2024-03-04T09:28:36.887656+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T09:44:03.678767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"91'10","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T09:30:09.522391+0000","last_change":"2024-03-04T09:28:37.900570+0000","last_active":"2024-03-04T09:30:09.522391+0000","last_peered":"2024-03-04T09:30:09.522391+0000","last_clean":"2024-03-04T09:30:09.522391+0000","last_became_active":"2024-03-04T09:28:37.900186+0000","last_became_peered":"2024-03-04T09:28:37.900186+0000","last_unstale":"2024-03-04T09:30:09.522391+0000","last_undegraded":"2024-03-04T09:30:09.522391+0000","last_fullsized":"2024-03-04T09:30:09.522391+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_clean_scrub_stamp":"2024-03-04T09:28:36.887656+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T21:17:12.915989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"91'7","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T09:30:09.525278+0000","last_change":"2024-03-04T09:28:37.920447+0000","last_active":"2024-03-04T09:30:09.525278+0000","last_peered":"2024-03-04T09:30:09.525278+0000","last_clean":"2024-03-04T09:30:09.525278+0000","last_became_active":"2024-03-04T09:28:37.919580+0000","last_became_peered":"2024-03-04T09:28:37.919580+0000","last_unstale":"2024-03-04T09:30:09.525278+0000","last_undegraded":"2024-03-04T09:30:09.525278+0000","last_fullsized":"2024-03-04T09:30:09.525278+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_clean_scrub_stamp":"2024-03-04T09:28:36.887656+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T13:25:56.434285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"91'7","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T09:30:09.522193+0000","last_change":"2024-03-04T09:28:37.913735+0000","last_active":"2024-03-04T09:30:09.522193+0000","last_peered":"2024-03-04T09:30:09.522193+0000","last_clean":"2024-03-04T09:30:09.522193+0000","last_became_active":"2024-03-04T09:28:37.913388+0000","last_became_peered":"2024-03-04T09:28:37.913388+0000","last_unstale":"2024-03-04T09:30:09.522193+0000","last_undegraded":"2024-03-04T09:30:09.522193+0000","last_fullsized":"2024-03-04T09:30:09.522193+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_clean_scrub_stamp":"2024-03-04T09:28:36.887656+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T16:04:28.468313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"91'9","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T09:30:09.528522+0000","last_change":"2024-03-04T09:28:37.925031+0000","last_active":"2024-03-04T09:30:09.528522+0000","last_peered":"2024-03-04T09:30:09.528522+0000","last_clean":"2024-03-04T09:30:09.528522+0000","last_became_active":"2024-03-04T09:28:37.923561+0000","last_became_peered":"2024-03-04T09:28:37.923561+0000","last_unstale":"2024-03-04T09:30:09.528522+0000","last_undegraded":"2024-03-04T09:30:09.528522+0000","last_fullsized":"2024-03-04T09:30:09.528522+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_clean_scrub_stamp":"2024-03-04T09:28:36.887656+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T16:29:04.266384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"91'8","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T09:30:09.527522+0000","last_change":"2024-03-04T09:28:37.912029+0000","last_active":"2024-03-04T09:30:09.527522+0000","last_peered":"2024-03-04T09:30:09.527522+0000","last_clean":"2024-03-04T09:30:09.527522+0000","last_became_active":"2024-03-04T09:28:37.911726+0000","last_became_peered":"2024-03-04T09:28:37.911726+0000","last_unstale":"2024-03-04T09:30:09.527522+0000","last_undegraded":"2024-03-04T09:30:09.527522+0000","last_fullsized":"2024-03-04T09:30:09.527522+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_clean_scrub_stamp":"2024-03-04T09:28:36.887656+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T12:32:03.385375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"91'10","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T09:30:09.526459+0000","last_change":"2024-03-04T09:28:37.914448+0000","last_active":"2024-03-04T09:30:09.526459+0000","last_peered":"2024-03-04T09:30:09.526459+0000","last_clean":"2024-03-04T09:30:09.526459+0000","last_became_active":"2024-03-04T09:28:37.914036+0000","last_became_peered":"2024-03-04T09:28:37.914036+0000","last_unstale":"2024-03-04T09:30:09.526459+0000","last_undegraded":"2024-03-04T09:30:09.526459+0000","last_fullsized":"2024-03-04T09:30:09.526459+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_clean_scrub_stamp":"2024-03-04T09:28:36.887656+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T20:57:10.348039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"91'7","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T09:30:09.517930+0000","last_change":"2024-03-04T09:28:37.907640+0000","last_active":"2024-03-04T09:30:09.517930+0000","last_peered":"2024-03-04T09:30:09.517930+0000","last_clean":"2024-03-04T09:30:09.517930+0000","last_became_active":"2024-03-04T09:28:37.907350+0000","last_became_peered":"2024-03-04T09:28:37.907350+0000","last_unstale":"2024-03-04T09:30:09.517930+0000","last_undegraded":"2024-03-04T09:30:09.517930+0000","last_fullsized":"2024-03-04T09:30:09.517930+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_clean_scrub_stamp":"2024-03-04T09:28:36.887656+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T10:17:37.576254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T09:29:41.061931+0000","last_change":"2024-03-04T09:28:41.950312+0000","last_active":"2024-03-04T09:29:41.061931+0000","last_peered":"2024-03-04T09:29:41.061931+0000","last_clean":"2024-03-04T09:29:41.061931+0000","last_became_active":"2024-03-04T09:28:41.949882+0000","last_became_peered":"2024-03-04T09:28:41.949882+0000","last_unstale":"2024-03-04T09:29:41.061931+0000","last_undegraded":"2024-03-04T09:29:41.061931+0000","last_fullsized":"2024-03-04T09:29:41.061931+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:40.917115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:40.917115+0000","last_clean_scrub_stamp":"2024-03-04T09:28:40.917115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T14:20:48.740518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"89'9","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T09:29:41.062915+0000","last_change":"2024-03-04T09:28:36.904805+0000","last_active":"2024-03-04T09:29:41.062915+0000","last_peered":"2024-03-04T09:29:41.062915+0000","last_clean":"2024-03-04T09:29:41.062915+0000","last_became_active":"2024-03-04T09:28:36.904469+0000","last_became_peered":"2024-03-04T09:28:36.904469+0000","last_unstale":"2024-03-04T09:29:41.062915+0000","last_undegraded":"2024-03-04T09:29:41.062915+0000","last_fullsized":"2024-03-04T09:29:41.062915+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:35.885776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:35.885776+0000","last_clean_scrub_stamp":"2024-03-04T09:28:35.885776+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T18:35:35.992042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"91'5","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T09:30:09.513701+0000","last_change":"2024-03-04T09:28:37.920102+0000","last_active":"2024-03-04T09:30:09.513701+0000","last_peered":"2024-03-04T09:30:09.513701+0000","last_clean":"2024-03-04T09:30:09.513701+0000","last_became_active":"2024-03-04T09:28:37.919749+0000","last_became_peered":"2024-03-04T09:28:37.919749+0000","last_unstale":"2024-03-04T09:30:09.513701+0000","last_undegraded":"2024-03-04T09:30:09.513701+0000","last_fullsized":"2024-03-04T09:30:09.513701+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_clean_scrub_stamp":"2024-03-04T09:28:36.887656+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T19:16:36.269782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T09:29:41.061792+0000","last_change":"2024-03-04T09:28:41.950477+0000","last_active":"2024-03-04T09:29:41.061792+0000","last_peered":"2024-03-04T09:29:41.061792+0000","last_clean":"2024-03-04T09:29:41.061792+0000","last_became_active":"2024-03-04T09:28:41.950087+0000","last_became_peered":"2024-03-04T09:28:41.950087+0000","last_unstale":"2024-03-04T09:29:41.061792+0000","last_undegraded":"2024-03-04T09:29:41.061792+0000","last_fullsized":"2024-03-04T09:29:41.061792+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:40.917115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:40.917115+0000","last_clean_scrub_stamp":"2024-03-04T09:28:40.917115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T14:48:17.327956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"89'2","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T09:29:41.071382+0000","last_change":"2024-03-04T09:28:36.913923+0000","last_active":"2024-03-04T09:29:41.071382+0000","last_peered":"2024-03-04T09:29:41.071382+0000","last_clean":"2024-03-04T09:29:41.071382+0000","last_became_active":"2024-03-04T09:28:36.911076+0000","last_became_peered":"2024-03-04T09:28:36.911076+0000","last_unstale":"2024-03-04T09:29:41.071382+0000","last_undegraded":"2024-03-04T09:29:41.071382+0000","last_fullsized":"2024-03-04T09:29:41.071382+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:35.885776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:35.885776+0000","last_clean_scrub_stamp":"2024-03-04T09:28:35.885776+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T16:13:23.004827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"91'6","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T09:30:09.527767+0000","last_change":"2024-03-04T09:28:37.912747+0000","last_active":"2024-03-04T09:30:09.527767+0000","last_peered":"2024-03-04T09:30:09.527767+0000","last_clean":"2024-03-04T09:30:09.527767+0000","last_became_active":"2024-03-04T09:28:37.912436+0000","last_became_peered":"2024-03-04T09:28:37.912436+0000","last_unstale":"2024-03-04T09:30:09.527767+0000","last_undegraded":"2024-03-04T09:30:09.527767+0000","last_fullsized":"2024-03-04T09:30:09.527767+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_clean_scrub_stamp":"2024-03-04T09:28:36.887656+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T14:52:47.318456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T09:29:41.411324+0000","last_change":"2024-03-04T09:29:41.411324+0000","last_active":"2024-03-04T09:29:41.411324+0000","last_peered":"2024-03-04T09:29:41.411324+0000","last_clean":"2024-03-04T09:29:41.411324+0000","last_became_active":"2024-03-04T09:29:41.410822+0000","last_became_peered":"2024-03-04T09:29:41.410822+0000","last_unstale":"2024-03-04T09:29:41.411324+0000","last_undegraded":"2024-03-04T09:29:41.411324+0000","last_fullsized":"2024-03-04T09:29:41.411324+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:40.917115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:40.917115+0000","last_clean_scrub_stamp":"2024-03-04T09:28:40.917115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T14:07:20.646108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,10,5],"acting":[4,3,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"89'4","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T09:29:41.066136+0000","last_change":"2024-03-04T09:28:36.910279+0000","last_active":"2024-03-04T09:29:41.066136+0000","last_peered":"2024-03-04T09:29:41.066136+0000","last_clean":"2024-03-04T09:29:41.066136+0000","last_became_active":"2024-03-04T09:28:36.909214+0000","last_became_peered":"2024-03-04T09:28:36.909214+0000","last_unstale":"2024-03-04T09:29:41.066136+0000","last_undegraded":"2024-03-04T09:29:41.066136+0000","last_fullsized":"2024-03-04T09:29:41.066136+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:35.885776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:35.885776+0000","last_clean_scrub_stamp":"2024-03-04T09:28:35.885776+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T19:22:45.808418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"91'9","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T09:30:09.528763+0000","last_change":"2024-03-04T09:28:37.914593+0000","last_active":"2024-03-04T09:30:09.528763+0000","last_peered":"2024-03-04T09:30:09.528763+0000","last_clean":"2024-03-04T09:30:09.528763+0000","last_became_active":"2024-03-04T09:28:37.914217+0000","last_became_peered":"2024-03-04T09:28:37.914217+0000","last_unstale":"2024-03-04T09:30:09.528763+0000","last_undegraded":"2024-03-04T09:30:09.528763+0000","last_fullsized":"2024-03-04T09:30:09.528763+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_clean_scrub_stamp":"2024-03-04T09:28:36.887656+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T19:32:39.893906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T09:29:41.061839+0000","last_change":"2024-03-04T09:28:41.951685+0000","last_active":"2024-03-04T09:29:41.061839+0000","last_peered":"2024-03-04T09:29:41.061839+0000","last_clean":"2024-03-04T09:29:41.061839+0000","last_became_active":"2024-03-04T09:28:41.951178+0000","last_became_peered":"2024-03-04T09:28:41.951178+0000","last_unstale":"2024-03-04T09:29:41.061839+0000","last_undegraded":"2024-03-04T09:29:41.061839+0000","last_fullsized":"2024-03-04T09:29:41.061839+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:40.917115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:40.917115+0000","last_clean_scrub_stamp":"2024-03-04T09:28:40.917115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T10:23:19.028344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"84'2","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T09:29:41.065632+0000","last_change":"2024-03-04T09:28:36.908066+0000","last_active":"2024-03-04T09:29:41.065632+0000","last_peered":"2024-03-04T09:29:41.065632+0000","last_clean":"2024-03-04T09:29:41.065632+0000","last_became_active":"2024-03-04T09:28:36.907761+0000","last_became_peered":"2024-03-04T09:28:36.907761+0000","last_unstale":"2024-03-04T09:29:41.065632+0000","last_undegraded":"2024-03-04T09:29:41.065632+0000","last_fullsized":"2024-03-04T09:29:41.065632+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:35.885776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:35.885776+0000","last_clean_scrub_stamp":"2024-03-04T09:28:35.885776+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T20:53:11.559719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"91'15","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T09:30:09.513186+0000","last_change":"2024-03-04T09:28:37.917667+0000","last_active":"2024-03-04T09:30:09.513186+0000","last_peered":"2024-03-04T09:30:09.513186+0000","last_clean":"2024-03-04T09:30:09.513186+0000","last_became_active":"2024-03-04T09:28:37.917320+0000","last_became_peered":"2024-03-04T09:28:37.917320+0000","last_unstale":"2024-03-04T09:30:09.513186+0000","last_undegraded":"2024-03-04T09:30:09.513186+0000","last_fullsized":"2024-03-04T09:30:09.513186+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_clean_scrub_stamp":"2024-03-04T09:28:36.887656+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T12:17:07.162334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T09:29:41.062750+0000","last_change":"2024-03-04T09:28:41.946078+0000","last_active":"2024-03-04T09:29:41.062750+0000","last_peered":"2024-03-04T09:29:41.062750+0000","last_clean":"2024-03-04T09:29:41.062750+0000","last_became_active":"2024-03-04T09:28:41.945639+0000","last_became_peered":"2024-03-04T09:28:41.945639+0000","last_unstale":"2024-03-04T09:29:41.062750+0000","last_undegraded":"2024-03-04T09:29:41.062750+0000","last_fullsized":"2024-03-04T09:29:41.062750+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:40.917115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:40.917115+0000","last_clean_scrub_stamp":"2024-03-04T09:28:40.917115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T16:31:41.939728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"89'5","reported_seq":107,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T09:30:15.085510+0000","last_change":"2024-03-04T09:28:36.905596+0000","last_active":"2024-03-04T09:30:15.085510+0000","last_peered":"2024-03-04T09:30:15.085510+0000","last_clean":"2024-03-04T09:30:15.085510+0000","last_became_active":"2024-03-04T09:28:36.905270+0000","last_became_peered":"2024-03-04T09:28:36.905270+0000","last_unstale":"2024-03-04T09:30:15.085510+0000","last_undegraded":"2024-03-04T09:30:15.085510+0000","last_fullsized":"2024-03-04T09:30:15.085510+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:35.885776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:35.885776+0000","last_clean_scrub_stamp":"2024-03-04T09:28:35.885776+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T09:43:32.986704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":75,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"91'7","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T09:30:09.519426+0000","last_change":"2024-03-04T09:28:37.906282+0000","last_active":"2024-03-04T09:30:09.519426+0000","last_peered":"2024-03-04T09:30:09.519426+0000","last_clean":"2024-03-04T09:30:09.519426+0000","last_became_active":"2024-03-04T09:28:37.905964+0000","last_became_peered":"2024-03-04T09:28:37.905964+0000","last_unstale":"2024-03-04T09:30:09.519426+0000","last_undegraded":"2024-03-04T09:30:09.519426+0000","last_fullsized":"2024-03-04T09:30:09.519426+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_clean_scrub_stamp":"2024-03-04T09:28:36.887656+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T11:13:16.287959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T09:29:41.062153+0000","last_change":"2024-03-04T09:28:41.949196+0000","last_active":"2024-03-04T09:29:41.062153+0000","last_peered":"2024-03-04T09:29:41.062153+0000","last_clean":"2024-03-04T09:29:41.062153+0000","last_became_active":"2024-03-04T09:28:41.948783+0000","last_became_peered":"2024-03-04T09:28:41.948783+0000","last_unstale":"2024-03-04T09:29:41.062153+0000","last_undegraded":"2024-03-04T09:29:41.062153+0000","last_fullsized":"2024-03-04T09:29:41.062153+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:40.917115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:40.917115+0000","last_clean_scrub_stamp":"2024-03-04T09:28:40.917115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T13:42:05.888482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"89'3","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T09:29:41.168040+0000","last_change":"2024-03-04T09:28:36.906900+0000","last_active":"2024-03-04T09:29:41.168040+0000","last_peered":"2024-03-04T09:29:41.168040+0000","last_clean":"2024-03-04T09:29:41.168040+0000","last_became_active":"2024-03-04T09:28:36.906556+0000","last_became_peered":"2024-03-04T09:28:36.906556+0000","last_unstale":"2024-03-04T09:29:41.168040+0000","last_undegraded":"2024-03-04T09:29:41.168040+0000","last_fullsized":"2024-03-04T09:29:41.168040+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:35.885776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:35.885776+0000","last_clean_scrub_stamp":"2024-03-04T09:28:35.885776+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T18:13:56.535621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"91'8","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T09:30:09.511567+0000","last_change":"2024-03-04T09:28:37.912123+0000","last_active":"2024-03-04T09:30:09.511567+0000","last_peered":"2024-03-04T09:30:09.511567+0000","last_clean":"2024-03-04T09:30:09.511567+0000","last_became_active":"2024-03-04T09:28:37.911810+0000","last_became_peered":"2024-03-04T09:28:37.911810+0000","last_unstale":"2024-03-04T09:30:09.511567+0000","last_undegraded":"2024-03-04T09:30:09.511567+0000","last_fullsized":"2024-03-04T09:30:09.511567+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_clean_scrub_stamp":"2024-03-04T09:28:36.887656+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T18:14:25.229989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T09:29:41.072635+0000","last_change":"2024-03-04T09:29:41.072635+0000","last_active":"2024-03-04T09:29:41.072635+0000","last_peered":"2024-03-04T09:29:41.072635+0000","last_clean":"2024-03-04T09:29:41.072635+0000","last_became_active":"2024-03-04T09:29:41.072213+0000","last_became_peered":"2024-03-04T09:29:41.072213+0000","last_unstale":"2024-03-04T09:29:41.072635+0000","last_undegraded":"2024-03-04T09:29:41.072635+0000","last_fullsized":"2024-03-04T09:29:41.072635+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:40.917115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:40.917115+0000","last_clean_scrub_stamp":"2024-03-04T09:28:40.917115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T13:09:43.707806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8,0],"acting":[11,4,8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"89'2","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T09:29:41.068264+0000","last_change":"2024-03-04T09:28:36.902206+0000","last_active":"2024-03-04T09:29:41.068264+0000","last_peered":"2024-03-04T09:29:41.068264+0000","last_clean":"2024-03-04T09:29:41.068264+0000","last_became_active":"2024-03-04T09:28:36.901847+0000","last_became_peered":"2024-03-04T09:28:36.901847+0000","last_unstale":"2024-03-04T09:29:41.068264+0000","last_undegraded":"2024-03-04T09:29:41.068264+0000","last_fullsized":"2024-03-04T09:29:41.068264+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:35.885776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:35.885776+0000","last_clean_scrub_stamp":"2024-03-04T09:28:35.885776+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T12:32:05.755876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"91'8","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T09:30:09.517891+0000","last_change":"2024-03-04T09:28:37.906175+0000","last_active":"2024-03-04T09:30:09.517891+0000","last_peered":"2024-03-04T09:30:09.517891+0000","last_clean":"2024-03-04T09:30:09.517891+0000","last_became_active":"2024-03-04T09:28:37.905845+0000","last_became_peered":"2024-03-04T09:28:37.905845+0000","last_unstale":"2024-03-04T09:30:09.517891+0000","last_undegraded":"2024-03-04T09:30:09.517891+0000","last_fullsized":"2024-03-04T09:30:09.517891+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_clean_scrub_stamp":"2024-03-04T09:28:36.887656+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T11:50:23.569240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T09:29:41.065675+0000","last_change":"2024-03-04T09:28:41.949290+0000","last_active":"2024-03-04T09:29:41.065675+0000","last_peered":"2024-03-04T09:29:41.065675+0000","last_clean":"2024-03-04T09:29:41.065675+0000","last_became_active":"2024-03-04T09:28:41.948866+0000","last_became_peered":"2024-03-04T09:28:41.948866+0000","last_unstale":"2024-03-04T09:29:41.065675+0000","last_undegraded":"2024-03-04T09:29:41.065675+0000","last_fullsized":"2024-03-04T09:29:41.065675+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:40.917115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:40.917115+0000","last_clean_scrub_stamp":"2024-03-04T09:28:40.917115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T15:30:11.980139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"89'1","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T09:29:41.410078+0000","last_change":"2024-03-04T09:28:36.910071+0000","last_active":"2024-03-04T09:29:41.410078+0000","last_peered":"2024-03-04T09:29:41.410078+0000","last_clean":"2024-03-04T09:29:41.410078+0000","last_became_active":"2024-03-04T09:28:36.908025+0000","last_became_peered":"2024-03-04T09:28:36.908025+0000","last_unstale":"2024-03-04T09:29:41.410078+0000","last_undegraded":"2024-03-04T09:29:41.410078+0000","last_fullsized":"2024-03-04T09:29:41.410078+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:35.885776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:35.885776+0000","last_clean_scrub_stamp":"2024-03-04T09:28:35.885776+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T10:45:42.310473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"91'11","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T09:30:09.526063+0000","last_change":"2024-03-04T09:28:37.915792+0000","last_active":"2024-03-04T09:30:09.526063+0000","last_peered":"2024-03-04T09:30:09.526063+0000","last_clean":"2024-03-04T09:30:09.526063+0000","last_became_active":"2024-03-04T09:28:37.914914+0000","last_became_peered":"2024-03-04T09:28:37.914914+0000","last_unstale":"2024-03-04T09:30:09.526063+0000","last_undegraded":"2024-03-04T09:30:09.526063+0000","last_fullsized":"2024-03-04T09:30:09.526063+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_clean_scrub_stamp":"2024-03-04T09:28:36.887656+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T14:13:57.828616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T09:29:41.070690+0000","last_change":"2024-03-04T09:28:41.947909+0000","last_active":"2024-03-04T09:29:41.070690+0000","last_peered":"2024-03-04T09:29:41.070690+0000","last_clean":"2024-03-04T09:29:41.070690+0000","last_became_active":"2024-03-04T09:28:41.947553+0000","last_became_peered":"2024-03-04T09:28:41.947553+0000","last_unstale":"2024-03-04T09:29:41.070690+0000","last_undegraded":"2024-03-04T09:29:41.070690+0000","last_fullsized":"2024-03-04T09:29:41.070690+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:40.917115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:40.917115+0000","last_clean_scrub_stamp":"2024-03-04T09:28:40.917115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T14:19:52.302321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"89'3","reported_seq":102,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T09:30:14.083953+0000","last_change":"2024-03-04T09:28:36.911823+0000","last_active":"2024-03-04T09:30:14.083953+0000","last_peered":"2024-03-04T09:30:14.083953+0000","last_clean":"2024-03-04T09:30:14.083953+0000","last_became_active":"2024-03-04T09:28:36.911502+0000","last_became_peered":"2024-03-04T09:28:36.911502+0000","last_unstale":"2024-03-04T09:30:14.083953+0000","last_undegraded":"2024-03-04T09:30:14.083953+0000","last_fullsized":"2024-03-04T09:30:14.083953+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:35.885776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:35.885776+0000","last_clean_scrub_stamp":"2024-03-04T09:28:35.885776+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T15:31:04.806604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":72,"num_read_kb":72,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"91'10","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T09:30:09.528752+0000","last_change":"2024-03-04T09:28:37.914093+0000","last_active":"2024-03-04T09:30:09.528752+0000","last_peered":"2024-03-04T09:30:09.528752+0000","last_clean":"2024-03-04T09:30:09.528752+0000","last_became_active":"2024-03-04T09:28:37.913661+0000","last_became_peered":"2024-03-04T09:28:37.913661+0000","last_unstale":"2024-03-04T09:30:09.528752+0000","last_undegraded":"2024-03-04T09:30:09.528752+0000","last_fullsized":"2024-03-04T09:30:09.528752+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_clean_scrub_stamp":"2024-03-04T09:28:36.887656+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T18:17:42.826342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T09:29:41.061387+0000","last_change":"2024-03-04T09:28:41.947371+0000","last_active":"2024-03-04T09:29:41.061387+0000","last_peered":"2024-03-04T09:29:41.061387+0000","last_clean":"2024-03-04T09:29:41.061387+0000","last_became_active":"2024-03-04T09:28:41.946944+0000","last_became_peered":"2024-03-04T09:28:41.946944+0000","last_unstale":"2024-03-04T09:29:41.061387+0000","last_undegraded":"2024-03-04T09:29:41.061387+0000","last_fullsized":"2024-03-04T09:29:41.061387+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:40.917115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:40.917115+0000","last_clean_scrub_stamp":"2024-03-04T09:28:40.917115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T11:01:34.348927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"89'2","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T09:29:41.064563+0000","last_change":"2024-03-04T09:28:36.907927+0000","last_active":"2024-03-04T09:29:41.064563+0000","last_peered":"2024-03-04T09:29:41.064563+0000","last_clean":"2024-03-04T09:29:41.064563+0000","last_became_active":"2024-03-04T09:28:36.907507+0000","last_became_peered":"2024-03-04T09:28:36.907507+0000","last_unstale":"2024-03-04T09:29:41.064563+0000","last_undegraded":"2024-03-04T09:29:41.064563+0000","last_fullsized":"2024-03-04T09:29:41.064563+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:35.885776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:35.885776+0000","last_clean_scrub_stamp":"2024-03-04T09:28:35.885776+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T21:11:24.942564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"91'9","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T09:30:09.522913+0000","last_change":"2024-03-04T09:28:37.913816+0000","last_active":"2024-03-04T09:30:09.522913+0000","last_peered":"2024-03-04T09:30:09.522913+0000","last_clean":"2024-03-04T09:30:09.522913+0000","last_became_active":"2024-03-04T09:28:37.913481+0000","last_became_peered":"2024-03-04T09:28:37.913481+0000","last_unstale":"2024-03-04T09:30:09.522913+0000","last_undegraded":"2024-03-04T09:30:09.522913+0000","last_fullsized":"2024-03-04T09:30:09.522913+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_clean_scrub_stamp":"2024-03-04T09:28:36.887656+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T14:33:07.371224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T09:29:41.062269+0000","last_change":"2024-03-04T09:28:41.948963+0000","last_active":"2024-03-04T09:29:41.062269+0000","last_peered":"2024-03-04T09:29:41.062269+0000","last_clean":"2024-03-04T09:29:41.062269+0000","last_became_active":"2024-03-04T09:28:41.948469+0000","last_became_peered":"2024-03-04T09:28:41.948469+0000","last_unstale":"2024-03-04T09:29:41.062269+0000","last_undegraded":"2024-03-04T09:29:41.062269+0000","last_fullsized":"2024-03-04T09:29:41.062269+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:40.917115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:40.917115+0000","last_clean_scrub_stamp":"2024-03-04T09:28:40.917115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T21:26:52.238364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"89'4","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T09:29:41.409973+0000","last_change":"2024-03-04T09:28:36.913162+0000","last_active":"2024-03-04T09:29:41.409973+0000","last_peered":"2024-03-04T09:29:41.409973+0000","last_clean":"2024-03-04T09:29:41.409973+0000","last_became_active":"2024-03-04T09:28:36.912834+0000","last_became_peered":"2024-03-04T09:28:36.912834+0000","last_unstale":"2024-03-04T09:29:41.409973+0000","last_undegraded":"2024-03-04T09:29:41.409973+0000","last_fullsized":"2024-03-04T09:29:41.409973+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:35.885776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:35.885776+0000","last_clean_scrub_stamp":"2024-03-04T09:28:35.885776+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T17:59:07.210048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"91'11","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T09:30:09.528998+0000","last_change":"2024-03-04T09:28:37.914801+0000","last_active":"2024-03-04T09:30:09.528998+0000","last_peered":"2024-03-04T09:30:09.528998+0000","last_clean":"2024-03-04T09:30:09.528998+0000","last_became_active":"2024-03-04T09:28:37.914517+0000","last_became_peered":"2024-03-04T09:28:37.914517+0000","last_unstale":"2024-03-04T09:30:09.528998+0000","last_undegraded":"2024-03-04T09:30:09.528998+0000","last_fullsized":"2024-03-04T09:30:09.528998+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_clean_scrub_stamp":"2024-03-04T09:28:36.887656+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T18:15:50.967364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T09:29:41.061526+0000","last_change":"2024-03-04T09:28:41.957821+0000","last_active":"2024-03-04T09:29:41.061526+0000","last_peered":"2024-03-04T09:29:41.061526+0000","last_clean":"2024-03-04T09:29:41.061526+0000","last_became_active":"2024-03-04T09:28:41.957392+0000","last_became_peered":"2024-03-04T09:28:41.957392+0000","last_unstale":"2024-03-04T09:29:41.061526+0000","last_undegraded":"2024-03-04T09:29:41.061526+0000","last_fullsized":"2024-03-04T09:29:41.061526+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:40.917115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:40.917115+0000","last_clean_scrub_stamp":"2024-03-04T09:28:40.917115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T19:44:24.294299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"91'34","reported_seq":213,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T09:30:15.084974+0000","last_change":"2024-03-04T09:28:36.909243+0000","last_active":"2024-03-04T09:30:15.084974+0000","last_peered":"2024-03-04T09:30:15.084974+0000","last_clean":"2024-03-04T09:30:15.084974+0000","last_became_active":"2024-03-04T09:28:36.908941+0000","last_became_peered":"2024-03-04T09:28:36.908941+0000","last_unstale":"2024-03-04T09:30:15.084974+0000","last_undegraded":"2024-03-04T09:30:15.084974+0000","last_fullsized":"2024-03-04T09:30:15.084974+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:35.885776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:35.885776+0000","last_clean_scrub_stamp":"2024-03-04T09:28:35.885776+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T16:46:26.014209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":62024,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":152,"num_read_kb":137,"num_write":35,"num_write_kb":77,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"91'6","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T09:30:09.510134+0000","last_change":"2024-03-04T09:28:37.910465+0000","last_active":"2024-03-04T09:30:09.510134+0000","last_peered":"2024-03-04T09:30:09.510134+0000","last_clean":"2024-03-04T09:30:09.510134+0000","last_became_active":"2024-03-04T09:28:37.910129+0000","last_became_peered":"2024-03-04T09:28:37.910129+0000","last_unstale":"2024-03-04T09:30:09.510134+0000","last_undegraded":"2024-03-04T09:30:09.510134+0000","last_fullsized":"2024-03-04T09:30:09.510134+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_clean_scrub_stamp":"2024-03-04T09:28:36.887656+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T21:25:48.272477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T09:29:41.410494+0000","last_change":"2024-03-04T09:28:41.948465+0000","last_active":"2024-03-04T09:29:41.410494+0000","last_peered":"2024-03-04T09:29:41.410494+0000","last_clean":"2024-03-04T09:29:41.410494+0000","last_became_active":"2024-03-04T09:28:41.946995+0000","last_became_peered":"2024-03-04T09:28:41.946995+0000","last_unstale":"2024-03-04T09:29:41.410494+0000","last_undegraded":"2024-03-04T09:29:41.410494+0000","last_fullsized":"2024-03-04T09:29:41.410494+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:40.917115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:40.917115+0000","last_clean_scrub_stamp":"2024-03-04T09:28:40.917115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T12:35:10.579279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"89'6","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T09:29:41.062009+0000","last_change":"2024-03-04T09:28:36.905641+0000","last_active":"2024-03-04T09:29:41.062009+0000","last_peered":"2024-03-04T09:29:41.062009+0000","last_clean":"2024-03-04T09:29:41.062009+0000","last_became_active":"2024-03-04T09:28:36.905309+0000","last_became_peered":"2024-03-04T09:28:36.905309+0000","last_unstale":"2024-03-04T09:29:41.062009+0000","last_undegraded":"2024-03-04T09:29:41.062009+0000","last_fullsized":"2024-03-04T09:29:41.062009+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:35.885776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:35.885776+0000","last_clean_scrub_stamp":"2024-03-04T09:28:35.885776+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T12:08:40.200654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"91'9","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T09:30:09.519509+0000","last_change":"2024-03-04T09:28:37.907744+0000","last_active":"2024-03-04T09:30:09.519509+0000","last_peered":"2024-03-04T09:30:09.519509+0000","last_clean":"2024-03-04T09:30:09.519509+0000","last_became_active":"2024-03-04T09:28:37.907393+0000","last_became_peered":"2024-03-04T09:28:37.907393+0000","last_unstale":"2024-03-04T09:30:09.519509+0000","last_undegraded":"2024-03-04T09:30:09.519509+0000","last_fullsized":"2024-03-04T09:30:09.519509+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_clean_scrub_stamp":"2024-03-04T09:28:36.887656+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T20:17:11.822846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T09:29:41.063487+0000","last_change":"2024-03-04T09:28:41.949824+0000","last_active":"2024-03-04T09:29:41.063487+0000","last_peered":"2024-03-04T09:29:41.063487+0000","last_clean":"2024-03-04T09:29:41.063487+0000","last_became_active":"2024-03-04T09:28:41.949252+0000","last_became_peered":"2024-03-04T09:28:41.949252+0000","last_unstale":"2024-03-04T09:29:41.063487+0000","last_undegraded":"2024-03-04T09:29:41.063487+0000","last_fullsized":"2024-03-04T09:29:41.063487+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:40.917115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:40.917115+0000","last_clean_scrub_stamp":"2024-03-04T09:28:40.917115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T21:22:19.045780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"89'3","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T09:29:41.409875+0000","last_change":"2024-03-04T09:28:36.908718+0000","last_active":"2024-03-04T09:29:41.409875+0000","last_peered":"2024-03-04T09:29:41.409875+0000","last_clean":"2024-03-04T09:29:41.409875+0000","last_became_active":"2024-03-04T09:28:36.908102+0000","last_became_peered":"2024-03-04T09:28:36.908102+0000","last_unstale":"2024-03-04T09:29:41.409875+0000","last_undegraded":"2024-03-04T09:29:41.409875+0000","last_fullsized":"2024-03-04T09:29:41.409875+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:35.885776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:35.885776+0000","last_clean_scrub_stamp":"2024-03-04T09:28:35.885776+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T20:53:39.603281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"91'15","reported_seq":46,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T09:30:09.518318+0000","last_change":"2024-03-04T09:29:41.068617+0000","last_active":"2024-03-04T09:30:09.518318+0000","last_peered":"2024-03-04T09:30:09.518318+0000","last_clean":"2024-03-04T09:30:09.518318+0000","last_became_active":"2024-03-04T09:29:41.068231+0000","last_became_peered":"2024-03-04T09:29:41.068231+0000","last_unstale":"2024-03-04T09:30:09.518318+0000","last_undegraded":"2024-03-04T09:30:09.518318+0000","last_fullsized":"2024-03-04T09:30:09.518318+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_clean_scrub_stamp":"2024-03-04T09:28:36.887656+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T12:40:15.456293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T09:29:41.072598+0000","last_change":"2024-03-04T09:29:41.072598+0000","last_active":"2024-03-04T09:29:41.072598+0000","last_peered":"2024-03-04T09:29:41.072598+0000","last_clean":"2024-03-04T09:29:41.072598+0000","last_became_active":"2024-03-04T09:29:41.072095+0000","last_became_peered":"2024-03-04T09:29:41.072095+0000","last_unstale":"2024-03-04T09:29:41.072598+0000","last_undegraded":"2024-03-04T09:29:41.072598+0000","last_fullsized":"2024-03-04T09:29:41.072598+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:40.917115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:40.917115+0000","last_clean_scrub_stamp":"2024-03-04T09:28:40.917115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T14:24:55.509528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,7,0,3],"acting":[8,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.11","version":"91'5","reported_seq":109,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T09:30:15.094268+0000","last_change":"2024-03-04T09:28:36.905097+0000","last_active":"2024-03-04T09:30:15.094268+0000","last_peered":"2024-03-04T09:30:15.094268+0000","last_clean":"2024-03-04T09:30:15.094268+0000","last_became_active":"2024-03-04T09:28:36.904709+0000","last_became_peered":"2024-03-04T09:28:36.904709+0000","last_unstale":"2024-03-04T09:30:15.094268+0000","last_undegraded":"2024-03-04T09:30:15.094268+0000","last_fullsized":"2024-03-04T09:30:15.094268+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:35.885776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:35.885776+0000","last_clean_scrub_stamp":"2024-03-04T09:28:35.885776+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T13:58:23.977258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1120,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":78,"num_read_kb":5,"num_write":4,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"91'6","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T09:30:09.505609+0000","last_change":"2024-03-04T09:28:37.918023+0000","last_active":"2024-03-04T09:30:09.505609+0000","last_peered":"2024-03-04T09:30:09.505609+0000","last_clean":"2024-03-04T09:30:09.505609+0000","last_became_active":"2024-03-04T09:28:37.917671+0000","last_became_peered":"2024-03-04T09:28:37.917671+0000","last_unstale":"2024-03-04T09:30:09.505609+0000","last_undegraded":"2024-03-04T09:30:09.505609+0000","last_fullsized":"2024-03-04T09:30:09.505609+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_clean_scrub_stamp":"2024-03-04T09:28:36.887656+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T09:51:15.618328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T09:29:41.067989+0000","last_change":"2024-03-04T09:28:41.958903+0000","last_active":"2024-03-04T09:29:41.067989+0000","last_peered":"2024-03-04T09:29:41.067989+0000","last_clean":"2024-03-04T09:29:41.067989+0000","last_became_active":"2024-03-04T09:28:41.958256+0000","last_became_peered":"2024-03-04T09:28:41.958256+0000","last_unstale":"2024-03-04T09:29:41.067989+0000","last_undegraded":"2024-03-04T09:29:41.067989+0000","last_fullsized":"2024-03-04T09:29:41.067989+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:40.917115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:40.917115+0000","last_clean_scrub_stamp":"2024-03-04T09:28:40.917115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T18:49:48.022650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"89'15","reported_seq":43,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T09:29:41.065368+0000","last_change":"2024-03-04T09:28:36.911924+0000","last_active":"2024-03-04T09:29:41.065368+0000","last_peered":"2024-03-04T09:29:41.065368+0000","last_clean":"2024-03-04T09:29:41.065368+0000","last_became_active":"2024-03-04T09:28:36.911623+0000","last_became_peered":"2024-03-04T09:28:36.911623+0000","last_unstale":"2024-03-04T09:29:41.065368+0000","last_undegraded":"2024-03-04T09:29:41.065368+0000","last_fullsized":"2024-03-04T09:29:41.065368+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:35.885776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:35.885776+0000","last_clean_scrub_stamp":"2024-03-04T09:28:35.885776+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T13:17:32.606767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"91'6","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T09:30:09.509820+0000","last_change":"2024-03-04T09:28:37.916578+0000","last_active":"2024-03-04T09:30:09.509820+0000","last_peered":"2024-03-04T09:30:09.509820+0000","last_clean":"2024-03-04T09:30:09.509820+0000","last_became_active":"2024-03-04T09:28:37.913830+0000","last_became_peered":"2024-03-04T09:28:37.913830+0000","last_unstale":"2024-03-04T09:30:09.509820+0000","last_undegraded":"2024-03-04T09:30:09.509820+0000","last_fullsized":"2024-03-04T09:30:09.509820+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_clean_scrub_stamp":"2024-03-04T09:28:36.887656+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T09:29:15.882496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T09:29:41.411504+0000","last_change":"2024-03-04T09:29:41.411504+0000","last_active":"2024-03-04T09:29:41.411504+0000","last_peered":"2024-03-04T09:29:41.411504+0000","last_clean":"2024-03-04T09:29:41.411504+0000","last_became_active":"2024-03-04T09:29:41.411024+0000","last_became_peered":"2024-03-04T09:29:41.411024+0000","last_unstale":"2024-03-04T09:29:41.411504+0000","last_undegraded":"2024-03-04T09:29:41.411504+0000","last_fullsized":"2024-03-04T09:29:41.411504+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:40.917115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:40.917115+0000","last_clean_scrub_stamp":"2024-03-04T09:28:40.917115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T18:10:56.888602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,3,1],"acting":[8,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.f","version":"89'2","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T09:29:41.066076+0000","last_change":"2024-03-04T09:28:36.909779+0000","last_active":"2024-03-04T09:29:41.066076+0000","last_peered":"2024-03-04T09:29:41.066076+0000","last_clean":"2024-03-04T09:29:41.066076+0000","last_became_active":"2024-03-04T09:28:36.909466+0000","last_became_peered":"2024-03-04T09:28:36.909466+0000","last_unstale":"2024-03-04T09:29:41.066076+0000","last_undegraded":"2024-03-04T09:29:41.066076+0000","last_fullsized":"2024-03-04T09:29:41.066076+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:35.885776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:35.885776+0000","last_clean_scrub_stamp":"2024-03-04T09:28:35.885776+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T17:16:28.183292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"91'5","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T09:30:09.514599+0000","last_change":"2024-03-04T09:28:37.916371+0000","last_active":"2024-03-04T09:30:09.514599+0000","last_peered":"2024-03-04T09:30:09.514599+0000","last_clean":"2024-03-04T09:30:09.514599+0000","last_became_active":"2024-03-04T09:28:37.916068+0000","last_became_peered":"2024-03-04T09:28:37.916068+0000","last_unstale":"2024-03-04T09:30:09.514599+0000","last_undegraded":"2024-03-04T09:30:09.514599+0000","last_fullsized":"2024-03-04T09:30:09.514599+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_clean_scrub_stamp":"2024-03-04T09:28:36.887656+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T13:18:18.850165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T09:29:41.065688+0000","last_change":"2024-03-04T09:28:41.955301+0000","last_active":"2024-03-04T09:29:41.065688+0000","last_peered":"2024-03-04T09:29:41.065688+0000","last_clean":"2024-03-04T09:29:41.065688+0000","last_became_active":"2024-03-04T09:28:41.954774+0000","last_became_peered":"2024-03-04T09:28:41.954774+0000","last_unstale":"2024-03-04T09:29:41.065688+0000","last_undegraded":"2024-03-04T09:29:41.065688+0000","last_fullsized":"2024-03-04T09:29:41.065688+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:40.917115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:40.917115+0000","last_clean_scrub_stamp":"2024-03-04T09:28:40.917115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T16:56:55.463344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T09:29:41.062163+0000","last_change":"2024-03-04T09:28:36.901659+0000","last_active":"2024-03-04T09:29:41.062163+0000","last_peered":"2024-03-04T09:29:41.062163+0000","last_clean":"2024-03-04T09:29:41.062163+0000","last_became_active":"2024-03-04T09:28:36.901235+0000","last_became_peered":"2024-03-04T09:28:36.901235+0000","last_unstale":"2024-03-04T09:29:41.062163+0000","last_undegraded":"2024-03-04T09:29:41.062163+0000","last_fullsized":"2024-03-04T09:29:41.062163+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:35.885776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:35.885776+0000","last_clean_scrub_stamp":"2024-03-04T09:28:35.885776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T12:59:50.237172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"91'4","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T09:30:09.524380+0000","last_change":"2024-03-04T09:28:37.907905+0000","last_active":"2024-03-04T09:30:09.524380+0000","last_peered":"2024-03-04T09:30:09.524380+0000","last_clean":"2024-03-04T09:30:09.524380+0000","last_became_active":"2024-03-04T09:28:37.907376+0000","last_became_peered":"2024-03-04T09:28:37.907376+0000","last_unstale":"2024-03-04T09:30:09.524380+0000","last_undegraded":"2024-03-04T09:30:09.524380+0000","last_fullsized":"2024-03-04T09:30:09.524380+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_clean_scrub_stamp":"2024-03-04T09:28:36.887656+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T14:25:31.784659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T09:29:41.065377+0000","last_change":"2024-03-04T09:28:41.947629+0000","last_active":"2024-03-04T09:29:41.065377+0000","last_peered":"2024-03-04T09:29:41.065377+0000","last_clean":"2024-03-04T09:29:41.065377+0000","last_became_active":"2024-03-04T09:28:41.946251+0000","last_became_peered":"2024-03-04T09:28:41.946251+0000","last_unstale":"2024-03-04T09:29:41.065377+0000","last_undegraded":"2024-03-04T09:29:41.065377+0000","last_fullsized":"2024-03-04T09:29:41.065377+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:40.917115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:40.917115+0000","last_clean_scrub_stamp":"2024-03-04T09:28:40.917115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T18:26:57.770657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"89'1","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T09:29:41.064265+0000","last_change":"2024-03-04T09:28:36.905323+0000","last_active":"2024-03-04T09:29:41.064265+0000","last_peered":"2024-03-04T09:29:41.064265+0000","last_clean":"2024-03-04T09:29:41.064265+0000","last_became_active":"2024-03-04T09:28:36.904517+0000","last_became_peered":"2024-03-04T09:28:36.904517+0000","last_unstale":"2024-03-04T09:29:41.064265+0000","last_undegraded":"2024-03-04T09:29:41.064265+0000","last_fullsized":"2024-03-04T09:29:41.064265+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:35.885776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:35.885776+0000","last_clean_scrub_stamp":"2024-03-04T09:28:35.885776+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T18:52:11.615107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"91'7","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T09:30:09.524345+0000","last_change":"2024-03-04T09:28:37.921000+0000","last_active":"2024-03-04T09:30:09.524345+0000","last_peered":"2024-03-04T09:30:09.524345+0000","last_clean":"2024-03-04T09:30:09.524345+0000","last_became_active":"2024-03-04T09:28:37.920620+0000","last_became_peered":"2024-03-04T09:28:37.920620+0000","last_unstale":"2024-03-04T09:30:09.524345+0000","last_undegraded":"2024-03-04T09:30:09.524345+0000","last_fullsized":"2024-03-04T09:30:09.524345+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_clean_scrub_stamp":"2024-03-04T09:28:36.887656+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T10:41:29.187701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T09:29:41.066953+0000","last_change":"2024-03-04T09:28:41.949132+0000","last_active":"2024-03-04T09:29:41.066953+0000","last_peered":"2024-03-04T09:29:41.066953+0000","last_clean":"2024-03-04T09:29:41.066953+0000","last_became_active":"2024-03-04T09:28:41.948618+0000","last_became_peered":"2024-03-04T09:28:41.948618+0000","last_unstale":"2024-03-04T09:29:41.066953+0000","last_undegraded":"2024-03-04T09:29:41.066953+0000","last_fullsized":"2024-03-04T09:29:41.066953+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:40.917115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:40.917115+0000","last_clean_scrub_stamp":"2024-03-04T09:28:40.917115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T19:45:45.001345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"91'8","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T09:30:09.513356+0000","last_change":"2024-03-04T09:28:37.910475+0000","last_active":"2024-03-04T09:30:09.513356+0000","last_peered":"2024-03-04T09:30:09.513356+0000","last_clean":"2024-03-04T09:30:09.513356+0000","last_became_active":"2024-03-04T09:28:37.910184+0000","last_became_peered":"2024-03-04T09:28:37.910184+0000","last_unstale":"2024-03-04T09:30:09.513356+0000","last_undegraded":"2024-03-04T09:30:09.513356+0000","last_fullsized":"2024-03-04T09:30:09.513356+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_clean_scrub_stamp":"2024-03-04T09:28:36.887656+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T10:16:56.244331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"91'5","reported_seq":104,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T09:30:14.092706+0000","last_change":"2024-03-04T09:28:36.913489+0000","last_active":"2024-03-04T09:30:14.092706+0000","last_peered":"2024-03-04T09:30:14.092706+0000","last_clean":"2024-03-04T09:30:14.092706+0000","last_became_active":"2024-03-04T09:28:36.913162+0000","last_became_peered":"2024-03-04T09:28:36.913162+0000","last_unstale":"2024-03-04T09:30:14.092706+0000","last_undegraded":"2024-03-04T09:30:14.092706+0000","last_fullsized":"2024-03-04T09:30:14.092706+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:35.885776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:35.885776+0000","last_clean_scrub_stamp":"2024-03-04T09:28:35.885776+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T12:20:35.870058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":72,"num_read_kb":72,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"89'1","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T09:29:41.064510+0000","last_change":"2024-03-04T09:28:41.945717+0000","last_active":"2024-03-04T09:29:41.064510+0000","last_peered":"2024-03-04T09:29:41.064510+0000","last_clean":"2024-03-04T09:29:41.064510+0000","last_became_active":"2024-03-04T09:28:41.945300+0000","last_became_peered":"2024-03-04T09:28:41.945300+0000","last_unstale":"2024-03-04T09:29:41.064510+0000","last_undegraded":"2024-03-04T09:29:41.064510+0000","last_fullsized":"2024-03-04T09:29:41.064510+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:40.917115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:40.917115+0000","last_clean_scrub_stamp":"2024-03-04T09:28:40.917115+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T11:05:30.407232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"91'8","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T09:30:09.506071+0000","last_change":"2024-03-04T09:28:37.921030+0000","last_active":"2024-03-04T09:30:09.506071+0000","last_peered":"2024-03-04T09:30:09.506071+0000","last_clean":"2024-03-04T09:30:09.506071+0000","last_became_active":"2024-03-04T09:28:37.920655+0000","last_became_peered":"2024-03-04T09:28:37.920655+0000","last_unstale":"2024-03-04T09:30:09.506071+0000","last_undegraded":"2024-03-04T09:30:09.506071+0000","last_fullsized":"2024-03-04T09:30:09.506071+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_clean_scrub_stamp":"2024-03-04T09:28:36.887656+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T19:37:34.063523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"89'3","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T09:29:41.062153+0000","last_change":"2024-03-04T09:28:36.902902+0000","last_active":"2024-03-04T09:29:41.062153+0000","last_peered":"2024-03-04T09:29:41.062153+0000","last_clean":"2024-03-04T09:29:41.062153+0000","last_became_active":"2024-03-04T09:28:36.902574+0000","last_became_peered":"2024-03-04T09:28:36.902574+0000","last_unstale":"2024-03-04T09:29:41.062153+0000","last_undegraded":"2024-03-04T09:29:41.062153+0000","last_fullsized":"2024-03-04T09:29:41.062153+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:35.885776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:35.885776+0000","last_clean_scrub_stamp":"2024-03-04T09:28:35.885776+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T14:11:17.544497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":362,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T09:29:41.064047+0000","last_change":"2024-03-04T09:27:28.478476+0000","last_active":"2024-03-04T09:29:41.064047+0000","last_peered":"2024-03-04T09:29:41.064047+0000","last_clean":"2024-03-04T09:29:41.064047+0000","last_became_active":"2024-03-04T09:27:28.478041+0000","last_became_peered":"2024-03-04T09:27:28.478041+0000","last_unstale":"2024-03-04T09:29:41.064047+0000","last_undegraded":"2024-03-04T09:29:41.064047+0000","last_fullsized":"2024-03-04T09:29:41.064047+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:24:01.742551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:24:01.742551+0000","last_clean_scrub_stamp":"2024-03-04T09:24:01.742551+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T12:51:11.585296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T09:29:41.409966+0000","last_change":"2024-03-04T09:28:41.958895+0000","last_active":"2024-03-04T09:29:41.409966+0000","last_peered":"2024-03-04T09:29:41.409966+0000","last_clean":"2024-03-04T09:29:41.409966+0000","last_became_active":"2024-03-04T09:28:41.958477+0000","last_became_peered":"2024-03-04T09:28:41.958477+0000","last_unstale":"2024-03-04T09:29:41.409966+0000","last_undegraded":"2024-03-04T09:29:41.409966+0000","last_fullsized":"2024-03-04T09:29:41.409966+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:40.917115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:40.917115+0000","last_clean_scrub_stamp":"2024-03-04T09:28:40.917115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T17:48:37.149539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"91'5","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T09:30:09.519445+0000","last_change":"2024-03-04T09:28:37.911872+0000","last_active":"2024-03-04T09:30:09.519445+0000","last_peered":"2024-03-04T09:30:09.519445+0000","last_clean":"2024-03-04T09:30:09.519445+0000","last_became_active":"2024-03-04T09:28:37.911583+0000","last_became_peered":"2024-03-04T09:28:37.911583+0000","last_unstale":"2024-03-04T09:30:09.519445+0000","last_undegraded":"2024-03-04T09:30:09.519445+0000","last_fullsized":"2024-03-04T09:30:09.519445+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_clean_scrub_stamp":"2024-03-04T09:28:36.887656+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T20:12:15.606046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"89'3","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T09:29:41.064006+0000","last_change":"2024-03-04T09:28:36.905156+0000","last_active":"2024-03-04T09:29:41.064006+0000","last_peered":"2024-03-04T09:29:41.064006+0000","last_clean":"2024-03-04T09:29:41.064006+0000","last_became_active":"2024-03-04T09:28:36.904702+0000","last_became_peered":"2024-03-04T09:28:36.904702+0000","last_unstale":"2024-03-04T09:29:41.064006+0000","last_undegraded":"2024-03-04T09:29:41.064006+0000","last_fullsized":"2024-03-04T09:29:41.064006+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:35.885776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:35.885776+0000","last_clean_scrub_stamp":"2024-03-04T09:28:35.885776+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T19:38:16.485245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T09:29:41.063757+0000","last_change":"2024-03-04T09:28:41.954087+0000","last_active":"2024-03-04T09:29:41.063757+0000","last_peered":"2024-03-04T09:29:41.063757+0000","last_clean":"2024-03-04T09:29:41.063757+0000","last_became_active":"2024-03-04T09:28:41.953681+0000","last_became_peered":"2024-03-04T09:28:41.953681+0000","last_unstale":"2024-03-04T09:29:41.063757+0000","last_undegraded":"2024-03-04T09:29:41.063757+0000","last_fullsized":"2024-03-04T09:29:41.063757+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:40.917115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:40.917115+0000","last_clean_scrub_stamp":"2024-03-04T09:28:40.917115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T19:45:18.525709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"89'1","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T09:29:41.066030+0000","last_change":"2024-03-04T09:28:36.912400+0000","last_active":"2024-03-04T09:29:41.066030+0000","last_peered":"2024-03-04T09:29:41.066030+0000","last_clean":"2024-03-04T09:29:41.066030+0000","last_became_active":"2024-03-04T09:28:36.912096+0000","last_became_peered":"2024-03-04T09:28:36.912096+0000","last_unstale":"2024-03-04T09:29:41.066030+0000","last_undegraded":"2024-03-04T09:29:41.066030+0000","last_fullsized":"2024-03-04T09:29:41.066030+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:35.885776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:35.885776+0000","last_clean_scrub_stamp":"2024-03-04T09:28:35.885776+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T16:45:54.750427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"91'9","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T09:30:09.519400+0000","last_change":"2024-03-04T09:28:37.902851+0000","last_active":"2024-03-04T09:30:09.519400+0000","last_peered":"2024-03-04T09:30:09.519400+0000","last_clean":"2024-03-04T09:30:09.519400+0000","last_became_active":"2024-03-04T09:28:37.902537+0000","last_became_peered":"2024-03-04T09:28:37.902537+0000","last_unstale":"2024-03-04T09:30:09.519400+0000","last_undegraded":"2024-03-04T09:30:09.519400+0000","last_fullsized":"2024-03-04T09:30:09.519400+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_clean_scrub_stamp":"2024-03-04T09:28:36.887656+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T14:30:50.350161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T09:29:41.070473+0000","last_change":"2024-03-04T09:28:41.948551+0000","last_active":"2024-03-04T09:29:41.070473+0000","last_peered":"2024-03-04T09:29:41.070473+0000","last_clean":"2024-03-04T09:29:41.070473+0000","last_became_active":"2024-03-04T09:28:41.946636+0000","last_became_peered":"2024-03-04T09:28:41.946636+0000","last_unstale":"2024-03-04T09:29:41.070473+0000","last_undegraded":"2024-03-04T09:29:41.070473+0000","last_fullsized":"2024-03-04T09:29:41.070473+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:40.917115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:40.917115+0000","last_clean_scrub_stamp":"2024-03-04T09:28:40.917115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T14:00:07.624296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"91'9","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T09:29:41.062700+0000","last_change":"2024-03-04T09:28:36.910605+0000","last_active":"2024-03-04T09:29:41.062700+0000","last_peered":"2024-03-04T09:29:41.062700+0000","last_clean":"2024-03-04T09:29:41.062700+0000","last_became_active":"2024-03-04T09:28:36.910253+0000","last_became_peered":"2024-03-04T09:28:36.910253+0000","last_unstale":"2024-03-04T09:29:41.062700+0000","last_undegraded":"2024-03-04T09:29:41.062700+0000","last_fullsized":"2024-03-04T09:29:41.062700+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:35.885776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:35.885776+0000","last_clean_scrub_stamp":"2024-03-04T09:28:35.885776+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T18:08:30.941310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"91'5","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T09:30:09.515703+0000","last_change":"2024-03-04T09:28:37.907342+0000","last_active":"2024-03-04T09:30:09.515703+0000","last_peered":"2024-03-04T09:30:09.515703+0000","last_clean":"2024-03-04T09:30:09.515703+0000","last_became_active":"2024-03-04T09:28:37.906974+0000","last_became_peered":"2024-03-04T09:28:37.906974+0000","last_unstale":"2024-03-04T09:30:09.515703+0000","last_undegraded":"2024-03-04T09:30:09.515703+0000","last_fullsized":"2024-03-04T09:30:09.515703+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_clean_scrub_stamp":"2024-03-04T09:28:36.887656+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T11:18:43.099544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T09:29:41.061765+0000","last_change":"2024-03-04T09:28:41.945713+0000","last_active":"2024-03-04T09:29:41.061765+0000","last_peered":"2024-03-04T09:29:41.061765+0000","last_clean":"2024-03-04T09:29:41.061765+0000","last_became_active":"2024-03-04T09:28:41.945245+0000","last_became_peered":"2024-03-04T09:28:41.945245+0000","last_unstale":"2024-03-04T09:29:41.061765+0000","last_undegraded":"2024-03-04T09:29:41.061765+0000","last_fullsized":"2024-03-04T09:29:41.061765+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:40.917115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:40.917115+0000","last_clean_scrub_stamp":"2024-03-04T09:28:40.917115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T11:06:39.595269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"89'4","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T09:29:41.062592+0000","last_change":"2024-03-04T09:28:36.902020+0000","last_active":"2024-03-04T09:29:41.062592+0000","last_peered":"2024-03-04T09:29:41.062592+0000","last_clean":"2024-03-04T09:29:41.062592+0000","last_became_active":"2024-03-04T09:28:36.901666+0000","last_became_peered":"2024-03-04T09:28:36.901666+0000","last_unstale":"2024-03-04T09:29:41.062592+0000","last_undegraded":"2024-03-04T09:29:41.062592+0000","last_fullsized":"2024-03-04T09:29:41.062592+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:35.885776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:35.885776+0000","last_clean_scrub_stamp":"2024-03-04T09:28:35.885776+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T12:48:17.270200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"91'7","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T09:30:09.525463+0000","last_change":"2024-03-04T09:28:37.912889+0000","last_active":"2024-03-04T09:30:09.525463+0000","last_peered":"2024-03-04T09:30:09.525463+0000","last_clean":"2024-03-04T09:30:09.525463+0000","last_became_active":"2024-03-04T09:28:37.912542+0000","last_became_peered":"2024-03-04T09:28:37.912542+0000","last_unstale":"2024-03-04T09:30:09.525463+0000","last_undegraded":"2024-03-04T09:30:09.525463+0000","last_fullsized":"2024-03-04T09:30:09.525463+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_clean_scrub_stamp":"2024-03-04T09:28:36.887656+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T20:51:12.540364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T09:29:41.062161+0000","last_change":"2024-03-04T09:28:41.949009+0000","last_active":"2024-03-04T09:29:41.062161+0000","last_peered":"2024-03-04T09:29:41.062161+0000","last_clean":"2024-03-04T09:29:41.062161+0000","last_became_active":"2024-03-04T09:28:41.948178+0000","last_became_peered":"2024-03-04T09:28:41.948178+0000","last_unstale":"2024-03-04T09:29:41.062161+0000","last_undegraded":"2024-03-04T09:29:41.062161+0000","last_fullsized":"2024-03-04T09:29:41.062161+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:40.917115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:40.917115+0000","last_clean_scrub_stamp":"2024-03-04T09:28:40.917115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T10:54:54.360229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"89'2","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T09:29:41.409452+0000","last_change":"2024-03-04T09:28:36.901847+0000","last_active":"2024-03-04T09:29:41.409452+0000","last_peered":"2024-03-04T09:29:41.409452+0000","last_clean":"2024-03-04T09:29:41.409452+0000","last_became_active":"2024-03-04T09:28:36.901461+0000","last_became_peered":"2024-03-04T09:28:36.901461+0000","last_unstale":"2024-03-04T09:29:41.409452+0000","last_undegraded":"2024-03-04T09:29:41.409452+0000","last_fullsized":"2024-03-04T09:29:41.409452+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:35.885776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:35.885776+0000","last_clean_scrub_stamp":"2024-03-04T09:28:35.885776+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T14:36:33.041447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"91'2","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T09:30:09.475901+0000","last_change":"2024-03-04T09:28:37.907958+0000","last_active":"2024-03-04T09:30:09.475901+0000","last_peered":"2024-03-04T09:30:09.475901+0000","last_clean":"2024-03-04T09:30:09.475901+0000","last_became_active":"2024-03-04T09:28:37.907653+0000","last_became_peered":"2024-03-04T09:28:37.907653+0000","last_unstale":"2024-03-04T09:30:09.475901+0000","last_undegraded":"2024-03-04T09:30:09.475901+0000","last_fullsized":"2024-03-04T09:30:09.475901+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_clean_scrub_stamp":"2024-03-04T09:28:36.887656+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T20:42:44.266275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T09:29:41.062815+0000","last_change":"2024-03-04T09:28:41.957479+0000","last_active":"2024-03-04T09:29:41.062815+0000","last_peered":"2024-03-04T09:29:41.062815+0000","last_clean":"2024-03-04T09:29:41.062815+0000","last_became_active":"2024-03-04T09:28:41.954950+0000","last_became_peered":"2024-03-04T09:28:41.954950+0000","last_unstale":"2024-03-04T09:29:41.062815+0000","last_undegraded":"2024-03-04T09:29:41.062815+0000","last_fullsized":"2024-03-04T09:29:41.062815+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:40.917115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:40.917115+0000","last_clean_scrub_stamp":"2024-03-04T09:28:40.917115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T14:53:16.636230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"89'3","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T09:29:41.070521+0000","last_change":"2024-03-04T09:28:36.908649+0000","last_active":"2024-03-04T09:29:41.070521+0000","last_peered":"2024-03-04T09:29:41.070521+0000","last_clean":"2024-03-04T09:29:41.070521+0000","last_became_active":"2024-03-04T09:28:36.908323+0000","last_became_peered":"2024-03-04T09:28:36.908323+0000","last_unstale":"2024-03-04T09:29:41.070521+0000","last_undegraded":"2024-03-04T09:29:41.070521+0000","last_fullsized":"2024-03-04T09:29:41.070521+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:35.885776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:35.885776+0000","last_clean_scrub_stamp":"2024-03-04T09:28:35.885776+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T16:49:40.367952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"91'9","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T09:30:09.523113+0000","last_change":"2024-03-04T09:28:37.905673+0000","last_active":"2024-03-04T09:30:09.523113+0000","last_peered":"2024-03-04T09:30:09.523113+0000","last_clean":"2024-03-04T09:30:09.523113+0000","last_became_active":"2024-03-04T09:28:37.905368+0000","last_became_peered":"2024-03-04T09:28:37.905368+0000","last_unstale":"2024-03-04T09:30:09.523113+0000","last_undegraded":"2024-03-04T09:30:09.523113+0000","last_fullsized":"2024-03-04T09:30:09.523113+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_clean_scrub_stamp":"2024-03-04T09:28:36.887656+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T17:25:35.058092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"89'1","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T09:29:41.061920+0000","last_change":"2024-03-04T09:28:41.948688+0000","last_active":"2024-03-04T09:29:41.061920+0000","last_peered":"2024-03-04T09:29:41.061920+0000","last_clean":"2024-03-04T09:29:41.061920+0000","last_became_active":"2024-03-04T09:28:41.948111+0000","last_became_peered":"2024-03-04T09:28:41.948111+0000","last_unstale":"2024-03-04T09:29:41.061920+0000","last_undegraded":"2024-03-04T09:29:41.061920+0000","last_fullsized":"2024-03-04T09:29:41.061920+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:40.917115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:40.917115+0000","last_clean_scrub_stamp":"2024-03-04T09:28:40.917115+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T12:27:37.121328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"89'1","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T09:29:41.061844+0000","last_change":"2024-03-04T09:28:36.908438+0000","last_active":"2024-03-04T09:29:41.061844+0000","last_peered":"2024-03-04T09:29:41.061844+0000","last_clean":"2024-03-04T09:29:41.061844+0000","last_became_active":"2024-03-04T09:28:36.906529+0000","last_became_peered":"2024-03-04T09:28:36.906529+0000","last_unstale":"2024-03-04T09:29:41.061844+0000","last_undegraded":"2024-03-04T09:29:41.061844+0000","last_fullsized":"2024-03-04T09:29:41.061844+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:35.885776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:35.885776+0000","last_clean_scrub_stamp":"2024-03-04T09:28:35.885776+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T10:12:54.833556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"91'8","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T09:30:09.510926+0000","last_change":"2024-03-04T09:28:37.927048+0000","last_active":"2024-03-04T09:30:09.510926+0000","last_peered":"2024-03-04T09:30:09.510926+0000","last_clean":"2024-03-04T09:30:09.510926+0000","last_became_active":"2024-03-04T09:28:37.925381+0000","last_became_peered":"2024-03-04T09:28:37.925381+0000","last_unstale":"2024-03-04T09:30:09.510926+0000","last_undegraded":"2024-03-04T09:30:09.510926+0000","last_fullsized":"2024-03-04T09:30:09.510926+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_clean_scrub_stamp":"2024-03-04T09:28:36.887656+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T20:19:48.554102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T09:29:41.062103+0000","last_change":"2024-03-04T09:28:41.945805+0000","last_active":"2024-03-04T09:29:41.062103+0000","last_peered":"2024-03-04T09:29:41.062103+0000","last_clean":"2024-03-04T09:29:41.062103+0000","last_became_active":"2024-03-04T09:28:41.945414+0000","last_became_peered":"2024-03-04T09:28:41.945414+0000","last_unstale":"2024-03-04T09:29:41.062103+0000","last_undegraded":"2024-03-04T09:29:41.062103+0000","last_fullsized":"2024-03-04T09:29:41.062103+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:40.917115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:40.917115+0000","last_clean_scrub_stamp":"2024-03-04T09:28:40.917115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T09:45:59.612507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"89'4","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T09:29:41.065464+0000","last_change":"2024-03-04T09:28:36.909808+0000","last_active":"2024-03-04T09:29:41.065464+0000","last_peered":"2024-03-04T09:29:41.065464+0000","last_clean":"2024-03-04T09:29:41.065464+0000","last_became_active":"2024-03-04T09:28:36.909486+0000","last_became_peered":"2024-03-04T09:28:36.909486+0000","last_unstale":"2024-03-04T09:29:41.065464+0000","last_undegraded":"2024-03-04T09:29:41.065464+0000","last_fullsized":"2024-03-04T09:29:41.065464+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:35.885776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:35.885776+0000","last_clean_scrub_stamp":"2024-03-04T09:28:35.885776+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T19:38:15.524941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"91'5","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T09:30:09.529632+0000","last_change":"2024-03-04T09:28:37.910807+0000","last_active":"2024-03-04T09:30:09.529632+0000","last_peered":"2024-03-04T09:30:09.529632+0000","last_clean":"2024-03-04T09:30:09.529632+0000","last_became_active":"2024-03-04T09:28:37.910504+0000","last_became_peered":"2024-03-04T09:28:37.910504+0000","last_unstale":"2024-03-04T09:30:09.529632+0000","last_undegraded":"2024-03-04T09:30:09.529632+0000","last_fullsized":"2024-03-04T09:30:09.529632+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_clean_scrub_stamp":"2024-03-04T09:28:36.887656+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T20:03:35.484747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T09:29:41.409454+0000","last_change":"2024-03-04T09:28:41.949798+0000","last_active":"2024-03-04T09:29:41.409454+0000","last_peered":"2024-03-04T09:29:41.409454+0000","last_clean":"2024-03-04T09:29:41.409454+0000","last_became_active":"2024-03-04T09:28:41.949358+0000","last_became_peered":"2024-03-04T09:28:41.949358+0000","last_unstale":"2024-03-04T09:29:41.409454+0000","last_undegraded":"2024-03-04T09:29:41.409454+0000","last_fullsized":"2024-03-04T09:29:41.409454+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:40.917115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:40.917115+0000","last_clean_scrub_stamp":"2024-03-04T09:28:40.917115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T13:30:26.519090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"89'3","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T09:29:41.062398+0000","last_change":"2024-03-04T09:28:36.912458+0000","last_active":"2024-03-04T09:29:41.062398+0000","last_peered":"2024-03-04T09:29:41.062398+0000","last_clean":"2024-03-04T09:29:41.062398+0000","last_became_active":"2024-03-04T09:28:36.912132+0000","last_became_peered":"2024-03-04T09:28:36.912132+0000","last_unstale":"2024-03-04T09:29:41.062398+0000","last_undegraded":"2024-03-04T09:29:41.062398+0000","last_fullsized":"2024-03-04T09:29:41.062398+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:35.885776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:35.885776+0000","last_clean_scrub_stamp":"2024-03-04T09:28:35.885776+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T13:51:29.713381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"91'7","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T09:30:09.517387+0000","last_change":"2024-03-04T09:28:37.915750+0000","last_active":"2024-03-04T09:30:09.517387+0000","last_peered":"2024-03-04T09:30:09.517387+0000","last_clean":"2024-03-04T09:30:09.517387+0000","last_became_active":"2024-03-04T09:28:37.915424+0000","last_became_peered":"2024-03-04T09:28:37.915424+0000","last_unstale":"2024-03-04T09:30:09.517387+0000","last_undegraded":"2024-03-04T09:30:09.517387+0000","last_fullsized":"2024-03-04T09:30:09.517387+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_clean_scrub_stamp":"2024-03-04T09:28:36.887656+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T19:15:39.555122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T09:29:41.066935+0000","last_change":"2024-03-04T09:28:41.949321+0000","last_active":"2024-03-04T09:29:41.066935+0000","last_peered":"2024-03-04T09:29:41.066935+0000","last_clean":"2024-03-04T09:29:41.066935+0000","last_became_active":"2024-03-04T09:28:41.948302+0000","last_became_peered":"2024-03-04T09:28:41.948302+0000","last_unstale":"2024-03-04T09:29:41.066935+0000","last_undegraded":"2024-03-04T09:29:41.066935+0000","last_fullsized":"2024-03-04T09:29:41.066935+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:40.917115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:40.917115+0000","last_clean_scrub_stamp":"2024-03-04T09:28:40.917115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T10:18:44.988651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"84'1","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T09:29:41.410258+0000","last_change":"2024-03-04T09:28:36.909585+0000","last_active":"2024-03-04T09:29:41.410258+0000","last_peered":"2024-03-04T09:29:41.410258+0000","last_clean":"2024-03-04T09:29:41.410258+0000","last_became_active":"2024-03-04T09:28:36.909286+0000","last_became_peered":"2024-03-04T09:28:36.909286+0000","last_unstale":"2024-03-04T09:29:41.410258+0000","last_undegraded":"2024-03-04T09:29:41.410258+0000","last_fullsized":"2024-03-04T09:29:41.410258+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:35.885776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:35.885776+0000","last_clean_scrub_stamp":"2024-03-04T09:28:35.885776+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T09:29:45.236355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"91'6","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T09:30:09.519785+0000","last_change":"2024-03-04T09:28:37.924411+0000","last_active":"2024-03-04T09:30:09.519785+0000","last_peered":"2024-03-04T09:30:09.519785+0000","last_clean":"2024-03-04T09:30:09.519785+0000","last_became_active":"2024-03-04T09:28:37.924147+0000","last_became_peered":"2024-03-04T09:28:37.924147+0000","last_unstale":"2024-03-04T09:30:09.519785+0000","last_undegraded":"2024-03-04T09:30:09.519785+0000","last_fullsized":"2024-03-04T09:30:09.519785+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_clean_scrub_stamp":"2024-03-04T09:28:36.887656+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T15:05:16.573937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T09:29:41.409623+0000","last_change":"2024-03-04T09:28:41.943710+0000","last_active":"2024-03-04T09:29:41.409623+0000","last_peered":"2024-03-04T09:29:41.409623+0000","last_clean":"2024-03-04T09:29:41.409623+0000","last_became_active":"2024-03-04T09:28:41.943258+0000","last_became_peered":"2024-03-04T09:28:41.943258+0000","last_unstale":"2024-03-04T09:29:41.409623+0000","last_undegraded":"2024-03-04T09:29:41.409623+0000","last_fullsized":"2024-03-04T09:29:41.409623+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:40.917115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:40.917115+0000","last_clean_scrub_stamp":"2024-03-04T09:28:40.917115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T15:32:52.597594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"84'2","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T09:29:41.075249+0000","last_change":"2024-03-04T09:29:41.075249+0000","last_active":"2024-03-04T09:29:41.075249+0000","last_peered":"2024-03-04T09:29:41.075249+0000","last_clean":"2024-03-04T09:29:41.075249+0000","last_became_active":"2024-03-04T09:29:41.072321+0000","last_became_peered":"2024-03-04T09:29:41.072321+0000","last_unstale":"2024-03-04T09:29:41.075249+0000","last_undegraded":"2024-03-04T09:29:41.075249+0000","last_fullsized":"2024-03-04T09:29:41.075249+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:35.885776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:35.885776+0000","last_clean_scrub_stamp":"2024-03-04T09:28:35.885776+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T09:59:10.757578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4],"acting":[6,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"91'9","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T09:30:09.513400+0000","last_change":"2024-03-04T09:28:37.916569+0000","last_active":"2024-03-04T09:30:09.513400+0000","last_peered":"2024-03-04T09:30:09.513400+0000","last_clean":"2024-03-04T09:30:09.513400+0000","last_became_active":"2024-03-04T09:28:37.916264+0000","last_became_peered":"2024-03-04T09:28:37.916264+0000","last_unstale":"2024-03-04T09:30:09.513400+0000","last_undegraded":"2024-03-04T09:30:09.513400+0000","last_fullsized":"2024-03-04T09:30:09.513400+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_clean_scrub_stamp":"2024-03-04T09:28:36.887656+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T16:59:51.981066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"91'9","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T09:30:09.516975+0000","last_change":"2024-03-04T09:29:41.068291+0000","last_active":"2024-03-04T09:30:09.516975+0000","last_peered":"2024-03-04T09:30:09.516975+0000","last_clean":"2024-03-04T09:30:09.516975+0000","last_became_active":"2024-03-04T09:29:41.067902+0000","last_became_peered":"2024-03-04T09:29:41.067902+0000","last_unstale":"2024-03-04T09:30:09.516975+0000","last_undegraded":"2024-03-04T09:30:09.516975+0000","last_fullsized":"2024-03-04T09:30:09.516975+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_clean_scrub_stamp":"2024-03-04T09:28:36.887656+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T12:40:21.605947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,9],"acting":[11,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"91'12","reported_seq":43,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T09:30:09.517696+0000","last_change":"2024-03-04T09:29:41.411161+0000","last_active":"2024-03-04T09:30:09.517696+0000","last_peered":"2024-03-04T09:30:09.517696+0000","last_clean":"2024-03-04T09:30:09.517696+0000","last_became_active":"2024-03-04T09:29:41.410703+0000","last_became_peered":"2024-03-04T09:29:41.410703+0000","last_unstale":"2024-03-04T09:30:09.517696+0000","last_undegraded":"2024-03-04T09:30:09.517696+0000","last_fullsized":"2024-03-04T09:30:09.517696+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_clean_scrub_stamp":"2024-03-04T09:28:36.887656+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T17:43:19.492432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,6],"acting":[2,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"91'8","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T09:30:09.511900+0000","last_change":"2024-03-04T09:28:37.906956+0000","last_active":"2024-03-04T09:30:09.511900+0000","last_peered":"2024-03-04T09:30:09.511900+0000","last_clean":"2024-03-04T09:30:09.511900+0000","last_became_active":"2024-03-04T09:28:37.906642+0000","last_became_peered":"2024-03-04T09:28:37.906642+0000","last_unstale":"2024-03-04T09:30:09.511900+0000","last_undegraded":"2024-03-04T09:30:09.511900+0000","last_fullsized":"2024-03-04T09:30:09.511900+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_clean_scrub_stamp":"2024-03-04T09:28:36.887656+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T20:58:18.387119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"91'7","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T09:30:09.525930+0000","last_change":"2024-03-04T09:29:41.071209+0000","last_active":"2024-03-04T09:30:09.525930+0000","last_peered":"2024-03-04T09:30:09.525930+0000","last_clean":"2024-03-04T09:30:09.525930+0000","last_became_active":"2024-03-04T09:29:41.070539+0000","last_became_peered":"2024-03-04T09:29:41.070539+0000","last_unstale":"2024-03-04T09:30:09.525930+0000","last_undegraded":"2024-03-04T09:30:09.525930+0000","last_fullsized":"2024-03-04T09:30:09.525930+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_clean_scrub_stamp":"2024-03-04T09:28:36.887656+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T11:47:08.175691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,8],"acting":[0,6,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"91'4","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T09:30:09.523479+0000","last_change":"2024-03-04T09:28:37.912702+0000","last_active":"2024-03-04T09:30:09.523479+0000","last_peered":"2024-03-04T09:30:09.523479+0000","last_clean":"2024-03-04T09:30:09.523479+0000","last_became_active":"2024-03-04T09:28:37.912397+0000","last_became_peered":"2024-03-04T09:28:37.912397+0000","last_unstale":"2024-03-04T09:30:09.523479+0000","last_undegraded":"2024-03-04T09:30:09.523479+0000","last_fullsized":"2024-03-04T09:30:09.523479+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_clean_scrub_stamp":"2024-03-04T09:28:36.887656+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T16:36:59.575111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"91'6","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T09:30:09.514765+0000","last_change":"2024-03-04T09:28:37.909801+0000","last_active":"2024-03-04T09:30:09.514765+0000","last_peered":"2024-03-04T09:30:09.514765+0000","last_clean":"2024-03-04T09:30:09.514765+0000","last_became_active":"2024-03-04T09:28:37.909505+0000","last_became_peered":"2024-03-04T09:28:37.909505+0000","last_unstale":"2024-03-04T09:30:09.514765+0000","last_undegraded":"2024-03-04T09:30:09.514765+0000","last_fullsized":"2024-03-04T09:30:09.514765+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_clean_scrub_stamp":"2024-03-04T09:28:36.887656+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T11:56:44.276188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"91'15","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T09:30:09.506625+0000","last_change":"2024-03-04T09:28:37.911076+0000","last_active":"2024-03-04T09:30:09.506625+0000","last_peered":"2024-03-04T09:30:09.506625+0000","last_clean":"2024-03-04T09:30:09.506625+0000","last_became_active":"2024-03-04T09:28:37.910786+0000","last_became_peered":"2024-03-04T09:28:37.910786+0000","last_unstale":"2024-03-04T09:30:09.506625+0000","last_undegraded":"2024-03-04T09:30:09.506625+0000","last_fullsized":"2024-03-04T09:30:09.506625+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_clean_scrub_stamp":"2024-03-04T09:28:36.887656+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T11:42:41.932235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"91'2","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T09:30:09.513363+0000","last_change":"2024-03-04T09:28:37.914661+0000","last_active":"2024-03-04T09:30:09.513363+0000","last_peered":"2024-03-04T09:30:09.513363+0000","last_clean":"2024-03-04T09:30:09.513363+0000","last_became_active":"2024-03-04T09:28:37.913851+0000","last_became_peered":"2024-03-04T09:28:37.913851+0000","last_unstale":"2024-03-04T09:30:09.513363+0000","last_undegraded":"2024-03-04T09:30:09.513363+0000","last_fullsized":"2024-03-04T09:30:09.513363+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_clean_scrub_stamp":"2024-03-04T09:28:36.887656+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T10:45:43.670006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"91'6","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T09:30:09.510744+0000","last_change":"2024-03-04T09:28:37.904746+0000","last_active":"2024-03-04T09:30:09.510744+0000","last_peered":"2024-03-04T09:30:09.510744+0000","last_clean":"2024-03-04T09:30:09.510744+0000","last_became_active":"2024-03-04T09:28:37.904466+0000","last_became_peered":"2024-03-04T09:28:37.904466+0000","last_unstale":"2024-03-04T09:30:09.510744+0000","last_undegraded":"2024-03-04T09:30:09.510744+0000","last_fullsized":"2024-03-04T09:30:09.510744+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_clean_scrub_stamp":"2024-03-04T09:28:36.887656+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T13:59:24.810820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"91'8","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T09:30:09.518488+0000","last_change":"2024-03-04T09:28:37.913252+0000","last_active":"2024-03-04T09:30:09.518488+0000","last_peered":"2024-03-04T09:30:09.518488+0000","last_clean":"2024-03-04T09:30:09.518488+0000","last_became_active":"2024-03-04T09:28:37.910819+0000","last_became_peered":"2024-03-04T09:28:37.910819+0000","last_unstale":"2024-03-04T09:30:09.518488+0000","last_undegraded":"2024-03-04T09:30:09.518488+0000","last_fullsized":"2024-03-04T09:30:09.518488+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_clean_scrub_stamp":"2024-03-04T09:28:36.887656+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T13:39:47.886505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"91'10","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T09:30:09.531136+0000","last_change":"2024-03-04T09:28:37.915716+0000","last_active":"2024-03-04T09:30:09.531136+0000","last_peered":"2024-03-04T09:30:09.531136+0000","last_clean":"2024-03-04T09:30:09.531136+0000","last_became_active":"2024-03-04T09:28:37.915398+0000","last_became_peered":"2024-03-04T09:28:37.915398+0000","last_unstale":"2024-03-04T09:30:09.531136+0000","last_undegraded":"2024-03-04T09:30:09.531136+0000","last_fullsized":"2024-03-04T09:30:09.531136+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_clean_scrub_stamp":"2024-03-04T09:28:36.887656+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T20:10:53.793715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"91'11","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T09:30:09.523748+0000","last_change":"2024-03-04T09:28:37.906671+0000","last_active":"2024-03-04T09:30:09.523748+0000","last_peered":"2024-03-04T09:30:09.523748+0000","last_clean":"2024-03-04T09:30:09.523748+0000","last_became_active":"2024-03-04T09:28:37.906344+0000","last_became_peered":"2024-03-04T09:28:37.906344+0000","last_unstale":"2024-03-04T09:30:09.523748+0000","last_undegraded":"2024-03-04T09:30:09.523748+0000","last_fullsized":"2024-03-04T09:30:09.523748+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_clean_scrub_stamp":"2024-03-04T09:28:36.887656+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T16:02:18.204424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"91'8","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T09:30:09.512715+0000","last_change":"2024-03-04T09:28:37.916895+0000","last_active":"2024-03-04T09:30:09.512715+0000","last_peered":"2024-03-04T09:30:09.512715+0000","last_clean":"2024-03-04T09:30:09.512715+0000","last_became_active":"2024-03-04T09:28:37.916585+0000","last_became_peered":"2024-03-04T09:28:37.916585+0000","last_unstale":"2024-03-04T09:30:09.512715+0000","last_undegraded":"2024-03-04T09:30:09.512715+0000","last_fullsized":"2024-03-04T09:30:09.512715+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_clean_scrub_stamp":"2024-03-04T09:28:36.887656+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T12:13:50.941390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":66166,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":462,"num_read_kb":301,"num_write":166,"num_write_kb":118,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":198498,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":144,"ondisk_log_size":144,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383713,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28016,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717136,"statfs":{"total":95995035648,"available":95966347264,"internally_reserved":0,"allocated":1024000,"data_stored":500831,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":72,"seq":309237645349,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":37008,"kb_used_data":1672,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708144,"statfs":{"total":95995035648,"available":95957139456,"internally_reserved":0,"allocated":1712128,"data_stored":1150655,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":67,"seq":287762808875,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36356,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708796,"statfs":{"total":95995035648,"available":95957807104,"internally_reserved":0,"allocated":1044480,"data_stored":505163,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":36174273},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":8,"up_from":59,"seq":253403070512,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28032,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717120,"statfs":{"total":95995035648,"available":95966330880,"internally_reserved":0,"allocated":1040384,"data_stored":500987,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332149,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28612,"kb_used_data":1596,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716540,"statfs":{"total":95995035648,"available":95965736960,"internally_reserved":0,"allocated":1634304,"data_stored":1097037,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495675,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28012,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717140,"statfs":{"total":95995035648,"available":95966351360,"internally_reserved":0,"allocated":1019904,"data_stored":496243,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574931,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28076,"kb_used_data":1060,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717076,"statfs":{"total":95995035648,"available":95966285824,"internally_reserved":0,"allocated":1085440,"data_stored":558177,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738456,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28036,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717116,"statfs":{"total":95995035648,"available":95966326784,"internally_reserved":0,"allocated":1044480,"data_stored":497631,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411407,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28100,"kb_used_data":1084,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717052,"statfs":{"total":95995035648,"available":95966261248,"internally_reserved":0,"allocated":1110016,"data_stored":559527,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182472,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36344,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708808,"statfs":{"total":95995035648,"available":95957819392,"internally_reserved":0,"allocated":1032192,"data_stored":496437,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920836,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28024,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717128,"statfs":{"total":95995035648,"available":95966339072,"internally_reserved":0,"allocated":1032192,"data_stored":501425,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757312,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36940,"kb_used_data":1604,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708212,"statfs":{"total":95995035648,"available":95957209088,"internally_reserved":0,"allocated":1642496,"data_stored":1092093,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":62024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":63374,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1176,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2324,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":64134,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-04T09:30:19.050 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd shell --fsid 4e059f08-da08-11ee-95c6-87774f69a715 -- ceph pg dump --format=json 2024-03-04T09:30:19.298 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/4e059f08-da08-11ee-95c6-87774f69a715/mon.a/config 2024-03-04T09:30:19.334 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:30:19 smithi052 ceph-mon[32473]: pgmap v340: 129 pgs: 129 active+clean; 641 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 426 B/s wr, 6 op/s 2024-03-04T09:30:19.335 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:30:19 smithi052 ceph-mon[32473]: from='client.15117 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-04T09:30:19.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:30:19 smithi191 ceph-mon[36816]: pgmap v340: 129 pgs: 129 active+clean; 641 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 426 B/s wr, 6 op/s 2024-03-04T09:30:19.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:30:19 smithi191 ceph-mon[36816]: from='client.15117 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-04T09:30:19.631 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:30:19 smithi106 ceph-mon[36484]: pgmap v340: 129 pgs: 129 active+clean; 641 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 426 B/s wr, 6 op/s 2024-03-04T09:30:19.631 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:30:19 smithi106 ceph-mon[36484]: from='client.15117 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-04T09:30:21.170 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:21.172+0000 7fa5c656e700 1 -- 172.21.15.52:0/2380731849 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa5c0107000 msgr2=0x7fa5c01073e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:30:21.171 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:21.172+0000 7fa5c656e700 1 --2- 172.21.15.52:0/2380731849 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa5c0107000 0x7fa5c01073e0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fa5b4009990 tx=0x7fa5b4009ca0 comp rx=0 tx=0).stop 2024-03-04T09:30:21.171 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:21.172+0000 7fa5c656e700 1 -- 172.21.15.52:0/2380731849 shutdown_connections 2024-03-04T09:30:21.171 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:21.172+0000 7fa5c656e700 1 --2- 172.21.15.52:0/2380731849 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa5c0101960 0x7fa5c010de40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:21.171 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:21.172+0000 7fa5c656e700 1 --2- 172.21.15.52:0/2380731849 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa5c0107000 0x7fa5c01073e0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:21.171 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:21.172+0000 7fa5c656e700 1 --2- 172.21.15.52:0/2380731849 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa5c0100fc0 0x7fa5c0101420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:21.171 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:21.172+0000 7fa5c656e700 1 -- 172.21.15.52:0/2380731849 >> 172.21.15.52:0/2380731849 conn(0x7fa5c0077ef0 msgr2=0x7fa5c0078300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:30:21.171 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:21.172+0000 7fa5c656e700 1 -- 172.21.15.52:0/2380731849 shutdown_connections 2024-03-04T09:30:21.171 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:21.172+0000 7fa5c656e700 1 -- 172.21.15.52:0/2380731849 wait complete. 2024-03-04T09:30:21.171 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:21.173+0000 7fa5c656e700 1 Processor -- start 2024-03-04T09:30:21.171 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:21.173+0000 7fa5c656e700 1 -- start start 2024-03-04T09:30:21.171 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:21.173+0000 7fa5c656e700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa5c0100fc0 0x7fa5c01976f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:30:21.171 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:21.173+0000 7fa5c656e700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa5c0101960 0x7fa5c0197c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:30:21.172 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:21.173+0000 7fa5c656e700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa5c0107000 0x7fa5c019bfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:30:21.172 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:21.173+0000 7fa5c656e700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5c0111240 con 0x7fa5c0101960 2024-03-04T09:30:21.172 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:21.173+0000 7fa5c656e700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5c01110c0 con 0x7fa5c0107000 2024-03-04T09:30:21.172 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:21.173+0000 7fa5c656e700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5c01113c0 con 0x7fa5c0100fc0 2024-03-04T09:30:21.172 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:21.173+0000 7fa5bf7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa5c0101960 0x7fa5c0197c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:30:21.172 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:21.174+0000 7fa5bf7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa5c0101960 0x7fa5c0197c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:47692/0 (socket says 172.21.15.52:47692) 2024-03-04T09:30:21.172 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:21.174+0000 7fa5bffff700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa5c0100fc0 0x7fa5c01976f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:30:21.172 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:21.174+0000 7fa5bf7fe700 1 -- 172.21.15.52:0/2673934301 learned_addr learned my addr 172.21.15.52:0/2673934301 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-04T09:30:21.172 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:21.174+0000 7fa5c4b0b700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa5c0107000 0x7fa5c019bfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:30:21.172 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:21.174+0000 7fa5bf7fe700 1 -- 172.21.15.52:0/2673934301 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa5c0100fc0 msgr2=0x7fa5c01976f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:30:21.172 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:21.174+0000 7fa5bf7fe700 1 --2- 172.21.15.52:0/2673934301 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa5c0100fc0 0x7fa5c01976f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:21.172 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:21.174+0000 7fa5bf7fe700 1 -- 172.21.15.52:0/2673934301 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa5c0107000 msgr2=0x7fa5c019bfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:30:21.172 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:21.174+0000 7fa5bf7fe700 1 --2- 172.21.15.52:0/2673934301 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa5c0107000 0x7fa5c019bfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:21.173 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:21.174+0000 7fa5bf7fe700 1 -- 172.21.15.52:0/2673934301 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5c019c700 con 0x7fa5c0101960 2024-03-04T09:30:21.174 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:21.174+0000 7fa5bf7fe700 1 --2- 172.21.15.52:0/2673934301 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa5c0101960 0x7fa5c0197c30 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fa5b000d980 tx=0x7fa5b000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:30:21.174 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:21.174+0000 7fa5bd7fa700 1 -- 172.21.15.52:0/2673934301 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5b0016070 con 0x7fa5c0101960 2024-03-04T09:30:21.174 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:21.174+0000 7fa5c656e700 1 -- 172.21.15.52:0/2673934301 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5c019c9f0 con 0x7fa5c0101960 2024-03-04T09:30:21.174 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:21.174+0000 7fa5c656e700 1 -- 172.21.15.52:0/2673934301 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5c01a4270 con 0x7fa5c0101960 2024-03-04T09:30:21.174 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:21.174+0000 7fa5bd7fa700 1 -- 172.21.15.52:0/2673934301 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa5b0004d10 con 0x7fa5c0101960 2024-03-04T09:30:21.174 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:21.174+0000 7fa5bd7fa700 1 -- 172.21.15.52:0/2673934301 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5b0005480 con 0x7fa5c0101960 2024-03-04T09:30:21.174 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:21.175+0000 7fa5c656e700 1 -- 172.21.15.52:0/2673934301 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa5ac005330 con 0x7fa5c0101960 2024-03-04T09:30:21.174 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:21.176+0000 7fa5bd7fa700 1 -- 172.21.15.52:0/2673934301 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) v1 ==== 136756+0+0 (secure 0 0 0) 0x7fa5b0004220 con 0x7fa5c0101960 2024-03-04T09:30:21.174 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:21.177+0000 7fa5bd7fa700 1 --2- 172.21.15.52:0/2673934301 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7fa5a80a1e10 0x7fa5a80a42d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:30:21.175 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:21.177+0000 7fa5bd7fa700 1 -- 172.21.15.52:0/2673934301 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9092+0+0 (secure 0 0 0) 0x7fa5b00cdec0 con 0x7fa5c0101960 2024-03-04T09:30:21.175 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:21.177+0000 7fa5bffff700 1 --2- 172.21.15.52:0/2673934301 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7fa5a80a1e10 0x7fa5a80a42d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:30:21.175 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:21.177+0000 7fa5bffff700 1 --2- 172.21.15.52:0/2673934301 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7fa5a80a1e10 0x7fa5a80a42d0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fa5b40098e0 tx=0x7fa5b4005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:30:21.178 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:21.181+0000 7fa5bd7fa700 1 -- 172.21.15.52:0/2673934301 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fa5b001f020 con 0x7fa5c0101960 2024-03-04T09:30:21.378 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:21.381+0000 7fa5c656e700 1 -- 172.21.15.52:0/2673934301 --> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fa5ac000bc0 con 0x7fa5a80a1e10 2024-03-04T09:30:21.388 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:21.391+0000 7fa5bd7fa700 1 -- 172.21.15.52:0/2673934301 <== mgr.14152 v2:172.21.15.52:6800/3036033923 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+340463 (secure 0 0 0) 0x7fa5ac000bc0 con 0x7fa5a80a1e10 2024-03-04T09:30:21.392 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-03-04T09:30:21.395 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:21.394+0000 7fa5c656e700 1 -- 172.21.15.52:0/2673934301 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7fa5a80a1e10 msgr2=0x7fa5a80a42d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:30:21.395 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:21.394+0000 7fa5c656e700 1 --2- 172.21.15.52:0/2673934301 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7fa5a80a1e10 0x7fa5a80a42d0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fa5b40098e0 tx=0x7fa5b4005960 comp rx=0 tx=0).stop 2024-03-04T09:30:21.395 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:21.394+0000 7fa5c656e700 1 -- 172.21.15.52:0/2673934301 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa5c0101960 msgr2=0x7fa5c0197c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:30:21.396 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:21.394+0000 7fa5c656e700 1 --2- 172.21.15.52:0/2673934301 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa5c0101960 0x7fa5c0197c30 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fa5b000d980 tx=0x7fa5b000dc90 comp rx=0 tx=0).stop 2024-03-04T09:30:21.396 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:21.394+0000 7fa5c656e700 1 -- 172.21.15.52:0/2673934301 shutdown_connections 2024-03-04T09:30:21.396 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:21.394+0000 7fa5c656e700 1 --2- 172.21.15.52:0/2673934301 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7fa5a80a1e10 0x7fa5a80a42d0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:21.396 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:21.394+0000 7fa5c656e700 1 --2- 172.21.15.52:0/2673934301 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa5c0100fc0 0x7fa5c01976f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:21.396 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:21.394+0000 7fa5c656e700 1 --2- 172.21.15.52:0/2673934301 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa5c0107000 0x7fa5c019bfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:21.396 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:21.394+0000 7fa5c656e700 1 --2- 172.21.15.52:0/2673934301 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa5c0101960 0x7fa5c0197c30 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:21.396 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:21.394+0000 7fa5c656e700 1 -- 172.21.15.52:0/2673934301 >> 172.21.15.52:0/2673934301 conn(0x7fa5c0077ef0 msgr2=0x7fa5c00ffcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:30:21.396 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:21.394+0000 7fa5c656e700 1 -- 172.21.15.52:0/2673934301 shutdown_connections 2024-03-04T09:30:21.396 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:21.394+0000 7fa5c656e700 1 -- 172.21.15.52:0/2673934301 wait complete. 2024-03-04T09:30:21.396 INFO:teuthology.orchestra.run.smithi052.stderr:dumped all 2024-03-04T09:30:21.585 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:30:21 smithi052 ceph-mon[32473]: pgmap v341: 129 pgs: 129 active+clean; 641 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 426 B/s wr, 5 op/s 2024-03-04T09:30:21.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:30:21 smithi191 ceph-mon[36816]: pgmap v341: 129 pgs: 129 active+clean; 641 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 426 B/s wr, 5 op/s 2024-03-04T09:30:21.631 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:30:21 smithi106 ceph-mon[36484]: pgmap v341: 129 pgs: 129 active+clean; 641 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 426 B/s wr, 5 op/s 2024-03-04T09:30:22.094 INFO:teuthology.orchestra.run.smithi052.stdout:{"pg_ready":true,"pg_map":{"version":341,"stamp":"2024-03-04T09:30:19.511625+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":656768,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":598,"num_read_kb":420,"num_write":349,"num_write_kb":3058,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":783,"ondisk_log_size":783,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":371556,"kb_used_data":14084,"kb_used_omap":18,"kb_used_meta":357357,"kb_avail":1124570268,"statfs":{"total":1151940427776,"available":1151559954432,"internally_reserved":0,"allocated":14422016,"data_stored":7956206,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19140,"internal_metadata":365933884},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":390,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":62,"num_read_kb":32,"num_write":7,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004423"},"pg_stats":[{"pgid":"3.3f","version":"91'8","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T09:30:09.527473+0000","last_change":"2024-03-04T09:28:37.914945+0000","last_active":"2024-03-04T09:30:09.527473+0000","last_peered":"2024-03-04T09:30:09.527473+0000","last_clean":"2024-03-04T09:30:09.527473+0000","last_became_active":"2024-03-04T09:28:37.914647+0000","last_became_peered":"2024-03-04T09:28:37.914647+0000","last_unstale":"2024-03-04T09:30:09.527473+0000","last_undegraded":"2024-03-04T09:30:09.527473+0000","last_fullsized":"2024-03-04T09:30:09.527473+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_clean_scrub_stamp":"2024-03-04T09:28:36.887656+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T13:46:41.262758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"91'4","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T09:30:09.520902+0000","last_change":"2024-03-04T09:28:37.918650+0000","last_active":"2024-03-04T09:30:09.520902+0000","last_peered":"2024-03-04T09:30:09.520902+0000","last_clean":"2024-03-04T09:30:09.520902+0000","last_became_active":"2024-03-04T09:28:37.918314+0000","last_became_peered":"2024-03-04T09:28:37.918314+0000","last_unstale":"2024-03-04T09:30:09.520902+0000","last_undegraded":"2024-03-04T09:30:09.520902+0000","last_fullsized":"2024-03-04T09:30:09.520902+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_clean_scrub_stamp":"2024-03-04T09:28:36.887656+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T18:36:41.948549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"91'7","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T09:30:09.527792+0000","last_change":"2024-03-04T09:28:37.920140+0000","last_active":"2024-03-04T09:30:09.527792+0000","last_peered":"2024-03-04T09:30:09.527792+0000","last_clean":"2024-03-04T09:30:09.527792+0000","last_became_active":"2024-03-04T09:28:37.917379+0000","last_became_peered":"2024-03-04T09:28:37.917379+0000","last_unstale":"2024-03-04T09:30:09.527792+0000","last_undegraded":"2024-03-04T09:30:09.527792+0000","last_fullsized":"2024-03-04T09:30:09.527792+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_clean_scrub_stamp":"2024-03-04T09:28:36.887656+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T17:23:33.144752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"91'8","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T09:30:09.511151+0000","last_change":"2024-03-04T09:28:37.916999+0000","last_active":"2024-03-04T09:30:09.511151+0000","last_peered":"2024-03-04T09:30:09.511151+0000","last_clean":"2024-03-04T09:30:09.511151+0000","last_became_active":"2024-03-04T09:28:37.916597+0000","last_became_peered":"2024-03-04T09:28:37.916597+0000","last_unstale":"2024-03-04T09:30:09.511151+0000","last_undegraded":"2024-03-04T09:30:09.511151+0000","last_fullsized":"2024-03-04T09:30:09.511151+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_clean_scrub_stamp":"2024-03-04T09:28:36.887656+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T19:23:20.556283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"91'4","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T09:30:09.505887+0000","last_change":"2024-03-04T09:28:37.904298+0000","last_active":"2024-03-04T09:30:09.505887+0000","last_peered":"2024-03-04T09:30:09.505887+0000","last_clean":"2024-03-04T09:30:09.505887+0000","last_became_active":"2024-03-04T09:28:37.903966+0000","last_became_peered":"2024-03-04T09:28:37.903966+0000","last_unstale":"2024-03-04T09:30:09.505887+0000","last_undegraded":"2024-03-04T09:30:09.505887+0000","last_fullsized":"2024-03-04T09:30:09.505887+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_clean_scrub_stamp":"2024-03-04T09:28:36.887656+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T20:14:21.615521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"91'6","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T09:30:09.526067+0000","last_change":"2024-03-04T09:28:37.917800+0000","last_active":"2024-03-04T09:30:09.526067+0000","last_peered":"2024-03-04T09:30:09.526067+0000","last_clean":"2024-03-04T09:30:09.526067+0000","last_became_active":"2024-03-04T09:28:37.917500+0000","last_became_peered":"2024-03-04T09:28:37.917500+0000","last_unstale":"2024-03-04T09:30:09.526067+0000","last_undegraded":"2024-03-04T09:30:09.526067+0000","last_fullsized":"2024-03-04T09:30:09.526067+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_clean_scrub_stamp":"2024-03-04T09:28:36.887656+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T20:06:31.330705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"91'5","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T09:30:09.502432+0000","last_change":"2024-03-04T09:28:37.917708+0000","last_active":"2024-03-04T09:30:09.502432+0000","last_peered":"2024-03-04T09:30:09.502432+0000","last_clean":"2024-03-04T09:30:09.502432+0000","last_became_active":"2024-03-04T09:28:37.917323+0000","last_became_peered":"2024-03-04T09:28:37.917323+0000","last_unstale":"2024-03-04T09:30:09.502432+0000","last_undegraded":"2024-03-04T09:30:09.502432+0000","last_fullsized":"2024-03-04T09:30:09.502432+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_clean_scrub_stamp":"2024-03-04T09:28:36.887656+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T11:10:46.345702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"91'9","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T09:30:09.524295+0000","last_change":"2024-03-04T09:29:41.411289+0000","last_active":"2024-03-04T09:30:09.524295+0000","last_peered":"2024-03-04T09:30:09.524295+0000","last_clean":"2024-03-04T09:30:09.524295+0000","last_became_active":"2024-03-04T09:29:41.410905+0000","last_became_peered":"2024-03-04T09:29:41.410905+0000","last_unstale":"2024-03-04T09:30:09.524295+0000","last_undegraded":"2024-03-04T09:30:09.524295+0000","last_fullsized":"2024-03-04T09:30:09.524295+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_clean_scrub_stamp":"2024-03-04T09:28:36.887656+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T17:55:14.687909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,10],"acting":[9,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.37","version":"91'11","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T09:30:09.529366+0000","last_change":"2024-03-04T09:28:37.928279+0000","last_active":"2024-03-04T09:30:09.529366+0000","last_peered":"2024-03-04T09:30:09.529366+0000","last_clean":"2024-03-04T09:30:09.529366+0000","last_became_active":"2024-03-04T09:28:37.927934+0000","last_became_peered":"2024-03-04T09:28:37.927934+0000","last_unstale":"2024-03-04T09:30:09.529366+0000","last_undegraded":"2024-03-04T09:30:09.529366+0000","last_fullsized":"2024-03-04T09:30:09.529366+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_clean_scrub_stamp":"2024-03-04T09:28:36.887656+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T21:23:06.592526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"91'16","reported_seq":41,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T09:30:09.525051+0000","last_change":"2024-03-04T09:28:37.918990+0000","last_active":"2024-03-04T09:30:09.525051+0000","last_peered":"2024-03-04T09:30:09.525051+0000","last_clean":"2024-03-04T09:30:09.525051+0000","last_became_active":"2024-03-04T09:28:37.918599+0000","last_became_peered":"2024-03-04T09:28:37.918599+0000","last_unstale":"2024-03-04T09:30:09.525051+0000","last_undegraded":"2024-03-04T09:30:09.525051+0000","last_fullsized":"2024-03-04T09:30:09.525051+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_clean_scrub_stamp":"2024-03-04T09:28:36.887656+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T20:39:09.603428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"91'6","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T09:30:09.517158+0000","last_change":"2024-03-04T09:28:37.914234+0000","last_active":"2024-03-04T09:30:09.517158+0000","last_peered":"2024-03-04T09:30:09.517158+0000","last_clean":"2024-03-04T09:30:09.517158+0000","last_became_active":"2024-03-04T09:28:37.913897+0000","last_became_peered":"2024-03-04T09:28:37.913897+0000","last_unstale":"2024-03-04T09:30:09.517158+0000","last_undegraded":"2024-03-04T09:30:09.517158+0000","last_fullsized":"2024-03-04T09:30:09.517158+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_clean_scrub_stamp":"2024-03-04T09:28:36.887656+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T15:43:53.634817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"91'7","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T09:30:09.509698+0000","last_change":"2024-03-04T09:28:37.919935+0000","last_active":"2024-03-04T09:30:09.509698+0000","last_peered":"2024-03-04T09:30:09.509698+0000","last_clean":"2024-03-04T09:30:09.509698+0000","last_became_active":"2024-03-04T09:28:37.919447+0000","last_became_peered":"2024-03-04T09:28:37.919447+0000","last_unstale":"2024-03-04T09:30:09.509698+0000","last_undegraded":"2024-03-04T09:30:09.509698+0000","last_fullsized":"2024-03-04T09:30:09.509698+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_clean_scrub_stamp":"2024-03-04T09:28:36.887656+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T09:44:03.678767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"91'10","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T09:30:09.522391+0000","last_change":"2024-03-04T09:28:37.900570+0000","last_active":"2024-03-04T09:30:09.522391+0000","last_peered":"2024-03-04T09:30:09.522391+0000","last_clean":"2024-03-04T09:30:09.522391+0000","last_became_active":"2024-03-04T09:28:37.900186+0000","last_became_peered":"2024-03-04T09:28:37.900186+0000","last_unstale":"2024-03-04T09:30:09.522391+0000","last_undegraded":"2024-03-04T09:30:09.522391+0000","last_fullsized":"2024-03-04T09:30:09.522391+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_clean_scrub_stamp":"2024-03-04T09:28:36.887656+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T21:17:12.915989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"91'7","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T09:30:09.525278+0000","last_change":"2024-03-04T09:28:37.920447+0000","last_active":"2024-03-04T09:30:09.525278+0000","last_peered":"2024-03-04T09:30:09.525278+0000","last_clean":"2024-03-04T09:30:09.525278+0000","last_became_active":"2024-03-04T09:28:37.919580+0000","last_became_peered":"2024-03-04T09:28:37.919580+0000","last_unstale":"2024-03-04T09:30:09.525278+0000","last_undegraded":"2024-03-04T09:30:09.525278+0000","last_fullsized":"2024-03-04T09:30:09.525278+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_clean_scrub_stamp":"2024-03-04T09:28:36.887656+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T13:25:56.434285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"91'7","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T09:30:09.522193+0000","last_change":"2024-03-04T09:28:37.913735+0000","last_active":"2024-03-04T09:30:09.522193+0000","last_peered":"2024-03-04T09:30:09.522193+0000","last_clean":"2024-03-04T09:30:09.522193+0000","last_became_active":"2024-03-04T09:28:37.913388+0000","last_became_peered":"2024-03-04T09:28:37.913388+0000","last_unstale":"2024-03-04T09:30:09.522193+0000","last_undegraded":"2024-03-04T09:30:09.522193+0000","last_fullsized":"2024-03-04T09:30:09.522193+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_clean_scrub_stamp":"2024-03-04T09:28:36.887656+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T16:04:28.468313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"91'9","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T09:30:09.528522+0000","last_change":"2024-03-04T09:28:37.925031+0000","last_active":"2024-03-04T09:30:09.528522+0000","last_peered":"2024-03-04T09:30:09.528522+0000","last_clean":"2024-03-04T09:30:09.528522+0000","last_became_active":"2024-03-04T09:28:37.923561+0000","last_became_peered":"2024-03-04T09:28:37.923561+0000","last_unstale":"2024-03-04T09:30:09.528522+0000","last_undegraded":"2024-03-04T09:30:09.528522+0000","last_fullsized":"2024-03-04T09:30:09.528522+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_clean_scrub_stamp":"2024-03-04T09:28:36.887656+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T16:29:04.266384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"91'8","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T09:30:09.527522+0000","last_change":"2024-03-04T09:28:37.912029+0000","last_active":"2024-03-04T09:30:09.527522+0000","last_peered":"2024-03-04T09:30:09.527522+0000","last_clean":"2024-03-04T09:30:09.527522+0000","last_became_active":"2024-03-04T09:28:37.911726+0000","last_became_peered":"2024-03-04T09:28:37.911726+0000","last_unstale":"2024-03-04T09:30:09.527522+0000","last_undegraded":"2024-03-04T09:30:09.527522+0000","last_fullsized":"2024-03-04T09:30:09.527522+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_clean_scrub_stamp":"2024-03-04T09:28:36.887656+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T12:32:03.385375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"91'10","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T09:30:09.526459+0000","last_change":"2024-03-04T09:28:37.914448+0000","last_active":"2024-03-04T09:30:09.526459+0000","last_peered":"2024-03-04T09:30:09.526459+0000","last_clean":"2024-03-04T09:30:09.526459+0000","last_became_active":"2024-03-04T09:28:37.914036+0000","last_became_peered":"2024-03-04T09:28:37.914036+0000","last_unstale":"2024-03-04T09:30:09.526459+0000","last_undegraded":"2024-03-04T09:30:09.526459+0000","last_fullsized":"2024-03-04T09:30:09.526459+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_clean_scrub_stamp":"2024-03-04T09:28:36.887656+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T20:57:10.348039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"91'7","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T09:30:09.517930+0000","last_change":"2024-03-04T09:28:37.907640+0000","last_active":"2024-03-04T09:30:09.517930+0000","last_peered":"2024-03-04T09:30:09.517930+0000","last_clean":"2024-03-04T09:30:09.517930+0000","last_became_active":"2024-03-04T09:28:37.907350+0000","last_became_peered":"2024-03-04T09:28:37.907350+0000","last_unstale":"2024-03-04T09:30:09.517930+0000","last_undegraded":"2024-03-04T09:30:09.517930+0000","last_fullsized":"2024-03-04T09:30:09.517930+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_clean_scrub_stamp":"2024-03-04T09:28:36.887656+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T10:17:37.576254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T09:29:41.061931+0000","last_change":"2024-03-04T09:28:41.950312+0000","last_active":"2024-03-04T09:29:41.061931+0000","last_peered":"2024-03-04T09:29:41.061931+0000","last_clean":"2024-03-04T09:29:41.061931+0000","last_became_active":"2024-03-04T09:28:41.949882+0000","last_became_peered":"2024-03-04T09:28:41.949882+0000","last_unstale":"2024-03-04T09:29:41.061931+0000","last_undegraded":"2024-03-04T09:29:41.061931+0000","last_fullsized":"2024-03-04T09:29:41.061931+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:40.917115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:40.917115+0000","last_clean_scrub_stamp":"2024-03-04T09:28:40.917115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T14:20:48.740518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"89'9","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T09:29:41.062915+0000","last_change":"2024-03-04T09:28:36.904805+0000","last_active":"2024-03-04T09:29:41.062915+0000","last_peered":"2024-03-04T09:29:41.062915+0000","last_clean":"2024-03-04T09:29:41.062915+0000","last_became_active":"2024-03-04T09:28:36.904469+0000","last_became_peered":"2024-03-04T09:28:36.904469+0000","last_unstale":"2024-03-04T09:29:41.062915+0000","last_undegraded":"2024-03-04T09:29:41.062915+0000","last_fullsized":"2024-03-04T09:29:41.062915+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:35.885776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:35.885776+0000","last_clean_scrub_stamp":"2024-03-04T09:28:35.885776+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T18:35:35.992042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"91'5","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T09:30:09.513701+0000","last_change":"2024-03-04T09:28:37.920102+0000","last_active":"2024-03-04T09:30:09.513701+0000","last_peered":"2024-03-04T09:30:09.513701+0000","last_clean":"2024-03-04T09:30:09.513701+0000","last_became_active":"2024-03-04T09:28:37.919749+0000","last_became_peered":"2024-03-04T09:28:37.919749+0000","last_unstale":"2024-03-04T09:30:09.513701+0000","last_undegraded":"2024-03-04T09:30:09.513701+0000","last_fullsized":"2024-03-04T09:30:09.513701+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_clean_scrub_stamp":"2024-03-04T09:28:36.887656+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T19:16:36.269782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T09:29:41.061792+0000","last_change":"2024-03-04T09:28:41.950477+0000","last_active":"2024-03-04T09:29:41.061792+0000","last_peered":"2024-03-04T09:29:41.061792+0000","last_clean":"2024-03-04T09:29:41.061792+0000","last_became_active":"2024-03-04T09:28:41.950087+0000","last_became_peered":"2024-03-04T09:28:41.950087+0000","last_unstale":"2024-03-04T09:29:41.061792+0000","last_undegraded":"2024-03-04T09:29:41.061792+0000","last_fullsized":"2024-03-04T09:29:41.061792+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:40.917115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:40.917115+0000","last_clean_scrub_stamp":"2024-03-04T09:28:40.917115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T14:48:17.327956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"89'2","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T09:29:41.071382+0000","last_change":"2024-03-04T09:28:36.913923+0000","last_active":"2024-03-04T09:29:41.071382+0000","last_peered":"2024-03-04T09:29:41.071382+0000","last_clean":"2024-03-04T09:29:41.071382+0000","last_became_active":"2024-03-04T09:28:36.911076+0000","last_became_peered":"2024-03-04T09:28:36.911076+0000","last_unstale":"2024-03-04T09:29:41.071382+0000","last_undegraded":"2024-03-04T09:29:41.071382+0000","last_fullsized":"2024-03-04T09:29:41.071382+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:35.885776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:35.885776+0000","last_clean_scrub_stamp":"2024-03-04T09:28:35.885776+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T16:13:23.004827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"91'6","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T09:30:09.527767+0000","last_change":"2024-03-04T09:28:37.912747+0000","last_active":"2024-03-04T09:30:09.527767+0000","last_peered":"2024-03-04T09:30:09.527767+0000","last_clean":"2024-03-04T09:30:09.527767+0000","last_became_active":"2024-03-04T09:28:37.912436+0000","last_became_peered":"2024-03-04T09:28:37.912436+0000","last_unstale":"2024-03-04T09:30:09.527767+0000","last_undegraded":"2024-03-04T09:30:09.527767+0000","last_fullsized":"2024-03-04T09:30:09.527767+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_clean_scrub_stamp":"2024-03-04T09:28:36.887656+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T14:52:47.318456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T09:29:41.411324+0000","last_change":"2024-03-04T09:29:41.411324+0000","last_active":"2024-03-04T09:29:41.411324+0000","last_peered":"2024-03-04T09:29:41.411324+0000","last_clean":"2024-03-04T09:29:41.411324+0000","last_became_active":"2024-03-04T09:29:41.410822+0000","last_became_peered":"2024-03-04T09:29:41.410822+0000","last_unstale":"2024-03-04T09:29:41.411324+0000","last_undegraded":"2024-03-04T09:29:41.411324+0000","last_fullsized":"2024-03-04T09:29:41.411324+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:40.917115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:40.917115+0000","last_clean_scrub_stamp":"2024-03-04T09:28:40.917115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T14:07:20.646108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,10,5],"acting":[4,3,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"89'4","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T09:29:41.066136+0000","last_change":"2024-03-04T09:28:36.910279+0000","last_active":"2024-03-04T09:29:41.066136+0000","last_peered":"2024-03-04T09:29:41.066136+0000","last_clean":"2024-03-04T09:29:41.066136+0000","last_became_active":"2024-03-04T09:28:36.909214+0000","last_became_peered":"2024-03-04T09:28:36.909214+0000","last_unstale":"2024-03-04T09:29:41.066136+0000","last_undegraded":"2024-03-04T09:29:41.066136+0000","last_fullsized":"2024-03-04T09:29:41.066136+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:35.885776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:35.885776+0000","last_clean_scrub_stamp":"2024-03-04T09:28:35.885776+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T19:22:45.808418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"91'9","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T09:30:09.528763+0000","last_change":"2024-03-04T09:28:37.914593+0000","last_active":"2024-03-04T09:30:09.528763+0000","last_peered":"2024-03-04T09:30:09.528763+0000","last_clean":"2024-03-04T09:30:09.528763+0000","last_became_active":"2024-03-04T09:28:37.914217+0000","last_became_peered":"2024-03-04T09:28:37.914217+0000","last_unstale":"2024-03-04T09:30:09.528763+0000","last_undegraded":"2024-03-04T09:30:09.528763+0000","last_fullsized":"2024-03-04T09:30:09.528763+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_clean_scrub_stamp":"2024-03-04T09:28:36.887656+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T19:32:39.893906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T09:29:41.061839+0000","last_change":"2024-03-04T09:28:41.951685+0000","last_active":"2024-03-04T09:29:41.061839+0000","last_peered":"2024-03-04T09:29:41.061839+0000","last_clean":"2024-03-04T09:29:41.061839+0000","last_became_active":"2024-03-04T09:28:41.951178+0000","last_became_peered":"2024-03-04T09:28:41.951178+0000","last_unstale":"2024-03-04T09:29:41.061839+0000","last_undegraded":"2024-03-04T09:29:41.061839+0000","last_fullsized":"2024-03-04T09:29:41.061839+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:40.917115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:40.917115+0000","last_clean_scrub_stamp":"2024-03-04T09:28:40.917115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T10:23:19.028344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"84'2","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T09:29:41.065632+0000","last_change":"2024-03-04T09:28:36.908066+0000","last_active":"2024-03-04T09:29:41.065632+0000","last_peered":"2024-03-04T09:29:41.065632+0000","last_clean":"2024-03-04T09:29:41.065632+0000","last_became_active":"2024-03-04T09:28:36.907761+0000","last_became_peered":"2024-03-04T09:28:36.907761+0000","last_unstale":"2024-03-04T09:29:41.065632+0000","last_undegraded":"2024-03-04T09:29:41.065632+0000","last_fullsized":"2024-03-04T09:29:41.065632+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:35.885776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:35.885776+0000","last_clean_scrub_stamp":"2024-03-04T09:28:35.885776+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T20:53:11.559719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"91'15","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T09:30:09.513186+0000","last_change":"2024-03-04T09:28:37.917667+0000","last_active":"2024-03-04T09:30:09.513186+0000","last_peered":"2024-03-04T09:30:09.513186+0000","last_clean":"2024-03-04T09:30:09.513186+0000","last_became_active":"2024-03-04T09:28:37.917320+0000","last_became_peered":"2024-03-04T09:28:37.917320+0000","last_unstale":"2024-03-04T09:30:09.513186+0000","last_undegraded":"2024-03-04T09:30:09.513186+0000","last_fullsized":"2024-03-04T09:30:09.513186+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_clean_scrub_stamp":"2024-03-04T09:28:36.887656+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T12:17:07.162334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T09:29:41.062750+0000","last_change":"2024-03-04T09:28:41.946078+0000","last_active":"2024-03-04T09:29:41.062750+0000","last_peered":"2024-03-04T09:29:41.062750+0000","last_clean":"2024-03-04T09:29:41.062750+0000","last_became_active":"2024-03-04T09:28:41.945639+0000","last_became_peered":"2024-03-04T09:28:41.945639+0000","last_unstale":"2024-03-04T09:29:41.062750+0000","last_undegraded":"2024-03-04T09:29:41.062750+0000","last_fullsized":"2024-03-04T09:29:41.062750+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:40.917115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:40.917115+0000","last_clean_scrub_stamp":"2024-03-04T09:28:40.917115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T16:31:41.939728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"89'5","reported_seq":107,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T09:30:15.085510+0000","last_change":"2024-03-04T09:28:36.905596+0000","last_active":"2024-03-04T09:30:15.085510+0000","last_peered":"2024-03-04T09:30:15.085510+0000","last_clean":"2024-03-04T09:30:15.085510+0000","last_became_active":"2024-03-04T09:28:36.905270+0000","last_became_peered":"2024-03-04T09:28:36.905270+0000","last_unstale":"2024-03-04T09:30:15.085510+0000","last_undegraded":"2024-03-04T09:30:15.085510+0000","last_fullsized":"2024-03-04T09:30:15.085510+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:35.885776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:35.885776+0000","last_clean_scrub_stamp":"2024-03-04T09:28:35.885776+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T09:43:32.986704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":75,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"91'7","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T09:30:09.519426+0000","last_change":"2024-03-04T09:28:37.906282+0000","last_active":"2024-03-04T09:30:09.519426+0000","last_peered":"2024-03-04T09:30:09.519426+0000","last_clean":"2024-03-04T09:30:09.519426+0000","last_became_active":"2024-03-04T09:28:37.905964+0000","last_became_peered":"2024-03-04T09:28:37.905964+0000","last_unstale":"2024-03-04T09:30:09.519426+0000","last_undegraded":"2024-03-04T09:30:09.519426+0000","last_fullsized":"2024-03-04T09:30:09.519426+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_clean_scrub_stamp":"2024-03-04T09:28:36.887656+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T11:13:16.287959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T09:29:41.062153+0000","last_change":"2024-03-04T09:28:41.949196+0000","last_active":"2024-03-04T09:29:41.062153+0000","last_peered":"2024-03-04T09:29:41.062153+0000","last_clean":"2024-03-04T09:29:41.062153+0000","last_became_active":"2024-03-04T09:28:41.948783+0000","last_became_peered":"2024-03-04T09:28:41.948783+0000","last_unstale":"2024-03-04T09:29:41.062153+0000","last_undegraded":"2024-03-04T09:29:41.062153+0000","last_fullsized":"2024-03-04T09:29:41.062153+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:40.917115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:40.917115+0000","last_clean_scrub_stamp":"2024-03-04T09:28:40.917115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T13:42:05.888482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"89'3","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T09:29:41.168040+0000","last_change":"2024-03-04T09:28:36.906900+0000","last_active":"2024-03-04T09:29:41.168040+0000","last_peered":"2024-03-04T09:29:41.168040+0000","last_clean":"2024-03-04T09:29:41.168040+0000","last_became_active":"2024-03-04T09:28:36.906556+0000","last_became_peered":"2024-03-04T09:28:36.906556+0000","last_unstale":"2024-03-04T09:29:41.168040+0000","last_undegraded":"2024-03-04T09:29:41.168040+0000","last_fullsized":"2024-03-04T09:29:41.168040+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:35.885776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:35.885776+0000","last_clean_scrub_stamp":"2024-03-04T09:28:35.885776+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T18:13:56.535621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"91'8","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T09:30:09.511567+0000","last_change":"2024-03-04T09:28:37.912123+0000","last_active":"2024-03-04T09:30:09.511567+0000","last_peered":"2024-03-04T09:30:09.511567+0000","last_clean":"2024-03-04T09:30:09.511567+0000","last_became_active":"2024-03-04T09:28:37.911810+0000","last_became_peered":"2024-03-04T09:28:37.911810+0000","last_unstale":"2024-03-04T09:30:09.511567+0000","last_undegraded":"2024-03-04T09:30:09.511567+0000","last_fullsized":"2024-03-04T09:30:09.511567+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_clean_scrub_stamp":"2024-03-04T09:28:36.887656+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T18:14:25.229989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T09:29:41.072635+0000","last_change":"2024-03-04T09:29:41.072635+0000","last_active":"2024-03-04T09:29:41.072635+0000","last_peered":"2024-03-04T09:29:41.072635+0000","last_clean":"2024-03-04T09:29:41.072635+0000","last_became_active":"2024-03-04T09:29:41.072213+0000","last_became_peered":"2024-03-04T09:29:41.072213+0000","last_unstale":"2024-03-04T09:29:41.072635+0000","last_undegraded":"2024-03-04T09:29:41.072635+0000","last_fullsized":"2024-03-04T09:29:41.072635+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:40.917115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:40.917115+0000","last_clean_scrub_stamp":"2024-03-04T09:28:40.917115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T13:09:43.707806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8,0],"acting":[11,4,8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"89'2","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T09:29:41.068264+0000","last_change":"2024-03-04T09:28:36.902206+0000","last_active":"2024-03-04T09:29:41.068264+0000","last_peered":"2024-03-04T09:29:41.068264+0000","last_clean":"2024-03-04T09:29:41.068264+0000","last_became_active":"2024-03-04T09:28:36.901847+0000","last_became_peered":"2024-03-04T09:28:36.901847+0000","last_unstale":"2024-03-04T09:29:41.068264+0000","last_undegraded":"2024-03-04T09:29:41.068264+0000","last_fullsized":"2024-03-04T09:29:41.068264+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:35.885776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:35.885776+0000","last_clean_scrub_stamp":"2024-03-04T09:28:35.885776+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T12:32:05.755876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"91'8","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T09:30:09.517891+0000","last_change":"2024-03-04T09:28:37.906175+0000","last_active":"2024-03-04T09:30:09.517891+0000","last_peered":"2024-03-04T09:30:09.517891+0000","last_clean":"2024-03-04T09:30:09.517891+0000","last_became_active":"2024-03-04T09:28:37.905845+0000","last_became_peered":"2024-03-04T09:28:37.905845+0000","last_unstale":"2024-03-04T09:30:09.517891+0000","last_undegraded":"2024-03-04T09:30:09.517891+0000","last_fullsized":"2024-03-04T09:30:09.517891+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_clean_scrub_stamp":"2024-03-04T09:28:36.887656+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T11:50:23.569240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T09:29:41.065675+0000","last_change":"2024-03-04T09:28:41.949290+0000","last_active":"2024-03-04T09:29:41.065675+0000","last_peered":"2024-03-04T09:29:41.065675+0000","last_clean":"2024-03-04T09:29:41.065675+0000","last_became_active":"2024-03-04T09:28:41.948866+0000","last_became_peered":"2024-03-04T09:28:41.948866+0000","last_unstale":"2024-03-04T09:29:41.065675+0000","last_undegraded":"2024-03-04T09:29:41.065675+0000","last_fullsized":"2024-03-04T09:29:41.065675+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:40.917115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:40.917115+0000","last_clean_scrub_stamp":"2024-03-04T09:28:40.917115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T15:30:11.980139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"89'1","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T09:29:41.410078+0000","last_change":"2024-03-04T09:28:36.910071+0000","last_active":"2024-03-04T09:29:41.410078+0000","last_peered":"2024-03-04T09:29:41.410078+0000","last_clean":"2024-03-04T09:29:41.410078+0000","last_became_active":"2024-03-04T09:28:36.908025+0000","last_became_peered":"2024-03-04T09:28:36.908025+0000","last_unstale":"2024-03-04T09:29:41.410078+0000","last_undegraded":"2024-03-04T09:29:41.410078+0000","last_fullsized":"2024-03-04T09:29:41.410078+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:35.885776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:35.885776+0000","last_clean_scrub_stamp":"2024-03-04T09:28:35.885776+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T10:45:42.310473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"91'11","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T09:30:09.526063+0000","last_change":"2024-03-04T09:28:37.915792+0000","last_active":"2024-03-04T09:30:09.526063+0000","last_peered":"2024-03-04T09:30:09.526063+0000","last_clean":"2024-03-04T09:30:09.526063+0000","last_became_active":"2024-03-04T09:28:37.914914+0000","last_became_peered":"2024-03-04T09:28:37.914914+0000","last_unstale":"2024-03-04T09:30:09.526063+0000","last_undegraded":"2024-03-04T09:30:09.526063+0000","last_fullsized":"2024-03-04T09:30:09.526063+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_clean_scrub_stamp":"2024-03-04T09:28:36.887656+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T14:13:57.828616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T09:29:41.070690+0000","last_change":"2024-03-04T09:28:41.947909+0000","last_active":"2024-03-04T09:29:41.070690+0000","last_peered":"2024-03-04T09:29:41.070690+0000","last_clean":"2024-03-04T09:29:41.070690+0000","last_became_active":"2024-03-04T09:28:41.947553+0000","last_became_peered":"2024-03-04T09:28:41.947553+0000","last_unstale":"2024-03-04T09:29:41.070690+0000","last_undegraded":"2024-03-04T09:29:41.070690+0000","last_fullsized":"2024-03-04T09:29:41.070690+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:40.917115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:40.917115+0000","last_clean_scrub_stamp":"2024-03-04T09:28:40.917115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T14:19:52.302321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"89'3","reported_seq":102,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T09:30:14.083953+0000","last_change":"2024-03-04T09:28:36.911823+0000","last_active":"2024-03-04T09:30:14.083953+0000","last_peered":"2024-03-04T09:30:14.083953+0000","last_clean":"2024-03-04T09:30:14.083953+0000","last_became_active":"2024-03-04T09:28:36.911502+0000","last_became_peered":"2024-03-04T09:28:36.911502+0000","last_unstale":"2024-03-04T09:30:14.083953+0000","last_undegraded":"2024-03-04T09:30:14.083953+0000","last_fullsized":"2024-03-04T09:30:14.083953+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:35.885776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:35.885776+0000","last_clean_scrub_stamp":"2024-03-04T09:28:35.885776+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T15:31:04.806604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":72,"num_read_kb":72,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"91'10","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T09:30:09.528752+0000","last_change":"2024-03-04T09:28:37.914093+0000","last_active":"2024-03-04T09:30:09.528752+0000","last_peered":"2024-03-04T09:30:09.528752+0000","last_clean":"2024-03-04T09:30:09.528752+0000","last_became_active":"2024-03-04T09:28:37.913661+0000","last_became_peered":"2024-03-04T09:28:37.913661+0000","last_unstale":"2024-03-04T09:30:09.528752+0000","last_undegraded":"2024-03-04T09:30:09.528752+0000","last_fullsized":"2024-03-04T09:30:09.528752+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_clean_scrub_stamp":"2024-03-04T09:28:36.887656+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T18:17:42.826342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T09:29:41.061387+0000","last_change":"2024-03-04T09:28:41.947371+0000","last_active":"2024-03-04T09:29:41.061387+0000","last_peered":"2024-03-04T09:29:41.061387+0000","last_clean":"2024-03-04T09:29:41.061387+0000","last_became_active":"2024-03-04T09:28:41.946944+0000","last_became_peered":"2024-03-04T09:28:41.946944+0000","last_unstale":"2024-03-04T09:29:41.061387+0000","last_undegraded":"2024-03-04T09:29:41.061387+0000","last_fullsized":"2024-03-04T09:29:41.061387+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:40.917115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:40.917115+0000","last_clean_scrub_stamp":"2024-03-04T09:28:40.917115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T11:01:34.348927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"89'2","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T09:29:41.064563+0000","last_change":"2024-03-04T09:28:36.907927+0000","last_active":"2024-03-04T09:29:41.064563+0000","last_peered":"2024-03-04T09:29:41.064563+0000","last_clean":"2024-03-04T09:29:41.064563+0000","last_became_active":"2024-03-04T09:28:36.907507+0000","last_became_peered":"2024-03-04T09:28:36.907507+0000","last_unstale":"2024-03-04T09:29:41.064563+0000","last_undegraded":"2024-03-04T09:29:41.064563+0000","last_fullsized":"2024-03-04T09:29:41.064563+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:35.885776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:35.885776+0000","last_clean_scrub_stamp":"2024-03-04T09:28:35.885776+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T21:11:24.942564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"91'9","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T09:30:09.522913+0000","last_change":"2024-03-04T09:28:37.913816+0000","last_active":"2024-03-04T09:30:09.522913+0000","last_peered":"2024-03-04T09:30:09.522913+0000","last_clean":"2024-03-04T09:30:09.522913+0000","last_became_active":"2024-03-04T09:28:37.913481+0000","last_became_peered":"2024-03-04T09:28:37.913481+0000","last_unstale":"2024-03-04T09:30:09.522913+0000","last_undegraded":"2024-03-04T09:30:09.522913+0000","last_fullsized":"2024-03-04T09:30:09.522913+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_clean_scrub_stamp":"2024-03-04T09:28:36.887656+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T14:33:07.371224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T09:29:41.062269+0000","last_change":"2024-03-04T09:28:41.948963+0000","last_active":"2024-03-04T09:29:41.062269+0000","last_peered":"2024-03-04T09:29:41.062269+0000","last_clean":"2024-03-04T09:29:41.062269+0000","last_became_active":"2024-03-04T09:28:41.948469+0000","last_became_peered":"2024-03-04T09:28:41.948469+0000","last_unstale":"2024-03-04T09:29:41.062269+0000","last_undegraded":"2024-03-04T09:29:41.062269+0000","last_fullsized":"2024-03-04T09:29:41.062269+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:40.917115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:40.917115+0000","last_clean_scrub_stamp":"2024-03-04T09:28:40.917115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T21:26:52.238364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"89'4","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T09:29:41.409973+0000","last_change":"2024-03-04T09:28:36.913162+0000","last_active":"2024-03-04T09:29:41.409973+0000","last_peered":"2024-03-04T09:29:41.409973+0000","last_clean":"2024-03-04T09:29:41.409973+0000","last_became_active":"2024-03-04T09:28:36.912834+0000","last_became_peered":"2024-03-04T09:28:36.912834+0000","last_unstale":"2024-03-04T09:29:41.409973+0000","last_undegraded":"2024-03-04T09:29:41.409973+0000","last_fullsized":"2024-03-04T09:29:41.409973+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:35.885776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:35.885776+0000","last_clean_scrub_stamp":"2024-03-04T09:28:35.885776+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T17:59:07.210048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"91'11","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T09:30:09.528998+0000","last_change":"2024-03-04T09:28:37.914801+0000","last_active":"2024-03-04T09:30:09.528998+0000","last_peered":"2024-03-04T09:30:09.528998+0000","last_clean":"2024-03-04T09:30:09.528998+0000","last_became_active":"2024-03-04T09:28:37.914517+0000","last_became_peered":"2024-03-04T09:28:37.914517+0000","last_unstale":"2024-03-04T09:30:09.528998+0000","last_undegraded":"2024-03-04T09:30:09.528998+0000","last_fullsized":"2024-03-04T09:30:09.528998+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_clean_scrub_stamp":"2024-03-04T09:28:36.887656+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T18:15:50.967364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T09:29:41.061526+0000","last_change":"2024-03-04T09:28:41.957821+0000","last_active":"2024-03-04T09:29:41.061526+0000","last_peered":"2024-03-04T09:29:41.061526+0000","last_clean":"2024-03-04T09:29:41.061526+0000","last_became_active":"2024-03-04T09:28:41.957392+0000","last_became_peered":"2024-03-04T09:28:41.957392+0000","last_unstale":"2024-03-04T09:29:41.061526+0000","last_undegraded":"2024-03-04T09:29:41.061526+0000","last_fullsized":"2024-03-04T09:29:41.061526+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:40.917115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:40.917115+0000","last_clean_scrub_stamp":"2024-03-04T09:28:40.917115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T19:44:24.294299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"91'34","reported_seq":213,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T09:30:15.084974+0000","last_change":"2024-03-04T09:28:36.909243+0000","last_active":"2024-03-04T09:30:15.084974+0000","last_peered":"2024-03-04T09:30:15.084974+0000","last_clean":"2024-03-04T09:30:15.084974+0000","last_became_active":"2024-03-04T09:28:36.908941+0000","last_became_peered":"2024-03-04T09:28:36.908941+0000","last_unstale":"2024-03-04T09:30:15.084974+0000","last_undegraded":"2024-03-04T09:30:15.084974+0000","last_fullsized":"2024-03-04T09:30:15.084974+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:35.885776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:35.885776+0000","last_clean_scrub_stamp":"2024-03-04T09:28:35.885776+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T16:46:26.014209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":62024,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":152,"num_read_kb":137,"num_write":35,"num_write_kb":77,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"91'6","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T09:30:09.510134+0000","last_change":"2024-03-04T09:28:37.910465+0000","last_active":"2024-03-04T09:30:09.510134+0000","last_peered":"2024-03-04T09:30:09.510134+0000","last_clean":"2024-03-04T09:30:09.510134+0000","last_became_active":"2024-03-04T09:28:37.910129+0000","last_became_peered":"2024-03-04T09:28:37.910129+0000","last_unstale":"2024-03-04T09:30:09.510134+0000","last_undegraded":"2024-03-04T09:30:09.510134+0000","last_fullsized":"2024-03-04T09:30:09.510134+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_clean_scrub_stamp":"2024-03-04T09:28:36.887656+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T21:25:48.272477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T09:29:41.410494+0000","last_change":"2024-03-04T09:28:41.948465+0000","last_active":"2024-03-04T09:29:41.410494+0000","last_peered":"2024-03-04T09:29:41.410494+0000","last_clean":"2024-03-04T09:29:41.410494+0000","last_became_active":"2024-03-04T09:28:41.946995+0000","last_became_peered":"2024-03-04T09:28:41.946995+0000","last_unstale":"2024-03-04T09:29:41.410494+0000","last_undegraded":"2024-03-04T09:29:41.410494+0000","last_fullsized":"2024-03-04T09:29:41.410494+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:40.917115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:40.917115+0000","last_clean_scrub_stamp":"2024-03-04T09:28:40.917115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T12:35:10.579279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"89'6","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T09:29:41.062009+0000","last_change":"2024-03-04T09:28:36.905641+0000","last_active":"2024-03-04T09:29:41.062009+0000","last_peered":"2024-03-04T09:29:41.062009+0000","last_clean":"2024-03-04T09:29:41.062009+0000","last_became_active":"2024-03-04T09:28:36.905309+0000","last_became_peered":"2024-03-04T09:28:36.905309+0000","last_unstale":"2024-03-04T09:29:41.062009+0000","last_undegraded":"2024-03-04T09:29:41.062009+0000","last_fullsized":"2024-03-04T09:29:41.062009+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:35.885776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:35.885776+0000","last_clean_scrub_stamp":"2024-03-04T09:28:35.885776+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T12:08:40.200654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"91'9","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T09:30:09.519509+0000","last_change":"2024-03-04T09:28:37.907744+0000","last_active":"2024-03-04T09:30:09.519509+0000","last_peered":"2024-03-04T09:30:09.519509+0000","last_clean":"2024-03-04T09:30:09.519509+0000","last_became_active":"2024-03-04T09:28:37.907393+0000","last_became_peered":"2024-03-04T09:28:37.907393+0000","last_unstale":"2024-03-04T09:30:09.519509+0000","last_undegraded":"2024-03-04T09:30:09.519509+0000","last_fullsized":"2024-03-04T09:30:09.519509+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_clean_scrub_stamp":"2024-03-04T09:28:36.887656+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T20:17:11.822846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T09:29:41.063487+0000","last_change":"2024-03-04T09:28:41.949824+0000","last_active":"2024-03-04T09:29:41.063487+0000","last_peered":"2024-03-04T09:29:41.063487+0000","last_clean":"2024-03-04T09:29:41.063487+0000","last_became_active":"2024-03-04T09:28:41.949252+0000","last_became_peered":"2024-03-04T09:28:41.949252+0000","last_unstale":"2024-03-04T09:29:41.063487+0000","last_undegraded":"2024-03-04T09:29:41.063487+0000","last_fullsized":"2024-03-04T09:29:41.063487+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:40.917115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:40.917115+0000","last_clean_scrub_stamp":"2024-03-04T09:28:40.917115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T21:22:19.045780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"89'3","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T09:29:41.409875+0000","last_change":"2024-03-04T09:28:36.908718+0000","last_active":"2024-03-04T09:29:41.409875+0000","last_peered":"2024-03-04T09:29:41.409875+0000","last_clean":"2024-03-04T09:29:41.409875+0000","last_became_active":"2024-03-04T09:28:36.908102+0000","last_became_peered":"2024-03-04T09:28:36.908102+0000","last_unstale":"2024-03-04T09:29:41.409875+0000","last_undegraded":"2024-03-04T09:29:41.409875+0000","last_fullsized":"2024-03-04T09:29:41.409875+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:35.885776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:35.885776+0000","last_clean_scrub_stamp":"2024-03-04T09:28:35.885776+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T20:53:39.603281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"91'15","reported_seq":46,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T09:30:09.518318+0000","last_change":"2024-03-04T09:29:41.068617+0000","last_active":"2024-03-04T09:30:09.518318+0000","last_peered":"2024-03-04T09:30:09.518318+0000","last_clean":"2024-03-04T09:30:09.518318+0000","last_became_active":"2024-03-04T09:29:41.068231+0000","last_became_peered":"2024-03-04T09:29:41.068231+0000","last_unstale":"2024-03-04T09:30:09.518318+0000","last_undegraded":"2024-03-04T09:30:09.518318+0000","last_fullsized":"2024-03-04T09:30:09.518318+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_clean_scrub_stamp":"2024-03-04T09:28:36.887656+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T12:40:15.456293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T09:29:41.072598+0000","last_change":"2024-03-04T09:29:41.072598+0000","last_active":"2024-03-04T09:29:41.072598+0000","last_peered":"2024-03-04T09:29:41.072598+0000","last_clean":"2024-03-04T09:29:41.072598+0000","last_became_active":"2024-03-04T09:29:41.072095+0000","last_became_peered":"2024-03-04T09:29:41.072095+0000","last_unstale":"2024-03-04T09:29:41.072598+0000","last_undegraded":"2024-03-04T09:29:41.072598+0000","last_fullsized":"2024-03-04T09:29:41.072598+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:40.917115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:40.917115+0000","last_clean_scrub_stamp":"2024-03-04T09:28:40.917115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T14:24:55.509528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,7,0,3],"acting":[8,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.11","version":"91'5","reported_seq":109,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T09:30:15.094268+0000","last_change":"2024-03-04T09:28:36.905097+0000","last_active":"2024-03-04T09:30:15.094268+0000","last_peered":"2024-03-04T09:30:15.094268+0000","last_clean":"2024-03-04T09:30:15.094268+0000","last_became_active":"2024-03-04T09:28:36.904709+0000","last_became_peered":"2024-03-04T09:28:36.904709+0000","last_unstale":"2024-03-04T09:30:15.094268+0000","last_undegraded":"2024-03-04T09:30:15.094268+0000","last_fullsized":"2024-03-04T09:30:15.094268+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:35.885776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:35.885776+0000","last_clean_scrub_stamp":"2024-03-04T09:28:35.885776+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T13:58:23.977258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1120,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":78,"num_read_kb":5,"num_write":4,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"91'6","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T09:30:09.505609+0000","last_change":"2024-03-04T09:28:37.918023+0000","last_active":"2024-03-04T09:30:09.505609+0000","last_peered":"2024-03-04T09:30:09.505609+0000","last_clean":"2024-03-04T09:30:09.505609+0000","last_became_active":"2024-03-04T09:28:37.917671+0000","last_became_peered":"2024-03-04T09:28:37.917671+0000","last_unstale":"2024-03-04T09:30:09.505609+0000","last_undegraded":"2024-03-04T09:30:09.505609+0000","last_fullsized":"2024-03-04T09:30:09.505609+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_clean_scrub_stamp":"2024-03-04T09:28:36.887656+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T09:51:15.618328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T09:29:41.067989+0000","last_change":"2024-03-04T09:28:41.958903+0000","last_active":"2024-03-04T09:29:41.067989+0000","last_peered":"2024-03-04T09:29:41.067989+0000","last_clean":"2024-03-04T09:29:41.067989+0000","last_became_active":"2024-03-04T09:28:41.958256+0000","last_became_peered":"2024-03-04T09:28:41.958256+0000","last_unstale":"2024-03-04T09:29:41.067989+0000","last_undegraded":"2024-03-04T09:29:41.067989+0000","last_fullsized":"2024-03-04T09:29:41.067989+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:40.917115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:40.917115+0000","last_clean_scrub_stamp":"2024-03-04T09:28:40.917115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T18:49:48.022650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"89'15","reported_seq":43,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T09:29:41.065368+0000","last_change":"2024-03-04T09:28:36.911924+0000","last_active":"2024-03-04T09:29:41.065368+0000","last_peered":"2024-03-04T09:29:41.065368+0000","last_clean":"2024-03-04T09:29:41.065368+0000","last_became_active":"2024-03-04T09:28:36.911623+0000","last_became_peered":"2024-03-04T09:28:36.911623+0000","last_unstale":"2024-03-04T09:29:41.065368+0000","last_undegraded":"2024-03-04T09:29:41.065368+0000","last_fullsized":"2024-03-04T09:29:41.065368+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:35.885776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:35.885776+0000","last_clean_scrub_stamp":"2024-03-04T09:28:35.885776+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T13:17:32.606767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"91'6","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T09:30:09.509820+0000","last_change":"2024-03-04T09:28:37.916578+0000","last_active":"2024-03-04T09:30:09.509820+0000","last_peered":"2024-03-04T09:30:09.509820+0000","last_clean":"2024-03-04T09:30:09.509820+0000","last_became_active":"2024-03-04T09:28:37.913830+0000","last_became_peered":"2024-03-04T09:28:37.913830+0000","last_unstale":"2024-03-04T09:30:09.509820+0000","last_undegraded":"2024-03-04T09:30:09.509820+0000","last_fullsized":"2024-03-04T09:30:09.509820+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_clean_scrub_stamp":"2024-03-04T09:28:36.887656+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T09:29:15.882496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T09:29:41.411504+0000","last_change":"2024-03-04T09:29:41.411504+0000","last_active":"2024-03-04T09:29:41.411504+0000","last_peered":"2024-03-04T09:29:41.411504+0000","last_clean":"2024-03-04T09:29:41.411504+0000","last_became_active":"2024-03-04T09:29:41.411024+0000","last_became_peered":"2024-03-04T09:29:41.411024+0000","last_unstale":"2024-03-04T09:29:41.411504+0000","last_undegraded":"2024-03-04T09:29:41.411504+0000","last_fullsized":"2024-03-04T09:29:41.411504+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:40.917115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:40.917115+0000","last_clean_scrub_stamp":"2024-03-04T09:28:40.917115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T18:10:56.888602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,3,1],"acting":[8,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.f","version":"89'2","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T09:29:41.066076+0000","last_change":"2024-03-04T09:28:36.909779+0000","last_active":"2024-03-04T09:29:41.066076+0000","last_peered":"2024-03-04T09:29:41.066076+0000","last_clean":"2024-03-04T09:29:41.066076+0000","last_became_active":"2024-03-04T09:28:36.909466+0000","last_became_peered":"2024-03-04T09:28:36.909466+0000","last_unstale":"2024-03-04T09:29:41.066076+0000","last_undegraded":"2024-03-04T09:29:41.066076+0000","last_fullsized":"2024-03-04T09:29:41.066076+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:35.885776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:35.885776+0000","last_clean_scrub_stamp":"2024-03-04T09:28:35.885776+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T17:16:28.183292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"91'5","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T09:30:09.514599+0000","last_change":"2024-03-04T09:28:37.916371+0000","last_active":"2024-03-04T09:30:09.514599+0000","last_peered":"2024-03-04T09:30:09.514599+0000","last_clean":"2024-03-04T09:30:09.514599+0000","last_became_active":"2024-03-04T09:28:37.916068+0000","last_became_peered":"2024-03-04T09:28:37.916068+0000","last_unstale":"2024-03-04T09:30:09.514599+0000","last_undegraded":"2024-03-04T09:30:09.514599+0000","last_fullsized":"2024-03-04T09:30:09.514599+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_clean_scrub_stamp":"2024-03-04T09:28:36.887656+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T13:18:18.850165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T09:29:41.065688+0000","last_change":"2024-03-04T09:28:41.955301+0000","last_active":"2024-03-04T09:29:41.065688+0000","last_peered":"2024-03-04T09:29:41.065688+0000","last_clean":"2024-03-04T09:29:41.065688+0000","last_became_active":"2024-03-04T09:28:41.954774+0000","last_became_peered":"2024-03-04T09:28:41.954774+0000","last_unstale":"2024-03-04T09:29:41.065688+0000","last_undegraded":"2024-03-04T09:29:41.065688+0000","last_fullsized":"2024-03-04T09:29:41.065688+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:40.917115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:40.917115+0000","last_clean_scrub_stamp":"2024-03-04T09:28:40.917115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T16:56:55.463344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T09:29:41.062163+0000","last_change":"2024-03-04T09:28:36.901659+0000","last_active":"2024-03-04T09:29:41.062163+0000","last_peered":"2024-03-04T09:29:41.062163+0000","last_clean":"2024-03-04T09:29:41.062163+0000","last_became_active":"2024-03-04T09:28:36.901235+0000","last_became_peered":"2024-03-04T09:28:36.901235+0000","last_unstale":"2024-03-04T09:29:41.062163+0000","last_undegraded":"2024-03-04T09:29:41.062163+0000","last_fullsized":"2024-03-04T09:29:41.062163+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:35.885776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:35.885776+0000","last_clean_scrub_stamp":"2024-03-04T09:28:35.885776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T12:59:50.237172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"91'4","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T09:30:09.524380+0000","last_change":"2024-03-04T09:28:37.907905+0000","last_active":"2024-03-04T09:30:09.524380+0000","last_peered":"2024-03-04T09:30:09.524380+0000","last_clean":"2024-03-04T09:30:09.524380+0000","last_became_active":"2024-03-04T09:28:37.907376+0000","last_became_peered":"2024-03-04T09:28:37.907376+0000","last_unstale":"2024-03-04T09:30:09.524380+0000","last_undegraded":"2024-03-04T09:30:09.524380+0000","last_fullsized":"2024-03-04T09:30:09.524380+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_clean_scrub_stamp":"2024-03-04T09:28:36.887656+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T14:25:31.784659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T09:29:41.065377+0000","last_change":"2024-03-04T09:28:41.947629+0000","last_active":"2024-03-04T09:29:41.065377+0000","last_peered":"2024-03-04T09:29:41.065377+0000","last_clean":"2024-03-04T09:29:41.065377+0000","last_became_active":"2024-03-04T09:28:41.946251+0000","last_became_peered":"2024-03-04T09:28:41.946251+0000","last_unstale":"2024-03-04T09:29:41.065377+0000","last_undegraded":"2024-03-04T09:29:41.065377+0000","last_fullsized":"2024-03-04T09:29:41.065377+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:40.917115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:40.917115+0000","last_clean_scrub_stamp":"2024-03-04T09:28:40.917115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T18:26:57.770657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"89'1","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T09:29:41.064265+0000","last_change":"2024-03-04T09:28:36.905323+0000","last_active":"2024-03-04T09:29:41.064265+0000","last_peered":"2024-03-04T09:29:41.064265+0000","last_clean":"2024-03-04T09:29:41.064265+0000","last_became_active":"2024-03-04T09:28:36.904517+0000","last_became_peered":"2024-03-04T09:28:36.904517+0000","last_unstale":"2024-03-04T09:29:41.064265+0000","last_undegraded":"2024-03-04T09:29:41.064265+0000","last_fullsized":"2024-03-04T09:29:41.064265+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:35.885776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:35.885776+0000","last_clean_scrub_stamp":"2024-03-04T09:28:35.885776+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T18:52:11.615107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"91'7","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T09:30:09.524345+0000","last_change":"2024-03-04T09:28:37.921000+0000","last_active":"2024-03-04T09:30:09.524345+0000","last_peered":"2024-03-04T09:30:09.524345+0000","last_clean":"2024-03-04T09:30:09.524345+0000","last_became_active":"2024-03-04T09:28:37.920620+0000","last_became_peered":"2024-03-04T09:28:37.920620+0000","last_unstale":"2024-03-04T09:30:09.524345+0000","last_undegraded":"2024-03-04T09:30:09.524345+0000","last_fullsized":"2024-03-04T09:30:09.524345+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_clean_scrub_stamp":"2024-03-04T09:28:36.887656+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T10:41:29.187701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T09:29:41.066953+0000","last_change":"2024-03-04T09:28:41.949132+0000","last_active":"2024-03-04T09:29:41.066953+0000","last_peered":"2024-03-04T09:29:41.066953+0000","last_clean":"2024-03-04T09:29:41.066953+0000","last_became_active":"2024-03-04T09:28:41.948618+0000","last_became_peered":"2024-03-04T09:28:41.948618+0000","last_unstale":"2024-03-04T09:29:41.066953+0000","last_undegraded":"2024-03-04T09:29:41.066953+0000","last_fullsized":"2024-03-04T09:29:41.066953+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:40.917115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:40.917115+0000","last_clean_scrub_stamp":"2024-03-04T09:28:40.917115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T19:45:45.001345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"91'8","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T09:30:09.513356+0000","last_change":"2024-03-04T09:28:37.910475+0000","last_active":"2024-03-04T09:30:09.513356+0000","last_peered":"2024-03-04T09:30:09.513356+0000","last_clean":"2024-03-04T09:30:09.513356+0000","last_became_active":"2024-03-04T09:28:37.910184+0000","last_became_peered":"2024-03-04T09:28:37.910184+0000","last_unstale":"2024-03-04T09:30:09.513356+0000","last_undegraded":"2024-03-04T09:30:09.513356+0000","last_fullsized":"2024-03-04T09:30:09.513356+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_clean_scrub_stamp":"2024-03-04T09:28:36.887656+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T10:16:56.244331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"91'5","reported_seq":104,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T09:30:14.092706+0000","last_change":"2024-03-04T09:28:36.913489+0000","last_active":"2024-03-04T09:30:14.092706+0000","last_peered":"2024-03-04T09:30:14.092706+0000","last_clean":"2024-03-04T09:30:14.092706+0000","last_became_active":"2024-03-04T09:28:36.913162+0000","last_became_peered":"2024-03-04T09:28:36.913162+0000","last_unstale":"2024-03-04T09:30:14.092706+0000","last_undegraded":"2024-03-04T09:30:14.092706+0000","last_fullsized":"2024-03-04T09:30:14.092706+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:35.885776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:35.885776+0000","last_clean_scrub_stamp":"2024-03-04T09:28:35.885776+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T12:20:35.870058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":72,"num_read_kb":72,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"89'1","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T09:29:41.064510+0000","last_change":"2024-03-04T09:28:41.945717+0000","last_active":"2024-03-04T09:29:41.064510+0000","last_peered":"2024-03-04T09:29:41.064510+0000","last_clean":"2024-03-04T09:29:41.064510+0000","last_became_active":"2024-03-04T09:28:41.945300+0000","last_became_peered":"2024-03-04T09:28:41.945300+0000","last_unstale":"2024-03-04T09:29:41.064510+0000","last_undegraded":"2024-03-04T09:29:41.064510+0000","last_fullsized":"2024-03-04T09:29:41.064510+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:40.917115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:40.917115+0000","last_clean_scrub_stamp":"2024-03-04T09:28:40.917115+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T11:05:30.407232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"91'8","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T09:30:09.506071+0000","last_change":"2024-03-04T09:28:37.921030+0000","last_active":"2024-03-04T09:30:09.506071+0000","last_peered":"2024-03-04T09:30:09.506071+0000","last_clean":"2024-03-04T09:30:09.506071+0000","last_became_active":"2024-03-04T09:28:37.920655+0000","last_became_peered":"2024-03-04T09:28:37.920655+0000","last_unstale":"2024-03-04T09:30:09.506071+0000","last_undegraded":"2024-03-04T09:30:09.506071+0000","last_fullsized":"2024-03-04T09:30:09.506071+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_clean_scrub_stamp":"2024-03-04T09:28:36.887656+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T19:37:34.063523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"89'3","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T09:29:41.062153+0000","last_change":"2024-03-04T09:28:36.902902+0000","last_active":"2024-03-04T09:29:41.062153+0000","last_peered":"2024-03-04T09:29:41.062153+0000","last_clean":"2024-03-04T09:29:41.062153+0000","last_became_active":"2024-03-04T09:28:36.902574+0000","last_became_peered":"2024-03-04T09:28:36.902574+0000","last_unstale":"2024-03-04T09:29:41.062153+0000","last_undegraded":"2024-03-04T09:29:41.062153+0000","last_fullsized":"2024-03-04T09:29:41.062153+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:35.885776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:35.885776+0000","last_clean_scrub_stamp":"2024-03-04T09:28:35.885776+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T14:11:17.544497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":362,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T09:29:41.064047+0000","last_change":"2024-03-04T09:27:28.478476+0000","last_active":"2024-03-04T09:29:41.064047+0000","last_peered":"2024-03-04T09:29:41.064047+0000","last_clean":"2024-03-04T09:29:41.064047+0000","last_became_active":"2024-03-04T09:27:28.478041+0000","last_became_peered":"2024-03-04T09:27:28.478041+0000","last_unstale":"2024-03-04T09:29:41.064047+0000","last_undegraded":"2024-03-04T09:29:41.064047+0000","last_fullsized":"2024-03-04T09:29:41.064047+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:24:01.742551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:24:01.742551+0000","last_clean_scrub_stamp":"2024-03-04T09:24:01.742551+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T12:51:11.585296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T09:29:41.409966+0000","last_change":"2024-03-04T09:28:41.958895+0000","last_active":"2024-03-04T09:29:41.409966+0000","last_peered":"2024-03-04T09:29:41.409966+0000","last_clean":"2024-03-04T09:29:41.409966+0000","last_became_active":"2024-03-04T09:28:41.958477+0000","last_became_peered":"2024-03-04T09:28:41.958477+0000","last_unstale":"2024-03-04T09:29:41.409966+0000","last_undegraded":"2024-03-04T09:29:41.409966+0000","last_fullsized":"2024-03-04T09:29:41.409966+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:40.917115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:40.917115+0000","last_clean_scrub_stamp":"2024-03-04T09:28:40.917115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T17:48:37.149539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"91'5","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T09:30:09.519445+0000","last_change":"2024-03-04T09:28:37.911872+0000","last_active":"2024-03-04T09:30:09.519445+0000","last_peered":"2024-03-04T09:30:09.519445+0000","last_clean":"2024-03-04T09:30:09.519445+0000","last_became_active":"2024-03-04T09:28:37.911583+0000","last_became_peered":"2024-03-04T09:28:37.911583+0000","last_unstale":"2024-03-04T09:30:09.519445+0000","last_undegraded":"2024-03-04T09:30:09.519445+0000","last_fullsized":"2024-03-04T09:30:09.519445+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_clean_scrub_stamp":"2024-03-04T09:28:36.887656+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T20:12:15.606046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"89'3","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T09:29:41.064006+0000","last_change":"2024-03-04T09:28:36.905156+0000","last_active":"2024-03-04T09:29:41.064006+0000","last_peered":"2024-03-04T09:29:41.064006+0000","last_clean":"2024-03-04T09:29:41.064006+0000","last_became_active":"2024-03-04T09:28:36.904702+0000","last_became_peered":"2024-03-04T09:28:36.904702+0000","last_unstale":"2024-03-04T09:29:41.064006+0000","last_undegraded":"2024-03-04T09:29:41.064006+0000","last_fullsized":"2024-03-04T09:29:41.064006+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:35.885776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:35.885776+0000","last_clean_scrub_stamp":"2024-03-04T09:28:35.885776+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T19:38:16.485245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T09:29:41.063757+0000","last_change":"2024-03-04T09:28:41.954087+0000","last_active":"2024-03-04T09:29:41.063757+0000","last_peered":"2024-03-04T09:29:41.063757+0000","last_clean":"2024-03-04T09:29:41.063757+0000","last_became_active":"2024-03-04T09:28:41.953681+0000","last_became_peered":"2024-03-04T09:28:41.953681+0000","last_unstale":"2024-03-04T09:29:41.063757+0000","last_undegraded":"2024-03-04T09:29:41.063757+0000","last_fullsized":"2024-03-04T09:29:41.063757+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:40.917115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:40.917115+0000","last_clean_scrub_stamp":"2024-03-04T09:28:40.917115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T19:45:18.525709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"89'1","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T09:29:41.066030+0000","last_change":"2024-03-04T09:28:36.912400+0000","last_active":"2024-03-04T09:29:41.066030+0000","last_peered":"2024-03-04T09:29:41.066030+0000","last_clean":"2024-03-04T09:29:41.066030+0000","last_became_active":"2024-03-04T09:28:36.912096+0000","last_became_peered":"2024-03-04T09:28:36.912096+0000","last_unstale":"2024-03-04T09:29:41.066030+0000","last_undegraded":"2024-03-04T09:29:41.066030+0000","last_fullsized":"2024-03-04T09:29:41.066030+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:35.885776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:35.885776+0000","last_clean_scrub_stamp":"2024-03-04T09:28:35.885776+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T16:45:54.750427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"91'9","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T09:30:09.519400+0000","last_change":"2024-03-04T09:28:37.902851+0000","last_active":"2024-03-04T09:30:09.519400+0000","last_peered":"2024-03-04T09:30:09.519400+0000","last_clean":"2024-03-04T09:30:09.519400+0000","last_became_active":"2024-03-04T09:28:37.902537+0000","last_became_peered":"2024-03-04T09:28:37.902537+0000","last_unstale":"2024-03-04T09:30:09.519400+0000","last_undegraded":"2024-03-04T09:30:09.519400+0000","last_fullsized":"2024-03-04T09:30:09.519400+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_clean_scrub_stamp":"2024-03-04T09:28:36.887656+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T14:30:50.350161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T09:29:41.070473+0000","last_change":"2024-03-04T09:28:41.948551+0000","last_active":"2024-03-04T09:29:41.070473+0000","last_peered":"2024-03-04T09:29:41.070473+0000","last_clean":"2024-03-04T09:29:41.070473+0000","last_became_active":"2024-03-04T09:28:41.946636+0000","last_became_peered":"2024-03-04T09:28:41.946636+0000","last_unstale":"2024-03-04T09:29:41.070473+0000","last_undegraded":"2024-03-04T09:29:41.070473+0000","last_fullsized":"2024-03-04T09:29:41.070473+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:40.917115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:40.917115+0000","last_clean_scrub_stamp":"2024-03-04T09:28:40.917115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T14:00:07.624296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"91'9","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T09:29:41.062700+0000","last_change":"2024-03-04T09:28:36.910605+0000","last_active":"2024-03-04T09:29:41.062700+0000","last_peered":"2024-03-04T09:29:41.062700+0000","last_clean":"2024-03-04T09:29:41.062700+0000","last_became_active":"2024-03-04T09:28:36.910253+0000","last_became_peered":"2024-03-04T09:28:36.910253+0000","last_unstale":"2024-03-04T09:29:41.062700+0000","last_undegraded":"2024-03-04T09:29:41.062700+0000","last_fullsized":"2024-03-04T09:29:41.062700+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:35.885776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:35.885776+0000","last_clean_scrub_stamp":"2024-03-04T09:28:35.885776+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T18:08:30.941310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"91'5","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T09:30:09.515703+0000","last_change":"2024-03-04T09:28:37.907342+0000","last_active":"2024-03-04T09:30:09.515703+0000","last_peered":"2024-03-04T09:30:09.515703+0000","last_clean":"2024-03-04T09:30:09.515703+0000","last_became_active":"2024-03-04T09:28:37.906974+0000","last_became_peered":"2024-03-04T09:28:37.906974+0000","last_unstale":"2024-03-04T09:30:09.515703+0000","last_undegraded":"2024-03-04T09:30:09.515703+0000","last_fullsized":"2024-03-04T09:30:09.515703+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_clean_scrub_stamp":"2024-03-04T09:28:36.887656+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T11:18:43.099544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T09:29:41.061765+0000","last_change":"2024-03-04T09:28:41.945713+0000","last_active":"2024-03-04T09:29:41.061765+0000","last_peered":"2024-03-04T09:29:41.061765+0000","last_clean":"2024-03-04T09:29:41.061765+0000","last_became_active":"2024-03-04T09:28:41.945245+0000","last_became_peered":"2024-03-04T09:28:41.945245+0000","last_unstale":"2024-03-04T09:29:41.061765+0000","last_undegraded":"2024-03-04T09:29:41.061765+0000","last_fullsized":"2024-03-04T09:29:41.061765+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:40.917115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:40.917115+0000","last_clean_scrub_stamp":"2024-03-04T09:28:40.917115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T11:06:39.595269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"89'4","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T09:29:41.062592+0000","last_change":"2024-03-04T09:28:36.902020+0000","last_active":"2024-03-04T09:29:41.062592+0000","last_peered":"2024-03-04T09:29:41.062592+0000","last_clean":"2024-03-04T09:29:41.062592+0000","last_became_active":"2024-03-04T09:28:36.901666+0000","last_became_peered":"2024-03-04T09:28:36.901666+0000","last_unstale":"2024-03-04T09:29:41.062592+0000","last_undegraded":"2024-03-04T09:29:41.062592+0000","last_fullsized":"2024-03-04T09:29:41.062592+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:35.885776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:35.885776+0000","last_clean_scrub_stamp":"2024-03-04T09:28:35.885776+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T12:48:17.270200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"91'7","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T09:30:09.525463+0000","last_change":"2024-03-04T09:28:37.912889+0000","last_active":"2024-03-04T09:30:09.525463+0000","last_peered":"2024-03-04T09:30:09.525463+0000","last_clean":"2024-03-04T09:30:09.525463+0000","last_became_active":"2024-03-04T09:28:37.912542+0000","last_became_peered":"2024-03-04T09:28:37.912542+0000","last_unstale":"2024-03-04T09:30:09.525463+0000","last_undegraded":"2024-03-04T09:30:09.525463+0000","last_fullsized":"2024-03-04T09:30:09.525463+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_clean_scrub_stamp":"2024-03-04T09:28:36.887656+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T20:51:12.540364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T09:29:41.062161+0000","last_change":"2024-03-04T09:28:41.949009+0000","last_active":"2024-03-04T09:29:41.062161+0000","last_peered":"2024-03-04T09:29:41.062161+0000","last_clean":"2024-03-04T09:29:41.062161+0000","last_became_active":"2024-03-04T09:28:41.948178+0000","last_became_peered":"2024-03-04T09:28:41.948178+0000","last_unstale":"2024-03-04T09:29:41.062161+0000","last_undegraded":"2024-03-04T09:29:41.062161+0000","last_fullsized":"2024-03-04T09:29:41.062161+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:40.917115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:40.917115+0000","last_clean_scrub_stamp":"2024-03-04T09:28:40.917115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T10:54:54.360229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"89'2","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T09:29:41.409452+0000","last_change":"2024-03-04T09:28:36.901847+0000","last_active":"2024-03-04T09:29:41.409452+0000","last_peered":"2024-03-04T09:29:41.409452+0000","last_clean":"2024-03-04T09:29:41.409452+0000","last_became_active":"2024-03-04T09:28:36.901461+0000","last_became_peered":"2024-03-04T09:28:36.901461+0000","last_unstale":"2024-03-04T09:29:41.409452+0000","last_undegraded":"2024-03-04T09:29:41.409452+0000","last_fullsized":"2024-03-04T09:29:41.409452+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:35.885776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:35.885776+0000","last_clean_scrub_stamp":"2024-03-04T09:28:35.885776+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T14:36:33.041447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"91'2","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T09:30:09.475901+0000","last_change":"2024-03-04T09:28:37.907958+0000","last_active":"2024-03-04T09:30:09.475901+0000","last_peered":"2024-03-04T09:30:09.475901+0000","last_clean":"2024-03-04T09:30:09.475901+0000","last_became_active":"2024-03-04T09:28:37.907653+0000","last_became_peered":"2024-03-04T09:28:37.907653+0000","last_unstale":"2024-03-04T09:30:09.475901+0000","last_undegraded":"2024-03-04T09:30:09.475901+0000","last_fullsized":"2024-03-04T09:30:09.475901+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_clean_scrub_stamp":"2024-03-04T09:28:36.887656+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T20:42:44.266275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T09:29:41.062815+0000","last_change":"2024-03-04T09:28:41.957479+0000","last_active":"2024-03-04T09:29:41.062815+0000","last_peered":"2024-03-04T09:29:41.062815+0000","last_clean":"2024-03-04T09:29:41.062815+0000","last_became_active":"2024-03-04T09:28:41.954950+0000","last_became_peered":"2024-03-04T09:28:41.954950+0000","last_unstale":"2024-03-04T09:29:41.062815+0000","last_undegraded":"2024-03-04T09:29:41.062815+0000","last_fullsized":"2024-03-04T09:29:41.062815+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:40.917115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:40.917115+0000","last_clean_scrub_stamp":"2024-03-04T09:28:40.917115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T14:53:16.636230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"89'3","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T09:29:41.070521+0000","last_change":"2024-03-04T09:28:36.908649+0000","last_active":"2024-03-04T09:29:41.070521+0000","last_peered":"2024-03-04T09:29:41.070521+0000","last_clean":"2024-03-04T09:29:41.070521+0000","last_became_active":"2024-03-04T09:28:36.908323+0000","last_became_peered":"2024-03-04T09:28:36.908323+0000","last_unstale":"2024-03-04T09:29:41.070521+0000","last_undegraded":"2024-03-04T09:29:41.070521+0000","last_fullsized":"2024-03-04T09:29:41.070521+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:35.885776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:35.885776+0000","last_clean_scrub_stamp":"2024-03-04T09:28:35.885776+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T16:49:40.367952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"91'9","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T09:30:09.523113+0000","last_change":"2024-03-04T09:28:37.905673+0000","last_active":"2024-03-04T09:30:09.523113+0000","last_peered":"2024-03-04T09:30:09.523113+0000","last_clean":"2024-03-04T09:30:09.523113+0000","last_became_active":"2024-03-04T09:28:37.905368+0000","last_became_peered":"2024-03-04T09:28:37.905368+0000","last_unstale":"2024-03-04T09:30:09.523113+0000","last_undegraded":"2024-03-04T09:30:09.523113+0000","last_fullsized":"2024-03-04T09:30:09.523113+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_clean_scrub_stamp":"2024-03-04T09:28:36.887656+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T17:25:35.058092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"89'1","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T09:29:41.061920+0000","last_change":"2024-03-04T09:28:41.948688+0000","last_active":"2024-03-04T09:29:41.061920+0000","last_peered":"2024-03-04T09:29:41.061920+0000","last_clean":"2024-03-04T09:29:41.061920+0000","last_became_active":"2024-03-04T09:28:41.948111+0000","last_became_peered":"2024-03-04T09:28:41.948111+0000","last_unstale":"2024-03-04T09:29:41.061920+0000","last_undegraded":"2024-03-04T09:29:41.061920+0000","last_fullsized":"2024-03-04T09:29:41.061920+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:40.917115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:40.917115+0000","last_clean_scrub_stamp":"2024-03-04T09:28:40.917115+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T12:27:37.121328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"89'1","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T09:29:41.061844+0000","last_change":"2024-03-04T09:28:36.908438+0000","last_active":"2024-03-04T09:29:41.061844+0000","last_peered":"2024-03-04T09:29:41.061844+0000","last_clean":"2024-03-04T09:29:41.061844+0000","last_became_active":"2024-03-04T09:28:36.906529+0000","last_became_peered":"2024-03-04T09:28:36.906529+0000","last_unstale":"2024-03-04T09:29:41.061844+0000","last_undegraded":"2024-03-04T09:29:41.061844+0000","last_fullsized":"2024-03-04T09:29:41.061844+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:35.885776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:35.885776+0000","last_clean_scrub_stamp":"2024-03-04T09:28:35.885776+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T10:12:54.833556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"91'8","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T09:30:09.510926+0000","last_change":"2024-03-04T09:28:37.927048+0000","last_active":"2024-03-04T09:30:09.510926+0000","last_peered":"2024-03-04T09:30:09.510926+0000","last_clean":"2024-03-04T09:30:09.510926+0000","last_became_active":"2024-03-04T09:28:37.925381+0000","last_became_peered":"2024-03-04T09:28:37.925381+0000","last_unstale":"2024-03-04T09:30:09.510926+0000","last_undegraded":"2024-03-04T09:30:09.510926+0000","last_fullsized":"2024-03-04T09:30:09.510926+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_clean_scrub_stamp":"2024-03-04T09:28:36.887656+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T20:19:48.554102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T09:29:41.062103+0000","last_change":"2024-03-04T09:28:41.945805+0000","last_active":"2024-03-04T09:29:41.062103+0000","last_peered":"2024-03-04T09:29:41.062103+0000","last_clean":"2024-03-04T09:29:41.062103+0000","last_became_active":"2024-03-04T09:28:41.945414+0000","last_became_peered":"2024-03-04T09:28:41.945414+0000","last_unstale":"2024-03-04T09:29:41.062103+0000","last_undegraded":"2024-03-04T09:29:41.062103+0000","last_fullsized":"2024-03-04T09:29:41.062103+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:40.917115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:40.917115+0000","last_clean_scrub_stamp":"2024-03-04T09:28:40.917115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T09:45:59.612507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"89'4","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T09:29:41.065464+0000","last_change":"2024-03-04T09:28:36.909808+0000","last_active":"2024-03-04T09:29:41.065464+0000","last_peered":"2024-03-04T09:29:41.065464+0000","last_clean":"2024-03-04T09:29:41.065464+0000","last_became_active":"2024-03-04T09:28:36.909486+0000","last_became_peered":"2024-03-04T09:28:36.909486+0000","last_unstale":"2024-03-04T09:29:41.065464+0000","last_undegraded":"2024-03-04T09:29:41.065464+0000","last_fullsized":"2024-03-04T09:29:41.065464+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:35.885776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:35.885776+0000","last_clean_scrub_stamp":"2024-03-04T09:28:35.885776+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T19:38:15.524941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"91'5","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T09:30:09.529632+0000","last_change":"2024-03-04T09:28:37.910807+0000","last_active":"2024-03-04T09:30:09.529632+0000","last_peered":"2024-03-04T09:30:09.529632+0000","last_clean":"2024-03-04T09:30:09.529632+0000","last_became_active":"2024-03-04T09:28:37.910504+0000","last_became_peered":"2024-03-04T09:28:37.910504+0000","last_unstale":"2024-03-04T09:30:09.529632+0000","last_undegraded":"2024-03-04T09:30:09.529632+0000","last_fullsized":"2024-03-04T09:30:09.529632+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_clean_scrub_stamp":"2024-03-04T09:28:36.887656+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T20:03:35.484747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T09:29:41.409454+0000","last_change":"2024-03-04T09:28:41.949798+0000","last_active":"2024-03-04T09:29:41.409454+0000","last_peered":"2024-03-04T09:29:41.409454+0000","last_clean":"2024-03-04T09:29:41.409454+0000","last_became_active":"2024-03-04T09:28:41.949358+0000","last_became_peered":"2024-03-04T09:28:41.949358+0000","last_unstale":"2024-03-04T09:29:41.409454+0000","last_undegraded":"2024-03-04T09:29:41.409454+0000","last_fullsized":"2024-03-04T09:29:41.409454+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:40.917115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:40.917115+0000","last_clean_scrub_stamp":"2024-03-04T09:28:40.917115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T13:30:26.519090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"89'3","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T09:29:41.062398+0000","last_change":"2024-03-04T09:28:36.912458+0000","last_active":"2024-03-04T09:29:41.062398+0000","last_peered":"2024-03-04T09:29:41.062398+0000","last_clean":"2024-03-04T09:29:41.062398+0000","last_became_active":"2024-03-04T09:28:36.912132+0000","last_became_peered":"2024-03-04T09:28:36.912132+0000","last_unstale":"2024-03-04T09:29:41.062398+0000","last_undegraded":"2024-03-04T09:29:41.062398+0000","last_fullsized":"2024-03-04T09:29:41.062398+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:35.885776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:35.885776+0000","last_clean_scrub_stamp":"2024-03-04T09:28:35.885776+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T13:51:29.713381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"91'7","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T09:30:09.517387+0000","last_change":"2024-03-04T09:28:37.915750+0000","last_active":"2024-03-04T09:30:09.517387+0000","last_peered":"2024-03-04T09:30:09.517387+0000","last_clean":"2024-03-04T09:30:09.517387+0000","last_became_active":"2024-03-04T09:28:37.915424+0000","last_became_peered":"2024-03-04T09:28:37.915424+0000","last_unstale":"2024-03-04T09:30:09.517387+0000","last_undegraded":"2024-03-04T09:30:09.517387+0000","last_fullsized":"2024-03-04T09:30:09.517387+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_clean_scrub_stamp":"2024-03-04T09:28:36.887656+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T19:15:39.555122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T09:29:41.066935+0000","last_change":"2024-03-04T09:28:41.949321+0000","last_active":"2024-03-04T09:29:41.066935+0000","last_peered":"2024-03-04T09:29:41.066935+0000","last_clean":"2024-03-04T09:29:41.066935+0000","last_became_active":"2024-03-04T09:28:41.948302+0000","last_became_peered":"2024-03-04T09:28:41.948302+0000","last_unstale":"2024-03-04T09:29:41.066935+0000","last_undegraded":"2024-03-04T09:29:41.066935+0000","last_fullsized":"2024-03-04T09:29:41.066935+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:40.917115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:40.917115+0000","last_clean_scrub_stamp":"2024-03-04T09:28:40.917115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T10:18:44.988651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"84'1","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T09:29:41.410258+0000","last_change":"2024-03-04T09:28:36.909585+0000","last_active":"2024-03-04T09:29:41.410258+0000","last_peered":"2024-03-04T09:29:41.410258+0000","last_clean":"2024-03-04T09:29:41.410258+0000","last_became_active":"2024-03-04T09:28:36.909286+0000","last_became_peered":"2024-03-04T09:28:36.909286+0000","last_unstale":"2024-03-04T09:29:41.410258+0000","last_undegraded":"2024-03-04T09:29:41.410258+0000","last_fullsized":"2024-03-04T09:29:41.410258+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:35.885776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:35.885776+0000","last_clean_scrub_stamp":"2024-03-04T09:28:35.885776+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T09:29:45.236355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"91'6","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T09:30:09.519785+0000","last_change":"2024-03-04T09:28:37.924411+0000","last_active":"2024-03-04T09:30:09.519785+0000","last_peered":"2024-03-04T09:30:09.519785+0000","last_clean":"2024-03-04T09:30:09.519785+0000","last_became_active":"2024-03-04T09:28:37.924147+0000","last_became_peered":"2024-03-04T09:28:37.924147+0000","last_unstale":"2024-03-04T09:30:09.519785+0000","last_undegraded":"2024-03-04T09:30:09.519785+0000","last_fullsized":"2024-03-04T09:30:09.519785+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_clean_scrub_stamp":"2024-03-04T09:28:36.887656+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T15:05:16.573937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T09:29:41.409623+0000","last_change":"2024-03-04T09:28:41.943710+0000","last_active":"2024-03-04T09:29:41.409623+0000","last_peered":"2024-03-04T09:29:41.409623+0000","last_clean":"2024-03-04T09:29:41.409623+0000","last_became_active":"2024-03-04T09:28:41.943258+0000","last_became_peered":"2024-03-04T09:28:41.943258+0000","last_unstale":"2024-03-04T09:29:41.409623+0000","last_undegraded":"2024-03-04T09:29:41.409623+0000","last_fullsized":"2024-03-04T09:29:41.409623+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:40.917115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:40.917115+0000","last_clean_scrub_stamp":"2024-03-04T09:28:40.917115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T15:32:52.597594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"84'2","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T09:29:41.075249+0000","last_change":"2024-03-04T09:29:41.075249+0000","last_active":"2024-03-04T09:29:41.075249+0000","last_peered":"2024-03-04T09:29:41.075249+0000","last_clean":"2024-03-04T09:29:41.075249+0000","last_became_active":"2024-03-04T09:29:41.072321+0000","last_became_peered":"2024-03-04T09:29:41.072321+0000","last_unstale":"2024-03-04T09:29:41.075249+0000","last_undegraded":"2024-03-04T09:29:41.075249+0000","last_fullsized":"2024-03-04T09:29:41.075249+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:35.885776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:35.885776+0000","last_clean_scrub_stamp":"2024-03-04T09:28:35.885776+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T09:59:10.757578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4],"acting":[6,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"91'9","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T09:30:09.513400+0000","last_change":"2024-03-04T09:28:37.916569+0000","last_active":"2024-03-04T09:30:09.513400+0000","last_peered":"2024-03-04T09:30:09.513400+0000","last_clean":"2024-03-04T09:30:09.513400+0000","last_became_active":"2024-03-04T09:28:37.916264+0000","last_became_peered":"2024-03-04T09:28:37.916264+0000","last_unstale":"2024-03-04T09:30:09.513400+0000","last_undegraded":"2024-03-04T09:30:09.513400+0000","last_fullsized":"2024-03-04T09:30:09.513400+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_clean_scrub_stamp":"2024-03-04T09:28:36.887656+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T16:59:51.981066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"91'9","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T09:30:09.516975+0000","last_change":"2024-03-04T09:29:41.068291+0000","last_active":"2024-03-04T09:30:09.516975+0000","last_peered":"2024-03-04T09:30:09.516975+0000","last_clean":"2024-03-04T09:30:09.516975+0000","last_became_active":"2024-03-04T09:29:41.067902+0000","last_became_peered":"2024-03-04T09:29:41.067902+0000","last_unstale":"2024-03-04T09:30:09.516975+0000","last_undegraded":"2024-03-04T09:30:09.516975+0000","last_fullsized":"2024-03-04T09:30:09.516975+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_clean_scrub_stamp":"2024-03-04T09:28:36.887656+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T12:40:21.605947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,9],"acting":[11,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"91'12","reported_seq":43,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T09:30:09.517696+0000","last_change":"2024-03-04T09:29:41.411161+0000","last_active":"2024-03-04T09:30:09.517696+0000","last_peered":"2024-03-04T09:30:09.517696+0000","last_clean":"2024-03-04T09:30:09.517696+0000","last_became_active":"2024-03-04T09:29:41.410703+0000","last_became_peered":"2024-03-04T09:29:41.410703+0000","last_unstale":"2024-03-04T09:30:09.517696+0000","last_undegraded":"2024-03-04T09:30:09.517696+0000","last_fullsized":"2024-03-04T09:30:09.517696+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_clean_scrub_stamp":"2024-03-04T09:28:36.887656+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T17:43:19.492432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,6],"acting":[2,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"91'8","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T09:30:09.511900+0000","last_change":"2024-03-04T09:28:37.906956+0000","last_active":"2024-03-04T09:30:09.511900+0000","last_peered":"2024-03-04T09:30:09.511900+0000","last_clean":"2024-03-04T09:30:09.511900+0000","last_became_active":"2024-03-04T09:28:37.906642+0000","last_became_peered":"2024-03-04T09:28:37.906642+0000","last_unstale":"2024-03-04T09:30:09.511900+0000","last_undegraded":"2024-03-04T09:30:09.511900+0000","last_fullsized":"2024-03-04T09:30:09.511900+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_clean_scrub_stamp":"2024-03-04T09:28:36.887656+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T20:58:18.387119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"91'7","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T09:30:09.525930+0000","last_change":"2024-03-04T09:29:41.071209+0000","last_active":"2024-03-04T09:30:09.525930+0000","last_peered":"2024-03-04T09:30:09.525930+0000","last_clean":"2024-03-04T09:30:09.525930+0000","last_became_active":"2024-03-04T09:29:41.070539+0000","last_became_peered":"2024-03-04T09:29:41.070539+0000","last_unstale":"2024-03-04T09:30:09.525930+0000","last_undegraded":"2024-03-04T09:30:09.525930+0000","last_fullsized":"2024-03-04T09:30:09.525930+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_clean_scrub_stamp":"2024-03-04T09:28:36.887656+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T11:47:08.175691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,8],"acting":[0,6,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"91'4","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T09:30:09.523479+0000","last_change":"2024-03-04T09:28:37.912702+0000","last_active":"2024-03-04T09:30:09.523479+0000","last_peered":"2024-03-04T09:30:09.523479+0000","last_clean":"2024-03-04T09:30:09.523479+0000","last_became_active":"2024-03-04T09:28:37.912397+0000","last_became_peered":"2024-03-04T09:28:37.912397+0000","last_unstale":"2024-03-04T09:30:09.523479+0000","last_undegraded":"2024-03-04T09:30:09.523479+0000","last_fullsized":"2024-03-04T09:30:09.523479+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_clean_scrub_stamp":"2024-03-04T09:28:36.887656+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T16:36:59.575111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"91'6","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T09:30:09.514765+0000","last_change":"2024-03-04T09:28:37.909801+0000","last_active":"2024-03-04T09:30:09.514765+0000","last_peered":"2024-03-04T09:30:09.514765+0000","last_clean":"2024-03-04T09:30:09.514765+0000","last_became_active":"2024-03-04T09:28:37.909505+0000","last_became_peered":"2024-03-04T09:28:37.909505+0000","last_unstale":"2024-03-04T09:30:09.514765+0000","last_undegraded":"2024-03-04T09:30:09.514765+0000","last_fullsized":"2024-03-04T09:30:09.514765+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_clean_scrub_stamp":"2024-03-04T09:28:36.887656+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T11:56:44.276188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"91'15","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T09:30:09.506625+0000","last_change":"2024-03-04T09:28:37.911076+0000","last_active":"2024-03-04T09:30:09.506625+0000","last_peered":"2024-03-04T09:30:09.506625+0000","last_clean":"2024-03-04T09:30:09.506625+0000","last_became_active":"2024-03-04T09:28:37.910786+0000","last_became_peered":"2024-03-04T09:28:37.910786+0000","last_unstale":"2024-03-04T09:30:09.506625+0000","last_undegraded":"2024-03-04T09:30:09.506625+0000","last_fullsized":"2024-03-04T09:30:09.506625+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_clean_scrub_stamp":"2024-03-04T09:28:36.887656+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T11:42:41.932235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"91'2","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T09:30:09.513363+0000","last_change":"2024-03-04T09:28:37.914661+0000","last_active":"2024-03-04T09:30:09.513363+0000","last_peered":"2024-03-04T09:30:09.513363+0000","last_clean":"2024-03-04T09:30:09.513363+0000","last_became_active":"2024-03-04T09:28:37.913851+0000","last_became_peered":"2024-03-04T09:28:37.913851+0000","last_unstale":"2024-03-04T09:30:09.513363+0000","last_undegraded":"2024-03-04T09:30:09.513363+0000","last_fullsized":"2024-03-04T09:30:09.513363+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_clean_scrub_stamp":"2024-03-04T09:28:36.887656+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T10:45:43.670006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"91'6","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T09:30:09.510744+0000","last_change":"2024-03-04T09:28:37.904746+0000","last_active":"2024-03-04T09:30:09.510744+0000","last_peered":"2024-03-04T09:30:09.510744+0000","last_clean":"2024-03-04T09:30:09.510744+0000","last_became_active":"2024-03-04T09:28:37.904466+0000","last_became_peered":"2024-03-04T09:28:37.904466+0000","last_unstale":"2024-03-04T09:30:09.510744+0000","last_undegraded":"2024-03-04T09:30:09.510744+0000","last_fullsized":"2024-03-04T09:30:09.510744+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_clean_scrub_stamp":"2024-03-04T09:28:36.887656+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T13:59:24.810820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"91'8","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T09:30:09.518488+0000","last_change":"2024-03-04T09:28:37.913252+0000","last_active":"2024-03-04T09:30:09.518488+0000","last_peered":"2024-03-04T09:30:09.518488+0000","last_clean":"2024-03-04T09:30:09.518488+0000","last_became_active":"2024-03-04T09:28:37.910819+0000","last_became_peered":"2024-03-04T09:28:37.910819+0000","last_unstale":"2024-03-04T09:30:09.518488+0000","last_undegraded":"2024-03-04T09:30:09.518488+0000","last_fullsized":"2024-03-04T09:30:09.518488+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_clean_scrub_stamp":"2024-03-04T09:28:36.887656+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T13:39:47.886505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"91'10","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T09:30:09.531136+0000","last_change":"2024-03-04T09:28:37.915716+0000","last_active":"2024-03-04T09:30:09.531136+0000","last_peered":"2024-03-04T09:30:09.531136+0000","last_clean":"2024-03-04T09:30:09.531136+0000","last_became_active":"2024-03-04T09:28:37.915398+0000","last_became_peered":"2024-03-04T09:28:37.915398+0000","last_unstale":"2024-03-04T09:30:09.531136+0000","last_undegraded":"2024-03-04T09:30:09.531136+0000","last_fullsized":"2024-03-04T09:30:09.531136+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_clean_scrub_stamp":"2024-03-04T09:28:36.887656+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T20:10:53.793715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"91'11","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T09:30:09.523748+0000","last_change":"2024-03-04T09:28:37.906671+0000","last_active":"2024-03-04T09:30:09.523748+0000","last_peered":"2024-03-04T09:30:09.523748+0000","last_clean":"2024-03-04T09:30:09.523748+0000","last_became_active":"2024-03-04T09:28:37.906344+0000","last_became_peered":"2024-03-04T09:28:37.906344+0000","last_unstale":"2024-03-04T09:30:09.523748+0000","last_undegraded":"2024-03-04T09:30:09.523748+0000","last_fullsized":"2024-03-04T09:30:09.523748+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_clean_scrub_stamp":"2024-03-04T09:28:36.887656+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T16:02:18.204424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"91'8","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-04T09:30:09.512715+0000","last_change":"2024-03-04T09:28:37.916895+0000","last_active":"2024-03-04T09:30:09.512715+0000","last_peered":"2024-03-04T09:30:09.512715+0000","last_clean":"2024-03-04T09:30:09.512715+0000","last_became_active":"2024-03-04T09:28:37.916585+0000","last_became_peered":"2024-03-04T09:28:37.916585+0000","last_unstale":"2024-03-04T09:30:09.512715+0000","last_undegraded":"2024-03-04T09:30:09.512715+0000","last_fullsized":"2024-03-04T09:30:09.512715+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T09:28:36.887656+0000","last_clean_scrub_stamp":"2024-03-04T09:28:36.887656+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T12:13:50.941390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":66166,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":462,"num_read_kb":301,"num_write":166,"num_write_kb":118,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":198498,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":144,"ondisk_log_size":144,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383713,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28016,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717136,"statfs":{"total":95995035648,"available":95966347264,"internally_reserved":0,"allocated":1024000,"data_stored":500831,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":72,"seq":309237645349,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":37008,"kb_used_data":1672,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708144,"statfs":{"total":95995035648,"available":95957139456,"internally_reserved":0,"allocated":1712128,"data_stored":1150655,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":67,"seq":287762808876,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36356,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708796,"statfs":{"total":95995035648,"available":95957807104,"internally_reserved":0,"allocated":1044480,"data_stored":505163,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":36174273},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":59,"seq":253403070512,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28032,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717120,"statfs":{"total":95995035648,"available":95966330880,"internally_reserved":0,"allocated":1040384,"data_stored":500987,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332149,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28612,"kb_used_data":1596,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716540,"statfs":{"total":95995035648,"available":95965736960,"internally_reserved":0,"allocated":1634304,"data_stored":1097037,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495675,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28012,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717140,"statfs":{"total":95995035648,"available":95966351360,"internally_reserved":0,"allocated":1019904,"data_stored":496243,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574931,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28076,"kb_used_data":1060,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717076,"statfs":{"total":95995035648,"available":95966285824,"internally_reserved":0,"allocated":1085440,"data_stored":558177,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738457,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28036,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717116,"statfs":{"total":95995035648,"available":95966326784,"internally_reserved":0,"allocated":1044480,"data_stored":497631,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411407,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28100,"kb_used_data":1084,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717052,"statfs":{"total":95995035648,"available":95966261248,"internally_reserved":0,"allocated":1110016,"data_stored":559527,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182472,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36344,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708808,"statfs":{"total":95995035648,"available":95957819392,"internally_reserved":0,"allocated":1032192,"data_stored":496437,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920836,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28024,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717128,"statfs":{"total":95995035648,"available":95966339072,"internally_reserved":0,"allocated":1032192,"data_stored":501425,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757312,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36940,"kb_used_data":1604,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708212,"statfs":{"total":95995035648,"available":95957209088,"internally_reserved":0,"allocated":1642496,"data_stored":1092093,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":62024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":63374,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1176,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2324,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":64134,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-04T09:30:22.100 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-03-04T09:30:22.100 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-03-04T09:30:22.100 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-03-04T09:30:22.100 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd shell --fsid 4e059f08-da08-11ee-95c6-87774f69a715 -- ceph health --format=json 2024-03-04T09:30:22.343 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/4e059f08-da08-11ee-95c6-87774f69a715/mon.a/config 2024-03-04T09:30:23.418 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:30:23 smithi052 ceph-mon[32473]: from='client.15123 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-04T09:30:23.418 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:30:23 smithi052 ceph-mon[32473]: pgmap v342: 129 pgs: 129 active+clean; 641 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 426 B/s wr, 8 op/s 2024-03-04T09:30:23.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:30:23 smithi191 ceph-mon[36816]: from='client.15123 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-04T09:30:23.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:30:23 smithi191 ceph-mon[36816]: pgmap v342: 129 pgs: 129 active+clean; 641 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 426 B/s wr, 8 op/s 2024-03-04T09:30:23.631 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:30:23 smithi106 ceph-mon[36484]: from='client.15123 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-04T09:30:23.631 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:30:23 smithi106 ceph-mon[36484]: pgmap v342: 129 pgs: 129 active+clean; 641 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 426 B/s wr, 8 op/s 2024-03-04T09:30:23.966 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:23.967+0000 7fda70130700 1 -- 172.21.15.52:0/3428868072 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fda68102ad0 msgr2=0x7fda6810ef70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:30:23.966 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:23.967+0000 7fda70130700 1 --2- 172.21.15.52:0/3428868072 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fda68102ad0 0x7fda6810ef70 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fda6000b0a0 tx=0x7fda6000b3b0 comp rx=0 tx=0).stop 2024-03-04T09:30:23.966 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:23.968+0000 7fda70130700 1 -- 172.21.15.52:0/3428868072 shutdown_connections 2024-03-04T09:30:23.967 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:23.968+0000 7fda70130700 1 --2- 172.21.15.52:0/3428868072 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fda68102ad0 0x7fda6810ef70 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:23.967 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:23.968+0000 7fda70130700 1 --2- 172.21.15.52:0/3428868072 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fda681081c0 0x7fda681085a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:23.967 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:23.968+0000 7fda70130700 1 --2- 172.21.15.52:0/3428868072 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fda68102110 0x7fda68102590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:23.967 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:23.968+0000 7fda70130700 1 -- 172.21.15.52:0/3428868072 >> 172.21.15.52:0/3428868072 conn(0x7fda680791f0 msgr2=0x7fda68079600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:30:23.967 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:23.968+0000 7fda70130700 1 -- 172.21.15.52:0/3428868072 shutdown_connections 2024-03-04T09:30:23.967 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:23.968+0000 7fda70130700 1 -- 172.21.15.52:0/3428868072 wait complete. 2024-03-04T09:30:23.967 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:23.968+0000 7fda70130700 1 Processor -- start 2024-03-04T09:30:23.967 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:23.968+0000 7fda70130700 1 -- start start 2024-03-04T09:30:23.967 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:23.968+0000 7fda70130700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fda68102110 0x7fda6819e570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:30:23.967 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:23.968+0000 7fda70130700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fda68102ad0 0x7fda6819eab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:30:23.967 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:23.968+0000 7fda70130700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fda681081c0 0x7fda68198640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:30:23.967 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:23.968+0000 7fda70130700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda68112420 con 0x7fda681081c0 2024-03-04T09:30:23.967 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:23.968+0000 7fda70130700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda681122a0 con 0x7fda68102110 2024-03-04T09:30:23.968 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:23.969+0000 7fda70130700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda681125a0 con 0x7fda68102ad0 2024-03-04T09:30:23.968 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:23.969+0000 7fda6e6cd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fda681081c0 0x7fda68198640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:30:23.968 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:23.969+0000 7fda6decc700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fda68102110 0x7fda6819e570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:30:23.968 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:23.969+0000 7fda6e6cd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fda681081c0 0x7fda68198640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:47702/0 (socket says 172.21.15.52:47702) 2024-03-04T09:30:23.968 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:23.969+0000 7fda6e6cd700 1 -- 172.21.15.52:0/2760377847 learned_addr learned my addr 172.21.15.52:0/2760377847 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-04T09:30:23.968 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:23.969+0000 7fda6d6cb700 1 --2- 172.21.15.52:0/2760377847 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fda68102ad0 0x7fda6819eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:30:23.968 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:23.969+0000 7fda6e6cd700 1 -- 172.21.15.52:0/2760377847 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fda68102ad0 msgr2=0x7fda6819eab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:30:23.968 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:23.969+0000 7fda6e6cd700 1 --2- 172.21.15.52:0/2760377847 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fda68102ad0 0x7fda6819eab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:23.968 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:23.969+0000 7fda6e6cd700 1 -- 172.21.15.52:0/2760377847 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fda68102110 msgr2=0x7fda6819e570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:30:23.968 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:23.969+0000 7fda6e6cd700 1 --2- 172.21.15.52:0/2760377847 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fda68102110 0x7fda6819e570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:23.968 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:23.969+0000 7fda6e6cd700 1 -- 172.21.15.52:0/2760377847 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda68198f20 con 0x7fda681081c0 2024-03-04T09:30:23.968 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:23.969+0000 7fda6decc700 1 --2- 172.21.15.52:0/2760377847 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fda68102110 0x7fda6819e570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-04T09:30:23.968 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:23.969+0000 7fda6d6cb700 1 --2- 172.21.15.52:0/2760377847 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fda68102ad0 0x7fda6819eab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-04T09:30:23.969 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:23.969+0000 7fda6e6cd700 1 --2- 172.21.15.52:0/2760377847 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fda681081c0 0x7fda68198640 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fda6000b070 tx=0x7fda60004c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:30:23.969 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:23.969+0000 7fda5effd700 1 -- 172.21.15.52:0/2760377847 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda60021070 con 0x7fda681081c0 2024-03-04T09:30:23.969 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:23.969+0000 7fda5effd700 1 -- 172.21.15.52:0/2760377847 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fda60003c00 con 0x7fda681081c0 2024-03-04T09:30:23.969 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:23.969+0000 7fda70130700 1 -- 172.21.15.52:0/2760377847 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda681991b0 con 0x7fda681081c0 2024-03-04T09:30:23.969 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:23.969+0000 7fda5effd700 1 -- 172.21.15.52:0/2760377847 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda60004740 con 0x7fda681081c0 2024-03-04T09:30:23.970 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:23.969+0000 7fda70130700 1 -- 172.21.15.52:0/2760377847 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda681a54f0 con 0x7fda681081c0 2024-03-04T09:30:23.970 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:23.970+0000 7fda70130700 1 -- 172.21.15.52:0/2760377847 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fda6810a510 con 0x7fda681081c0 2024-03-04T09:30:23.970 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:23.972+0000 7fda5effd700 1 -- 172.21.15.52:0/2760377847 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) v1 ==== 136756+0+0 (secure 0 0 0) 0x7fda60007980 con 0x7fda681081c0 2024-03-04T09:30:23.970 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:23.972+0000 7fda5effd700 1 --2- 172.21.15.52:0/2760377847 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7fda540a1e90 0x7fda540a4350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:30:23.970 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:23.972+0000 7fda6decc700 1 --2- 172.21.15.52:0/2760377847 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7fda540a1e90 0x7fda540a4350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:30:23.970 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:23.972+0000 7fda5effd700 1 -- 172.21.15.52:0/2760377847 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9092+0+0 (secure 0 0 0) 0x7fda6000f020 con 0x7fda681081c0 2024-03-04T09:30:23.970 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:23.973+0000 7fda6decc700 1 --2- 172.21.15.52:0/2760377847 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7fda540a1e90 0x7fda540a4350 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fda64011490 tx=0x7fda64009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:30:23.972 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:23.976+0000 7fda5effd700 1 -- 172.21.15.52:0/2760377847 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fda6002a020 con 0x7fda681081c0 2024-03-04T09:30:24.221 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:24.224+0000 7fda70130700 1 -- 172.21.15.52:0/2760377847 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7fda68199b70 con 0x7fda681081c0 2024-03-04T09:30:24.221 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:24.224+0000 7fda5effd700 1 -- 172.21.15.52:0/2760377847 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7fda6009c930 con 0x7fda681081c0 2024-03-04T09:30:24.221 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-03-04T09:30:24.221 INFO:teuthology.orchestra.run.smithi052.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-03-04T09:30:24.224 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:24.226+0000 7fda70130700 1 -- 172.21.15.52:0/2760377847 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7fda540a1e90 msgr2=0x7fda540a4350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:30:24.224 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:24.226+0000 7fda70130700 1 --2- 172.21.15.52:0/2760377847 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7fda540a1e90 0x7fda540a4350 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fda64011490 tx=0x7fda64009260 comp rx=0 tx=0).stop 2024-03-04T09:30:24.224 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:24.226+0000 7fda70130700 1 -- 172.21.15.52:0/2760377847 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fda681081c0 msgr2=0x7fda68198640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:30:24.224 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:24.226+0000 7fda70130700 1 --2- 172.21.15.52:0/2760377847 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fda681081c0 0x7fda68198640 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fda6000b070 tx=0x7fda60004c60 comp rx=0 tx=0).stop 2024-03-04T09:30:24.224 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:24.226+0000 7fda70130700 1 -- 172.21.15.52:0/2760377847 shutdown_connections 2024-03-04T09:30:24.224 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:24.226+0000 7fda70130700 1 --2- 172.21.15.52:0/2760377847 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7fda540a1e90 0x7fda540a4350 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:24.224 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:24.226+0000 7fda70130700 1 --2- 172.21.15.52:0/2760377847 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fda681081c0 0x7fda68198640 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:24.224 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:24.226+0000 7fda70130700 1 --2- 172.21.15.52:0/2760377847 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fda68102110 0x7fda6819e570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:24.224 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:24.226+0000 7fda70130700 1 --2- 172.21.15.52:0/2760377847 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fda68102ad0 0x7fda6819eab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:24.224 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:24.226+0000 7fda70130700 1 -- 172.21.15.52:0/2760377847 >> 172.21.15.52:0/2760377847 conn(0x7fda680791f0 msgr2=0x7fda68100e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:30:24.224 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:24.227+0000 7fda70130700 1 -- 172.21.15.52:0/2760377847 shutdown_connections 2024-03-04T09:30:24.224 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:24.227+0000 7fda70130700 1 -- 172.21.15.52:0/2760377847 wait complete. 2024-03-04T09:30:25.071 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-03-04T09:30:25.071 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-04T09:30:25.186 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:25.188+0000 7f6dd9a5c640 1 -- 172.21.15.52:0/2493645775 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6dd41015f0 msgr2=0x7f6dd4101a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:30:25.187 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:25.188+0000 7f6dd9a5c640 1 --2- 172.21.15.52:0/2493645775 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6dd41015f0 0x7f6dd4101a10 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f6dc4009ca0 tx=0x7f6dc402c0f0 comp rx=0 tx=0).stop 2024-03-04T09:30:25.187 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:25.188+0000 7f6dd9a5c640 1 -- 172.21.15.52:0/2493645775 shutdown_connections 2024-03-04T09:30:25.187 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:25.188+0000 7f6dd9a5c640 1 --2- 172.21.15.52:0/2493645775 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6dd41031a0 0x7f6dd4109a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:25.187 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:25.188+0000 7f6dd9a5c640 1 --2- 172.21.15.52:0/2493645775 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6dd41027e0 0x7f6dd4102c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:25.187 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:25.188+0000 7f6dd9a5c640 1 --2- 172.21.15.52:0/2493645775 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6dd41015f0 0x7f6dd4101a10 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:25.187 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:25.188+0000 7f6dd9a5c640 1 -- 172.21.15.52:0/2493645775 >> 172.21.15.52:0/2493645775 conn(0x7f6dd40fcb80 msgr2=0x7f6dd40fefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:30:25.187 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:25.188+0000 7f6dd9a5c640 1 -- 172.21.15.52:0/2493645775 shutdown_connections 2024-03-04T09:30:25.187 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:25.188+0000 7f6dd9a5c640 1 -- 172.21.15.52:0/2493645775 wait complete. 2024-03-04T09:30:25.187 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:25.188+0000 7f6dd9a5c640 1 Processor -- start 2024-03-04T09:30:25.187 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:25.188+0000 7f6dd9a5c640 1 -- start start 2024-03-04T09:30:25.187 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:25.189+0000 7f6dd9a5c640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6dd41015f0 0x7f6dd41978e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:30:25.187 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:25.189+0000 7f6dd9a5c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6dd41027e0 0x7f6dd4197e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:30:25.188 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:25.189+0000 7f6dd9a5c640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6dd41031a0 0x7f6dd419ee90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:30:25.188 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:25.189+0000 7f6dd9a5c640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6dd410d240 con 0x7f6dd41015f0 2024-03-04T09:30:25.188 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:25.189+0000 7f6dd9a5c640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6dd410d0c0 con 0x7f6dd41027e0 2024-03-04T09:30:25.188 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:25.189+0000 7f6dd9a5c640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6dd410d3c0 con 0x7f6dd41031a0 2024-03-04T09:30:25.188 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:25.189+0000 7f6dd2ffd640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6dd41015f0 0x7f6dd41978e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:30:25.188 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:25.189+0000 7f6dd2ffd640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6dd41015f0 0x7f6dd41978e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:51968/0 (socket says 172.21.15.52:51968) 2024-03-04T09:30:25.188 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:25.189+0000 7f6dd2ffd640 1 -- 172.21.15.52:0/4278522654 learned_addr learned my addr 172.21.15.52:0/4278522654 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-04T09:30:25.188 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:25.189+0000 7f6dd37fe640 1 --2- 172.21.15.52:0/4278522654 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6dd41031a0 0x7f6dd419ee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:30:25.188 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:25.189+0000 7f6dd27fc640 1 --2- 172.21.15.52:0/4278522654 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6dd41027e0 0x7f6dd4197e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:30:25.188 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:25.189+0000 7f6dd2ffd640 1 -- 172.21.15.52:0/4278522654 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6dd41031a0 msgr2=0x7f6dd419ee90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:30:25.188 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:25.189+0000 7f6dd2ffd640 1 --2- 172.21.15.52:0/4278522654 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6dd41031a0 0x7f6dd419ee90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:25.188 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:25.189+0000 7f6dd2ffd640 1 -- 172.21.15.52:0/4278522654 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6dd41027e0 msgr2=0x7f6dd4197e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:30:25.188 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:25.189+0000 7f6dd2ffd640 1 --2- 172.21.15.52:0/4278522654 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6dd41027e0 0x7f6dd4197e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:25.189 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:25.189+0000 7f6dd2ffd640 1 -- 172.21.15.52:0/4278522654 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6dd419f500 con 0x7f6dd41015f0 2024-03-04T09:30:25.189 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:25.189+0000 7f6dd2ffd640 1 --2- 172.21.15.52:0/4278522654 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6dd41015f0 0x7f6dd41978e0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f6dc40324c0 tx=0x7f6dc4032f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:30:25.189 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:25.189+0000 7f6dd8a5a640 1 -- 172.21.15.52:0/4278522654 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dc4041020 con 0x7f6dd41015f0 2024-03-04T09:30:25.189 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:25.189+0000 7f6dd8a5a640 1 -- 172.21.15.52:0/4278522654 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6dc4034cd0 con 0x7f6dd41015f0 2024-03-04T09:30:25.189 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:25.189+0000 7f6dd9a5c640 1 -- 172.21.15.52:0/4278522654 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6dd419f790 con 0x7f6dd41015f0 2024-03-04T09:30:25.189 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:25.189+0000 7f6dd8a5a640 1 -- 172.21.15.52:0/4278522654 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dc4039040 con 0x7f6dd41015f0 2024-03-04T09:30:25.189 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:25.189+0000 7f6dd9a5c640 1 -- 172.21.15.52:0/4278522654 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6dd419fc80 con 0x7f6dd41015f0 2024-03-04T09:30:25.190 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:25.190+0000 7f6dd9a5c640 1 -- 172.21.15.52:0/4278522654 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6dd4108910 con 0x7f6dd41015f0 2024-03-04T09:30:25.190 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:25.191+0000 7f6dd8a5a640 1 -- 172.21.15.52:0/4278522654 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) v1 ==== 136756+0+0 (secure 0 0 0) 0x7f6dc4044020 con 0x7f6dd41015f0 2024-03-04T09:30:25.190 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:25.192+0000 7f6dd8a5a640 1 --2- 172.21.15.52:0/4278522654 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f6dbc0a1d80 0x7f6dbc0a4240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:30:25.190 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:25.192+0000 7f6dd27fc640 1 --2- 172.21.15.52:0/4278522654 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f6dbc0a1d80 0x7f6dbc0a4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:30:25.190 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:25.192+0000 7f6dd8a5a640 1 -- 172.21.15.52:0/4278522654 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9092+0+0 (secure 0 0 0) 0x7f6dc40effa0 con 0x7f6dd41015f0 2024-03-04T09:30:25.190 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:25.192+0000 7f6dd27fc640 1 --2- 172.21.15.52:0/4278522654 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f6dbc0a1d80 0x7f6dbc0a4240 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f6dd4198df0 tx=0x7f6dc8005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:30:25.193 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:25.196+0000 7f6dd8a5a640 1 -- 172.21.15.52:0/4278522654 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f6dc404b020 con 0x7f6dd41015f0 2024-03-04T09:30:25.334 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:30:25 smithi052 ceph-mon[32473]: pgmap v343: 129 pgs: 129 active+clean; 641 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 5 op/s 2024-03-04T09:30:25.335 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:30:25 smithi052 ceph-mon[32473]: from='client.? 172.21.15.52:0/2760377847' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-03-04T09:30:25.378 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:25.381+0000 7f6dd9a5c640 1 -- 172.21.15.52:0/4278522654 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6dd4062200 con 0x7f6dd41015f0 2024-03-04T09:30:25.379 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:25.382+0000 7f6dd8a5a640 1 -- 172.21.15.52:0/4278522654 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) v1 ==== 74+0+24635 (secure 0 0 0) 0x7f6dc40bd100 con 0x7f6dd41015f0 2024-03-04T09:30:25.380 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-03-04T09:30:25.380 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":91,"fsid":"4e059f08-da08-11ee-95c6-87774f69a715","created":"2024-03-04T09:20:46.104963+0000","modified":"2024-03-04T09:29:41.058759+0000","last_up_change":"2024-03-04T09:27:49.261365+0000","last_in_change":"2024-03-04T09:27:30.024964+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-04T09:24:01.742461+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-04T09:28:35.835600+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-04T09:28:36.453099+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-04T09:28:40.168831+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"583172be-602f-4bf4-bfcb-42663cf5b277","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6802","nonce":1371221397},{"type":"v1","addr":"172.21.15.52:6803","nonce":1371221397}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6804","nonce":1371221397},{"type":"v1","addr":"172.21.15.52:6805","nonce":1371221397}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6808","nonce":1371221397},{"type":"v1","addr":"172.21.15.52:6809","nonce":1371221397}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6806","nonce":1371221397},{"type":"v1","addr":"172.21.15.52:6807","nonce":1371221397}]},"public_addr":"172.21.15.52:6803/1371221397","cluster_addr":"172.21.15.52:6805/1371221397","heartbeat_back_addr":"172.21.15.52:6809/1371221397","heartbeat_front_addr":"172.21.15.52:6807/1371221397","state":["exists","up"]},{"osd":1,"uuid":"5ae0081c-a2f6-46eb-ace0-0368208377b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":2540865537},{"type":"v1","addr":"172.21.15.106:6801","nonce":2540865537}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":2540865537},{"type":"v1","addr":"172.21.15.106:6803","nonce":2540865537}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":2540865537},{"type":"v1","addr":"172.21.15.106:6807","nonce":2540865537}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":2540865537},{"type":"v1","addr":"172.21.15.106:6805","nonce":2540865537}]},"public_addr":"172.21.15.106:6801/2540865537","cluster_addr":"172.21.15.106:6803/2540865537","heartbeat_back_addr":"172.21.15.106:6807/2540865537","heartbeat_front_addr":"172.21.15.106:6805/2540865537","state":["exists","up"]},{"osd":2,"uuid":"1948bac7-2bb2-4218-8b93-4e4743a0de04","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":1377083164},{"type":"v1","addr":"172.21.15.191:6801","nonce":1377083164}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":1377083164},{"type":"v1","addr":"172.21.15.191:6803","nonce":1377083164}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":1377083164},{"type":"v1","addr":"172.21.15.191:6807","nonce":1377083164}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":1377083164},{"type":"v1","addr":"172.21.15.191:6805","nonce":1377083164}]},"public_addr":"172.21.15.191:6801/1377083164","cluster_addr":"172.21.15.191:6803/1377083164","heartbeat_back_addr":"172.21.15.191:6807/1377083164","heartbeat_front_addr":"172.21.15.191:6805/1377083164","state":["exists","up"]},{"osd":3,"uuid":"16c9e9fe-7167-4992-bf12-1a7398e0498a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6810","nonce":2388571810},{"type":"v1","addr":"172.21.15.52:6811","nonce":2388571810}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6812","nonce":2388571810},{"type":"v1","addr":"172.21.15.52:6813","nonce":2388571810}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6816","nonce":2388571810},{"type":"v1","addr":"172.21.15.52:6817","nonce":2388571810}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6814","nonce":2388571810},{"type":"v1","addr":"172.21.15.52:6815","nonce":2388571810}]},"public_addr":"172.21.15.52:6811/2388571810","cluster_addr":"172.21.15.52:6813/2388571810","heartbeat_back_addr":"172.21.15.52:6817/2388571810","heartbeat_front_addr":"172.21.15.52:6815/2388571810","state":["exists","up"]},{"osd":4,"uuid":"c8ef48c7-7c78-44d3-a9e5-99ec750abd3d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6808","nonce":3465868015},{"type":"v1","addr":"172.21.15.106:6809","nonce":3465868015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6810","nonce":3465868015},{"type":"v1","addr":"172.21.15.106:6811","nonce":3465868015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6814","nonce":3465868015},{"type":"v1","addr":"172.21.15.106:6815","nonce":3465868015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6812","nonce":3465868015},{"type":"v1","addr":"172.21.15.106:6813","nonce":3465868015}]},"public_addr":"172.21.15.106:6809/3465868015","cluster_addr":"172.21.15.106:6811/3465868015","heartbeat_back_addr":"172.21.15.106:6815/3465868015","heartbeat_front_addr":"172.21.15.106:6813/3465868015","state":["exists","up"]},{"osd":5,"uuid":"9e41824a-c5a2-4f88-be50-2ba1dceb3220","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":1485092643},{"type":"v1","addr":"172.21.15.191:6809","nonce":1485092643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":1485092643},{"type":"v1","addr":"172.21.15.191:6811","nonce":1485092643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":1485092643},{"type":"v1","addr":"172.21.15.191:6815","nonce":1485092643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":1485092643},{"type":"v1","addr":"172.21.15.191:6813","nonce":1485092643}]},"public_addr":"172.21.15.191:6809/1485092643","cluster_addr":"172.21.15.191:6811/1485092643","heartbeat_back_addr":"172.21.15.191:6815/1485092643","heartbeat_front_addr":"172.21.15.191:6813/1485092643","state":["exists","up"]},{"osd":6,"uuid":"b35dccee-ec16-452e-ac69-3ac2fb7b27fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6818","nonce":1989797856},{"type":"v1","addr":"172.21.15.52:6819","nonce":1989797856}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6820","nonce":1989797856},{"type":"v1","addr":"172.21.15.52:6821","nonce":1989797856}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6824","nonce":1989797856},{"type":"v1","addr":"172.21.15.52:6825","nonce":1989797856}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6822","nonce":1989797856},{"type":"v1","addr":"172.21.15.52:6823","nonce":1989797856}]},"public_addr":"172.21.15.52:6819/1989797856","cluster_addr":"172.21.15.52:6821/1989797856","heartbeat_back_addr":"172.21.15.52:6825/1989797856","heartbeat_front_addr":"172.21.15.52:6823/1989797856","state":["exists","up"]},{"osd":7,"uuid":"71303e3d-14ec-4cbd-b19c-67645a1de3c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6816","nonce":1450870329},{"type":"v1","addr":"172.21.15.106:6817","nonce":1450870329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":1450870329},{"type":"v1","addr":"172.21.15.106:6819","nonce":1450870329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":1450870329},{"type":"v1","addr":"172.21.15.106:6823","nonce":1450870329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6820","nonce":1450870329},{"type":"v1","addr":"172.21.15.106:6821","nonce":1450870329}]},"public_addr":"172.21.15.106:6817/1450870329","cluster_addr":"172.21.15.106:6819/1450870329","heartbeat_back_addr":"172.21.15.106:6823/1450870329","heartbeat_front_addr":"172.21.15.106:6821/1450870329","state":["exists","up"]},{"osd":8,"uuid":"6f5786c4-4402-435a-bdad-adff46fe9473","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":1114119072},{"type":"v1","addr":"172.21.15.191:6817","nonce":1114119072}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6818","nonce":1114119072},{"type":"v1","addr":"172.21.15.191:6819","nonce":1114119072}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":1114119072},{"type":"v1","addr":"172.21.15.191:6823","nonce":1114119072}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":1114119072},{"type":"v1","addr":"172.21.15.191:6821","nonce":1114119072}]},"public_addr":"172.21.15.191:6817/1114119072","cluster_addr":"172.21.15.191:6819/1114119072","heartbeat_back_addr":"172.21.15.191:6823/1114119072","heartbeat_front_addr":"172.21.15.191:6821/1114119072","state":["exists","up"]},{"osd":9,"uuid":"062bbaa5-1227-4684-b859-450f5594a90c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6826","nonce":3485821914},{"type":"v1","addr":"172.21.15.52:6827","nonce":3485821914}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6828","nonce":3485821914},{"type":"v1","addr":"172.21.15.52:6829","nonce":3485821914}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6832","nonce":3485821914},{"type":"v1","addr":"172.21.15.52:6833","nonce":3485821914}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6830","nonce":3485821914},{"type":"v1","addr":"172.21.15.52:6831","nonce":3485821914}]},"public_addr":"172.21.15.52:6827/3485821914","cluster_addr":"172.21.15.52:6829/3485821914","heartbeat_back_addr":"172.21.15.52:6833/3485821914","heartbeat_front_addr":"172.21.15.52:6831/3485821914","state":["exists","up"]},{"osd":10,"uuid":"1a285a5a-f793-48da-a8eb-5326d9528cd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":3815586576},{"type":"v1","addr":"172.21.15.106:6825","nonce":3815586576}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6826","nonce":3815586576},{"type":"v1","addr":"172.21.15.106:6827","nonce":3815586576}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":3815586576},{"type":"v1","addr":"172.21.15.106:6831","nonce":3815586576}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":3815586576},{"type":"v1","addr":"172.21.15.106:6829","nonce":3815586576}]},"public_addr":"172.21.15.106:6825/3815586576","cluster_addr":"172.21.15.106:6827/3815586576","heartbeat_back_addr":"172.21.15.106:6831/3815586576","heartbeat_front_addr":"172.21.15.106:6829/3815586576","state":["exists","up"]},{"osd":11,"uuid":"1f8b9a43-57fa-41f4-9969-3b803a74bf45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":2009632843},{"type":"v1","addr":"172.21.15.191:6825","nonce":2009632843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":2009632843},{"type":"v1","addr":"172.21.15.191:6827","nonce":2009632843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":2009632843},{"type":"v1","addr":"172.21.15.191:6831","nonce":2009632843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":2009632843},{"type":"v1","addr":"172.21.15.191:6829","nonce":2009632843}]},"public_addr":"172.21.15.191:6825/2009632843","cluster_addr":"172.21.15.191:6827/2009632843","heartbeat_back_addr":"172.21.15.191:6831/2009632843","heartbeat_front_addr":"172.21.15.191:6829/2009632843","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T09:23:10.894763+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T09:23:34.420055+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T09:23:58.288324+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T09:24:23.673457+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T09:24:48.208561+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T09:25:13.405473+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T09:25:38.670915+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T09:26:04.451257+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T09:26:30.281380+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T09:26:56.543143+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T09:27:22.412930+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T09:27:48.003866+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.20","mappings":[{"from":5,"to":9}]},{"pgid":"3.21","mappings":[{"from":4,"to":6}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]},{"pgid":"4.9","mappings":[{"from":5,"to":8}]},{"pgid":"4.17","mappings":[{"from":9,"to":8}]},{"pgid":"4.1b","mappings":[{"from":9,"to":10}]},{"pgid":"4.1f","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.52:0/4082645718":"2024-03-05T09:21:39.401935+0000","172.21.15.52:0/1271949988":"2024-03-05T09:21:39.401935+0000","172.21.15.52:6801/2379361333":"2024-03-05T09:21:12.760938+0000","172.21.15.52:0/497075075":"2024-03-05T09:21:12.760938+0000","172.21.15.52:6800/2379361333":"2024-03-05T09:21:12.760938+0000","172.21.15.52:0/568242577":"2024-03-05T09:21:12.760938+0000","172.21.15.52:6801/2425730636":"2024-03-05T09:21:39.401935+0000","172.21.15.52:6800/2425730636":"2024-03-05T09:21:39.401935+0000","172.21.15.52:0/2743084243":"2024-03-05T09:21:39.401935+0000","172.21.15.52:0/1035878034":"2024-03-05T09:21:12.760938+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-04T09:30:25.382 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:25.384+0000 7f6dd9a5c640 1 -- 172.21.15.52:0/4278522654 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f6dbc0a1d80 msgr2=0x7f6dbc0a4240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:30:25.382 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:25.384+0000 7f6dd9a5c640 1 --2- 172.21.15.52:0/4278522654 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f6dbc0a1d80 0x7f6dbc0a4240 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f6dd4198df0 tx=0x7f6dc8005f90 comp rx=0 tx=0).stop 2024-03-04T09:30:25.382 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:25.384+0000 7f6dd9a5c640 1 -- 172.21.15.52:0/4278522654 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6dd41015f0 msgr2=0x7f6dd41978e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:30:25.382 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:25.384+0000 7f6dd9a5c640 1 --2- 172.21.15.52:0/4278522654 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6dd41015f0 0x7f6dd41978e0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f6dc40324c0 tx=0x7f6dc4032f40 comp rx=0 tx=0).stop 2024-03-04T09:30:25.382 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:25.385+0000 7f6dd9a5c640 1 -- 172.21.15.52:0/4278522654 shutdown_connections 2024-03-04T09:30:25.382 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:25.385+0000 7f6dd9a5c640 1 --2- 172.21.15.52:0/4278522654 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f6dbc0a1d80 0x7f6dbc0a4240 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:25.382 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:25.385+0000 7f6dd9a5c640 1 --2- 172.21.15.52:0/4278522654 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6dd41031a0 0x7f6dd419ee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:25.382 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:25.385+0000 7f6dd9a5c640 1 --2- 172.21.15.52:0/4278522654 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6dd41027e0 0x7f6dd4197e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:25.382 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:25.385+0000 7f6dd9a5c640 1 --2- 172.21.15.52:0/4278522654 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6dd41015f0 0x7f6dd41978e0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:25.382 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:25.385+0000 7f6dd9a5c640 1 -- 172.21.15.52:0/4278522654 >> 172.21.15.52:0/4278522654 conn(0x7f6dd40fcb80 msgr2=0x7f6dd4105ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:30:25.382 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:25.385+0000 7f6dd9a5c640 1 -- 172.21.15.52:0/4278522654 shutdown_connections 2024-03-04T09:30:25.383 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:25.385+0000 7f6dd9a5c640 1 -- 172.21.15.52:0/4278522654 wait complete. 2024-03-04T09:30:25.393 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-04T09:24:01.742461+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-04T09:28:35.835600+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-04T09:28:36.453099+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-04T09:28:40.168831+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '89', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-04T09:30:25.394 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-04T09:30:25.503 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:25.502+0000 7f2d4eb93640 1 -- 172.21.15.52:0/915723432 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2d481027e0 msgr2=0x7f2d48102c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:30:25.503 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:25.502+0000 7f2d4eb93640 1 --2- 172.21.15.52:0/915723432 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2d481027e0 0x7f2d48102c60 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f2d34009900 tx=0x7f2d3402bc30 comp rx=0 tx=0).stop 2024-03-04T09:30:25.503 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:25.502+0000 7f2d4eb93640 1 -- 172.21.15.52:0/915723432 shutdown_connections 2024-03-04T09:30:25.503 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:25.502+0000 7f2d4eb93640 1 --2- 172.21.15.52:0/915723432 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2d481031a0 0x7f2d48109a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:25.504 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:25.502+0000 7f2d4eb93640 1 --2- 172.21.15.52:0/915723432 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2d481027e0 0x7f2d48102c60 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:25.504 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:25.502+0000 7f2d4eb93640 1 --2- 172.21.15.52:0/915723432 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2d481015f0 0x7f2d48101a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:25.504 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:25.502+0000 7f2d4eb93640 1 -- 172.21.15.52:0/915723432 >> 172.21.15.52:0/915723432 conn(0x7f2d480fcb40 msgr2=0x7f2d480fefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:30:25.504 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:25.502+0000 7f2d4eb93640 1 -- 172.21.15.52:0/915723432 shutdown_connections 2024-03-04T09:30:25.504 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:25.502+0000 7f2d4eb93640 1 -- 172.21.15.52:0/915723432 wait complete. 2024-03-04T09:30:25.504 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:25.503+0000 7f2d4eb93640 1 Processor -- start 2024-03-04T09:30:25.504 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:25.503+0000 7f2d4eb93640 1 -- start start 2024-03-04T09:30:25.504 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:25.503+0000 7f2d4eb93640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2d481015f0 0x7f2d48197910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:30:25.504 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:25.503+0000 7f2d4eb93640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2d481027e0 0x7f2d48197e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:30:25.504 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:25.503+0000 7f2d4eb93640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2d481031a0 0x7f2d4819eec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:30:25.504 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:25.503+0000 7f2d4eb93640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d4810d270 con 0x7f2d481027e0 2024-03-04T09:30:25.505 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:25.503+0000 7f2d4eb93640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d4810d0f0 con 0x7f2d481015f0 2024-03-04T09:30:25.505 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:25.503+0000 7f2d4eb93640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d4810d3f0 con 0x7f2d481031a0 2024-03-04T09:30:25.505 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:25.503+0000 7f2d47fff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2d481027e0 0x7f2d48197e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:30:25.505 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:25.503+0000 7f2d47fff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2d481027e0 0x7f2d48197e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:51992/0 (socket says 172.21.15.52:51992) 2024-03-04T09:30:25.505 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:25.503+0000 7f2d47fff640 1 -- 172.21.15.52:0/643014779 learned_addr learned my addr 172.21.15.52:0/643014779 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-04T09:30:25.505 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:25.503+0000 7f2d4c92f640 1 --2- 172.21.15.52:0/643014779 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2d481015f0 0x7f2d48197910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:30:25.505 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:25.503+0000 7f2d4d130640 1 --2- 172.21.15.52:0/643014779 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2d481031a0 0x7f2d4819eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:30:25.505 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:25.503+0000 7f2d47fff640 1 -- 172.21.15.52:0/643014779 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2d481031a0 msgr2=0x7f2d4819eec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:30:25.505 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:25.503+0000 7f2d47fff640 1 --2- 172.21.15.52:0/643014779 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2d481031a0 0x7f2d4819eec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:25.505 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:25.503+0000 7f2d47fff640 1 -- 172.21.15.52:0/643014779 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2d481015f0 msgr2=0x7f2d48197910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:30:25.505 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:25.503+0000 7f2d47fff640 1 --2- 172.21.15.52:0/643014779 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2d481015f0 0x7f2d48197910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:25.505 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:25.503+0000 7f2d47fff640 1 -- 172.21.15.52:0/643014779 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d4819f530 con 0x7f2d481027e0 2024-03-04T09:30:25.506 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:25.504+0000 7f2d47fff640 1 --2- 172.21.15.52:0/643014779 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2d481027e0 0x7f2d48197e50 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f2d34009230 tx=0x7f2d3402c650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:30:25.506 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:25.504+0000 7f2d45ffb640 1 -- 172.21.15.52:0/643014779 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d34041070 con 0x7f2d481027e0 2024-03-04T09:30:25.506 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:25.504+0000 7f2d45ffb640 1 -- 172.21.15.52:0/643014779 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2d34004450 con 0x7f2d481027e0 2024-03-04T09:30:25.506 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:25.504+0000 7f2d4eb93640 1 -- 172.21.15.52:0/643014779 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d4819f7c0 con 0x7f2d481027e0 2024-03-04T09:30:25.506 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:25.504+0000 7f2d45ffb640 1 -- 172.21.15.52:0/643014779 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d34003db0 con 0x7f2d481027e0 2024-03-04T09:30:25.506 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:25.504+0000 7f2d4eb93640 1 -- 172.21.15.52:0/643014779 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d4819fd00 con 0x7f2d481027e0 2024-03-04T09:30:25.506 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:25.505+0000 7f2d4eb93640 1 -- 172.21.15.52:0/643014779 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d48108990 con 0x7f2d481027e0 2024-03-04T09:30:25.506 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:25.506+0000 7f2d45ffb640 1 -- 172.21.15.52:0/643014779 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) v1 ==== 136756+0+0 (secure 0 0 0) 0x7f2d34044070 con 0x7f2d481027e0 2024-03-04T09:30:25.507 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:25.507+0000 7f2d45ffb640 1 --2- 172.21.15.52:0/643014779 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f2d380a1e00 0x7f2d380a42c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:30:25.507 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:25.507+0000 7f2d4c92f640 1 --2- 172.21.15.52:0/643014779 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f2d380a1e00 0x7f2d380a42c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:30:25.507 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:25.507+0000 7f2d45ffb640 1 -- 172.21.15.52:0/643014779 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9092+0+0 (secure 0 0 0) 0x7f2d340efc10 con 0x7f2d481027e0 2024-03-04T09:30:25.507 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:25.507+0000 7f2d4c92f640 1 --2- 172.21.15.52:0/643014779 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f2d380a1e00 0x7f2d380a42c0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f2d48102640 tx=0x7f2d3c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:30:25.508 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:25.511+0000 7f2d45ffb640 1 -- 172.21.15.52:0/643014779 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f2d340bcce0 con 0x7f2d481027e0 2024-03-04T09:30:25.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:30:25 smithi191 ceph-mon[36816]: pgmap v343: 129 pgs: 129 active+clean; 641 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 5 op/s 2024-03-04T09:30:25.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:30:25 smithi191 ceph-mon[36816]: from='client.? 172.21.15.52:0/2760377847' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-03-04T09:30:25.631 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:30:25 smithi106 ceph-mon[36484]: pgmap v343: 129 pgs: 129 active+clean; 641 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 5 op/s 2024-03-04T09:30:25.631 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:30:25 smithi106 ceph-mon[36484]: from='client.? 172.21.15.52:0/2760377847' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-03-04T09:30:25.689 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:25.692+0000 7f2d4eb93640 1 -- 172.21.15.52:0/643014779 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f2d480621d0 con 0x7f2d481027e0 2024-03-04T09:30:25.689 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:25.692+0000 7f2d45ffb640 1 -- 172.21.15.52:0/643014779 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v91) v1 ==== 93+0+10 (secure 0 0 0) 0x7f2d340bc5e0 con 0x7f2d481027e0 2024-03-04T09:30:25.689 INFO:teuthology.orchestra.run.smithi052.stdout:pg_num: 1 2024-03-04T09:30:25.691 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:25.694+0000 7f2d4eb93640 1 -- 172.21.15.52:0/643014779 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f2d380a1e00 msgr2=0x7f2d380a42c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:30:25.691 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:25.694+0000 7f2d4eb93640 1 --2- 172.21.15.52:0/643014779 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f2d380a1e00 0x7f2d380a42c0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f2d48102640 tx=0x7f2d3c005f90 comp rx=0 tx=0).stop 2024-03-04T09:30:25.692 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:25.694+0000 7f2d4eb93640 1 -- 172.21.15.52:0/643014779 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2d481027e0 msgr2=0x7f2d48197e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:30:25.692 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:25.694+0000 7f2d4eb93640 1 --2- 172.21.15.52:0/643014779 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2d481027e0 0x7f2d48197e50 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f2d34009230 tx=0x7f2d3402c650 comp rx=0 tx=0).stop 2024-03-04T09:30:25.692 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:25.694+0000 7f2d4eb93640 1 -- 172.21.15.52:0/643014779 shutdown_connections 2024-03-04T09:30:25.692 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:25.694+0000 7f2d4eb93640 1 --2- 172.21.15.52:0/643014779 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f2d380a1e00 0x7f2d380a42c0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:25.692 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:25.694+0000 7f2d4eb93640 1 --2- 172.21.15.52:0/643014779 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2d481031a0 0x7f2d4819eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:25.692 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:25.694+0000 7f2d4eb93640 1 --2- 172.21.15.52:0/643014779 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2d481027e0 0x7f2d48197e50 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:25.692 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:25.694+0000 7f2d4eb93640 1 --2- 172.21.15.52:0/643014779 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2d481015f0 0x7f2d48197910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:25.692 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:25.694+0000 7f2d4eb93640 1 -- 172.21.15.52:0/643014779 >> 172.21.15.52:0/643014779 conn(0x7f2d480fcb40 msgr2=0x7f2d48105b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:30:25.692 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:25.694+0000 7f2d4eb93640 1 -- 172.21.15.52:0/643014779 shutdown_connections 2024-03-04T09:30:25.692 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:25.694+0000 7f2d4eb93640 1 -- 172.21.15.52:0/643014779 wait complete. 2024-03-04T09:30:25.702 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-04T09:30:25.810 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:25.811+0000 7f3233015640 1 -- 172.21.15.52:0/3285144457 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f322c1049a0 msgr2=0x7f322c106d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:30:25.810 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:25.811+0000 7f3233015640 1 --2- 172.21.15.52:0/3285144457 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f322c1049a0 0x7f322c106d90 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f3218009900 tx=0x7f321802bc30 comp rx=0 tx=0).stop 2024-03-04T09:30:25.810 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:25.811+0000 7f3233015640 1 -- 172.21.15.52:0/3285144457 shutdown_connections 2024-03-04T09:30:25.810 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:25.811+0000 7f3233015640 1 --2- 172.21.15.52:0/3285144457 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f322c1072d0 0x7f322c109720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:25.810 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:25.811+0000 7f3233015640 1 --2- 172.21.15.52:0/3285144457 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f322c1049a0 0x7f322c106d90 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:25.810 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:25.811+0000 7f3233015640 1 --2- 172.21.15.52:0/3285144457 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f322c102070 0x7f322c104460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:25.810 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:25.811+0000 7f3233015640 1 -- 172.21.15.52:0/3285144457 >> 172.21.15.52:0/3285144457 conn(0x7f322c0fba80 msgr2=0x7f322c0fdea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:30:25.811 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:25.811+0000 7f3233015640 1 -- 172.21.15.52:0/3285144457 shutdown_connections 2024-03-04T09:30:25.811 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:25.811+0000 7f3233015640 1 -- 172.21.15.52:0/3285144457 wait complete. 2024-03-04T09:30:25.811 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:25.812+0000 7f3233015640 1 Processor -- start 2024-03-04T09:30:25.811 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:25.812+0000 7f3233015640 1 -- start start 2024-03-04T09:30:25.811 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:25.812+0000 7f3233015640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f322c102070 0x7f322c197930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:30:25.811 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:25.812+0000 7f3233015640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f322c1049a0 0x7f322c197e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:30:25.811 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:25.812+0000 7f3233015640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f322c1072d0 0x7f322c19eee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:30:25.811 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:25.812+0000 7f3233015640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f322c10d270 con 0x7f322c1049a0 2024-03-04T09:30:25.811 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:25.812+0000 7f3233015640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f322c10d0f0 con 0x7f322c1072d0 2024-03-04T09:30:25.811 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:25.812+0000 7f3233015640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f322c10d3f0 con 0x7f322c102070 2024-03-04T09:30:25.811 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:25.812+0000 7f322bfff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f322c1049a0 0x7f322c197e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:30:25.811 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:25.812+0000 7f3230db1640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f322c102070 0x7f322c197930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:30:25.812 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:25.812+0000 7f322bfff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f322c1049a0 0x7f322c197e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:52004/0 (socket says 172.21.15.52:52004) 2024-03-04T09:30:25.812 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:25.812+0000 7f322bfff640 1 -- 172.21.15.52:0/1287363241 learned_addr learned my addr 172.21.15.52:0/1287363241 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-04T09:30:25.812 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:25.812+0000 7f32315b2640 1 --2- 172.21.15.52:0/1287363241 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f322c1072d0 0x7f322c19eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:30:25.812 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:25.812+0000 7f3230db1640 1 -- 172.21.15.52:0/1287363241 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f322c1072d0 msgr2=0x7f322c19eee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:30:25.812 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:25.812+0000 7f3230db1640 1 --2- 172.21.15.52:0/1287363241 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f322c1072d0 0x7f322c19eee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:25.812 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:25.812+0000 7f3230db1640 1 -- 172.21.15.52:0/1287363241 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f322c1049a0 msgr2=0x7f322c197e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:30:25.812 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:25.812+0000 7f3230db1640 1 --2- 172.21.15.52:0/1287363241 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f322c1049a0 0x7f322c197e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:25.812 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:25.812+0000 7f3230db1640 1 -- 172.21.15.52:0/1287363241 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f322c19f550 con 0x7f322c102070 2024-03-04T09:30:25.812 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:25.812+0000 7f322bfff640 1 --2- 172.21.15.52:0/1287363241 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f322c1049a0 0x7f322c197e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-04T09:30:25.812 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:25.812+0000 7f3230db1640 1 --2- 172.21.15.52:0/1287363241 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f322c102070 0x7f322c197930 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f322000e9f0 tx=0x7f322000edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:30:25.812 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:25.813+0000 7f3229ffb640 1 -- 172.21.15.52:0/1287363241 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f322000ccc0 con 0x7f322c102070 2024-03-04T09:30:25.812 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:25.813+0000 7f3233015640 1 -- 172.21.15.52:0/1287363241 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f322c19f840 con 0x7f322c102070 2024-03-04T09:30:25.812 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:25.813+0000 7f3229ffb640 1 -- 172.21.15.52:0/1287363241 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3220005020 con 0x7f322c102070 2024-03-04T09:30:25.813 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:25.813+0000 7f3229ffb640 1 -- 172.21.15.52:0/1287363241 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3220004a00 con 0x7f322c102070 2024-03-04T09:30:25.813 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:25.813+0000 7f3233015640 1 -- 172.21.15.52:0/1287363241 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f322c19fd60 con 0x7f322c102070 2024-03-04T09:30:25.814 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:25.814+0000 7f3233015640 1 -- 172.21.15.52:0/1287363241 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f322c0fd570 con 0x7f322c102070 2024-03-04T09:30:25.814 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:25.815+0000 7f3229ffb640 1 -- 172.21.15.52:0/1287363241 <== mon.1 v2:172.21.15.191:3300/0 4 ==== mgrmap(e 18) v1 ==== 136756+0+0 (secure 0 0 0) 0x7f32200051d0 con 0x7f322c102070 2024-03-04T09:30:25.814 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:25.815+0000 7f3229ffb640 1 --2- 172.21.15.52:0/1287363241 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f321c0a1d80 0x7f321c0a4240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:30:25.814 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:25.815+0000 7f322bfff640 1 --2- 172.21.15.52:0/1287363241 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f321c0a1d80 0x7f321c0a4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:30:25.814 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:25.815+0000 7f3229ffb640 1 -- 172.21.15.52:0/1287363241 <== mon.1 v2:172.21.15.191:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9092+0+0 (secure 0 0 0) 0x7f32200ce020 con 0x7f322c102070 2024-03-04T09:30:25.814 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:25.815+0000 7f322bfff640 1 --2- 172.21.15.52:0/1287363241 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f321c0a1d80 0x7f321c0a4240 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f321802c8e0 tx=0x7f3218005b50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:30:25.816 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:25.820+0000 7f3229ffb640 1 -- 172.21.15.52:0/1287363241 <== mon.1 v2:172.21.15.191:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f322009b0f0 con 0x7f322c102070 2024-03-04T09:30:26.001 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:26.004+0000 7f3233015640 1 -- 172.21.15.52:0/1287363241 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f322c0621d0 con 0x7f322c102070 2024-03-04T09:30:26.001 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:26.004+0000 7f3229ffb640 1 -- 172.21.15.52:0/1287363241 <== mon.1 v2:172.21.15.191:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v91) v1 ==== 104+0+11 (secure 0 0 0) 0x7f322009a9f0 con 0x7f322c102070 2024-03-04T09:30:26.002 INFO:teuthology.orchestra.run.smithi052.stdout:pg_num: 32 2024-03-04T09:30:26.004 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:26.006+0000 7f3233015640 1 -- 172.21.15.52:0/1287363241 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f321c0a1d80 msgr2=0x7f321c0a4240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:30:26.004 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:26.006+0000 7f3233015640 1 --2- 172.21.15.52:0/1287363241 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f321c0a1d80 0x7f321c0a4240 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f321802c8e0 tx=0x7f3218005b50 comp rx=0 tx=0).stop 2024-03-04T09:30:26.004 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:26.006+0000 7f3233015640 1 -- 172.21.15.52:0/1287363241 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f322c102070 msgr2=0x7f322c197930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:30:26.004 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:26.006+0000 7f3233015640 1 --2- 172.21.15.52:0/1287363241 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f322c102070 0x7f322c197930 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f322000e9f0 tx=0x7f322000edf0 comp rx=0 tx=0).stop 2024-03-04T09:30:26.004 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:26.007+0000 7f3233015640 1 -- 172.21.15.52:0/1287363241 shutdown_connections 2024-03-04T09:30:26.004 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:26.007+0000 7f3233015640 1 --2- 172.21.15.52:0/1287363241 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f321c0a1d80 0x7f321c0a4240 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:26.004 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:26.007+0000 7f3233015640 1 --2- 172.21.15.52:0/1287363241 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f322c1072d0 0x7f322c19eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:26.004 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:26.007+0000 7f3233015640 1 --2- 172.21.15.52:0/1287363241 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f322c1049a0 0x7f322c197e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:26.004 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:26.007+0000 7f3233015640 1 --2- 172.21.15.52:0/1287363241 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f322c102070 0x7f322c197930 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:26.004 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:26.007+0000 7f3233015640 1 -- 172.21.15.52:0/1287363241 >> 172.21.15.52:0/1287363241 conn(0x7f322c0fba80 msgr2=0x7f322c1029d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:30:26.005 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:26.007+0000 7f3233015640 1 -- 172.21.15.52:0/1287363241 shutdown_connections 2024-03-04T09:30:26.005 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:26.007+0000 7f3233015640 1 -- 172.21.15.52:0/1287363241 wait complete. 2024-03-04T09:30:26.015 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-04T09:30:26.124 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:26.124+0000 7feded581640 1 -- 172.21.15.52:0/3938418786 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fede81020a0 msgr2=0x7fede8108930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:30:26.124 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:26.124+0000 7feded581640 1 --2- 172.21.15.52:0/3938418786 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fede81020a0 0x7fede8108930 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fede0009ca0 tx=0x7fede002c0f0 comp rx=0 tx=0).stop 2024-03-04T09:30:26.124 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:26.124+0000 7feded581640 1 -- 172.21.15.52:0/3938418786 shutdown_connections 2024-03-04T09:30:26.124 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:26.124+0000 7feded581640 1 --2- 172.21.15.52:0/3938418786 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fede81020a0 0x7fede8108930 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:26.124 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:26.124+0000 7feded581640 1 --2- 172.21.15.52:0/3938418786 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fede81016e0 0x7fede8101b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:26.124 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:26.124+0000 7feded581640 1 --2- 172.21.15.52:0/3938418786 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fede81004f0 0x7fede8100910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:26.124 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:26.124+0000 7feded581640 1 -- 172.21.15.52:0/3938418786 >> 172.21.15.52:0/3938418786 conn(0x7fede80fba80 msgr2=0x7fede80fdea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:30:26.124 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:26.124+0000 7feded581640 1 -- 172.21.15.52:0/3938418786 shutdown_connections 2024-03-04T09:30:26.124 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:26.124+0000 7feded581640 1 -- 172.21.15.52:0/3938418786 wait complete. 2024-03-04T09:30:26.124 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:26.125+0000 7feded581640 1 Processor -- start 2024-03-04T09:30:26.125 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:26.125+0000 7feded581640 1 -- start start 2024-03-04T09:30:26.125 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:26.125+0000 7feded581640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fede81004f0 0x7fede81978f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:30:26.125 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:26.125+0000 7feded581640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fede81016e0 0x7fede8197e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:30:26.125 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:26.125+0000 7feded581640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fede81020a0 0x7fede819eea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:30:26.125 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:26.125+0000 7feded581640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fede810c180 con 0x7fede81016e0 2024-03-04T09:30:26.125 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:26.125+0000 7feded581640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fede810c000 con 0x7fede81004f0 2024-03-04T09:30:26.125 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:26.125+0000 7feded581640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fede810c300 con 0x7fede81020a0 2024-03-04T09:30:26.125 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:26.125+0000 7feddffff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fede81016e0 0x7fede8197e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:30:26.125 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:26.125+0000 7feddffff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fede81016e0 0x7fede8197e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:52028/0 (socket says 172.21.15.52:52028) 2024-03-04T09:30:26.125 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:26.125+0000 7feddffff640 1 -- 172.21.15.52:0/479513724 learned_addr learned my addr 172.21.15.52:0/479513724 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-04T09:30:26.125 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:26.125+0000 7fede6ffd640 1 --2- 172.21.15.52:0/479513724 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fede81004f0 0x7fede81978f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:30:26.125 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:26.125+0000 7fede77fe640 1 --2- 172.21.15.52:0/479513724 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fede81020a0 0x7fede819eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:30:26.125 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:26.125+0000 7feddffff640 1 -- 172.21.15.52:0/479513724 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fede81020a0 msgr2=0x7fede819eea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:30:26.126 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:26.125+0000 7feddffff640 1 --2- 172.21.15.52:0/479513724 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fede81020a0 0x7fede819eea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:26.126 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:26.125+0000 7feddffff640 1 -- 172.21.15.52:0/479513724 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fede81004f0 msgr2=0x7fede81978f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:30:26.126 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:26.125+0000 7feddffff640 1 --2- 172.21.15.52:0/479513724 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fede81004f0 0x7fede81978f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:26.126 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:26.125+0000 7feddffff640 1 -- 172.21.15.52:0/479513724 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fede819f510 con 0x7fede81016e0 2024-03-04T09:30:26.126 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:26.125+0000 7fede77fe640 1 --2- 172.21.15.52:0/479513724 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fede81020a0 0x7fede819eea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T09:30:26.126 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:26.126+0000 7feddffff640 1 --2- 172.21.15.52:0/479513724 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fede81016e0 0x7fede8197e30 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fedd000e9f0 tx=0x7fedd000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:30:26.126 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:26.126+0000 7fede4ff9640 1 -- 172.21.15.52:0/479513724 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedd000cd80 con 0x7fede81016e0 2024-03-04T09:30:26.126 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:26.126+0000 7feded581640 1 -- 172.21.15.52:0/479513724 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fede819f800 con 0x7fede81016e0 2024-03-04T09:30:26.126 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:26.126+0000 7fede4ff9640 1 -- 172.21.15.52:0/479513724 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fedd0010910 con 0x7fede81016e0 2024-03-04T09:30:26.126 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:26.126+0000 7fede4ff9640 1 -- 172.21.15.52:0/479513724 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedd00188f0 con 0x7fede81016e0 2024-03-04T09:30:26.126 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:26.126+0000 7feded581640 1 -- 172.21.15.52:0/479513724 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fede819fd50 con 0x7fede81016e0 2024-03-04T09:30:26.126 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:26.127+0000 7feded581640 1 -- 172.21.15.52:0/479513724 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fede819ffb0 con 0x7fede81016e0 2024-03-04T09:30:26.127 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:26.128+0000 7fede4ff9640 1 -- 172.21.15.52:0/479513724 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) v1 ==== 136756+0+0 (secure 0 0 0) 0x7fedd0010ac0 con 0x7fede81016e0 2024-03-04T09:30:26.127 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:26.128+0000 7fede4ff9640 1 --2- 172.21.15.52:0/479513724 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7fedd40a1d80 0x7fedd40a4240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:30:26.127 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:26.128+0000 7fede4ff9640 1 -- 172.21.15.52:0/479513724 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9092+0+0 (secure 0 0 0) 0x7fedd00ce560 con 0x7fede81016e0 2024-03-04T09:30:26.128 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:26.128+0000 7fede6ffd640 1 --2- 172.21.15.52:0/479513724 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7fedd40a1d80 0x7fedd40a4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:30:26.128 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:26.129+0000 7fede6ffd640 1 --2- 172.21.15.52:0/479513724 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7fedd40a1d80 0x7fedd40a4240 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fede8101540 tx=0x7fedd8005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:30:26.130 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:26.133+0000 7fede4ff9640 1 -- 172.21.15.52:0/479513724 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fedd009b630 con 0x7fede81016e0 2024-03-04T09:30:26.310 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:26.314+0000 7feded581640 1 -- 172.21.15.52:0/479513724 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fede81a07e0 con 0x7fede81016e0 2024-03-04T09:30:26.311 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:26.314+0000 7fede4ff9640 1 -- 172.21.15.52:0/479513724 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v91) v1 ==== 100+0+11 (secure 0 0 0) 0x7fedd009af30 con 0x7fede81016e0 2024-03-04T09:30:26.311 INFO:teuthology.orchestra.run.smithi052.stdout:pg_num: 64 2024-03-04T09:30:26.313 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:26.316+0000 7feded581640 1 -- 172.21.15.52:0/479513724 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7fedd40a1d80 msgr2=0x7fedd40a4240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:30:26.313 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:26.316+0000 7feded581640 1 --2- 172.21.15.52:0/479513724 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7fedd40a1d80 0x7fedd40a4240 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fede8101540 tx=0x7fedd8005f90 comp rx=0 tx=0).stop 2024-03-04T09:30:26.313 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:26.316+0000 7feded581640 1 -- 172.21.15.52:0/479513724 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fede81016e0 msgr2=0x7fede8197e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:30:26.314 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:26.316+0000 7feded581640 1 --2- 172.21.15.52:0/479513724 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fede81016e0 0x7fede8197e30 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fedd000e9f0 tx=0x7fedd000eeb0 comp rx=0 tx=0).stop 2024-03-04T09:30:26.314 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:26.316+0000 7feded581640 1 -- 172.21.15.52:0/479513724 shutdown_connections 2024-03-04T09:30:26.314 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:26.316+0000 7feded581640 1 --2- 172.21.15.52:0/479513724 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7fedd40a1d80 0x7fedd40a4240 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:26.314 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:26.316+0000 7feded581640 1 --2- 172.21.15.52:0/479513724 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fede81020a0 0x7fede819eea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:26.314 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:26.316+0000 7feded581640 1 --2- 172.21.15.52:0/479513724 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fede81016e0 0x7fede8197e30 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:26.314 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:26.316+0000 7feded581640 1 --2- 172.21.15.52:0/479513724 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fede81004f0 0x7fede81978f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:26.314 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:26.316+0000 7feded581640 1 -- 172.21.15.52:0/479513724 >> 172.21.15.52:0/479513724 conn(0x7fede80fba80 msgr2=0x7fede80fd4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:30:26.314 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:26.316+0000 7feded581640 1 -- 172.21.15.52:0/479513724 shutdown_connections 2024-03-04T09:30:26.314 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:26.316+0000 7feded581640 1 -- 172.21.15.52:0/479513724 wait complete. 2024-03-04T09:30:26.323 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:30:26 smithi052 ceph-mon[32473]: from='client.? 172.21.15.52:0/4278522654' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-04T09:30:26.324 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:30:26 smithi052 ceph-mon[32473]: from='client.? 172.21.15.52:0/643014779' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-04T09:30:26.324 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:30:26 smithi052 ceph-mon[32473]: from='client.? 172.21.15.52:0/1287363241' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-04T09:30:26.325 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-04T09:30:26.432 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:26.433+0000 7f6a133bc640 1 -- 172.21.15.52:0/1664514713 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6a0c107310 msgr2=0x7f6a0c109700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:30:26.432 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:26.433+0000 7f6a133bc640 1 --2- 172.21.15.52:0/1664514713 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6a0c107310 0x7f6a0c109700 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f69fc009ca0 tx=0x7f69fc02c0f0 comp rx=0 tx=0).stop 2024-03-04T09:30:26.432 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:26.433+0000 7f6a133bc640 1 -- 172.21.15.52:0/1664514713 shutdown_connections 2024-03-04T09:30:26.433 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:26.433+0000 7f6a133bc640 1 --2- 172.21.15.52:0/1664514713 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6a0c107310 0x7f6a0c109700 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:26.433 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:26.433+0000 7f6a133bc640 1 --2- 172.21.15.52:0/1664514713 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6a0c1049e0 0x7f6a0c106dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:26.433 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:26.433+0000 7f6a133bc640 1 --2- 172.21.15.52:0/1664514713 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6a0c1020b0 0x7f6a0c1044a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:26.433 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:26.433+0000 7f6a133bc640 1 -- 172.21.15.52:0/1664514713 >> 172.21.15.52:0/1664514713 conn(0x7f6a0c0fba60 msgr2=0x7f6a0c0fde80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:30:26.433 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:26.434+0000 7f6a133bc640 1 -- 172.21.15.52:0/1664514713 shutdown_connections 2024-03-04T09:30:26.433 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:26.434+0000 7f6a133bc640 1 -- 172.21.15.52:0/1664514713 wait complete. 2024-03-04T09:30:26.433 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:26.434+0000 7f6a133bc640 1 Processor -- start 2024-03-04T09:30:26.433 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:26.434+0000 7f6a133bc640 1 -- start start 2024-03-04T09:30:26.433 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:26.434+0000 7f6a133bc640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6a0c1020b0 0x7f6a0c1978e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:30:26.433 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:26.434+0000 7f6a133bc640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6a0c1049e0 0x7f6a0c197e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:30:26.433 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:26.434+0000 7f6a133bc640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6a0c107310 0x7f6a0c19ee90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:30:26.433 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:26.434+0000 7f6a133bc640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a0c10d260 con 0x7f6a0c107310 2024-03-04T09:30:26.433 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:26.434+0000 7f6a133bc640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a0c10d0e0 con 0x7f6a0c1049e0 2024-03-04T09:30:26.434 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:26.434+0000 7f6a133bc640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a0c10d3e0 con 0x7f6a0c1020b0 2024-03-04T09:30:26.434 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:26.434+0000 7f6a11959640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6a0c107310 0x7f6a0c19ee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:30:26.434 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:26.434+0000 7f6a11959640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6a0c107310 0x7f6a0c19ee90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:52042/0 (socket says 172.21.15.52:52042) 2024-03-04T09:30:26.434 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:26.434+0000 7f6a11959640 1 -- 172.21.15.52:0/1081317327 learned_addr learned my addr 172.21.15.52:0/1081317327 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-04T09:30:26.434 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:26.434+0000 7f6a10957640 1 --2- 172.21.15.52:0/1081317327 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6a0c1049e0 0x7f6a0c197e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:30:26.434 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:26.434+0000 7f6a11158640 1 --2- 172.21.15.52:0/1081317327 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6a0c1020b0 0x7f6a0c1978e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:30:26.434 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:26.435+0000 7f6a11959640 1 -- 172.21.15.52:0/1081317327 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6a0c1020b0 msgr2=0x7f6a0c1978e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:30:26.434 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:26.435+0000 7f6a11959640 1 --2- 172.21.15.52:0/1081317327 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6a0c1020b0 0x7f6a0c1978e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:26.434 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:26.435+0000 7f6a11959640 1 -- 172.21.15.52:0/1081317327 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6a0c1049e0 msgr2=0x7f6a0c197e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:30:26.434 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:26.435+0000 7f6a11959640 1 --2- 172.21.15.52:0/1081317327 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6a0c1049e0 0x7f6a0c197e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:26.434 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:26.435+0000 7f6a11959640 1 -- 172.21.15.52:0/1081317327 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a0c19f500 con 0x7f6a0c107310 2024-03-04T09:30:26.434 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:26.435+0000 7f6a10957640 1 --2- 172.21.15.52:0/1081317327 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6a0c1049e0 0x7f6a0c197e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-04T09:30:26.434 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:26.435+0000 7f6a11959640 1 --2- 172.21.15.52:0/1081317327 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6a0c107310 0x7f6a0c19ee90 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f69fc037040 tx=0x7f69fc032f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:30:26.434 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:26.435+0000 7f6a0a7fc640 1 -- 172.21.15.52:0/1081317327 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69fc041020 con 0x7f6a0c107310 2024-03-04T09:30:26.435 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:26.435+0000 7f6a0a7fc640 1 -- 172.21.15.52:0/1081317327 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f69fc034cd0 con 0x7f6a0c107310 2024-03-04T09:30:26.435 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:26.435+0000 7f6a133bc640 1 -- 172.21.15.52:0/1081317327 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a0c19f790 con 0x7f6a0c107310 2024-03-04T09:30:26.435 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:26.435+0000 7f6a0a7fc640 1 -- 172.21.15.52:0/1081317327 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69fc039040 con 0x7f6a0c107310 2024-03-04T09:30:26.435 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:26.435+0000 7f6a133bc640 1 -- 172.21.15.52:0/1081317327 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a0c19fc80 con 0x7f6a0c107310 2024-03-04T09:30:26.436 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:26.436+0000 7f6a133bc640 1 -- 172.21.15.52:0/1081317327 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a0c0fd550 con 0x7f6a0c107310 2024-03-04T09:30:26.436 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:26.437+0000 7f6a0a7fc640 1 -- 172.21.15.52:0/1081317327 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) v1 ==== 136756+0+0 (secure 0 0 0) 0x7f69fc044020 con 0x7f6a0c107310 2024-03-04T09:30:26.436 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:26.438+0000 7f6a0a7fc640 1 --2- 172.21.15.52:0/1081317327 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f69f40a1d80 0x7f69f40a4240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:30:26.436 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:26.438+0000 7f6a11158640 1 --2- 172.21.15.52:0/1081317327 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f69f40a1d80 0x7f69f40a4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:30:26.436 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:26.438+0000 7f6a0a7fc640 1 -- 172.21.15.52:0/1081317327 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9092+0+0 (secure 0 0 0) 0x7f69fc0f0330 con 0x7f6a0c107310 2024-03-04T09:30:26.436 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:26.438+0000 7f6a11158640 1 --2- 172.21.15.52:0/1081317327 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f69f40a1d80 0x7f69f40a4240 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f6a04009bd0 tx=0x7f6a04005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:30:26.438 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:26.441+0000 7f6a0a7fc640 1 -- 172.21.15.52:0/1081317327 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f69fc04b020 con 0x7f6a0c107310 2024-03-04T09:30:26.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:30:26 smithi191 ceph-mon[36816]: from='client.? 172.21.15.52:0/4278522654' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-04T09:30:26.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:30:26 smithi191 ceph-mon[36816]: from='client.? 172.21.15.52:0/643014779' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-04T09:30:26.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:30:26 smithi191 ceph-mon[36816]: from='client.? 172.21.15.52:0/1287363241' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-04T09:30:26.620 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:26.622+0000 7f6a133bc640 1 -- 172.21.15.52:0/1081317327 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f6a0c02cff0 con 0x7f6a0c107310 2024-03-04T09:30:26.620 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:26.623+0000 7f6a0a7fc640 1 -- 172.21.15.52:0/1081317327 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v91) v1 ==== 103+0+11 (secure 0 0 0) 0x7f69fc0bd490 con 0x7f6a0c107310 2024-03-04T09:30:26.620 INFO:teuthology.orchestra.run.smithi052.stdout:pg_num: 32 2024-03-04T09:30:26.622 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:26.624+0000 7f6a133bc640 1 -- 172.21.15.52:0/1081317327 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f69f40a1d80 msgr2=0x7f69f40a4240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:30:26.622 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:26.624+0000 7f6a133bc640 1 --2- 172.21.15.52:0/1081317327 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f69f40a1d80 0x7f69f40a4240 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f6a04009bd0 tx=0x7f6a04005f90 comp rx=0 tx=0).stop 2024-03-04T09:30:26.622 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:26.624+0000 7f6a133bc640 1 -- 172.21.15.52:0/1081317327 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6a0c107310 msgr2=0x7f6a0c19ee90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:30:26.622 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:26.624+0000 7f6a133bc640 1 --2- 172.21.15.52:0/1081317327 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6a0c107310 0x7f6a0c19ee90 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f69fc037040 tx=0x7f69fc032f40 comp rx=0 tx=0).stop 2024-03-04T09:30:26.622 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:26.625+0000 7f6a133bc640 1 -- 172.21.15.52:0/1081317327 shutdown_connections 2024-03-04T09:30:26.622 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:26.625+0000 7f6a133bc640 1 --2- 172.21.15.52:0/1081317327 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f69f40a1d80 0x7f69f40a4240 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:26.622 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:26.625+0000 7f6a133bc640 1 --2- 172.21.15.52:0/1081317327 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6a0c107310 0x7f6a0c19ee90 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:26.622 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:26.625+0000 7f6a133bc640 1 --2- 172.21.15.52:0/1081317327 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6a0c1049e0 0x7f6a0c197e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:26.622 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:26.625+0000 7f6a133bc640 1 --2- 172.21.15.52:0/1081317327 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6a0c1020b0 0x7f6a0c1978e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:26.623 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:26.625+0000 7f6a133bc640 1 -- 172.21.15.52:0/1081317327 >> 172.21.15.52:0/1081317327 conn(0x7f6a0c0fba60 msgr2=0x7f6a0c1029b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:30:26.623 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:26.625+0000 7f6a133bc640 1 -- 172.21.15.52:0/1081317327 shutdown_connections 2024-03-04T09:30:26.623 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:26.625+0000 7f6a133bc640 1 -- 172.21.15.52:0/1081317327 wait complete. 2024-03-04T09:30:26.631 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:30:26 smithi106 ceph-mon[36484]: from='client.? 172.21.15.52:0/4278522654' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-04T09:30:26.631 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:30:26 smithi106 ceph-mon[36484]: from='client.? 172.21.15.52:0/643014779' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-04T09:30:26.631 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:30:26 smithi106 ceph-mon[36484]: from='client.? 172.21.15.52:0/1287363241' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-04T09:30:26.633 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T09:30:26.738 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:26.739+0000 7f158f94b640 1 -- 172.21.15.52:0/169184840 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1588101700 msgr2=0x7f1588101b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:30:26.738 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:26.739+0000 7f158f94b640 1 --2- 172.21.15.52:0/169184840 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1588101700 0x7f1588101b80 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f157c009900 tx=0x7f157c02bc30 comp rx=0 tx=0).stop 2024-03-04T09:30:26.738 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:26.739+0000 7f158f94b640 1 -- 172.21.15.52:0/169184840 shutdown_connections 2024-03-04T09:30:26.738 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:26.739+0000 7f158f94b640 1 --2- 172.21.15.52:0/169184840 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f15881020c0 0x7f1588108950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:26.738 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:26.739+0000 7f158f94b640 1 --2- 172.21.15.52:0/169184840 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1588101700 0x7f1588101b80 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:26.738 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:26.739+0000 7f158f94b640 1 --2- 172.21.15.52:0/169184840 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1588100510 0x7f1588100930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:26.739 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:26.739+0000 7f158f94b640 1 -- 172.21.15.52:0/169184840 >> 172.21.15.52:0/169184840 conn(0x7f15880fba80 msgr2=0x7f15880fdec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:30:26.739 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:26.739+0000 7f158f94b640 1 -- 172.21.15.52:0/169184840 shutdown_connections 2024-03-04T09:30:26.739 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:26.739+0000 7f158f94b640 1 -- 172.21.15.52:0/169184840 wait complete. 2024-03-04T09:30:26.739 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:26.739+0000 7f158f94b640 1 Processor -- start 2024-03-04T09:30:26.739 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:26.739+0000 7f158f94b640 1 -- start start 2024-03-04T09:30:26.739 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:26.740+0000 7f158f94b640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1588100510 0x7f1588197930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:30:26.739 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:26.740+0000 7f158f94b640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1588101700 0x7f1588197e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:30:26.739 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:26.740+0000 7f158f94b640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f15881020c0 0x7f158819eee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:30:26.739 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:26.740+0000 7f158f94b640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f158810c1a0 con 0x7f1588101700 2024-03-04T09:30:26.739 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:26.740+0000 7f158f94b640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f158810c020 con 0x7f1588100510 2024-03-04T09:30:26.739 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:26.740+0000 7f158f94b640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f158810c320 con 0x7f15881020c0 2024-03-04T09:30:26.739 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:26.740+0000 7f158cee6640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1588101700 0x7f1588197e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:30:26.739 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:26.740+0000 7f158cee6640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1588101700 0x7f1588197e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:52058/0 (socket says 172.21.15.52:52058) 2024-03-04T09:30:26.739 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:26.740+0000 7f158cee6640 1 -- 172.21.15.52:0/2586934638 learned_addr learned my addr 172.21.15.52:0/2586934638 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-04T09:30:26.740 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:26.740+0000 7f158d6e7640 1 --2- 172.21.15.52:0/2586934638 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1588100510 0x7f1588197930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:30:26.740 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:26.740+0000 7f158dee8640 1 --2- 172.21.15.52:0/2586934638 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f15881020c0 0x7f158819eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:30:26.740 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:26.740+0000 7f158cee6640 1 -- 172.21.15.52:0/2586934638 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f15881020c0 msgr2=0x7f158819eee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:30:26.740 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:26.740+0000 7f158cee6640 1 --2- 172.21.15.52:0/2586934638 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f15881020c0 0x7f158819eee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:26.740 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:26.740+0000 7f158cee6640 1 -- 172.21.15.52:0/2586934638 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1588100510 msgr2=0x7f1588197930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:30:26.740 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:26.740+0000 7f158cee6640 1 --2- 172.21.15.52:0/2586934638 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1588100510 0x7f1588197930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:26.740 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:26.740+0000 7f158cee6640 1 -- 172.21.15.52:0/2586934638 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f158819f550 con 0x7f1588101700 2024-03-04T09:30:26.740 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:26.740+0000 7f158d6e7640 1 --2- 172.21.15.52:0/2586934638 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1588100510 0x7f1588197930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-04T09:30:26.740 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:26.740+0000 7f158cee6640 1 --2- 172.21.15.52:0/2586934638 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1588101700 0x7f1588197e70 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f157c009230 tx=0x7f157c02c650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:30:26.740 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:26.740+0000 7f15867fc640 1 -- 172.21.15.52:0/2586934638 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f157c041070 con 0x7f1588101700 2024-03-04T09:30:26.740 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:26.740+0000 7f15867fc640 1 -- 172.21.15.52:0/2586934638 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f157c004450 con 0x7f1588101700 2024-03-04T09:30:26.740 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:26.740+0000 7f158f94b640 1 -- 172.21.15.52:0/2586934638 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f158819f7e0 con 0x7f1588101700 2024-03-04T09:30:26.740 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:26.740+0000 7f15867fc640 1 -- 172.21.15.52:0/2586934638 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f157c003e20 con 0x7f1588101700 2024-03-04T09:30:26.741 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:26.740+0000 7f158f94b640 1 -- 172.21.15.52:0/2586934638 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f158819fcd0 con 0x7f1588101700 2024-03-04T09:30:26.741 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:26.741+0000 7f158f94b640 1 -- 172.21.15.52:0/2586934638 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f158819ff30 con 0x7f1588101700 2024-03-04T09:30:26.741 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:26.742+0000 7f15867fc640 1 -- 172.21.15.52:0/2586934638 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) v1 ==== 136756+0+0 (secure 0 0 0) 0x7f157c044070 con 0x7f1588101700 2024-03-04T09:30:26.741 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:26.743+0000 7f15867fc640 1 --2- 172.21.15.52:0/2586934638 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f15700a1d80 0x7f15700a4240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:30:26.742 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:26.743+0000 7f158d6e7640 1 --2- 172.21.15.52:0/2586934638 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f15700a1d80 0x7f15700a4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:30:26.742 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:26.743+0000 7f15867fc640 1 -- 172.21.15.52:0/2586934638 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9092+0+0 (secure 0 0 0) 0x7f157c0f0050 con 0x7f1588101700 2024-03-04T09:30:26.742 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:26.743+0000 7f158d6e7640 1 --2- 172.21.15.52:0/2586934638 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f15700a1d80 0x7f15700a4240 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f1588101560 tx=0x7f1580009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:30:26.743 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:26.746+0000 7f15867fc640 1 -- 172.21.15.52:0/2586934638 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f157c0bcad0 con 0x7f1588101700 2024-03-04T09:30:26.963 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:26.966+0000 7f158f94b640 1 -- 172.21.15.52:0/2586934638 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f15881a0100 con 0x7f1588101700 2024-03-04T09:30:26.964 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:26.967+0000 7f15867fc640 1 -- 172.21.15.52:0/2586934638 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+11406 (secure 0 0 0) 0x7f157c0c0b70 con 0x7f1588101700 2024-03-04T09:30:26.964 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-03-04T09:30:26.964 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6835/1007961019","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6834","nonce":1007961019},{"type":"v1","addr":"172.21.15.52:6835","nonce":1007961019}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6837/3753881283","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6836","nonce":3753881283},{"type":"v1","addr":"172.21.15.52:6837","nonce":3753881283}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14628,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6839/3467897808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6838","nonce":3467897808},{"type":"v1","addr":"172.21.15.52:6839","nonce":3467897808}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14640,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6841/2804292291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6840","nonce":2804292291},{"type":"v1","addr":"172.21.15.52:6841","nonce":2804292291}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24407,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6833/1818480323","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6832","nonce":1818480323},{"type":"v1","addr":"172.21.15.106:6833","nonce":1818480323}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24412,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6833/765768204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":765768204},{"type":"v1","addr":"172.21.15.191:6833","nonce":765768204}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-04T09:28:37.264970+0000","modified":"2024-03-04T09:29:04.001848+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24506,"mds_1":24490,"mds_2":24473},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24440":{"gid":24440,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.106:6835/1541069897","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6834","nonce":1541069897},{"type":"v1","addr":"172.21.15.106:6835","nonce":1541069897}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24451":{"gid":24451,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.191:6835/105048699","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":105048699},{"type":"v1","addr":"172.21.15.191:6835","nonce":105048699}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24463":{"gid":24463,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.191:6837/4071064225","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6836","nonce":4071064225},{"type":"v1","addr":"172.21.15.191:6837","nonce":4071064225}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24473":{"gid":24473,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.106:6837/2901755066","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6836","nonce":2901755066},{"type":"v1","addr":"172.21.15.106:6837","nonce":2901755066}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24490":{"gid":24490,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.191:6839/2194324891","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6838","nonce":2194324891},{"type":"v1","addr":"172.21.15.191:6839","nonce":2194324891}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24506":{"gid":24506,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.106:6839/1995853093","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6838","nonce":1995853093},{"type":"v1","addr":"172.21.15.106:6839","nonce":1995853093}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24506,"qdb_cluster":[24473,24490,24506]},"id":1}]} 2024-03-04T09:30:26.964 INFO:teuthology.orchestra.run.smithi052.stderr:dumped fsmap epoch 24 2024-03-04T09:30:26.966 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:26.969+0000 7f158f94b640 1 -- 172.21.15.52:0/2586934638 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f15700a1d80 msgr2=0x7f15700a4240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:30:26.966 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:26.969+0000 7f158f94b640 1 --2- 172.21.15.52:0/2586934638 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f15700a1d80 0x7f15700a4240 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f1588101560 tx=0x7f1580009500 comp rx=0 tx=0).stop 2024-03-04T09:30:26.966 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:26.969+0000 7f158f94b640 1 -- 172.21.15.52:0/2586934638 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1588101700 msgr2=0x7f1588197e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:30:26.966 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:26.969+0000 7f158f94b640 1 --2- 172.21.15.52:0/2586934638 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1588101700 0x7f1588197e70 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f157c009230 tx=0x7f157c02c650 comp rx=0 tx=0).stop 2024-03-04T09:30:26.966 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:26.969+0000 7f158f94b640 1 -- 172.21.15.52:0/2586934638 shutdown_connections 2024-03-04T09:30:26.966 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:26.969+0000 7f158f94b640 1 --2- 172.21.15.52:0/2586934638 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f15700a1d80 0x7f15700a4240 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:26.966 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:26.969+0000 7f158f94b640 1 --2- 172.21.15.52:0/2586934638 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f15881020c0 0x7f158819eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:26.967 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:26.969+0000 7f158f94b640 1 --2- 172.21.15.52:0/2586934638 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1588101700 0x7f1588197e70 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:26.967 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:26.969+0000 7f158f94b640 1 --2- 172.21.15.52:0/2586934638 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1588100510 0x7f1588197930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:26.967 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:26.969+0000 7f158f94b640 1 -- 172.21.15.52:0/2586934638 >> 172.21.15.52:0/2586934638 conn(0x7f15880fba80 msgr2=0x7f15880fd540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:30:26.967 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:26.969+0000 7f158f94b640 1 -- 172.21.15.52:0/2586934638 shutdown_connections 2024-03-04T09:30:26.967 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:26.969+0000 7f158f94b640 1 -- 172.21.15.52:0/2586934638 wait complete. 2024-03-04T09:30:26.977 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-04T09:30:27.080 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:27.083+0000 7f2658e15640 1 -- 172.21.15.52:0/3776286090 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f26541027e0 msgr2=0x7f2654102c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:30:27.080 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:27.083+0000 7f2658e15640 1 --2- 172.21.15.52:0/3776286090 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f26541027e0 0x7f2654102c60 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f263c009900 tx=0x7f263c02bc30 comp rx=0 tx=0).stop 2024-03-04T09:30:27.080 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:27.083+0000 7f2658e15640 1 -- 172.21.15.52:0/3776286090 shutdown_connections 2024-03-04T09:30:27.081 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:27.083+0000 7f2658e15640 1 --2- 172.21.15.52:0/3776286090 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f26541031a0 0x7f2654109a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:27.081 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:27.083+0000 7f2658e15640 1 --2- 172.21.15.52:0/3776286090 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f26541027e0 0x7f2654102c60 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:27.081 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:27.083+0000 7f2658e15640 1 --2- 172.21.15.52:0/3776286090 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f26541015f0 0x7f2654101a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:27.081 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:27.083+0000 7f2658e15640 1 -- 172.21.15.52:0/3776286090 >> 172.21.15.52:0/3776286090 conn(0x7f26540fcb80 msgr2=0x7f26540fefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:30:27.081 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:27.083+0000 7f2658e15640 1 -- 172.21.15.52:0/3776286090 shutdown_connections 2024-03-04T09:30:27.081 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:27.083+0000 7f2658e15640 1 -- 172.21.15.52:0/3776286090 wait complete. 2024-03-04T09:30:27.081 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:27.084+0000 7f2658e15640 1 Processor -- start 2024-03-04T09:30:27.081 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:27.084+0000 7f2658e15640 1 -- start start 2024-03-04T09:30:27.083 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:27.084+0000 7f2658e15640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f26541015f0 0x7f2654199b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:30:27.084 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:27.084+0000 7f2658e15640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f26541027e0 0x7f265419a040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:30:27.084 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:27.084+0000 7f2658e15640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f26541031a0 0x7f26541a10b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:30:27.084 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:27.084+0000 7f2658e15640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f265410d250 con 0x7f26541027e0 2024-03-04T09:30:27.084 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:27.084+0000 7f2658e15640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f265410d0d0 con 0x7f26541031a0 2024-03-04T09:30:27.084 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:27.084+0000 7f2658e15640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f265410d3d0 con 0x7f26541015f0 2024-03-04T09:30:27.084 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:27.084+0000 7f2651d9b640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f26541027e0 0x7f265419a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:30:27.084 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:27.084+0000 7f2651d9b640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f26541027e0 0x7f265419a040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:52072/0 (socket says 172.21.15.52:52072) 2024-03-04T09:30:27.084 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:27.084+0000 7f2651d9b640 1 -- 172.21.15.52:0/3533441867 learned_addr learned my addr 172.21.15.52:0/3533441867 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-04T09:30:27.084 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:27.084+0000 7f2652d9d640 1 --2- 172.21.15.52:0/3533441867 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f26541031a0 0x7f26541a10b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:30:27.084 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:27.084+0000 7f265259c640 1 --2- 172.21.15.52:0/3533441867 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f26541015f0 0x7f2654199b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:30:27.084 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:27.084+0000 7f2651d9b640 1 -- 172.21.15.52:0/3533441867 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f26541015f0 msgr2=0x7f2654199b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:30:27.085 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:27.084+0000 7f2651d9b640 1 --2- 172.21.15.52:0/3533441867 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f26541015f0 0x7f2654199b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:27.085 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:27.084+0000 7f2651d9b640 1 -- 172.21.15.52:0/3533441867 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f26541031a0 msgr2=0x7f26541a10b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:30:27.085 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:27.084+0000 7f2651d9b640 1 --2- 172.21.15.52:0/3533441867 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f26541031a0 0x7f26541a10b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:27.085 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:27.084+0000 7f2651d9b640 1 -- 172.21.15.52:0/3533441867 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26541a1720 con 0x7f26541027e0 2024-03-04T09:30:27.085 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:27.084+0000 7f2652d9d640 1 --2- 172.21.15.52:0/3533441867 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f26541031a0 0x7f26541a10b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-04T09:30:27.085 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:27.084+0000 7f2651d9b640 1 --2- 172.21.15.52:0/3533441867 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f26541027e0 0x7f265419a040 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f263c009230 tx=0x7f263c02c650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:30:27.085 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:27.085+0000 7f264b7fe640 1 -- 172.21.15.52:0/3533441867 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f263c041070 con 0x7f26541027e0 2024-03-04T09:30:27.085 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:27.085+0000 7f2658e15640 1 -- 172.21.15.52:0/3533441867 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26541a19b0 con 0x7f26541027e0 2024-03-04T09:30:27.085 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:27.085+0000 7f264b7fe640 1 -- 172.21.15.52:0/3533441867 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f263c004450 con 0x7f26541027e0 2024-03-04T09:30:27.085 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:27.085+0000 7f264b7fe640 1 -- 172.21.15.52:0/3533441867 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f263c003e20 con 0x7f26541027e0 2024-03-04T09:30:27.085 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:27.085+0000 7f2658e15640 1 -- 172.21.15.52:0/3533441867 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26541a1ea0 con 0x7f26541027e0 2024-03-04T09:30:27.085 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:27.086+0000 7f2658e15640 1 -- 172.21.15.52:0/3533441867 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f26541a20d0 con 0x7f26541027e0 2024-03-04T09:30:27.085 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:27.087+0000 7f264b7fe640 1 -- 172.21.15.52:0/3533441867 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) v1 ==== 136756+0+0 (secure 0 0 0) 0x7f263c044070 con 0x7f26541027e0 2024-03-04T09:30:27.086 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:27.087+0000 7f264b7fe640 1 --2- 172.21.15.52:0/3533441867 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f26400a1d80 0x7f26400a4240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:30:27.086 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:27.087+0000 7f265259c640 1 --2- 172.21.15.52:0/3533441867 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f26400a1d80 0x7f26400a4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:30:27.086 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:27.087+0000 7f264b7fe640 1 -- 172.21.15.52:0/3533441867 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9092+0+0 (secure 0 0 0) 0x7f263c0efc80 con 0x7f26541027e0 2024-03-04T09:30:27.086 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:27.088+0000 7f265259c640 1 --2- 172.21.15.52:0/3533441867 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f26400a1d80 0x7f26400a4240 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f2654102640 tx=0x7f264c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:30:27.088 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:27.092+0000 7f264b7fe640 1 -- 172.21.15.52:0/3533441867 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f263c0bc700 con 0x7f26541027e0 2024-03-04T09:30:27.274 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:27.277+0000 7f2658e15640 1 -- 172.21.15.52:0/3533441867 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f26541a2860 con 0x7f26541027e0 2024-03-04T09:30:27.275 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:27.278+0000 7f264b7fe640 1 -- 172.21.15.52:0/3533441867 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) v1 ==== 74+0+24635 (secure 0 0 0) 0x7f263c0c07a0 con 0x7f26541027e0 2024-03-04T09:30:27.276 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-03-04T09:30:27.276 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":91,"fsid":"4e059f08-da08-11ee-95c6-87774f69a715","created":"2024-03-04T09:20:46.104963+0000","modified":"2024-03-04T09:29:41.058759+0000","last_up_change":"2024-03-04T09:27:49.261365+0000","last_in_change":"2024-03-04T09:27:30.024964+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-04T09:24:01.742461+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-04T09:28:35.835600+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-04T09:28:36.453099+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-04T09:28:40.168831+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"583172be-602f-4bf4-bfcb-42663cf5b277","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6802","nonce":1371221397},{"type":"v1","addr":"172.21.15.52:6803","nonce":1371221397}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6804","nonce":1371221397},{"type":"v1","addr":"172.21.15.52:6805","nonce":1371221397}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6808","nonce":1371221397},{"type":"v1","addr":"172.21.15.52:6809","nonce":1371221397}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6806","nonce":1371221397},{"type":"v1","addr":"172.21.15.52:6807","nonce":1371221397}]},"public_addr":"172.21.15.52:6803/1371221397","cluster_addr":"172.21.15.52:6805/1371221397","heartbeat_back_addr":"172.21.15.52:6809/1371221397","heartbeat_front_addr":"172.21.15.52:6807/1371221397","state":["exists","up"]},{"osd":1,"uuid":"5ae0081c-a2f6-46eb-ace0-0368208377b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":2540865537},{"type":"v1","addr":"172.21.15.106:6801","nonce":2540865537}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":2540865537},{"type":"v1","addr":"172.21.15.106:6803","nonce":2540865537}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":2540865537},{"type":"v1","addr":"172.21.15.106:6807","nonce":2540865537}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":2540865537},{"type":"v1","addr":"172.21.15.106:6805","nonce":2540865537}]},"public_addr":"172.21.15.106:6801/2540865537","cluster_addr":"172.21.15.106:6803/2540865537","heartbeat_back_addr":"172.21.15.106:6807/2540865537","heartbeat_front_addr":"172.21.15.106:6805/2540865537","state":["exists","up"]},{"osd":2,"uuid":"1948bac7-2bb2-4218-8b93-4e4743a0de04","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":1377083164},{"type":"v1","addr":"172.21.15.191:6801","nonce":1377083164}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":1377083164},{"type":"v1","addr":"172.21.15.191:6803","nonce":1377083164}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":1377083164},{"type":"v1","addr":"172.21.15.191:6807","nonce":1377083164}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":1377083164},{"type":"v1","addr":"172.21.15.191:6805","nonce":1377083164}]},"public_addr":"172.21.15.191:6801/1377083164","cluster_addr":"172.21.15.191:6803/1377083164","heartbeat_back_addr":"172.21.15.191:6807/1377083164","heartbeat_front_addr":"172.21.15.191:6805/1377083164","state":["exists","up"]},{"osd":3,"uuid":"16c9e9fe-7167-4992-bf12-1a7398e0498a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6810","nonce":2388571810},{"type":"v1","addr":"172.21.15.52:6811","nonce":2388571810}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6812","nonce":2388571810},{"type":"v1","addr":"172.21.15.52:6813","nonce":2388571810}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6816","nonce":2388571810},{"type":"v1","addr":"172.21.15.52:6817","nonce":2388571810}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6814","nonce":2388571810},{"type":"v1","addr":"172.21.15.52:6815","nonce":2388571810}]},"public_addr":"172.21.15.52:6811/2388571810","cluster_addr":"172.21.15.52:6813/2388571810","heartbeat_back_addr":"172.21.15.52:6817/2388571810","heartbeat_front_addr":"172.21.15.52:6815/2388571810","state":["exists","up"]},{"osd":4,"uuid":"c8ef48c7-7c78-44d3-a9e5-99ec750abd3d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6808","nonce":3465868015},{"type":"v1","addr":"172.21.15.106:6809","nonce":3465868015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6810","nonce":3465868015},{"type":"v1","addr":"172.21.15.106:6811","nonce":3465868015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6814","nonce":3465868015},{"type":"v1","addr":"172.21.15.106:6815","nonce":3465868015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6812","nonce":3465868015},{"type":"v1","addr":"172.21.15.106:6813","nonce":3465868015}]},"public_addr":"172.21.15.106:6809/3465868015","cluster_addr":"172.21.15.106:6811/3465868015","heartbeat_back_addr":"172.21.15.106:6815/3465868015","heartbeat_front_addr":"172.21.15.106:6813/3465868015","state":["exists","up"]},{"osd":5,"uuid":"9e41824a-c5a2-4f88-be50-2ba1dceb3220","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":1485092643},{"type":"v1","addr":"172.21.15.191:6809","nonce":1485092643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":1485092643},{"type":"v1","addr":"172.21.15.191:6811","nonce":1485092643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":1485092643},{"type":"v1","addr":"172.21.15.191:6815","nonce":1485092643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":1485092643},{"type":"v1","addr":"172.21.15.191:6813","nonce":1485092643}]},"public_addr":"172.21.15.191:6809/1485092643","cluster_addr":"172.21.15.191:6811/1485092643","heartbeat_back_addr":"172.21.15.191:6815/1485092643","heartbeat_front_addr":"172.21.15.191:6813/1485092643","state":["exists","up"]},{"osd":6,"uuid":"b35dccee-ec16-452e-ac69-3ac2fb7b27fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6818","nonce":1989797856},{"type":"v1","addr":"172.21.15.52:6819","nonce":1989797856}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6820","nonce":1989797856},{"type":"v1","addr":"172.21.15.52:6821","nonce":1989797856}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6824","nonce":1989797856},{"type":"v1","addr":"172.21.15.52:6825","nonce":1989797856}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6822","nonce":1989797856},{"type":"v1","addr":"172.21.15.52:6823","nonce":1989797856}]},"public_addr":"172.21.15.52:6819/1989797856","cluster_addr":"172.21.15.52:6821/1989797856","heartbeat_back_addr":"172.21.15.52:6825/1989797856","heartbeat_front_addr":"172.21.15.52:6823/1989797856","state":["exists","up"]},{"osd":7,"uuid":"71303e3d-14ec-4cbd-b19c-67645a1de3c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6816","nonce":1450870329},{"type":"v1","addr":"172.21.15.106:6817","nonce":1450870329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":1450870329},{"type":"v1","addr":"172.21.15.106:6819","nonce":1450870329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":1450870329},{"type":"v1","addr":"172.21.15.106:6823","nonce":1450870329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6820","nonce":1450870329},{"type":"v1","addr":"172.21.15.106:6821","nonce":1450870329}]},"public_addr":"172.21.15.106:6817/1450870329","cluster_addr":"172.21.15.106:6819/1450870329","heartbeat_back_addr":"172.21.15.106:6823/1450870329","heartbeat_front_addr":"172.21.15.106:6821/1450870329","state":["exists","up"]},{"osd":8,"uuid":"6f5786c4-4402-435a-bdad-adff46fe9473","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":1114119072},{"type":"v1","addr":"172.21.15.191:6817","nonce":1114119072}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6818","nonce":1114119072},{"type":"v1","addr":"172.21.15.191:6819","nonce":1114119072}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":1114119072},{"type":"v1","addr":"172.21.15.191:6823","nonce":1114119072}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":1114119072},{"type":"v1","addr":"172.21.15.191:6821","nonce":1114119072}]},"public_addr":"172.21.15.191:6817/1114119072","cluster_addr":"172.21.15.191:6819/1114119072","heartbeat_back_addr":"172.21.15.191:6823/1114119072","heartbeat_front_addr":"172.21.15.191:6821/1114119072","state":["exists","up"]},{"osd":9,"uuid":"062bbaa5-1227-4684-b859-450f5594a90c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6826","nonce":3485821914},{"type":"v1","addr":"172.21.15.52:6827","nonce":3485821914}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6828","nonce":3485821914},{"type":"v1","addr":"172.21.15.52:6829","nonce":3485821914}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6832","nonce":3485821914},{"type":"v1","addr":"172.21.15.52:6833","nonce":3485821914}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6830","nonce":3485821914},{"type":"v1","addr":"172.21.15.52:6831","nonce":3485821914}]},"public_addr":"172.21.15.52:6827/3485821914","cluster_addr":"172.21.15.52:6829/3485821914","heartbeat_back_addr":"172.21.15.52:6833/3485821914","heartbeat_front_addr":"172.21.15.52:6831/3485821914","state":["exists","up"]},{"osd":10,"uuid":"1a285a5a-f793-48da-a8eb-5326d9528cd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":3815586576},{"type":"v1","addr":"172.21.15.106:6825","nonce":3815586576}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6826","nonce":3815586576},{"type":"v1","addr":"172.21.15.106:6827","nonce":3815586576}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":3815586576},{"type":"v1","addr":"172.21.15.106:6831","nonce":3815586576}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":3815586576},{"type":"v1","addr":"172.21.15.106:6829","nonce":3815586576}]},"public_addr":"172.21.15.106:6825/3815586576","cluster_addr":"172.21.15.106:6827/3815586576","heartbeat_back_addr":"172.21.15.106:6831/3815586576","heartbeat_front_addr":"172.21.15.106:6829/3815586576","state":["exists","up"]},{"osd":11,"uuid":"1f8b9a43-57fa-41f4-9969-3b803a74bf45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":2009632843},{"type":"v1","addr":"172.21.15.191:6825","nonce":2009632843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":2009632843},{"type":"v1","addr":"172.21.15.191:6827","nonce":2009632843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":2009632843},{"type":"v1","addr":"172.21.15.191:6831","nonce":2009632843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":2009632843},{"type":"v1","addr":"172.21.15.191:6829","nonce":2009632843}]},"public_addr":"172.21.15.191:6825/2009632843","cluster_addr":"172.21.15.191:6827/2009632843","heartbeat_back_addr":"172.21.15.191:6831/2009632843","heartbeat_front_addr":"172.21.15.191:6829/2009632843","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T09:23:10.894763+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T09:23:34.420055+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T09:23:58.288324+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T09:24:23.673457+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T09:24:48.208561+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T09:25:13.405473+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T09:25:38.670915+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T09:26:04.451257+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T09:26:30.281380+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T09:26:56.543143+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T09:27:22.412930+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T09:27:48.003866+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.20","mappings":[{"from":5,"to":9}]},{"pgid":"3.21","mappings":[{"from":4,"to":6}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]},{"pgid":"4.9","mappings":[{"from":5,"to":8}]},{"pgid":"4.17","mappings":[{"from":9,"to":8}]},{"pgid":"4.1b","mappings":[{"from":9,"to":10}]},{"pgid":"4.1f","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.52:0/4082645718":"2024-03-05T09:21:39.401935+0000","172.21.15.52:0/1271949988":"2024-03-05T09:21:39.401935+0000","172.21.15.52:6801/2379361333":"2024-03-05T09:21:12.760938+0000","172.21.15.52:0/497075075":"2024-03-05T09:21:12.760938+0000","172.21.15.52:6800/2379361333":"2024-03-05T09:21:12.760938+0000","172.21.15.52:0/568242577":"2024-03-05T09:21:12.760938+0000","172.21.15.52:6801/2425730636":"2024-03-05T09:21:39.401935+0000","172.21.15.52:6800/2425730636":"2024-03-05T09:21:39.401935+0000","172.21.15.52:0/2743084243":"2024-03-05T09:21:39.401935+0000","172.21.15.52:0/1035878034":"2024-03-05T09:21:12.760938+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-04T09:30:27.277 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:27.280+0000 7f2658e15640 1 -- 172.21.15.52:0/3533441867 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f26400a1d80 msgr2=0x7f26400a4240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:30:27.277 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:27.280+0000 7f2658e15640 1 --2- 172.21.15.52:0/3533441867 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f26400a1d80 0x7f26400a4240 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f2654102640 tx=0x7f264c005f90 comp rx=0 tx=0).stop 2024-03-04T09:30:27.277 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:27.280+0000 7f2658e15640 1 -- 172.21.15.52:0/3533441867 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f26541027e0 msgr2=0x7f265419a040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:30:27.278 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:27.280+0000 7f2658e15640 1 --2- 172.21.15.52:0/3533441867 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f26541027e0 0x7f265419a040 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f263c009230 tx=0x7f263c02c650 comp rx=0 tx=0).stop 2024-03-04T09:30:27.278 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:27.280+0000 7f2658e15640 1 -- 172.21.15.52:0/3533441867 shutdown_connections 2024-03-04T09:30:27.278 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:27.280+0000 7f2658e15640 1 --2- 172.21.15.52:0/3533441867 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f26400a1d80 0x7f26400a4240 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:27.278 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:27.280+0000 7f2658e15640 1 --2- 172.21.15.52:0/3533441867 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f26541031a0 0x7f26541a10b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:27.278 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:27.280+0000 7f2658e15640 1 --2- 172.21.15.52:0/3533441867 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f26541027e0 0x7f265419a040 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:27.278 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:27.280+0000 7f2658e15640 1 --2- 172.21.15.52:0/3533441867 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f26541015f0 0x7f2654199b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:27.278 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:27.280+0000 7f2658e15640 1 -- 172.21.15.52:0/3533441867 >> 172.21.15.52:0/3533441867 conn(0x7f26540fcb80 msgr2=0x7f2654073800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:30:27.278 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:27.281+0000 7f2658e15640 1 -- 172.21.15.52:0/3533441867 shutdown_connections 2024-03-04T09:30:27.278 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:27.281+0000 7f2658e15640 1 -- 172.21.15.52:0/3533441867 wait complete. 2024-03-04T09:30:27.288 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:30:27 smithi052 ceph-mon[32473]: pgmap v344: 129 pgs: 129 active+clean; 641 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 0 B/s wr, 6 op/s 2024-03-04T09:30:27.288 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:30:27 smithi052 ceph-mon[32473]: from='client.? 172.21.15.52:0/479513724' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-04T09:30:27.288 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:30:27 smithi052 ceph-mon[32473]: from='client.? 172.21.15.52:0/1081317327' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-04T09:30:27.288 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:30:27 smithi052 ceph-mon[32473]: from='client.? 172.21.15.52:0/2586934638' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-04T09:30:27.289 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-04T09:24:01.742461+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-04T09:28:35.835600+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-04T09:28:36.453099+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-04T09:28:40.168831+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '89', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-04T09:30:27.290 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-04T09:30:27.398 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:27.399+0000 7f118dd83640 1 -- 172.21.15.52:0/439454376 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f11881027d0 msgr2=0x7f1188102c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:30:27.398 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:27.399+0000 7f118dd83640 1 --2- 172.21.15.52:0/439454376 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f11881027d0 0x7f1188102c50 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f1170009900 tx=0x7f117002bc30 comp rx=0 tx=0).stop 2024-03-04T09:30:27.398 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:27.399+0000 7f118dd83640 1 -- 172.21.15.52:0/439454376 shutdown_connections 2024-03-04T09:30:27.398 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:27.399+0000 7f118dd83640 1 --2- 172.21.15.52:0/439454376 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1188103190 0x7f1188109a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:27.399 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:27.399+0000 7f118dd83640 1 --2- 172.21.15.52:0/439454376 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f11881027d0 0x7f1188102c50 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:27.399 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:27.399+0000 7f118dd83640 1 --2- 172.21.15.52:0/439454376 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f11881015e0 0x7f1188101a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:27.399 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:27.399+0000 7f118dd83640 1 -- 172.21.15.52:0/439454376 >> 172.21.15.52:0/439454376 conn(0x7f11880fcb70 msgr2=0x7f11880fef90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:30:27.399 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:27.399+0000 7f118dd83640 1 -- 172.21.15.52:0/439454376 shutdown_connections 2024-03-04T09:30:27.399 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:27.399+0000 7f118dd83640 1 -- 172.21.15.52:0/439454376 wait complete. 2024-03-04T09:30:27.399 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:27.399+0000 7f118dd83640 1 Processor -- start 2024-03-04T09:30:27.399 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:27.400+0000 7f118dd83640 1 -- start start 2024-03-04T09:30:27.399 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:27.400+0000 7f118dd83640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f11881015e0 0x7f1188197910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:30:27.399 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:27.400+0000 7f118dd83640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f11881027d0 0x7f1188197e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:30:27.399 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:27.400+0000 7f118dd83640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1188103190 0x7f118819eec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:30:27.399 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:27.400+0000 7f118dd83640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f118810d2a0 con 0x7f11881027d0 2024-03-04T09:30:27.399 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:27.400+0000 7f118dd83640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f118810d120 con 0x7f1188103190 2024-03-04T09:30:27.400 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:27.400+0000 7f118dd83640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f118810d420 con 0x7f11881015e0 2024-03-04T09:30:27.400 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:27.400+0000 7f1186ffd640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f11881027d0 0x7f1188197e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:30:27.400 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:27.400+0000 7f1186ffd640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f11881027d0 0x7f1188197e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:52102/0 (socket says 172.21.15.52:52102) 2024-03-04T09:30:27.400 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:27.400+0000 7f1186ffd640 1 -- 172.21.15.52:0/3938195849 learned_addr learned my addr 172.21.15.52:0/3938195849 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-04T09:30:27.400 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:27.400+0000 7f11877fe640 1 --2- 172.21.15.52:0/3938195849 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f11881015e0 0x7f1188197910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:30:27.400 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:27.400+0000 7f1187fff640 1 --2- 172.21.15.52:0/3938195849 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1188103190 0x7f118819eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:30:27.400 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:27.400+0000 7f1186ffd640 1 -- 172.21.15.52:0/3938195849 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f11881015e0 msgr2=0x7f1188197910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:30:27.400 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:27.400+0000 7f1186ffd640 1 --2- 172.21.15.52:0/3938195849 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f11881015e0 0x7f1188197910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:27.401 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:27.400+0000 7f1186ffd640 1 -- 172.21.15.52:0/3938195849 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1188103190 msgr2=0x7f118819eec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:30:27.401 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:27.400+0000 7f1186ffd640 1 --2- 172.21.15.52:0/3938195849 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1188103190 0x7f118819eec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:27.401 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:27.400+0000 7f1186ffd640 1 -- 172.21.15.52:0/3938195849 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f118819f530 con 0x7f11881027d0 2024-03-04T09:30:27.401 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:27.400+0000 7f1186ffd640 1 --2- 172.21.15.52:0/3938195849 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f11881027d0 0x7f1188197e50 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f1170000c00 tx=0x7f117002c650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:30:27.401 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:27.400+0000 7f1184ff9640 1 -- 172.21.15.52:0/3938195849 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1170043020 con 0x7f11881027d0 2024-03-04T09:30:27.401 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:27.400+0000 7f1184ff9640 1 -- 172.21.15.52:0/3938195849 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f11700054e0 con 0x7f11881027d0 2024-03-04T09:30:27.401 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:27.400+0000 7f118dd83640 1 -- 172.21.15.52:0/3938195849 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f118819f7c0 con 0x7f11881027d0 2024-03-04T09:30:27.401 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:27.400+0000 7f1184ff9640 1 -- 172.21.15.52:0/3938195849 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1170003c40 con 0x7f11881027d0 2024-03-04T09:30:27.401 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:27.401+0000 7f118dd83640 1 -- 172.21.15.52:0/3938195849 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f118819fd10 con 0x7f11881027d0 2024-03-04T09:30:27.401 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:27.401+0000 7f118dd83640 1 -- 172.21.15.52:0/3938195849 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1188108980 con 0x7f11881027d0 2024-03-04T09:30:27.403 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:27.402+0000 7f1184ff9640 1 -- 172.21.15.52:0/3938195849 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) v1 ==== 136756+0+0 (secure 0 0 0) 0x7f1170046020 con 0x7f11881027d0 2024-03-04T09:30:27.404 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:27.403+0000 7f1184ff9640 1 --2- 172.21.15.52:0/3938195849 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f11740a1d80 0x7f11740a4240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:30:27.404 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:27.403+0000 7f11877fe640 1 --2- 172.21.15.52:0/3938195849 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f11740a1d80 0x7f11740a4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:30:27.404 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:27.403+0000 7f1184ff9640 1 -- 172.21.15.52:0/3938195849 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9092+0+0 (secure 0 0 0) 0x7f11700f00f0 con 0x7f11881027d0 2024-03-04T09:30:27.404 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:27.404+0000 7f11877fe640 1 --2- 172.21.15.52:0/3938195849 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f11740a1d80 0x7f11740a4240 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f1188102630 tx=0x7f1178009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:30:27.404 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:27.408+0000 7f1184ff9640 1 -- 172.21.15.52:0/3938195849 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f117004d020 con 0x7f11881027d0 2024-03-04T09:30:27.584 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:27.587+0000 7f118dd83640 1 -- 172.21.15.52:0/3938195849 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f11880621d0 con 0x7f11881027d0 2024-03-04T09:30:27.585 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:27.588+0000 7f1184ff9640 1 -- 172.21.15.52:0/3938195849 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v91) v1 ==== 93+0+10 (secure 0 0 0) 0x7f11700bd1c0 con 0x7f11881027d0 2024-03-04T09:30:27.585 INFO:teuthology.orchestra.run.smithi052.stdout:pg_num: 1 2024-03-04T09:30:27.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:30:27 smithi191 ceph-mon[36816]: pgmap v344: 129 pgs: 129 active+clean; 641 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 0 B/s wr, 6 op/s 2024-03-04T09:30:27.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:30:27 smithi191 ceph-mon[36816]: from='client.? 172.21.15.52:0/479513724' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-04T09:30:27.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:30:27 smithi191 ceph-mon[36816]: from='client.? 172.21.15.52:0/1081317327' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-04T09:30:27.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:30:27 smithi191 ceph-mon[36816]: from='client.? 172.21.15.52:0/2586934638' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-04T09:30:27.587 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:27.590+0000 7f118dd83640 1 -- 172.21.15.52:0/3938195849 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f11740a1d80 msgr2=0x7f11740a4240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:30:27.587 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:27.590+0000 7f118dd83640 1 --2- 172.21.15.52:0/3938195849 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f11740a1d80 0x7f11740a4240 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f1188102630 tx=0x7f1178009500 comp rx=0 tx=0).stop 2024-03-04T09:30:27.587 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:27.590+0000 7f118dd83640 1 -- 172.21.15.52:0/3938195849 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f11881027d0 msgr2=0x7f1188197e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:30:27.587 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:27.590+0000 7f118dd83640 1 --2- 172.21.15.52:0/3938195849 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f11881027d0 0x7f1188197e50 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f1170000c00 tx=0x7f117002c650 comp rx=0 tx=0).stop 2024-03-04T09:30:27.587 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:27.590+0000 7f118dd83640 1 -- 172.21.15.52:0/3938195849 shutdown_connections 2024-03-04T09:30:27.587 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:27.590+0000 7f118dd83640 1 --2- 172.21.15.52:0/3938195849 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f11740a1d80 0x7f11740a4240 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:27.587 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:27.590+0000 7f118dd83640 1 --2- 172.21.15.52:0/3938195849 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1188103190 0x7f118819eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:27.587 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:27.590+0000 7f118dd83640 1 --2- 172.21.15.52:0/3938195849 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f11881027d0 0x7f1188197e50 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:27.587 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:27.590+0000 7f118dd83640 1 --2- 172.21.15.52:0/3938195849 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f11881015e0 0x7f1188197910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:27.587 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:27.590+0000 7f118dd83640 1 -- 172.21.15.52:0/3938195849 >> 172.21.15.52:0/3938195849 conn(0x7f11880fcb70 msgr2=0x7f1188105b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:30:27.587 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:27.590+0000 7f118dd83640 1 -- 172.21.15.52:0/3938195849 shutdown_connections 2024-03-04T09:30:27.588 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:27.590+0000 7f118dd83640 1 -- 172.21.15.52:0/3938195849 wait complete. 2024-03-04T09:30:27.598 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-04T09:30:27.631 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:30:27 smithi106 ceph-mon[36484]: pgmap v344: 129 pgs: 129 active+clean; 641 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 0 B/s wr, 6 op/s 2024-03-04T09:30:27.631 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:30:27 smithi106 ceph-mon[36484]: from='client.? 172.21.15.52:0/479513724' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-04T09:30:27.631 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:30:27 smithi106 ceph-mon[36484]: from='client.? 172.21.15.52:0/1081317327' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-04T09:30:27.631 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:30:27 smithi106 ceph-mon[36484]: from='client.? 172.21.15.52:0/2586934638' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-04T09:30:27.705 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:27.706+0000 7f8828f5c640 1 -- 172.21.15.52:0/2770816165 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8824102080 msgr2=0x7f8824108910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:30:27.706 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:27.706+0000 7f8828f5c640 1 --2- 172.21.15.52:0/2770816165 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8824102080 0x7f8824108910 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f8814009ca0 tx=0x7f881402c0f0 comp rx=0 tx=0).stop 2024-03-04T09:30:27.706 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:27.706+0000 7f8828f5c640 1 -- 172.21.15.52:0/2770816165 shutdown_connections 2024-03-04T09:30:27.706 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:27.706+0000 7f8828f5c640 1 --2- 172.21.15.52:0/2770816165 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8824102080 0x7f8824108910 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:27.706 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:27.706+0000 7f8828f5c640 1 --2- 172.21.15.52:0/2770816165 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f88241016c0 0x7f8824101b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:27.706 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:27.706+0000 7f8828f5c640 1 --2- 172.21.15.52:0/2770816165 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f88241004d0 0x7f88241008f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:27.706 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:27.706+0000 7f8828f5c640 1 -- 172.21.15.52:0/2770816165 >> 172.21.15.52:0/2770816165 conn(0x7f88240fba60 msgr2=0x7f88240fde80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:30:27.706 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:27.706+0000 7f8828f5c640 1 -- 172.21.15.52:0/2770816165 shutdown_connections 2024-03-04T09:30:27.706 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:27.706+0000 7f8828f5c640 1 -- 172.21.15.52:0/2770816165 wait complete. 2024-03-04T09:30:27.706 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:27.707+0000 7f8828f5c640 1 Processor -- start 2024-03-04T09:30:27.706 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:27.707+0000 7f8828f5c640 1 -- start start 2024-03-04T09:30:27.706 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:27.707+0000 7f8828f5c640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f88241004d0 0x7f8824104460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:30:27.707 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:27.707+0000 7f8828f5c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f88241016c0 0x7f8824102aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:30:27.707 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:27.707+0000 7f8828f5c640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8824102080 0x7f8824102fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:30:27.707 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:27.707+0000 7f8828f5c640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f882410c160 con 0x7f8824102080 2024-03-04T09:30:27.707 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:27.707+0000 7f8828f5c640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f882410bfe0 con 0x7f88241016c0 2024-03-04T09:30:27.707 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:27.707+0000 7f8828f5c640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f882410c2e0 con 0x7f88241004d0 2024-03-04T09:30:27.707 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:27.708+0000 7f8822d9d640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8824102080 0x7f8824102fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:30:27.707 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:27.708+0000 7f8822d9d640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8824102080 0x7f8824102fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:52122/0 (socket says 172.21.15.52:52122) 2024-03-04T09:30:27.707 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:27.708+0000 7f8822d9d640 1 -- 172.21.15.52:0/3744334532 learned_addr learned my addr 172.21.15.52:0/3744334532 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-04T09:30:27.707 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:27.708+0000 7f8821d9b640 1 --2- 172.21.15.52:0/3744334532 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f88241016c0 0x7f8824102aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:30:27.707 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:27.708+0000 7f882259c640 1 --2- 172.21.15.52:0/3744334532 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f88241004d0 0x7f8824104460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:30:27.707 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:27.708+0000 7f8822d9d640 1 -- 172.21.15.52:0/3744334532 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f88241004d0 msgr2=0x7f8824104460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:30:27.707 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:27.708+0000 7f8822d9d640 1 --2- 172.21.15.52:0/3744334532 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f88241004d0 0x7f8824104460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:27.707 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:27.708+0000 7f8822d9d640 1 -- 172.21.15.52:0/3744334532 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f88241016c0 msgr2=0x7f8824102aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:30:27.708 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:27.708+0000 7f8822d9d640 1 --2- 172.21.15.52:0/3744334532 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f88241016c0 0x7f8824102aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:27.708 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:27.708+0000 7f8822d9d640 1 -- 172.21.15.52:0/3744334532 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8824103870 con 0x7f8824102080 2024-03-04T09:30:27.708 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:27.708+0000 7f8822d9d640 1 --2- 172.21.15.52:0/3744334532 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8824102080 0x7f8824102fe0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f8814037040 tx=0x7f8814032f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:30:27.708 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:27.708+0000 7f881b7fe640 1 -- 172.21.15.52:0/3744334532 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8814041020 con 0x7f8824102080 2024-03-04T09:30:27.708 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:27.708+0000 7f8828f5c640 1 -- 172.21.15.52:0/3744334532 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88241a1990 con 0x7f8824102080 2024-03-04T09:30:27.708 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:27.708+0000 7f881b7fe640 1 -- 172.21.15.52:0/3744334532 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8814034cd0 con 0x7f8824102080 2024-03-04T09:30:27.708 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:27.708+0000 7f881b7fe640 1 -- 172.21.15.52:0/3744334532 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8814039040 con 0x7f8824102080 2024-03-04T09:30:27.708 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:27.708+0000 7f8828f5c640 1 -- 172.21.15.52:0/3744334532 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88241a1f30 con 0x7f8824102080 2024-03-04T09:30:27.709 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:27.709+0000 7f8828f5c640 1 -- 172.21.15.52:0/3744334532 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f88241a2100 con 0x7f8824102080 2024-03-04T09:30:27.709 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:27.710+0000 7f881b7fe640 1 -- 172.21.15.52:0/3744334532 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) v1 ==== 136756+0+0 (secure 0 0 0) 0x7f8814044020 con 0x7f8824102080 2024-03-04T09:30:27.709 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:27.711+0000 7f881b7fe640 1 --2- 172.21.15.52:0/3744334532 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f88100a1d80 0x7f88100a4240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:30:27.709 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:27.711+0000 7f882259c640 1 --2- 172.21.15.52:0/3744334532 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f88100a1d80 0x7f88100a4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:30:27.709 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:27.711+0000 7f881b7fe640 1 -- 172.21.15.52:0/3744334532 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9092+0+0 (secure 0 0 0) 0x7f88140eff10 con 0x7f8824102080 2024-03-04T09:30:27.709 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:27.711+0000 7f882259c640 1 --2- 172.21.15.52:0/3744334532 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f88100a1d80 0x7f88100a4240 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f8824101520 tx=0x7f881c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:30:27.712 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:27.715+0000 7f881b7fe640 1 -- 172.21.15.52:0/3744334532 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f881404b020 con 0x7f8824102080 2024-03-04T09:30:27.892 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:27.895+0000 7f8828f5c640 1 -- 172.21.15.52:0/3744334532 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f88241a22d0 con 0x7f8824102080 2024-03-04T09:30:27.892 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:27.895+0000 7f881b7fe640 1 -- 172.21.15.52:0/3744334532 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v91) v1 ==== 104+0+11 (secure 0 0 0) 0x7f88140bd070 con 0x7f8824102080 2024-03-04T09:30:27.892 INFO:teuthology.orchestra.run.smithi052.stdout:pg_num: 32 2024-03-04T09:30:27.894 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:27.897+0000 7f8828f5c640 1 -- 172.21.15.52:0/3744334532 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f88100a1d80 msgr2=0x7f88100a4240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:30:27.894 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:27.897+0000 7f8828f5c640 1 --2- 172.21.15.52:0/3744334532 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f88100a1d80 0x7f88100a4240 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f8824101520 tx=0x7f881c005f90 comp rx=0 tx=0).stop 2024-03-04T09:30:27.894 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:27.897+0000 7f8828f5c640 1 -- 172.21.15.52:0/3744334532 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8824102080 msgr2=0x7f8824102fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:30:27.894 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:27.897+0000 7f8828f5c640 1 --2- 172.21.15.52:0/3744334532 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8824102080 0x7f8824102fe0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f8814037040 tx=0x7f8814032f40 comp rx=0 tx=0).stop 2024-03-04T09:30:27.895 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:27.897+0000 7f8828f5c640 1 -- 172.21.15.52:0/3744334532 shutdown_connections 2024-03-04T09:30:27.895 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:27.897+0000 7f8828f5c640 1 --2- 172.21.15.52:0/3744334532 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f88100a1d80 0x7f88100a4240 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:27.895 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:27.897+0000 7f8828f5c640 1 --2- 172.21.15.52:0/3744334532 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8824102080 0x7f8824102fe0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:27.895 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:27.897+0000 7f8828f5c640 1 --2- 172.21.15.52:0/3744334532 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f88241016c0 0x7f8824102aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:27.895 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:27.897+0000 7f8828f5c640 1 --2- 172.21.15.52:0/3744334532 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f88241004d0 0x7f8824104460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:27.895 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:27.897+0000 7f8828f5c640 1 -- 172.21.15.52:0/3744334532 >> 172.21.15.52:0/3744334532 conn(0x7f88240fba60 msgr2=0x7f88240fd4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:30:27.895 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:27.897+0000 7f8828f5c640 1 -- 172.21.15.52:0/3744334532 shutdown_connections 2024-03-04T09:30:27.895 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:27.897+0000 7f8828f5c640 1 -- 172.21.15.52:0/3744334532 wait complete. 2024-03-04T09:30:27.906 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-04T09:30:28.014 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:28.014+0000 7f9e0f377640 1 -- 172.21.15.52:0/1664714299 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9e081015f0 msgr2=0x7f9e08101a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:30:28.014 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:28.014+0000 7f9e0f377640 1 --2- 172.21.15.52:0/1664714299 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9e081015f0 0x7f9e08101a10 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f9e00009900 tx=0x7f9e0002bc30 comp rx=0 tx=0).stop 2024-03-04T09:30:28.014 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:28.014+0000 7f9e0f377640 1 -- 172.21.15.52:0/1664714299 shutdown_connections 2024-03-04T09:30:28.014 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:28.014+0000 7f9e0f377640 1 --2- 172.21.15.52:0/1664714299 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9e081031a0 0x7f9e08109a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:28.014 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:28.014+0000 7f9e0f377640 1 --2- 172.21.15.52:0/1664714299 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9e081027e0 0x7f9e08102c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:28.014 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:28.014+0000 7f9e0f377640 1 --2- 172.21.15.52:0/1664714299 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9e081015f0 0x7f9e08101a10 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:28.014 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:28.014+0000 7f9e0f377640 1 -- 172.21.15.52:0/1664714299 >> 172.21.15.52:0/1664714299 conn(0x7f9e080fcb60 msgr2=0x7f9e080fefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:30:28.014 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:28.014+0000 7f9e0f377640 1 -- 172.21.15.52:0/1664714299 shutdown_connections 2024-03-04T09:30:28.014 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:28.014+0000 7f9e0f377640 1 -- 172.21.15.52:0/1664714299 wait complete. 2024-03-04T09:30:28.014 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:28.015+0000 7f9e0f377640 1 Processor -- start 2024-03-04T09:30:28.015 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:28.015+0000 7f9e0f377640 1 -- start start 2024-03-04T09:30:28.015 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:28.015+0000 7f9e0f377640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9e081015f0 0x7f9e08197910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:30:28.015 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:28.015+0000 7f9e0f377640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9e081027e0 0x7f9e08197e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:30:28.015 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:28.015+0000 7f9e0f377640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9e081031a0 0x7f9e0819eec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:30:28.015 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:28.015+0000 7f9e0f377640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e0810d240 con 0x7f9e081015f0 2024-03-04T09:30:28.015 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:28.015+0000 7f9e0f377640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e0810d0c0 con 0x7f9e081031a0 2024-03-04T09:30:28.015 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:28.015+0000 7f9e0f377640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e0810d3c0 con 0x7f9e081027e0 2024-03-04T09:30:28.015 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:28.015+0000 7f9e0d113640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9e081015f0 0x7f9e08197910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:30:28.015 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:28.015+0000 7f9e0d113640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9e081015f0 0x7f9e08197910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:52142/0 (socket says 172.21.15.52:52142) 2024-03-04T09:30:28.015 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:28.015+0000 7f9e0d113640 1 -- 172.21.15.52:0/4180968862 learned_addr learned my addr 172.21.15.52:0/4180968862 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-04T09:30:28.015 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:28.015+0000 7f9e0c912640 1 --2- 172.21.15.52:0/4180968862 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9e081027e0 0x7f9e08197e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:30:28.015 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:28.015+0000 7f9e0d914640 1 --2- 172.21.15.52:0/4180968862 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9e081031a0 0x7f9e0819eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:30:28.015 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:28.015+0000 7f9e0d113640 1 -- 172.21.15.52:0/4180968862 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9e081027e0 msgr2=0x7f9e08197e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:30:28.016 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:28.016+0000 7f9e0d113640 1 --2- 172.21.15.52:0/4180968862 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9e081027e0 0x7f9e08197e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:28.016 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:28.016+0000 7f9e0d113640 1 -- 172.21.15.52:0/4180968862 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9e081031a0 msgr2=0x7f9e0819eec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:30:28.016 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:28.016+0000 7f9e0d113640 1 --2- 172.21.15.52:0/4180968862 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9e081031a0 0x7f9e0819eec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:28.016 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:28.016+0000 7f9e0d113640 1 -- 172.21.15.52:0/4180968862 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e0819f530 con 0x7f9e081015f0 2024-03-04T09:30:28.016 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:28.016+0000 7f9e0d914640 1 --2- 172.21.15.52:0/4180968862 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9e081031a0 0x7f9e0819eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T09:30:28.016 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:28.016+0000 7f9e0d113640 1 --2- 172.21.15.52:0/4180968862 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9e081015f0 0x7f9e08197910 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f9e00000c00 tx=0x7f9e00005d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:30:28.016 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:28.016+0000 7f9dfe7fc640 1 -- 172.21.15.52:0/4180968862 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e00041070 con 0x7f9e081015f0 2024-03-04T09:30:28.016 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:28.016+0000 7f9dfe7fc640 1 -- 172.21.15.52:0/4180968862 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9e000044d0 con 0x7f9e081015f0 2024-03-04T09:30:28.016 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:28.016+0000 7f9dfe7fc640 1 -- 172.21.15.52:0/4180968862 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e00034440 con 0x7f9e081015f0 2024-03-04T09:30:28.016 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:28.016+0000 7f9e0f377640 1 -- 172.21.15.52:0/4180968862 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e0819f7c0 con 0x7f9e081015f0 2024-03-04T09:30:28.016 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:28.016+0000 7f9e0f377640 1 -- 172.21.15.52:0/4180968862 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e0819fcb0 con 0x7f9e081015f0 2024-03-04T09:30:28.016 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:28.017+0000 7f9e0f377640 1 -- 172.21.15.52:0/4180968862 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9e08108930 con 0x7f9e081015f0 2024-03-04T09:30:28.017 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:28.019+0000 7f9dfe7fc640 1 -- 172.21.15.52:0/4180968862 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) v1 ==== 136756+0+0 (secure 0 0 0) 0x7f9e00044070 con 0x7f9e081015f0 2024-03-04T09:30:28.017 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:28.019+0000 7f9dfe7fc640 1 --2- 172.21.15.52:0/4180968862 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f9df40a1d80 0x7f9df40a4240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:30:28.017 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:28.019+0000 7f9e0c912640 1 --2- 172.21.15.52:0/4180968862 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f9df40a1d80 0x7f9df40a4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:30:28.017 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:28.019+0000 7f9dfe7fc640 1 -- 172.21.15.52:0/4180968862 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9092+0+0 (secure 0 0 0) 0x7f9e000efef0 con 0x7f9e081015f0 2024-03-04T09:30:28.018 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:28.020+0000 7f9e0c912640 1 --2- 172.21.15.52:0/4180968862 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f9df40a1d80 0x7f9df40a4240 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f9e08198e20 tx=0x7f9df8005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:30:28.019 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:28.023+0000 7f9dfe7fc640 1 -- 172.21.15.52:0/4180968862 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f9e000bcfc0 con 0x7f9e081015f0 2024-03-04T09:30:28.202 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:28.205+0000 7f9e0f377640 1 -- 172.21.15.52:0/4180968862 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f9e0802d180 con 0x7f9e081015f0 2024-03-04T09:30:28.202 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:28.205+0000 7f9dfe7fc640 1 -- 172.21.15.52:0/4180968862 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v91) v1 ==== 100+0+11 (secure 0 0 0) 0x7f9e000bc8c0 con 0x7f9e081015f0 2024-03-04T09:30:28.202 INFO:teuthology.orchestra.run.smithi052.stdout:pg_num: 64 2024-03-04T09:30:28.204 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:28.207+0000 7f9e0f377640 1 -- 172.21.15.52:0/4180968862 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f9df40a1d80 msgr2=0x7f9df40a4240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:30:28.204 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:28.207+0000 7f9e0f377640 1 --2- 172.21.15.52:0/4180968862 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f9df40a1d80 0x7f9df40a4240 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f9e08198e20 tx=0x7f9df8005f90 comp rx=0 tx=0).stop 2024-03-04T09:30:28.204 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:28.207+0000 7f9e0f377640 1 -- 172.21.15.52:0/4180968862 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9e081015f0 msgr2=0x7f9e08197910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:30:28.204 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:28.207+0000 7f9e0f377640 1 --2- 172.21.15.52:0/4180968862 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9e081015f0 0x7f9e08197910 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f9e00000c00 tx=0x7f9e00005d50 comp rx=0 tx=0).stop 2024-03-04T09:30:28.205 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:28.207+0000 7f9e0f377640 1 -- 172.21.15.52:0/4180968862 shutdown_connections 2024-03-04T09:30:28.205 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:28.207+0000 7f9e0f377640 1 --2- 172.21.15.52:0/4180968862 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f9df40a1d80 0x7f9df40a4240 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:28.205 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:28.207+0000 7f9e0f377640 1 --2- 172.21.15.52:0/4180968862 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9e081031a0 0x7f9e0819eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:28.205 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:28.207+0000 7f9e0f377640 1 --2- 172.21.15.52:0/4180968862 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9e081027e0 0x7f9e08197e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:28.205 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:28.207+0000 7f9e0f377640 1 --2- 172.21.15.52:0/4180968862 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9e081015f0 0x7f9e08197910 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:28.205 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:28.207+0000 7f9e0f377640 1 -- 172.21.15.52:0/4180968862 >> 172.21.15.52:0/4180968862 conn(0x7f9e080fcb60 msgr2=0x7f9e08105b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:30:28.205 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:28.207+0000 7f9e0f377640 1 -- 172.21.15.52:0/4180968862 shutdown_connections 2024-03-04T09:30:28.205 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:28.207+0000 7f9e0f377640 1 -- 172.21.15.52:0/4180968862 wait complete. 2024-03-04T09:30:28.216 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-04T09:30:28.323 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:28.324+0000 7f6914281640 1 -- 172.21.15.52:0/1401986915 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f690c102080 msgr2=0x7f690c108910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:30:28.324 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:28.324+0000 7f6914281640 1 --2- 172.21.15.52:0/1401986915 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f690c102080 0x7f690c108910 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f6904009ca0 tx=0x7f690402c0f0 comp rx=0 tx=0).stop 2024-03-04T09:30:28.324 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:28.325+0000 7f6914281640 1 -- 172.21.15.52:0/1401986915 shutdown_connections 2024-03-04T09:30:28.324 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:28.325+0000 7f6914281640 1 --2- 172.21.15.52:0/1401986915 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f690c102080 0x7f690c108910 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:28.324 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:28.325+0000 7f6914281640 1 --2- 172.21.15.52:0/1401986915 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f690c1016c0 0x7f690c101b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:28.324 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:28.325+0000 7f6914281640 1 --2- 172.21.15.52:0/1401986915 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f690c1004d0 0x7f690c1008f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:28.324 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:28.325+0000 7f6914281640 1 -- 172.21.15.52:0/1401986915 >> 172.21.15.52:0/1401986915 conn(0x7f690c0fba60 msgr2=0x7f690c0fde80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:30:28.324 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:28.325+0000 7f6914281640 1 -- 172.21.15.52:0/1401986915 shutdown_connections 2024-03-04T09:30:28.324 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:28.325+0000 7f6914281640 1 -- 172.21.15.52:0/1401986915 wait complete. 2024-03-04T09:30:28.324 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:28.325+0000 7f6914281640 1 Processor -- start 2024-03-04T09:30:28.324 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:28.325+0000 7f6914281640 1 -- start start 2024-03-04T09:30:28.324 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:28.325+0000 7f6914281640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f690c1004d0 0x7f690c0692a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:30:28.324 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:28.325+0000 7f6914281640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f690c1016c0 0x7f690c19e9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:30:28.324 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:28.325+0000 7f6914281640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f690c102080 0x7f690c1a0f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:30:28.325 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:28.325+0000 7f6914281640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f690c10c160 con 0x7f690c1004d0 2024-03-04T09:30:28.325 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:28.325+0000 7f6914281640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f690c10bfe0 con 0x7f690c1016c0 2024-03-04T09:30:28.325 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:28.325+0000 7f6914281640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f690c10c2e0 con 0x7f690c102080 2024-03-04T09:30:28.325 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:28.325+0000 7f691201d640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f690c1004d0 0x7f690c0692a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:30:28.325 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:28.325+0000 7f691201d640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f690c1004d0 0x7f690c0692a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:52160/0 (socket says 172.21.15.52:52160) 2024-03-04T09:30:28.325 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:28.325+0000 7f691201d640 1 -- 172.21.15.52:0/617865048 learned_addr learned my addr 172.21.15.52:0/617865048 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-04T09:30:28.325 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:28.326+0000 7f691181c640 1 --2- 172.21.15.52:0/617865048 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f690c1016c0 0x7f690c19e9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:30:28.325 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:28.326+0000 7f691281e640 1 --2- 172.21.15.52:0/617865048 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f690c102080 0x7f690c1a0f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:30:28.325 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:28.326+0000 7f691201d640 1 -- 172.21.15.52:0/617865048 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f690c102080 msgr2=0x7f690c1a0f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:30:28.325 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:28.326+0000 7f691201d640 1 --2- 172.21.15.52:0/617865048 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f690c102080 0x7f690c1a0f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:28.325 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:28.326+0000 7f691201d640 1 -- 172.21.15.52:0/617865048 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f690c1016c0 msgr2=0x7f690c19e9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:30:28.325 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:28.326+0000 7f691201d640 1 --2- 172.21.15.52:0/617865048 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f690c1016c0 0x7f690c19e9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:28.325 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:28.326+0000 7f691201d640 1 -- 172.21.15.52:0/617865048 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f690c1a1720 con 0x7f690c1004d0 2024-03-04T09:30:28.325 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:28.326+0000 7f691201d640 1 --2- 172.21.15.52:0/617865048 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f690c1004d0 0x7f690c0692a0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f690800e9c0 tx=0x7f690800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:30:28.326 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:28.326+0000 7f69037fe640 1 -- 172.21.15.52:0/617865048 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f690800cd80 con 0x7f690c1004d0 2024-03-04T09:30:28.326 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:28.326+0000 7f6914281640 1 -- 172.21.15.52:0/617865048 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f690c1a19b0 con 0x7f690c1004d0 2024-03-04T09:30:28.326 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:28.326+0000 7f69037fe640 1 -- 172.21.15.52:0/617865048 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f69080035f0 con 0x7f690c1004d0 2024-03-04T09:30:28.326 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:28.326+0000 7f69037fe640 1 -- 172.21.15.52:0/617865048 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6908010690 con 0x7f690c1004d0 2024-03-04T09:30:28.326 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:28.326+0000 7f6914281640 1 -- 172.21.15.52:0/617865048 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f690c1a1f00 con 0x7f690c1004d0 2024-03-04T09:30:28.327 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:28.327+0000 7f6914281640 1 -- 172.21.15.52:0/617865048 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f690c1077f0 con 0x7f690c1004d0 2024-03-04T09:30:28.327 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:28.328+0000 7f69037fe640 1 -- 172.21.15.52:0/617865048 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) v1 ==== 136756+0+0 (secure 0 0 0) 0x7f69080037a0 con 0x7f690c1004d0 2024-03-04T09:30:28.327 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:28.329+0000 7f69037fe640 1 --2- 172.21.15.52:0/617865048 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f68f80a1db0 0x7f68f80a4270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:30:28.327 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:28.329+0000 7f691181c640 1 --2- 172.21.15.52:0/617865048 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f68f80a1db0 0x7f68f80a4270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:30:28.327 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:28.329+0000 7f69037fe640 1 -- 172.21.15.52:0/617865048 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9092+0+0 (secure 0 0 0) 0x7f69080ce470 con 0x7f690c1004d0 2024-03-04T09:30:28.327 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:28.329+0000 7f691181c640 1 --2- 172.21.15.52:0/617865048 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f68f80a1db0 0x7f68f80a4270 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f68fc0095f0 tx=0x7f68fc005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:30:28.330 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:28.333+0000 7f69037fe640 1 -- 172.21.15.52:0/617865048 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f690809b4e0 con 0x7f690c1004d0 2024-03-04T09:30:28.511 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:28.514+0000 7f6914281640 1 -- 172.21.15.52:0/617865048 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f690c0621d0 con 0x7f690c1004d0 2024-03-04T09:30:28.511 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:28.514+0000 7f69037fe640 1 -- 172.21.15.52:0/617865048 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v91) v1 ==== 103+0+11 (secure 0 0 0) 0x7f690809ade0 con 0x7f690c1004d0 2024-03-04T09:30:28.511 INFO:teuthology.orchestra.run.smithi052.stdout:pg_num: 32 2024-03-04T09:30:28.513 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:28.516+0000 7f6914281640 1 -- 172.21.15.52:0/617865048 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f68f80a1db0 msgr2=0x7f68f80a4270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:30:28.513 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:28.516+0000 7f6914281640 1 --2- 172.21.15.52:0/617865048 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f68f80a1db0 0x7f68f80a4270 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f68fc0095f0 tx=0x7f68fc005f90 comp rx=0 tx=0).stop 2024-03-04T09:30:28.513 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:28.516+0000 7f6914281640 1 -- 172.21.15.52:0/617865048 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f690c1004d0 msgr2=0x7f690c0692a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:30:28.513 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:28.516+0000 7f6914281640 1 --2- 172.21.15.52:0/617865048 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f690c1004d0 0x7f690c0692a0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f690800e9c0 tx=0x7f690800ee80 comp rx=0 tx=0).stop 2024-03-04T09:30:28.513 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:28.516+0000 7f6914281640 1 -- 172.21.15.52:0/617865048 shutdown_connections 2024-03-04T09:30:28.513 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:28.516+0000 7f6914281640 1 --2- 172.21.15.52:0/617865048 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f68f80a1db0 0x7f68f80a4270 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:28.514 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:28.516+0000 7f6914281640 1 --2- 172.21.15.52:0/617865048 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f690c102080 0x7f690c1a0f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:28.514 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:28.516+0000 7f6914281640 1 --2- 172.21.15.52:0/617865048 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f690c1016c0 0x7f690c19e9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:28.514 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:28.516+0000 7f6914281640 1 --2- 172.21.15.52:0/617865048 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f690c1004d0 0x7f690c0692a0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:28.514 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:28.516+0000 7f6914281640 1 -- 172.21.15.52:0/617865048 >> 172.21.15.52:0/617865048 conn(0x7f690c0fba60 msgr2=0x7f690c1049c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:30:28.514 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:28.516+0000 7f6914281640 1 -- 172.21.15.52:0/617865048 shutdown_connections 2024-03-04T09:30:28.514 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:28.516+0000 7f6914281640 1 -- 172.21.15.52:0/617865048 wait complete. 2024-03-04T09:30:28.523 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:30:28 smithi052 ceph-mon[32473]: from='client.? 172.21.15.52:0/3533441867' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-04T09:30:28.523 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:30:28 smithi052 ceph-mon[32473]: from='client.? 172.21.15.52:0/3938195849' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-04T09:30:28.523 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:30:28 smithi052 ceph-mon[32473]: from='client.? 172.21.15.52:0/3744334532' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-04T09:30:28.523 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:30:28 smithi052 ceph-mon[32473]: from='client.? 172.21.15.52:0/4180968862' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-04T09:30:28.524 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T09:30:28.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:30:28 smithi191 ceph-mon[36816]: from='client.? 172.21.15.52:0/3533441867' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-04T09:30:28.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:30:28 smithi191 ceph-mon[36816]: from='client.? 172.21.15.52:0/3938195849' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-04T09:30:28.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:30:28 smithi191 ceph-mon[36816]: from='client.? 172.21.15.52:0/3744334532' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-04T09:30:28.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:30:28 smithi191 ceph-mon[36816]: from='client.? 172.21.15.52:0/4180968862' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-04T09:30:28.630 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:28.631+0000 7f2986993640 1 -- 172.21.15.52:0/1096220747 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2980100570 msgr2=0x7f2980100990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:30:28.630 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:28.631+0000 7f2986993640 1 --2- 172.21.15.52:0/1096220747 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2980100570 0x7f2980100990 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f2970009860 tx=0x7f297002bb70 comp rx=0 tx=0).stop 2024-03-04T09:30:28.630 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:28.631+0000 7f2986993640 1 -- 172.21.15.52:0/1096220747 shutdown_connections 2024-03-04T09:30:28.630 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:28.631+0000 7f2986993640 1 --2- 172.21.15.52:0/1096220747 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2980102120 0x7f29801089b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:28.630 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:28.631+0000 7f2986993640 1 --2- 172.21.15.52:0/1096220747 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2980101760 0x7f2980101be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:28.630 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:28.631+0000 7f2986993640 1 --2- 172.21.15.52:0/1096220747 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2980100570 0x7f2980100990 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:28.630 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:28.631+0000 7f2986993640 1 -- 172.21.15.52:0/1096220747 >> 172.21.15.52:0/1096220747 conn(0x7f29800fbb00 msgr2=0x7f29800fdf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:30:28.630 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:28.631+0000 7f2986993640 1 -- 172.21.15.52:0/1096220747 shutdown_connections 2024-03-04T09:30:28.630 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:28.631+0000 7f2986993640 1 -- 172.21.15.52:0/1096220747 wait complete. 2024-03-04T09:30:28.631 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:28.631+0000 7f2986993640 1 Processor -- start 2024-03-04T09:30:28.631 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:28.631+0000 7f2986993640 1 -- start start 2024-03-04T09:30:28.631 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:28.632+0000 7f2986993640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2980100570 0x7f2980196820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:30:28.631 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:28.632+0000 7f2986993640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2980101760 0x7f2980196d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:30:28.631 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:28.632+0000 7f2986993640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2980102120 0x7f298019ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:30:28.631 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:28.632+0000 7f2986993640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f298010c190 con 0x7f2980100570 2024-03-04T09:30:28.631 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:28.632+0000 7f2986993640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f298010c010 con 0x7f2980101760 2024-03-04T09:30:28.631 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:28.632+0000 7f2986993640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f298010c310 con 0x7f2980102120 2024-03-04T09:30:28.631 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:28.632+0000 7f297ffff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2980100570 0x7f2980196820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:30:28.631 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:28.632+0000 7f297ffff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2980100570 0x7f2980196820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:52174/0 (socket says 172.21.15.52:52174) 2024-03-04T09:30:28.631 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:28.632+0000 7f297ffff640 1 -- 172.21.15.52:0/3102207095 learned_addr learned my addr 172.21.15.52:0/3102207095 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-04T09:30:28.631 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:28.632+0000 7f297f7fe640 1 --2- 172.21.15.52:0/3102207095 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2980101760 0x7f2980196d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:30:28.631 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:28.632+0000 7f2984f30640 1 --2- 172.21.15.52:0/3102207095 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2980102120 0x7f298019ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:30:28.631 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:28.632+0000 7f297ffff640 1 -- 172.21.15.52:0/3102207095 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2980102120 msgr2=0x7f298019ddd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:30:28.632 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:28.632+0000 7f297ffff640 1 --2- 172.21.15.52:0/3102207095 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2980102120 0x7f298019ddd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:28.632 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:28.632+0000 7f297ffff640 1 -- 172.21.15.52:0/3102207095 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2980101760 msgr2=0x7f2980196d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:30:28.632 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:28.632+0000 7f297ffff640 1 --2- 172.21.15.52:0/3102207095 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2980101760 0x7f2980196d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:28.632 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:28.632+0000 7f297ffff640 1 -- 172.21.15.52:0/3102207095 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f298019e440 con 0x7f2980100570 2024-03-04T09:30:28.632 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:28.632+0000 7f297ffff640 1 --2- 172.21.15.52:0/3102207095 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2980100570 0x7f2980196820 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f297002c0b0 tx=0x7f2970032430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:30:28.632 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:28.632+0000 7f297d7fa640 1 -- 172.21.15.52:0/3102207095 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2970041020 con 0x7f2980100570 2024-03-04T09:30:28.632 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:28.632+0000 7f297d7fa640 1 -- 172.21.15.52:0/3102207095 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f297002c9f0 con 0x7f2980100570 2024-03-04T09:30:28.632 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:28.632+0000 7f2986993640 1 -- 172.21.15.52:0/3102207095 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f298019e6d0 con 0x7f2980100570 2024-03-04T09:30:28.632 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:28.632+0000 7f297d7fa640 1 -- 172.21.15.52:0/3102207095 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2970039040 con 0x7f2980100570 2024-03-04T09:30:28.632 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:28.632+0000 7f2986993640 1 -- 172.21.15.52:0/3102207095 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f298019ec10 con 0x7f2980100570 2024-03-04T09:30:28.633 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:30:28 smithi106 ceph-mon[36484]: from='client.? 172.21.15.52:0/3533441867' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-04T09:30:28.633 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:30:28 smithi106 ceph-mon[36484]: from='client.? 172.21.15.52:0/3938195849' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-04T09:30:28.633 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:30:28 smithi106 ceph-mon[36484]: from='client.? 172.21.15.52:0/3744334532' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-04T09:30:28.633 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:30:28 smithi106 ceph-mon[36484]: from='client.? 172.21.15.52:0/4180968862' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-04T09:30:28.634 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:28.633+0000 7f2986993640 1 -- 172.21.15.52:0/3102207095 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2980107890 con 0x7f2980100570 2024-03-04T09:30:28.634 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:28.634+0000 7f297d7fa640 1 -- 172.21.15.52:0/3102207095 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) v1 ==== 136756+0+0 (secure 0 0 0) 0x7f2970044020 con 0x7f2980100570 2024-03-04T09:30:28.634 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:28.635+0000 7f297d7fa640 1 --2- 172.21.15.52:0/3102207095 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f296c0a1e00 0x7f296c0a42c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:30:28.634 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:28.635+0000 7f297f7fe640 1 --2- 172.21.15.52:0/3102207095 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f296c0a1e00 0x7f296c0a42c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:30:28.634 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:28.635+0000 7f297d7fa640 1 -- 172.21.15.52:0/3102207095 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9092+0+0 (secure 0 0 0) 0x7f29700ef950 con 0x7f2980100570 2024-03-04T09:30:28.634 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:28.635+0000 7f297f7fe640 1 --2- 172.21.15.52:0/3102207095 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f296c0a1e00 0x7f296c0a42c0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f2980197d30 tx=0x7f2968005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:30:28.636 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:28.639+0000 7f297d7fa640 1 -- 172.21.15.52:0/3102207095 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f297004b020 con 0x7f2980100570 2024-03-04T09:30:28.859 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:28.862+0000 7f2986993640 1 -- 172.21.15.52:0/3102207095 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f298002cff0 con 0x7f2980100570 2024-03-04T09:30:28.859 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:28.863+0000 7f297d7fa640 1 -- 172.21.15.52:0/3102207095 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+11406 (secure 0 0 0) 0x7f29700bca20 con 0x7f2980100570 2024-03-04T09:30:28.860 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-03-04T09:30:28.860 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6835/1007961019","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6834","nonce":1007961019},{"type":"v1","addr":"172.21.15.52:6835","nonce":1007961019}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6837/3753881283","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6836","nonce":3753881283},{"type":"v1","addr":"172.21.15.52:6837","nonce":3753881283}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14628,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6839/3467897808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6838","nonce":3467897808},{"type":"v1","addr":"172.21.15.52:6839","nonce":3467897808}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14640,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6841/2804292291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6840","nonce":2804292291},{"type":"v1","addr":"172.21.15.52:6841","nonce":2804292291}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24407,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6833/1818480323","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6832","nonce":1818480323},{"type":"v1","addr":"172.21.15.106:6833","nonce":1818480323}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24412,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6833/765768204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":765768204},{"type":"v1","addr":"172.21.15.191:6833","nonce":765768204}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-04T09:28:37.264970+0000","modified":"2024-03-04T09:29:04.001848+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24506,"mds_1":24490,"mds_2":24473},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24440":{"gid":24440,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.106:6835/1541069897","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6834","nonce":1541069897},{"type":"v1","addr":"172.21.15.106:6835","nonce":1541069897}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24451":{"gid":24451,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.191:6835/105048699","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":105048699},{"type":"v1","addr":"172.21.15.191:6835","nonce":105048699}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24463":{"gid":24463,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.191:6837/4071064225","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6836","nonce":4071064225},{"type":"v1","addr":"172.21.15.191:6837","nonce":4071064225}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24473":{"gid":24473,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.106:6837/2901755066","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6836","nonce":2901755066},{"type":"v1","addr":"172.21.15.106:6837","nonce":2901755066}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24490":{"gid":24490,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.191:6839/2194324891","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6838","nonce":2194324891},{"type":"v1","addr":"172.21.15.191:6839","nonce":2194324891}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24506":{"gid":24506,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.106:6839/1995853093","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6838","nonce":1995853093},{"type":"v1","addr":"172.21.15.106:6839","nonce":1995853093}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24506,"qdb_cluster":[24473,24490,24506]},"id":1}]} 2024-03-04T09:30:28.860 INFO:teuthology.orchestra.run.smithi052.stderr:dumped fsmap epoch 24 2024-03-04T09:30:28.862 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:28.865+0000 7f2986993640 1 -- 172.21.15.52:0/3102207095 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f296c0a1e00 msgr2=0x7f296c0a42c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:30:28.862 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:28.865+0000 7f2986993640 1 --2- 172.21.15.52:0/3102207095 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f296c0a1e00 0x7f296c0a42c0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f2980197d30 tx=0x7f2968005f90 comp rx=0 tx=0).stop 2024-03-04T09:30:28.862 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:28.865+0000 7f2986993640 1 -- 172.21.15.52:0/3102207095 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2980100570 msgr2=0x7f2980196820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:30:28.862 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:28.865+0000 7f2986993640 1 --2- 172.21.15.52:0/3102207095 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2980100570 0x7f2980196820 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f297002c0b0 tx=0x7f2970032430 comp rx=0 tx=0).stop 2024-03-04T09:30:28.863 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:28.865+0000 7f2986993640 1 -- 172.21.15.52:0/3102207095 shutdown_connections 2024-03-04T09:30:28.863 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:28.865+0000 7f2986993640 1 --2- 172.21.15.52:0/3102207095 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f296c0a1e00 0x7f296c0a42c0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:28.863 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:28.865+0000 7f2986993640 1 --2- 172.21.15.52:0/3102207095 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2980102120 0x7f298019ddd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:28.863 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:28.865+0000 7f2986993640 1 --2- 172.21.15.52:0/3102207095 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2980101760 0x7f2980196d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:28.863 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:28.865+0000 7f2986993640 1 --2- 172.21.15.52:0/3102207095 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2980100570 0x7f2980196820 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:28.863 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:28.865+0000 7f2986993640 1 -- 172.21.15.52:0/3102207095 >> 172.21.15.52:0/3102207095 conn(0x7f29800fbb00 msgr2=0x7f2980104a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:30:28.863 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:28.865+0000 7f2986993640 1 -- 172.21.15.52:0/3102207095 shutdown_connections 2024-03-04T09:30:28.863 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:28.865+0000 7f2986993640 1 -- 172.21.15.52:0/3102207095 wait complete. 2024-03-04T09:30:28.873 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-04T09:30:28.980 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:28.980+0000 7f6f21702640 1 -- 172.21.15.52:0/3199549243 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6f1c1027e0 msgr2=0x7f6f1c102c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:30:28.980 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:28.980+0000 7f6f21702640 1 --2- 172.21.15.52:0/3199549243 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6f1c1027e0 0x7f6f1c102c60 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f6f04009900 tx=0x7f6f0402bc30 comp rx=0 tx=0).stop 2024-03-04T09:30:28.980 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:28.980+0000 7f6f21702640 1 -- 172.21.15.52:0/3199549243 shutdown_connections 2024-03-04T09:30:28.980 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:28.980+0000 7f6f21702640 1 --2- 172.21.15.52:0/3199549243 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6f1c1031a0 0x7f6f1c109a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:28.980 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:28.980+0000 7f6f21702640 1 --2- 172.21.15.52:0/3199549243 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6f1c1027e0 0x7f6f1c102c60 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:28.980 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:28.980+0000 7f6f21702640 1 --2- 172.21.15.52:0/3199549243 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6f1c1015f0 0x7f6f1c101a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:28.980 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:28.980+0000 7f6f21702640 1 -- 172.21.15.52:0/3199549243 >> 172.21.15.52:0/3199549243 conn(0x7f6f1c0fcb80 msgr2=0x7f6f1c0fefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:30:28.980 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:28.980+0000 7f6f21702640 1 -- 172.21.15.52:0/3199549243 shutdown_connections 2024-03-04T09:30:28.980 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:28.980+0000 7f6f21702640 1 -- 172.21.15.52:0/3199549243 wait complete. 2024-03-04T09:30:28.980 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:28.981+0000 7f6f21702640 1 Processor -- start 2024-03-04T09:30:28.980 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:28.981+0000 7f6f21702640 1 -- start start 2024-03-04T09:30:28.981 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:28.981+0000 7f6f21702640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6f1c1015f0 0x7f6f1c1978e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:30:28.981 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:28.981+0000 7f6f21702640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6f1c1027e0 0x7f6f1c197e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:30:28.981 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:28.981+0000 7f6f21702640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6f1c1031a0 0x7f6f1c19ee90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:30:28.981 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:28.981+0000 7f6f21702640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f1c10d270 con 0x7f6f1c1031a0 2024-03-04T09:30:28.981 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:28.981+0000 7f6f21702640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f1c10d0f0 con 0x7f6f1c1027e0 2024-03-04T09:30:28.981 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:28.981+0000 7f6f21702640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f1c10d3f0 con 0x7f6f1c1015f0 2024-03-04T09:30:28.981 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:28.981+0000 7f6f1b7fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6f1c1031a0 0x7f6f1c19ee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:30:28.981 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:28.981+0000 7f6f1b7fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6f1c1031a0 0x7f6f1c19ee90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:52196/0 (socket says 172.21.15.52:52196) 2024-03-04T09:30:28.981 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:28.981+0000 7f6f1b7fe640 1 -- 172.21.15.52:0/3100044586 learned_addr learned my addr 172.21.15.52:0/3100044586 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-04T09:30:28.981 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:28.981+0000 7f6f13fff640 1 --2- 172.21.15.52:0/3100044586 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6f1c1027e0 0x7f6f1c197e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:30:28.981 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:28.981+0000 7f6f1affd640 1 --2- 172.21.15.52:0/3100044586 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6f1c1015f0 0x7f6f1c1978e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:30:28.981 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:28.981+0000 7f6f1b7fe640 1 -- 172.21.15.52:0/3100044586 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6f1c1015f0 msgr2=0x7f6f1c1978e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:30:28.981 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:28.981+0000 7f6f1b7fe640 1 --2- 172.21.15.52:0/3100044586 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6f1c1015f0 0x7f6f1c1978e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:28.982 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:28.981+0000 7f6f1b7fe640 1 -- 172.21.15.52:0/3100044586 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6f1c1027e0 msgr2=0x7f6f1c197e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:30:28.982 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:28.981+0000 7f6f1b7fe640 1 --2- 172.21.15.52:0/3100044586 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6f1c1027e0 0x7f6f1c197e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:28.982 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:28.981+0000 7f6f1b7fe640 1 -- 172.21.15.52:0/3100044586 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f1c19f500 con 0x7f6f1c1031a0 2024-03-04T09:30:28.982 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:28.981+0000 7f6f13fff640 1 --2- 172.21.15.52:0/3100044586 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6f1c1027e0 0x7f6f1c197e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-04T09:30:28.982 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:28.981+0000 7f6f1b7fe640 1 --2- 172.21.15.52:0/3100044586 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6f1c1031a0 0x7f6f1c19ee90 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f6f1400eb10 tx=0x7f6f1400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:30:28.982 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:28.982+0000 7f6f18ff9640 1 -- 172.21.15.52:0/3100044586 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f14004b20 con 0x7f6f1c1031a0 2024-03-04T09:30:28.982 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:28.982+0000 7f6f18ff9640 1 -- 172.21.15.52:0/3100044586 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6f14004cc0 con 0x7f6f1c1031a0 2024-03-04T09:30:28.982 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:28.982+0000 7f6f21702640 1 -- 172.21.15.52:0/3100044586 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f1c19f7f0 con 0x7f6f1c1031a0 2024-03-04T09:30:28.982 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:28.982+0000 7f6f18ff9640 1 -- 172.21.15.52:0/3100044586 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f14010760 con 0x7f6f1c1031a0 2024-03-04T09:30:28.982 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:28.982+0000 7f6f21702640 1 -- 172.21.15.52:0/3100044586 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f1c19fcc0 con 0x7f6f1c1031a0 2024-03-04T09:30:28.982 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:28.983+0000 7f6f21702640 1 -- 172.21.15.52:0/3100044586 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6f1c108910 con 0x7f6f1c1031a0 2024-03-04T09:30:28.983 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:28.984+0000 7f6f18ff9640 1 -- 172.21.15.52:0/3100044586 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) v1 ==== 136756+0+0 (secure 0 0 0) 0x7f6f14005020 con 0x7f6f1c1031a0 2024-03-04T09:30:28.983 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:28.984+0000 7f6f18ff9640 1 --2- 172.21.15.52:0/3100044586 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f6f080a1d80 0x7f6f080a4240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:30:28.983 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:28.984+0000 7f6f1affd640 1 --2- 172.21.15.52:0/3100044586 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f6f080a1d80 0x7f6f080a4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:30:28.983 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:28.984+0000 7f6f18ff9640 1 -- 172.21.15.52:0/3100044586 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9092+0+0 (secure 0 0 0) 0x7f6f140ce6c0 con 0x7f6f1c1031a0 2024-03-04T09:30:28.983 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:28.985+0000 7f6f1affd640 1 --2- 172.21.15.52:0/3100044586 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f6f080a1d80 0x7f6f080a4240 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f6f1c102640 tx=0x7f6f0c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:30:28.985 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:28.989+0000 7f6f18ff9640 1 -- 172.21.15.52:0/3100044586 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f6f1409b790 con 0x7f6f1c1031a0 2024-03-04T09:30:29.168 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.172+0000 7f6f21702640 1 -- 172.21.15.52:0/3100044586 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6f1c0621d0 con 0x7f6f1c1031a0 2024-03-04T09:30:29.170 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.173+0000 7f6f18ff9640 1 -- 172.21.15.52:0/3100044586 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) v1 ==== 74+0+24635 (secure 0 0 0) 0x7f6f1409b090 con 0x7f6f1c1031a0 2024-03-04T09:30:29.170 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-03-04T09:30:29.171 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":91,"fsid":"4e059f08-da08-11ee-95c6-87774f69a715","created":"2024-03-04T09:20:46.104963+0000","modified":"2024-03-04T09:29:41.058759+0000","last_up_change":"2024-03-04T09:27:49.261365+0000","last_in_change":"2024-03-04T09:27:30.024964+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-04T09:24:01.742461+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-04T09:28:35.835600+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-04T09:28:36.453099+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-04T09:28:40.168831+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"583172be-602f-4bf4-bfcb-42663cf5b277","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6802","nonce":1371221397},{"type":"v1","addr":"172.21.15.52:6803","nonce":1371221397}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6804","nonce":1371221397},{"type":"v1","addr":"172.21.15.52:6805","nonce":1371221397}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6808","nonce":1371221397},{"type":"v1","addr":"172.21.15.52:6809","nonce":1371221397}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6806","nonce":1371221397},{"type":"v1","addr":"172.21.15.52:6807","nonce":1371221397}]},"public_addr":"172.21.15.52:6803/1371221397","cluster_addr":"172.21.15.52:6805/1371221397","heartbeat_back_addr":"172.21.15.52:6809/1371221397","heartbeat_front_addr":"172.21.15.52:6807/1371221397","state":["exists","up"]},{"osd":1,"uuid":"5ae0081c-a2f6-46eb-ace0-0368208377b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":2540865537},{"type":"v1","addr":"172.21.15.106:6801","nonce":2540865537}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":2540865537},{"type":"v1","addr":"172.21.15.106:6803","nonce":2540865537}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":2540865537},{"type":"v1","addr":"172.21.15.106:6807","nonce":2540865537}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":2540865537},{"type":"v1","addr":"172.21.15.106:6805","nonce":2540865537}]},"public_addr":"172.21.15.106:6801/2540865537","cluster_addr":"172.21.15.106:6803/2540865537","heartbeat_back_addr":"172.21.15.106:6807/2540865537","heartbeat_front_addr":"172.21.15.106:6805/2540865537","state":["exists","up"]},{"osd":2,"uuid":"1948bac7-2bb2-4218-8b93-4e4743a0de04","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":1377083164},{"type":"v1","addr":"172.21.15.191:6801","nonce":1377083164}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":1377083164},{"type":"v1","addr":"172.21.15.191:6803","nonce":1377083164}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":1377083164},{"type":"v1","addr":"172.21.15.191:6807","nonce":1377083164}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":1377083164},{"type":"v1","addr":"172.21.15.191:6805","nonce":1377083164}]},"public_addr":"172.21.15.191:6801/1377083164","cluster_addr":"172.21.15.191:6803/1377083164","heartbeat_back_addr":"172.21.15.191:6807/1377083164","heartbeat_front_addr":"172.21.15.191:6805/1377083164","state":["exists","up"]},{"osd":3,"uuid":"16c9e9fe-7167-4992-bf12-1a7398e0498a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6810","nonce":2388571810},{"type":"v1","addr":"172.21.15.52:6811","nonce":2388571810}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6812","nonce":2388571810},{"type":"v1","addr":"172.21.15.52:6813","nonce":2388571810}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6816","nonce":2388571810},{"type":"v1","addr":"172.21.15.52:6817","nonce":2388571810}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6814","nonce":2388571810},{"type":"v1","addr":"172.21.15.52:6815","nonce":2388571810}]},"public_addr":"172.21.15.52:6811/2388571810","cluster_addr":"172.21.15.52:6813/2388571810","heartbeat_back_addr":"172.21.15.52:6817/2388571810","heartbeat_front_addr":"172.21.15.52:6815/2388571810","state":["exists","up"]},{"osd":4,"uuid":"c8ef48c7-7c78-44d3-a9e5-99ec750abd3d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6808","nonce":3465868015},{"type":"v1","addr":"172.21.15.106:6809","nonce":3465868015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6810","nonce":3465868015},{"type":"v1","addr":"172.21.15.106:6811","nonce":3465868015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6814","nonce":3465868015},{"type":"v1","addr":"172.21.15.106:6815","nonce":3465868015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6812","nonce":3465868015},{"type":"v1","addr":"172.21.15.106:6813","nonce":3465868015}]},"public_addr":"172.21.15.106:6809/3465868015","cluster_addr":"172.21.15.106:6811/3465868015","heartbeat_back_addr":"172.21.15.106:6815/3465868015","heartbeat_front_addr":"172.21.15.106:6813/3465868015","state":["exists","up"]},{"osd":5,"uuid":"9e41824a-c5a2-4f88-be50-2ba1dceb3220","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":1485092643},{"type":"v1","addr":"172.21.15.191:6809","nonce":1485092643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":1485092643},{"type":"v1","addr":"172.21.15.191:6811","nonce":1485092643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":1485092643},{"type":"v1","addr":"172.21.15.191:6815","nonce":1485092643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":1485092643},{"type":"v1","addr":"172.21.15.191:6813","nonce":1485092643}]},"public_addr":"172.21.15.191:6809/1485092643","cluster_addr":"172.21.15.191:6811/1485092643","heartbeat_back_addr":"172.21.15.191:6815/1485092643","heartbeat_front_addr":"172.21.15.191:6813/1485092643","state":["exists","up"]},{"osd":6,"uuid":"b35dccee-ec16-452e-ac69-3ac2fb7b27fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6818","nonce":1989797856},{"type":"v1","addr":"172.21.15.52:6819","nonce":1989797856}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6820","nonce":1989797856},{"type":"v1","addr":"172.21.15.52:6821","nonce":1989797856}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6824","nonce":1989797856},{"type":"v1","addr":"172.21.15.52:6825","nonce":1989797856}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6822","nonce":1989797856},{"type":"v1","addr":"172.21.15.52:6823","nonce":1989797856}]},"public_addr":"172.21.15.52:6819/1989797856","cluster_addr":"172.21.15.52:6821/1989797856","heartbeat_back_addr":"172.21.15.52:6825/1989797856","heartbeat_front_addr":"172.21.15.52:6823/1989797856","state":["exists","up"]},{"osd":7,"uuid":"71303e3d-14ec-4cbd-b19c-67645a1de3c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6816","nonce":1450870329},{"type":"v1","addr":"172.21.15.106:6817","nonce":1450870329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":1450870329},{"type":"v1","addr":"172.21.15.106:6819","nonce":1450870329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":1450870329},{"type":"v1","addr":"172.21.15.106:6823","nonce":1450870329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6820","nonce":1450870329},{"type":"v1","addr":"172.21.15.106:6821","nonce":1450870329}]},"public_addr":"172.21.15.106:6817/1450870329","cluster_addr":"172.21.15.106:6819/1450870329","heartbeat_back_addr":"172.21.15.106:6823/1450870329","heartbeat_front_addr":"172.21.15.106:6821/1450870329","state":["exists","up"]},{"osd":8,"uuid":"6f5786c4-4402-435a-bdad-adff46fe9473","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":1114119072},{"type":"v1","addr":"172.21.15.191:6817","nonce":1114119072}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6818","nonce":1114119072},{"type":"v1","addr":"172.21.15.191:6819","nonce":1114119072}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":1114119072},{"type":"v1","addr":"172.21.15.191:6823","nonce":1114119072}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":1114119072},{"type":"v1","addr":"172.21.15.191:6821","nonce":1114119072}]},"public_addr":"172.21.15.191:6817/1114119072","cluster_addr":"172.21.15.191:6819/1114119072","heartbeat_back_addr":"172.21.15.191:6823/1114119072","heartbeat_front_addr":"172.21.15.191:6821/1114119072","state":["exists","up"]},{"osd":9,"uuid":"062bbaa5-1227-4684-b859-450f5594a90c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6826","nonce":3485821914},{"type":"v1","addr":"172.21.15.52:6827","nonce":3485821914}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6828","nonce":3485821914},{"type":"v1","addr":"172.21.15.52:6829","nonce":3485821914}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6832","nonce":3485821914},{"type":"v1","addr":"172.21.15.52:6833","nonce":3485821914}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6830","nonce":3485821914},{"type":"v1","addr":"172.21.15.52:6831","nonce":3485821914}]},"public_addr":"172.21.15.52:6827/3485821914","cluster_addr":"172.21.15.52:6829/3485821914","heartbeat_back_addr":"172.21.15.52:6833/3485821914","heartbeat_front_addr":"172.21.15.52:6831/3485821914","state":["exists","up"]},{"osd":10,"uuid":"1a285a5a-f793-48da-a8eb-5326d9528cd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":3815586576},{"type":"v1","addr":"172.21.15.106:6825","nonce":3815586576}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6826","nonce":3815586576},{"type":"v1","addr":"172.21.15.106:6827","nonce":3815586576}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":3815586576},{"type":"v1","addr":"172.21.15.106:6831","nonce":3815586576}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":3815586576},{"type":"v1","addr":"172.21.15.106:6829","nonce":3815586576}]},"public_addr":"172.21.15.106:6825/3815586576","cluster_addr":"172.21.15.106:6827/3815586576","heartbeat_back_addr":"172.21.15.106:6831/3815586576","heartbeat_front_addr":"172.21.15.106:6829/3815586576","state":["exists","up"]},{"osd":11,"uuid":"1f8b9a43-57fa-41f4-9969-3b803a74bf45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":2009632843},{"type":"v1","addr":"172.21.15.191:6825","nonce":2009632843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":2009632843},{"type":"v1","addr":"172.21.15.191:6827","nonce":2009632843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":2009632843},{"type":"v1","addr":"172.21.15.191:6831","nonce":2009632843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":2009632843},{"type":"v1","addr":"172.21.15.191:6829","nonce":2009632843}]},"public_addr":"172.21.15.191:6825/2009632843","cluster_addr":"172.21.15.191:6827/2009632843","heartbeat_back_addr":"172.21.15.191:6831/2009632843","heartbeat_front_addr":"172.21.15.191:6829/2009632843","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T09:23:10.894763+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T09:23:34.420055+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T09:23:58.288324+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T09:24:23.673457+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T09:24:48.208561+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T09:25:13.405473+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T09:25:38.670915+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T09:26:04.451257+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T09:26:30.281380+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T09:26:56.543143+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T09:27:22.412930+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T09:27:48.003866+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.20","mappings":[{"from":5,"to":9}]},{"pgid":"3.21","mappings":[{"from":4,"to":6}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]},{"pgid":"4.9","mappings":[{"from":5,"to":8}]},{"pgid":"4.17","mappings":[{"from":9,"to":8}]},{"pgid":"4.1b","mappings":[{"from":9,"to":10}]},{"pgid":"4.1f","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.52:0/4082645718":"2024-03-05T09:21:39.401935+0000","172.21.15.52:0/1271949988":"2024-03-05T09:21:39.401935+0000","172.21.15.52:6801/2379361333":"2024-03-05T09:21:12.760938+0000","172.21.15.52:0/497075075":"2024-03-05T09:21:12.760938+0000","172.21.15.52:6800/2379361333":"2024-03-05T09:21:12.760938+0000","172.21.15.52:0/568242577":"2024-03-05T09:21:12.760938+0000","172.21.15.52:6801/2425730636":"2024-03-05T09:21:39.401935+0000","172.21.15.52:6800/2425730636":"2024-03-05T09:21:39.401935+0000","172.21.15.52:0/2743084243":"2024-03-05T09:21:39.401935+0000","172.21.15.52:0/1035878034":"2024-03-05T09:21:12.760938+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-04T09:30:29.172 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.175+0000 7f6f21702640 1 -- 172.21.15.52:0/3100044586 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f6f080a1d80 msgr2=0x7f6f080a4240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:30:29.173 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.175+0000 7f6f21702640 1 --2- 172.21.15.52:0/3100044586 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f6f080a1d80 0x7f6f080a4240 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f6f1c102640 tx=0x7f6f0c005f90 comp rx=0 tx=0).stop 2024-03-04T09:30:29.173 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.175+0000 7f6f21702640 1 -- 172.21.15.52:0/3100044586 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6f1c1031a0 msgr2=0x7f6f1c19ee90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:30:29.173 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.175+0000 7f6f21702640 1 --2- 172.21.15.52:0/3100044586 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6f1c1031a0 0x7f6f1c19ee90 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f6f1400eb10 tx=0x7f6f1400c2d0 comp rx=0 tx=0).stop 2024-03-04T09:30:29.173 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.175+0000 7f6f21702640 1 -- 172.21.15.52:0/3100044586 shutdown_connections 2024-03-04T09:30:29.173 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.175+0000 7f6f21702640 1 --2- 172.21.15.52:0/3100044586 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f6f080a1d80 0x7f6f080a4240 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:29.173 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.175+0000 7f6f21702640 1 --2- 172.21.15.52:0/3100044586 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6f1c1031a0 0x7f6f1c19ee90 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:29.173 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.175+0000 7f6f21702640 1 --2- 172.21.15.52:0/3100044586 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6f1c1027e0 0x7f6f1c197e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:29.173 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.175+0000 7f6f21702640 1 --2- 172.21.15.52:0/3100044586 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6f1c1015f0 0x7f6f1c1978e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:29.173 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.175+0000 7f6f21702640 1 -- 172.21.15.52:0/3100044586 >> 172.21.15.52:0/3100044586 conn(0x7f6f1c0fcb80 msgr2=0x7f6f1c105ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:30:29.173 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.175+0000 7f6f21702640 1 -- 172.21.15.52:0/3100044586 shutdown_connections 2024-03-04T09:30:29.173 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.175+0000 7f6f21702640 1 -- 172.21.15.52:0/3100044586 wait complete. 2024-03-04T09:30:29.184 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T09:30:29.289 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.290+0000 7f923f39a640 1 -- 172.21.15.52:0/1589072212 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9238102120 msgr2=0x7f92381089b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:30:29.289 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.290+0000 7f923f39a640 1 --2- 172.21.15.52:0/1589072212 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9238102120 0x7f92381089b0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f9230009ca0 tx=0x7f923002c0f0 comp rx=0 tx=0).stop 2024-03-04T09:30:29.289 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.290+0000 7f923f39a640 1 -- 172.21.15.52:0/1589072212 shutdown_connections 2024-03-04T09:30:29.290 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.290+0000 7f923f39a640 1 --2- 172.21.15.52:0/1589072212 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9238102120 0x7f92381089b0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:29.290 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.290+0000 7f923f39a640 1 --2- 172.21.15.52:0/1589072212 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9238101760 0x7f9238101be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:29.290 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.290+0000 7f923f39a640 1 --2- 172.21.15.52:0/1589072212 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9238100570 0x7f9238100990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:29.290 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.290+0000 7f923f39a640 1 -- 172.21.15.52:0/1589072212 >> 172.21.15.52:0/1589072212 conn(0x7f92380fbb00 msgr2=0x7f92380fdf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:30:29.290 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.290+0000 7f923f39a640 1 -- 172.21.15.52:0/1589072212 shutdown_connections 2024-03-04T09:30:29.290 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.290+0000 7f923f39a640 1 -- 172.21.15.52:0/1589072212 wait complete. 2024-03-04T09:30:29.290 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.291+0000 7f923f39a640 1 Processor -- start 2024-03-04T09:30:29.290 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.291+0000 7f923f39a640 1 -- start start 2024-03-04T09:30:29.290 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.291+0000 7f923f39a640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9238100570 0x7f9238196780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:30:29.290 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.291+0000 7f923f39a640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9238101760 0x7f9238196cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:30:29.290 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.291+0000 7f923f39a640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9238102120 0x7f923819dd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:30:29.290 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.291+0000 7f923f39a640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f923810c190 con 0x7f9238102120 2024-03-04T09:30:29.290 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.291+0000 7f923f39a640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f923810c010 con 0x7f9238101760 2024-03-04T09:30:29.291 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.291+0000 7f923f39a640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f923810c310 con 0x7f9238100570 2024-03-04T09:30:29.291 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.291+0000 7f923d937640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9238102120 0x7f923819dd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:30:29.291 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.291+0000 7f923d937640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9238102120 0x7f923819dd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:52212/0 (socket says 172.21.15.52:52212) 2024-03-04T09:30:29.291 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.291+0000 7f923d937640 1 -- 172.21.15.52:0/345658193 learned_addr learned my addr 172.21.15.52:0/345658193 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-04T09:30:29.291 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.291+0000 7f923d136640 1 --2- 172.21.15.52:0/345658193 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9238100570 0x7f9238196780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:30:29.291 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.291+0000 7f923c935640 1 --2- 172.21.15.52:0/345658193 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9238101760 0x7f9238196cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:30:29.291 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.291+0000 7f923d937640 1 -- 172.21.15.52:0/345658193 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9238100570 msgr2=0x7f9238196780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:30:29.291 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.291+0000 7f923d937640 1 --2- 172.21.15.52:0/345658193 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9238100570 0x7f9238196780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:29.291 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.291+0000 7f923d937640 1 -- 172.21.15.52:0/345658193 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9238101760 msgr2=0x7f9238196cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:30:29.291 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.291+0000 7f923d937640 1 --2- 172.21.15.52:0/345658193 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9238101760 0x7f9238196cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:29.291 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.291+0000 7f923d937640 1 -- 172.21.15.52:0/345658193 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f923819e3a0 con 0x7f9238102120 2024-03-04T09:30:29.291 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.291+0000 7f923d136640 1 --2- 172.21.15.52:0/345658193 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9238100570 0x7f9238196780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-04T09:30:29.291 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.291+0000 7f923d937640 1 --2- 172.21.15.52:0/345658193 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9238102120 0x7f923819dd30 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f9230037040 tx=0x7f9230032f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:30:29.291 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.292+0000 7f922e7fc640 1 -- 172.21.15.52:0/345658193 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9230041020 con 0x7f9238102120 2024-03-04T09:30:29.292 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.292+0000 7f922e7fc640 1 -- 172.21.15.52:0/345658193 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9230034cd0 con 0x7f9238102120 2024-03-04T09:30:29.292 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.292+0000 7f922e7fc640 1 -- 172.21.15.52:0/345658193 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9230039040 con 0x7f9238102120 2024-03-04T09:30:29.292 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.292+0000 7f923f39a640 1 -- 172.21.15.52:0/345658193 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f923819e630 con 0x7f9238102120 2024-03-04T09:30:29.292 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.292+0000 7f923f39a640 1 -- 172.21.15.52:0/345658193 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f923819eb20 con 0x7f9238102120 2024-03-04T09:30:29.293 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.293+0000 7f923f39a640 1 -- 172.21.15.52:0/345658193 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9238107830 con 0x7f9238102120 2024-03-04T09:30:29.293 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.294+0000 7f922e7fc640 1 -- 172.21.15.52:0/345658193 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) v1 ==== 136756+0+0 (secure 0 0 0) 0x7f9230044020 con 0x7f9238102120 2024-03-04T09:30:29.293 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.295+0000 7f922e7fc640 1 --2- 172.21.15.52:0/345658193 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f92240a1d80 0x7f92240a4240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:30:29.293 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.295+0000 7f923d136640 1 --2- 172.21.15.52:0/345658193 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f92240a1d80 0x7f92240a4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:30:29.293 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.295+0000 7f922e7fc640 1 -- 172.21.15.52:0/345658193 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9092+0+0 (secure 0 0 0) 0x7f92300effa0 con 0x7f9238102120 2024-03-04T09:30:29.293 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.295+0000 7f923d136640 1 --2- 172.21.15.52:0/345658193 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f92240a1d80 0x7f92240a4240 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f92381015c0 tx=0x7f9234005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:30:29.296 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.299+0000 7f922e7fc640 1 -- 172.21.15.52:0/345658193 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f923004b020 con 0x7f9238102120 2024-03-04T09:30:29.518 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.521+0000 7f923f39a640 1 -- 172.21.15.52:0/345658193 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f92380621d0 con 0x7f9238102120 2024-03-04T09:30:29.518 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.521+0000 7f922e7fc640 1 -- 172.21.15.52:0/345658193 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+11406 (secure 0 0 0) 0x7f92300bd100 con 0x7f9238102120 2024-03-04T09:30:29.519 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-03-04T09:30:29.519 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6835/1007961019","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6834","nonce":1007961019},{"type":"v1","addr":"172.21.15.52:6835","nonce":1007961019}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6837/3753881283","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6836","nonce":3753881283},{"type":"v1","addr":"172.21.15.52:6837","nonce":3753881283}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14628,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6839/3467897808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6838","nonce":3467897808},{"type":"v1","addr":"172.21.15.52:6839","nonce":3467897808}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14640,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6841/2804292291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6840","nonce":2804292291},{"type":"v1","addr":"172.21.15.52:6841","nonce":2804292291}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24407,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6833/1818480323","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6832","nonce":1818480323},{"type":"v1","addr":"172.21.15.106:6833","nonce":1818480323}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24412,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6833/765768204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":765768204},{"type":"v1","addr":"172.21.15.191:6833","nonce":765768204}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-04T09:28:37.264970+0000","modified":"2024-03-04T09:29:04.001848+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24506,"mds_1":24490,"mds_2":24473},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24440":{"gid":24440,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.106:6835/1541069897","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6834","nonce":1541069897},{"type":"v1","addr":"172.21.15.106:6835","nonce":1541069897}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24451":{"gid":24451,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.191:6835/105048699","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":105048699},{"type":"v1","addr":"172.21.15.191:6835","nonce":105048699}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24463":{"gid":24463,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.191:6837/4071064225","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6836","nonce":4071064225},{"type":"v1","addr":"172.21.15.191:6837","nonce":4071064225}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24473":{"gid":24473,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.106:6837/2901755066","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6836","nonce":2901755066},{"type":"v1","addr":"172.21.15.106:6837","nonce":2901755066}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24490":{"gid":24490,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.191:6839/2194324891","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6838","nonce":2194324891},{"type":"v1","addr":"172.21.15.191:6839","nonce":2194324891}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24506":{"gid":24506,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.106:6839/1995853093","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6838","nonce":1995853093},{"type":"v1","addr":"172.21.15.106:6839","nonce":1995853093}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24506,"qdb_cluster":[24473,24490,24506]},"id":1}]} 2024-03-04T09:30:29.519 INFO:teuthology.orchestra.run.smithi052.stderr:dumped fsmap epoch 24 2024-03-04T09:30:29.520 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.523+0000 7f923f39a640 1 -- 172.21.15.52:0/345658193 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f92240a1d80 msgr2=0x7f92240a4240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:30:29.520 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.523+0000 7f923f39a640 1 --2- 172.21.15.52:0/345658193 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f92240a1d80 0x7f92240a4240 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f92381015c0 tx=0x7f9234005f90 comp rx=0 tx=0).stop 2024-03-04T09:30:29.521 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.523+0000 7f923f39a640 1 -- 172.21.15.52:0/345658193 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9238102120 msgr2=0x7f923819dd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:30:29.521 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.523+0000 7f923f39a640 1 --2- 172.21.15.52:0/345658193 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9238102120 0x7f923819dd30 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f9230037040 tx=0x7f9230032f40 comp rx=0 tx=0).stop 2024-03-04T09:30:29.521 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.523+0000 7f923f39a640 1 -- 172.21.15.52:0/345658193 shutdown_connections 2024-03-04T09:30:29.521 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.523+0000 7f923f39a640 1 --2- 172.21.15.52:0/345658193 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f92240a1d80 0x7f92240a4240 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:29.521 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.523+0000 7f923f39a640 1 --2- 172.21.15.52:0/345658193 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9238102120 0x7f923819dd30 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:29.521 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.523+0000 7f923f39a640 1 --2- 172.21.15.52:0/345658193 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9238101760 0x7f9238196cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:29.521 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.523+0000 7f923f39a640 1 --2- 172.21.15.52:0/345658193 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9238100570 0x7f9238196780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:29.521 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.523+0000 7f923f39a640 1 -- 172.21.15.52:0/345658193 >> 172.21.15.52:0/345658193 conn(0x7f92380fbb00 msgr2=0x7f9238104a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:30:29.521 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.524+0000 7f923f39a640 1 -- 172.21.15.52:0/345658193 shutdown_connections 2024-03-04T09:30:29.521 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.524+0000 7f923f39a640 1 -- 172.21.15.52:0/345658193 wait complete. 2024-03-04T09:30:29.530 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:30:29 smithi052 ceph-mon[32473]: pgmap v345: 129 pgs: 129 active+clean; 641 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-03-04T09:30:29.531 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:30:29 smithi052 ceph-mon[32473]: from='client.? 172.21.15.52:0/617865048' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-04T09:30:29.531 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:30:29 smithi052 ceph-mon[32473]: from='client.? 172.21.15.52:0/3102207095' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-04T09:30:29.531 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:30:29 smithi052 ceph-mon[32473]: from='client.? 172.21.15.52:0/3100044586' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-04T09:30:29.532 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 24, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-03-04T09:28:37.264970+0000', 'modified': '2024-03-04T09:29:04.001848+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 24506, 'mds_1': 24490, 'mds_2': 24473}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24440': {'gid': 24440, 'name': 'e', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.106:6835/1541069897', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.106:6834', 'nonce': 1541069897}, {'type': 'v1', 'addr': '172.21.15.106:6835', 'nonce': 1541069897}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24451': {'gid': 24451, 'name': 'f', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.191:6835/105048699', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.191:6834', 'nonce': 105048699}, {'type': 'v1', 'addr': '172.21.15.191:6835', 'nonce': 105048699}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24463': {'gid': 24463, 'name': 'c', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.191:6837/4071064225', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.191:6836', 'nonce': 4071064225}, {'type': 'v1', 'addr': '172.21.15.191:6837', 'nonce': 4071064225}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24473': {'gid': 24473, 'name': 'h', 'rank': 2, 'incarnation': 22, 'state': 'up:active', 'state_seq': 12, 'addr': '172.21.15.106:6837/2901755066', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.106:6836', 'nonce': 2901755066}, {'type': 'v1', 'addr': '172.21.15.106:6837', 'nonce': 2901755066}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24490': {'gid': 24490, 'name': 'i', 'rank': 1, 'incarnation': 20, 'state': 'up:active', 'state_seq': 11, 'addr': '172.21.15.191:6839/2194324891', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.191:6838', 'nonce': 2194324891}, {'type': 'v1', 'addr': '172.21.15.191:6839', 'nonce': 2194324891}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24506': {'gid': 24506, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.106:6839/1995853093', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.106:6838', 'nonce': 1995853093}, {'type': 'v1', 'addr': '172.21.15.106:6839', 'nonce': 1995853093}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24506, 'qdb_cluster': [24473, 24490, 24506]} 2024-03-04T09:30:29.532 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-03-04T09:30:29.533 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.h status 2024-03-04T09:30:29.533 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:30:29 smithi106 ceph-mon[36484]: pgmap v345: 129 pgs: 129 active+clean; 641 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-03-04T09:30:29.533 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:30:29 smithi106 ceph-mon[36484]: from='client.? 172.21.15.52:0/617865048' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-04T09:30:29.533 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:30:29 smithi106 ceph-mon[36484]: from='client.? 172.21.15.52:0/3102207095' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-04T09:30:29.533 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:30:29 smithi106 ceph-mon[36484]: from='client.? 172.21.15.52:0/3100044586' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-04T09:30:29.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:30:29 smithi191 ceph-mon[36816]: pgmap v345: 129 pgs: 129 active+clean; 641 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-03-04T09:30:29.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:30:29 smithi191 ceph-mon[36816]: from='client.? 172.21.15.52:0/617865048' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-04T09:30:29.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:30:29 smithi191 ceph-mon[36816]: from='client.? 172.21.15.52:0/3102207095' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-04T09:30:29.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:30:29 smithi191 ceph-mon[36816]: from='client.? 172.21.15.52:0/3100044586' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-04T09:30:29.639 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.639+0000 7fbe8fceb640 1 -- 172.21.15.52:0/361829738 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbe88103190 msgr2=0x7fbe88109a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:30:29.640 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.639+0000 7fbe8fceb640 1 --2- 172.21.15.52:0/361829738 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbe88103190 0x7fbe88109a20 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fbe80009ca0 tx=0x7fbe8002c0f0 comp rx=0 tx=0).stop 2024-03-04T09:30:29.640 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.639+0000 7fbe8fceb640 1 -- 172.21.15.52:0/361829738 shutdown_connections 2024-03-04T09:30:29.640 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.639+0000 7fbe8fceb640 1 --2- 172.21.15.52:0/361829738 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbe88103190 0x7fbe88109a20 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:29.640 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.639+0000 7fbe8fceb640 1 --2- 172.21.15.52:0/361829738 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbe881027d0 0x7fbe88102c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:29.640 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.639+0000 7fbe8fceb640 1 --2- 172.21.15.52:0/361829738 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbe881015e0 0x7fbe88101a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:29.640 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.639+0000 7fbe8fceb640 1 -- 172.21.15.52:0/361829738 >> 172.21.15.52:0/361829738 conn(0x7fbe880fcb70 msgr2=0x7fbe880fef90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:30:29.640 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.639+0000 7fbe8fceb640 1 -- 172.21.15.52:0/361829738 shutdown_connections 2024-03-04T09:30:29.641 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.639+0000 7fbe8fceb640 1 -- 172.21.15.52:0/361829738 wait complete. 2024-03-04T09:30:29.641 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.640+0000 7fbe8fceb640 1 Processor -- start 2024-03-04T09:30:29.641 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.640+0000 7fbe8fceb640 1 -- start start 2024-03-04T09:30:29.641 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.640+0000 7fbe8fceb640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbe881015e0 0x7fbe881978a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:30:29.641 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.640+0000 7fbe8fceb640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbe881027d0 0x7fbe88197de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:30:29.641 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.640+0000 7fbe8fceb640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbe88103190 0x7fbe8819ee50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:30:29.641 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.640+0000 7fbe8fceb640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe8810d260 con 0x7fbe881027d0 2024-03-04T09:30:29.641 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.640+0000 7fbe8fceb640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe8810d0e0 con 0x7fbe881015e0 2024-03-04T09:30:29.641 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.640+0000 7fbe8fceb640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe8810d3e0 con 0x7fbe88103190 2024-03-04T09:30:29.641 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.640+0000 7fbe8d286640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbe881027d0 0x7fbe88197de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:30:29.642 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.640+0000 7fbe8da87640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbe881015e0 0x7fbe881978a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:30:29.642 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.640+0000 7fbe8d286640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbe881027d0 0x7fbe88197de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:52222/0 (socket says 172.21.15.52:52222) 2024-03-04T09:30:29.642 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.640+0000 7fbe8d286640 1 -- 172.21.15.52:0/392474011 learned_addr learned my addr 172.21.15.52:0/392474011 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-04T09:30:29.642 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.640+0000 7fbe8e288640 1 --2- 172.21.15.52:0/392474011 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbe88103190 0x7fbe8819ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:30:29.642 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.640+0000 7fbe8d286640 1 -- 172.21.15.52:0/392474011 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbe88103190 msgr2=0x7fbe8819ee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:30:29.642 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.640+0000 7fbe8d286640 1 --2- 172.21.15.52:0/392474011 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbe88103190 0x7fbe8819ee50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:29.642 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.640+0000 7fbe8d286640 1 -- 172.21.15.52:0/392474011 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbe881015e0 msgr2=0x7fbe881978a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:30:29.642 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.640+0000 7fbe8d286640 1 --2- 172.21.15.52:0/392474011 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbe881015e0 0x7fbe881978a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:29.642 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.640+0000 7fbe8d286640 1 -- 172.21.15.52:0/392474011 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe8819f4c0 con 0x7fbe881027d0 2024-03-04T09:30:29.642 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.640+0000 7fbe8d286640 1 --2- 172.21.15.52:0/392474011 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbe881027d0 0x7fbe88197de0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fbe7800d950 tx=0x7fbe7800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:30:29.642 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.641+0000 7fbe7effd640 1 -- 172.21.15.52:0/392474011 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe78017030 con 0x7fbe881027d0 2024-03-04T09:30:29.642 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.641+0000 7fbe7effd640 1 -- 172.21.15.52:0/392474011 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fbe78004020 con 0x7fbe881027d0 2024-03-04T09:30:29.643 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.641+0000 7fbe8fceb640 1 -- 172.21.15.52:0/392474011 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe8819f7b0 con 0x7fbe881027d0 2024-03-04T09:30:29.643 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.641+0000 7fbe7effd640 1 -- 172.21.15.52:0/392474011 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe780047a0 con 0x7fbe881027d0 2024-03-04T09:30:29.643 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.641+0000 7fbe8fceb640 1 -- 172.21.15.52:0/392474011 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe8819fd50 con 0x7fbe881027d0 2024-03-04T09:30:29.647 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.643+0000 7fbe7effd640 1 -- 172.21.15.52:0/392474011 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) v1 ==== 136756+0+0 (secure 0 0 0) 0x7fbe780041d0 con 0x7fbe881027d0 2024-03-04T09:30:29.647 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.643+0000 7fbe929aa740 1 Processor -- start 2024-03-04T09:30:29.647 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.644+0000 7fbe929aa740 1 -- start start 2024-03-04T09:30:29.647 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.644+0000 7fbe929aa740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55663d5867b0 0x55663d586bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:30:29.647 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.644+0000 7fbe929aa740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55663d587980 0x55663d587e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:30:29.647 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.644+0000 7fbe929aa740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55663d588340 0x55663d58ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:30:29.647 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.644+0000 7fbe929aa740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x55663d4eb6b0 con 0x55663d5867b0 2024-03-04T09:30:29.648 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.644+0000 7fbe929aa740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x55663d494a60 con 0x55663d588340 2024-03-04T09:30:29.648 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.644+0000 7fbe929aa740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x55663d4e7750 con 0x55663d587980 2024-03-04T09:30:29.648 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.644+0000 7fbe7effd640 1 --2- 172.21.15.52:0/392474011 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7fbe740a1e00 0x7fbe740a42c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:30:29.648 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.644+0000 7fbe8e288640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55663d5867b0 0x55663d586bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:30:29.648 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.644+0000 7fbe8da87640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55663d587980 0x55663d587e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:30:29.648 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.644+0000 7fbe8e288640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55663d5867b0 0x55663d586bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:52236/0 (socket says 172.21.15.52:52236) 2024-03-04T09:30:29.648 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.644+0000 7fbe8e288640 1 -- 172.21.15.52:0/2415698256 learned_addr learned my addr 172.21.15.52:0/2415698256 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-04T09:30:29.648 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.644+0000 7fbe8d286640 1 --2- 172.21.15.52:0/2415698256 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55663d588340 0x55663d58ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:30:29.648 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.644+0000 7fbe8e288640 1 --2- 172.21.15.52:0/392474011 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7fbe740a1e00 0x7fbe740a42c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:30:29.649 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.644+0000 7fbe7effd640 1 -- 172.21.15.52:0/392474011 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9092+0+0 (secure 0 0 0) 0x7fbe780cd9e0 con 0x7fbe881027d0 2024-03-04T09:30:29.649 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.644+0000 7fbe8da87640 1 -- 172.21.15.52:0/2415698256 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55663d588340 msgr2=0x55663d58ebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T09:30:29.649 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.644+0000 7fbe8da87640 1 --2- 172.21.15.52:0/2415698256 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55663d588340 0x55663d58ebd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:29.649 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.644+0000 7fbe8da87640 1 -- 172.21.15.52:0/2415698256 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55663d5867b0 msgr2=0x55663d586bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T09:30:29.649 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.644+0000 7fbe8da87640 1 --2- 172.21.15.52:0/2415698256 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55663d5867b0 0x55663d586bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:29.649 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.644+0000 7fbe8da87640 1 -- 172.21.15.52:0/2415698256 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55663d4b2f30 con 0x55663d587980 2024-03-04T09:30:29.649 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.645+0000 7fbe8e288640 1 --2- 172.21.15.52:0/392474011 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7fbe740a1e00 0x7fbe740a42c0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x55663d587880 tx=0x7fbe80032590 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:30:29.649 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.645+0000 7fbe8da87640 1 --2- 172.21.15.52:0/2415698256 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55663d587980 0x55663d587e00 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fbe84009bd0 tx=0x7fbe84011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bd022628bc28d1eb server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:30:29.649 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.645+0000 7fbe73fff640 1 -- 172.21.15.52:0/2415698256 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe84004230 con 0x55663d587980 2024-03-04T09:30:29.649 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.645+0000 7fbe73fff640 1 -- 172.21.15.52:0/2415698256 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fbe84009e40 con 0x55663d587980 2024-03-04T09:30:29.649 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.645+0000 7fbe73fff640 1 -- 172.21.15.52:0/2415698256 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe84019750 con 0x55663d587980 2024-03-04T09:30:29.649 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.645+0000 7fbe929aa740 1 -- 172.21.15.52:0/2415698256 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55663d587980 msgr2=0x55663d587e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:30:29.649 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.645+0000 7fbe929aa740 1 --2- 172.21.15.52:0/2415698256 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55663d587980 0x55663d587e00 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fbe84009bd0 tx=0x7fbe84011810 comp rx=0 tx=0).stop 2024-03-04T09:30:29.649 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.645+0000 7fbe929aa740 1 -- 172.21.15.52:0/2415698256 shutdown_connections 2024-03-04T09:30:29.650 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.645+0000 7fbe929aa740 1 --2- 172.21.15.52:0/2415698256 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55663d588340 0x55663d58ebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:29.650 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.645+0000 7fbe929aa740 1 --2- 172.21.15.52:0/2415698256 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55663d587980 0x55663d587e00 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:29.650 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.645+0000 7fbe929aa740 1 --2- 172.21.15.52:0/2415698256 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55663d5867b0 0x55663d586bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:29.650 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.645+0000 7fbe929aa740 1 -- 172.21.15.52:0/2415698256 >> 172.21.15.52:0/2415698256 conn(0x55663d581ba0 msgr2=0x55663d581fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:30:29.650 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.646+0000 7fbe929aa740 1 -- 172.21.15.52:0/2415698256 shutdown_connections 2024-03-04T09:30:29.650 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.646+0000 7fbe929aa740 1 -- 172.21.15.52:0/2415698256 wait complete. 2024-03-04T09:30:29.650 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.646+0000 7fbe929aa740 1 -- start start 2024-03-04T09:30:29.650 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.646+0000 7fbe929aa740 1 Processor -- start 2024-03-04T09:30:29.650 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.646+0000 7fbe929aa740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55663d5867b0 0x55663d590cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:30:29.650 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.646+0000 7fbe929aa740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55663d587980 0x55663d591210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:30:29.650 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.646+0000 7fbe929aa740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55663d588340 0x55663d598250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:30:29.650 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.646+0000 7fbe929aa740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x55663d58f350 con 0x55663d588340 2024-03-04T09:30:29.650 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.646+0000 7fbe929aa740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x55663d4a7ea0 con 0x55663d587980 2024-03-04T09:30:29.651 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.646+0000 7fbe929aa740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x55663d58f4d0 con 0x55663d5867b0 2024-03-04T09:30:29.651 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.646+0000 7fbe929aa740 10 client.0 _ll_register_callbacks cb 0x55663d4a7870 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-04T09:30:29.651 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.646+0000 7fbe8e288640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55663d588340 0x55663d598250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:30:29.651 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.646+0000 7fbe8da87640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55663d5867b0 0x55663d590cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:30:29.651 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.646+0000 7fbe8e288640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55663d588340 0x55663d598250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:52240/0 (socket says 172.21.15.52:52240) 2024-03-04T09:30:29.651 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.646+0000 7fbe8e288640 1 -- 172.21.15.52:0/1635572089 learned_addr learned my addr 172.21.15.52:0/1635572089 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-04T09:30:29.651 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.646+0000 7fbe8d286640 1 --2- 172.21.15.52:0/1635572089 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55663d587980 0x55663d591210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:30:29.651 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.647+0000 7fbe8da87640 1 -- 172.21.15.52:0/1635572089 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55663d587980 msgr2=0x55663d591210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T09:30:29.651 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.647+0000 7fbe8da87640 1 --2- 172.21.15.52:0/1635572089 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55663d587980 0x55663d591210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:29.651 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.647+0000 7fbe8da87640 1 -- 172.21.15.52:0/1635572089 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55663d588340 msgr2=0x55663d598250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T09:30:29.651 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.647+0000 7fbe8da87640 1 --2- 172.21.15.52:0/1635572089 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55663d588340 0x55663d598250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:29.651 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.647+0000 7fbe8da87640 1 -- 172.21.15.52:0/1635572089 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55663d4a8270 con 0x55663d5867b0 2024-03-04T09:30:29.651 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.647+0000 7fbe8d286640 1 --2- 172.21.15.52:0/1635572089 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55663d587980 0x55663d591210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-04T09:30:29.651 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.647+0000 7fbe8e288640 1 --2- 172.21.15.52:0/1635572089 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55663d588340 0x55663d598250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-04T09:30:29.652 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.647+0000 7fbe8da87640 1 --2- 172.21.15.52:0/1635572089 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55663d5867b0 0x55663d590cd0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fbe8401b040 tx=0x7fbe840041d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=200a1be5ef332500 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:30:29.652 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.647+0000 7fbe717fa640 10 client.0 ms_handle_connect on v2:172.21.15.191:3300/0 2024-03-04T09:30:29.656 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.647+0000 7fbe717fa640 1 -- 172.21.15.52:0/1635572089 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe84028070 con 0x55663d5867b0 2024-03-04T09:30:29.656 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.647+0000 7fbe929aa740 1 -- 172.21.15.52:0/1635572089 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55663d4f8bf0 con 0x55663d5867b0 2024-03-04T09:30:29.656 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.647+0000 7fbe717fa640 1 -- 172.21.15.52:0/1635572089 <== mon.1 v2:172.21.15.191:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9092+0+0 (secure 0 0 0) 0x7fbe84005040 con 0x55663d5867b0 2024-03-04T09:30:29.657 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.647+0000 7fbe717fa640 1 -- 172.21.15.52:0/1635572089 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbe84034050 con 0x55663d5867b0 2024-03-04T09:30:29.657 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.647+0000 7fbe929aa740 10 client.25006 fetch_fsmap learned FSMap version 24 2024-03-04T09:30:29.657 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.647+0000 7fbe929aa740 1 -- 172.21.15.52:0/1635572089 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55663d2002d0 con 0x55663d5867b0 2024-03-04T09:30:29.657 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.647+0000 7fbe717fa640 1 -- 172.21.15.52:0/1635572089 <== mon.1 v2:172.21.15.191:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55663d4f8bf0 con 0x55663d5867b0 2024-03-04T09:30:29.657 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.648+0000 7fbe717fa640 1 -- 172.21.15.52:0/1635572089 <== mon.1 v2:172.21.15.191:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7fbe84004ba0 con 0x55663d5867b0 2024-03-04T09:30:29.657 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.648+0000 7fbe929aa740 10 client.25006 fetch_fsmap finished waiting for FSMap version 24 2024-03-04T09:30:29.657 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.648+0000 7fbe929aa740 10 client.25006 resolve_mds: resolved name 'h' to daemon mds.h 2024-03-04T09:30:29.657 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.648+0000 7fbe929aa740 20 client.25006 populate_metadata read hostname 'smithi052' 2024-03-04T09:30:29.657 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.648+0000 7fbe929aa740 1 --2- 172.21.15.52:0/1635572089 >> [v2:172.21.15.106:6836/2901755066,v1:172.21.15.106:6837/2901755066] conn(0x55663d59dca0 0x55663d59c800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:30:29.657 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.648+0000 7fbe929aa740 4 client.25006 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-04T09:30:29.657 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.648+0000 7fbe929aa740 1 -- 172.21.15.52:0/1635572089 --> [v2:172.21.15.106:6836/2901755066,v1:172.21.15.106:6837/2901755066] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55663d4f2f40 con 0x55663d59dca0 2024-03-04T09:30:29.657 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.648+0000 7fbe8d286640 1 --2- 172.21.15.52:0/1635572089 >> [v2:172.21.15.106:6836/2901755066,v1:172.21.15.106:6837/2901755066] conn(0x55663d59dca0 0x55663d59c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:30:29.657 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.648+0000 7fbe8d286640 1 --2- 172.21.15.52:0/1635572089 >> [v2:172.21.15.106:6836/2901755066,v1:172.21.15.106:6837/2901755066] conn(0x55663d59dca0 0x55663d59c800 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=8389862a656a989b server_cookie=6bd89d8e053eebea in_seq=0 out_seq=0 2024-03-04T09:30:29.658 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.648+0000 7fbe717fa640 10 client.25006 ms_handle_connect on v2:172.21.15.106:6836/2901755066 2024-03-04T09:30:29.658 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.649+0000 7fbe717fa640 1 -- 172.21.15.52:0/1635572089 <== mds.2 v2:172.21.15.106:6836/2901755066 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55663d4f2f40 con 0x55663d59dca0 2024-03-04T09:30:29.658 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.649+0000 7fbe717fa640 10 client.25006 handle_command_reply: tid=0 2024-03-04T09:30:29.658 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.649+0000 7fbe929aa740 1 client.25006 shutdown 2024-03-04T09:30:29.658 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.649+0000 7fbe929aa740 1 -- 172.21.15.52:0/1635572089 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55663d5867b0 msgr2=0x55663d590cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:30:29.658 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.649+0000 7fbe929aa740 1 --2- 172.21.15.52:0/1635572089 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55663d5867b0 0x55663d590cd0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fbe8401b040 tx=0x7fbe840041d0 comp rx=0 tx=0).stop 2024-03-04T09:30:29.658 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.649+0000 7fbe929aa740 1 -- 172.21.15.52:0/1635572089 shutdown_connections 2024-03-04T09:30:29.658 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.649+0000 7fbe929aa740 1 --2- 172.21.15.52:0/1635572089 >> [v2:172.21.15.106:6836/2901755066,v1:172.21.15.106:6837/2901755066] conn(0x55663d59dca0 0x55663d59c800 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:29.658 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.649+0000 7fbe929aa740 1 --2- 172.21.15.52:0/1635572089 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55663d588340 0x55663d598250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:29.658 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.649+0000 7fbe717fa640 0 client.25006 ms_handle_reset on v2:172.21.15.106:6836/2901755066 2024-03-04T09:30:29.658 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.649+0000 7fbe929aa740 1 --2- 172.21.15.52:0/1635572089 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55663d587980 0x55663d591210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:29.658 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.649+0000 7fbe929aa740 1 --2- 172.21.15.52:0/1635572089 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55663d5867b0 0x55663d590cd0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:29.658 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.649+0000 7fbe929aa740 1 -- 172.21.15.52:0/1635572089 >> 172.21.15.52:0/1635572089 conn(0x55663d581ba0 msgr2=0x55663d580da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:30:29.658 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.649+0000 7fbe929aa740 1 -- 172.21.15.52:0/1635572089 shutdown_connections 2024-03-04T09:30:29.659 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.649+0000 7fbe929aa740 1 -- 172.21.15.52:0/1635572089 wait complete. 2024-03-04T09:30:29.659 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.649+0000 7fbe929aa740 20 client.25006 trim_cache size 0 max 16384 2024-03-04T09:30:29.668 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.666+0000 7fbe929aa740 1 Processor -- start 2024-03-04T09:30:29.669 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.666+0000 7fbe929aa740 1 -- start start 2024-03-04T09:30:29.669 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.666+0000 7fbe929aa740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55663d5867b0 0x55663d581f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:30:29.669 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.666+0000 7fbe929aa740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55663d587980 0x55663d598210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:30:29.669 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.666+0000 7fbe929aa740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55663d588340 0x55663d580510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:30:29.669 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.666+0000 7fbe929aa740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x55663d59cfc0 con 0x55663d588340 2024-03-04T09:30:29.669 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.666+0000 7fbe929aa740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x55663d59d4c0 con 0x55663d587980 2024-03-04T09:30:29.669 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.666+0000 7fbe929aa740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x55663d59db20 con 0x55663d5867b0 2024-03-04T09:30:29.669 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.666+0000 7fbe8e288640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55663d588340 0x55663d580510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:30:29.669 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.666+0000 7fbe8e288640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55663d588340 0x55663d580510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:52250/0 (socket says 172.21.15.52:52250) 2024-03-04T09:30:29.669 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.666+0000 7fbe8e288640 1 -- 172.21.15.52:0/2312832367 learned_addr learned my addr 172.21.15.52:0/2312832367 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-04T09:30:29.669 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.666+0000 7fbe8da87640 1 --2- 172.21.15.52:0/2312832367 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55663d5867b0 0x55663d581f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:30:29.670 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.666+0000 7fbe8d286640 1 --2- 172.21.15.52:0/2312832367 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55663d587980 0x55663d598210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:30:29.670 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.667+0000 7fbe8e288640 1 -- 172.21.15.52:0/2312832367 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55663d5867b0 msgr2=0x55663d581f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T09:30:29.670 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.667+0000 7fbe8e288640 1 --2- 172.21.15.52:0/2312832367 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55663d5867b0 0x55663d581f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:29.670 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.667+0000 7fbe8e288640 1 -- 172.21.15.52:0/2312832367 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55663d587980 msgr2=0x55663d598210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T09:30:29.670 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.667+0000 7fbe8e288640 1 --2- 172.21.15.52:0/2312832367 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55663d587980 0x55663d598210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:29.670 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.667+0000 7fbe8e288640 1 -- 172.21.15.52:0/2312832367 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55663d2ca6b0 con 0x55663d588340 2024-03-04T09:30:29.670 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.667+0000 7fbe8da87640 1 --2- 172.21.15.52:0/2312832367 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55663d5867b0 0x55663d581f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-04T09:30:29.670 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.667+0000 7fbe8e288640 1 --2- 172.21.15.52:0/2312832367 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55663d588340 0x55663d580510 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fbe800324c0 tx=0x7fbe8002e040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0a528707372b8f0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:30:29.670 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.667+0000 7fbe737fe640 1 -- 172.21.15.52:0/2312832367 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe80039040 con 0x55663d588340 2024-03-04T09:30:29.670 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.667+0000 7fbe737fe640 1 -- 172.21.15.52:0/2312832367 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fbe80055070 con 0x55663d588340 2024-03-04T09:30:29.670 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.667+0000 7fbe737fe640 1 -- 172.21.15.52:0/2312832367 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe80050a50 con 0x55663d588340 2024-03-04T09:30:29.670 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.668+0000 7fbe929aa740 1 -- 172.21.15.52:0/2312832367 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55663d588340 msgr2=0x55663d580510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:30:29.670 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.668+0000 7fbe929aa740 1 --2- 172.21.15.52:0/2312832367 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55663d588340 0x55663d580510 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fbe800324c0 tx=0x7fbe8002e040 comp rx=0 tx=0).stop 2024-03-04T09:30:29.671 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.668+0000 7fbe929aa740 1 -- 172.21.15.52:0/2312832367 shutdown_connections 2024-03-04T09:30:29.671 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.668+0000 7fbe929aa740 1 --2- 172.21.15.52:0/2312832367 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55663d588340 0x55663d580510 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:29.671 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.668+0000 7fbe929aa740 1 --2- 172.21.15.52:0/2312832367 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55663d587980 0x55663d598210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:29.671 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.668+0000 7fbe929aa740 1 --2- 172.21.15.52:0/2312832367 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55663d5867b0 0x55663d581f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:29.671 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.668+0000 7fbe929aa740 1 -- 172.21.15.52:0/2312832367 >> 172.21.15.52:0/2312832367 conn(0x55663d581ba0 msgr2=0x55663d59c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:30:29.671 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.668+0000 7fbe929aa740 1 -- 172.21.15.52:0/2312832367 shutdown_connections 2024-03-04T09:30:29.671 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.668+0000 7fbe929aa740 1 -- 172.21.15.52:0/2312832367 wait complete. 2024-03-04T09:30:29.671 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.668+0000 7fbe929aa740 1 -- start start 2024-03-04T09:30:29.671 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.668+0000 7fbe929aa740 1 Processor -- start 2024-03-04T09:30:29.671 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.668+0000 7fbe929aa740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55663d5867b0 0x55663d594440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:30:29.671 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.668+0000 7fbe929aa740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55663d587980 0x55663d594980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:30:29.671 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.668+0000 7fbe929aa740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55663d588340 0x55663d597250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:30:29.671 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.668+0000 7fbe929aa740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x55663d59e7e0 con 0x55663d587980 2024-03-04T09:30:29.671 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.668+0000 7fbe929aa740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x55663d59e180 con 0x55663d588340 2024-03-04T09:30:29.672 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.668+0000 7fbe929aa740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x55663d59ee40 con 0x55663d5867b0 2024-03-04T09:30:29.672 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.668+0000 7fbe929aa740 10 client.0 _ll_register_callbacks cb 0x7fbe84009a50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-04T09:30:29.672 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.668+0000 7fbe8d286640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55663d587980 0x55663d594980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:30:29.672 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.668+0000 7fbe8d286640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55663d587980 0x55663d594980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:52266/0 (socket says 172.21.15.52:52266) 2024-03-04T09:30:29.672 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.668+0000 7fbe8d286640 1 -- 172.21.15.52:0/1252670954 learned_addr learned my addr 172.21.15.52:0/1252670954 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-04T09:30:29.672 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.668+0000 7fbe8e288640 1 --2- 172.21.15.52:0/1252670954 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55663d588340 0x55663d597250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:30:29.672 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.668+0000 7fbe8da87640 1 --2- 172.21.15.52:0/1252670954 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55663d5867b0 0x55663d594440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:30:29.672 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.669+0000 7fbe8e288640 1 -- 172.21.15.52:0/1252670954 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55663d5867b0 msgr2=0x55663d594440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T09:30:29.672 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.669+0000 7fbe8e288640 1 --2- 172.21.15.52:0/1252670954 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55663d5867b0 0x55663d594440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:29.672 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.669+0000 7fbe8e288640 1 -- 172.21.15.52:0/1252670954 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55663d587980 msgr2=0x55663d594980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T09:30:29.672 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.669+0000 7fbe8e288640 1 --2- 172.21.15.52:0/1252670954 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55663d587980 0x55663d594980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:29.672 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.669+0000 7fbe8e288640 1 -- 172.21.15.52:0/1252670954 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55663d597810 con 0x55663d588340 2024-03-04T09:30:29.672 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.669+0000 7fbe8d286640 1 --2- 172.21.15.52:0/1252670954 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55663d587980 0x55663d594980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-04T09:30:29.673 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.669+0000 7fbe8e288640 1 --2- 172.21.15.52:0/1252670954 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55663d588340 0x55663d597250 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fbe80005e30 tx=0x7fbe80050cd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=636cd3596d9d50f4 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:30:29.673 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.669+0000 7fbe717fa640 10 client.0 ms_handle_connect on v2:172.21.15.106:3300/0 2024-03-04T09:30:29.673 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.669+0000 7fbe717fa640 1 -- 172.21.15.52:0/1252670954 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe8005e060 con 0x55663d588340 2024-03-04T09:30:29.673 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.669+0000 7fbe929aa740 1 -- 172.21.15.52:0/1252670954 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fbe80055070 con 0x55663d588340 2024-03-04T09:30:29.673 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.669+0000 7fbe717fa640 1 -- 172.21.15.52:0/1252670954 <== mon.2 v2:172.21.15.106:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9092+0+0 (secure 0 0 0) 0x7fbe80004100 con 0x55663d588340 2024-03-04T09:30:29.673 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.669+0000 7fbe717fa640 1 -- 172.21.15.52:0/1252670954 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbe80039030 con 0x55663d588340 2024-03-04T09:30:29.673 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.669+0000 7fbe929aa740 10 client.25079 fetch_fsmap learned FSMap version 24 2024-03-04T09:30:29.673 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.669+0000 7fbe929aa740 1 -- 172.21.15.52:0/1252670954 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55663d5f89c0 con 0x55663d588340 2024-03-04T09:30:29.673 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.669+0000 7fbe717fa640 1 -- 172.21.15.52:0/1252670954 <== mon.2 v2:172.21.15.106:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fbe80055070 con 0x55663d588340 2024-03-04T09:30:29.673 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.670+0000 7fbe717fa640 1 -- 172.21.15.52:0/1252670954 <== mon.2 v2:172.21.15.106:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7fbe800344d0 con 0x55663d588340 2024-03-04T09:30:29.673 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.670+0000 7fbe929aa740 10 client.25079 fetch_fsmap finished waiting for FSMap version 24 2024-03-04T09:30:29.673 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.670+0000 7fbe929aa740 10 client.25079 resolve_mds: resolved name 'h' to daemon mds.h 2024-03-04T09:30:29.673 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.670+0000 7fbe929aa740 20 client.25079 populate_metadata read hostname 'smithi052' 2024-03-04T09:30:29.674 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.670+0000 7fbe929aa740 1 --2- 172.21.15.52:0/1252670954 >> [v2:172.21.15.106:6836/2901755066,v1:172.21.15.106:6837/2901755066] conn(0x55663d5fcbb0 0x55663d5f8b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:30:29.674 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.670+0000 7fbe929aa740 4 client.25079 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-04T09:30:29.674 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.670+0000 7fbe929aa740 1 -- 172.21.15.52:0/1252670954 --> [v2:172.21.15.106:6836/2901755066,v1:172.21.15.106:6837/2901755066] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55663d4719a0 con 0x55663d5fcbb0 2024-03-04T09:30:29.674 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.670+0000 7fbe8da87640 1 --2- 172.21.15.52:0/1252670954 >> [v2:172.21.15.106:6836/2901755066,v1:172.21.15.106:6837/2901755066] conn(0x55663d5fcbb0 0x55663d5f8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:30:29.674 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.670+0000 7fbe8da87640 1 --2- 172.21.15.52:0/1252670954 >> [v2:172.21.15.106:6836/2901755066,v1:172.21.15.106:6837/2901755066] conn(0x55663d5fcbb0 0x55663d5f8b90 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=be53c89263ddc244 server_cookie=d8435cf5fc0e2f6e in_seq=0 out_seq=0 2024-03-04T09:30:29.674 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.670+0000 7fbe717fa640 10 client.25079 ms_handle_connect on v2:172.21.15.106:6836/2901755066 2024-03-04T09:30:29.674 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.670+0000 7fbe717fa640 1 -- 172.21.15.52:0/1252670954 <== mds.2 v2:172.21.15.106:6836/2901755066 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+327 (crc 0 0 0) 0x55663d4719a0 con 0x55663d5fcbb0 2024-03-04T09:30:29.674 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.670+0000 7fbe717fa640 10 client.25079 handle_command_reply: tid=0 2024-03-04T09:30:29.674 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.670+0000 7fbe929aa740 1 client.25079 shutdown 2024-03-04T09:30:29.674 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.670+0000 7fbe929aa740 1 -- 172.21.15.52:0/1252670954 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55663d588340 msgr2=0x55663d597250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:30:29.674 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.670+0000 7fbe929aa740 1 --2- 172.21.15.52:0/1252670954 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55663d588340 0x55663d597250 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fbe80005e30 tx=0x7fbe80050cd0 comp rx=0 tx=0).stop 2024-03-04T09:30:29.674 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.671+0000 7fbe929aa740 1 -- 172.21.15.52:0/1252670954 shutdown_connections 2024-03-04T09:30:29.674 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.671+0000 7fbe929aa740 1 --2- 172.21.15.52:0/1252670954 >> [v2:172.21.15.106:6836/2901755066,v1:172.21.15.106:6837/2901755066] conn(0x55663d5fcbb0 0x55663d5f8b90 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:29.674 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.671+0000 7fbe929aa740 1 --2- 172.21.15.52:0/1252670954 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55663d588340 0x55663d597250 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:29.675 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.671+0000 7fbe929aa740 1 --2- 172.21.15.52:0/1252670954 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55663d587980 0x55663d594980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:29.675 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.671+0000 7fbe929aa740 1 --2- 172.21.15.52:0/1252670954 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55663d5867b0 0x55663d594440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:29.675 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.671+0000 7fbe717fa640 0 client.25079 ms_handle_reset on v2:172.21.15.106:6836/2901755066 2024-03-04T09:30:29.675 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.671+0000 7fbe929aa740 1 -- 172.21.15.52:0/1252670954 >> 172.21.15.52:0/1252670954 conn(0x55663d581ba0 msgr2=0x55663d58e690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:30:29.675 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.671+0000 7fbe929aa740 1 -- 172.21.15.52:0/1252670954 shutdown_connections 2024-03-04T09:30:29.675 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.671+0000 7fbe929aa740 1 -- 172.21.15.52:0/1252670954 wait complete. 2024-03-04T09:30:29.675 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.671+0000 7fbe929aa740 20 client.25079 trim_cache size 0 max 16384 2024-03-04T09:30:29.675 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2024-03-04T09:30:29.676 INFO:teuthology.orchestra.run.smithi052.stdout: "cluster_fsid": "4e059f08-da08-11ee-95c6-87774f69a715", 2024-03-04T09:30:29.676 INFO:teuthology.orchestra.run.smithi052.stdout: "whoami": 2, 2024-03-04T09:30:29.676 INFO:teuthology.orchestra.run.smithi052.stdout: "id": 24473, 2024-03-04T09:30:29.676 INFO:teuthology.orchestra.run.smithi052.stdout: "want_state": "up:active", 2024-03-04T09:30:29.676 INFO:teuthology.orchestra.run.smithi052.stdout: "state": "up:active", 2024-03-04T09:30:29.676 INFO:teuthology.orchestra.run.smithi052.stdout: "fs_name": "cephfs", 2024-03-04T09:30:29.676 INFO:teuthology.orchestra.run.smithi052.stdout: "rank_uptime": 86.679810226000001, 2024-03-04T09:30:29.676 INFO:teuthology.orchestra.run.smithi052.stdout: "mdsmap_epoch": 24, 2024-03-04T09:30:29.676 INFO:teuthology.orchestra.run.smithi052.stdout: "osdmap_epoch": 91, 2024-03-04T09:30:29.676 INFO:teuthology.orchestra.run.smithi052.stdout: "osdmap_epoch_barrier": 89, 2024-03-04T09:30:29.676 INFO:teuthology.orchestra.run.smithi052.stdout: "uptime": 127.348895809 2024-03-04T09:30:29.676 INFO:teuthology.orchestra.run.smithi052.stdout:} 2024-03-04T09:30:29.677 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.671+0000 7fbe8fceb640 1 -- 172.21.15.52:0/392474011 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7fbe740a1e00 msgr2=0x7fbe740a42c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:30:29.677 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.671+0000 7fbe8fceb640 1 --2- 172.21.15.52:0/392474011 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7fbe740a1e00 0x7fbe740a42c0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x55663d587880 tx=0x7fbe80032590 comp rx=0 tx=0).stop 2024-03-04T09:30:29.677 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.671+0000 7fbe8fceb640 1 -- 172.21.15.52:0/392474011 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbe881027d0 msgr2=0x7fbe88197de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:30:29.677 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.671+0000 7fbe8fceb640 1 --2- 172.21.15.52:0/392474011 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbe881027d0 0x7fbe88197de0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fbe7800d950 tx=0x7fbe7800de10 comp rx=0 tx=0).stop 2024-03-04T09:30:29.678 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.671+0000 7fbe8fceb640 1 -- 172.21.15.52:0/392474011 shutdown_connections 2024-03-04T09:30:29.678 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.671+0000 7fbe8fceb640 1 --2- 172.21.15.52:0/392474011 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7fbe740a1e00 0x7fbe740a42c0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:29.678 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.671+0000 7fbe8fceb640 1 --2- 172.21.15.52:0/392474011 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbe88103190 0x7fbe8819ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:29.678 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.671+0000 7fbe8fceb640 1 --2- 172.21.15.52:0/392474011 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbe881027d0 0x7fbe88197de0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:29.678 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.671+0000 7fbe8fceb640 1 --2- 172.21.15.52:0/392474011 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbe881015e0 0x7fbe881978a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:29.678 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.671+0000 7fbe8fceb640 1 -- 172.21.15.52:0/392474011 >> 172.21.15.52:0/392474011 conn(0x7fbe880fcb70 msgr2=0x7fbe88105ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:30:29.678 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.671+0000 7fbe8fceb640 1 -- 172.21.15.52:0/392474011 shutdown_connections 2024-03-04T09:30:29.678 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.671+0000 7fbe8fceb640 1 -- 172.21.15.52:0/392474011 wait complete. 2024-03-04T09:30:29.680 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.i status 2024-03-04T09:30:29.787 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.790+0000 7f9fe7e6d640 1 -- 172.21.15.52:0/480518369 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9fe01015e0 msgr2=0x7f9fe0101a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:30:29.787 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.790+0000 7f9fe7e6d640 1 --2- 172.21.15.52:0/480518369 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9fe01015e0 0x7f9fe0101a00 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f9fd8009900 tx=0x7f9fd802bc30 comp rx=0 tx=0).stop 2024-03-04T09:30:29.787 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.790+0000 7f9fe7e6d640 1 -- 172.21.15.52:0/480518369 shutdown_connections 2024-03-04T09:30:29.787 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.790+0000 7f9fe7e6d640 1 --2- 172.21.15.52:0/480518369 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9fe0103190 0x7f9fe0109a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:29.787 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.790+0000 7f9fe7e6d640 1 --2- 172.21.15.52:0/480518369 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9fe01027d0 0x7f9fe0102c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:29.787 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.790+0000 7f9fe7e6d640 1 --2- 172.21.15.52:0/480518369 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9fe01015e0 0x7f9fe0101a00 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:29.787 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.790+0000 7f9fe7e6d640 1 -- 172.21.15.52:0/480518369 >> 172.21.15.52:0/480518369 conn(0x7f9fe00fcb70 msgr2=0x7f9fe00fef90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:30:29.788 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.790+0000 7f9fe7e6d640 1 -- 172.21.15.52:0/480518369 shutdown_connections 2024-03-04T09:30:29.788 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.790+0000 7f9fe7e6d640 1 -- 172.21.15.52:0/480518369 wait complete. 2024-03-04T09:30:29.790 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.790+0000 7f9fe7e6d640 1 Processor -- start 2024-03-04T09:30:29.790 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.790+0000 7f9fe7e6d640 1 -- start start 2024-03-04T09:30:29.790 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.790+0000 7f9fe7e6d640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9fe01015e0 0x7f9fe0197950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:30:29.790 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.790+0000 7f9fe7e6d640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9fe01027d0 0x7f9fe0197e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:30:29.790 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.790+0000 7f9fe7e6d640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9fe0103190 0x7f9fe019ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:30:29.790 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.790+0000 7f9fe7e6d640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fe010d2b0 con 0x7f9fe01027d0 2024-03-04T09:30:29.791 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.790+0000 7f9fe7e6d640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fe010d130 con 0x7f9fe0103190 2024-03-04T09:30:29.791 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.790+0000 7f9fe7e6d640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fe010d430 con 0x7f9fe01015e0 2024-03-04T09:30:29.791 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.791+0000 7f9fe5408640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9fe01027d0 0x7f9fe0197e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:30:29.791 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.791+0000 7f9fe5408640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9fe01027d0 0x7f9fe0197e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:52284/0 (socket says 172.21.15.52:52284) 2024-03-04T09:30:29.791 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.791+0000 7f9fe5408640 1 -- 172.21.15.52:0/586846173 learned_addr learned my addr 172.21.15.52:0/586846173 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-04T09:30:29.791 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.791+0000 7f9fe5c09640 1 --2- 172.21.15.52:0/586846173 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9fe01015e0 0x7f9fe0197950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:30:29.791 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.791+0000 7f9fe640a640 1 --2- 172.21.15.52:0/586846173 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9fe0103190 0x7f9fe019ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:30:29.791 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.791+0000 7f9fe5408640 1 -- 172.21.15.52:0/586846173 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9fe01015e0 msgr2=0x7f9fe0197950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:30:29.791 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.791+0000 7f9fe5408640 1 --2- 172.21.15.52:0/586846173 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9fe01015e0 0x7f9fe0197950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:29.791 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.791+0000 7f9fe5408640 1 -- 172.21.15.52:0/586846173 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9fe0103190 msgr2=0x7f9fe019ef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:30:29.791 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.791+0000 7f9fe5408640 1 --2- 172.21.15.52:0/586846173 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9fe0103190 0x7f9fe019ef00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:29.792 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.791+0000 7f9fe5408640 1 -- 172.21.15.52:0/586846173 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fe019f570 con 0x7f9fe01027d0 2024-03-04T09:30:29.792 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.791+0000 7f9fe5408640 1 --2- 172.21.15.52:0/586846173 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9fe01027d0 0x7f9fe0197e90 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f9fd400e9f0 tx=0x7f9fd400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:30:29.792 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.791+0000 7f9fdeffd640 1 -- 172.21.15.52:0/586846173 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fd400cd80 con 0x7f9fe01027d0 2024-03-04T09:30:29.792 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.791+0000 7f9fdeffd640 1 -- 172.21.15.52:0/586846173 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9fd4005500 con 0x7f9fe01027d0 2024-03-04T09:30:29.792 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.791+0000 7f9fe7e6d640 1 -- 172.21.15.52:0/586846173 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9fe019f860 con 0x7f9fe01027d0 2024-03-04T09:30:29.792 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.791+0000 7f9fdeffd640 1 -- 172.21.15.52:0/586846173 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fd4010690 con 0x7f9fe01027d0 2024-03-04T09:30:29.792 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.791+0000 7f9fe7e6d640 1 -- 172.21.15.52:0/586846173 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9fe019fdb0 con 0x7f9fe01027d0 2024-03-04T09:30:29.800 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.794+0000 7f9fdeffd640 1 -- 172.21.15.52:0/586846173 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) v1 ==== 136756+0+0 (secure 0 0 0) 0x7f9fd40046c0 con 0x7f9fe01027d0 2024-03-04T09:30:29.800 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.794+0000 7f9feaa4d740 1 Processor -- start 2024-03-04T09:30:29.800 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.794+0000 7f9feaa4d740 1 -- start start 2024-03-04T09:30:29.800 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.794+0000 7f9feaa4d740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c8ba93a7b0 0x55c8ba93abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:30:29.800 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.795+0000 7f9feaa4d740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c8ba93b980 0x55c8ba93be00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:30:29.800 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.795+0000 7f9feaa4d740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55c8ba93c340 0x55c8ba942bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:30:29.800 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.795+0000 7f9feaa4d740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c8ba89f6b0 con 0x55c8ba93a7b0 2024-03-04T09:30:29.800 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.795+0000 7f9feaa4d740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c8ba848a60 con 0x55c8ba93b980 2024-03-04T09:30:29.800 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.795+0000 7f9feaa4d740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c8ba89b750 con 0x55c8ba93c340 2024-03-04T09:30:29.800 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.795+0000 7f9fe640a640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c8ba93a7b0 0x55c8ba93abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:30:29.800 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.795+0000 7f9fe640a640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c8ba93a7b0 0x55c8ba93abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:52294/0 (socket says 172.21.15.52:52294) 2024-03-04T09:30:29.801 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.795+0000 7f9fe640a640 1 -- 172.21.15.52:0/2457449160 learned_addr learned my addr 172.21.15.52:0/2457449160 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-04T09:30:29.801 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.795+0000 7f9fe5c09640 1 --2- 172.21.15.52:0/2457449160 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c8ba93b980 0x55c8ba93be00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:30:29.801 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.795+0000 7f9fe5408640 1 --2- 172.21.15.52:0/2457449160 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55c8ba93c340 0x55c8ba942bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:30:29.801 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.795+0000 7f9fdeffd640 1 --2- 172.21.15.52:0/586846173 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f9fc80a1e00 0x7f9fc80a42c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:30:29.801 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.795+0000 7f9fe5c09640 1 -- 172.21.15.52:0/2457449160 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55c8ba93c340 msgr2=0x55c8ba942bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T09:30:29.801 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.795+0000 7f9fe5c09640 1 --2- 172.21.15.52:0/2457449160 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55c8ba93c340 0x55c8ba942bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:29.801 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.795+0000 7f9fe640a640 1 --2- 172.21.15.52:0/586846173 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f9fc80a1e00 0x7f9fc80a42c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:30:29.801 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.795+0000 7f9fe5c09640 1 -- 172.21.15.52:0/2457449160 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c8ba93a7b0 msgr2=0x55c8ba93abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T09:30:29.801 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.795+0000 7f9fe5c09640 1 --2- 172.21.15.52:0/2457449160 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c8ba93a7b0 0x55c8ba93abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:29.801 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.795+0000 7f9fdeffd640 1 -- 172.21.15.52:0/586846173 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9092+0+0 (secure 0 0 0) 0x7f9fd40ce500 con 0x7f9fe01027d0 2024-03-04T09:30:29.801 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.795+0000 7f9fe5c09640 1 -- 172.21.15.52:0/2457449160 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c8ba866f30 con 0x55c8ba93b980 2024-03-04T09:30:29.801 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.795+0000 7f9fe5c09640 1 --2- 172.21.15.52:0/2457449160 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c8ba93b980 0x55c8ba93be00 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f9fd8009230 tx=0x7f9fd802c670 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3d17027deeedf462 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:30:29.802 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.795+0000 7f9fc7fff640 1 -- 172.21.15.52:0/2457449160 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fd802c980 con 0x55c8ba93b980 2024-03-04T09:30:29.802 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.795+0000 7f9fe640a640 1 --2- 172.21.15.52:0/586846173 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f9fc80a1e00 0x7f9fc80a42c0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x55c8ba93b880 tx=0x7f9fd0004ab0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:30:29.802 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.796+0000 7f9fc7fff640 1 -- 172.21.15.52:0/2457449160 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9fd802cb20 con 0x55c8ba93b980 2024-03-04T09:30:29.802 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.796+0000 7f9fc7fff640 1 -- 172.21.15.52:0/2457449160 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fd8004920 con 0x55c8ba93b980 2024-03-04T09:30:29.802 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.796+0000 7f9feaa4d740 1 -- 172.21.15.52:0/2457449160 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c8ba93b980 msgr2=0x55c8ba93be00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:30:29.802 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.796+0000 7f9feaa4d740 1 --2- 172.21.15.52:0/2457449160 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c8ba93b980 0x55c8ba93be00 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f9fd8009230 tx=0x7f9fd802c670 comp rx=0 tx=0).stop 2024-03-04T09:30:29.802 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.796+0000 7f9feaa4d740 1 -- 172.21.15.52:0/2457449160 shutdown_connections 2024-03-04T09:30:29.802 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.796+0000 7f9feaa4d740 1 --2- 172.21.15.52:0/2457449160 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55c8ba93c340 0x55c8ba942bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:29.802 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.796+0000 7f9feaa4d740 1 --2- 172.21.15.52:0/2457449160 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c8ba93b980 0x55c8ba93be00 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:29.802 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.796+0000 7f9feaa4d740 1 --2- 172.21.15.52:0/2457449160 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c8ba93a7b0 0x55c8ba93abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:29.802 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.796+0000 7f9feaa4d740 1 -- 172.21.15.52:0/2457449160 >> 172.21.15.52:0/2457449160 conn(0x55c8ba935ba0 msgr2=0x55c8ba935fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:30:29.802 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.796+0000 7f9feaa4d740 1 -- 172.21.15.52:0/2457449160 shutdown_connections 2024-03-04T09:30:29.803 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.796+0000 7f9feaa4d740 1 -- 172.21.15.52:0/2457449160 wait complete. 2024-03-04T09:30:29.803 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.797+0000 7f9feaa4d740 1 -- start start 2024-03-04T09:30:29.803 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.797+0000 7f9feaa4d740 1 Processor -- start 2024-03-04T09:30:29.803 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.797+0000 7f9feaa4d740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c8ba93a7b0 0x55c8ba944c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:30:29.803 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.797+0000 7f9feaa4d740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55c8ba93b980 0x55c8ba9451d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:30:29.803 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.797+0000 7f9feaa4d740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c8ba93c340 0x55c8ba94c210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:30:29.803 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.797+0000 7f9feaa4d740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c8ba9432f0 con 0x55c8ba93a7b0 2024-03-04T09:30:29.803 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.797+0000 7f9feaa4d740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c8ba85bea0 con 0x55c8ba93c340 2024-03-04T09:30:29.803 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.797+0000 7f9feaa4d740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c8ba943470 con 0x55c8ba93b980 2024-03-04T09:30:29.803 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.797+0000 7f9feaa4d740 10 client.0 _ll_register_callbacks cb 0x55c8ba85b870 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-04T09:30:29.803 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.797+0000 7f9fe5c09640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c8ba93a7b0 0x55c8ba944c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:30:29.803 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.797+0000 7f9fe5c09640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c8ba93a7b0 0x55c8ba944c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:52304/0 (socket says 172.21.15.52:52304) 2024-03-04T09:30:29.804 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.797+0000 7f9fe5c09640 1 -- 172.21.15.52:0/3566646031 learned_addr learned my addr 172.21.15.52:0/3566646031 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-04T09:30:29.804 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.797+0000 7f9fe5408640 1 --2- 172.21.15.52:0/3566646031 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55c8ba93b980 0x55c8ba9451d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:30:29.804 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.797+0000 7f9fe640a640 1 --2- 172.21.15.52:0/3566646031 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c8ba93c340 0x55c8ba94c210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:30:29.804 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.797+0000 7f9fe5c09640 1 -- 172.21.15.52:0/3566646031 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55c8ba93b980 msgr2=0x55c8ba9451d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T09:30:29.804 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.797+0000 7f9fe5c09640 1 --2- 172.21.15.52:0/3566646031 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55c8ba93b980 0x55c8ba9451d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:29.804 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.797+0000 7f9fe5c09640 1 -- 172.21.15.52:0/3566646031 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c8ba93c340 msgr2=0x55c8ba94c210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T09:30:29.804 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.797+0000 7f9fe5c09640 1 --2- 172.21.15.52:0/3566646031 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c8ba93c340 0x55c8ba94c210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:29.804 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.798+0000 7f9fe5c09640 1 -- 172.21.15.52:0/3566646031 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c8ba85c270 con 0x55c8ba93a7b0 2024-03-04T09:30:29.804 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.798+0000 7f9fe5408640 1 --2- 172.21.15.52:0/3566646031 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55c8ba93b980 0x55c8ba9451d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-04T09:30:29.804 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.798+0000 7f9fe5c09640 1 --2- 172.21.15.52:0/3566646031 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c8ba93a7b0 0x55c8ba944c90 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f9fd802c170 tx=0x7f9fd8009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5274673fe67cf3ad server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:30:29.804 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.798+0000 7f9fc57fa640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-03-04T09:30:29.804 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.798+0000 7f9fc57fa640 1 -- 172.21.15.52:0/3566646031 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fd802c980 con 0x55c8ba93a7b0 2024-03-04T09:30:29.804 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.798+0000 7f9fc57fa640 1 -- 172.21.15.52:0/3566646031 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9092+0+0 (secure 0 0 0) 0x7f9fd8003a00 con 0x55c8ba93a7b0 2024-03-04T09:30:29.805 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.798+0000 7f9feaa4d740 1 -- 172.21.15.52:0/3566646031 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9fd802cb20 con 0x55c8ba93a7b0 2024-03-04T09:30:29.805 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.798+0000 7f9fc57fa640 1 -- 172.21.15.52:0/3566646031 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9fd805a050 con 0x55c8ba93a7b0 2024-03-04T09:30:29.805 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.798+0000 7f9feaa4d740 10 client.15243 fetch_fsmap learned FSMap version 24 2024-03-04T09:30:29.805 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.798+0000 7f9feaa4d740 1 -- 172.21.15.52:0/3566646031 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55c8ba5b42d0 con 0x55c8ba93a7b0 2024-03-04T09:30:29.805 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.798+0000 7f9fc57fa640 1 -- 172.21.15.52:0/3566646031 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9fd802cb20 con 0x55c8ba93a7b0 2024-03-04T09:30:29.805 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.798+0000 7f9fc57fa640 1 -- 172.21.15.52:0/3566646031 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f9fd803fb80 con 0x55c8ba93a7b0 2024-03-04T09:30:29.805 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.798+0000 7f9feaa4d740 10 client.15243 fetch_fsmap finished waiting for FSMap version 24 2024-03-04T09:30:29.805 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.798+0000 7f9feaa4d740 10 client.15243 resolve_mds: resolved name 'i' to daemon mds.i 2024-03-04T09:30:29.805 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.798+0000 7f9feaa4d740 20 client.15243 populate_metadata read hostname 'smithi052' 2024-03-04T09:30:29.806 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.798+0000 7f9feaa4d740 1 --2- 172.21.15.52:0/3566646031 >> [v2:172.21.15.191:6838/2194324891,v1:172.21.15.191:6839/2194324891] conn(0x55c8ba9513d0 0x55c8ba955850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:30:29.806 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.799+0000 7f9feaa4d740 4 client.15243 mds_command: new command op to 24490 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-04T09:30:29.806 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.799+0000 7f9feaa4d740 1 -- 172.21.15.52:0/3566646031 --> [v2:172.21.15.191:6838/2194324891,v1:172.21.15.191:6839/2194324891] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c8ba8acbf0 con 0x55c8ba9513d0 2024-03-04T09:30:29.806 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.799+0000 7f9fe5408640 1 --2- 172.21.15.52:0/3566646031 >> [v2:172.21.15.191:6838/2194324891,v1:172.21.15.191:6839/2194324891] conn(0x55c8ba9513d0 0x55c8ba955850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:30:29.806 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.799+0000 7f9fe5408640 1 --2- 172.21.15.52:0/3566646031 >> [v2:172.21.15.191:6838/2194324891,v1:172.21.15.191:6839/2194324891] conn(0x55c8ba9513d0 0x55c8ba955850 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=15fc77880c257d84 server_cookie=fbdb75631a85adbe in_seq=0 out_seq=0 2024-03-04T09:30:29.806 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.799+0000 7f9fc57fa640 10 client.15243 ms_handle_connect on v2:172.21.15.191:6838/2194324891 2024-03-04T09:30:29.806 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.800+0000 7f9fc57fa640 1 -- 172.21.15.52:0/3566646031 <== mds.1 v2:172.21.15.191:6838/2194324891 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55c8ba8acbf0 con 0x55c8ba9513d0 2024-03-04T09:30:29.806 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.800+0000 7f9fc57fa640 10 client.15243 handle_command_reply: tid=0 2024-03-04T09:30:29.807 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.800+0000 7f9feaa4d740 1 client.15243 shutdown 2024-03-04T09:30:29.807 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.800+0000 7f9feaa4d740 1 -- 172.21.15.52:0/3566646031 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c8ba93a7b0 msgr2=0x55c8ba944c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:30:29.807 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.800+0000 7f9feaa4d740 1 --2- 172.21.15.52:0/3566646031 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c8ba93a7b0 0x55c8ba944c90 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f9fd802c170 tx=0x7f9fd8009260 comp rx=0 tx=0).stop 2024-03-04T09:30:29.807 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.800+0000 7f9feaa4d740 1 -- 172.21.15.52:0/3566646031 shutdown_connections 2024-03-04T09:30:29.807 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.800+0000 7f9feaa4d740 1 --2- 172.21.15.52:0/3566646031 >> [v2:172.21.15.191:6838/2194324891,v1:172.21.15.191:6839/2194324891] conn(0x55c8ba9513d0 0x55c8ba955850 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:29.807 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.800+0000 7f9feaa4d740 1 --2- 172.21.15.52:0/3566646031 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c8ba93c340 0x55c8ba94c210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:29.807 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.800+0000 7f9feaa4d740 1 --2- 172.21.15.52:0/3566646031 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55c8ba93b980 0x55c8ba9451d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:29.807 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.800+0000 7f9fc57fa640 0 client.15243 ms_handle_reset on v2:172.21.15.191:6838/2194324891 2024-03-04T09:30:29.807 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.800+0000 7f9feaa4d740 1 --2- 172.21.15.52:0/3566646031 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c8ba93a7b0 0x55c8ba944c90 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:29.808 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.800+0000 7f9feaa4d740 1 -- 172.21.15.52:0/3566646031 >> 172.21.15.52:0/3566646031 conn(0x55c8ba935ba0 msgr2=0x55c8ba934da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:30:29.808 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.800+0000 7f9feaa4d740 1 -- 172.21.15.52:0/3566646031 shutdown_connections 2024-03-04T09:30:29.808 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.800+0000 7f9feaa4d740 1 -- 172.21.15.52:0/3566646031 wait complete. 2024-03-04T09:30:29.808 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.800+0000 7f9feaa4d740 20 client.15243 trim_cache size 0 max 16384 2024-03-04T09:30:29.814 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.817+0000 7f9feaa4d740 1 Processor -- start 2024-03-04T09:30:29.814 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.817+0000 7f9feaa4d740 1 -- start start 2024-03-04T09:30:29.815 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.817+0000 7f9feaa4d740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55c8ba93a7b0 0x55c8ba93f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:30:29.815 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.817+0000 7f9feaa4d740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c8ba93b980 0x55c8ba93fa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:30:29.815 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.817+0000 7f9feaa4d740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c8ba93c340 0x55c8ba94c130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:30:29.815 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.817+0000 7f9feaa4d740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c8ba950bf0 con 0x55c8ba93b980 2024-03-04T09:30:29.815 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.817+0000 7f9feaa4d740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c8ba951250 con 0x55c8ba93c340 2024-03-04T09:30:29.821 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.817+0000 7f9feaa4d740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c8ba9518b0 con 0x55c8ba93a7b0 2024-03-04T09:30:29.822 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.817+0000 7f9fe5408640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c8ba93b980 0x55c8ba93fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:30:29.822 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.817+0000 7f9fe5408640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c8ba93b980 0x55c8ba93fa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:52318/0 (socket says 172.21.15.52:52318) 2024-03-04T09:30:29.822 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.817+0000 7f9fe5408640 1 -- 172.21.15.52:0/2309214153 learned_addr learned my addr 172.21.15.52:0/2309214153 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-04T09:30:29.822 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.817+0000 7f9fe5c09640 1 --2- 172.21.15.52:0/2309214153 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55c8ba93a7b0 0x55c8ba93f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:30:29.822 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.817+0000 7f9fe640a640 1 --2- 172.21.15.52:0/2309214153 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c8ba93c340 0x55c8ba94c130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:30:29.822 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.818+0000 7f9fe5408640 1 -- 172.21.15.52:0/2309214153 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55c8ba93a7b0 msgr2=0x55c8ba93f4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T09:30:29.822 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.818+0000 7f9fe5408640 1 --2- 172.21.15.52:0/2309214153 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55c8ba93a7b0 0x55c8ba93f4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:29.822 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.818+0000 7f9fe5408640 1 -- 172.21.15.52:0/2309214153 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c8ba93c340 msgr2=0x55c8ba94c130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T09:30:29.823 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.818+0000 7f9fe5408640 1 --2- 172.21.15.52:0/2309214153 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c8ba93c340 0x55c8ba94c130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:29.823 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.818+0000 7f9fe5408640 1 -- 172.21.15.52:0/2309214153 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c8ba67e6b0 con 0x55c8ba93b980 2024-03-04T09:30:29.823 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.818+0000 7f9fe5408640 1 --2- 172.21.15.52:0/2309214153 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c8ba93b980 0x55c8ba93fa00 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f9fd40972d0 tx=0x7f9fd40d0040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b0b073095df9b90 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:30:29.823 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.818+0000 7f9fc77fe640 1 -- 172.21.15.52:0/2309214153 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fd40ce040 con 0x55c8ba93b980 2024-03-04T09:30:29.824 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.818+0000 7f9fc77fe640 1 -- 172.21.15.52:0/2309214153 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9fd409efc0 con 0x55c8ba93b980 2024-03-04T09:30:29.824 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.818+0000 7f9fc77fe640 1 -- 172.21.15.52:0/2309214153 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fd40ce1e0 con 0x55c8ba93b980 2024-03-04T09:30:29.824 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.818+0000 7f9feaa4d740 1 -- 172.21.15.52:0/2309214153 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c8ba93b980 msgr2=0x55c8ba93fa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:30:29.824 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.818+0000 7f9feaa4d740 1 --2- 172.21.15.52:0/2309214153 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c8ba93b980 0x55c8ba93fa00 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f9fd40972d0 tx=0x7f9fd40d0040 comp rx=0 tx=0).stop 2024-03-04T09:30:29.824 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.818+0000 7f9feaa4d740 1 -- 172.21.15.52:0/2309214153 shutdown_connections 2024-03-04T09:30:29.824 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.819+0000 7f9feaa4d740 1 --2- 172.21.15.52:0/2309214153 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c8ba93c340 0x55c8ba94c130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:29.824 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.819+0000 7f9feaa4d740 1 --2- 172.21.15.52:0/2309214153 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c8ba93b980 0x55c8ba93fa00 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:29.824 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.819+0000 7f9feaa4d740 1 --2- 172.21.15.52:0/2309214153 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55c8ba93a7b0 0x55c8ba93f4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:29.824 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.819+0000 7f9feaa4d740 1 -- 172.21.15.52:0/2309214153 >> 172.21.15.52:0/2309214153 conn(0x55c8ba935ba0 msgr2=0x55c8ba935f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:30:29.824 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.819+0000 7f9feaa4d740 1 -- 172.21.15.52:0/2309214153 shutdown_connections 2024-03-04T09:30:29.824 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.819+0000 7f9feaa4d740 1 -- 172.21.15.52:0/2309214153 wait complete. 2024-03-04T09:30:29.825 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.819+0000 7f9feaa4d740 1 -- start start 2024-03-04T09:30:29.825 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.819+0000 7f9feaa4d740 1 Processor -- start 2024-03-04T09:30:29.825 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.819+0000 7f9feaa4d740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c8ba93a7b0 0x55c8ba9478b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:30:29.825 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.819+0000 7f9feaa4d740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55c8ba93b980 0x55c8ba947df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:30:29.825 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.819+0000 7f9feaa4d740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c8ba93c340 0x55c8ba94ae10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:30:29.825 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.819+0000 7f9feaa4d740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c8ba952570 con 0x55c8ba93c340 2024-03-04T09:30:29.826 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.819+0000 7f9feaa4d740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c8ba951f10 con 0x55c8ba93a7b0 2024-03-04T09:30:29.826 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.819+0000 7f9feaa4d740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c8ba952bd0 con 0x55c8ba93b980 2024-03-04T09:30:29.826 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.819+0000 7f9feaa4d740 10 client.0 _ll_register_callbacks cb 0x7f9fd8009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-04T09:30:29.827 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.819+0000 7f9fe640a640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c8ba93c340 0x55c8ba94ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:30:29.827 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.819+0000 7f9fe640a640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c8ba93c340 0x55c8ba94ae10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:52322/0 (socket says 172.21.15.52:52322) 2024-03-04T09:30:29.827 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.819+0000 7f9fe640a640 1 -- 172.21.15.52:0/1054103976 learned_addr learned my addr 172.21.15.52:0/1054103976 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-04T09:30:29.827 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.819+0000 7f9fe5c09640 1 --2- 172.21.15.52:0/1054103976 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c8ba93a7b0 0x55c8ba9478b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:30:29.827 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.819+0000 7f9fe5408640 1 --2- 172.21.15.52:0/1054103976 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55c8ba93b980 0x55c8ba947df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:30:29.827 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.820+0000 7f9fe640a640 1 -- 172.21.15.52:0/1054103976 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55c8ba93b980 msgr2=0x55c8ba947df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T09:30:29.827 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.820+0000 7f9fe640a640 1 --2- 172.21.15.52:0/1054103976 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55c8ba93b980 0x55c8ba947df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:29.828 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.820+0000 7f9fe640a640 1 -- 172.21.15.52:0/1054103976 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c8ba93a7b0 msgr2=0x55c8ba9478b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T09:30:29.828 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.820+0000 7f9fe640a640 1 --2- 172.21.15.52:0/1054103976 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c8ba93a7b0 0x55c8ba9478b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:29.828 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.820+0000 7f9fe640a640 1 -- 172.21.15.52:0/1054103976 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c8ba94b400 con 0x55c8ba93c340 2024-03-04T09:30:29.828 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.820+0000 7f9fe5c09640 1 --2- 172.21.15.52:0/1054103976 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c8ba93a7b0 0x55c8ba9478b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-04T09:30:29.828 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.820+0000 7f9fe640a640 1 --2- 172.21.15.52:0/1054103976 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c8ba93c340 0x55c8ba94ae10 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f9fd00043c0 tx=0x7f9fd0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eafb8222a04f08b server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:30:29.828 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.820+0000 7f9fc57fa640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-03-04T09:30:29.828 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.820+0000 7f9fc57fa640 1 -- 172.21.15.52:0/1054103976 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fd0010ac0 con 0x55c8ba93c340 2024-03-04T09:30:29.828 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.820+0000 7f9feaa4d740 1 -- 172.21.15.52:0/1054103976 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c8ba8a6f40 con 0x55c8ba93c340 2024-03-04T09:30:29.829 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.820+0000 7f9fc57fa640 1 -- 172.21.15.52:0/1054103976 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9092+0+0 (secure 0 0 0) 0x7f9fd0010c60 con 0x55c8ba93c340 2024-03-04T09:30:29.829 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.820+0000 7f9fc57fa640 1 -- 172.21.15.52:0/1054103976 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9fd0010e60 con 0x55c8ba93c340 2024-03-04T09:30:29.829 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.820+0000 7f9feaa4d740 10 client.15249 fetch_fsmap learned FSMap version 24 2024-03-04T09:30:29.829 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.820+0000 7f9feaa4d740 1 -- 172.21.15.52:0/1054103976 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55c8ba9ac970 con 0x55c8ba93c340 2024-03-04T09:30:29.829 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.820+0000 7f9fc57fa640 1 -- 172.21.15.52:0/1054103976 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55c8ba8a6f40 con 0x55c8ba93c340 2024-03-04T09:30:29.829 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.821+0000 7f9fc57fa640 1 -- 172.21.15.52:0/1054103976 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f9fd0023660 con 0x55c8ba93c340 2024-03-04T09:30:29.829 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.821+0000 7f9feaa4d740 10 client.15249 fetch_fsmap finished waiting for FSMap version 24 2024-03-04T09:30:29.829 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.821+0000 7f9feaa4d740 10 client.15249 resolve_mds: resolved name 'i' to daemon mds.i 2024-03-04T09:30:29.829 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.821+0000 7f9feaa4d740 20 client.15249 populate_metadata read hostname 'smithi052' 2024-03-04T09:30:29.829 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.821+0000 7f9feaa4d740 1 --2- 172.21.15.52:0/1054103976 >> [v2:172.21.15.191:6838/2194324891,v1:172.21.15.191:6839/2194324891] conn(0x55c8ba9513d0 0x55c8ba9acc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:30:29.830 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.821+0000 7f9feaa4d740 4 client.15249 mds_command: new command op to 24490 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-04T09:30:29.830 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.821+0000 7f9feaa4d740 1 -- 172.21.15.52:0/1054103976 --> [v2:172.21.15.191:6838/2194324891,v1:172.21.15.191:6839/2194324891] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55c8ba8259a0 con 0x55c8ba9513d0 2024-03-04T09:30:29.830 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.821+0000 7f9fe5c09640 1 --2- 172.21.15.52:0/1054103976 >> [v2:172.21.15.191:6838/2194324891,v1:172.21.15.191:6839/2194324891] conn(0x55c8ba9513d0 0x55c8ba9acc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:30:29.830 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.821+0000 7f9fc57fa640 10 client.15249 ms_handle_connect on v2:172.21.15.191:6838/2194324891 2024-03-04T09:30:29.830 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.821+0000 7f9fe5c09640 1 --2- 172.21.15.52:0/1054103976 >> [v2:172.21.15.191:6838/2194324891,v1:172.21.15.191:6839/2194324891] conn(0x55c8ba9513d0 0x55c8ba9acc50 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a269761be9eeea3b server_cookie=fc4611e5476d71bf in_seq=0 out_seq=0 2024-03-04T09:30:29.830 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.821+0000 7f9fc57fa640 1 -- 172.21.15.52:0/1054103976 <== mds.1 v2:172.21.15.191:6838/2194324891 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+327 (crc 0 0 0) 0x55c8ba8259a0 con 0x55c8ba9513d0 2024-03-04T09:30:29.830 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.821+0000 7f9fc57fa640 10 client.15249 handle_command_reply: tid=0 2024-03-04T09:30:29.830 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.821+0000 7f9feaa4d740 1 client.15249 shutdown 2024-03-04T09:30:29.830 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.821+0000 7f9feaa4d740 1 -- 172.21.15.52:0/1054103976 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c8ba93c340 msgr2=0x55c8ba94ae10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:30:29.830 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.821+0000 7f9feaa4d740 1 --2- 172.21.15.52:0/1054103976 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c8ba93c340 0x55c8ba94ae10 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f9fd00043c0 tx=0x7f9fd0004a40 comp rx=0 tx=0).stop 2024-03-04T09:30:29.830 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.822+0000 7f9feaa4d740 1 -- 172.21.15.52:0/1054103976 shutdown_connections 2024-03-04T09:30:29.830 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.822+0000 7f9feaa4d740 1 --2- 172.21.15.52:0/1054103976 >> [v2:172.21.15.191:6838/2194324891,v1:172.21.15.191:6839/2194324891] conn(0x55c8ba9513d0 0x55c8ba9acc50 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:29.831 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.822+0000 7f9feaa4d740 1 --2- 172.21.15.52:0/1054103976 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c8ba93c340 0x55c8ba94ae10 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:29.831 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.822+0000 7f9feaa4d740 1 --2- 172.21.15.52:0/1054103976 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55c8ba93b980 0x55c8ba947df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:29.831 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.822+0000 7f9fc57fa640 0 client.15249 ms_handle_reset on v2:172.21.15.191:6838/2194324891 2024-03-04T09:30:29.831 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.822+0000 7f9feaa4d740 1 --2- 172.21.15.52:0/1054103976 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c8ba93a7b0 0x55c8ba9478b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:29.831 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.822+0000 7f9feaa4d740 1 -- 172.21.15.52:0/1054103976 >> 172.21.15.52:0/1054103976 conn(0x55c8ba935ba0 msgr2=0x55c8ba93f4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:30:29.831 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.822+0000 7f9feaa4d740 1 -- 172.21.15.52:0/1054103976 shutdown_connections 2024-03-04T09:30:29.831 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.822+0000 7f9feaa4d740 1 -- 172.21.15.52:0/1054103976 wait complete. 2024-03-04T09:30:29.831 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.822+0000 7f9feaa4d740 20 client.15249 trim_cache size 0 max 16384 2024-03-04T09:30:29.832 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2024-03-04T09:30:29.832 INFO:teuthology.orchestra.run.smithi052.stdout: "cluster_fsid": "4e059f08-da08-11ee-95c6-87774f69a715", 2024-03-04T09:30:29.832 INFO:teuthology.orchestra.run.smithi052.stdout: "whoami": 1, 2024-03-04T09:30:29.832 INFO:teuthology.orchestra.run.smithi052.stdout: "id": 24490, 2024-03-04T09:30:29.832 INFO:teuthology.orchestra.run.smithi052.stdout: "want_state": "up:active", 2024-03-04T09:30:29.832 INFO:teuthology.orchestra.run.smithi052.stdout: "state": "up:active", 2024-03-04T09:30:29.832 INFO:teuthology.orchestra.run.smithi052.stdout: "fs_name": "cephfs", 2024-03-04T09:30:29.832 INFO:teuthology.orchestra.run.smithi052.stdout: "rank_uptime": 87.833128134999995, 2024-03-04T09:30:29.832 INFO:teuthology.orchestra.run.smithi052.stdout: "mdsmap_epoch": 24, 2024-03-04T09:30:29.833 INFO:teuthology.orchestra.run.smithi052.stdout: "osdmap_epoch": 91, 2024-03-04T09:30:29.833 INFO:teuthology.orchestra.run.smithi052.stdout: "osdmap_epoch_barrier": 89, 2024-03-04T09:30:29.833 INFO:teuthology.orchestra.run.smithi052.stdout: "uptime": 124.657995231 2024-03-04T09:30:29.833 INFO:teuthology.orchestra.run.smithi052.stdout:} 2024-03-04T09:30:29.834 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.822+0000 7f9fe7e6d640 1 -- 172.21.15.52:0/586846173 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f9fc80a1e00 msgr2=0x7f9fc80a42c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:30:29.834 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.822+0000 7f9fe7e6d640 1 --2- 172.21.15.52:0/586846173 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f9fc80a1e00 0x7f9fc80a42c0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x55c8ba93b880 tx=0x7f9fd0004ab0 comp rx=0 tx=0).stop 2024-03-04T09:30:29.834 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.822+0000 7f9fe7e6d640 1 -- 172.21.15.52:0/586846173 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9fe01027d0 msgr2=0x7f9fe0197e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:30:29.834 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.822+0000 7f9fe7e6d640 1 --2- 172.21.15.52:0/586846173 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9fe01027d0 0x7f9fe0197e90 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f9fd400e9f0 tx=0x7f9fd400edf0 comp rx=0 tx=0).stop 2024-03-04T09:30:29.835 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.822+0000 7f9fe7e6d640 1 -- 172.21.15.52:0/586846173 shutdown_connections 2024-03-04T09:30:29.835 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.822+0000 7f9fe7e6d640 1 --2- 172.21.15.52:0/586846173 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f9fc80a1e00 0x7f9fc80a42c0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:29.835 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.822+0000 7f9fe7e6d640 1 --2- 172.21.15.52:0/586846173 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9fe0103190 0x7f9fe019ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:29.835 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.822+0000 7f9fe7e6d640 1 --2- 172.21.15.52:0/586846173 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9fe01027d0 0x7f9fe0197e90 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:29.835 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.822+0000 7f9fe7e6d640 1 --2- 172.21.15.52:0/586846173 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9fe01015e0 0x7f9fe0197950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:29.835 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.822+0000 7f9fe7e6d640 1 -- 172.21.15.52:0/586846173 >> 172.21.15.52:0/586846173 conn(0x7f9fe00fcb70 msgr2=0x7f9fe0105b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:30:29.835 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.822+0000 7f9fe7e6d640 1 -- 172.21.15.52:0/586846173 shutdown_connections 2024-03-04T09:30:29.835 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.822+0000 7f9fe7e6d640 1 -- 172.21.15.52:0/586846173 wait complete. 2024-03-04T09:30:29.836 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-03-04T09:30:29.942 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.944+0000 7f76acd32640 1 -- 172.21.15.52:0/4156634438 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f76a8102080 msgr2=0x7f76a8108910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:30:29.942 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.944+0000 7f76acd32640 1 --2- 172.21.15.52:0/4156634438 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f76a8102080 0x7f76a8108910 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f7698009ca0 tx=0x7f769802c0f0 comp rx=0 tx=0).stop 2024-03-04T09:30:29.942 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.944+0000 7f76acd32640 1 -- 172.21.15.52:0/4156634438 shutdown_connections 2024-03-04T09:30:29.942 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.944+0000 7f76acd32640 1 --2- 172.21.15.52:0/4156634438 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f76a8102080 0x7f76a8108910 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:29.942 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.944+0000 7f76acd32640 1 --2- 172.21.15.52:0/4156634438 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f76a81016c0 0x7f76a8101b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:29.942 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.944+0000 7f76acd32640 1 --2- 172.21.15.52:0/4156634438 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f76a81004d0 0x7f76a81008f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:29.942 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.944+0000 7f76acd32640 1 -- 172.21.15.52:0/4156634438 >> 172.21.15.52:0/4156634438 conn(0x7f76a80fba60 msgr2=0x7f76a80fde80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:30:29.942 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.944+0000 7f76acd32640 1 -- 172.21.15.52:0/4156634438 shutdown_connections 2024-03-04T09:30:29.942 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.944+0000 7f76acd32640 1 -- 172.21.15.52:0/4156634438 wait complete. 2024-03-04T09:30:29.944 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.945+0000 7f76acd32640 1 Processor -- start 2024-03-04T09:30:29.944 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.945+0000 7f76acd32640 1 -- start start 2024-03-04T09:30:29.945 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.945+0000 7f76acd32640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f76a81004d0 0x7f76a8196790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:30:29.945 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.945+0000 7f76acd32640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f76a81016c0 0x7f76a8196cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:30:29.945 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.945+0000 7f76acd32640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f76a8102080 0x7f76a819dd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:30:29.945 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.945+0000 7f76acd32640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76a810c160 con 0x7f76a81004d0 2024-03-04T09:30:29.945 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.945+0000 7f76acd32640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76a810bfe0 con 0x7f76a8102080 2024-03-04T09:30:29.945 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.945+0000 7f76acd32640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76a810c2e0 con 0x7f76a81016c0 2024-03-04T09:30:29.945 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.945+0000 7f76a659c640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f76a81004d0 0x7f76a8196790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:30:29.945 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.945+0000 7f76a659c640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f76a81004d0 0x7f76a8196790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:52340/0 (socket says 172.21.15.52:52340) 2024-03-04T09:30:29.945 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.945+0000 7f76a659c640 1 -- 172.21.15.52:0/3612659578 learned_addr learned my addr 172.21.15.52:0/3612659578 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-04T09:30:29.945 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.946+0000 7f76a5d9b640 1 --2- 172.21.15.52:0/3612659578 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f76a81016c0 0x7f76a8196cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:30:29.945 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.946+0000 7f76a6d9d640 1 --2- 172.21.15.52:0/3612659578 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f76a8102080 0x7f76a819dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:30:29.945 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.946+0000 7f76a659c640 1 -- 172.21.15.52:0/3612659578 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f76a81016c0 msgr2=0x7f76a8196cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:30:29.945 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.946+0000 7f76a659c640 1 --2- 172.21.15.52:0/3612659578 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f76a81016c0 0x7f76a8196cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:29.946 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.946+0000 7f76a659c640 1 -- 172.21.15.52:0/3612659578 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f76a8102080 msgr2=0x7f76a819dd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:30:29.946 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.946+0000 7f76a659c640 1 --2- 172.21.15.52:0/3612659578 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f76a8102080 0x7f76a819dd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:29.946 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.946+0000 7f76a659c640 1 -- 172.21.15.52:0/3612659578 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76a819e3b0 con 0x7f76a81004d0 2024-03-04T09:30:29.946 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.946+0000 7f76a659c640 1 --2- 172.21.15.52:0/3612659578 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f76a81004d0 0x7f76a8196790 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f76a000e9c0 tx=0x7f76a000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:30:29.946 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.946+0000 7f769f7fe640 1 -- 172.21.15.52:0/3612659578 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76a000cd80 con 0x7f76a81004d0 2024-03-04T09:30:29.946 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.946+0000 7f76acd32640 1 -- 172.21.15.52:0/3612659578 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76a819e6a0 con 0x7f76a81004d0 2024-03-04T09:30:29.946 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.946+0000 7f769f7fe640 1 -- 172.21.15.52:0/3612659578 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f76a00035f0 con 0x7f76a81004d0 2024-03-04T09:30:29.946 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.946+0000 7f769f7fe640 1 -- 172.21.15.52:0/3612659578 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76a0010690 con 0x7f76a81004d0 2024-03-04T09:30:29.946 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.946+0000 7f76acd32640 1 -- 172.21.15.52:0/3612659578 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76a819eba0 con 0x7f76a81004d0 2024-03-04T09:30:29.954 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.949+0000 7f769f7fe640 1 -- 172.21.15.52:0/3612659578 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) v1 ==== 136756+0+0 (secure 0 0 0) 0x7f76a00037a0 con 0x7f76a81004d0 2024-03-04T09:30:29.954 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.949+0000 7f76af9f0740 1 Processor -- start 2024-03-04T09:30:29.954 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.949+0000 7f76af9f0740 1 -- start start 2024-03-04T09:30:29.954 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.949+0000 7f76af9f0740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560952d879c0 0x560952d87de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:30:29.954 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.949+0000 7f76af9f0740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560952d88b90 0x560952d89010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:30:29.954 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.949+0000 7f76af9f0740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560952d89550 0x560952d8fde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:30:29.954 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.949+0000 7f76af9f0740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x560952cea6b0 con 0x560952d88b90 2024-03-04T09:30:29.954 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.949+0000 7f76af9f0740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x560952c93a60 con 0x560952d879c0 2024-03-04T09:30:29.954 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.949+0000 7f76af9f0740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x560952ce6750 con 0x560952d89550 2024-03-04T09:30:29.954 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.950+0000 7f76a659c640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560952d88b90 0x560952d89010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:30:29.954 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.950+0000 7f76a5d9b640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560952d89550 0x560952d8fde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:30:29.954 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.950+0000 7f76a659c640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560952d88b90 0x560952d89010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:52356/0 (socket says 172.21.15.52:52356) 2024-03-04T09:30:29.954 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.950+0000 7f76a5d9b640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560952d89550 0x560952d8fde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.52:53618/0 (socket says 172.21.15.52:53618) 2024-03-04T09:30:29.955 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.950+0000 7f76a659c640 1 -- 172.21.15.52:0/746465454 learned_addr learned my addr 172.21.15.52:0/746465454 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-04T09:30:29.955 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.950+0000 7f76a6d9d640 1 --2- 172.21.15.52:0/746465454 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560952d879c0 0x560952d87de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:30:29.955 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.950+0000 7f769f7fe640 1 --2- 172.21.15.52:0/3612659578 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f76940a1d80 0x7f76940a4240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:30:29.955 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.950+0000 7f76a6d9d640 1 -- 172.21.15.52:0/746465454 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560952d89550 msgr2=0x560952d8fde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T09:30:29.955 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.950+0000 7f76a6d9d640 1 --2- 172.21.15.52:0/746465454 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560952d89550 0x560952d8fde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:29.955 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.950+0000 7f76a6d9d640 1 -- 172.21.15.52:0/746465454 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560952d88b90 msgr2=0x560952d89010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T09:30:29.955 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.950+0000 7f76a6d9d640 1 --2- 172.21.15.52:0/746465454 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560952d88b90 0x560952d89010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:29.955 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.950+0000 7f76a6d9d640 1 -- 172.21.15.52:0/746465454 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560952cb1f30 con 0x560952d879c0 2024-03-04T09:30:29.955 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.950+0000 7f76a6d9d640 1 --2- 172.21.15.52:0/3612659578 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f76940a1d80 0x7f76940a4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:30:29.955 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.950+0000 7f769f7fe640 1 -- 172.21.15.52:0/3612659578 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9092+0+0 (secure 0 0 0) 0x7f76a00ce460 con 0x7f76a81004d0 2024-03-04T09:30:29.955 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.950+0000 7f76a6d9d640 1 --2- 172.21.15.52:0/746465454 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560952d879c0 0x560952d87de0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f769802c630 tx=0x7f7698005f70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=14ffd7a57d4e5d88 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:30:29.955 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.950+0000 7f769cff9640 1 -- 172.21.15.52:0/746465454 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7698039040 con 0x560952d879c0 2024-03-04T09:30:29.956 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.950+0000 7f769cff9640 1 -- 172.21.15.52:0/746465454 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7698003e50 con 0x560952d879c0 2024-03-04T09:30:29.956 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.950+0000 7f769cff9640 1 -- 172.21.15.52:0/746465454 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7698034440 con 0x560952d879c0 2024-03-04T09:30:29.956 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.951+0000 7f76af9f0740 1 -- 172.21.15.52:0/746465454 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560952d879c0 msgr2=0x560952d87de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:30:29.956 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.951+0000 7f76a6d9d640 1 --2- 172.21.15.52:0/3612659578 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f76940a1d80 0x7f76940a4240 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f7698048000 tx=0x7f7698045000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:30:29.956 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.951+0000 7f76af9f0740 1 --2- 172.21.15.52:0/746465454 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560952d879c0 0x560952d87de0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f769802c630 tx=0x7f7698005f70 comp rx=0 tx=0).stop 2024-03-04T09:30:29.956 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.951+0000 7f76af9f0740 1 -- 172.21.15.52:0/746465454 shutdown_connections 2024-03-04T09:30:29.956 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.951+0000 7f76af9f0740 1 --2- 172.21.15.52:0/746465454 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560952d89550 0x560952d8fde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:29.956 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.951+0000 7f76af9f0740 1 --2- 172.21.15.52:0/746465454 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560952d88b90 0x560952d89010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:29.956 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.951+0000 7f76af9f0740 1 --2- 172.21.15.52:0/746465454 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560952d879c0 0x560952d87de0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:29.956 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.951+0000 7f76af9f0740 1 -- 172.21.15.52:0/746465454 >> 172.21.15.52:0/746465454 conn(0x560952d80ba0 msgr2=0x560952d80fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:30:29.956 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.951+0000 7f76af9f0740 1 -- 172.21.15.52:0/746465454 shutdown_connections 2024-03-04T09:30:29.956 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.951+0000 7f76af9f0740 1 -- 172.21.15.52:0/746465454 wait complete. 2024-03-04T09:30:29.956 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.951+0000 7f76af9f0740 1 -- start start 2024-03-04T09:30:29.957 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.951+0000 7f76af9f0740 1 Processor -- start 2024-03-04T09:30:29.957 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.952+0000 7f76af9f0740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560952d879c0 0x560952d91f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:30:29.957 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.952+0000 7f76af9f0740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560952d88b90 0x560952d924d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:30:29.957 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.952+0000 7f76af9f0740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560952d89550 0x560952d99510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:30:29.957 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.952+0000 7f76af9f0740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x560952d90560 con 0x560952d88b90 2024-03-04T09:30:29.957 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.952+0000 7f76af9f0740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x560952ca6ea0 con 0x560952d89550 2024-03-04T09:30:29.957 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.952+0000 7f76af9f0740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x560952d906e0 con 0x560952d879c0 2024-03-04T09:30:29.957 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.952+0000 7f76af9f0740 10 client.0 _ll_register_callbacks cb 0x560952ca6870 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-04T09:30:29.957 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.952+0000 7f76a5d9b640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560952d88b90 0x560952d924d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:30:29.957 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.952+0000 7f76a5d9b640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560952d88b90 0x560952d924d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:52364/0 (socket says 172.21.15.52:52364) 2024-03-04T09:30:29.957 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.952+0000 7f76a5d9b640 1 -- 172.21.15.52:0/737660258 learned_addr learned my addr 172.21.15.52:0/737660258 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-04T09:30:29.957 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.952+0000 7f76a659c640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560952d879c0 0x560952d91f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:30:29.957 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.952+0000 7f76a6d9d640 1 --2- 172.21.15.52:0/737660258 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560952d89550 0x560952d99510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:30:29.957 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.952+0000 7f76a5d9b640 1 -- 172.21.15.52:0/737660258 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560952d879c0 msgr2=0x560952d91f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T09:30:29.958 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.952+0000 7f76a5d9b640 1 --2- 172.21.15.52:0/737660258 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560952d879c0 0x560952d91f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:29.958 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.952+0000 7f76a5d9b640 1 -- 172.21.15.52:0/737660258 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560952d89550 msgr2=0x560952d99510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T09:30:29.958 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.952+0000 7f76a5d9b640 1 --2- 172.21.15.52:0/737660258 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560952d89550 0x560952d99510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:29.958 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.952+0000 7f76a5d9b640 1 -- 172.21.15.52:0/737660258 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560952ca7270 con 0x560952d88b90 2024-03-04T09:30:29.958 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.952+0000 7f76a659c640 1 --2- 172.21.15.52:0/737660258 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560952d879c0 0x560952d91f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-04T09:30:29.958 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.952+0000 7f76a6d9d640 1 --2- 172.21.15.52:0/737660258 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560952d89550 0x560952d99510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-04T09:30:29.958 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.952+0000 7f76a5d9b640 1 --2- 172.21.15.52:0/737660258 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560952d88b90 0x560952d924d0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f769000bf80 tx=0x7f7690011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70d89b0075efcba4 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:30:29.958 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.952+0000 7f768dffb640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-03-04T09:30:29.958 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.952+0000 7f768dffb640 1 -- 172.21.15.52:0/737660258 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7690004020 con 0x560952d88b90 2024-03-04T09:30:29.958 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.952+0000 7f768dffb640 1 -- 172.21.15.52:0/737660258 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9092+0+0 (secure 0 0 0) 0x7f76900041c0 con 0x560952d88b90 2024-03-04T09:30:29.958 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.952+0000 7f76af9f0740 1 -- 172.21.15.52:0/737660258 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7698003e50 con 0x560952d88b90 2024-03-04T09:30:29.958 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.953+0000 7f768dffb640 1 -- 172.21.15.52:0/737660258 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7690024050 con 0x560952d88b90 2024-03-04T09:30:29.958 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.953+0000 7f76af9f0740 10 client.15261 fetch_fsmap learned FSMap version 24 2024-03-04T09:30:29.959 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.953+0000 7f76af9f0740 1 -- 172.21.15.52:0/737660258 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x5609529ff2d0 con 0x560952d88b90 2024-03-04T09:30:29.959 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.953+0000 7f768dffb640 1 -- 172.21.15.52:0/737660258 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7698003e50 con 0x560952d88b90 2024-03-04T09:30:29.959 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.953+0000 7f768dffb640 1 -- 172.21.15.52:0/737660258 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f7690004be0 con 0x560952d88b90 2024-03-04T09:30:29.959 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.953+0000 7f76af9f0740 10 client.15261 fetch_fsmap finished waiting for FSMap version 24 2024-03-04T09:30:29.959 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.953+0000 7f76af9f0740 10 client.15261 resolve_mds: resolved name 'b' to daemon mds.b 2024-03-04T09:30:29.959 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.953+0000 7f76af9f0740 20 client.15261 populate_metadata read hostname 'smithi052' 2024-03-04T09:30:29.959 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.953+0000 7f76af9f0740 1 --2- 172.21.15.52:0/737660258 >> [v2:172.21.15.106:6838/1995853093,v1:172.21.15.106:6839/1995853093] conn(0x560952d9eb50 0x560952d9dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:30:29.959 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.953+0000 7f76af9f0740 4 client.15261 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-04T09:30:29.959 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.953+0000 7f76af9f0740 1 -- 172.21.15.52:0/737660258 --> [v2:172.21.15.106:6838/1995853093,v1:172.21.15.106:6839/1995853093] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560952cf7bf0 con 0x560952d9eb50 2024-03-04T09:30:29.959 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.953+0000 7f76a659c640 1 --2- 172.21.15.52:0/737660258 >> [v2:172.21.15.106:6838/1995853093,v1:172.21.15.106:6839/1995853093] conn(0x560952d9eb50 0x560952d9dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:30:29.959 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.953+0000 7f768dffb640 10 client.15261 ms_handle_connect on v2:172.21.15.106:6838/1995853093 2024-03-04T09:30:29.959 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.953+0000 7f76a659c640 1 --2- 172.21.15.52:0/737660258 >> [v2:172.21.15.106:6838/1995853093,v1:172.21.15.106:6839/1995853093] conn(0x560952d9eb50 0x560952d9dbe0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2a989564648e4b8b server_cookie=c5cefe911a1d244e in_seq=0 out_seq=0 2024-03-04T09:30:29.959 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.954+0000 7f768dffb640 1 -- 172.21.15.52:0/737660258 <== mds.0 v2:172.21.15.106:6838/1995853093 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x560952cf7bf0 con 0x560952d9eb50 2024-03-04T09:30:29.959 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.954+0000 7f768dffb640 10 client.15261 handle_command_reply: tid=0 2024-03-04T09:30:29.960 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.954+0000 7f76af9f0740 1 client.15261 shutdown 2024-03-04T09:30:29.960 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.955+0000 7f76af9f0740 1 -- 172.21.15.52:0/737660258 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560952d88b90 msgr2=0x560952d924d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:30:29.960 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.955+0000 7f76af9f0740 1 --2- 172.21.15.52:0/737660258 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560952d88b90 0x560952d924d0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f769000bf80 tx=0x7f7690011810 comp rx=0 tx=0).stop 2024-03-04T09:30:29.960 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.955+0000 7f76af9f0740 1 -- 172.21.15.52:0/737660258 shutdown_connections 2024-03-04T09:30:29.960 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.955+0000 7f76af9f0740 1 --2- 172.21.15.52:0/737660258 >> [v2:172.21.15.106:6838/1995853093,v1:172.21.15.106:6839/1995853093] conn(0x560952d9eb50 0x560952d9dbe0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:29.960 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.955+0000 7f76af9f0740 1 --2- 172.21.15.52:0/737660258 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560952d89550 0x560952d99510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:29.960 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.955+0000 7f76af9f0740 1 --2- 172.21.15.52:0/737660258 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560952d88b90 0x560952d924d0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:29.960 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.955+0000 7f76af9f0740 1 --2- 172.21.15.52:0/737660258 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560952d879c0 0x560952d91f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:29.960 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.955+0000 7f768dffb640 0 client.15261 ms_handle_reset on v2:172.21.15.106:6838/1995853093 2024-03-04T09:30:29.960 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.955+0000 7f76af9f0740 1 -- 172.21.15.52:0/737660258 >> 172.21.15.52:0/737660258 conn(0x560952d80ba0 msgr2=0x560952d7fda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:30:29.960 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.955+0000 7f76af9f0740 1 -- 172.21.15.52:0/737660258 shutdown_connections 2024-03-04T09:30:29.960 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.955+0000 7f76af9f0740 1 -- 172.21.15.52:0/737660258 wait complete. 2024-03-04T09:30:29.960 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.955+0000 7f76af9f0740 20 client.15261 trim_cache size 0 max 16384 2024-03-04T09:30:29.974 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.971+0000 7f76af9f0740 1 Processor -- start 2024-03-04T09:30:29.975 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.971+0000 7f76af9f0740 1 -- start start 2024-03-04T09:30:29.975 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.971+0000 7f76af9f0740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560952d879c0 0x560952d80f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:30:29.975 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.971+0000 7f76af9f0740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560952d88b90 0x560952d994d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:30:29.975 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.971+0000 7f76af9f0740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560952d89550 0x560952d91f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:30:29.975 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.971+0000 7f76af9f0740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x560952d9e370 con 0x560952d88b90 2024-03-04T09:30:29.975 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.971+0000 7f76af9f0740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x560952d9e9d0 con 0x560952d879c0 2024-03-04T09:30:29.975 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.971+0000 7f76af9f0740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x560952d9f030 con 0x560952d89550 2024-03-04T09:30:29.975 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.972+0000 7f76a5d9b640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560952d88b90 0x560952d994d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:30:29.975 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.972+0000 7f76a5d9b640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560952d88b90 0x560952d994d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:52370/0 (socket says 172.21.15.52:52370) 2024-03-04T09:30:29.975 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.972+0000 7f76a5d9b640 1 -- 172.21.15.52:0/249351709 learned_addr learned my addr 172.21.15.52:0/249351709 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-04T09:30:29.975 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.972+0000 7f76a659c640 1 --2- 172.21.15.52:0/249351709 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560952d879c0 0x560952d80f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:30:29.975 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.972+0000 7f76a6d9d640 1 --2- 172.21.15.52:0/249351709 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560952d89550 0x560952d91f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:30:29.975 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.972+0000 7f76a5d9b640 1 -- 172.21.15.52:0/249351709 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560952d89550 msgr2=0x560952d91f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T09:30:29.976 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.972+0000 7f76a5d9b640 1 --2- 172.21.15.52:0/249351709 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560952d89550 0x560952d91f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:29.976 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.972+0000 7f76a5d9b640 1 -- 172.21.15.52:0/249351709 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560952d879c0 msgr2=0x560952d80f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T09:30:29.976 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.972+0000 7f76a5d9b640 1 --2- 172.21.15.52:0/249351709 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560952d879c0 0x560952d80f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:29.976 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.972+0000 7f76a5d9b640 1 -- 172.21.15.52:0/249351709 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560952ac96b0 con 0x560952d88b90 2024-03-04T09:30:29.976 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.972+0000 7f76a5d9b640 1 --2- 172.21.15.52:0/249351709 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560952d88b90 0x560952d994d0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f7690010700 tx=0x7f7690011d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c0115464634af43 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:30:29.976 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.972+0000 7f768ffff640 1 -- 172.21.15.52:0/249351709 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7690010ca0 con 0x560952d88b90 2024-03-04T09:30:29.976 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.972+0000 7f768ffff640 1 -- 172.21.15.52:0/249351709 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7690010e40 con 0x560952d88b90 2024-03-04T09:30:29.976 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.972+0000 7f768ffff640 1 -- 172.21.15.52:0/249351709 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7690004a40 con 0x560952d88b90 2024-03-04T09:30:29.976 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.973+0000 7f76af9f0740 1 -- 172.21.15.52:0/249351709 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560952d88b90 msgr2=0x560952d994d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:30:29.976 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.973+0000 7f76af9f0740 1 --2- 172.21.15.52:0/249351709 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560952d88b90 0x560952d994d0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f7690010700 tx=0x7f7690011d10 comp rx=0 tx=0).stop 2024-03-04T09:30:29.976 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.973+0000 7f76af9f0740 1 -- 172.21.15.52:0/249351709 shutdown_connections 2024-03-04T09:30:29.976 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.973+0000 7f76af9f0740 1 --2- 172.21.15.52:0/249351709 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560952d89550 0x560952d91f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:29.976 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.973+0000 7f76af9f0740 1 --2- 172.21.15.52:0/249351709 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560952d88b90 0x560952d994d0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:29.977 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.973+0000 7f76af9f0740 1 --2- 172.21.15.52:0/249351709 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560952d879c0 0x560952d80f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:29.977 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.973+0000 7f76af9f0740 1 -- 172.21.15.52:0/249351709 >> 172.21.15.52:0/249351709 conn(0x560952d80ba0 msgr2=0x560952d8bc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:30:29.977 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.973+0000 7f76af9f0740 1 -- 172.21.15.52:0/249351709 shutdown_connections 2024-03-04T09:30:29.977 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.973+0000 7f76af9f0740 1 -- 172.21.15.52:0/249351709 wait complete. 2024-03-04T09:30:29.977 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.973+0000 7f76af9f0740 1 -- start start 2024-03-04T09:30:29.977 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.973+0000 7f76af9f0740 1 Processor -- start 2024-03-04T09:30:29.977 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.973+0000 7f76af9f0740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560952d879c0 0x560952d93650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:30:29.977 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.973+0000 7f76af9f0740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560952d88b90 0x560952d93b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:30:29.977 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.973+0000 7f76af9f0740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560952d89550 0x560952d98630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:30:29.977 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.973+0000 7f76af9f0740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x560952d9fcf0 con 0x560952d88b90 2024-03-04T09:30:29.977 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.973+0000 7f76af9f0740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x560952d9f690 con 0x560952d89550 2024-03-04T09:30:29.977 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.973+0000 7f76af9f0740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x560952da0350 con 0x560952d879c0 2024-03-04T09:30:29.977 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.973+0000 7f76af9f0740 10 client.0 _ll_register_callbacks cb 0x7f769000ba50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-04T09:30:29.977 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.973+0000 7f76a5d9b640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560952d88b90 0x560952d93b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:30:29.978 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.973+0000 7f76a5d9b640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560952d88b90 0x560952d93b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:52382/0 (socket says 172.21.15.52:52382) 2024-03-04T09:30:29.978 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.973+0000 7f76a5d9b640 1 -- 172.21.15.52:0/1949091650 learned_addr learned my addr 172.21.15.52:0/1949091650 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-04T09:30:29.978 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.974+0000 7f76a659c640 1 --2- 172.21.15.52:0/1949091650 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560952d879c0 0x560952d93650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:30:29.978 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.974+0000 7f76a6d9d640 1 --2- 172.21.15.52:0/1949091650 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560952d89550 0x560952d98630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:30:29.978 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.974+0000 7f76a5d9b640 1 -- 172.21.15.52:0/1949091650 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560952d879c0 msgr2=0x560952d93650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T09:30:29.978 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.974+0000 7f76a5d9b640 1 --2- 172.21.15.52:0/1949091650 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560952d879c0 0x560952d93650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:29.978 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.974+0000 7f76a5d9b640 1 -- 172.21.15.52:0/1949091650 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560952d89550 msgr2=0x560952d98630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T09:30:29.978 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.974+0000 7f76a5d9b640 1 --2- 172.21.15.52:0/1949091650 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560952d89550 0x560952d98630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:29.978 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.974+0000 7f76a5d9b640 1 -- 172.21.15.52:0/1949091650 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560952d98c20 con 0x560952d88b90 2024-03-04T09:30:29.978 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.974+0000 7f76a5d9b640 1 --2- 172.21.15.52:0/1949091650 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560952d88b90 0x560952d93b90 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f7690005d00 tx=0x7f76900049f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d25162ecf12947bb server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:30:29.978 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.974+0000 7f768dffb640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-03-04T09:30:29.978 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.974+0000 7f768dffb640 1 -- 172.21.15.52:0/1949091650 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7690010920 con 0x560952d88b90 2024-03-04T09:30:29.978 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.974+0000 7f768dffb640 1 -- 172.21.15.52:0/1949091650 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9092+0+0 (secure 0 0 0) 0x7f7690004fd0 con 0x560952d88b90 2024-03-04T09:30:29.979 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.974+0000 7f76af9f0740 1 -- 172.21.15.52:0/1949091650 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7690010e40 con 0x560952d88b90 2024-03-04T09:30:29.979 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.974+0000 7f768dffb640 1 -- 172.21.15.52:0/1949091650 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7690043050 con 0x560952d88b90 2024-03-04T09:30:29.979 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.974+0000 7f76af9f0740 10 client.15267 fetch_fsmap learned FSMap version 24 2024-03-04T09:30:29.979 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.974+0000 7f76af9f0740 1 -- 172.21.15.52:0/1949091650 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x560952dfa010 con 0x560952d88b90 2024-03-04T09:30:29.979 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.974+0000 7f768dffb640 1 -- 172.21.15.52:0/1949091650 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7690010e40 con 0x560952d88b90 2024-03-04T09:30:29.979 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.974+0000 7f768dffb640 1 -- 172.21.15.52:0/1949091650 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f769001b8f0 con 0x560952d88b90 2024-03-04T09:30:29.979 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.974+0000 7f76af9f0740 10 client.15267 fetch_fsmap finished waiting for FSMap version 24 2024-03-04T09:30:29.979 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.974+0000 7f76af9f0740 10 client.15267 resolve_mds: resolved name 'b' to daemon mds.b 2024-03-04T09:30:29.979 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.974+0000 7f76af9f0740 20 client.15267 populate_metadata read hostname 'smithi052' 2024-03-04T09:30:29.979 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.975+0000 7f76af9f0740 1 --2- 172.21.15.52:0/1949091650 >> [v2:172.21.15.106:6838/1995853093,v1:172.21.15.106:6839/1995853093] conn(0x560952dfe280 0x560952dfa2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:30:29.979 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.975+0000 7f76af9f0740 4 client.15267 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-04T09:30:29.979 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.975+0000 7f76af9f0740 1 -- 172.21.15.52:0/1949091650 --> [v2:172.21.15.106:6838/1995853093,v1:172.21.15.106:6839/1995853093] -- command(tid 0: {"prefix": "status"}) v1 -- 0x560952cf1f40 con 0x560952dfe280 2024-03-04T09:30:29.979 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.975+0000 7f76a659c640 1 --2- 172.21.15.52:0/1949091650 >> [v2:172.21.15.106:6838/1995853093,v1:172.21.15.106:6839/1995853093] conn(0x560952dfe280 0x560952dfa2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:30:29.979 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.975+0000 7f76a659c640 1 --2- 172.21.15.52:0/1949091650 >> [v2:172.21.15.106:6838/1995853093,v1:172.21.15.106:6839/1995853093] conn(0x560952dfe280 0x560952dfa2a0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3637ab4128bac059 server_cookie=d233ad1eb48a0e6b in_seq=0 out_seq=0 2024-03-04T09:30:29.980 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.975+0000 7f768dffb640 10 client.15267 ms_handle_connect on v2:172.21.15.106:6838/1995853093 2024-03-04T09:30:29.980 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.975+0000 7f768dffb640 1 -- 172.21.15.52:0/1949091650 <== mds.0 v2:172.21.15.106:6838/1995853093 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+322 (crc 0 0 0) 0x560952cf1f40 con 0x560952dfe280 2024-03-04T09:30:29.980 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.975+0000 7f768dffb640 10 client.15267 handle_command_reply: tid=0 2024-03-04T09:30:29.980 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.975+0000 7f76af9f0740 1 client.15267 shutdown 2024-03-04T09:30:29.980 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.975+0000 7f76af9f0740 1 -- 172.21.15.52:0/1949091650 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560952d88b90 msgr2=0x560952d93b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:30:29.980 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.975+0000 7f76af9f0740 1 --2- 172.21.15.52:0/1949091650 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560952d88b90 0x560952d93b90 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f7690005d00 tx=0x7f76900049f0 comp rx=0 tx=0).stop 2024-03-04T09:30:29.980 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.975+0000 7f76af9f0740 1 -- 172.21.15.52:0/1949091650 shutdown_connections 2024-03-04T09:30:29.980 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.975+0000 7f76af9f0740 1 --2- 172.21.15.52:0/1949091650 >> [v2:172.21.15.106:6838/1995853093,v1:172.21.15.106:6839/1995853093] conn(0x560952dfe280 0x560952dfa2a0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:29.980 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.975+0000 7f76af9f0740 1 --2- 172.21.15.52:0/1949091650 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560952d89550 0x560952d98630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:29.980 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.975+0000 7f76af9f0740 1 --2- 172.21.15.52:0/1949091650 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560952d88b90 0x560952d93b90 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:29.980 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.975+0000 7f768dffb640 0 client.15267 ms_handle_reset on v2:172.21.15.106:6838/1995853093 2024-03-04T09:30:29.980 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.975+0000 7f76af9f0740 1 --2- 172.21.15.52:0/1949091650 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560952d879c0 0x560952d93650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:29.980 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.975+0000 7f76af9f0740 1 -- 172.21.15.52:0/1949091650 >> 172.21.15.52:0/1949091650 conn(0x560952d80ba0 msgr2=0x560952d91f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:30:29.981 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.975+0000 7f76af9f0740 1 -- 172.21.15.52:0/1949091650 shutdown_connections 2024-03-04T09:30:29.981 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.975+0000 7f76af9f0740 1 -- 172.21.15.52:0/1949091650 wait complete. 2024-03-04T09:30:29.981 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.976+0000 7f76af9f0740 20 client.15267 trim_cache size 0 max 16384 2024-03-04T09:30:29.981 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2024-03-04T09:30:29.981 INFO:teuthology.orchestra.run.smithi052.stdout: "cluster_fsid": "4e059f08-da08-11ee-95c6-87774f69a715", 2024-03-04T09:30:29.981 INFO:teuthology.orchestra.run.smithi052.stdout: "whoami": 0, 2024-03-04T09:30:29.981 INFO:teuthology.orchestra.run.smithi052.stdout: "id": 24506, 2024-03-04T09:30:29.981 INFO:teuthology.orchestra.run.smithi052.stdout: "want_state": "up:active", 2024-03-04T09:30:29.981 INFO:teuthology.orchestra.run.smithi052.stdout: "state": "up:active", 2024-03-04T09:30:29.981 INFO:teuthology.orchestra.run.smithi052.stdout: "fs_name": "cephfs", 2024-03-04T09:30:29.982 INFO:teuthology.orchestra.run.smithi052.stdout: "rank_uptime": 112.070703025, 2024-03-04T09:30:29.982 INFO:teuthology.orchestra.run.smithi052.stdout: "mdsmap_epoch": 24, 2024-03-04T09:30:29.982 INFO:teuthology.orchestra.run.smithi052.stdout: "osdmap_epoch": 91, 2024-03-04T09:30:29.982 INFO:teuthology.orchestra.run.smithi052.stdout: "osdmap_epoch_barrier": 89, 2024-03-04T09:30:29.982 INFO:teuthology.orchestra.run.smithi052.stdout: "uptime": 116.314750252 2024-03-04T09:30:29.982 INFO:teuthology.orchestra.run.smithi052.stdout:} 2024-03-04T09:30:29.983 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.976+0000 7f76acd32640 1 -- 172.21.15.52:0/3612659578 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f76940a1d80 msgr2=0x7f76940a4240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:30:29.983 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.976+0000 7f76acd32640 1 --2- 172.21.15.52:0/3612659578 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f76940a1d80 0x7f76940a4240 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f7698048000 tx=0x7f7698045000 comp rx=0 tx=0).stop 2024-03-04T09:30:29.983 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.976+0000 7f76acd32640 1 -- 172.21.15.52:0/3612659578 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f76a81004d0 msgr2=0x7f76a8196790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:30:29.983 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.976+0000 7f76acd32640 1 --2- 172.21.15.52:0/3612659578 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f76a81004d0 0x7f76a8196790 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f76a000e9c0 tx=0x7f76a000ee80 comp rx=0 tx=0).stop 2024-03-04T09:30:29.983 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.976+0000 7f76acd32640 1 -- 172.21.15.52:0/3612659578 shutdown_connections 2024-03-04T09:30:29.983 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.976+0000 7f76acd32640 1 --2- 172.21.15.52:0/3612659578 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f76940a1d80 0x7f76940a4240 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:29.983 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.976+0000 7f76acd32640 1 --2- 172.21.15.52:0/3612659578 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f76a8102080 0x7f76a819dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:29.983 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.976+0000 7f76acd32640 1 --2- 172.21.15.52:0/3612659578 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f76a81016c0 0x7f76a8196cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:29.983 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.976+0000 7f76acd32640 1 --2- 172.21.15.52:0/3612659578 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f76a81004d0 0x7f76a8196790 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:29.983 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.976+0000 7f76acd32640 1 -- 172.21.15.52:0/3612659578 >> 172.21.15.52:0/3612659578 conn(0x7f76a80fba60 msgr2=0x7f76a81049c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:30:29.983 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.976+0000 7f76acd32640 1 -- 172.21.15.52:0/3612659578 shutdown_connections 2024-03-04T09:30:29.984 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:29.976+0000 7f76acd32640 1 -- 172.21.15.52:0/3612659578 wait complete. 2024-03-04T09:30:29.985 INFO:tasks.cephadm:Setup complete, yielding 2024-03-04T09:30:29.985 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-03-04T09:30:29.996 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi052.front.sepia.ceph.com 2024-03-04T09:30:29.997 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4e059f08-da08-11ee-95c6-87774f69a715 -- bash -c 'ceph orch status' 2024-03-04T09:30:30.239 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/4e059f08-da08-11ee-95c6-87774f69a715/mon.a/config 2024-03-04T09:30:30.545 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:30:30 smithi052 ceph-mon[32473]: from='client.? 172.21.15.52:0/345658193' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-04T09:30:30.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:30:30 smithi191 ceph-mon[36816]: from='client.? 172.21.15.52:0/345658193' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-04T09:30:30.631 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:30:30 smithi106 ceph-mon[36484]: from='client.? 172.21.15.52:0/345658193' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-04T09:30:31.505 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:30:31 smithi052 ceph-mon[32473]: pgmap v346: 129 pgs: 129 active+clean; 641 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-04T09:30:31.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:30:31 smithi191 ceph-mon[36816]: pgmap v346: 129 pgs: 129 active+clean; 641 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-04T09:30:31.631 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:30:31 smithi106 ceph-mon[36484]: pgmap v346: 129 pgs: 129 active+clean; 641 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-04T09:30:32.034 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:32.036+0000 7fb74495b700 1 -- 172.21.15.52:0/106849820 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb740102b90 msgr2=0x7fb740109440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:30:32.034 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:32.036+0000 7fb74495b700 1 --2- 172.21.15.52:0/106849820 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb740102b90 0x7fb740109440 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fb73800b0d0 tx=0x7fb73800b3e0 comp rx=0 tx=0).stop 2024-03-04T09:30:32.034 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:32.036+0000 7fb74495b700 1 -- 172.21.15.52:0/106849820 shutdown_connections 2024-03-04T09:30:32.034 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:32.036+0000 7fb74495b700 1 --2- 172.21.15.52:0/106849820 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb740100ff0 0x7fb740101410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:32.034 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:32.036+0000 7fb74495b700 1 --2- 172.21.15.52:0/106849820 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb740102b90 0x7fb740109440 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:32.034 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:32.036+0000 7fb74495b700 1 --2- 172.21.15.52:0/106849820 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb7401021f0 0x7fb740102650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:32.034 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:32.036+0000 7fb74495b700 1 -- 172.21.15.52:0/106849820 >> 172.21.15.52:0/106849820 conn(0x7fb7400fc590 msgr2=0x7fb7400fe9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:30:32.034 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:32.036+0000 7fb74495b700 1 -- 172.21.15.52:0/106849820 shutdown_connections 2024-03-04T09:30:32.034 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:32.036+0000 7fb74495b700 1 -- 172.21.15.52:0/106849820 wait complete. 2024-03-04T09:30:32.036 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:32.037+0000 7fb74495b700 1 Processor -- start 2024-03-04T09:30:32.036 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:32.037+0000 7fb74495b700 1 -- start start 2024-03-04T09:30:32.036 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:32.037+0000 7fb74495b700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb740100ff0 0x7fb74006beb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:30:32.036 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:32.037+0000 7fb74495b700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb7401021f0 0x7fb74006c3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:30:32.036 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:32.037+0000 7fb74495b700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb740102b90 0x7fb740073670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:30:32.037 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:32.037+0000 7fb74495b700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb74010cbe0 con 0x7fb740102b90 2024-03-04T09:30:32.037 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:32.037+0000 7fb74495b700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb74010ca60 con 0x7fb7401021f0 2024-03-04T09:30:32.037 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:32.037+0000 7fb74495b700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb74010cd60 con 0x7fb740100ff0 2024-03-04T09:30:32.037 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:32.038+0000 7fb73ed9d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb740102b90 0x7fb740073670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:30:32.037 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:32.038+0000 7fb73dd9b700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb7401021f0 0x7fb74006c3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:30:32.037 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:32.038+0000 7fb73e59c700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb740100ff0 0x7fb74006beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:30:32.037 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:32.038+0000 7fb73dd9b700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb7401021f0 0x7fb74006c3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.52:33052/0 (socket says 172.21.15.52:33052) 2024-03-04T09:30:32.038 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:32.038+0000 7fb73ed9d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb740102b90 0x7fb740073670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:52398/0 (socket says 172.21.15.52:52398) 2024-03-04T09:30:32.038 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:32.038+0000 7fb73dd9b700 1 -- 172.21.15.52:0/2435137735 learned_addr learned my addr 172.21.15.52:0/2435137735 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-04T09:30:32.038 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:32.038+0000 7fb73e59c700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb740100ff0 0x7fb74006beb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.52:53664/0 (socket says 172.21.15.52:53664) 2024-03-04T09:30:32.038 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:32.038+0000 7fb73dd9b700 1 -- 172.21.15.52:0/2435137735 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb740100ff0 msgr2=0x7fb74006beb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:30:32.038 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:32.038+0000 7fb73dd9b700 1 --2- 172.21.15.52:0/2435137735 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb740100ff0 0x7fb74006beb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:32.038 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:32.038+0000 7fb73dd9b700 1 -- 172.21.15.52:0/2435137735 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb740102b90 msgr2=0x7fb740073670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:30:32.038 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:32.038+0000 7fb73dd9b700 1 --2- 172.21.15.52:0/2435137735 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb740102b90 0x7fb740073670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:32.038 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:32.038+0000 7fb73dd9b700 1 -- 172.21.15.52:0/2435137735 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb740073d40 con 0x7fb7401021f0 2024-03-04T09:30:32.038 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:32.038+0000 7fb73dd9b700 1 --2- 172.21.15.52:0/2435137735 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb7401021f0 0x7fb74006c3f0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fb73000e9a0 tx=0x7fb73000ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:30:32.038 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:32.038+0000 7fb72f7fe700 1 -- 172.21.15.52:0/2435137735 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb73000ca10 con 0x7fb7401021f0 2024-03-04T09:30:32.038 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:32.038+0000 7fb72f7fe700 1 -- 172.21.15.52:0/2435137735 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb73000cbb0 con 0x7fb7401021f0 2024-03-04T09:30:32.038 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:32.038+0000 7fb74495b700 1 -- 172.21.15.52:0/2435137735 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb740074030 con 0x7fb7401021f0 2024-03-04T09:30:32.038 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:32.038+0000 7fb72f7fe700 1 -- 172.21.15.52:0/2435137735 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb730018680 con 0x7fb7401021f0 2024-03-04T09:30:32.039 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:32.038+0000 7fb74495b700 1 -- 172.21.15.52:0/2435137735 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb740069ff0 con 0x7fb7401021f0 2024-03-04T09:30:32.039 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:32.039+0000 7fb74495b700 1 -- 172.21.15.52:0/2435137735 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb740068d80 con 0x7fb7401021f0 2024-03-04T09:30:32.039 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:32.040+0000 7fb72f7fe700 1 -- 172.21.15.52:0/2435137735 <== mon.2 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 18) v1 ==== 136756+0+0 (secure 0 0 0) 0x7fb730047850 con 0x7fb7401021f0 2024-03-04T09:30:32.039 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:32.041+0000 7fb72f7fe700 1 --2- 172.21.15.52:0/2435137735 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7fb7280a1e10 0x7fb7280a42d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:30:32.040 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:32.041+0000 7fb73e59c700 1 --2- 172.21.15.52:0/2435137735 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7fb7280a1e10 0x7fb7280a42d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:30:32.040 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:32.041+0000 7fb72f7fe700 1 -- 172.21.15.52:0/2435137735 <== mon.2 v2:172.21.15.106:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9092+0+0 (secure 0 0 0) 0x7fb7300ce340 con 0x7fb7401021f0 2024-03-04T09:30:32.040 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:32.041+0000 7fb73e59c700 1 --2- 172.21.15.52:0/2435137735 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7fb7280a1e10 0x7fb7280a42d0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fb740102050 tx=0x7fb734009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:30:32.042 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:32.045+0000 7fb72f7fe700 1 -- 172.21.15.52:0/2435137735 <== mon.2 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fb73009b3b0 con 0x7fb7401021f0 2024-03-04T09:30:32.262 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:32.265+0000 7fb74495b700 1 -- 172.21.15.52:0/2435137735 --> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) v1 -- 0x7fb740108140 con 0x7fb7280a1e10 2024-03-04T09:30:32.263 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:32.266+0000 7fb72f7fe700 1 -- 172.21.15.52:0/2435137735 <== mgr.14152 v2:172.21.15.52:6800/3036033923 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+43 (secure 0 0 0) 0x7fb740108140 con 0x7fb7280a1e10 2024-03-04T09:30:32.263 INFO:teuthology.orchestra.run.smithi052.stdout:Backend: cephadm 2024-03-04T09:30:32.263 INFO:teuthology.orchestra.run.smithi052.stdout:Available: Yes 2024-03-04T09:30:32.263 INFO:teuthology.orchestra.run.smithi052.stdout:Paused: No 2024-03-04T09:30:32.265 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:32.268+0000 7fb74495b700 1 -- 172.21.15.52:0/2435137735 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7fb7280a1e10 msgr2=0x7fb7280a42d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:30:32.265 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:32.268+0000 7fb74495b700 1 --2- 172.21.15.52:0/2435137735 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7fb7280a1e10 0x7fb7280a42d0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fb740102050 tx=0x7fb734009260 comp rx=0 tx=0).stop 2024-03-04T09:30:32.265 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:32.268+0000 7fb74495b700 1 -- 172.21.15.52:0/2435137735 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb7401021f0 msgr2=0x7fb74006c3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:30:32.265 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:32.268+0000 7fb74495b700 1 --2- 172.21.15.52:0/2435137735 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb7401021f0 0x7fb74006c3f0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fb73000e9a0 tx=0x7fb73000ecb0 comp rx=0 tx=0).stop 2024-03-04T09:30:32.265 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:32.268+0000 7fb74495b700 1 -- 172.21.15.52:0/2435137735 shutdown_connections 2024-03-04T09:30:32.266 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:32.268+0000 7fb74495b700 1 --2- 172.21.15.52:0/2435137735 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7fb7280a1e10 0x7fb7280a42d0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:32.266 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:32.268+0000 7fb74495b700 1 --2- 172.21.15.52:0/2435137735 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb740100ff0 0x7fb74006beb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:32.266 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:32.268+0000 7fb74495b700 1 --2- 172.21.15.52:0/2435137735 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb740102b90 0x7fb740073670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:32.266 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:32.268+0000 7fb74495b700 1 --2- 172.21.15.52:0/2435137735 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb7401021f0 0x7fb74006c3f0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:32.266 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:32.268+0000 7fb74495b700 1 -- 172.21.15.52:0/2435137735 >> 172.21.15.52:0/2435137735 conn(0x7fb7400fc590 msgr2=0x7fb7401054f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:30:32.266 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:32.268+0000 7fb74495b700 1 -- 172.21.15.52:0/2435137735 shutdown_connections 2024-03-04T09:30:32.266 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:32.268+0000 7fb74495b700 1 -- 172.21.15.52:0/2435137735 wait complete. 2024-03-04T09:30:33.053 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4e059f08-da08-11ee-95c6-87774f69a715 -- bash -c 'ceph orch ps' 2024-03-04T09:30:33.302 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/4e059f08-da08-11ee-95c6-87774f69a715/mon.a/config 2024-03-04T09:30:33.334 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:30:33 smithi052 ceph-mon[32473]: pgmap v347: 129 pgs: 129 active+clean; 641 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-04T09:30:33.335 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:30:33 smithi052 ceph-mon[32473]: from='client.25121 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T09:30:33.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:30:33 smithi191 ceph-mon[36816]: pgmap v347: 129 pgs: 129 active+clean; 641 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-04T09:30:33.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:30:33 smithi191 ceph-mon[36816]: from='client.25121 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T09:30:33.631 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:30:33 smithi106 ceph-mon[36484]: pgmap v347: 129 pgs: 129 active+clean; 641 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-04T09:30:33.631 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:30:33 smithi106 ceph-mon[36484]: from='client.25121 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T09:30:35.063 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:35.063+0000 7f215245d700 1 -- 172.21.15.52:0/2306276246 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f214c103cd0 msgr2=0x7f214c10a580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:30:35.063 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:35.063+0000 7f215245d700 1 --2- 172.21.15.52:0/2306276246 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f214c103cd0 0x7f214c10a580 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f214400b0d0 tx=0x7f214400b3e0 comp rx=0 tx=0).stop 2024-03-04T09:30:35.063 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:35.063+0000 7f215245d700 1 -- 172.21.15.52:0/2306276246 shutdown_connections 2024-03-04T09:30:35.063 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:35.063+0000 7f215245d700 1 --2- 172.21.15.52:0/2306276246 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f214c103cd0 0x7f214c10a580 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:35.063 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:35.063+0000 7f215245d700 1 --2- 172.21.15.52:0/2306276246 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f214c102130 0x7f214c102550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:35.063 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:35.063+0000 7f215245d700 1 --2- 172.21.15.52:0/2306276246 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f214c103330 0x7f214c103790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:35.063 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:35.063+0000 7f215245d700 1 -- 172.21.15.52:0/2306276246 >> 172.21.15.52:0/2306276246 conn(0x7f214c0fd710 msgr2=0x7f214c0ffb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:30:35.063 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:35.063+0000 7f215245d700 1 -- 172.21.15.52:0/2306276246 shutdown_connections 2024-03-04T09:30:35.063 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:35.063+0000 7f215245d700 1 -- 172.21.15.52:0/2306276246 wait complete. 2024-03-04T09:30:35.063 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:35.064+0000 7f215245d700 1 Processor -- start 2024-03-04T09:30:35.064 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:35.064+0000 7f215245d700 1 -- start start 2024-03-04T09:30:35.064 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:35.064+0000 7f215245d700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f214c102130 0x7f214c1982d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:30:35.064 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:35.064+0000 7f215245d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f214c103330 0x7f214c198810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:30:35.064 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:35.064+0000 7f215245d700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f214c103cd0 0x7f214c19f890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:30:35.064 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:35.064+0000 7f215245d700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f214c10dd70 con 0x7f214c103330 2024-03-04T09:30:35.064 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:35.064+0000 7f215245d700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f214c10dbf0 con 0x7f214c102130 2024-03-04T09:30:35.064 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:35.064+0000 7f215245d700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f214c10def0 con 0x7f214c103cd0 2024-03-04T09:30:35.064 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:35.065+0000 7f2143fff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f214c103330 0x7f214c198810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:30:35.064 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:35.065+0000 7f2143fff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f214c103330 0x7f214c198810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:35528/0 (socket says 172.21.15.52:35528) 2024-03-04T09:30:35.064 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:35.065+0000 7f2143fff700 1 -- 172.21.15.52:0/1423278609 learned_addr learned my addr 172.21.15.52:0/1423278609 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-04T09:30:35.064 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:35.065+0000 7f214bfff700 1 --2- 172.21.15.52:0/1423278609 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f214c102130 0x7f214c1982d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:30:35.064 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:35.065+0000 7f21509fa700 1 --2- 172.21.15.52:0/1423278609 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f214c103cd0 0x7f214c19f890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:30:35.064 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:35.065+0000 7f2143fff700 1 -- 172.21.15.52:0/1423278609 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f214c103cd0 msgr2=0x7f214c19f890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:30:35.065 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:35.065+0000 7f2143fff700 1 --2- 172.21.15.52:0/1423278609 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f214c103cd0 0x7f214c19f890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:35.065 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:35.065+0000 7f2143fff700 1 -- 172.21.15.52:0/1423278609 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f214c102130 msgr2=0x7f214c1982d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:30:35.065 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:35.065+0000 7f2143fff700 1 --2- 172.21.15.52:0/1423278609 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f214c102130 0x7f214c1982d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:35.065 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:35.065+0000 7f2143fff700 1 -- 172.21.15.52:0/1423278609 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f214c19ff30 con 0x7f214c103330 2024-03-04T09:30:35.065 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:35.065+0000 7f2143fff700 1 --2- 172.21.15.52:0/1423278609 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f214c103330 0x7f214c198810 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f213400e9f0 tx=0x7f213400ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:30:35.065 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:35.065+0000 7f2149ffb700 1 -- 172.21.15.52:0/1423278609 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f213400cbd0 con 0x7f214c103330 2024-03-04T09:30:35.065 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:35.065+0000 7f2149ffb700 1 -- 172.21.15.52:0/1423278609 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f213400ee90 con 0x7f214c103330 2024-03-04T09:30:35.065 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:35.065+0000 7f215245d700 1 -- 172.21.15.52:0/1423278609 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f214c1a0220 con 0x7f214c103330 2024-03-04T09:30:35.065 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:35.065+0000 7f2149ffb700 1 -- 172.21.15.52:0/1423278609 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21340056c0 con 0x7f214c103330 2024-03-04T09:30:35.065 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:35.065+0000 7f215245d700 1 -- 172.21.15.52:0/1423278609 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f214c1a0760 con 0x7f214c103330 2024-03-04T09:30:35.066 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:35.066+0000 7f215245d700 1 -- 172.21.15.52:0/1423278609 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f214c068d80 con 0x7f214c103330 2024-03-04T09:30:35.066 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:35.067+0000 7f2149ffb700 1 -- 172.21.15.52:0/1423278609 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) v1 ==== 136756+0+0 (secure 0 0 0) 0x7f2134004750 con 0x7f214c103330 2024-03-04T09:30:35.066 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:35.068+0000 7f2149ffb700 1 --2- 172.21.15.52:0/1423278609 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f21380a1e90 0x7f21380a4350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:30:35.066 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:35.068+0000 7f214bfff700 1 --2- 172.21.15.52:0/1423278609 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f21380a1e90 0x7f21380a4350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:30:35.066 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:35.068+0000 7f2149ffb700 1 -- 172.21.15.52:0/1423278609 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9092+0+0 (secure 0 0 0) 0x7f21340ce300 con 0x7f214c103330 2024-03-04T09:30:35.066 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:35.069+0000 7f214bfff700 1 --2- 172.21.15.52:0/1423278609 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f21380a1e90 0x7f21380a4350 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f213c009680 tx=0x7f213c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:30:35.069 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:35.073+0000 7f2149ffb700 1 -- 172.21.15.52:0/1423278609 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f213409b3d0 con 0x7f214c103330 2024-03-04T09:30:35.283 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:35.286+0000 7f215245d700 1 -- 172.21.15.52:0/1423278609 --> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f214c062500 con 0x7f21380a1e90 2024-03-04T09:30:35.297 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:35.300+0000 7f2149ffb700 1 -- 172.21.15.52:0/1423278609 <== mgr.14152 v2:172.21.15.52:6800/3036033923 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+4092 (secure 0 0 0) 0x7f214c062500 con 0x7f21380a1e90 2024-03-04T09:30:35.297 INFO:teuthology.orchestra.run.smithi052.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-04T09:30:35.297 INFO:teuthology.orchestra.run.smithi052.stdout:mds.a smithi052 running (2m) 20s ago 2m 13.9M - 19.0.0-1739-g1bc250c7 f88ffb32aa6a f1c2020db07b 2024-03-04T09:30:35.298 INFO:teuthology.orchestra.run.smithi052.stdout:mds.b smithi106 running (2m) 54s ago 2m 24.9M - 19.0.0-1739-g1bc250c7 f88ffb32aa6a a2424e9e9412 2024-03-04T09:30:35.298 INFO:teuthology.orchestra.run.smithi052.stdout:mds.c smithi191 running (2m) 54s ago 2m 18.4M - 19.0.0-1739-g1bc250c7 f88ffb32aa6a ea63df47849d 2024-03-04T09:30:35.298 INFO:teuthology.orchestra.run.smithi052.stdout:mds.d smithi052 running (2m) 20s ago 2m 12.9M - 19.0.0-1739-g1bc250c7 f88ffb32aa6a 4e33c573a820 2024-03-04T09:30:35.298 INFO:teuthology.orchestra.run.smithi052.stdout:mds.e smithi106 running (2m) 54s ago 2m 16.0M - 19.0.0-1739-g1bc250c7 f88ffb32aa6a 2bbb1b9af83d 2024-03-04T09:30:35.298 INFO:teuthology.orchestra.run.smithi052.stdout:mds.f smithi191 running (2m) 54s ago 2m 18.1M - 19.0.0-1739-g1bc250c7 f88ffb32aa6a 701812dde879 2024-03-04T09:30:35.298 INFO:teuthology.orchestra.run.smithi052.stdout:mds.g smithi052 running (2m) 20s ago 2m 13.8M - 19.0.0-1739-g1bc250c7 f88ffb32aa6a 828128f1c501 2024-03-04T09:30:35.298 INFO:teuthology.orchestra.run.smithi052.stdout:mds.h smithi106 running (2m) 54s ago 2m 18.4M - 19.0.0-1739-g1bc250c7 f88ffb32aa6a 7a759fae8efb 2024-03-04T09:30:35.298 INFO:teuthology.orchestra.run.smithi052.stdout:mds.i smithi191 running (2m) 54s ago 2m 17.0M - 19.0.0-1739-g1bc250c7 f88ffb32aa6a a07ca64ad21d 2024-03-04T09:30:35.298 INFO:teuthology.orchestra.run.smithi052.stdout:mds.j smithi052 running (2m) 20s ago 2m 16.5M - 19.0.0-1739-g1bc250c7 f88ffb32aa6a 942ac769c5f2 2024-03-04T09:30:35.298 INFO:teuthology.orchestra.run.smithi052.stdout:mds.k smithi106 running (2m) 54s ago 2m 15.3M - 19.0.0-1739-g1bc250c7 f88ffb32aa6a a3f695264ac9 2024-03-04T09:30:35.298 INFO:teuthology.orchestra.run.smithi052.stdout:mds.l smithi191 running (2m) 54s ago 2m 16.5M - 19.0.0-1739-g1bc250c7 f88ffb32aa6a 08126d739d3a 2024-03-04T09:30:35.298 INFO:teuthology.orchestra.run.smithi052.stdout:mgr.x smithi052 *:9283,8765 running (9m) 20s ago 9m 514M - 19.0.0-1739-g1bc250c7 f88ffb32aa6a 4852946a56df 2024-03-04T09:30:35.298 INFO:teuthology.orchestra.run.smithi052.stdout:mgr.y smithi106 *:8443,8765 running (7m) 54s ago 7m 451M - 19.0.0-1739-g1bc250c7 f88ffb32aa6a d19f286d79b1 2024-03-04T09:30:35.298 INFO:teuthology.orchestra.run.smithi052.stdout:mgr.z smithi191 *:8443,8765 running (7m) 54s ago 7m 452M - 19.0.0-1739-g1bc250c7 f88ffb32aa6a f5a6a77297b8 2024-03-04T09:30:35.299 INFO:teuthology.orchestra.run.smithi052.stdout:mon.a smithi052 running (9m) 20s ago 9m 51.3M 2048M 19.0.0-1739-g1bc250c7 f88ffb32aa6a b840a3470a2b 2024-03-04T09:30:35.299 INFO:teuthology.orchestra.run.smithi052.stdout:mon.b smithi106 running (8m) 54s ago 8m 42.3M 2048M 19.0.0-1739-g1bc250c7 f88ffb32aa6a d74eb04dccfb 2024-03-04T09:30:35.299 INFO:teuthology.orchestra.run.smithi052.stdout:mon.c smithi191 running (8m) 54s ago 8m 38.2M 2048M 19.0.0-1739-g1bc250c7 f88ffb32aa6a d08b1b732845 2024-03-04T09:30:35.299 INFO:teuthology.orchestra.run.smithi052.stdout:osd.0 smithi052 running (7m) 20s ago 7m 48.5M 4273M 19.0.0-1739-g1bc250c7 f88ffb32aa6a d30d901a490a 2024-03-04T09:30:35.299 INFO:teuthology.orchestra.run.smithi052.stdout:osd.1 smithi106 running (7m) 54s ago 7m 46.9M 4270M 19.0.0-1739-g1bc250c7 f88ffb32aa6a d0c65575d03e 2024-03-04T09:30:35.299 INFO:teuthology.orchestra.run.smithi052.stdout:osd.2 smithi191 running (6m) 54s ago 6m 47.6M 4270M 19.0.0-1739-g1bc250c7 f88ffb32aa6a da5629b7b744 2024-03-04T09:30:35.299 INFO:teuthology.orchestra.run.smithi052.stdout:osd.3 smithi052 running (6m) 20s ago 6m 52.5M 4273M 19.0.0-1739-g1bc250c7 f88ffb32aa6a 42b56a6f5c48 2024-03-04T09:30:35.299 INFO:teuthology.orchestra.run.smithi052.stdout:osd.4 smithi106 running (5m) 54s ago 5m 42.5M 4270M 19.0.0-1739-g1bc250c7 f88ffb32aa6a 703e793e68e4 2024-03-04T09:30:35.299 INFO:teuthology.orchestra.run.smithi052.stdout:osd.5 smithi191 running (5m) 54s ago 5m 51.6M 4270M 19.0.0-1739-g1bc250c7 f88ffb32aa6a e7711eace7bd 2024-03-04T09:30:35.299 INFO:teuthology.orchestra.run.smithi052.stdout:osd.6 smithi052 running (5m) 20s ago 5m 45.7M 4273M 19.0.0-1739-g1bc250c7 f88ffb32aa6a 95ef300435cc 2024-03-04T09:30:35.299 INFO:teuthology.orchestra.run.smithi052.stdout:osd.7 smithi106 running (4m) 54s ago 4m 44.3M 4270M 19.0.0-1739-g1bc250c7 f88ffb32aa6a 1c595e243785 2024-03-04T09:30:35.299 INFO:teuthology.orchestra.run.smithi052.stdout:osd.8 smithi191 running (4m) 54s ago 4m 43.4M 4270M 19.0.0-1739-g1bc250c7 f88ffb32aa6a 34223926f7ee 2024-03-04T09:30:35.299 INFO:teuthology.orchestra.run.smithi052.stdout:osd.9 smithi052 running (3m) 20s ago 3m 52.1M 4273M 19.0.0-1739-g1bc250c7 f88ffb32aa6a d6717d8248e5 2024-03-04T09:30:35.299 INFO:teuthology.orchestra.run.smithi052.stdout:osd.10 smithi106 running (3m) 54s ago 3m 55.4M 4270M 19.0.0-1739-g1bc250c7 f88ffb32aa6a 2cdd66ca240b 2024-03-04T09:30:35.300 INFO:teuthology.orchestra.run.smithi052.stdout:osd.11 smithi191 running (2m) 54s ago 2m 41.6M 4270M 19.0.0-1739-g1bc250c7 f88ffb32aa6a 1cb2ddf0e1ef 2024-03-04T09:30:35.300 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:35.303+0000 7f215245d700 1 -- 172.21.15.52:0/1423278609 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f21380a1e90 msgr2=0x7f21380a4350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:30:35.301 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:35.303+0000 7f215245d700 1 --2- 172.21.15.52:0/1423278609 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f21380a1e90 0x7f21380a4350 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f213c009680 tx=0x7f213c009260 comp rx=0 tx=0).stop 2024-03-04T09:30:35.301 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:35.303+0000 7f215245d700 1 -- 172.21.15.52:0/1423278609 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f214c103330 msgr2=0x7f214c198810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:30:35.301 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:35.303+0000 7f215245d700 1 --2- 172.21.15.52:0/1423278609 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f214c103330 0x7f214c198810 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f213400e9f0 tx=0x7f213400ed00 comp rx=0 tx=0).stop 2024-03-04T09:30:35.301 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:35.303+0000 7f215245d700 1 -- 172.21.15.52:0/1423278609 shutdown_connections 2024-03-04T09:30:35.301 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:35.303+0000 7f215245d700 1 --2- 172.21.15.52:0/1423278609 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f21380a1e90 0x7f21380a4350 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:35.301 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:35.303+0000 7f215245d700 1 --2- 172.21.15.52:0/1423278609 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f214c103cd0 0x7f214c19f890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:35.301 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:35.303+0000 7f215245d700 1 --2- 172.21.15.52:0/1423278609 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f214c102130 0x7f214c1982d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:35.301 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:35.303+0000 7f215245d700 1 --2- 172.21.15.52:0/1423278609 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f214c103330 0x7f214c198810 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:35.301 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:35.303+0000 7f215245d700 1 -- 172.21.15.52:0/1423278609 >> 172.21.15.52:0/1423278609 conn(0x7f214c0fd710 msgr2=0x7f214c106560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:30:35.301 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:35.303+0000 7f215245d700 1 -- 172.21.15.52:0/1423278609 shutdown_connections 2024-03-04T09:30:35.301 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:35.303+0000 7f215245d700 1 -- 172.21.15.52:0/1423278609 wait complete. 2024-03-04T09:30:35.334 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:30:35 smithi052 ceph-mon[32473]: pgmap v348: 129 pgs: 129 active+clean; 641 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-04T09:30:35.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:30:35 smithi191 ceph-mon[36816]: pgmap v348: 129 pgs: 129 active+clean; 641 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-04T09:30:35.631 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:30:35 smithi106 ceph-mon[36484]: pgmap v348: 129 pgs: 129 active+clean; 641 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-04T09:30:35.943 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4e059f08-da08-11ee-95c6-87774f69a715 -- bash -c 'ceph orch ls' 2024-03-04T09:30:36.186 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/4e059f08-da08-11ee-95c6-87774f69a715/mon.a/config 2024-03-04T09:30:36.570 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:30:36 smithi052 ceph-mon[32473]: from='client.15279 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T09:30:36.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:30:36 smithi191 ceph-mon[36816]: from='client.15279 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T09:30:36.631 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:30:36 smithi106 ceph-mon[36484]: from='client.15279 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T09:30:37.576 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:30:37 smithi052 ceph-mon[32473]: pgmap v349: 129 pgs: 129 active+clean; 641 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-04T09:30:37.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:30:37 smithi191 ceph-mon[36816]: pgmap v349: 129 pgs: 129 active+clean; 641 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-04T09:30:37.631 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:30:37 smithi106 ceph-mon[36484]: pgmap v349: 129 pgs: 129 active+clean; 641 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-04T09:30:38.153 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:38.155+0000 7f15665fb700 1 -- 172.21.15.52:0/1862019270 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1560100b50 msgr2=0x7f156010a720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:30:38.153 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:38.155+0000 7f15665fb700 1 --2- 172.21.15.52:0/1862019270 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1560100b50 0x7f156010a720 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f155800b0a0 tx=0x7f155800b3b0 comp rx=0 tx=0).stop 2024-03-04T09:30:38.154 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:38.155+0000 7f15665fb700 1 -- 172.21.15.52:0/1862019270 shutdown_connections 2024-03-04T09:30:38.154 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:38.155+0000 7f15665fb700 1 --2- 172.21.15.52:0/1862019270 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1560103730 0x7f1560103b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:38.154 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:38.155+0000 7f15665fb700 1 --2- 172.21.15.52:0/1862019270 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1560100b50 0x7f156010a720 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:38.154 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:38.155+0000 7f15665fb700 1 --2- 172.21.15.52:0/1862019270 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f15601001b0 0x7f1560100610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:38.154 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:38.155+0000 7f15665fb700 1 -- 172.21.15.52:0/1862019270 >> 172.21.15.52:0/1862019270 conn(0x7f15600fbd50 msgr2=0x7f15600fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:30:38.154 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:38.156+0000 7f15665fb700 1 -- 172.21.15.52:0/1862019270 shutdown_connections 2024-03-04T09:30:38.154 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:38.156+0000 7f15665fb700 1 -- 172.21.15.52:0/1862019270 wait complete. 2024-03-04T09:30:38.154 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:38.156+0000 7f15665fb700 1 Processor -- start 2024-03-04T09:30:38.154 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:38.156+0000 7f15665fb700 1 -- start start 2024-03-04T09:30:38.154 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:38.156+0000 7f15665fb700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f15601001b0 0x7f1560198230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:30:38.154 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:38.156+0000 7f15665fb700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1560100b50 0x7f1560198770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:30:38.154 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:38.156+0000 7f15665fb700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1560103730 0x7f156019f7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:30:38.154 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:38.156+0000 7f15665fb700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f156010dcd0 con 0x7f1560103730 2024-03-04T09:30:38.154 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:38.156+0000 7f15665fb700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f156010db50 con 0x7f15601001b0 2024-03-04T09:30:38.155 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:38.156+0000 7f15665fb700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f156010de50 con 0x7f1560100b50 2024-03-04T09:30:38.155 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:38.157+0000 7f1564b98700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1560103730 0x7f156019f7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:30:38.155 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:38.157+0000 7f155ffff700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f15601001b0 0x7f1560198230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:30:38.155 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:38.157+0000 7f1564b98700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1560103730 0x7f156019f7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:35552/0 (socket says 172.21.15.52:35552) 2024-03-04T09:30:38.155 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:38.157+0000 7f155ffff700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f15601001b0 0x7f1560198230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.52:40618/0 (socket says 172.21.15.52:40618) 2024-03-04T09:30:38.155 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:38.157+0000 7f1564b98700 1 -- 172.21.15.52:0/327433946 learned_addr learned my addr 172.21.15.52:0/327433946 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-04T09:30:38.155 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:38.157+0000 7f155f7fe700 1 --2- 172.21.15.52:0/327433946 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1560100b50 0x7f1560198770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:30:38.156 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:38.157+0000 7f1564b98700 1 -- 172.21.15.52:0/327433946 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1560100b50 msgr2=0x7f1560198770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:30:38.156 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:38.157+0000 7f1564b98700 1 --2- 172.21.15.52:0/327433946 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1560100b50 0x7f1560198770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:38.156 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:38.157+0000 7f1564b98700 1 -- 172.21.15.52:0/327433946 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f15601001b0 msgr2=0x7f1560198230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:30:38.156 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:38.157+0000 7f1564b98700 1 --2- 172.21.15.52:0/327433946 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f15601001b0 0x7f1560198230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:38.156 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:38.157+0000 7f1564b98700 1 -- 172.21.15.52:0/327433946 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f156019fe90 con 0x7f1560103730 2024-03-04T09:30:38.157 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:38.157+0000 7f155ffff700 1 --2- 172.21.15.52:0/327433946 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f15601001b0 0x7f1560198230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-04T09:30:38.157 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:38.157+0000 7f1564b98700 1 --2- 172.21.15.52:0/327433946 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1560103730 0x7f156019f7f0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f1558004d00 tx=0x7f1558004d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:30:38.157 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:38.157+0000 7f155d7fa700 1 -- 172.21.15.52:0/327433946 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1558021070 con 0x7f1560103730 2024-03-04T09:30:38.157 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:38.157+0000 7f15665fb700 1 -- 172.21.15.52:0/327433946 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15601012c0 con 0x7f1560103730 2024-03-04T09:30:38.157 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:38.157+0000 7f155d7fa700 1 -- 172.21.15.52:0/327433946 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1558007b80 con 0x7f1560103730 2024-03-04T09:30:38.157 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:38.157+0000 7f155d7fa700 1 -- 172.21.15.52:0/327433946 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15580052a0 con 0x7f1560103730 2024-03-04T09:30:38.157 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:38.157+0000 7f15665fb700 1 -- 172.21.15.52:0/327433946 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1560101800 con 0x7f1560103730 2024-03-04T09:30:38.157 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:38.158+0000 7f15665fb700 1 -- 172.21.15.52:0/327433946 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1560068d80 con 0x7f1560103730 2024-03-04T09:30:38.157 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:38.160+0000 7f155d7fa700 1 -- 172.21.15.52:0/327433946 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) v1 ==== 136756+0+0 (secure 0 0 0) 0x7f1558024070 con 0x7f1560103730 2024-03-04T09:30:38.158 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:38.160+0000 7f155d7fa700 1 --2- 172.21.15.52:0/327433946 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f15480a1e10 0x7f15480a42d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:30:38.158 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:38.160+0000 7f155ffff700 1 --2- 172.21.15.52:0/327433946 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f15480a1e10 0x7f15480a42d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:30:38.158 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:38.160+0000 7f155d7fa700 1 -- 172.21.15.52:0/327433946 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9092+0+0 (secure 0 0 0) 0x7f155800f020 con 0x7f1560103730 2024-03-04T09:30:38.158 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:38.161+0000 7f155ffff700 1 --2- 172.21.15.52:0/327433946 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f15480a1e10 0x7f15480a42d0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f156006a500 tx=0x7f1550009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:30:38.160 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:38.164+0000 7f155d7fa700 1 -- 172.21.15.52:0/327433946 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f155802b020 con 0x7f1560103730 2024-03-04T09:30:38.377 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:38.380+0000 7f15665fb700 1 -- 172.21.15.52:0/327433946 --> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f1560062500 con 0x7f15480a1e10 2024-03-04T09:30:38.388 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:38.391+0000 7f155d7fa700 1 -- 172.21.15.52:0/327433946 <== mgr.14152 v2:172.21.15.52:6800/3036033923 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+980 (secure 0 0 0) 0x7f1560062500 con 0x7f15480a1e10 2024-03-04T09:30:38.389 INFO:teuthology.orchestra.run.smithi052.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-03-04T09:30:38.389 INFO:teuthology.orchestra.run.smithi052.stdout:mds.all 12/12 58s ago 2m smithi052=a;smithi052=d;smithi052=g;smithi052=j;smithi106=b;smithi106=e;smithi106=h;smithi106=k;smithi191=c;smithi191=f;smithi191=i;smithi191=l;count:12 2024-03-04T09:30:38.389 INFO:teuthology.orchestra.run.smithi052.stdout:mgr 3/3 58s ago 7m smithi052=x;smithi106=y;smithi191=z;count:3 2024-03-04T09:30:38.389 INFO:teuthology.orchestra.run.smithi052.stdout:mon 3/3 58s ago 8m smithi052:172.21.15.52=a;smithi106:172.21.15.106=b;smithi191:172.21.15.191=c;count:3 2024-03-04T09:30:38.389 INFO:teuthology.orchestra.run.smithi052.stdout:osd 12 58s ago - 2024-03-04T09:30:38.391 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:38.394+0000 7f15665fb700 1 -- 172.21.15.52:0/327433946 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f15480a1e10 msgr2=0x7f15480a42d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:30:38.391 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:38.394+0000 7f15665fb700 1 --2- 172.21.15.52:0/327433946 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f15480a1e10 0x7f15480a42d0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f156006a500 tx=0x7f1550009260 comp rx=0 tx=0).stop 2024-03-04T09:30:38.391 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:38.394+0000 7f15665fb700 1 -- 172.21.15.52:0/327433946 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1560103730 msgr2=0x7f156019f7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:30:38.391 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:38.394+0000 7f15665fb700 1 --2- 172.21.15.52:0/327433946 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1560103730 0x7f156019f7f0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f1558004d00 tx=0x7f1558004d30 comp rx=0 tx=0).stop 2024-03-04T09:30:38.391 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:38.394+0000 7f15665fb700 1 -- 172.21.15.52:0/327433946 shutdown_connections 2024-03-04T09:30:38.391 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:38.394+0000 7f15665fb700 1 --2- 172.21.15.52:0/327433946 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f15480a1e10 0x7f15480a42d0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:38.391 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:38.394+0000 7f15665fb700 1 --2- 172.21.15.52:0/327433946 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1560103730 0x7f156019f7f0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:38.391 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:38.394+0000 7f15665fb700 1 --2- 172.21.15.52:0/327433946 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f15601001b0 0x7f1560198230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:38.391 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:38.394+0000 7f15665fb700 1 --2- 172.21.15.52:0/327433946 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1560100b50 0x7f1560198770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:38.391 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:38.394+0000 7f15665fb700 1 -- 172.21.15.52:0/327433946 >> 172.21.15.52:0/327433946 conn(0x7f15600fbd50 msgr2=0x7f1560106700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:30:38.392 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:38.394+0000 7f15665fb700 1 -- 172.21.15.52:0/327433946 shutdown_connections 2024-03-04T09:30:38.392 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:38.394+0000 7f15665fb700 1 -- 172.21.15.52:0/327433946 wait complete. 2024-03-04T09:30:39.046 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4e059f08-da08-11ee-95c6-87774f69a715 -- bash -c 'ceph orch host ls' 2024-03-04T09:30:39.307 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/4e059f08-da08-11ee-95c6-87774f69a715/mon.a/config 2024-03-04T09:30:39.335 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:30:39 smithi052 ceph-mon[32473]: pgmap v350: 129 pgs: 129 active+clean; 641 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-03-04T09:30:39.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:30:39 smithi191 ceph-mon[36816]: pgmap v350: 129 pgs: 129 active+clean; 641 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-03-04T09:30:39.631 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:30:39 smithi106 ceph-mon[36484]: pgmap v350: 129 pgs: 129 active+clean; 641 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-03-04T09:30:40.585 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:30:40 smithi052 ceph-mon[32473]: from='client.15285 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T09:30:40.585 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:30:40 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [8, 11]}]: dispatch 2024-03-04T09:30:40.585 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:30:40 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [2, 1, 8, 11]}]: dispatch 2024-03-04T09:30:40.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:30:40 smithi191 ceph-mon[36816]: from='client.15285 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T09:30:40.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:30:40 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [8, 11]}]: dispatch 2024-03-04T09:30:40.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:30:40 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [2, 1, 8, 11]}]: dispatch 2024-03-04T09:30:40.631 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:30:40 smithi106 ceph-mon[36484]: from='client.15285 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T09:30:40.631 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:30:40 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [8, 11]}]: dispatch 2024-03-04T09:30:40.631 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:30:40 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [2, 1, 8, 11]}]: dispatch 2024-03-04T09:30:41.141 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:41.143+0000 7f623883a700 1 -- 172.21.15.52:0/1512803091 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f62341019b0 msgr2=0x7f6234101e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:30:41.141 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:41.143+0000 7f623883a700 1 --2- 172.21.15.52:0/1512803091 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f62341019b0 0x7f6234101e10 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f62280099c0 tx=0x7f6228009cd0 comp rx=0 tx=0).stop 2024-03-04T09:30:41.141 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:41.144+0000 7f623883a700 1 -- 172.21.15.52:0/1512803091 shutdown_connections 2024-03-04T09:30:41.141 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:41.144+0000 7f623883a700 1 --2- 172.21.15.52:0/1512803091 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6234102350 0x7f6234108c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:41.141 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:41.144+0000 7f623883a700 1 --2- 172.21.15.52:0/1512803091 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f62341007b0 0x7f6234100bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:41.141 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:41.144+0000 7f623883a700 1 --2- 172.21.15.52:0/1512803091 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f62341019b0 0x7f6234101e10 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:41.141 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:41.144+0000 7f623883a700 1 -- 172.21.15.52:0/1512803091 >> 172.21.15.52:0/1512803091 conn(0x7f62340fbd50 msgr2=0x7f62340fe190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:30:41.141 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:41.144+0000 7f623883a700 1 -- 172.21.15.52:0/1512803091 shutdown_connections 2024-03-04T09:30:41.141 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:41.144+0000 7f623883a700 1 -- 172.21.15.52:0/1512803091 wait complete. 2024-03-04T09:30:41.143 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:41.145+0000 7f623883a700 1 Processor -- start 2024-03-04T09:30:41.143 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:41.145+0000 7f623883a700 1 -- start start 2024-03-04T09:30:41.143 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:41.145+0000 7f623883a700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f62341007b0 0x7f6234198340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:30:41.143 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:41.145+0000 7f623883a700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f62341019b0 0x7f6234198880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:30:41.143 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:41.145+0000 7f623883a700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6234102350 0x7f623419f8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:30:41.143 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:41.145+0000 7f623883a700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f623410cc20 con 0x7f6234102350 2024-03-04T09:30:41.143 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:41.145+0000 7f623883a700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f623410caa0 con 0x7f62341007b0 2024-03-04T09:30:41.144 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:41.145+0000 7f623883a700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f623410cda0 con 0x7f62341019b0 2024-03-04T09:30:41.144 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:41.145+0000 7f6232d9d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6234102350 0x7f623419f8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:30:41.144 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:41.145+0000 7f6232d9d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6234102350 0x7f623419f8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:35560/0 (socket says 172.21.15.52:35560) 2024-03-04T09:30:41.144 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:41.145+0000 7f6232d9d700 1 -- 172.21.15.52:0/1558818031 learned_addr learned my addr 172.21.15.52:0/1558818031 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-04T09:30:41.144 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:41.145+0000 7f6232d9d700 1 -- 172.21.15.52:0/1558818031 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f62341019b0 msgr2=0x7f6234198880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:30:41.144 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:41.145+0000 7f6232d9d700 1 --2- 172.21.15.52:0/1558818031 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f62341019b0 0x7f6234198880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:41.144 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:41.145+0000 7f6232d9d700 1 -- 172.21.15.52:0/1558818031 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f62341007b0 msgr2=0x7f6234198340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:30:41.144 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:41.145+0000 7f6232d9d700 1 --2- 172.21.15.52:0/1558818031 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f62341007b0 0x7f6234198340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:41.144 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:41.145+0000 7f6232d9d700 1 -- 172.21.15.52:0/1558818031 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6234102d90 con 0x7f6234102350 2024-03-04T09:30:41.144 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:41.146+0000 7f6232d9d700 1 --2- 172.21.15.52:0/1558818031 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6234102350 0x7f623419f8b0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f622c00ef30 tx=0x7f622c00c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:30:41.145 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:41.146+0000 7f62237fe700 1 -- 172.21.15.52:0/1558818031 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f622c0092e0 con 0x7f6234102350 2024-03-04T09:30:41.145 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:41.146+0000 7f62237fe700 1 -- 172.21.15.52:0/1558818031 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f622c004bb0 con 0x7f6234102350 2024-03-04T09:30:41.145 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:41.146+0000 7f62237fe700 1 -- 172.21.15.52:0/1558818031 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f622c004280 con 0x7f6234102350 2024-03-04T09:30:41.145 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:41.146+0000 7f623883a700 1 -- 172.21.15.52:0/1558818031 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6234103020 con 0x7f6234102350 2024-03-04T09:30:41.145 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:41.146+0000 7f623883a700 1 -- 172.21.15.52:0/1558818031 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6234103510 con 0x7f6234102350 2024-03-04T09:30:41.145 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:41.147+0000 7f623883a700 1 -- 172.21.15.52:0/1558818031 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6234068d80 con 0x7f6234102350 2024-03-04T09:30:41.145 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:41.148+0000 7f62237fe700 1 -- 172.21.15.52:0/1558818031 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) v1 ==== 136756+0+0 (secure 0 0 0) 0x7f622c01b030 con 0x7f6234102350 2024-03-04T09:30:41.146 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:41.149+0000 7f62237fe700 1 --2- 172.21.15.52:0/1558818031 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f621c0a1e90 0x7f621c0a4350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:30:41.146 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:41.149+0000 7f623259c700 1 --2- 172.21.15.52:0/1558818031 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f621c0a1e90 0x7f621c0a4350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:30:41.147 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:41.149+0000 7f62237fe700 1 -- 172.21.15.52:0/1558818031 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9158+0+0 (secure 0 0 0) 0x7f622c0cdc00 con 0x7f6234102350 2024-03-04T09:30:41.147 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:41.150+0000 7f623259c700 1 --2- 172.21.15.52:0/1558818031 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f621c0a1e90 0x7f621c0a4350 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f6224005fd0 tx=0x7f6224005d00 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:30:41.150 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:41.154+0000 7f62237fe700 1 -- 172.21.15.52:0/1558818031 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f622c0d4080 con 0x7f6234102350 2024-03-04T09:30:41.366 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:41.369+0000 7f623883a700 1 -- 172.21.15.52:0/1558818031 --> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f62340fde80 con 0x7f621c0a1e90 2024-03-04T09:30:41.367 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:41.371+0000 7f62237fe700 1 -- 172.21.15.52:0/1558818031 <== mgr.14152 v2:172.21.15.52:6800/3036033923 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+191 (secure 0 0 0) 0x7f62340fde80 con 0x7f621c0a1e90 2024-03-04T09:30:41.368 INFO:teuthology.orchestra.run.smithi052.stdout:HOST ADDR LABELS STATUS 2024-03-04T09:30:41.368 INFO:teuthology.orchestra.run.smithi052.stdout:smithi052 172.21.15.52 2024-03-04T09:30:41.368 INFO:teuthology.orchestra.run.smithi052.stdout:smithi106 172.21.15.106 2024-03-04T09:30:41.368 INFO:teuthology.orchestra.run.smithi052.stdout:smithi191 172.21.15.191 2024-03-04T09:30:41.368 INFO:teuthology.orchestra.run.smithi052.stdout:3 hosts in cluster 2024-03-04T09:30:41.370 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:41.373+0000 7f623883a700 1 -- 172.21.15.52:0/1558818031 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f621c0a1e90 msgr2=0x7f621c0a4350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:30:41.370 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:41.373+0000 7f623883a700 1 --2- 172.21.15.52:0/1558818031 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f621c0a1e90 0x7f621c0a4350 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f6224005fd0 tx=0x7f6224005d00 comp rx=0 tx=0).stop 2024-03-04T09:30:41.370 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:41.373+0000 7f623883a700 1 -- 172.21.15.52:0/1558818031 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6234102350 msgr2=0x7f623419f8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:30:41.370 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:41.373+0000 7f623883a700 1 --2- 172.21.15.52:0/1558818031 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6234102350 0x7f623419f8b0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f622c00ef30 tx=0x7f622c00c450 comp rx=0 tx=0).stop 2024-03-04T09:30:41.370 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:41.373+0000 7f623883a700 1 -- 172.21.15.52:0/1558818031 shutdown_connections 2024-03-04T09:30:41.370 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:41.373+0000 7f623883a700 1 --2- 172.21.15.52:0/1558818031 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f621c0a1e90 0x7f621c0a4350 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:41.370 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:41.373+0000 7f623883a700 1 --2- 172.21.15.52:0/1558818031 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6234102350 0x7f623419f8b0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:41.370 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:41.373+0000 7f623883a700 1 --2- 172.21.15.52:0/1558818031 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f62341007b0 0x7f6234198340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:41.371 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:41.373+0000 7f623883a700 1 --2- 172.21.15.52:0/1558818031 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f62341019b0 0x7f6234198880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:41.371 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:41.373+0000 7f623883a700 1 -- 172.21.15.52:0/1558818031 >> 172.21.15.52:0/1558818031 conn(0x7f62340fbd50 msgr2=0x7f62340fd770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:30:41.371 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:41.373+0000 7f623883a700 1 -- 172.21.15.52:0/1558818031 shutdown_connections 2024-03-04T09:30:41.371 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:41.373+0000 7f623883a700 1 -- 172.21.15.52:0/1558818031 wait complete. 2024-03-04T09:30:41.584 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:30:41 smithi052 ceph-mon[32473]: pgmap v351: 129 pgs: 129 active+clean; 641 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-04T09:30:41.585 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:30:41 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [8, 11]}]': finished 2024-03-04T09:30:41.585 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:30:41 smithi052 ceph-mon[32473]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [2, 1, 8, 11]}]': finished 2024-03-04T09:30:41.585 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:30:41 smithi052 ceph-mon[32473]: osdmap e92: 12 total, 12 up, 12 in 2024-03-04T09:30:41.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:30:41 smithi191 ceph-mon[36816]: pgmap v351: 129 pgs: 129 active+clean; 641 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-04T09:30:41.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:30:41 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [8, 11]}]': finished 2024-03-04T09:30:41.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:30:41 smithi191 ceph-mon[36816]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [2, 1, 8, 11]}]': finished 2024-03-04T09:30:41.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:30:41 smithi191 ceph-mon[36816]: osdmap e92: 12 total, 12 up, 12 in 2024-03-04T09:30:41.631 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:30:41 smithi106 ceph-mon[36484]: pgmap v351: 129 pgs: 129 active+clean; 641 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-04T09:30:41.631 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:30:41 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [8, 11]}]': finished 2024-03-04T09:30:41.631 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:30:41 smithi106 ceph-mon[36484]: from='mgr.14152 172.21.15.52:0/2817070231' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [2, 1, 8, 11]}]': finished 2024-03-04T09:30:41.631 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:30:41 smithi106 ceph-mon[36484]: osdmap e92: 12 total, 12 up, 12 in 2024-03-04T09:30:42.039 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4e059f08-da08-11ee-95c6-87774f69a715 -- bash -c 'ceph orch device ls' 2024-03-04T09:30:42.286 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/4e059f08-da08-11ee-95c6-87774f69a715/mon.a/config 2024-03-04T09:30:42.335 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:30:42 smithi052 ceph-mon[32473]: osdmap e93: 12 total, 12 up, 12 in 2024-03-04T09:30:42.335 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:30:42 smithi052 ceph-mon[32473]: from='client.15291 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T09:30:42.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:30:42 smithi191 ceph-mon[36816]: osdmap e93: 12 total, 12 up, 12 in 2024-03-04T09:30:42.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:30:42 smithi191 ceph-mon[36816]: from='client.15291 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T09:30:42.631 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:30:42 smithi106 ceph-mon[36484]: osdmap e93: 12 total, 12 up, 12 in 2024-03-04T09:30:42.631 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:30:42 smithi106 ceph-mon[36484]: from='client.15291 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T09:30:43.585 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:30:43 smithi052 ceph-mon[32473]: pgmap v354: 129 pgs: 2 peering, 127 active+clean; 641 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 7 op/s 2024-03-04T09:30:43.585 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:30:43 smithi052 ceph-mon[32473]: Health check failed: Reduced data availability: 2 pgs peering (PG_AVAILABILITY) 2024-03-04T09:30:43.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:30:43 smithi191 ceph-mon[36816]: pgmap v354: 129 pgs: 2 peering, 127 active+clean; 641 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 7 op/s 2024-03-04T09:30:43.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:30:43 smithi191 ceph-mon[36816]: Health check failed: Reduced data availability: 2 pgs peering (PG_AVAILABILITY) 2024-03-04T09:30:43.631 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:30:43 smithi106 ceph-mon[36484]: pgmap v354: 129 pgs: 2 peering, 127 active+clean; 641 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 7 op/s 2024-03-04T09:30:43.631 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:30:43 smithi106 ceph-mon[36484]: Health check failed: Reduced data availability: 2 pgs peering (PG_AVAILABILITY) 2024-03-04T09:30:43.986 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:43.979+0000 7f3056cc6700 1 -- 172.21.15.52:0/3869990516 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f30501021f0 msgr2=0x7f3050102650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:30:43.986 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:43.979+0000 7f3056cc6700 1 --2- 172.21.15.52:0/3869990516 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f30501021f0 0x7f3050102650 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f303c0099c0 tx=0x7f303c009cd0 comp rx=0 tx=0).stop 2024-03-04T09:30:43.986 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:43.979+0000 7f3056cc6700 1 -- 172.21.15.52:0/3869990516 shutdown_connections 2024-03-04T09:30:43.986 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:43.979+0000 7f3056cc6700 1 --2- 172.21.15.52:0/3869990516 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3050102b90 0x7f3050109440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:43.986 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:43.979+0000 7f3056cc6700 1 --2- 172.21.15.52:0/3869990516 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3050100ff0 0x7f3050101410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:43.986 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:43.979+0000 7f3056cc6700 1 --2- 172.21.15.52:0/3869990516 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f30501021f0 0x7f3050102650 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:43.986 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:43.979+0000 7f3056cc6700 1 -- 172.21.15.52:0/3869990516 >> 172.21.15.52:0/3869990516 conn(0x7f30500fc590 msgr2=0x7f30500fe9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:30:43.986 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:43.979+0000 7f3056cc6700 1 -- 172.21.15.52:0/3869990516 shutdown_connections 2024-03-04T09:30:43.986 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:43.979+0000 7f3056cc6700 1 -- 172.21.15.52:0/3869990516 wait complete. 2024-03-04T09:30:43.987 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:43.979+0000 7f3056cc6700 1 Processor -- start 2024-03-04T09:30:43.987 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:43.979+0000 7f3056cc6700 1 -- start start 2024-03-04T09:30:43.987 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:43.979+0000 7f3056cc6700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3050100ff0 0x7f305006b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:30:43.987 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:43.980+0000 7f3056cc6700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f30501021f0 0x7f305019f460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:30:43.987 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:43.980+0000 7f3056cc6700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3050102b90 0x7f30501a19b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:30:43.987 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:43.980+0000 7f3056cc6700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f305010cbb0 con 0x7f3050100ff0 2024-03-04T09:30:43.987 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:43.980+0000 7f3056cc6700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f305010ca30 con 0x7f3050102b90 2024-03-04T09:30:43.987 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:43.980+0000 7f3056cc6700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f305010cd30 con 0x7f30501021f0 2024-03-04T09:30:43.987 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:43.980+0000 7f3054a62700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3050100ff0 0x7f305006b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:30:43.987 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:43.980+0000 7f3054a62700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3050100ff0 0x7f305006b2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:35574/0 (socket says 172.21.15.52:35574) 2024-03-04T09:30:43.987 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:43.980+0000 7f3054a62700 1 -- 172.21.15.52:0/1003061547 learned_addr learned my addr 172.21.15.52:0/1003061547 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-04T09:30:43.987 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:43.980+0000 7f304ffff700 1 --2- 172.21.15.52:0/1003061547 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f30501021f0 0x7f305019f460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:30:43.987 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:43.980+0000 7f3055263700 1 --2- 172.21.15.52:0/1003061547 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3050102b90 0x7f30501a19b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:30:43.988 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:43.980+0000 7f3054a62700 1 -- 172.21.15.52:0/1003061547 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f30501021f0 msgr2=0x7f305019f460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:30:43.988 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:43.980+0000 7f3054a62700 1 --2- 172.21.15.52:0/1003061547 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f30501021f0 0x7f305019f460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:43.988 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:43.980+0000 7f3054a62700 1 -- 172.21.15.52:0/1003061547 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3050102b90 msgr2=0x7f30501a19b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:30:43.988 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:43.980+0000 7f3054a62700 1 --2- 172.21.15.52:0/1003061547 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3050102b90 0x7f30501a19b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:43.988 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:43.980+0000 7f3054a62700 1 -- 172.21.15.52:0/1003061547 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30501a2190 con 0x7f3050100ff0 2024-03-04T09:30:43.988 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:43.980+0000 7f3054a62700 1 --2- 172.21.15.52:0/1003061547 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3050100ff0 0x7f305006b2c0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f304400e970 tx=0x7f304400ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:30:43.992 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:43.980+0000 7f304dffb700 1 -- 172.21.15.52:0/1003061547 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3044004220 con 0x7f3050100ff0 2024-03-04T09:30:43.992 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:43.980+0000 7f304dffb700 1 -- 172.21.15.52:0/1003061547 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f30440043c0 con 0x7f3050100ff0 2024-03-04T09:30:43.992 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:43.980+0000 7f3056cc6700 1 -- 172.21.15.52:0/1003061547 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30501a2480 con 0x7f3050100ff0 2024-03-04T09:30:43.992 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:43.980+0000 7f304dffb700 1 -- 172.21.15.52:0/1003061547 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3044003dc0 con 0x7f3050100ff0 2024-03-04T09:30:43.992 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:43.981+0000 7f3056cc6700 1 -- 172.21.15.52:0/1003061547 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30501a2990 con 0x7f3050100ff0 2024-03-04T09:30:43.992 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:43.982+0000 7f3056cc6700 1 -- 172.21.15.52:0/1003061547 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3050068d80 con 0x7f3050100ff0 2024-03-04T09:30:43.992 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:43.983+0000 7f304dffb700 1 -- 172.21.15.52:0/1003061547 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) v1 ==== 136756+0+0 (secure 0 0 0) 0x7f304400c7e0 con 0x7f3050100ff0 2024-03-04T09:30:43.992 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:43.983+0000 7f304dffb700 1 --2- 172.21.15.52:0/1003061547 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f30400a1e10 0x7f30400a42d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:30:43.992 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:43.983+0000 7f304ffff700 1 --2- 172.21.15.52:0/1003061547 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f30400a1e10 0x7f30400a42d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:30:43.992 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:43.983+0000 7f304dffb700 1 -- 172.21.15.52:0/1003061547 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9158+0+0 (secure 0 0 0) 0x7f30440ce310 con 0x7f3050100ff0 2024-03-04T09:30:43.992 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:43.984+0000 7f304ffff700 1 --2- 172.21.15.52:0/1003061547 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f30400a1e10 0x7f30400a42d0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f303c000c00 tx=0x7f303c005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:30:43.992 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:43.988+0000 7f304dffb700 1 -- 172.21.15.52:0/1003061547 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f304409b3a0 con 0x7f3050100ff0 2024-03-04T09:30:44.199 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:44.202+0000 7f3056cc6700 1 -- 172.21.15.52:0/1003061547 --> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f3050062500 con 0x7f30400a1e10 2024-03-04T09:30:44.201 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:44.204+0000 7f304dffb700 1 -- 172.21.15.52:0/1003061547 <== mgr.14152 v2:172.21.15.52:6800/3036033923 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+532 (secure 0 0 0) 0x7f3050062500 con 0x7f30400a1e10 2024-03-04T09:30:44.201 INFO:teuthology.orchestra.run.smithi052.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-03-04T09:30:44.201 INFO:teuthology.orchestra.run.smithi052.stdout:smithi052 /dev/nvme0n1 ssd INTEL_SSDPED1D480GA_PHMB7513010N480DGN 447G No 3m ago Has a FileSystem, LVM detected 2024-03-04T09:30:44.201 INFO:teuthology.orchestra.run.smithi052.stdout:smithi106 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT6204013R400BGN 372G No 3m ago Has a FileSystem, LVM detected 2024-03-04T09:30:44.201 INFO:teuthology.orchestra.run.smithi052.stdout:smithi191 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT6303008M400BGN 372G No 2m ago Has a FileSystem, LVM detected 2024-03-04T09:30:44.203 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:44.206+0000 7f3056cc6700 1 -- 172.21.15.52:0/1003061547 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f30400a1e10 msgr2=0x7f30400a42d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:30:44.203 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:44.206+0000 7f3056cc6700 1 --2- 172.21.15.52:0/1003061547 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f30400a1e10 0x7f30400a42d0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f303c000c00 tx=0x7f303c005fb0 comp rx=0 tx=0).stop 2024-03-04T09:30:44.203 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:44.206+0000 7f3056cc6700 1 -- 172.21.15.52:0/1003061547 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3050100ff0 msgr2=0x7f305006b2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:30:44.203 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:44.206+0000 7f3056cc6700 1 --2- 172.21.15.52:0/1003061547 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3050100ff0 0x7f305006b2c0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f304400e970 tx=0x7f304400ec80 comp rx=0 tx=0).stop 2024-03-04T09:30:44.203 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:44.206+0000 7f3056cc6700 1 -- 172.21.15.52:0/1003061547 shutdown_connections 2024-03-04T09:30:44.203 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:44.206+0000 7f3056cc6700 1 --2- 172.21.15.52:0/1003061547 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f30400a1e10 0x7f30400a42d0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:44.204 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:44.206+0000 7f3056cc6700 1 --2- 172.21.15.52:0/1003061547 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3050102b90 0x7f30501a19b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:44.204 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:44.206+0000 7f3056cc6700 1 --2- 172.21.15.52:0/1003061547 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3050100ff0 0x7f305006b2c0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:44.204 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:44.206+0000 7f3056cc6700 1 --2- 172.21.15.52:0/1003061547 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f30501021f0 0x7f305019f460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:44.204 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:44.206+0000 7f3056cc6700 1 -- 172.21.15.52:0/1003061547 >> 172.21.15.52:0/1003061547 conn(0x7f30500fc590 msgr2=0x7f30501054f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:30:44.204 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:44.206+0000 7f3056cc6700 1 -- 172.21.15.52:0/1003061547 shutdown_connections 2024-03-04T09:30:44.204 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:44.206+0000 7f3056cc6700 1 -- 172.21.15.52:0/1003061547 wait complete. 2024-03-04T09:30:44.897 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-03-04T09:30:44.907 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi052.front.sepia.ceph.com 2024-03-04T09:30:44.907 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4e059f08-da08-11ee-95c6-87774f69a715 -- bash -c 'ceph fs dump' 2024-03-04T09:30:45.156 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/4e059f08-da08-11ee-95c6-87774f69a715/mon.a/config 2024-03-04T09:30:45.537 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:30:45 smithi052 ceph-mon[32473]: pgmap v355: 129 pgs: 2 peering, 127 active+clean; 641 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-03-04T09:30:45.537 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:30:45 smithi052 ceph-mon[32473]: from='client.15297 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T09:30:45.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:30:45 smithi191 ceph-mon[36816]: pgmap v355: 129 pgs: 2 peering, 127 active+clean; 641 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-03-04T09:30:45.587 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:30:45 smithi191 ceph-mon[36816]: from='client.15297 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T09:30:45.631 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:30:45 smithi106 ceph-mon[36484]: pgmap v355: 129 pgs: 2 peering, 127 active+clean; 641 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-03-04T09:30:45.631 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:30:45 smithi106 ceph-mon[36484]: from='client.15297 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T09:30:46.585 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:30:46 smithi052 ceph-mon[32473]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 2 pgs peering) 2024-03-04T09:30:46.585 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:30:46 smithi052 ceph-mon[32473]: Cluster is now healthy 2024-03-04T09:30:46.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:30:46 smithi191 ceph-mon[36816]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 2 pgs peering) 2024-03-04T09:30:46.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:30:46 smithi191 ceph-mon[36816]: Cluster is now healthy 2024-03-04T09:30:46.631 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:30:46 smithi106 ceph-mon[36484]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 2 pgs peering) 2024-03-04T09:30:46.631 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:30:46 smithi106 ceph-mon[36484]: Cluster is now healthy 2024-03-04T09:30:46.926 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:46.926+0000 7f194ebf9700 1 -- 172.21.15.52:0/1696082903 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f19481032f0 msgr2=0x7f1948103750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:30:46.926 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:46.926+0000 7f194ebf9700 1 --2- 172.21.15.52:0/1696082903 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f19481032f0 0x7f1948103750 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f193c0099c0 tx=0x7f193c009cd0 comp rx=0 tx=0).stop 2024-03-04T09:30:46.926 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:46.926+0000 7f194ebf9700 1 -- 172.21.15.52:0/1696082903 shutdown_connections 2024-03-04T09:30:46.926 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:46.926+0000 7f194ebf9700 1 --2- 172.21.15.52:0/1696082903 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1948103c90 0x7f194810a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:46.926 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:46.926+0000 7f194ebf9700 1 --2- 172.21.15.52:0/1696082903 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f19481020f0 0x7f1948102510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:46.927 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:46.926+0000 7f194ebf9700 1 --2- 172.21.15.52:0/1696082903 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f19481032f0 0x7f1948103750 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:46.927 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:46.927+0000 7f194ebf9700 1 -- 172.21.15.52:0/1696082903 >> 172.21.15.52:0/1696082903 conn(0x7f19480fd6d0 msgr2=0x7f19480ffaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:30:46.927 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:46.927+0000 7f194ebf9700 1 -- 172.21.15.52:0/1696082903 shutdown_connections 2024-03-04T09:30:46.927 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:46.927+0000 7f194ebf9700 1 -- 172.21.15.52:0/1696082903 wait complete. 2024-03-04T09:30:46.927 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:46.927+0000 7f194ebf9700 1 Processor -- start 2024-03-04T09:30:46.927 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:46.927+0000 7f194ebf9700 1 -- start start 2024-03-04T09:30:46.927 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:46.927+0000 7f194ebf9700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f19481020f0 0x7f194819a510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:30:46.927 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:46.927+0000 7f194ebf9700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f19481032f0 0x7f194819aa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:30:46.927 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:46.927+0000 7f194ebf9700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1948103c90 0x7f19481a1ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:30:46.927 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:46.927+0000 7f194ebf9700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f194810dcb0 con 0x7f19481020f0 2024-03-04T09:30:46.927 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:46.927+0000 7f194ebf9700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f194810db30 con 0x7f1948103c90 2024-03-04T09:30:46.927 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:46.927+0000 7f194ebf9700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f194810de30 con 0x7f19481032f0 2024-03-04T09:30:46.928 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:46.928+0000 7f194d196700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1948103c90 0x7f19481a1ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:30:46.928 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:46.928+0000 7f194d196700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1948103c90 0x7f19481a1ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.52:49578/0 (socket says 172.21.15.52:49578) 2024-03-04T09:30:46.928 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:46.928+0000 7f194c995700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f19481020f0 0x7f194819a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:30:46.928 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:46.928+0000 7f194d196700 1 -- 172.21.15.52:0/218109252 learned_addr learned my addr 172.21.15.52:0/218109252 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-04T09:30:46.928 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:46.928+0000 7f1947fff700 1 --2- 172.21.15.52:0/218109252 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f19481032f0 0x7f194819aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:30:46.928 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:46.928+0000 7f194d196700 1 -- 172.21.15.52:0/218109252 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f19481032f0 msgr2=0x7f194819aa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:30:46.928 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:46.928+0000 7f194d196700 1 --2- 172.21.15.52:0/218109252 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f19481032f0 0x7f194819aa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:46.928 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:46.928+0000 7f194d196700 1 -- 172.21.15.52:0/218109252 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f19481020f0 msgr2=0x7f194819a510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:30:46.928 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:46.928+0000 7f194d196700 1 --2- 172.21.15.52:0/218109252 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f19481020f0 0x7f194819a510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:46.928 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:46.928+0000 7f194d196700 1 -- 172.21.15.52:0/218109252 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19481a2170 con 0x7f1948103c90 2024-03-04T09:30:46.928 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:46.928+0000 7f194c995700 1 --2- 172.21.15.52:0/218109252 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f19481020f0 0x7f194819a510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-04T09:30:46.928 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:46.928+0000 7f194d196700 1 --2- 172.21.15.52:0/218109252 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1948103c90 0x7f19481a1ad0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f194000ef30 tx=0x7f194000c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:30:46.929 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:46.928+0000 7f1945ffb700 1 -- 172.21.15.52:0/218109252 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19400092e0 con 0x7f1948103c90 2024-03-04T09:30:46.929 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:46.928+0000 7f1945ffb700 1 -- 172.21.15.52:0/218109252 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1940004bb0 con 0x7f1948103c90 2024-03-04T09:30:46.929 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:46.928+0000 7f1945ffb700 1 -- 172.21.15.52:0/218109252 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1940004280 con 0x7f1948103c90 2024-03-04T09:30:46.929 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:46.928+0000 7f194ebf9700 1 -- 172.21.15.52:0/218109252 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19481a2400 con 0x7f1948103c90 2024-03-04T09:30:46.929 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:46.929+0000 7f194ebf9700 1 -- 172.21.15.52:0/218109252 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19481a28f0 con 0x7f1948103c90 2024-03-04T09:30:46.930 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:46.930+0000 7f194ebf9700 1 -- 172.21.15.52:0/218109252 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1948068d80 con 0x7f1948103c90 2024-03-04T09:30:46.930 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:46.931+0000 7f1945ffb700 1 -- 172.21.15.52:0/218109252 <== mon.2 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 18) v1 ==== 136756+0+0 (secure 0 0 0) 0x7f194001b030 con 0x7f1948103c90 2024-03-04T09:30:46.930 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:46.931+0000 7f1945ffb700 1 --2- 172.21.15.52:0/218109252 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f19300a1e10 0x7f19300a42d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:30:46.930 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:46.931+0000 7f194c995700 1 --2- 172.21.15.52:0/218109252 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f19300a1e10 0x7f19300a42d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:30:46.930 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:46.931+0000 7f1945ffb700 1 -- 172.21.15.52:0/218109252 <== mon.2 v2:172.21.15.106:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9158+0+0 (secure 0 0 0) 0x7f19400cdc50 con 0x7f1948103c90 2024-03-04T09:30:46.931 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:46.932+0000 7f194c995700 1 --2- 172.21.15.52:0/218109252 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f19300a1e10 0x7f19300a42d0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f1948103150 tx=0x7f1938009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:30:46.937 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:46.940+0000 7f1945ffb700 1 -- 172.21.15.52:0/218109252 <== mon.2 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f1940022020 con 0x7f1948103c90 2024-03-04T09:30:47.185 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:47.187+0000 7f194ebf9700 1 -- 172.21.15.52:0/218109252 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f19481a3270 con 0x7f1948103c90 2024-03-04T09:30:47.185 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:47.188+0000 7f1945ffb700 1 -- 172.21.15.52:0/218109252 <== mon.2 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 24 v24) v1 ==== 76+0+3002 (secure 0 0 0) 0x7f1940010040 con 0x7f1948103c90 2024-03-04T09:30:47.185 INFO:teuthology.orchestra.run.smithi052.stdout:e24 2024-03-04T09:30:47.185 INFO:teuthology.orchestra.run.smithi052.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-04T09:30:47.185 INFO:teuthology.orchestra.run.smithi052.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments} 2024-03-04T09:30:47.185 INFO:teuthology.orchestra.run.smithi052.stdout:legacy client fscid: 1 2024-03-04T09:30:47.185 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-03-04T09:30:47.186 INFO:teuthology.orchestra.run.smithi052.stdout:Filesystem 'cephfs' (1) 2024-03-04T09:30:47.186 INFO:teuthology.orchestra.run.smithi052.stdout:fs_name cephfs 2024-03-04T09:30:47.186 INFO:teuthology.orchestra.run.smithi052.stdout:epoch 24 2024-03-04T09:30:47.186 INFO:teuthology.orchestra.run.smithi052.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-03-04T09:30:47.186 INFO:teuthology.orchestra.run.smithi052.stdout:created 2024-03-04T09:28:37.264970+0000 2024-03-04T09:30:47.186 INFO:teuthology.orchestra.run.smithi052.stdout:modified 2024-03-04T09:29:04.001848+0000 2024-03-04T09:30:47.186 INFO:teuthology.orchestra.run.smithi052.stdout:tableserver 0 2024-03-04T09:30:47.186 INFO:teuthology.orchestra.run.smithi052.stdout:root 0 2024-03-04T09:30:47.186 INFO:teuthology.orchestra.run.smithi052.stdout:session_timeout 300 2024-03-04T09:30:47.186 INFO:teuthology.orchestra.run.smithi052.stdout:session_autoclose 300 2024-03-04T09:30:47.186 INFO:teuthology.orchestra.run.smithi052.stdout:max_file_size 1099511627776 2024-03-04T09:30:47.187 INFO:teuthology.orchestra.run.smithi052.stdout:max_xattr_size 65536 2024-03-04T09:30:47.187 INFO:teuthology.orchestra.run.smithi052.stdout:required_client_features {} 2024-03-04T09:30:47.187 INFO:teuthology.orchestra.run.smithi052.stdout:last_failure 0 2024-03-04T09:30:47.187 INFO:teuthology.orchestra.run.smithi052.stdout:last_failure_osd_epoch 0 2024-03-04T09:30:47.187 INFO:teuthology.orchestra.run.smithi052.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments} 2024-03-04T09:30:47.187 INFO:teuthology.orchestra.run.smithi052.stdout:max_mds 3 2024-03-04T09:30:47.187 INFO:teuthology.orchestra.run.smithi052.stdout:in 0,1,2 2024-03-04T09:30:47.187 INFO:teuthology.orchestra.run.smithi052.stdout:up {0=24506,1=24490,2=24473} 2024-03-04T09:30:47.187 INFO:teuthology.orchestra.run.smithi052.stdout:failed 2024-03-04T09:30:47.187 INFO:teuthology.orchestra.run.smithi052.stdout:damaged 2024-03-04T09:30:47.187 INFO:teuthology.orchestra.run.smithi052.stdout:stopped 2024-03-04T09:30:47.188 INFO:teuthology.orchestra.run.smithi052.stdout:data_pools [3,4] 2024-03-04T09:30:47.188 INFO:teuthology.orchestra.run.smithi052.stdout:metadata_pool 2 2024-03-04T09:30:47.188 INFO:teuthology.orchestra.run.smithi052.stdout:inline_data disabled 2024-03-04T09:30:47.188 INFO:teuthology.orchestra.run.smithi052.stdout:balancer 2024-03-04T09:30:47.188 INFO:teuthology.orchestra.run.smithi052.stdout:bal_rank_mask -1 2024-03-04T09:30:47.188 INFO:teuthology.orchestra.run.smithi052.stdout:standby_count_wanted 0 2024-03-04T09:30:47.188 INFO:teuthology.orchestra.run.smithi052.stdout:qdb_cluster leader: 24506 members: 24473,24490,24506 2024-03-04T09:30:47.188 INFO:teuthology.orchestra.run.smithi052.stdout:[mds.b{0:24506} state up:active seq 3 addr [v2:172.21.15.106:6838/1995853093,v1:172.21.15.106:6839/1995853093] compat {c=[1],r=[1],i=[fff]}] 2024-03-04T09:30:47.188 INFO:teuthology.orchestra.run.smithi052.stdout:[mds.c{0:24463} state up:standby-replay seq 1 addr [v2:172.21.15.191:6836/4071064225,v1:172.21.15.191:6837/4071064225] compat {c=[1],r=[1],i=[fff]}] 2024-03-04T09:30:47.188 INFO:teuthology.orchestra.run.smithi052.stdout:[mds.i{1:24490} state up:active seq 11 addr [v2:172.21.15.191:6838/2194324891,v1:172.21.15.191:6839/2194324891] compat {c=[1],r=[1],i=[fff]}] 2024-03-04T09:30:47.188 INFO:teuthology.orchestra.run.smithi052.stdout:[mds.f{1:24451} state up:standby-replay seq 1 addr [v2:172.21.15.191:6834/105048699,v1:172.21.15.191:6835/105048699] compat {c=[1],r=[1],i=[fff]}] 2024-03-04T09:30:47.189 INFO:teuthology.orchestra.run.smithi052.stdout:[mds.h{2:24473} state up:active seq 12 addr [v2:172.21.15.106:6836/2901755066,v1:172.21.15.106:6837/2901755066] compat {c=[1],r=[1],i=[fff]}] 2024-03-04T09:30:47.189 INFO:teuthology.orchestra.run.smithi052.stdout:[mds.e{2:24440} state up:standby-replay seq 1 addr [v2:172.21.15.106:6834/1541069897,v1:172.21.15.106:6835/1541069897] compat {c=[1],r=[1],i=[fff]}] 2024-03-04T09:30:47.189 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-03-04T09:30:47.189 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-03-04T09:30:47.189 INFO:teuthology.orchestra.run.smithi052.stdout:Standby daemons: 2024-03-04T09:30:47.189 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-03-04T09:30:47.189 INFO:teuthology.orchestra.run.smithi052.stdout:[mds.j{-1:14547} state up:standby seq 1 addr [v2:172.21.15.52:6834/1007961019,v1:172.21.15.52:6835/1007961019] compat {c=[1],r=[1],i=[fff]}] 2024-03-04T09:30:47.189 INFO:teuthology.orchestra.run.smithi052.stdout:[mds.d{-1:14559} state up:standby seq 1 addr [v2:172.21.15.52:6836/3753881283,v1:172.21.15.52:6837/3753881283] compat {c=[1],r=[1],i=[fff]}] 2024-03-04T09:30:47.189 INFO:teuthology.orchestra.run.smithi052.stdout:[mds.g{-1:14628} state up:standby seq 1 addr [v2:172.21.15.52:6838/3467897808,v1:172.21.15.52:6839/3467897808] compat {c=[1],r=[1],i=[fff]}] 2024-03-04T09:30:47.189 INFO:teuthology.orchestra.run.smithi052.stdout:[mds.a{-1:14640} state up:standby seq 1 addr [v2:172.21.15.52:6840/2804292291,v1:172.21.15.52:6841/2804292291] compat {c=[1],r=[1],i=[fff]}] 2024-03-04T09:30:47.189 INFO:teuthology.orchestra.run.smithi052.stdout:[mds.k{-1:24407} state up:standby seq 1 addr [v2:172.21.15.106:6832/1818480323,v1:172.21.15.106:6833/1818480323] compat {c=[1],r=[1],i=[fff]}] 2024-03-04T09:30:47.189 INFO:teuthology.orchestra.run.smithi052.stdout:[mds.l{-1:24412} state up:standby seq 1 addr [v2:172.21.15.191:6832/765768204,v1:172.21.15.191:6833/765768204] compat {c=[1],r=[1],i=[fff]}] 2024-03-04T09:30:47.190 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:47.191+0000 7f194ebf9700 1 -- 172.21.15.52:0/218109252 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f19300a1e10 msgr2=0x7f19300a42d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:30:47.191 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:47.191+0000 7f194ebf9700 1 --2- 172.21.15.52:0/218109252 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f19300a1e10 0x7f19300a42d0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f1948103150 tx=0x7f1938009260 comp rx=0 tx=0).stop 2024-03-04T09:30:47.191 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:47.191+0000 7f194ebf9700 1 -- 172.21.15.52:0/218109252 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1948103c90 msgr2=0x7f19481a1ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:30:47.191 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:47.191+0000 7f194ebf9700 1 --2- 172.21.15.52:0/218109252 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1948103c90 0x7f19481a1ad0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f194000ef30 tx=0x7f194000c3a0 comp rx=0 tx=0).stop 2024-03-04T09:30:47.191 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:47.191+0000 7f194ebf9700 1 -- 172.21.15.52:0/218109252 shutdown_connections 2024-03-04T09:30:47.191 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:47.191+0000 7f194ebf9700 1 --2- 172.21.15.52:0/218109252 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f19300a1e10 0x7f19300a42d0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:47.191 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:47.191+0000 7f194ebf9700 1 --2- 172.21.15.52:0/218109252 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1948103c90 0x7f19481a1ad0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:47.191 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:47.191+0000 7f194ebf9700 1 --2- 172.21.15.52:0/218109252 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f19481020f0 0x7f194819a510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:47.191 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:47.191+0000 7f194ebf9700 1 --2- 172.21.15.52:0/218109252 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f19481032f0 0x7f194819aa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:47.191 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:47.191+0000 7f194ebf9700 1 -- 172.21.15.52:0/218109252 >> 172.21.15.52:0/218109252 conn(0x7f19480fd6d0 msgr2=0x7f19480742f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:30:47.191 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:47.191+0000 7f194ebf9700 1 -- 172.21.15.52:0/218109252 shutdown_connections 2024-03-04T09:30:47.191 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:47.191+0000 7f194ebf9700 1 -- 172.21.15.52:0/218109252 wait complete. 2024-03-04T09:30:47.192 INFO:teuthology.orchestra.run.smithi052.stderr:dumped fsmap epoch 24 2024-03-04T09:30:47.584 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:30:47 smithi052 ceph-mon[32473]: pgmap v356: 129 pgs: 129 active+clean; 641 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s; 2 B/s, 1 keys/s, 0 objects/s recovering 2024-03-04T09:30:47.585 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:30:47 smithi052 ceph-mon[32473]: from='client.? 172.21.15.52:0/218109252' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-04T09:30:47.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:30:47 smithi191 ceph-mon[36816]: pgmap v356: 129 pgs: 129 active+clean; 641 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s; 2 B/s, 1 keys/s, 0 objects/s recovering 2024-03-04T09:30:47.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:30:47 smithi191 ceph-mon[36816]: from='client.? 172.21.15.52:0/218109252' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-04T09:30:47.631 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:30:47 smithi106 ceph-mon[36484]: pgmap v356: 129 pgs: 129 active+clean; 641 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s; 2 B/s, 1 keys/s, 0 objects/s recovering 2024-03-04T09:30:47.631 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:30:47 smithi106 ceph-mon[36484]: from='client.? 172.21.15.52:0/218109252' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-04T09:30:47.898 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4e059f08-da08-11ee-95c6-87774f69a715 -- bash -c 'ceph osd dump' 2024-03-04T09:30:48.145 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/4e059f08-da08-11ee-95c6-87774f69a715/mon.a/config 2024-03-04T09:30:49.585 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:30:49 smithi052 ceph-mon[32473]: pgmap v357: 129 pgs: 129 active+clean; 641 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 7 op/s; 2 B/s, 1 keys/s, 0 objects/s recovering 2024-03-04T09:30:49.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:30:49 smithi191 ceph-mon[36816]: pgmap v357: 129 pgs: 129 active+clean; 641 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 7 op/s; 2 B/s, 1 keys/s, 0 objects/s recovering 2024-03-04T09:30:49.631 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:30:49 smithi106 ceph-mon[36484]: pgmap v357: 129 pgs: 129 active+clean; 641 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 7 op/s; 2 B/s, 1 keys/s, 0 objects/s recovering 2024-03-04T09:30:49.873 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:49.875+0000 7feab6bd8700 1 -- 172.21.15.52:0/8712555 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7feab01036f0 msgr2=0x7feab0103b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:30:49.873 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:49.875+0000 7feab6bd8700 1 --2- 172.21.15.52:0/8712555 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7feab01036f0 0x7feab0103b10 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7feaa4009990 tx=0x7feaa4009ca0 comp rx=0 tx=0).stop 2024-03-04T09:30:49.873 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:49.876+0000 7feab6bd8700 1 -- 172.21.15.52:0/8712555 shutdown_connections 2024-03-04T09:30:49.873 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:49.876+0000 7feab6bd8700 1 --2- 172.21.15.52:0/8712555 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7feab0100b10 0x7feab010a6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:49.873 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:49.876+0000 7feab6bd8700 1 --2- 172.21.15.52:0/8712555 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7feab01036f0 0x7feab0103b10 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:49.873 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:49.876+0000 7feab6bd8700 1 --2- 172.21.15.52:0/8712555 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7feab0100170 0x7feab01005d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:49.873 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:49.876+0000 7feab6bd8700 1 -- 172.21.15.52:0/8712555 >> 172.21.15.52:0/8712555 conn(0x7feab00fbd10 msgr2=0x7feab00fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:30:49.873 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:49.876+0000 7feab6bd8700 1 -- 172.21.15.52:0/8712555 shutdown_connections 2024-03-04T09:30:49.873 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:49.876+0000 7feab6bd8700 1 -- 172.21.15.52:0/8712555 wait complete. 2024-03-04T09:30:49.875 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:49.876+0000 7feab6bd8700 1 Processor -- start 2024-03-04T09:30:49.875 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:49.876+0000 7feab6bd8700 1 -- start start 2024-03-04T09:30:49.875 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:49.877+0000 7feab6bd8700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7feab0100170 0x7feab0198340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:30:49.875 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:49.877+0000 7feab6bd8700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7feab0100b10 0x7feab0198880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:30:49.875 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:49.877+0000 7feab6bd8700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7feab01036f0 0x7feab019f900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:30:49.875 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:49.877+0000 7feab6bd8700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feab010dd00 con 0x7feab01036f0 2024-03-04T09:30:49.876 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:49.877+0000 7feab6bd8700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feab010db80 con 0x7feab0100b10 2024-03-04T09:30:49.876 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:49.877+0000 7feab6bd8700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feab010de80 con 0x7feab0100170 2024-03-04T09:30:49.876 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:49.877+0000 7feab5175700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7feab01036f0 0x7feab019f900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:30:49.876 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:49.877+0000 7feab4974700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7feab0100170 0x7feab0198340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:30:49.876 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:49.877+0000 7feaaffff700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7feab0100b10 0x7feab0198880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:30:49.876 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:49.877+0000 7feab4974700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7feab0100170 0x7feab0198340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.52:47074/0 (socket says 172.21.15.52:47074) 2024-03-04T09:30:49.876 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:49.877+0000 7feaaffff700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7feab0100b10 0x7feab0198880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.52:49606/0 (socket says 172.21.15.52:49606) 2024-03-04T09:30:49.876 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:49.877+0000 7feab4974700 1 -- 172.21.15.52:0/894869983 learned_addr learned my addr 172.21.15.52:0/894869983 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-04T09:30:49.876 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:49.877+0000 7feab5175700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7feab01036f0 0x7feab019f900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:58036/0 (socket says 172.21.15.52:58036) 2024-03-04T09:30:49.876 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:49.877+0000 7feab5175700 1 -- 172.21.15.52:0/894869983 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7feab0100170 msgr2=0x7feab0198340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:30:49.876 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:49.877+0000 7feab5175700 1 --2- 172.21.15.52:0/894869983 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7feab0100170 0x7feab0198340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:49.876 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:49.877+0000 7feab5175700 1 -- 172.21.15.52:0/894869983 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7feab0100b10 msgr2=0x7feab0198880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:30:49.876 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:49.877+0000 7feab5175700 1 --2- 172.21.15.52:0/894869983 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7feab0100b10 0x7feab0198880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:49.877 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:49.877+0000 7feab5175700 1 -- 172.21.15.52:0/894869983 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feab019ffa0 con 0x7feab01036f0 2024-03-04T09:30:49.877 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:49.877+0000 7feab4974700 1 --2- 172.21.15.52:0/894869983 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7feab0100170 0x7feab0198340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-04T09:30:49.877 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:49.877+0000 7feaaffff700 1 --2- 172.21.15.52:0/894869983 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7feab0100b10 0x7feab0198880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-04T09:30:49.877 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:49.877+0000 7feab5175700 1 --2- 172.21.15.52:0/894869983 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7feab01036f0 0x7feab019f900 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7feaa8009f60 tx=0x7feaa8009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:30:49.877 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:49.878+0000 7feaadffb700 1 -- 172.21.15.52:0/894869983 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feaa80092e0 con 0x7feab01036f0 2024-03-04T09:30:49.877 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:49.878+0000 7feaadffb700 1 -- 172.21.15.52:0/894869983 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7feaa8004bb0 con 0x7feab01036f0 2024-03-04T09:30:49.877 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:49.878+0000 7feaadffb700 1 -- 172.21.15.52:0/894869983 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feaa8004280 con 0x7feab01036f0 2024-03-04T09:30:49.877 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:49.878+0000 7feab6bd8700 1 -- 172.21.15.52:0/894869983 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feab01a0230 con 0x7feab01036f0 2024-03-04T09:30:49.877 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:49.878+0000 7feab6bd8700 1 -- 172.21.15.52:0/894869983 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feab01a0770 con 0x7feab01036f0 2024-03-04T09:30:49.878 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:49.879+0000 7feab6bd8700 1 -- 172.21.15.52:0/894869983 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feab0068d80 con 0x7feab01036f0 2024-03-04T09:30:49.878 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:49.880+0000 7feaadffb700 1 -- 172.21.15.52:0/894869983 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) v1 ==== 136756+0+0 (secure 0 0 0) 0x7feaa801b030 con 0x7feab01036f0 2024-03-04T09:30:49.878 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:49.880+0000 7feaadffb700 1 --2- 172.21.15.52:0/894869983 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7feaa00a1e90 0x7feaa00a4350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:30:49.878 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:49.880+0000 7feaadffb700 1 -- 172.21.15.52:0/894869983 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9158+0+0 (secure 0 0 0) 0x7feaa80ce500 con 0x7feab01036f0 2024-03-04T09:30:49.878 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:49.880+0000 7feab4974700 1 --2- 172.21.15.52:0/894869983 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7feaa00a1e90 0x7feaa00a4350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:30:49.878 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:49.881+0000 7feab4974700 1 --2- 172.21.15.52:0/894869983 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7feaa00a1e90 0x7feaa00a4350 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7feaa4006010 tx=0x7feaa401a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:30:49.881 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:49.884+0000 7feaadffb700 1 -- 172.21.15.52:0/894869983 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7feaa8022020 con 0x7feab01036f0 2024-03-04T09:30:50.090 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:50.093+0000 7feab6bd8700 1 -- 172.21.15.52:0/894869983 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd dump"} v 0) v1 -- 0x7feab002d9e0 con 0x7feab01036f0 2024-03-04T09:30:50.091 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:50.094+0000 7feaadffb700 1 -- 172.21.15.52:0/894869983 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v93) v1 ==== 56+0+5671 (secure 0 0 0) 0x7feaa8010040 con 0x7feab01036f0 2024-03-04T09:30:50.091 INFO:teuthology.orchestra.run.smithi052.stdout:epoch 93 2024-03-04T09:30:50.092 INFO:teuthology.orchestra.run.smithi052.stdout:fsid 4e059f08-da08-11ee-95c6-87774f69a715 2024-03-04T09:30:50.092 INFO:teuthology.orchestra.run.smithi052.stdout:created 2024-03-04T09:20:46.104963+0000 2024-03-04T09:30:50.092 INFO:teuthology.orchestra.run.smithi052.stdout:modified 2024-03-04T09:30:41.237792+0000 2024-03-04T09:30:50.092 INFO:teuthology.orchestra.run.smithi052.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-03-04T09:30:50.092 INFO:teuthology.orchestra.run.smithi052.stdout:crush_version 27 2024-03-04T09:30:50.092 INFO:teuthology.orchestra.run.smithi052.stdout:full_ratio 0.95 2024-03-04T09:30:50.092 INFO:teuthology.orchestra.run.smithi052.stdout:backfillfull_ratio 0.9 2024-03-04T09:30:50.092 INFO:teuthology.orchestra.run.smithi052.stdout:nearfull_ratio 0.85 2024-03-04T09:30:50.092 INFO:teuthology.orchestra.run.smithi052.stdout:require_min_compat_client luminous 2024-03-04T09:30:50.092 INFO:teuthology.orchestra.run.smithi052.stdout:min_compat_client luminous 2024-03-04T09:30:50.092 INFO:teuthology.orchestra.run.smithi052.stdout:require_osd_release squid 2024-03-04T09:30:50.093 INFO:teuthology.orchestra.run.smithi052.stdout:stretch_mode_enabled false 2024-03-04T09:30:50.093 INFO:teuthology.orchestra.run.smithi052.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 21 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 12.00 2024-03-04T09:30:50.093 INFO:teuthology.orchestra.run.smithi052.stdout:pool 2 'cephfs_metadata' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 84 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 1.88 2024-03-04T09:30:50.093 INFO:teuthology.orchestra.run.smithi052.stdout:pool 3 'cephfs_data' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 85 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.05 application cephfs read_balance_score 1.50 2024-03-04T09:30:50.093 INFO:teuthology.orchestra.run.smithi052.stdout:pool 4 'cephfs_data_ec' erasure profile cephfs_data_ec size 4 min_size 3 crush_rule 1 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 89 flags hashpspool,ec_overwrites stripe_width 8192 pg_num_min 64 target_size_ratio 0.9 application cephfs 2024-03-04T09:30:50.093 INFO:teuthology.orchestra.run.smithi052.stdout:max_osd 12 2024-03-04T09:30:50.093 INFO:teuthology.orchestra.run.smithi052.stdout:osd.0 up in weight 1 up_from 8 up_thru 90 down_at 0 last_clean_interval [0,0) [v2:172.21.15.52:6802/1371221397,v1:172.21.15.52:6803/1371221397] [v2:172.21.15.52:6804/1371221397,v1:172.21.15.52:6805/1371221397] exists,up 583172be-602f-4bf4-bfcb-42663cf5b277 2024-03-04T09:30:50.093 INFO:teuthology.orchestra.run.smithi052.stdout:osd.1 up in weight 1 up_from 13 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.106:6800/2540865537,v1:172.21.15.106:6801/2540865537] [v2:172.21.15.106:6802/2540865537,v1:172.21.15.106:6803/2540865537] exists,up 5ae0081c-a2f6-46eb-ace0-0368208377b3 2024-03-04T09:30:50.093 INFO:teuthology.orchestra.run.smithi052.stdout:osd.2 up in weight 1 up_from 18 up_thru 90 down_at 0 last_clean_interval [0,0) [v2:172.21.15.191:6800/1377083164,v1:172.21.15.191:6801/1377083164] [v2:172.21.15.191:6802/1377083164,v1:172.21.15.191:6803/1377083164] exists,up 1948bac7-2bb2-4218-8b93-4e4743a0de04 2024-03-04T09:30:50.093 INFO:teuthology.orchestra.run.smithi052.stdout:osd.3 up in weight 1 up_from 25 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.52:6810/2388571810,v1:172.21.15.52:6811/2388571810] [v2:172.21.15.52:6812/2388571810,v1:172.21.15.52:6813/2388571810] exists,up 16c9e9fe-7167-4992-bf12-1a7398e0498a 2024-03-04T09:30:50.094 INFO:teuthology.orchestra.run.smithi052.stdout:osd.4 up in weight 1 up_from 33 up_thru 90 down_at 0 last_clean_interval [0,0) [v2:172.21.15.106:6808/3465868015,v1:172.21.15.106:6809/3465868015] [v2:172.21.15.106:6810/3465868015,v1:172.21.15.106:6811/3465868015] exists,up c8ef48c7-7c78-44d3-a9e5-99ec750abd3d 2024-03-04T09:30:50.094 INFO:teuthology.orchestra.run.smithi052.stdout:osd.5 up in weight 1 up_from 38 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.191:6808/1485092643,v1:172.21.15.191:6809/1485092643] [v2:172.21.15.191:6810/1485092643,v1:172.21.15.191:6811/1485092643] exists,up 9e41824a-c5a2-4f88-be50-2ba1dceb3220 2024-03-04T09:30:50.094 INFO:teuthology.orchestra.run.smithi052.stdout:osd.6 up in weight 1 up_from 46 up_thru 90 down_at 0 last_clean_interval [0,0) [v2:172.21.15.52:6818/1989797856,v1:172.21.15.52:6819/1989797856] [v2:172.21.15.52:6820/1989797856,v1:172.21.15.52:6821/1989797856] exists,up b35dccee-ec16-452e-ac69-3ac2fb7b27fa 2024-03-04T09:30:50.094 INFO:teuthology.orchestra.run.smithi052.stdout:osd.7 up in weight 1 up_from 51 up_thru 90 down_at 0 last_clean_interval [0,0) [v2:172.21.15.106:6816/1450870329,v1:172.21.15.106:6817/1450870329] [v2:172.21.15.106:6818/1450870329,v1:172.21.15.106:6819/1450870329] exists,up 71303e3d-14ec-4cbd-b19c-67645a1de3c4 2024-03-04T09:30:50.094 INFO:teuthology.orchestra.run.smithi052.stdout:osd.8 up in weight 1 up_from 59 up_thru 90 down_at 0 last_clean_interval [0,0) [v2:172.21.15.191:6816/1114119072,v1:172.21.15.191:6817/1114119072] [v2:172.21.15.191:6818/1114119072,v1:172.21.15.191:6819/1114119072] exists,up 6f5786c4-4402-435a-bdad-adff46fe9473 2024-03-04T09:30:50.094 INFO:teuthology.orchestra.run.smithi052.stdout:osd.9 up in weight 1 up_from 67 up_thru 90 down_at 0 last_clean_interval [0,0) [v2:172.21.15.52:6826/3485821914,v1:172.21.15.52:6827/3485821914] [v2:172.21.15.52:6828/3485821914,v1:172.21.15.52:6829/3485821914] exists,up 062bbaa5-1227-4684-b859-450f5594a90c 2024-03-04T09:30:50.094 INFO:teuthology.orchestra.run.smithi052.stdout:osd.10 up in weight 1 up_from 72 up_thru 83 down_at 0 last_clean_interval [0,0) [v2:172.21.15.106:6824/3815586576,v1:172.21.15.106:6825/3815586576] [v2:172.21.15.106:6826/3815586576,v1:172.21.15.106:6827/3815586576] exists,up 1a285a5a-f793-48da-a8eb-5326d9528cd8 2024-03-04T09:30:50.094 INFO:teuthology.orchestra.run.smithi052.stdout:osd.11 up in weight 1 up_from 80 up_thru 92 down_at 0 last_clean_interval [0,0) [v2:172.21.15.191:6824/2009632843,v1:172.21.15.191:6825/2009632843] [v2:172.21.15.191:6826/2009632843,v1:172.21.15.191:6827/2009632843] exists,up 1f8b9a43-57fa-41f4-9969-3b803a74bf45 2024-03-04T09:30:50.094 INFO:teuthology.orchestra.run.smithi052.stdout:pg_upmap_items 2.5 [8,11] 2024-03-04T09:30:50.094 INFO:teuthology.orchestra.run.smithi052.stdout:pg_upmap_items 2.e [2,4] 2024-03-04T09:30:50.094 INFO:teuthology.orchestra.run.smithi052.stdout:pg_upmap_items 2.1d [2,1,8,11] 2024-03-04T09:30:50.095 INFO:teuthology.orchestra.run.smithi052.stdout:pg_upmap_items 3.13 [4,9] 2024-03-04T09:30:50.095 INFO:teuthology.orchestra.run.smithi052.stdout:pg_upmap_items 3.20 [5,9] 2024-03-04T09:30:50.095 INFO:teuthology.orchestra.run.smithi052.stdout:pg_upmap_items 3.21 [4,6] 2024-03-04T09:30:50.095 INFO:teuthology.orchestra.run.smithi052.stdout:pg_upmap_items 3.23 [4,8] 2024-03-04T09:30:50.095 INFO:teuthology.orchestra.run.smithi052.stdout:pg_upmap_items 3.38 [3,9] 2024-03-04T09:30:50.095 INFO:teuthology.orchestra.run.smithi052.stdout:pg_upmap_items 4.9 [5,8] 2024-03-04T09:30:50.095 INFO:teuthology.orchestra.run.smithi052.stdout:pg_upmap_items 4.17 [9,8] 2024-03-04T09:30:50.095 INFO:teuthology.orchestra.run.smithi052.stdout:pg_upmap_items 4.1b [9,10] 2024-03-04T09:30:50.095 INFO:teuthology.orchestra.run.smithi052.stdout:pg_upmap_items 4.1f [9,8] 2024-03-04T09:30:50.095 INFO:teuthology.orchestra.run.smithi052.stdout:blocklist 172.21.15.52:0/4082645718 expires 2024-03-05T09:21:39.401935+0000 2024-03-04T09:30:50.095 INFO:teuthology.orchestra.run.smithi052.stdout:blocklist 172.21.15.52:0/1271949988 expires 2024-03-05T09:21:39.401935+0000 2024-03-04T09:30:50.095 INFO:teuthology.orchestra.run.smithi052.stdout:blocklist 172.21.15.52:6801/2379361333 expires 2024-03-05T09:21:12.760938+0000 2024-03-04T09:30:50.095 INFO:teuthology.orchestra.run.smithi052.stdout:blocklist 172.21.15.52:0/497075075 expires 2024-03-05T09:21:12.760938+0000 2024-03-04T09:30:50.096 INFO:teuthology.orchestra.run.smithi052.stdout:blocklist 172.21.15.52:6800/2379361333 expires 2024-03-05T09:21:12.760938+0000 2024-03-04T09:30:50.096 INFO:teuthology.orchestra.run.smithi052.stdout:blocklist 172.21.15.52:0/568242577 expires 2024-03-05T09:21:12.760938+0000 2024-03-04T09:30:50.096 INFO:teuthology.orchestra.run.smithi052.stdout:blocklist 172.21.15.52:6801/2425730636 expires 2024-03-05T09:21:39.401935+0000 2024-03-04T09:30:50.096 INFO:teuthology.orchestra.run.smithi052.stdout:blocklist 172.21.15.52:6800/2425730636 expires 2024-03-05T09:21:39.401935+0000 2024-03-04T09:30:50.096 INFO:teuthology.orchestra.run.smithi052.stdout:blocklist 172.21.15.52:0/2743084243 expires 2024-03-05T09:21:39.401935+0000 2024-03-04T09:30:50.096 INFO:teuthology.orchestra.run.smithi052.stdout:blocklist 172.21.15.52:0/1035878034 expires 2024-03-05T09:21:12.760938+0000 2024-03-04T09:30:50.097 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:50.097+0000 7feab6bd8700 1 -- 172.21.15.52:0/894869983 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7feaa00a1e90 msgr2=0x7feaa00a4350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:30:50.097 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:50.097+0000 7feab6bd8700 1 --2- 172.21.15.52:0/894869983 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7feaa00a1e90 0x7feaa00a4350 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7feaa4006010 tx=0x7feaa401a040 comp rx=0 tx=0).stop 2024-03-04T09:30:50.097 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:50.097+0000 7feab6bd8700 1 -- 172.21.15.52:0/894869983 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7feab01036f0 msgr2=0x7feab019f900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:30:50.097 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:50.097+0000 7feab6bd8700 1 --2- 172.21.15.52:0/894869983 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7feab01036f0 0x7feab019f900 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7feaa8009f60 tx=0x7feaa8009f90 comp rx=0 tx=0).stop 2024-03-04T09:30:50.097 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:50.097+0000 7feab6bd8700 1 -- 172.21.15.52:0/894869983 shutdown_connections 2024-03-04T09:30:50.097 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:50.097+0000 7feab6bd8700 1 --2- 172.21.15.52:0/894869983 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7feaa00a1e90 0x7feaa00a4350 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:50.098 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:50.097+0000 7feab6bd8700 1 --2- 172.21.15.52:0/894869983 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7feab0100170 0x7feab0198340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:50.098 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:50.097+0000 7feab6bd8700 1 --2- 172.21.15.52:0/894869983 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7feab01036f0 0x7feab019f900 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:50.098 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:50.097+0000 7feab6bd8700 1 --2- 172.21.15.52:0/894869983 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7feab0100b10 0x7feab0198880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:50.098 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:50.097+0000 7feab6bd8700 1 -- 172.21.15.52:0/894869983 >> 172.21.15.52:0/894869983 conn(0x7feab00fbd10 msgr2=0x7feab0106790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:30:50.098 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:50.098+0000 7feab6bd8700 1 -- 172.21.15.52:0/894869983 shutdown_connections 2024-03-04T09:30:50.098 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:50.098+0000 7feab6bd8700 1 -- 172.21.15.52:0/894869983 wait complete. 2024-03-04T09:30:50.467 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:30:50 smithi052 ceph-mon[32473]: from='client.? 172.21.15.52:0/894869983' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-03-04T09:30:50.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:30:50 smithi191 ceph-mon[36816]: from='client.? 172.21.15.52:0/894869983' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-03-04T09:30:50.631 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:30:50 smithi106 ceph-mon[36484]: from='client.? 172.21.15.52:0/894869983' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-03-04T09:30:50.798 INFO:teuthology.run_tasks:Running task fs.ready... 2024-03-04T09:30:50.809 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-04T09:30:50.918 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:50.920+0000 7fbc704c2640 1 -- 172.21.15.52:0/2714031498 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbc680ffca0 msgr2=0x7fbc681000c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:30:50.918 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:50.920+0000 7fbc704c2640 1 --2- 172.21.15.52:0/2714031498 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbc680ffca0 0x7fbc681000c0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fbc64009900 tx=0x7fbc6402bc30 comp rx=0 tx=0).stop 2024-03-04T09:30:50.918 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:50.920+0000 7fbc704c2640 1 -- 172.21.15.52:0/2714031498 shutdown_connections 2024-03-04T09:30:50.918 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:50.920+0000 7fbc704c2640 1 --2- 172.21.15.52:0/2714031498 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbc68101850 0x7fbc681080e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:50.918 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:50.920+0000 7fbc704c2640 1 --2- 172.21.15.52:0/2714031498 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbc68100e90 0x7fbc68101310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:50.918 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:50.920+0000 7fbc704c2640 1 --2- 172.21.15.52:0/2714031498 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbc680ffca0 0x7fbc681000c0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:50.918 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:50.920+0000 7fbc704c2640 1 -- 172.21.15.52:0/2714031498 >> 172.21.15.52:0/2714031498 conn(0x7fbc680fb210 msgr2=0x7fbc680fd650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:30:50.918 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:50.921+0000 7fbc704c2640 1 -- 172.21.15.52:0/2714031498 shutdown_connections 2024-03-04T09:30:50.918 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:50.921+0000 7fbc704c2640 1 -- 172.21.15.52:0/2714031498 wait complete. 2024-03-04T09:30:50.921 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:50.921+0000 7fbc704c2640 1 Processor -- start 2024-03-04T09:30:50.921 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:50.921+0000 7fbc704c2640 1 -- start start 2024-03-04T09:30:50.921 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:50.921+0000 7fbc704c2640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbc680ffca0 0x7fbc68109670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:30:50.921 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:50.922+0000 7fbc704c2640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbc68100e90 0x7fbc6819c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:30:50.921 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:50.922+0000 7fbc704c2640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbc68101850 0x7fbc6819eca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:30:50.921 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:50.922+0000 7fbc704c2640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc6810c1b0 con 0x7fbc680ffca0 2024-03-04T09:30:50.921 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:50.922+0000 7fbc704c2640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc6810c030 con 0x7fbc68100e90 2024-03-04T09:30:50.921 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:50.922+0000 7fbc704c2640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc6810c330 con 0x7fbc68101850 2024-03-04T09:30:50.921 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:50.922+0000 7fbc6e25e640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbc680ffca0 0x7fbc68109670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:30:50.922 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:50.922+0000 7fbc6e25e640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbc680ffca0 0x7fbc68109670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:58066/0 (socket says 172.21.15.52:58066) 2024-03-04T09:30:50.922 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:50.922+0000 7fbc6e25e640 1 -- 172.21.15.52:0/4113041432 learned_addr learned my addr 172.21.15.52:0/4113041432 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-04T09:30:50.922 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:50.922+0000 7fbc6da5d640 1 --2- 172.21.15.52:0/4113041432 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbc68100e90 0x7fbc6819c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:30:50.922 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:50.922+0000 7fbc6ea5f640 1 --2- 172.21.15.52:0/4113041432 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbc68101850 0x7fbc6819eca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:30:50.922 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:50.922+0000 7fbc6e25e640 1 -- 172.21.15.52:0/4113041432 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbc68101850 msgr2=0x7fbc6819eca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:30:50.922 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:50.922+0000 7fbc6e25e640 1 --2- 172.21.15.52:0/4113041432 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbc68101850 0x7fbc6819eca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:50.922 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:50.922+0000 7fbc6e25e640 1 -- 172.21.15.52:0/4113041432 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbc68100e90 msgr2=0x7fbc6819c630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:30:50.922 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:50.922+0000 7fbc6e25e640 1 --2- 172.21.15.52:0/4113041432 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbc68100e90 0x7fbc6819c630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:50.922 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:50.922+0000 7fbc6e25e640 1 -- 172.21.15.52:0/4113041432 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc6819f560 con 0x7fbc680ffca0 2024-03-04T09:30:50.922 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:50.922+0000 7fbc6da5d640 1 --2- 172.21.15.52:0/4113041432 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbc68100e90 0x7fbc6819c630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-04T09:30:50.923 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:50.922+0000 7fbc6e25e640 1 --2- 172.21.15.52:0/4113041432 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbc680ffca0 0x7fbc68109670 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fbc64000c00 tx=0x7fbc6402c640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:30:50.923 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:50.922+0000 7fbc5f7fe640 1 -- 172.21.15.52:0/4113041432 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc64041070 con 0x7fbc680ffca0 2024-03-04T09:30:50.923 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:50.922+0000 7fbc704c2640 1 -- 172.21.15.52:0/4113041432 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc6819f7f0 con 0x7fbc680ffca0 2024-03-04T09:30:50.923 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:50.922+0000 7fbc5f7fe640 1 -- 172.21.15.52:0/4113041432 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fbc64004450 con 0x7fbc680ffca0 2024-03-04T09:30:50.923 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:50.922+0000 7fbc5f7fe640 1 -- 172.21.15.52:0/4113041432 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc64034440 con 0x7fbc680ffca0 2024-03-04T09:30:50.923 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:50.922+0000 7fbc704c2640 1 -- 172.21.15.52:0/4113041432 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc6819fce0 con 0x7fbc680ffca0 2024-03-04T09:30:50.923 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:50.923+0000 7fbc704c2640 1 -- 172.21.15.52:0/4113041432 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc68106fc0 con 0x7fbc680ffca0 2024-03-04T09:30:50.924 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:50.925+0000 7fbc5f7fe640 1 -- 172.21.15.52:0/4113041432 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) v1 ==== 136756+0+0 (secure 0 0 0) 0x7fbc64044070 con 0x7fbc680ffca0 2024-03-04T09:30:50.924 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:50.925+0000 7fbc5f7fe640 1 --2- 172.21.15.52:0/4113041432 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7fbc540a1d80 0x7fbc540a4240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:30:50.924 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:50.925+0000 7fbc6da5d640 1 --2- 172.21.15.52:0/4113041432 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7fbc540a1d80 0x7fbc540a4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:30:50.924 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:50.925+0000 7fbc5f7fe640 1 -- 172.21.15.52:0/4113041432 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9158+0+0 (secure 0 0 0) 0x7fbc640efe10 con 0x7fbc680ffca0 2024-03-04T09:30:50.924 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:50.925+0000 7fbc6da5d640 1 --2- 172.21.15.52:0/4113041432 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7fbc540a1d80 0x7fbc540a4240 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fbc681093e0 tx=0x7fbc58005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:30:50.925 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:50.929+0000 7fbc5f7fe640 1 -- 172.21.15.52:0/4113041432 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fbc640bc8e0 con 0x7fbc680ffca0 2024-03-04T09:30:51.108 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:51.112+0000 7fbc704c2640 1 -- 172.21.15.52:0/4113041432 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbc6802d1c0 con 0x7fbc680ffca0 2024-03-04T09:30:51.110 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:51.113+0000 7fbc5f7fe640 1 -- 172.21.15.52:0/4113041432 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v93) v1 ==== 74+0+24804 (secure 0 0 0) 0x7fbc640c0980 con 0x7fbc680ffca0 2024-03-04T09:30:51.110 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-03-04T09:30:51.111 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":93,"fsid":"4e059f08-da08-11ee-95c6-87774f69a715","created":"2024-03-04T09:20:46.104963+0000","modified":"2024-03-04T09:30:41.237792+0000","last_up_change":"2024-03-04T09:27:49.261365+0000","last_in_change":"2024-03-04T09:27:30.024964+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-04T09:24:01.742461+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-04T09:28:35.835600+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-04T09:28:36.453099+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-04T09:28:40.168831+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"583172be-602f-4bf4-bfcb-42663cf5b277","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6802","nonce":1371221397},{"type":"v1","addr":"172.21.15.52:6803","nonce":1371221397}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6804","nonce":1371221397},{"type":"v1","addr":"172.21.15.52:6805","nonce":1371221397}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6808","nonce":1371221397},{"type":"v1","addr":"172.21.15.52:6809","nonce":1371221397}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6806","nonce":1371221397},{"type":"v1","addr":"172.21.15.52:6807","nonce":1371221397}]},"public_addr":"172.21.15.52:6803/1371221397","cluster_addr":"172.21.15.52:6805/1371221397","heartbeat_back_addr":"172.21.15.52:6809/1371221397","heartbeat_front_addr":"172.21.15.52:6807/1371221397","state":["exists","up"]},{"osd":1,"uuid":"5ae0081c-a2f6-46eb-ace0-0368208377b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":2540865537},{"type":"v1","addr":"172.21.15.106:6801","nonce":2540865537}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":2540865537},{"type":"v1","addr":"172.21.15.106:6803","nonce":2540865537}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":2540865537},{"type":"v1","addr":"172.21.15.106:6807","nonce":2540865537}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":2540865537},{"type":"v1","addr":"172.21.15.106:6805","nonce":2540865537}]},"public_addr":"172.21.15.106:6801/2540865537","cluster_addr":"172.21.15.106:6803/2540865537","heartbeat_back_addr":"172.21.15.106:6807/2540865537","heartbeat_front_addr":"172.21.15.106:6805/2540865537","state":["exists","up"]},{"osd":2,"uuid":"1948bac7-2bb2-4218-8b93-4e4743a0de04","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":1377083164},{"type":"v1","addr":"172.21.15.191:6801","nonce":1377083164}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":1377083164},{"type":"v1","addr":"172.21.15.191:6803","nonce":1377083164}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":1377083164},{"type":"v1","addr":"172.21.15.191:6807","nonce":1377083164}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":1377083164},{"type":"v1","addr":"172.21.15.191:6805","nonce":1377083164}]},"public_addr":"172.21.15.191:6801/1377083164","cluster_addr":"172.21.15.191:6803/1377083164","heartbeat_back_addr":"172.21.15.191:6807/1377083164","heartbeat_front_addr":"172.21.15.191:6805/1377083164","state":["exists","up"]},{"osd":3,"uuid":"16c9e9fe-7167-4992-bf12-1a7398e0498a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6810","nonce":2388571810},{"type":"v1","addr":"172.21.15.52:6811","nonce":2388571810}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6812","nonce":2388571810},{"type":"v1","addr":"172.21.15.52:6813","nonce":2388571810}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6816","nonce":2388571810},{"type":"v1","addr":"172.21.15.52:6817","nonce":2388571810}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6814","nonce":2388571810},{"type":"v1","addr":"172.21.15.52:6815","nonce":2388571810}]},"public_addr":"172.21.15.52:6811/2388571810","cluster_addr":"172.21.15.52:6813/2388571810","heartbeat_back_addr":"172.21.15.52:6817/2388571810","heartbeat_front_addr":"172.21.15.52:6815/2388571810","state":["exists","up"]},{"osd":4,"uuid":"c8ef48c7-7c78-44d3-a9e5-99ec750abd3d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6808","nonce":3465868015},{"type":"v1","addr":"172.21.15.106:6809","nonce":3465868015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6810","nonce":3465868015},{"type":"v1","addr":"172.21.15.106:6811","nonce":3465868015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6814","nonce":3465868015},{"type":"v1","addr":"172.21.15.106:6815","nonce":3465868015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6812","nonce":3465868015},{"type":"v1","addr":"172.21.15.106:6813","nonce":3465868015}]},"public_addr":"172.21.15.106:6809/3465868015","cluster_addr":"172.21.15.106:6811/3465868015","heartbeat_back_addr":"172.21.15.106:6815/3465868015","heartbeat_front_addr":"172.21.15.106:6813/3465868015","state":["exists","up"]},{"osd":5,"uuid":"9e41824a-c5a2-4f88-be50-2ba1dceb3220","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":1485092643},{"type":"v1","addr":"172.21.15.191:6809","nonce":1485092643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":1485092643},{"type":"v1","addr":"172.21.15.191:6811","nonce":1485092643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":1485092643},{"type":"v1","addr":"172.21.15.191:6815","nonce":1485092643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":1485092643},{"type":"v1","addr":"172.21.15.191:6813","nonce":1485092643}]},"public_addr":"172.21.15.191:6809/1485092643","cluster_addr":"172.21.15.191:6811/1485092643","heartbeat_back_addr":"172.21.15.191:6815/1485092643","heartbeat_front_addr":"172.21.15.191:6813/1485092643","state":["exists","up"]},{"osd":6,"uuid":"b35dccee-ec16-452e-ac69-3ac2fb7b27fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6818","nonce":1989797856},{"type":"v1","addr":"172.21.15.52:6819","nonce":1989797856}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6820","nonce":1989797856},{"type":"v1","addr":"172.21.15.52:6821","nonce":1989797856}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6824","nonce":1989797856},{"type":"v1","addr":"172.21.15.52:6825","nonce":1989797856}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6822","nonce":1989797856},{"type":"v1","addr":"172.21.15.52:6823","nonce":1989797856}]},"public_addr":"172.21.15.52:6819/1989797856","cluster_addr":"172.21.15.52:6821/1989797856","heartbeat_back_addr":"172.21.15.52:6825/1989797856","heartbeat_front_addr":"172.21.15.52:6823/1989797856","state":["exists","up"]},{"osd":7,"uuid":"71303e3d-14ec-4cbd-b19c-67645a1de3c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6816","nonce":1450870329},{"type":"v1","addr":"172.21.15.106:6817","nonce":1450870329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":1450870329},{"type":"v1","addr":"172.21.15.106:6819","nonce":1450870329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":1450870329},{"type":"v1","addr":"172.21.15.106:6823","nonce":1450870329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6820","nonce":1450870329},{"type":"v1","addr":"172.21.15.106:6821","nonce":1450870329}]},"public_addr":"172.21.15.106:6817/1450870329","cluster_addr":"172.21.15.106:6819/1450870329","heartbeat_back_addr":"172.21.15.106:6823/1450870329","heartbeat_front_addr":"172.21.15.106:6821/1450870329","state":["exists","up"]},{"osd":8,"uuid":"6f5786c4-4402-435a-bdad-adff46fe9473","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":1114119072},{"type":"v1","addr":"172.21.15.191:6817","nonce":1114119072}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6818","nonce":1114119072},{"type":"v1","addr":"172.21.15.191:6819","nonce":1114119072}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":1114119072},{"type":"v1","addr":"172.21.15.191:6823","nonce":1114119072}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":1114119072},{"type":"v1","addr":"172.21.15.191:6821","nonce":1114119072}]},"public_addr":"172.21.15.191:6817/1114119072","cluster_addr":"172.21.15.191:6819/1114119072","heartbeat_back_addr":"172.21.15.191:6823/1114119072","heartbeat_front_addr":"172.21.15.191:6821/1114119072","state":["exists","up"]},{"osd":9,"uuid":"062bbaa5-1227-4684-b859-450f5594a90c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6826","nonce":3485821914},{"type":"v1","addr":"172.21.15.52:6827","nonce":3485821914}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6828","nonce":3485821914},{"type":"v1","addr":"172.21.15.52:6829","nonce":3485821914}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6832","nonce":3485821914},{"type":"v1","addr":"172.21.15.52:6833","nonce":3485821914}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6830","nonce":3485821914},{"type":"v1","addr":"172.21.15.52:6831","nonce":3485821914}]},"public_addr":"172.21.15.52:6827/3485821914","cluster_addr":"172.21.15.52:6829/3485821914","heartbeat_back_addr":"172.21.15.52:6833/3485821914","heartbeat_front_addr":"172.21.15.52:6831/3485821914","state":["exists","up"]},{"osd":10,"uuid":"1a285a5a-f793-48da-a8eb-5326d9528cd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":3815586576},{"type":"v1","addr":"172.21.15.106:6825","nonce":3815586576}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6826","nonce":3815586576},{"type":"v1","addr":"172.21.15.106:6827","nonce":3815586576}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":3815586576},{"type":"v1","addr":"172.21.15.106:6831","nonce":3815586576}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":3815586576},{"type":"v1","addr":"172.21.15.106:6829","nonce":3815586576}]},"public_addr":"172.21.15.106:6825/3815586576","cluster_addr":"172.21.15.106:6827/3815586576","heartbeat_back_addr":"172.21.15.106:6831/3815586576","heartbeat_front_addr":"172.21.15.106:6829/3815586576","state":["exists","up"]},{"osd":11,"uuid":"1f8b9a43-57fa-41f4-9969-3b803a74bf45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":2009632843},{"type":"v1","addr":"172.21.15.191:6825","nonce":2009632843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":2009632843},{"type":"v1","addr":"172.21.15.191:6827","nonce":2009632843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":2009632843},{"type":"v1","addr":"172.21.15.191:6831","nonce":2009632843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":2009632843},{"type":"v1","addr":"172.21.15.191:6829","nonce":2009632843}]},"public_addr":"172.21.15.191:6825/2009632843","cluster_addr":"172.21.15.191:6827/2009632843","heartbeat_back_addr":"172.21.15.191:6831/2009632843","heartbeat_front_addr":"172.21.15.191:6829/2009632843","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T09:23:10.894763+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T09:23:34.420055+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T09:23:58.288324+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T09:24:23.673457+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T09:24:48.208561+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T09:25:13.405473+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T09:25:38.670915+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T09:26:04.451257+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T09:26:30.281380+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T09:26:56.543143+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T09:27:22.412930+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T09:27:48.003866+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.1d","mappings":[{"from":2,"to":1},{"from":8,"to":11}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.20","mappings":[{"from":5,"to":9}]},{"pgid":"3.21","mappings":[{"from":4,"to":6}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]},{"pgid":"4.9","mappings":[{"from":5,"to":8}]},{"pgid":"4.17","mappings":[{"from":9,"to":8}]},{"pgid":"4.1b","mappings":[{"from":9,"to":10}]},{"pgid":"4.1f","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.52:0/4082645718":"2024-03-05T09:21:39.401935+0000","172.21.15.52:0/1271949988":"2024-03-05T09:21:39.401935+0000","172.21.15.52:6801/2379361333":"2024-03-05T09:21:12.760938+0000","172.21.15.52:0/497075075":"2024-03-05T09:21:12.760938+0000","172.21.15.52:6800/2379361333":"2024-03-05T09:21:12.760938+0000","172.21.15.52:0/568242577":"2024-03-05T09:21:12.760938+0000","172.21.15.52:6801/2425730636":"2024-03-05T09:21:39.401935+0000","172.21.15.52:6800/2425730636":"2024-03-05T09:21:39.401935+0000","172.21.15.52:0/2743084243":"2024-03-05T09:21:39.401935+0000","172.21.15.52:0/1035878034":"2024-03-05T09:21:12.760938+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-04T09:30:51.112 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:51.115+0000 7fbc704c2640 1 -- 172.21.15.52:0/4113041432 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7fbc540a1d80 msgr2=0x7fbc540a4240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:30:51.112 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:51.115+0000 7fbc704c2640 1 --2- 172.21.15.52:0/4113041432 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7fbc540a1d80 0x7fbc540a4240 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fbc681093e0 tx=0x7fbc58005f90 comp rx=0 tx=0).stop 2024-03-04T09:30:51.112 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:51.115+0000 7fbc704c2640 1 -- 172.21.15.52:0/4113041432 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbc680ffca0 msgr2=0x7fbc68109670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:30:51.113 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:51.115+0000 7fbc704c2640 1 --2- 172.21.15.52:0/4113041432 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbc680ffca0 0x7fbc68109670 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fbc64000c00 tx=0x7fbc6402c640 comp rx=0 tx=0).stop 2024-03-04T09:30:51.113 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:51.115+0000 7fbc704c2640 1 -- 172.21.15.52:0/4113041432 shutdown_connections 2024-03-04T09:30:51.113 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:51.115+0000 7fbc704c2640 1 --2- 172.21.15.52:0/4113041432 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7fbc540a1d80 0x7fbc540a4240 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:51.113 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:51.115+0000 7fbc704c2640 1 --2- 172.21.15.52:0/4113041432 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbc68101850 0x7fbc6819eca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:51.113 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:51.115+0000 7fbc704c2640 1 --2- 172.21.15.52:0/4113041432 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbc68100e90 0x7fbc6819c630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:51.113 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:51.115+0000 7fbc704c2640 1 --2- 172.21.15.52:0/4113041432 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbc680ffca0 0x7fbc68109670 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:51.113 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:51.115+0000 7fbc704c2640 1 -- 172.21.15.52:0/4113041432 >> 172.21.15.52:0/4113041432 conn(0x7fbc680fb210 msgr2=0x7fbc68104190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:30:51.113 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:51.115+0000 7fbc704c2640 1 -- 172.21.15.52:0/4113041432 shutdown_connections 2024-03-04T09:30:51.113 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:51.115+0000 7fbc704c2640 1 -- 172.21.15.52:0/4113041432 wait complete. 2024-03-04T09:30:51.124 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-04T09:24:01.742461+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-04T09:28:35.835600+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-04T09:28:36.453099+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-04T09:28:40.168831+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '89', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-04T09:30:51.124 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-04T09:30:51.238 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:51.238+0000 7f15ee3bc640 1 -- 172.21.15.52:0/293334867 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f15e81015e0 msgr2=0x7f15e8101a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:30:51.238 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:51.238+0000 7f15ee3bc640 1 --2- 172.21.15.52:0/293334867 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f15e81015e0 0x7f15e8101a00 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f15d8009860 tx=0x7f15d802bb70 comp rx=0 tx=0).stop 2024-03-04T09:30:51.238 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:51.238+0000 7f15ee3bc640 1 -- 172.21.15.52:0/293334867 shutdown_connections 2024-03-04T09:30:51.238 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:51.238+0000 7f15ee3bc640 1 --2- 172.21.15.52:0/293334867 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f15e8103190 0x7f15e8109a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:51.238 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:51.238+0000 7f15ee3bc640 1 --2- 172.21.15.52:0/293334867 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f15e81027d0 0x7f15e8102c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:51.238 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:51.238+0000 7f15ee3bc640 1 --2- 172.21.15.52:0/293334867 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f15e81015e0 0x7f15e8101a00 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:51.238 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:51.238+0000 7f15ee3bc640 1 -- 172.21.15.52:0/293334867 >> 172.21.15.52:0/293334867 conn(0x7f15e80fcb70 msgr2=0x7f15e80fef90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:30:51.238 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:51.238+0000 7f15ee3bc640 1 -- 172.21.15.52:0/293334867 shutdown_connections 2024-03-04T09:30:51.238 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:51.238+0000 7f15ee3bc640 1 -- 172.21.15.52:0/293334867 wait complete. 2024-03-04T09:30:51.239 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:51.239+0000 7f15ee3bc640 1 Processor -- start 2024-03-04T09:30:51.239 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:51.239+0000 7f15ee3bc640 1 -- start start 2024-03-04T09:30:51.239 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:51.239+0000 7f15ee3bc640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f15e81015e0 0x7f15e8197920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:30:51.239 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:51.239+0000 7f15ee3bc640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f15e81027d0 0x7f15e8197e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:30:51.239 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:51.239+0000 7f15ee3bc640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f15e8103190 0x7f15e819eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:30:51.239 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:51.239+0000 7f15ee3bc640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15e810d230 con 0x7f15e81027d0 2024-03-04T09:30:51.239 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:51.239+0000 7f15ee3bc640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15e810d0b0 con 0x7f15e8103190 2024-03-04T09:30:51.239 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:51.239+0000 7f15ee3bc640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15e810d3b0 con 0x7f15e81015e0 2024-03-04T09:30:51.239 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:51.239+0000 7f15e77fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f15e81027d0 0x7f15e8197e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:30:51.239 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:51.239+0000 7f15e77fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f15e81027d0 0x7f15e8197e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:58086/0 (socket says 172.21.15.52:58086) 2024-03-04T09:30:51.239 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:51.239+0000 7f15e77fe640 1 -- 172.21.15.52:0/337026709 learned_addr learned my addr 172.21.15.52:0/337026709 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-04T09:30:51.239 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:51.239+0000 7f15e7fff640 1 --2- 172.21.15.52:0/337026709 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f15e81015e0 0x7f15e8197920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:30:51.239 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:51.239+0000 7f15ec959640 1 --2- 172.21.15.52:0/337026709 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f15e8103190 0x7f15e819eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:30:51.239 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:51.240+0000 7f15e7fff640 1 -- 172.21.15.52:0/337026709 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f15e8103190 msgr2=0x7f15e819eed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:30:51.240 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:51.240+0000 7f15e7fff640 1 --2- 172.21.15.52:0/337026709 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f15e8103190 0x7f15e819eed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:51.240 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:51.240+0000 7f15e7fff640 1 -- 172.21.15.52:0/337026709 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f15e81027d0 msgr2=0x7f15e8197e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:30:51.240 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:51.240+0000 7f15e7fff640 1 --2- 172.21.15.52:0/337026709 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f15e81027d0 0x7f15e8197e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:51.240 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:51.240+0000 7f15e7fff640 1 -- 172.21.15.52:0/337026709 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15e819f540 con 0x7f15e81015e0 2024-03-04T09:30:51.240 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:51.240+0000 7f15e77fe640 1 --2- 172.21.15.52:0/337026709 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f15e81027d0 0x7f15e8197e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-04T09:30:51.240 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:51.240+0000 7f15ec959640 1 --2- 172.21.15.52:0/337026709 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f15e8103190 0x7f15e819eed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-04T09:30:51.240 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:51.240+0000 7f15e7fff640 1 --2- 172.21.15.52:0/337026709 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f15e81015e0 0x7f15e8197920 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f15d802c0b0 tx=0x7f15d8032430 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:30:51.240 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:51.240+0000 7f15e57fa640 1 -- 172.21.15.52:0/337026709 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15d8041020 con 0x7f15e81015e0 2024-03-04T09:30:51.240 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:51.240+0000 7f15ee3bc640 1 -- 172.21.15.52:0/337026709 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15e819f7d0 con 0x7f15e81015e0 2024-03-04T09:30:51.240 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:51.240+0000 7f15e57fa640 1 -- 172.21.15.52:0/337026709 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f15d802c9f0 con 0x7f15e81015e0 2024-03-04T09:30:51.240 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:51.240+0000 7f15ee3bc640 1 -- 172.21.15.52:0/337026709 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15e819fd10 con 0x7f15e81015e0 2024-03-04T09:30:51.240 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:51.240+0000 7f15e57fa640 1 -- 172.21.15.52:0/337026709 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15d8039040 con 0x7f15e81015e0 2024-03-04T09:30:51.240 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:51.241+0000 7f15ee3bc640 1 -- 172.21.15.52:0/337026709 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f15e8108a20 con 0x7f15e81015e0 2024-03-04T09:30:51.241 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:51.242+0000 7f15e57fa640 1 -- 172.21.15.52:0/337026709 <== mon.1 v2:172.21.15.191:3300/0 4 ==== mgrmap(e 18) v1 ==== 136756+0+0 (secure 0 0 0) 0x7f15d8044020 con 0x7f15e81015e0 2024-03-04T09:30:51.241 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:51.243+0000 7f15e57fa640 1 --2- 172.21.15.52:0/337026709 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f15d00a1d80 0x7f15d00a4240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:30:51.241 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:51.243+0000 7f15e77fe640 1 --2- 172.21.15.52:0/337026709 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f15d00a1d80 0x7f15d00a4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:30:51.241 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:51.243+0000 7f15e57fa640 1 -- 172.21.15.52:0/337026709 <== mon.1 v2:172.21.15.191:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9158+0+0 (secure 0 0 0) 0x7f15d80ef9d0 con 0x7f15e81015e0 2024-03-04T09:30:51.242 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:51.243+0000 7f15e77fe640 1 --2- 172.21.15.52:0/337026709 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f15d00a1d80 0x7f15d00a4240 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f15e8198e30 tx=0x7f15dc005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:30:51.243 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:51.246+0000 7f15e57fa640 1 -- 172.21.15.52:0/337026709 <== mon.1 v2:172.21.15.191:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f15d804b020 con 0x7f15e81015e0 2024-03-04T09:30:51.427 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:51.431+0000 7f15ee3bc640 1 -- 172.21.15.52:0/337026709 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f15e80621d0 con 0x7f15e81015e0 2024-03-04T09:30:51.428 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:51.431+0000 7f15e57fa640 1 -- 172.21.15.52:0/337026709 <== mon.1 v2:172.21.15.191:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v93) v1 ==== 93+0+10 (secure 0 0 0) 0x7f15d80bca00 con 0x7f15e81015e0 2024-03-04T09:30:51.428 INFO:teuthology.orchestra.run.smithi052.stdout:pg_num: 1 2024-03-04T09:30:51.430 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:51.433+0000 7f15ee3bc640 1 -- 172.21.15.52:0/337026709 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f15d00a1d80 msgr2=0x7f15d00a4240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:30:51.430 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:51.433+0000 7f15ee3bc640 1 --2- 172.21.15.52:0/337026709 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f15d00a1d80 0x7f15d00a4240 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f15e8198e30 tx=0x7f15dc005f90 comp rx=0 tx=0).stop 2024-03-04T09:30:51.430 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:51.433+0000 7f15ee3bc640 1 -- 172.21.15.52:0/337026709 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f15e81015e0 msgr2=0x7f15e8197920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:30:51.430 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:51.433+0000 7f15ee3bc640 1 --2- 172.21.15.52:0/337026709 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f15e81015e0 0x7f15e8197920 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f15d802c0b0 tx=0x7f15d8032430 comp rx=0 tx=0).stop 2024-03-04T09:30:51.430 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:51.433+0000 7f15ee3bc640 1 -- 172.21.15.52:0/337026709 shutdown_connections 2024-03-04T09:30:51.431 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:51.433+0000 7f15ee3bc640 1 --2- 172.21.15.52:0/337026709 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f15d00a1d80 0x7f15d00a4240 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:51.431 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:51.433+0000 7f15ee3bc640 1 --2- 172.21.15.52:0/337026709 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f15e8103190 0x7f15e819eed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:51.431 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:51.433+0000 7f15ee3bc640 1 --2- 172.21.15.52:0/337026709 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f15e81027d0 0x7f15e8197e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:51.431 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:51.433+0000 7f15ee3bc640 1 --2- 172.21.15.52:0/337026709 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f15e81015e0 0x7f15e8197920 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:51.431 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:51.433+0000 7f15ee3bc640 1 -- 172.21.15.52:0/337026709 >> 172.21.15.52:0/337026709 conn(0x7f15e80fcb70 msgr2=0x7f15e8105bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:30:51.431 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:51.433+0000 7f15ee3bc640 1 -- 172.21.15.52:0/337026709 shutdown_connections 2024-03-04T09:30:51.431 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:51.433+0000 7f15ee3bc640 1 -- 172.21.15.52:0/337026709 wait complete. 2024-03-04T09:30:51.440 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:30:51 smithi052 ceph-mon[32473]: pgmap v358: 129 pgs: 129 active+clean; 641 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s; 2 B/s, 1 keys/s, 0 objects/s recovering 2024-03-04T09:30:51.440 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:30:51 smithi052 ceph-mon[32473]: from='client.? 172.21.15.52:0/4113041432' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-04T09:30:51.441 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-04T09:30:51.550 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:51.550+0000 7fa4b720e640 1 -- 172.21.15.52:0/3239294741 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa4b01016d0 msgr2=0x7fa4b0101b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:30:51.550 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:51.550+0000 7fa4b720e640 1 --2- 172.21.15.52:0/3239294741 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa4b01016d0 0x7fa4b0101b50 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fa4a4009900 tx=0x7fa4a402bc30 comp rx=0 tx=0).stop 2024-03-04T09:30:51.550 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:51.550+0000 7fa4b720e640 1 -- 172.21.15.52:0/3239294741 shutdown_connections 2024-03-04T09:30:51.550 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:51.550+0000 7fa4b720e640 1 --2- 172.21.15.52:0/3239294741 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa4b0102090 0x7fa4b0108920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:51.550 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:51.550+0000 7fa4b720e640 1 --2- 172.21.15.52:0/3239294741 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa4b01016d0 0x7fa4b0101b50 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:51.550 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:51.550+0000 7fa4b720e640 1 --2- 172.21.15.52:0/3239294741 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa4b01004e0 0x7fa4b0100900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:51.550 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:51.550+0000 7fa4b720e640 1 -- 172.21.15.52:0/3239294741 >> 172.21.15.52:0/3239294741 conn(0x7fa4b00fba70 msgr2=0x7fa4b00fde90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:30:51.550 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:51.550+0000 7fa4b720e640 1 -- 172.21.15.52:0/3239294741 shutdown_connections 2024-03-04T09:30:51.551 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:51.550+0000 7fa4b720e640 1 -- 172.21.15.52:0/3239294741 wait complete. 2024-03-04T09:30:51.551 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:51.551+0000 7fa4b720e640 1 Processor -- start 2024-03-04T09:30:51.551 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:51.551+0000 7fa4b720e640 1 -- start start 2024-03-04T09:30:51.551 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:51.551+0000 7fa4b720e640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa4b01004e0 0x7fa4b0104470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:30:51.551 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:51.551+0000 7fa4b720e640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa4b01016d0 0x7fa4b0102ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:30:51.551 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:51.551+0000 7fa4b720e640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa4b0102090 0x7fa4b0102ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:30:51.551 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:51.551+0000 7fa4b720e640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4b010c170 con 0x7fa4b01016d0 2024-03-04T09:30:51.551 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:51.551+0000 7fa4b720e640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4b010bff0 con 0x7fa4b01004e0 2024-03-04T09:30:51.551 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:51.551+0000 7fa4b720e640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4b010c2f0 con 0x7fa4b0102090 2024-03-04T09:30:51.551 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:51.551+0000 7fa4affff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa4b01016d0 0x7fa4b0102ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:30:51.551 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:51.551+0000 7fa4affff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa4b01016d0 0x7fa4b0102ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:58110/0 (socket says 172.21.15.52:58110) 2024-03-04T09:30:51.551 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:51.551+0000 7fa4affff640 1 -- 172.21.15.52:0/362375957 learned_addr learned my addr 172.21.15.52:0/362375957 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-04T09:30:51.551 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:51.551+0000 7fa4b4faa640 1 --2- 172.21.15.52:0/362375957 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa4b01004e0 0x7fa4b0104470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:30:51.551 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:51.551+0000 7fa4b57ab640 1 --2- 172.21.15.52:0/362375957 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa4b0102090 0x7fa4b0102ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:30:51.552 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:51.551+0000 7fa4affff640 1 -- 172.21.15.52:0/362375957 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa4b0102090 msgr2=0x7fa4b0102ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:30:51.552 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:51.551+0000 7fa4affff640 1 --2- 172.21.15.52:0/362375957 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa4b0102090 0x7fa4b0102ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:51.552 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:51.551+0000 7fa4affff640 1 -- 172.21.15.52:0/362375957 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa4b01004e0 msgr2=0x7fa4b0104470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:30:51.552 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:51.551+0000 7fa4affff640 1 --2- 172.21.15.52:0/362375957 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa4b01004e0 0x7fa4b0104470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:51.552 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:51.551+0000 7fa4affff640 1 -- 172.21.15.52:0/362375957 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4b0103880 con 0x7fa4b01016d0 2024-03-04T09:30:51.552 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:51.552+0000 7fa4affff640 1 --2- 172.21.15.52:0/362375957 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa4b01016d0 0x7fa4b0102ab0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fa4a4009230 tx=0x7fa4a402c650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:30:51.552 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:51.552+0000 7fa4adffb640 1 -- 172.21.15.52:0/362375957 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4a4041070 con 0x7fa4b01016d0 2024-03-04T09:30:51.552 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:51.552+0000 7fa4adffb640 1 -- 172.21.15.52:0/362375957 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa4a4004450 con 0x7fa4b01016d0 2024-03-04T09:30:51.552 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:51.552+0000 7fa4b720e640 1 -- 172.21.15.52:0/362375957 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa4b01a1a00 con 0x7fa4b01016d0 2024-03-04T09:30:51.552 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:51.552+0000 7fa4adffb640 1 -- 172.21.15.52:0/362375957 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4a4003e20 con 0x7fa4b01016d0 2024-03-04T09:30:51.552 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:51.552+0000 7fa4b720e640 1 -- 172.21.15.52:0/362375957 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4b01a1fa0 con 0x7fa4b01016d0 2024-03-04T09:30:51.552 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:51.553+0000 7fa4b720e640 1 -- 172.21.15.52:0/362375957 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa4b01a2170 con 0x7fa4b01016d0 2024-03-04T09:30:51.553 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:51.554+0000 7fa4adffb640 1 -- 172.21.15.52:0/362375957 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) v1 ==== 136756+0+0 (secure 0 0 0) 0x7fa4a4044070 con 0x7fa4b01016d0 2024-03-04T09:30:51.554 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:51.555+0000 7fa4adffb640 1 --2- 172.21.15.52:0/362375957 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7fa4980a1e00 0x7fa4980a42c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:30:51.554 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:51.555+0000 7fa4b4faa640 1 --2- 172.21.15.52:0/362375957 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7fa4980a1e00 0x7fa4980a42c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:30:51.554 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:51.555+0000 7fa4adffb640 1 -- 172.21.15.52:0/362375957 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9158+0+0 (secure 0 0 0) 0x7fa4a40efca0 con 0x7fa4b01016d0 2024-03-04T09:30:51.554 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:51.555+0000 7fa4b4faa640 1 --2- 172.21.15.52:0/362375957 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7fa4980a1e00 0x7fa4980a42c0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fa4b0101530 tx=0x7fa4a8009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:30:51.556 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:51.559+0000 7fa4adffb640 1 -- 172.21.15.52:0/362375957 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fa4a40bc680 con 0x7fa4b01016d0 2024-03-04T09:30:51.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:30:51 smithi191 ceph-mon[36816]: pgmap v358: 129 pgs: 129 active+clean; 641 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s; 2 B/s, 1 keys/s, 0 objects/s recovering 2024-03-04T09:30:51.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:30:51 smithi191 ceph-mon[36816]: from='client.? 172.21.15.52:0/4113041432' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-04T09:30:51.631 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:30:51 smithi106 ceph-mon[36484]: pgmap v358: 129 pgs: 129 active+clean; 641 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s; 2 B/s, 1 keys/s, 0 objects/s recovering 2024-03-04T09:30:51.631 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:30:51 smithi106 ceph-mon[36484]: from='client.? 172.21.15.52:0/4113041432' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-04T09:30:51.736 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:51.739+0000 7fa4b720e640 1 -- 172.21.15.52:0/362375957 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fa4b01a2340 con 0x7fa4b01016d0 2024-03-04T09:30:51.737 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:51.740+0000 7fa4adffb640 1 -- 172.21.15.52:0/362375957 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v93) v1 ==== 104+0+11 (secure 0 0 0) 0x7fa4a40c0720 con 0x7fa4b01016d0 2024-03-04T09:30:51.737 INFO:teuthology.orchestra.run.smithi052.stdout:pg_num: 32 2024-03-04T09:30:51.739 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:51.742+0000 7fa4b720e640 1 -- 172.21.15.52:0/362375957 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7fa4980a1e00 msgr2=0x7fa4980a42c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:30:51.739 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:51.742+0000 7fa4b720e640 1 --2- 172.21.15.52:0/362375957 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7fa4980a1e00 0x7fa4980a42c0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fa4b0101530 tx=0x7fa4a8009500 comp rx=0 tx=0).stop 2024-03-04T09:30:51.739 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:51.742+0000 7fa4b720e640 1 -- 172.21.15.52:0/362375957 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa4b01016d0 msgr2=0x7fa4b0102ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:30:51.739 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:51.742+0000 7fa4b720e640 1 --2- 172.21.15.52:0/362375957 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa4b01016d0 0x7fa4b0102ab0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fa4a4009230 tx=0x7fa4a402c650 comp rx=0 tx=0).stop 2024-03-04T09:30:51.739 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:51.742+0000 7fa4b720e640 1 -- 172.21.15.52:0/362375957 shutdown_connections 2024-03-04T09:30:51.739 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:51.742+0000 7fa4b720e640 1 --2- 172.21.15.52:0/362375957 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7fa4980a1e00 0x7fa4980a42c0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:51.740 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:51.742+0000 7fa4b720e640 1 --2- 172.21.15.52:0/362375957 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa4b0102090 0x7fa4b0102ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:51.740 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:51.742+0000 7fa4b720e640 1 --2- 172.21.15.52:0/362375957 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa4b01016d0 0x7fa4b0102ab0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:51.740 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:51.742+0000 7fa4b720e640 1 --2- 172.21.15.52:0/362375957 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa4b01004e0 0x7fa4b0104470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:51.740 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:51.742+0000 7fa4b720e640 1 -- 172.21.15.52:0/362375957 >> 172.21.15.52:0/362375957 conn(0x7fa4b00fba70 msgr2=0x7fa4b00fd540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:30:51.740 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:51.742+0000 7fa4b720e640 1 -- 172.21.15.52:0/362375957 shutdown_connections 2024-03-04T09:30:51.740 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:51.742+0000 7fa4b720e640 1 -- 172.21.15.52:0/362375957 wait complete. 2024-03-04T09:30:51.750 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-04T09:30:51.857 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:51.858+0000 7f02e1415640 1 -- 172.21.15.52:0/2771656389 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f02dc102050 msgr2=0x7f02dc104440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:30:51.858 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:51.858+0000 7f02e1415640 1 --2- 172.21.15.52:0/2771656389 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f02dc102050 0x7f02dc104440 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f02d4009900 tx=0x7f02d402bc30 comp rx=0 tx=0).stop 2024-03-04T09:30:51.858 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:51.858+0000 7f02e1415640 1 -- 172.21.15.52:0/2771656389 shutdown_connections 2024-03-04T09:30:51.858 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:51.858+0000 7f02e1415640 1 --2- 172.21.15.52:0/2771656389 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f02dc1072b0 0x7f02dc109700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:51.858 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:51.858+0000 7f02e1415640 1 --2- 172.21.15.52:0/2771656389 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f02dc104980 0x7f02dc106d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:51.858 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:51.858+0000 7f02e1415640 1 --2- 172.21.15.52:0/2771656389 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f02dc102050 0x7f02dc104440 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:51.858 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:51.858+0000 7f02e1415640 1 -- 172.21.15.52:0/2771656389 >> 172.21.15.52:0/2771656389 conn(0x7f02dc0fba60 msgr2=0x7f02dc0fde80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:30:51.858 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:51.858+0000 7f02e1415640 1 -- 172.21.15.52:0/2771656389 shutdown_connections 2024-03-04T09:30:51.858 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:51.858+0000 7f02e1415640 1 -- 172.21.15.52:0/2771656389 wait complete. 2024-03-04T09:30:51.858 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:51.859+0000 7f02e1415640 1 Processor -- start 2024-03-04T09:30:51.858 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:51.859+0000 7f02e1415640 1 -- start start 2024-03-04T09:30:51.859 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:51.859+0000 7f02e1415640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f02dc102050 0x7f02dc199b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:30:51.859 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:51.859+0000 7f02e1415640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f02dc104980 0x7f02dc19a0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:30:51.859 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:51.859+0000 7f02e1415640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f02dc1072b0 0x7f02dc1a1130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:30:51.859 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:51.859+0000 7f02e1415640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02dc10d2b0 con 0x7f02dc102050 2024-03-04T09:30:51.859 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:51.859+0000 7f02e1415640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02dc10d130 con 0x7f02dc104980 2024-03-04T09:30:51.859 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:51.859+0000 7f02e1415640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02dc10d430 con 0x7f02dc1072b0 2024-03-04T09:30:51.859 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:51.859+0000 7f02daffd640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f02dc102050 0x7f02dc199b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:30:51.859 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:51.859+0000 7f02daffd640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f02dc102050 0x7f02dc199b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:58126/0 (socket says 172.21.15.52:58126) 2024-03-04T09:30:51.859 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:51.859+0000 7f02daffd640 1 -- 172.21.15.52:0/3296452024 learned_addr learned my addr 172.21.15.52:0/3296452024 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-04T09:30:51.859 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:51.859+0000 7f02db7fe640 1 --2- 172.21.15.52:0/3296452024 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f02dc1072b0 0x7f02dc1a1130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:30:51.859 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:51.859+0000 7f02da7fc640 1 --2- 172.21.15.52:0/3296452024 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f02dc104980 0x7f02dc19a0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:30:51.859 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:51.860+0000 7f02daffd640 1 -- 172.21.15.52:0/3296452024 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f02dc1072b0 msgr2=0x7f02dc1a1130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:30:51.860 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:51.860+0000 7f02daffd640 1 --2- 172.21.15.52:0/3296452024 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f02dc1072b0 0x7f02dc1a1130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:51.860 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:51.860+0000 7f02daffd640 1 -- 172.21.15.52:0/3296452024 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f02dc104980 msgr2=0x7f02dc19a0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:30:51.860 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:51.860+0000 7f02daffd640 1 --2- 172.21.15.52:0/3296452024 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f02dc104980 0x7f02dc19a0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:51.860 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:51.860+0000 7f02daffd640 1 -- 172.21.15.52:0/3296452024 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02dc1a17a0 con 0x7f02dc102050 2024-03-04T09:30:51.860 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:51.860+0000 7f02daffd640 1 --2- 172.21.15.52:0/3296452024 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f02dc102050 0x7f02dc199b80 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f02d4000c00 tx=0x7f02d402c650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:30:51.860 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:51.860+0000 7f02d3fff640 1 -- 172.21.15.52:0/3296452024 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02d4043020 con 0x7f02dc102050 2024-03-04T09:30:51.860 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:51.860+0000 7f02d3fff640 1 -- 172.21.15.52:0/3296452024 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f02d40054e0 con 0x7f02dc102050 2024-03-04T09:30:51.860 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:51.860+0000 7f02e1415640 1 -- 172.21.15.52:0/3296452024 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02dc1a1a30 con 0x7f02dc102050 2024-03-04T09:30:51.860 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:51.860+0000 7f02d3fff640 1 -- 172.21.15.52:0/3296452024 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02d4003c40 con 0x7f02dc102050 2024-03-04T09:30:51.860 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:51.860+0000 7f02e1415640 1 -- 172.21.15.52:0/3296452024 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02dc1a1f80 con 0x7f02dc102050 2024-03-04T09:30:51.861 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:51.861+0000 7f02e1415640 1 -- 172.21.15.52:0/3296452024 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f02dc0683c0 con 0x7f02dc102050 2024-03-04T09:30:51.861 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:51.862+0000 7f02d3fff640 1 -- 172.21.15.52:0/3296452024 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) v1 ==== 136756+0+0 (secure 0 0 0) 0x7f02d4046020 con 0x7f02dc102050 2024-03-04T09:30:51.861 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:51.863+0000 7f02d3fff640 1 --2- 172.21.15.52:0/3296452024 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f02c80a1d80 0x7f02c80a4240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:30:51.861 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:51.863+0000 7f02da7fc640 1 --2- 172.21.15.52:0/3296452024 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f02c80a1d80 0x7f02c80a4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:30:51.861 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:51.863+0000 7f02d3fff640 1 -- 172.21.15.52:0/3296452024 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9158+0+0 (secure 0 0 0) 0x7f02d40f0190 con 0x7f02dc102050 2024-03-04T09:30:51.862 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:51.863+0000 7f02da7fc640 1 --2- 172.21.15.52:0/3296452024 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f02c80a1d80 0x7f02c80a4240 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f02dc19b090 tx=0x7f02c4005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:30:51.864 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:51.867+0000 7f02d3fff640 1 -- 172.21.15.52:0/3296452024 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f02d404d020 con 0x7f02dc102050 2024-03-04T09:30:52.045 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:52.048+0000 7f02e1415640 1 -- 172.21.15.52:0/3296452024 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f02dc0621d0 con 0x7f02dc102050 2024-03-04T09:30:52.045 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:52.049+0000 7f02d3fff640 1 -- 172.21.15.52:0/3296452024 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v93) v1 ==== 100+0+11 (secure 0 0 0) 0x7f02d40bd1c0 con 0x7f02dc102050 2024-03-04T09:30:52.046 INFO:teuthology.orchestra.run.smithi052.stdout:pg_num: 64 2024-03-04T09:30:52.048 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:52.050+0000 7f02e1415640 1 -- 172.21.15.52:0/3296452024 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f02c80a1d80 msgr2=0x7f02c80a4240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:30:52.048 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:52.050+0000 7f02e1415640 1 --2- 172.21.15.52:0/3296452024 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f02c80a1d80 0x7f02c80a4240 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f02dc19b090 tx=0x7f02c4005f90 comp rx=0 tx=0).stop 2024-03-04T09:30:52.048 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:52.050+0000 7f02e1415640 1 -- 172.21.15.52:0/3296452024 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f02dc102050 msgr2=0x7f02dc199b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:30:52.048 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:52.050+0000 7f02e1415640 1 --2- 172.21.15.52:0/3296452024 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f02dc102050 0x7f02dc199b80 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f02d4000c00 tx=0x7f02d402c650 comp rx=0 tx=0).stop 2024-03-04T09:30:52.048 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:52.050+0000 7f02e1415640 1 -- 172.21.15.52:0/3296452024 shutdown_connections 2024-03-04T09:30:52.048 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:52.050+0000 7f02e1415640 1 --2- 172.21.15.52:0/3296452024 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f02c80a1d80 0x7f02c80a4240 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:52.048 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:52.050+0000 7f02e1415640 1 --2- 172.21.15.52:0/3296452024 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f02dc1072b0 0x7f02dc1a1130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:52.048 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:52.050+0000 7f02e1415640 1 --2- 172.21.15.52:0/3296452024 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f02dc104980 0x7f02dc19a0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:52.048 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:52.050+0000 7f02e1415640 1 --2- 172.21.15.52:0/3296452024 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f02dc102050 0x7f02dc199b80 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:52.048 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:52.050+0000 7f02e1415640 1 -- 172.21.15.52:0/3296452024 >> 172.21.15.52:0/3296452024 conn(0x7f02dc0fba60 msgr2=0x7f02dc101840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:30:52.049 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:52.051+0000 7f02e1415640 1 -- 172.21.15.52:0/3296452024 shutdown_connections 2024-03-04T09:30:52.049 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:52.051+0000 7f02e1415640 1 -- 172.21.15.52:0/3296452024 wait complete. 2024-03-04T09:30:52.059 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-04T09:30:52.167 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:52.168+0000 7fb22ab81640 1 -- 172.21.15.52:0/829808781 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb2241031a0 msgr2=0x7fb224109a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:30:52.167 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:52.168+0000 7fb22ab81640 1 --2- 172.21.15.52:0/829808781 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb2241031a0 0x7fb224109a30 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fb21c009be0 tx=0x7fb21c02bef0 comp rx=0 tx=0).stop 2024-03-04T09:30:52.167 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:52.168+0000 7fb22ab81640 1 -- 172.21.15.52:0/829808781 shutdown_connections 2024-03-04T09:30:52.167 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:52.168+0000 7fb22ab81640 1 --2- 172.21.15.52:0/829808781 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb2241031a0 0x7fb224109a30 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:52.167 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:52.168+0000 7fb22ab81640 1 --2- 172.21.15.52:0/829808781 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb2241027e0 0x7fb224102c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:52.167 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:52.168+0000 7fb22ab81640 1 --2- 172.21.15.52:0/829808781 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb2241015f0 0x7fb224101a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:52.167 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:52.168+0000 7fb22ab81640 1 -- 172.21.15.52:0/829808781 >> 172.21.15.52:0/829808781 conn(0x7fb2240fcb40 msgr2=0x7fb2240fefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:30:52.167 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:52.168+0000 7fb22ab81640 1 -- 172.21.15.52:0/829808781 shutdown_connections 2024-03-04T09:30:52.168 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:52.168+0000 7fb22ab81640 1 -- 172.21.15.52:0/829808781 wait complete. 2024-03-04T09:30:52.168 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:52.168+0000 7fb22ab81640 1 Processor -- start 2024-03-04T09:30:52.168 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:52.168+0000 7fb22ab81640 1 -- start start 2024-03-04T09:30:52.168 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:52.168+0000 7fb22ab81640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb2241015f0 0x7fb224197880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:30:52.172 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:52.168+0000 7fb22ab81640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb2241027e0 0x7fb224197dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:30:52.172 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:52.168+0000 7fb22ab81640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb2241031a0 0x7fb22419ee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:30:52.172 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:52.168+0000 7fb22ab81640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb22410d270 con 0x7fb2241027e0 2024-03-04T09:30:52.172 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:52.168+0000 7fb22ab81640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb22410d0f0 con 0x7fb2241015f0 2024-03-04T09:30:52.172 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:52.168+0000 7fb22ab81640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb22410d3f0 con 0x7fb2241031a0 2024-03-04T09:30:52.172 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:52.169+0000 7fb223fff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb2241027e0 0x7fb224197dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:30:52.172 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:52.169+0000 7fb22891d640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb2241015f0 0x7fb224197880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:30:52.172 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:52.169+0000 7fb223fff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb2241027e0 0x7fb224197dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:58154/0 (socket says 172.21.15.52:58154) 2024-03-04T09:30:52.172 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:52.169+0000 7fb223fff640 1 -- 172.21.15.52:0/1180570880 learned_addr learned my addr 172.21.15.52:0/1180570880 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-04T09:30:52.172 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:52.169+0000 7fb22911e640 1 --2- 172.21.15.52:0/1180570880 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb2241031a0 0x7fb22419ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:30:52.172 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:52.169+0000 7fb223fff640 1 -- 172.21.15.52:0/1180570880 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb2241031a0 msgr2=0x7fb22419ee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:30:52.173 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:52.169+0000 7fb223fff640 1 --2- 172.21.15.52:0/1180570880 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb2241031a0 0x7fb22419ee30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:52.173 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:52.169+0000 7fb223fff640 1 -- 172.21.15.52:0/1180570880 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb2241015f0 msgr2=0x7fb224197880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:30:52.173 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:52.169+0000 7fb223fff640 1 --2- 172.21.15.52:0/1180570880 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb2241015f0 0x7fb224197880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:52.173 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:52.169+0000 7fb223fff640 1 -- 172.21.15.52:0/1180570880 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb22419f4a0 con 0x7fb2241027e0 2024-03-04T09:30:52.173 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:52.169+0000 7fb223fff640 1 --2- 172.21.15.52:0/1180570880 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb2241027e0 0x7fb224197dc0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fb21000e9c0 tx=0x7fb21000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:30:52.173 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:52.169+0000 7fb221ffb640 1 -- 172.21.15.52:0/1180570880 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb21000cde0 con 0x7fb2241027e0 2024-03-04T09:30:52.173 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:52.169+0000 7fb22ab81640 1 -- 172.21.15.52:0/1180570880 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb22419f790 con 0x7fb2241027e0 2024-03-04T09:30:52.173 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:52.169+0000 7fb221ffb640 1 -- 172.21.15.52:0/1180570880 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb2100035f0 con 0x7fb2241027e0 2024-03-04T09:30:52.173 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:52.169+0000 7fb221ffb640 1 -- 172.21.15.52:0/1180570880 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb210010690 con 0x7fb2241027e0 2024-03-04T09:30:52.173 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:52.169+0000 7fb22ab81640 1 -- 172.21.15.52:0/1180570880 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb22419fce0 con 0x7fb2241027e0 2024-03-04T09:30:52.175 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:52.170+0000 7fb22ab81640 1 -- 172.21.15.52:0/1180570880 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb2241088b0 con 0x7fb2241027e0 2024-03-04T09:30:52.175 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:52.171+0000 7fb221ffb640 1 -- 172.21.15.52:0/1180570880 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) v1 ==== 136756+0+0 (secure 0 0 0) 0x7fb2100037a0 con 0x7fb2241027e0 2024-03-04T09:30:52.175 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:52.172+0000 7fb221ffb640 1 --2- 172.21.15.52:0/1180570880 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7fb2140a1d80 0x7fb2140a4240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:30:52.175 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:52.172+0000 7fb22891d640 1 --2- 172.21.15.52:0/1180570880 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7fb2140a1d80 0x7fb2140a4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:30:52.175 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:52.172+0000 7fb221ffb640 1 -- 172.21.15.52:0/1180570880 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9158+0+0 (secure 0 0 0) 0x7fb2100ce4b0 con 0x7fb2241027e0 2024-03-04T09:30:52.175 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:52.172+0000 7fb22891d640 1 --2- 172.21.15.52:0/1180570880 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7fb2140a1d80 0x7fb2140a4240 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fb224102640 tx=0x7fb218005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:30:52.176 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:52.176+0000 7fb221ffb640 1 -- 172.21.15.52:0/1180570880 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fb21009b4e0 con 0x7fb2241027e0 2024-03-04T09:30:52.357 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:52.359+0000 7fb22ab81640 1 -- 172.21.15.52:0/1180570880 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fb2240621d0 con 0x7fb2241027e0 2024-03-04T09:30:52.357 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:52.360+0000 7fb221ffb640 1 -- 172.21.15.52:0/1180570880 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v93) v1 ==== 103+0+11 (secure 0 0 0) 0x7fb21009ade0 con 0x7fb2241027e0 2024-03-04T09:30:52.357 INFO:teuthology.orchestra.run.smithi052.stdout:pg_num: 32 2024-03-04T09:30:52.359 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:52.362+0000 7fb22ab81640 1 -- 172.21.15.52:0/1180570880 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7fb2140a1d80 msgr2=0x7fb2140a4240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:30:52.360 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:52.362+0000 7fb22ab81640 1 --2- 172.21.15.52:0/1180570880 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7fb2140a1d80 0x7fb2140a4240 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fb224102640 tx=0x7fb218005f90 comp rx=0 tx=0).stop 2024-03-04T09:30:52.360 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:52.362+0000 7fb22ab81640 1 -- 172.21.15.52:0/1180570880 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb2241027e0 msgr2=0x7fb224197dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:30:52.360 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:52.362+0000 7fb22ab81640 1 --2- 172.21.15.52:0/1180570880 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb2241027e0 0x7fb224197dc0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fb21000e9c0 tx=0x7fb21000ee80 comp rx=0 tx=0).stop 2024-03-04T09:30:52.360 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:52.362+0000 7fb22ab81640 1 -- 172.21.15.52:0/1180570880 shutdown_connections 2024-03-04T09:30:52.360 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:52.362+0000 7fb22ab81640 1 --2- 172.21.15.52:0/1180570880 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7fb2140a1d80 0x7fb2140a4240 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:52.361 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:52.362+0000 7fb22ab81640 1 --2- 172.21.15.52:0/1180570880 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb2241031a0 0x7fb22419ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:52.361 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:52.362+0000 7fb22ab81640 1 --2- 172.21.15.52:0/1180570880 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb2241027e0 0x7fb224197dc0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:52.361 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:52.362+0000 7fb22ab81640 1 --2- 172.21.15.52:0/1180570880 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb2241015f0 0x7fb224197880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:52.361 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:52.362+0000 7fb22ab81640 1 -- 172.21.15.52:0/1180570880 >> 172.21.15.52:0/1180570880 conn(0x7fb2240fcb40 msgr2=0x7fb224105a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:30:52.361 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:52.362+0000 7fb22ab81640 1 -- 172.21.15.52:0/1180570880 shutdown_connections 2024-03-04T09:30:52.361 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:52.362+0000 7fb22ab81640 1 -- 172.21.15.52:0/1180570880 wait complete. 2024-03-04T09:30:52.370 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T09:30:52.479 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:52.477+0000 7f2658365640 1 -- 172.21.15.52:0/4086258773 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f26501027e0 msgr2=0x7f2650102c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:30:52.479 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:52.477+0000 7f2658365640 1 --2- 172.21.15.52:0/4086258773 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f26501027e0 0x7f2650102c60 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f2640009900 tx=0x7f264002bc30 comp rx=0 tx=0).stop 2024-03-04T09:30:52.479 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:52.477+0000 7f2658365640 1 -- 172.21.15.52:0/4086258773 shutdown_connections 2024-03-04T09:30:52.479 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:52.477+0000 7f2658365640 1 --2- 172.21.15.52:0/4086258773 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f26501031a0 0x7f2650109a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:52.479 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:52.477+0000 7f2658365640 1 --2- 172.21.15.52:0/4086258773 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f26501027e0 0x7f2650102c60 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:52.479 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:52.477+0000 7f2658365640 1 --2- 172.21.15.52:0/4086258773 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f26501015f0 0x7f2650101a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:52.480 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:52.477+0000 7f2658365640 1 -- 172.21.15.52:0/4086258773 >> 172.21.15.52:0/4086258773 conn(0x7f26500fcb80 msgr2=0x7f26500fefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:30:52.480 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:52.477+0000 7f2658365640 1 -- 172.21.15.52:0/4086258773 shutdown_connections 2024-03-04T09:30:52.480 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:52.477+0000 7f2658365640 1 -- 172.21.15.52:0/4086258773 wait complete. 2024-03-04T09:30:52.480 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:52.478+0000 7f2658365640 1 Processor -- start 2024-03-04T09:30:52.480 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:52.478+0000 7f2658365640 1 -- start start 2024-03-04T09:30:52.480 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:52.478+0000 7f2658365640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f26501015f0 0x7f26501978e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:30:52.480 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:52.478+0000 7f2658365640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f26501027e0 0x7f2650197e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:30:52.480 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:52.478+0000 7f2658365640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f26501031a0 0x7f265019ee90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:30:52.481 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:52.478+0000 7f2658365640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f265010d270 con 0x7f26501031a0 2024-03-04T09:30:52.481 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:52.478+0000 7f2658365640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f265010d0f0 con 0x7f26501015f0 2024-03-04T09:30:52.481 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:52.478+0000 7f2658365640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f265010d3f0 con 0x7f26501027e0 2024-03-04T09:30:52.481 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:52.478+0000 7f2656902640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f26501031a0 0x7f265019ee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:30:52.481 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:52.478+0000 7f2656101640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f26501015f0 0x7f26501978e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:30:52.481 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:52.478+0000 7f2656902640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f26501031a0 0x7f265019ee90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:58170/0 (socket says 172.21.15.52:58170) 2024-03-04T09:30:52.481 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:52.478+0000 7f2656902640 1 -- 172.21.15.52:0/3650944609 learned_addr learned my addr 172.21.15.52:0/3650944609 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-04T09:30:52.481 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:52.478+0000 7f2655900640 1 --2- 172.21.15.52:0/3650944609 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f26501027e0 0x7f2650197e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:30:52.481 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:52.478+0000 7f2656902640 1 -- 172.21.15.52:0/3650944609 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f26501027e0 msgr2=0x7f2650197e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:30:52.482 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:52.478+0000 7f2656902640 1 --2- 172.21.15.52:0/3650944609 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f26501027e0 0x7f2650197e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:52.482 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:52.478+0000 7f2656902640 1 -- 172.21.15.52:0/3650944609 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f26501015f0 msgr2=0x7f26501978e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:30:52.482 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:52.478+0000 7f2656902640 1 --2- 172.21.15.52:0/3650944609 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f26501015f0 0x7f26501978e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:52.482 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:52.478+0000 7f2656902640 1 -- 172.21.15.52:0/3650944609 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f265019f500 con 0x7f26501031a0 2024-03-04T09:30:52.482 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:52.479+0000 7f2656902640 1 --2- 172.21.15.52:0/3650944609 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f26501031a0 0x7f265019ee90 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f264800eb10 tx=0x7f264800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:30:52.482 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:52.479+0000 7f26477fe640 1 -- 172.21.15.52:0/3650944609 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2648004b20 con 0x7f26501031a0 2024-03-04T09:30:52.482 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:52.479+0000 7f26477fe640 1 -- 172.21.15.52:0/3650944609 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2648004cc0 con 0x7f26501031a0 2024-03-04T09:30:52.482 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:52.479+0000 7f2658365640 1 -- 172.21.15.52:0/3650944609 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f265019f7f0 con 0x7f26501031a0 2024-03-04T09:30:52.483 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:52.479+0000 7f26477fe640 1 -- 172.21.15.52:0/3650944609 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2648010760 con 0x7f26501031a0 2024-03-04T09:30:52.483 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:52.479+0000 7f2658365640 1 -- 172.21.15.52:0/3650944609 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f265019fd10 con 0x7f26501031a0 2024-03-04T09:30:52.483 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:52.480+0000 7f2658365640 1 -- 172.21.15.52:0/3650944609 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2650108910 con 0x7f26501031a0 2024-03-04T09:30:52.483 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:52.481+0000 7f26477fe640 1 -- 172.21.15.52:0/3650944609 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) v1 ==== 136756+0+0 (secure 0 0 0) 0x7f2648005000 con 0x7f26501031a0 2024-03-04T09:30:52.483 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:52.481+0000 7f26477fe640 1 --2- 172.21.15.52:0/3650944609 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f263c0a1d80 0x7f263c0a4240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:30:52.483 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:52.482+0000 7f26477fe640 1 -- 172.21.15.52:0/3650944609 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9158+0+0 (secure 0 0 0) 0x7f26480ceb70 con 0x7f26501031a0 2024-03-04T09:30:52.483 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:52.482+0000 7f2656101640 1 --2- 172.21.15.52:0/3650944609 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f263c0a1d80 0x7f263c0a4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:30:52.483 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:52.482+0000 7f2656101640 1 --2- 172.21.15.52:0/3650944609 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f263c0a1d80 0x7f263c0a4240 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f2650102640 tx=0x7f264c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:30:52.484 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:52.485+0000 7f26477fe640 1 -- 172.21.15.52:0/3650944609 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f264809bc00 con 0x7f26501031a0 2024-03-04T09:30:52.584 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:30:52 smithi052 ceph-mon[32473]: from='client.? 172.21.15.52:0/337026709' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-04T09:30:52.585 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:30:52 smithi052 ceph-mon[32473]: from='client.? 172.21.15.52:0/362375957' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-04T09:30:52.585 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:30:52 smithi052 ceph-mon[32473]: from='client.? 172.21.15.52:0/3296452024' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-04T09:30:52.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:30:52 smithi191 ceph-mon[36816]: from='client.? 172.21.15.52:0/337026709' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-04T09:30:52.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:30:52 smithi191 ceph-mon[36816]: from='client.? 172.21.15.52:0/362375957' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-04T09:30:52.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:30:52 smithi191 ceph-mon[36816]: from='client.? 172.21.15.52:0/3296452024' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-04T09:30:52.631 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:30:52 smithi106 ceph-mon[36484]: from='client.? 172.21.15.52:0/337026709' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-04T09:30:52.631 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:30:52 smithi106 ceph-mon[36484]: from='client.? 172.21.15.52:0/362375957' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-04T09:30:52.631 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:30:52 smithi106 ceph-mon[36484]: from='client.? 172.21.15.52:0/3296452024' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-04T09:30:52.702 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:52.706+0000 7f2658365640 1 -- 172.21.15.52:0/3650944609 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f265002cff0 con 0x7f26501031a0 2024-03-04T09:30:52.703 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:52.706+0000 7f26477fe640 1 -- 172.21.15.52:0/3650944609 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+11406 (secure 0 0 0) 0x7f264809b500 con 0x7f26501031a0 2024-03-04T09:30:52.703 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-03-04T09:30:52.704 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6835/1007961019","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6834","nonce":1007961019},{"type":"v1","addr":"172.21.15.52:6835","nonce":1007961019}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6837/3753881283","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6836","nonce":3753881283},{"type":"v1","addr":"172.21.15.52:6837","nonce":3753881283}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14628,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6839/3467897808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6838","nonce":3467897808},{"type":"v1","addr":"172.21.15.52:6839","nonce":3467897808}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14640,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6841/2804292291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6840","nonce":2804292291},{"type":"v1","addr":"172.21.15.52:6841","nonce":2804292291}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24407,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6833/1818480323","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6832","nonce":1818480323},{"type":"v1","addr":"172.21.15.106:6833","nonce":1818480323}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24412,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6833/765768204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":765768204},{"type":"v1","addr":"172.21.15.191:6833","nonce":765768204}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-04T09:28:37.264970+0000","modified":"2024-03-04T09:29:04.001848+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24506,"mds_1":24490,"mds_2":24473},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24440":{"gid":24440,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.106:6835/1541069897","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6834","nonce":1541069897},{"type":"v1","addr":"172.21.15.106:6835","nonce":1541069897}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24451":{"gid":24451,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.191:6835/105048699","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":105048699},{"type":"v1","addr":"172.21.15.191:6835","nonce":105048699}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24463":{"gid":24463,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.191:6837/4071064225","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6836","nonce":4071064225},{"type":"v1","addr":"172.21.15.191:6837","nonce":4071064225}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24473":{"gid":24473,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.106:6837/2901755066","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6836","nonce":2901755066},{"type":"v1","addr":"172.21.15.106:6837","nonce":2901755066}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24490":{"gid":24490,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.191:6839/2194324891","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6838","nonce":2194324891},{"type":"v1","addr":"172.21.15.191:6839","nonce":2194324891}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24506":{"gid":24506,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.106:6839/1995853093","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6838","nonce":1995853093},{"type":"v1","addr":"172.21.15.106:6839","nonce":1995853093}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24506,"qdb_cluster":[24473,24490,24506]},"id":1}]} 2024-03-04T09:30:52.704 INFO:teuthology.orchestra.run.smithi052.stderr:dumped fsmap epoch 24 2024-03-04T09:30:52.705 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:52.708+0000 7f2658365640 1 -- 172.21.15.52:0/3650944609 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f263c0a1d80 msgr2=0x7f263c0a4240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:30:52.706 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:52.708+0000 7f2658365640 1 --2- 172.21.15.52:0/3650944609 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f263c0a1d80 0x7f263c0a4240 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f2650102640 tx=0x7f264c005f90 comp rx=0 tx=0).stop 2024-03-04T09:30:52.706 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:52.708+0000 7f2658365640 1 -- 172.21.15.52:0/3650944609 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f26501031a0 msgr2=0x7f265019ee90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:30:52.706 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:52.708+0000 7f2658365640 1 --2- 172.21.15.52:0/3650944609 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f26501031a0 0x7f265019ee90 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f264800eb10 tx=0x7f264800c2d0 comp rx=0 tx=0).stop 2024-03-04T09:30:52.706 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:52.708+0000 7f2658365640 1 -- 172.21.15.52:0/3650944609 shutdown_connections 2024-03-04T09:30:52.706 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:52.708+0000 7f2658365640 1 --2- 172.21.15.52:0/3650944609 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f263c0a1d80 0x7f263c0a4240 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:52.706 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:52.708+0000 7f2658365640 1 --2- 172.21.15.52:0/3650944609 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f26501031a0 0x7f265019ee90 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:52.706 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:52.708+0000 7f2658365640 1 --2- 172.21.15.52:0/3650944609 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f26501027e0 0x7f2650197e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:52.706 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:52.708+0000 7f2658365640 1 --2- 172.21.15.52:0/3650944609 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f26501015f0 0x7f26501978e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:52.706 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:52.708+0000 7f2658365640 1 -- 172.21.15.52:0/3650944609 >> 172.21.15.52:0/3650944609 conn(0x7f26500fcb80 msgr2=0x7f2650105ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:30:52.706 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:52.709+0000 7f2658365640 1 -- 172.21.15.52:0/3650944609 shutdown_connections 2024-03-04T09:30:52.706 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:52.709+0000 7f2658365640 1 -- 172.21.15.52:0/3650944609 wait complete. 2024-03-04T09:30:52.717 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-04T09:30:52.822 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:52.822+0000 7fe3a084e640 1 -- 172.21.15.52:0/1804901738 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe39c100510 msgr2=0x7fe39c100930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:30:52.822 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:52.822+0000 7fe3a084e640 1 --2- 172.21.15.52:0/1804901738 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe39c100510 0x7fe39c100930 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fe38c009900 tx=0x7fe38c02bc30 comp rx=0 tx=0).stop 2024-03-04T09:30:52.822 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:52.822+0000 7fe3a084e640 1 -- 172.21.15.52:0/1804901738 shutdown_connections 2024-03-04T09:30:52.822 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:52.822+0000 7fe3a084e640 1 --2- 172.21.15.52:0/1804901738 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe39c1020c0 0x7fe39c108950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:52.822 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:52.822+0000 7fe3a084e640 1 --2- 172.21.15.52:0/1804901738 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe39c101700 0x7fe39c101b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:52.822 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:52.822+0000 7fe3a084e640 1 --2- 172.21.15.52:0/1804901738 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe39c100510 0x7fe39c100930 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:52.822 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:52.822+0000 7fe3a084e640 1 -- 172.21.15.52:0/1804901738 >> 172.21.15.52:0/1804901738 conn(0x7fe39c0fba80 msgr2=0x7fe39c0fdec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:30:52.822 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:52.822+0000 7fe3a084e640 1 -- 172.21.15.52:0/1804901738 shutdown_connections 2024-03-04T09:30:52.822 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:52.823+0000 7fe3a084e640 1 -- 172.21.15.52:0/1804901738 wait complete. 2024-03-04T09:30:52.822 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:52.823+0000 7fe3a084e640 1 Processor -- start 2024-03-04T09:30:52.822 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:52.823+0000 7fe3a084e640 1 -- start start 2024-03-04T09:30:52.822 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:52.823+0000 7fe3a084e640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe39c100510 0x7fe39c06a790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:30:52.823 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:52.823+0000 7fe3a084e640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe39c101700 0x7fe39c19c670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:30:52.823 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:52.823+0000 7fe3a084e640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe39c1020c0 0x7fe39c19ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:30:52.823 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:52.823+0000 7fe3a084e640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe39c10c170 con 0x7fe39c1020c0 2024-03-04T09:30:52.823 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:52.823+0000 7fe3a084e640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe39c10bff0 con 0x7fe39c101700 2024-03-04T09:30:52.823 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:52.823+0000 7fe3a084e640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe39c10c2f0 con 0x7fe39c100510 2024-03-04T09:30:52.823 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:52.824+0000 7fe39ad9d640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe39c1020c0 0x7fe39c19ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:30:52.823 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:52.824+0000 7fe39ad9d640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe39c1020c0 0x7fe39c19ecb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:58188/0 (socket says 172.21.15.52:58188) 2024-03-04T09:30:52.823 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:52.824+0000 7fe39ad9d640 1 -- 172.21.15.52:0/2013879280 learned_addr learned my addr 172.21.15.52:0/2013879280 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-04T09:30:52.823 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:52.824+0000 7fe393fff640 1 --2- 172.21.15.52:0/2013879280 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe39c101700 0x7fe39c19c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:30:52.823 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:52.824+0000 7fe39a59c640 1 --2- 172.21.15.52:0/2013879280 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe39c100510 0x7fe39c06a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:30:52.823 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:52.824+0000 7fe39ad9d640 1 -- 172.21.15.52:0/2013879280 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe39c100510 msgr2=0x7fe39c06a790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:30:52.823 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:52.824+0000 7fe39ad9d640 1 --2- 172.21.15.52:0/2013879280 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe39c100510 0x7fe39c06a790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:52.823 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:52.824+0000 7fe39ad9d640 1 -- 172.21.15.52:0/2013879280 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe39c101700 msgr2=0x7fe39c19c670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:30:52.824 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:52.824+0000 7fe39ad9d640 1 --2- 172.21.15.52:0/2013879280 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe39c101700 0x7fe39c19c670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:52.824 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:52.824+0000 7fe39ad9d640 1 -- 172.21.15.52:0/2013879280 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe39c19f570 con 0x7fe39c1020c0 2024-03-04T09:30:52.824 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:52.824+0000 7fe393fff640 1 --2- 172.21.15.52:0/2013879280 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe39c101700 0x7fe39c19c670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-04T09:30:52.824 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:52.824+0000 7fe39ad9d640 1 --2- 172.21.15.52:0/2013879280 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe39c1020c0 0x7fe39c19ecb0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fe39400eb10 tx=0x7fe39400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:30:52.824 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:52.824+0000 7fe3937fe640 1 -- 172.21.15.52:0/2013879280 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe394004b20 con 0x7fe39c1020c0 2024-03-04T09:30:52.824 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:52.824+0000 7fe3937fe640 1 -- 172.21.15.52:0/2013879280 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe394004cc0 con 0x7fe39c1020c0 2024-03-04T09:30:52.824 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:52.824+0000 7fe3a084e640 1 -- 172.21.15.52:0/2013879280 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe39c19f860 con 0x7fe39c1020c0 2024-03-04T09:30:52.824 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:52.824+0000 7fe3937fe640 1 -- 172.21.15.52:0/2013879280 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe394010760 con 0x7fe39c1020c0 2024-03-04T09:30:52.824 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:52.824+0000 7fe3a084e640 1 -- 172.21.15.52:0/2013879280 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe39c19fd60 con 0x7fe39c1020c0 2024-03-04T09:30:52.825 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:52.825+0000 7fe3a084e640 1 -- 172.21.15.52:0/2013879280 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe39c107830 con 0x7fe39c1020c0 2024-03-04T09:30:52.825 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:52.826+0000 7fe3937fe640 1 -- 172.21.15.52:0/2013879280 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) v1 ==== 136756+0+0 (secure 0 0 0) 0x7fe394005020 con 0x7fe39c1020c0 2024-03-04T09:30:52.826 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:52.827+0000 7fe3937fe640 1 --2- 172.21.15.52:0/2013879280 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7fe3880a1d80 0x7fe3880a4240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:30:52.826 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:52.827+0000 7fe39a59c640 1 --2- 172.21.15.52:0/2013879280 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7fe3880a1d80 0x7fe3880a4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:30:52.826 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:52.827+0000 7fe3937fe640 1 -- 172.21.15.52:0/2013879280 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9158+0+0 (secure 0 0 0) 0x7fe3940ce760 con 0x7fe39c1020c0 2024-03-04T09:30:52.826 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:52.828+0000 7fe39a59c640 1 --2- 172.21.15.52:0/2013879280 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7fe3880a1d80 0x7fe3880a4240 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fe38c000c00 tx=0x7fe38c005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:30:52.828 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:52.832+0000 7fe3937fe640 1 -- 172.21.15.52:0/2013879280 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fe39409b790 con 0x7fe39c1020c0 2024-03-04T09:30:53.014 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:53.017+0000 7fe3a084e640 1 -- 172.21.15.52:0/2013879280 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe39c0621d0 con 0x7fe39c1020c0 2024-03-04T09:30:53.016 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:53.019+0000 7fe3937fe640 1 -- 172.21.15.52:0/2013879280 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v93) v1 ==== 74+0+24804 (secure 0 0 0) 0x7fe39409b090 con 0x7fe39c1020c0 2024-03-04T09:30:53.016 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-03-04T09:30:53.017 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":93,"fsid":"4e059f08-da08-11ee-95c6-87774f69a715","created":"2024-03-04T09:20:46.104963+0000","modified":"2024-03-04T09:30:41.237792+0000","last_up_change":"2024-03-04T09:27:49.261365+0000","last_in_change":"2024-03-04T09:27:30.024964+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-04T09:24:01.742461+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-04T09:28:35.835600+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-04T09:28:36.453099+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-04T09:28:40.168831+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"583172be-602f-4bf4-bfcb-42663cf5b277","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6802","nonce":1371221397},{"type":"v1","addr":"172.21.15.52:6803","nonce":1371221397}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6804","nonce":1371221397},{"type":"v1","addr":"172.21.15.52:6805","nonce":1371221397}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6808","nonce":1371221397},{"type":"v1","addr":"172.21.15.52:6809","nonce":1371221397}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6806","nonce":1371221397},{"type":"v1","addr":"172.21.15.52:6807","nonce":1371221397}]},"public_addr":"172.21.15.52:6803/1371221397","cluster_addr":"172.21.15.52:6805/1371221397","heartbeat_back_addr":"172.21.15.52:6809/1371221397","heartbeat_front_addr":"172.21.15.52:6807/1371221397","state":["exists","up"]},{"osd":1,"uuid":"5ae0081c-a2f6-46eb-ace0-0368208377b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":2540865537},{"type":"v1","addr":"172.21.15.106:6801","nonce":2540865537}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":2540865537},{"type":"v1","addr":"172.21.15.106:6803","nonce":2540865537}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":2540865537},{"type":"v1","addr":"172.21.15.106:6807","nonce":2540865537}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":2540865537},{"type":"v1","addr":"172.21.15.106:6805","nonce":2540865537}]},"public_addr":"172.21.15.106:6801/2540865537","cluster_addr":"172.21.15.106:6803/2540865537","heartbeat_back_addr":"172.21.15.106:6807/2540865537","heartbeat_front_addr":"172.21.15.106:6805/2540865537","state":["exists","up"]},{"osd":2,"uuid":"1948bac7-2bb2-4218-8b93-4e4743a0de04","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":1377083164},{"type":"v1","addr":"172.21.15.191:6801","nonce":1377083164}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":1377083164},{"type":"v1","addr":"172.21.15.191:6803","nonce":1377083164}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":1377083164},{"type":"v1","addr":"172.21.15.191:6807","nonce":1377083164}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":1377083164},{"type":"v1","addr":"172.21.15.191:6805","nonce":1377083164}]},"public_addr":"172.21.15.191:6801/1377083164","cluster_addr":"172.21.15.191:6803/1377083164","heartbeat_back_addr":"172.21.15.191:6807/1377083164","heartbeat_front_addr":"172.21.15.191:6805/1377083164","state":["exists","up"]},{"osd":3,"uuid":"16c9e9fe-7167-4992-bf12-1a7398e0498a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6810","nonce":2388571810},{"type":"v1","addr":"172.21.15.52:6811","nonce":2388571810}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6812","nonce":2388571810},{"type":"v1","addr":"172.21.15.52:6813","nonce":2388571810}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6816","nonce":2388571810},{"type":"v1","addr":"172.21.15.52:6817","nonce":2388571810}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6814","nonce":2388571810},{"type":"v1","addr":"172.21.15.52:6815","nonce":2388571810}]},"public_addr":"172.21.15.52:6811/2388571810","cluster_addr":"172.21.15.52:6813/2388571810","heartbeat_back_addr":"172.21.15.52:6817/2388571810","heartbeat_front_addr":"172.21.15.52:6815/2388571810","state":["exists","up"]},{"osd":4,"uuid":"c8ef48c7-7c78-44d3-a9e5-99ec750abd3d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6808","nonce":3465868015},{"type":"v1","addr":"172.21.15.106:6809","nonce":3465868015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6810","nonce":3465868015},{"type":"v1","addr":"172.21.15.106:6811","nonce":3465868015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6814","nonce":3465868015},{"type":"v1","addr":"172.21.15.106:6815","nonce":3465868015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6812","nonce":3465868015},{"type":"v1","addr":"172.21.15.106:6813","nonce":3465868015}]},"public_addr":"172.21.15.106:6809/3465868015","cluster_addr":"172.21.15.106:6811/3465868015","heartbeat_back_addr":"172.21.15.106:6815/3465868015","heartbeat_front_addr":"172.21.15.106:6813/3465868015","state":["exists","up"]},{"osd":5,"uuid":"9e41824a-c5a2-4f88-be50-2ba1dceb3220","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":1485092643},{"type":"v1","addr":"172.21.15.191:6809","nonce":1485092643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":1485092643},{"type":"v1","addr":"172.21.15.191:6811","nonce":1485092643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":1485092643},{"type":"v1","addr":"172.21.15.191:6815","nonce":1485092643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":1485092643},{"type":"v1","addr":"172.21.15.191:6813","nonce":1485092643}]},"public_addr":"172.21.15.191:6809/1485092643","cluster_addr":"172.21.15.191:6811/1485092643","heartbeat_back_addr":"172.21.15.191:6815/1485092643","heartbeat_front_addr":"172.21.15.191:6813/1485092643","state":["exists","up"]},{"osd":6,"uuid":"b35dccee-ec16-452e-ac69-3ac2fb7b27fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6818","nonce":1989797856},{"type":"v1","addr":"172.21.15.52:6819","nonce":1989797856}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6820","nonce":1989797856},{"type":"v1","addr":"172.21.15.52:6821","nonce":1989797856}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6824","nonce":1989797856},{"type":"v1","addr":"172.21.15.52:6825","nonce":1989797856}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6822","nonce":1989797856},{"type":"v1","addr":"172.21.15.52:6823","nonce":1989797856}]},"public_addr":"172.21.15.52:6819/1989797856","cluster_addr":"172.21.15.52:6821/1989797856","heartbeat_back_addr":"172.21.15.52:6825/1989797856","heartbeat_front_addr":"172.21.15.52:6823/1989797856","state":["exists","up"]},{"osd":7,"uuid":"71303e3d-14ec-4cbd-b19c-67645a1de3c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6816","nonce":1450870329},{"type":"v1","addr":"172.21.15.106:6817","nonce":1450870329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":1450870329},{"type":"v1","addr":"172.21.15.106:6819","nonce":1450870329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":1450870329},{"type":"v1","addr":"172.21.15.106:6823","nonce":1450870329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6820","nonce":1450870329},{"type":"v1","addr":"172.21.15.106:6821","nonce":1450870329}]},"public_addr":"172.21.15.106:6817/1450870329","cluster_addr":"172.21.15.106:6819/1450870329","heartbeat_back_addr":"172.21.15.106:6823/1450870329","heartbeat_front_addr":"172.21.15.106:6821/1450870329","state":["exists","up"]},{"osd":8,"uuid":"6f5786c4-4402-435a-bdad-adff46fe9473","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":1114119072},{"type":"v1","addr":"172.21.15.191:6817","nonce":1114119072}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6818","nonce":1114119072},{"type":"v1","addr":"172.21.15.191:6819","nonce":1114119072}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":1114119072},{"type":"v1","addr":"172.21.15.191:6823","nonce":1114119072}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":1114119072},{"type":"v1","addr":"172.21.15.191:6821","nonce":1114119072}]},"public_addr":"172.21.15.191:6817/1114119072","cluster_addr":"172.21.15.191:6819/1114119072","heartbeat_back_addr":"172.21.15.191:6823/1114119072","heartbeat_front_addr":"172.21.15.191:6821/1114119072","state":["exists","up"]},{"osd":9,"uuid":"062bbaa5-1227-4684-b859-450f5594a90c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6826","nonce":3485821914},{"type":"v1","addr":"172.21.15.52:6827","nonce":3485821914}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6828","nonce":3485821914},{"type":"v1","addr":"172.21.15.52:6829","nonce":3485821914}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6832","nonce":3485821914},{"type":"v1","addr":"172.21.15.52:6833","nonce":3485821914}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6830","nonce":3485821914},{"type":"v1","addr":"172.21.15.52:6831","nonce":3485821914}]},"public_addr":"172.21.15.52:6827/3485821914","cluster_addr":"172.21.15.52:6829/3485821914","heartbeat_back_addr":"172.21.15.52:6833/3485821914","heartbeat_front_addr":"172.21.15.52:6831/3485821914","state":["exists","up"]},{"osd":10,"uuid":"1a285a5a-f793-48da-a8eb-5326d9528cd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":3815586576},{"type":"v1","addr":"172.21.15.106:6825","nonce":3815586576}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6826","nonce":3815586576},{"type":"v1","addr":"172.21.15.106:6827","nonce":3815586576}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":3815586576},{"type":"v1","addr":"172.21.15.106:6831","nonce":3815586576}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":3815586576},{"type":"v1","addr":"172.21.15.106:6829","nonce":3815586576}]},"public_addr":"172.21.15.106:6825/3815586576","cluster_addr":"172.21.15.106:6827/3815586576","heartbeat_back_addr":"172.21.15.106:6831/3815586576","heartbeat_front_addr":"172.21.15.106:6829/3815586576","state":["exists","up"]},{"osd":11,"uuid":"1f8b9a43-57fa-41f4-9969-3b803a74bf45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":2009632843},{"type":"v1","addr":"172.21.15.191:6825","nonce":2009632843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":2009632843},{"type":"v1","addr":"172.21.15.191:6827","nonce":2009632843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":2009632843},{"type":"v1","addr":"172.21.15.191:6831","nonce":2009632843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":2009632843},{"type":"v1","addr":"172.21.15.191:6829","nonce":2009632843}]},"public_addr":"172.21.15.191:6825/2009632843","cluster_addr":"172.21.15.191:6827/2009632843","heartbeat_back_addr":"172.21.15.191:6831/2009632843","heartbeat_front_addr":"172.21.15.191:6829/2009632843","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T09:23:10.894763+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T09:23:34.420055+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T09:23:58.288324+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T09:24:23.673457+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T09:24:48.208561+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T09:25:13.405473+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T09:25:38.670915+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T09:26:04.451257+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T09:26:30.281380+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T09:26:56.543143+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T09:27:22.412930+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T09:27:48.003866+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.1d","mappings":[{"from":2,"to":1},{"from":8,"to":11}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.20","mappings":[{"from":5,"to":9}]},{"pgid":"3.21","mappings":[{"from":4,"to":6}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]},{"pgid":"4.9","mappings":[{"from":5,"to":8}]},{"pgid":"4.17","mappings":[{"from":9,"to":8}]},{"pgid":"4.1b","mappings":[{"from":9,"to":10}]},{"pgid":"4.1f","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.52:0/4082645718":"2024-03-05T09:21:39.401935+0000","172.21.15.52:0/1271949988":"2024-03-05T09:21:39.401935+0000","172.21.15.52:6801/2379361333":"2024-03-05T09:21:12.760938+0000","172.21.15.52:0/497075075":"2024-03-05T09:21:12.760938+0000","172.21.15.52:6800/2379361333":"2024-03-05T09:21:12.760938+0000","172.21.15.52:0/568242577":"2024-03-05T09:21:12.760938+0000","172.21.15.52:6801/2425730636":"2024-03-05T09:21:39.401935+0000","172.21.15.52:6800/2425730636":"2024-03-05T09:21:39.401935+0000","172.21.15.52:0/2743084243":"2024-03-05T09:21:39.401935+0000","172.21.15.52:0/1035878034":"2024-03-05T09:21:12.760938+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-04T09:30:53.018 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:53.021+0000 7fe3a084e640 1 -- 172.21.15.52:0/2013879280 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7fe3880a1d80 msgr2=0x7fe3880a4240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:30:53.018 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:53.021+0000 7fe3a084e640 1 --2- 172.21.15.52:0/2013879280 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7fe3880a1d80 0x7fe3880a4240 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fe38c000c00 tx=0x7fe38c005fd0 comp rx=0 tx=0).stop 2024-03-04T09:30:53.018 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:53.021+0000 7fe3a084e640 1 -- 172.21.15.52:0/2013879280 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe39c1020c0 msgr2=0x7fe39c19ecb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:30:53.018 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:53.021+0000 7fe3a084e640 1 --2- 172.21.15.52:0/2013879280 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe39c1020c0 0x7fe39c19ecb0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fe39400eb10 tx=0x7fe39400c2d0 comp rx=0 tx=0).stop 2024-03-04T09:30:53.019 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:53.021+0000 7fe3a084e640 1 -- 172.21.15.52:0/2013879280 shutdown_connections 2024-03-04T09:30:53.019 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:53.021+0000 7fe3a084e640 1 --2- 172.21.15.52:0/2013879280 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7fe3880a1d80 0x7fe3880a4240 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:53.019 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:53.021+0000 7fe3a084e640 1 --2- 172.21.15.52:0/2013879280 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe39c1020c0 0x7fe39c19ecb0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:53.019 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:53.021+0000 7fe3a084e640 1 --2- 172.21.15.52:0/2013879280 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe39c101700 0x7fe39c19c670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:53.019 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:53.021+0000 7fe3a084e640 1 --2- 172.21.15.52:0/2013879280 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe39c100510 0x7fe39c06a790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:53.019 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:53.021+0000 7fe3a084e640 1 -- 172.21.15.52:0/2013879280 >> 172.21.15.52:0/2013879280 conn(0x7fe39c0fba80 msgr2=0x7fe39c104a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:30:53.019 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:53.021+0000 7fe3a084e640 1 -- 172.21.15.52:0/2013879280 shutdown_connections 2024-03-04T09:30:53.019 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:53.021+0000 7fe3a084e640 1 -- 172.21.15.52:0/2013879280 wait complete. 2024-03-04T09:30:53.030 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-04T09:24:01.742461+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-04T09:28:35.835600+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-04T09:28:36.453099+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-04T09:28:40.168831+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '89', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-04T09:30:53.030 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-04T09:30:53.138 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:53.139+0000 7f9947e39640 1 -- 172.21.15.52:0/1032618348 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f99401020a0 msgr2=0x7f9940108930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:30:53.138 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:53.139+0000 7f9947e39640 1 --2- 172.21.15.52:0/1032618348 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f99401020a0 0x7f9940108930 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f9938009ca0 tx=0x7f993802c0f0 comp rx=0 tx=0).stop 2024-03-04T09:30:53.138 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:53.139+0000 7f9947e39640 1 -- 172.21.15.52:0/1032618348 shutdown_connections 2024-03-04T09:30:53.138 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:53.139+0000 7f9947e39640 1 --2- 172.21.15.52:0/1032618348 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f99401020a0 0x7f9940108930 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:53.138 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:53.139+0000 7f9947e39640 1 --2- 172.21.15.52:0/1032618348 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f99401016e0 0x7f9940101b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:53.138 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:53.139+0000 7f9947e39640 1 --2- 172.21.15.52:0/1032618348 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f99401004f0 0x7f9940100910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:53.138 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:53.139+0000 7f9947e39640 1 -- 172.21.15.52:0/1032618348 >> 172.21.15.52:0/1032618348 conn(0x7f99400fba80 msgr2=0x7f99400fdea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:30:53.138 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:53.139+0000 7f9947e39640 1 -- 172.21.15.52:0/1032618348 shutdown_connections 2024-03-04T09:30:53.139 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:53.139+0000 7f9947e39640 1 -- 172.21.15.52:0/1032618348 wait complete. 2024-03-04T09:30:53.139 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:53.140+0000 7f9947e39640 1 Processor -- start 2024-03-04T09:30:53.139 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:53.140+0000 7f9947e39640 1 -- start start 2024-03-04T09:30:53.139 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:53.140+0000 7f9947e39640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f99401004f0 0x7f9940104480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:30:53.139 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:53.140+0000 7f9947e39640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f99401016e0 0x7f9940102ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:30:53.139 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:53.140+0000 7f9947e39640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f99401020a0 0x7f9940103000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:30:53.139 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:53.140+0000 7f9947e39640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f994010c180 con 0x7f99401004f0 2024-03-04T09:30:53.139 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:53.140+0000 7f9947e39640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f994010c000 con 0x7f99401020a0 2024-03-04T09:30:53.139 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:53.140+0000 7f9947e39640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f994010c300 con 0x7f99401016e0 2024-03-04T09:30:53.139 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:53.140+0000 7f9945bd5640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f99401004f0 0x7f9940104480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:30:53.139 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:53.140+0000 7f9945bd5640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f99401004f0 0x7f9940104480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:58208/0 (socket says 172.21.15.52:58208) 2024-03-04T09:30:53.139 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:53.140+0000 7f9945bd5640 1 -- 172.21.15.52:0/24504963 learned_addr learned my addr 172.21.15.52:0/24504963 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-04T09:30:53.139 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:53.140+0000 7f99453d4640 1 --2- 172.21.15.52:0/24504963 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f99401016e0 0x7f9940102ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:30:53.139 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:53.140+0000 7f99463d6640 1 --2- 172.21.15.52:0/24504963 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f99401020a0 0x7f9940103000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:30:53.140 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:53.140+0000 7f9945bd5640 1 -- 172.21.15.52:0/24504963 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f99401016e0 msgr2=0x7f9940102ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:30:53.140 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:53.140+0000 7f9945bd5640 1 --2- 172.21.15.52:0/24504963 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f99401016e0 0x7f9940102ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:53.140 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:53.140+0000 7f9945bd5640 1 -- 172.21.15.52:0/24504963 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f99401020a0 msgr2=0x7f9940103000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:30:53.140 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:53.140+0000 7f9945bd5640 1 --2- 172.21.15.52:0/24504963 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f99401020a0 0x7f9940103000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:53.140 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:53.140+0000 7f9945bd5640 1 -- 172.21.15.52:0/24504963 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9940103890 con 0x7f99401004f0 2024-03-04T09:30:53.140 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:53.140+0000 7f9945bd5640 1 --2- 172.21.15.52:0/24504963 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f99401004f0 0x7f9940104480 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f993c00e9c0 tx=0x7f993c00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:30:53.140 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:53.141+0000 7f9936ffd640 1 -- 172.21.15.52:0/24504963 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f993c00cd80 con 0x7f99401004f0 2024-03-04T09:30:53.140 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:53.141+0000 7f9936ffd640 1 -- 172.21.15.52:0/24504963 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f993c0035f0 con 0x7f99401004f0 2024-03-04T09:30:53.140 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:53.141+0000 7f9947e39640 1 -- 172.21.15.52:0/24504963 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99401a19c0 con 0x7f99401004f0 2024-03-04T09:30:53.140 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:53.141+0000 7f9936ffd640 1 -- 172.21.15.52:0/24504963 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f993c010690 con 0x7f99401004f0 2024-03-04T09:30:53.140 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:53.141+0000 7f9947e39640 1 -- 172.21.15.52:0/24504963 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99401a1f60 con 0x7f99401004f0 2024-03-04T09:30:53.141 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:53.142+0000 7f9947e39640 1 -- 172.21.15.52:0/24504963 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f99401a21f0 con 0x7f99401004f0 2024-03-04T09:30:53.141 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:53.143+0000 7f9936ffd640 1 -- 172.21.15.52:0/24504963 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) v1 ==== 136756+0+0 (secure 0 0 0) 0x7f993c0037a0 con 0x7f99401004f0 2024-03-04T09:30:53.141 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:53.143+0000 7f9936ffd640 1 --2- 172.21.15.52:0/24504963 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f992c0a1e00 0x7f992c0a42c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:30:53.141 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:53.143+0000 7f9936ffd640 1 -- 172.21.15.52:0/24504963 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9158+0+0 (secure 0 0 0) 0x7f993c0ce510 con 0x7f99401004f0 2024-03-04T09:30:53.141 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:53.143+0000 7f99453d4640 1 --2- 172.21.15.52:0/24504963 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f992c0a1e00 0x7f992c0a42c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:30:53.142 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:53.144+0000 7f99453d4640 1 --2- 172.21.15.52:0/24504963 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f992c0a1e00 0x7f992c0a42c0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f99401041f0 tx=0x7f9930005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:30:53.144 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:53.148+0000 7f9936ffd640 1 -- 172.21.15.52:0/24504963 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f993c09b540 con 0x7f99401004f0 2024-03-04T09:30:53.327 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:53.329+0000 7f9947e39640 1 -- 172.21.15.52:0/24504963 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f99401a23c0 con 0x7f99401004f0 2024-03-04T09:30:53.327 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:53.330+0000 7f9936ffd640 1 -- 172.21.15.52:0/24504963 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v93) v1 ==== 93+0+10 (secure 0 0 0) 0x7f993c09ae40 con 0x7f99401004f0 2024-03-04T09:30:53.327 INFO:teuthology.orchestra.run.smithi052.stdout:pg_num: 1 2024-03-04T09:30:53.329 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:53.332+0000 7f9947e39640 1 -- 172.21.15.52:0/24504963 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f992c0a1e00 msgr2=0x7f992c0a42c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:30:53.329 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:53.332+0000 7f9947e39640 1 --2- 172.21.15.52:0/24504963 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f992c0a1e00 0x7f992c0a42c0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f99401041f0 tx=0x7f9930005f90 comp rx=0 tx=0).stop 2024-03-04T09:30:53.329 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:53.332+0000 7f9947e39640 1 -- 172.21.15.52:0/24504963 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f99401004f0 msgr2=0x7f9940104480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:30:53.329 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:53.332+0000 7f9947e39640 1 --2- 172.21.15.52:0/24504963 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f99401004f0 0x7f9940104480 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f993c00e9c0 tx=0x7f993c00ee80 comp rx=0 tx=0).stop 2024-03-04T09:30:53.329 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:53.332+0000 7f9947e39640 1 -- 172.21.15.52:0/24504963 shutdown_connections 2024-03-04T09:30:53.329 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:53.332+0000 7f9947e39640 1 --2- 172.21.15.52:0/24504963 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f992c0a1e00 0x7f992c0a42c0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:53.329 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:53.332+0000 7f9947e39640 1 --2- 172.21.15.52:0/24504963 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f99401020a0 0x7f9940103000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:53.329 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:53.332+0000 7f9947e39640 1 --2- 172.21.15.52:0/24504963 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f99401016e0 0x7f9940102ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:53.329 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:53.332+0000 7f9947e39640 1 --2- 172.21.15.52:0/24504963 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f99401004f0 0x7f9940104480 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:53.329 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:53.332+0000 7f9947e39640 1 -- 172.21.15.52:0/24504963 >> 172.21.15.52:0/24504963 conn(0x7f99400fba80 msgr2=0x7f99400fd4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:30:53.330 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:53.332+0000 7f9947e39640 1 -- 172.21.15.52:0/24504963 shutdown_connections 2024-03-04T09:30:53.330 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:53.332+0000 7f9947e39640 1 -- 172.21.15.52:0/24504963 wait complete. 2024-03-04T09:30:53.340 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-04T09:30:53.447 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:53.448+0000 7f590d447640 1 -- 172.21.15.52:0/1125188824 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f59081015e0 msgr2=0x7f5908101a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:30:53.448 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:53.448+0000 7f590d447640 1 --2- 172.21.15.52:0/1125188824 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f59081015e0 0x7f5908101a00 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f58f8009900 tx=0x7f58f802bc30 comp rx=0 tx=0).stop 2024-03-04T09:30:53.448 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:53.449+0000 7f590d447640 1 -- 172.21.15.52:0/1125188824 shutdown_connections 2024-03-04T09:30:53.448 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:53.449+0000 7f590d447640 1 --2- 172.21.15.52:0/1125188824 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5908103190 0x7f5908109a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:53.448 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:53.449+0000 7f590d447640 1 --2- 172.21.15.52:0/1125188824 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f59081027d0 0x7f5908102c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:53.448 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:53.449+0000 7f590d447640 1 --2- 172.21.15.52:0/1125188824 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f59081015e0 0x7f5908101a00 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:53.448 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:53.449+0000 7f590d447640 1 -- 172.21.15.52:0/1125188824 >> 172.21.15.52:0/1125188824 conn(0x7f59080fcb70 msgr2=0x7f59080fef90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:30:53.448 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:53.449+0000 7f590d447640 1 -- 172.21.15.52:0/1125188824 shutdown_connections 2024-03-04T09:30:53.448 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:53.449+0000 7f590d447640 1 -- 172.21.15.52:0/1125188824 wait complete. 2024-03-04T09:30:53.448 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:53.449+0000 7f590d447640 1 Processor -- start 2024-03-04T09:30:53.448 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:53.449+0000 7f590d447640 1 -- start start 2024-03-04T09:30:53.448 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:53.449+0000 7f590d447640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f59081015e0 0x7f5908197900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:30:53.448 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:53.449+0000 7f590d447640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f59081027d0 0x7f5908197e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:30:53.449 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:53.449+0000 7f590d447640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5908103190 0x7f590819eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:30:53.449 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:53.449+0000 7f590d447640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f590810d230 con 0x7f5908103190 2024-03-04T09:30:53.449 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:53.449+0000 7f590d447640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f590810d0b0 con 0x7f59081015e0 2024-03-04T09:30:53.449 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:53.449+0000 7f590d447640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f590810d3b0 con 0x7f59081027d0 2024-03-04T09:30:53.449 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:53.449+0000 7f59077fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5908103190 0x7f590819eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:30:53.449 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:53.450+0000 7f59077fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5908103190 0x7f590819eeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:58224/0 (socket says 172.21.15.52:58224) 2024-03-04T09:30:53.449 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:53.450+0000 7f59077fe640 1 -- 172.21.15.52:0/714504996 learned_addr learned my addr 172.21.15.52:0/714504996 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-04T09:30:53.449 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:53.450+0000 7f5906ffd640 1 --2- 172.21.15.52:0/714504996 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f59081015e0 0x7f5908197900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:30:53.449 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:53.450+0000 7f59067fc640 1 --2- 172.21.15.52:0/714504996 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f59081027d0 0x7f5908197e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:30:53.449 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:53.450+0000 7f59077fe640 1 -- 172.21.15.52:0/714504996 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f59081027d0 msgr2=0x7f5908197e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:30:53.449 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:53.450+0000 7f59077fe640 1 --2- 172.21.15.52:0/714504996 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f59081027d0 0x7f5908197e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:53.449 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:53.450+0000 7f59077fe640 1 -- 172.21.15.52:0/714504996 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f59081015e0 msgr2=0x7f5908197900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:30:53.449 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:53.450+0000 7f59077fe640 1 --2- 172.21.15.52:0/714504996 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f59081015e0 0x7f5908197900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:53.449 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:53.450+0000 7f59077fe640 1 -- 172.21.15.52:0/714504996 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f590819f520 con 0x7f5908103190 2024-03-04T09:30:53.449 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:53.450+0000 7f59077fe640 1 --2- 172.21.15.52:0/714504996 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5908103190 0x7f590819eeb0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f590000eb10 tx=0x7f590000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:30:53.450 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:53.450+0000 7f58effff640 1 -- 172.21.15.52:0/714504996 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5900004b20 con 0x7f5908103190 2024-03-04T09:30:53.450 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:53.450+0000 7f58effff640 1 -- 172.21.15.52:0/714504996 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5900004cc0 con 0x7f5908103190 2024-03-04T09:30:53.450 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:53.450+0000 7f590d447640 1 -- 172.21.15.52:0/714504996 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f590819f810 con 0x7f5908103190 2024-03-04T09:30:53.450 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:53.450+0000 7f58effff640 1 -- 172.21.15.52:0/714504996 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5900010760 con 0x7f5908103190 2024-03-04T09:30:53.450 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:53.450+0000 7f590d447640 1 -- 172.21.15.52:0/714504996 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f590819fd10 con 0x7f5908103190 2024-03-04T09:30:53.451 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:53.451+0000 7f590d447640 1 -- 172.21.15.52:0/714504996 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5908108a20 con 0x7f5908103190 2024-03-04T09:30:53.451 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:53.452+0000 7f58effff640 1 -- 172.21.15.52:0/714504996 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) v1 ==== 136756+0+0 (secure 0 0 0) 0x7f5900005020 con 0x7f5908103190 2024-03-04T09:30:53.451 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:53.453+0000 7f58effff640 1 --2- 172.21.15.52:0/714504996 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f58f00a1d80 0x7f58f00a4240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:30:53.451 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:53.453+0000 7f58effff640 1 -- 172.21.15.52:0/714504996 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9158+0+0 (secure 0 0 0) 0x7f59000ce700 con 0x7f5908103190 2024-03-04T09:30:53.451 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:53.453+0000 7f5906ffd640 1 --2- 172.21.15.52:0/714504996 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f58f00a1d80 0x7f58f00a4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:30:53.451 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:53.453+0000 7f5906ffd640 1 --2- 172.21.15.52:0/714504996 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f58f00a1d80 0x7f58f00a4240 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f58f8000c00 tx=0x7f58f8005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:30:53.454 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:53.457+0000 7f58effff640 1 -- 172.21.15.52:0/714504996 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f590009b790 con 0x7f5908103190 2024-03-04T09:30:53.585 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:30:53 smithi052 ceph-mon[32473]: pgmap v359: 129 pgs: 129 active+clean; 641 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s; 2 B/s, 0 keys/s, 0 objects/s recovering 2024-03-04T09:30:53.585 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:30:53 smithi052 ceph-mon[32473]: from='client.? 172.21.15.52:0/1180570880' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-04T09:30:53.585 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:30:53 smithi052 ceph-mon[32473]: from='client.? 172.21.15.52:0/3650944609' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-04T09:30:53.585 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:30:53 smithi052 ceph-mon[32473]: from='client.? 172.21.15.52:0/2013879280' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-04T09:30:53.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:30:53 smithi191 ceph-mon[36816]: pgmap v359: 129 pgs: 129 active+clean; 641 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s; 2 B/s, 0 keys/s, 0 objects/s recovering 2024-03-04T09:30:53.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:30:53 smithi191 ceph-mon[36816]: from='client.? 172.21.15.52:0/1180570880' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-04T09:30:53.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:30:53 smithi191 ceph-mon[36816]: from='client.? 172.21.15.52:0/3650944609' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-04T09:30:53.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:30:53 smithi191 ceph-mon[36816]: from='client.? 172.21.15.52:0/2013879280' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-04T09:30:53.631 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:30:53 smithi106 ceph-mon[36484]: pgmap v359: 129 pgs: 129 active+clean; 641 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s; 2 B/s, 0 keys/s, 0 objects/s recovering 2024-03-04T09:30:53.631 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:30:53 smithi106 ceph-mon[36484]: from='client.? 172.21.15.52:0/1180570880' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-04T09:30:53.631 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:30:53 smithi106 ceph-mon[36484]: from='client.? 172.21.15.52:0/3650944609' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-04T09:30:53.631 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:30:53 smithi106 ceph-mon[36484]: from='client.? 172.21.15.52:0/2013879280' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-04T09:30:53.637 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:53.640+0000 7f590d447640 1 -- 172.21.15.52:0/714504996 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f59080621d0 con 0x7f5908103190 2024-03-04T09:30:53.637 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:53.641+0000 7f58effff640 1 -- 172.21.15.52:0/714504996 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v93) v1 ==== 104+0+11 (secure 0 0 0) 0x7f590009b090 con 0x7f5908103190 2024-03-04T09:30:53.638 INFO:teuthology.orchestra.run.smithi052.stdout:pg_num: 32 2024-03-04T09:30:53.640 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:53.643+0000 7f590d447640 1 -- 172.21.15.52:0/714504996 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f58f00a1d80 msgr2=0x7f58f00a4240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:30:53.640 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:53.643+0000 7f590d447640 1 --2- 172.21.15.52:0/714504996 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f58f00a1d80 0x7f58f00a4240 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f58f8000c00 tx=0x7f58f8005fd0 comp rx=0 tx=0).stop 2024-03-04T09:30:53.640 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:53.643+0000 7f590d447640 1 -- 172.21.15.52:0/714504996 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5908103190 msgr2=0x7f590819eeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:30:53.640 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:53.643+0000 7f590d447640 1 --2- 172.21.15.52:0/714504996 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5908103190 0x7f590819eeb0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f590000eb10 tx=0x7f590000c2d0 comp rx=0 tx=0).stop 2024-03-04T09:30:53.640 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:53.643+0000 7f590d447640 1 -- 172.21.15.52:0/714504996 shutdown_connections 2024-03-04T09:30:53.640 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:53.643+0000 7f590d447640 1 --2- 172.21.15.52:0/714504996 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f58f00a1d80 0x7f58f00a4240 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:53.640 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:53.643+0000 7f590d447640 1 --2- 172.21.15.52:0/714504996 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5908103190 0x7f590819eeb0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:53.641 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:53.643+0000 7f590d447640 1 --2- 172.21.15.52:0/714504996 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f59081027d0 0x7f5908197e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:53.641 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:53.643+0000 7f590d447640 1 --2- 172.21.15.52:0/714504996 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f59081015e0 0x7f5908197900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:53.641 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:53.643+0000 7f590d447640 1 -- 172.21.15.52:0/714504996 >> 172.21.15.52:0/714504996 conn(0x7f59080fcb70 msgr2=0x7f5908105bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:30:53.641 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:53.643+0000 7f590d447640 1 -- 172.21.15.52:0/714504996 shutdown_connections 2024-03-04T09:30:53.641 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:53.643+0000 7f590d447640 1 -- 172.21.15.52:0/714504996 wait complete. 2024-03-04T09:30:53.651 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-04T09:30:53.759 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:53.760+0000 7f6705381640 1 -- 172.21.15.52:0/2852947816 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f67001004f0 msgr2=0x7f6700100910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:30:53.759 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:53.760+0000 7f6705381640 1 --2- 172.21.15.52:0/2852947816 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f67001004f0 0x7f6700100910 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f66f0009900 tx=0x7f66f002bc30 comp rx=0 tx=0).stop 2024-03-04T09:30:53.759 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:53.760+0000 7f6705381640 1 -- 172.21.15.52:0/2852947816 shutdown_connections 2024-03-04T09:30:53.759 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:53.760+0000 7f6705381640 1 --2- 172.21.15.52:0/2852947816 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f67001020a0 0x7f6700108930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:53.760 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:53.760+0000 7f6705381640 1 --2- 172.21.15.52:0/2852947816 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f67001016e0 0x7f6700101b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:53.760 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:53.760+0000 7f6705381640 1 --2- 172.21.15.52:0/2852947816 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f67001004f0 0x7f6700100910 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:53.760 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:53.760+0000 7f6705381640 1 -- 172.21.15.52:0/2852947816 >> 172.21.15.52:0/2852947816 conn(0x7f67000fba80 msgr2=0x7f67000fdea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:30:53.760 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:53.760+0000 7f6705381640 1 -- 172.21.15.52:0/2852947816 shutdown_connections 2024-03-04T09:30:53.760 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:53.760+0000 7f6705381640 1 -- 172.21.15.52:0/2852947816 wait complete. 2024-03-04T09:30:53.760 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:53.761+0000 7f6705381640 1 Processor -- start 2024-03-04T09:30:53.760 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:53.761+0000 7f6705381640 1 -- start start 2024-03-04T09:30:53.760 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:53.761+0000 7f6705381640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f67001004f0 0x7f6700197970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:30:53.760 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:53.761+0000 7f6705381640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f67001016e0 0x7f6700197eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:30:53.760 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:53.761+0000 7f6705381640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f67001020a0 0x7f670019ef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:30:53.760 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:53.761+0000 7f6705381640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f670010d250 con 0x7f67001016e0 2024-03-04T09:30:53.760 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:53.761+0000 7f6705381640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f670010d0d0 con 0x7f67001020a0 2024-03-04T09:30:53.761 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:53.761+0000 7f6705381640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f670010d3d0 con 0x7f67001004f0 2024-03-04T09:30:53.761 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:53.761+0000 7f66feffd640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f67001004f0 0x7f6700197970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:30:53.761 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:53.761+0000 7f66f7fff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f67001016e0 0x7f6700197eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:30:53.761 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:53.761+0000 7f66feffd640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f67001004f0 0x7f6700197970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.52:47268/0 (socket says 172.21.15.52:47268) 2024-03-04T09:30:53.761 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:53.761+0000 7f66feffd640 1 -- 172.21.15.52:0/960956854 learned_addr learned my addr 172.21.15.52:0/960956854 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-04T09:30:53.761 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:53.761+0000 7f66ff7fe640 1 --2- 172.21.15.52:0/960956854 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f67001020a0 0x7f670019ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:30:53.761 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:53.761+0000 7f66feffd640 1 -- 172.21.15.52:0/960956854 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f67001020a0 msgr2=0x7f670019ef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:30:53.761 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:53.761+0000 7f66feffd640 1 --2- 172.21.15.52:0/960956854 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f67001020a0 0x7f670019ef20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:53.761 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:53.761+0000 7f66feffd640 1 -- 172.21.15.52:0/960956854 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f67001016e0 msgr2=0x7f6700197eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:30:53.761 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:53.761+0000 7f66feffd640 1 --2- 172.21.15.52:0/960956854 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f67001016e0 0x7f6700197eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:53.761 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:53.761+0000 7f66feffd640 1 -- 172.21.15.52:0/960956854 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f670019f590 con 0x7f67001004f0 2024-03-04T09:30:53.761 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:53.761+0000 7f66f7fff640 1 --2- 172.21.15.52:0/960956854 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f67001016e0 0x7f6700197eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-04T09:30:53.761 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:53.761+0000 7f66feffd640 1 --2- 172.21.15.52:0/960956854 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f67001004f0 0x7f6700197970 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f66f0000c00 tx=0x7f66f002c640 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:30:53.761 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:53.762+0000 7f66fcff9640 1 -- 172.21.15.52:0/960956854 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66f0041070 con 0x7f67001004f0 2024-03-04T09:30:53.762 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:53.762+0000 7f6705381640 1 -- 172.21.15.52:0/960956854 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f670019f820 con 0x7f67001004f0 2024-03-04T09:30:53.762 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:53.762+0000 7f66fcff9640 1 -- 172.21.15.52:0/960956854 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f66f0004450 con 0x7f67001004f0 2024-03-04T09:30:53.762 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:53.762+0000 7f66fcff9640 1 -- 172.21.15.52:0/960956854 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66f0003df0 con 0x7f67001004f0 2024-03-04T09:30:53.762 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:53.762+0000 7f6705381640 1 -- 172.21.15.52:0/960956854 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f670019fd60 con 0x7f67001004f0 2024-03-04T09:30:53.762 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:53.763+0000 7f6705381640 1 -- 172.21.15.52:0/960956854 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6700107930 con 0x7f67001004f0 2024-03-04T09:30:53.763 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:53.764+0000 7f66fcff9640 1 -- 172.21.15.52:0/960956854 <== mon.1 v2:172.21.15.191:3300/0 4 ==== mgrmap(e 18) v1 ==== 136756+0+0 (secure 0 0 0) 0x7f66f0044070 con 0x7f67001004f0 2024-03-04T09:30:53.763 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:53.764+0000 7f66fcff9640 1 --2- 172.21.15.52:0/960956854 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f66ec0a1d80 0x7f66ec0a4240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:30:53.763 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:53.764+0000 7f66f7fff640 1 --2- 172.21.15.52:0/960956854 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f66ec0a1d80 0x7f66ec0a4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:30:53.763 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:53.764+0000 7f66fcff9640 1 -- 172.21.15.52:0/960956854 <== mon.1 v2:172.21.15.191:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9158+0+0 (secure 0 0 0) 0x7f66f00efe30 con 0x7f67001004f0 2024-03-04T09:30:53.763 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:53.765+0000 7f66f7fff640 1 --2- 172.21.15.52:0/960956854 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f66ec0a1d80 0x7f66ec0a4240 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f6700198e80 tx=0x7f66e8005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:30:53.769 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:53.773+0000 7f66fcff9640 1 -- 172.21.15.52:0/960956854 <== mon.1 v2:172.21.15.191:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f66f00bcef0 con 0x7f67001004f0 2024-03-04T09:30:53.953 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:53.956+0000 7f6705381640 1 -- 172.21.15.52:0/960956854 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f67000621d0 con 0x7f67001004f0 2024-03-04T09:30:53.954 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:53.957+0000 7f66fcff9640 1 -- 172.21.15.52:0/960956854 <== mon.1 v2:172.21.15.191:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v93) v1 ==== 100+0+11 (secure 0 0 0) 0x7f66f00bc7f0 con 0x7f67001004f0 2024-03-04T09:30:53.954 INFO:teuthology.orchestra.run.smithi052.stdout:pg_num: 64 2024-03-04T09:30:53.956 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:53.960+0000 7f6705381640 1 -- 172.21.15.52:0/960956854 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f66ec0a1d80 msgr2=0x7f66ec0a4240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:30:53.956 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:53.960+0000 7f6705381640 1 --2- 172.21.15.52:0/960956854 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f66ec0a1d80 0x7f66ec0a4240 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f6700198e80 tx=0x7f66e8005f90 comp rx=0 tx=0).stop 2024-03-04T09:30:53.957 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:53.960+0000 7f6705381640 1 -- 172.21.15.52:0/960956854 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f67001004f0 msgr2=0x7f6700197970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:30:53.957 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:53.960+0000 7f6705381640 1 --2- 172.21.15.52:0/960956854 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f67001004f0 0x7f6700197970 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f66f0000c00 tx=0x7f66f002c640 comp rx=0 tx=0).stop 2024-03-04T09:30:53.957 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:53.960+0000 7f6705381640 1 -- 172.21.15.52:0/960956854 shutdown_connections 2024-03-04T09:30:53.957 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:53.960+0000 7f6705381640 1 --2- 172.21.15.52:0/960956854 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f66ec0a1d80 0x7f66ec0a4240 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:53.957 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:53.960+0000 7f6705381640 1 --2- 172.21.15.52:0/960956854 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f67001020a0 0x7f670019ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:53.957 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:53.960+0000 7f6705381640 1 --2- 172.21.15.52:0/960956854 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f67001016e0 0x7f6700197eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:53.957 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:53.960+0000 7f6705381640 1 --2- 172.21.15.52:0/960956854 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f67001004f0 0x7f6700197970 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:53.957 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:53.960+0000 7f6705381640 1 -- 172.21.15.52:0/960956854 >> 172.21.15.52:0/960956854 conn(0x7f67000fba80 msgr2=0x7f6700104b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:30:53.957 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:53.960+0000 7f6705381640 1 -- 172.21.15.52:0/960956854 shutdown_connections 2024-03-04T09:30:53.957 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:53.960+0000 7f6705381640 1 -- 172.21.15.52:0/960956854 wait complete. 2024-03-04T09:30:53.967 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-04T09:30:54.075 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:54.076+0000 7f33ac11e640 1 -- 172.21.15.52:0/319696063 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f33a41020b0 msgr2=0x7f33a41044a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:30:54.075 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:54.076+0000 7f33ac11e640 1 --2- 172.21.15.52:0/319696063 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f33a41020b0 0x7f33a41044a0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f33a0009900 tx=0x7f33a002bc30 comp rx=0 tx=0).stop 2024-03-04T09:30:54.075 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:54.076+0000 7f33ac11e640 1 -- 172.21.15.52:0/319696063 shutdown_connections 2024-03-04T09:30:54.075 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:54.076+0000 7f33ac11e640 1 --2- 172.21.15.52:0/319696063 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f33a4107310 0x7f33a4109700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:54.076 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:54.076+0000 7f33ac11e640 1 --2- 172.21.15.52:0/319696063 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f33a41049e0 0x7f33a4106dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:54.076 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:54.076+0000 7f33ac11e640 1 --2- 172.21.15.52:0/319696063 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f33a41020b0 0x7f33a41044a0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:54.076 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:54.076+0000 7f33ac11e640 1 -- 172.21.15.52:0/319696063 >> 172.21.15.52:0/319696063 conn(0x7f33a40fba60 msgr2=0x7f33a40fde80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:30:54.076 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:54.076+0000 7f33ac11e640 1 -- 172.21.15.52:0/319696063 shutdown_connections 2024-03-04T09:30:54.076 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:54.076+0000 7f33ac11e640 1 -- 172.21.15.52:0/319696063 wait complete. 2024-03-04T09:30:54.076 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:54.077+0000 7f33ac11e640 1 Processor -- start 2024-03-04T09:30:54.076 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:54.077+0000 7f33ac11e640 1 -- start start 2024-03-04T09:30:54.076 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:54.077+0000 7f33ac11e640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f33a41020b0 0x7f33a4197940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:30:54.076 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:54.077+0000 7f33ac11e640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f33a41049e0 0x7f33a4197e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:30:54.076 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:54.077+0000 7f33ac11e640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f33a4107310 0x7f33a419eef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:30:54.076 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:54.077+0000 7f33ac11e640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33a410d260 con 0x7f33a41049e0 2024-03-04T09:30:54.076 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:54.077+0000 7f33ac11e640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33a410d0e0 con 0x7f33a41020b0 2024-03-04T09:30:54.077 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:54.077+0000 7f33ac11e640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33a410d3e0 con 0x7f33a4107310 2024-03-04T09:30:54.077 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:54.077+0000 7f33a96b9640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f33a41049e0 0x7f33a4197e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:30:54.077 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:54.077+0000 7f33a96b9640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f33a41049e0 0x7f33a4197e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:58256/0 (socket says 172.21.15.52:58256) 2024-03-04T09:30:54.077 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:54.077+0000 7f33a96b9640 1 -- 172.21.15.52:0/378790348 learned_addr learned my addr 172.21.15.52:0/378790348 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-04T09:30:54.077 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:54.077+0000 7f33a9eba640 1 --2- 172.21.15.52:0/378790348 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f33a41020b0 0x7f33a4197940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:30:54.077 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:54.077+0000 7f33aa6bb640 1 --2- 172.21.15.52:0/378790348 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f33a4107310 0x7f33a419eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:30:54.077 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:54.078+0000 7f33a96b9640 1 -- 172.21.15.52:0/378790348 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f33a4107310 msgr2=0x7f33a419eef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:30:54.077 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:54.078+0000 7f33a96b9640 1 --2- 172.21.15.52:0/378790348 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f33a4107310 0x7f33a419eef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:54.077 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:54.078+0000 7f33a96b9640 1 -- 172.21.15.52:0/378790348 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f33a41020b0 msgr2=0x7f33a4197940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:30:54.077 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:54.078+0000 7f33a96b9640 1 --2- 172.21.15.52:0/378790348 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f33a41020b0 0x7f33a4197940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:54.077 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:54.078+0000 7f33a96b9640 1 -- 172.21.15.52:0/378790348 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33a419f560 con 0x7f33a41049e0 2024-03-04T09:30:54.077 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:54.078+0000 7f33a96b9640 1 --2- 172.21.15.52:0/378790348 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f33a41049e0 0x7f33a4197e80 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f339400e9c0 tx=0x7f339400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:30:54.077 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:54.078+0000 7f339affd640 1 -- 172.21.15.52:0/378790348 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f339400cd80 con 0x7f33a41049e0 2024-03-04T09:30:54.077 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:54.078+0000 7f339affd640 1 -- 172.21.15.52:0/378790348 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3394005500 con 0x7f33a41049e0 2024-03-04T09:30:54.077 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:54.078+0000 7f339affd640 1 -- 172.21.15.52:0/378790348 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3394010690 con 0x7f33a41049e0 2024-03-04T09:30:54.078 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:54.078+0000 7f33ac11e640 1 -- 172.21.15.52:0/378790348 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33a419f850 con 0x7f33a41049e0 2024-03-04T09:30:54.078 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:54.078+0000 7f33ac11e640 1 -- 172.21.15.52:0/378790348 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33a419fdf0 con 0x7f33a41049e0 2024-03-04T09:30:54.079 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:54.079+0000 7f33ac11e640 1 -- 172.21.15.52:0/378790348 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f33a40fd550 con 0x7f33a41049e0 2024-03-04T09:30:54.079 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:54.080+0000 7f339affd640 1 -- 172.21.15.52:0/378790348 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) v1 ==== 136756+0+0 (secure 0 0 0) 0x7f33940046c0 con 0x7f33a41049e0 2024-03-04T09:30:54.079 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:54.081+0000 7f339affd640 1 --2- 172.21.15.52:0/378790348 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f33900a1e00 0x7f33900a42c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:30:54.079 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:54.081+0000 7f33a9eba640 1 --2- 172.21.15.52:0/378790348 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f33900a1e00 0x7f33900a42c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:30:54.079 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:54.081+0000 7f339affd640 1 -- 172.21.15.52:0/378790348 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9158+0+0 (secure 0 0 0) 0x7f33940ce690 con 0x7f33a41049e0 2024-03-04T09:30:54.079 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:54.081+0000 7f33a9eba640 1 --2- 172.21.15.52:0/378790348 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f33900a1e00 0x7f33900a42c0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f33a002c7b0 tx=0x7f33a0005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:30:54.081 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:54.084+0000 7f339affd640 1 -- 172.21.15.52:0/378790348 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f3394053280 con 0x7f33a41049e0 2024-03-04T09:30:54.262 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:54.265+0000 7f33ac11e640 1 -- 172.21.15.52:0/378790348 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f33a402d240 con 0x7f33a41049e0 2024-03-04T09:30:54.263 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:54.266+0000 7f339affd640 1 -- 172.21.15.52:0/378790348 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v93) v1 ==== 103+0+11 (secure 0 0 0) 0x7f339409b6c0 con 0x7f33a41049e0 2024-03-04T09:30:54.263 INFO:teuthology.orchestra.run.smithi052.stdout:pg_num: 32 2024-03-04T09:30:54.265 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:54.267+0000 7f33ac11e640 1 -- 172.21.15.52:0/378790348 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f33900a1e00 msgr2=0x7f33900a42c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:30:54.265 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:54.267+0000 7f33ac11e640 1 --2- 172.21.15.52:0/378790348 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f33900a1e00 0x7f33900a42c0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f33a002c7b0 tx=0x7f33a0005fd0 comp rx=0 tx=0).stop 2024-03-04T09:30:54.265 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:54.267+0000 7f33ac11e640 1 -- 172.21.15.52:0/378790348 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f33a41049e0 msgr2=0x7f33a4197e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:30:54.265 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:54.267+0000 7f33ac11e640 1 --2- 172.21.15.52:0/378790348 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f33a41049e0 0x7f33a4197e80 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f339400e9c0 tx=0x7f339400ee80 comp rx=0 tx=0).stop 2024-03-04T09:30:54.265 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:54.268+0000 7f33ac11e640 1 -- 172.21.15.52:0/378790348 shutdown_connections 2024-03-04T09:30:54.265 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:54.268+0000 7f33ac11e640 1 --2- 172.21.15.52:0/378790348 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f33900a1e00 0x7f33900a42c0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:54.265 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:54.268+0000 7f33ac11e640 1 --2- 172.21.15.52:0/378790348 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f33a4107310 0x7f33a419eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:54.265 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:54.268+0000 7f33ac11e640 1 --2- 172.21.15.52:0/378790348 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f33a41049e0 0x7f33a4197e80 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:54.265 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:54.268+0000 7f33ac11e640 1 --2- 172.21.15.52:0/378790348 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f33a41020b0 0x7f33a4197940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:54.265 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:54.268+0000 7f33ac11e640 1 -- 172.21.15.52:0/378790348 >> 172.21.15.52:0/378790348 conn(0x7f33a40fba60 msgr2=0x7f33a4102970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:30:54.265 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:54.268+0000 7f33ac11e640 1 -- 172.21.15.52:0/378790348 shutdown_connections 2024-03-04T09:30:54.265 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:54.268+0000 7f33ac11e640 1 -- 172.21.15.52:0/378790348 wait complete. 2024-03-04T09:30:54.276 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T09:30:54.383 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:54.384+0000 7fb1a7278640 1 -- 172.21.15.52:0/3551451404 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb1a0100030 msgr2=0x7fb1a01004e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:30:54.383 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:54.384+0000 7fb1a7278640 1 --2- 172.21.15.52:0/3551451404 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb1a0100030 0x7fb1a01004e0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fb198009ca0 tx=0x7fb19802c0f0 comp rx=0 tx=0).stop 2024-03-04T09:30:54.383 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:54.384+0000 7fb1a7278640 1 -- 172.21.15.52:0/3551451404 shutdown_connections 2024-03-04T09:30:54.384 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:54.384+0000 7fb1a7278640 1 --2- 172.21.15.52:0/3551451404 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb1a0100030 0x7fb1a01004e0 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:54.384 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:54.384+0000 7fb1a7278640 1 --2- 172.21.15.52:0/3551451404 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb1a00ff670 0x7fb1a00ffaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:54.384 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:54.384+0000 7fb1a7278640 1 --2- 172.21.15.52:0/3551451404 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb1a0101b80 0x7fb1a0101fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:54.384 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:54.384+0000 7fb1a7278640 1 -- 172.21.15.52:0/3551451404 >> 172.21.15.52:0/3551451404 conn(0x7fb1a00fb1f0 msgr2=0x7fb1a00fd630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:30:54.384 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:54.384+0000 7fb1a7278640 1 -- 172.21.15.52:0/3551451404 shutdown_connections 2024-03-04T09:30:54.384 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:54.384+0000 7fb1a7278640 1 -- 172.21.15.52:0/3551451404 wait complete. 2024-03-04T09:30:54.384 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:54.385+0000 7fb1a7278640 1 Processor -- start 2024-03-04T09:30:54.384 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:54.385+0000 7fb1a7278640 1 -- start start 2024-03-04T09:30:54.384 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:54.385+0000 7fb1a7278640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb1a00ff670 0x7fb1a0199a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:30:54.384 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:54.385+0000 7fb1a7278640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb1a0100030 0x7fb1a0199f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:30:54.384 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:54.385+0000 7fb1a7278640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb1a0101b80 0x7fb1a01a1000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:30:54.384 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:54.385+0000 7fb1a7278640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1a010f470 con 0x7fb1a0101b80 2024-03-04T09:30:54.384 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:54.385+0000 7fb1a7278640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1a010f2f0 con 0x7fb1a0100030 2024-03-04T09:30:54.385 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:54.385+0000 7fb1a7278640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1a010f5f0 con 0x7fb1a00ff670 2024-03-04T09:30:54.385 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:54.385+0000 7fb1a5815640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb1a0101b80 0x7fb1a01a1000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:30:54.385 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:54.385+0000 7fb1a5014640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb1a00ff670 0x7fb1a0199a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:30:54.385 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:54.385+0000 7fb1a5014640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb1a00ff670 0x7fb1a0199a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.52:40736/0 (socket says 172.21.15.52:40736) 2024-03-04T09:30:54.385 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:54.385+0000 7fb1a5014640 1 -- 172.21.15.52:0/4142970534 learned_addr learned my addr 172.21.15.52:0/4142970534 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-04T09:30:54.385 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:54.385+0000 7fb1a4813640 1 --2- 172.21.15.52:0/4142970534 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb1a0100030 0x7fb1a0199f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:30:54.385 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:54.385+0000 7fb1a5815640 1 -- 172.21.15.52:0/4142970534 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb1a00ff670 msgr2=0x7fb1a0199a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:30:54.385 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:54.385+0000 7fb1a5815640 1 --2- 172.21.15.52:0/4142970534 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb1a00ff670 0x7fb1a0199a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:54.385 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:54.385+0000 7fb1a5815640 1 -- 172.21.15.52:0/4142970534 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb1a0100030 msgr2=0x7fb1a0199f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:30:54.385 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:54.385+0000 7fb1a5815640 1 --2- 172.21.15.52:0/4142970534 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb1a0100030 0x7fb1a0199f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:54.385 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:54.385+0000 7fb1a5815640 1 -- 172.21.15.52:0/4142970534 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1a01a1670 con 0x7fb1a0101b80 2024-03-04T09:30:54.385 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:54.385+0000 7fb1a5815640 1 --2- 172.21.15.52:0/4142970534 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb1a0101b80 0x7fb1a01a1000 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fb19802c630 tx=0x7fb198032f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:30:54.385 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:54.386+0000 7fb1967fc640 1 -- 172.21.15.52:0/4142970534 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb198041020 con 0x7fb1a0101b80 2024-03-04T09:30:54.385 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:54.386+0000 7fb1967fc640 1 -- 172.21.15.52:0/4142970534 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb198034cd0 con 0x7fb1a0101b80 2024-03-04T09:30:54.386 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:54.386+0000 7fb1a7278640 1 -- 172.21.15.52:0/4142970534 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1a01a1900 con 0x7fb1a0101b80 2024-03-04T09:30:54.386 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:54.386+0000 7fb1967fc640 1 -- 172.21.15.52:0/4142970534 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb198039040 con 0x7fb1a0101b80 2024-03-04T09:30:54.386 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:54.386+0000 7fb1a7278640 1 -- 172.21.15.52:0/4142970534 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1a01a1dc0 con 0x7fb1a0101b80 2024-03-04T09:30:54.386 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:54.387+0000 7fb1a7278640 1 -- 172.21.15.52:0/4142970534 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb1a010b590 con 0x7fb1a0101b80 2024-03-04T09:30:54.386 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:54.388+0000 7fb1967fc640 1 -- 172.21.15.52:0/4142970534 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) v1 ==== 136756+0+0 (secure 0 0 0) 0x7fb198044020 con 0x7fb1a0101b80 2024-03-04T09:30:54.387 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:54.388+0000 7fb1967fc640 1 --2- 172.21.15.52:0/4142970534 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7fb18c0a1d80 0x7fb18c0a4240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:30:54.387 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:54.389+0000 7fb1a5014640 1 --2- 172.21.15.52:0/4142970534 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7fb18c0a1d80 0x7fb18c0a4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:30:54.387 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:54.389+0000 7fb1967fc640 1 -- 172.21.15.52:0/4142970534 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9158+0+0 (secure 0 0 0) 0x7fb1980f0320 con 0x7fb1a0101b80 2024-03-04T09:30:54.387 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:54.389+0000 7fb1a5014640 1 --2- 172.21.15.52:0/4142970534 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7fb18c0a1d80 0x7fb18c0a4240 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fb19c009c90 tx=0x7fb19c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:30:54.389 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:54.393+0000 7fb1967fc640 1 -- 172.21.15.52:0/4142970534 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fb19804b020 con 0x7fb1a0101b80 2024-03-04T09:30:54.584 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:30:54 smithi052 ceph-mon[32473]: from='client.? 172.21.15.52:0/24504963' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-04T09:30:54.585 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:30:54 smithi052 ceph-mon[32473]: from='client.? 172.21.15.52:0/714504996' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-04T09:30:54.585 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:30:54 smithi052 ceph-mon[32473]: from='client.? 172.21.15.52:0/960956854' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-04T09:30:54.585 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:30:54 smithi052 ceph-mon[32473]: from='client.? 172.21.15.52:0/378790348' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-04T09:30:54.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:30:54 smithi191 ceph-mon[36816]: from='client.? 172.21.15.52:0/24504963' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-04T09:30:54.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:30:54 smithi191 ceph-mon[36816]: from='client.? 172.21.15.52:0/714504996' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-04T09:30:54.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:30:54 smithi191 ceph-mon[36816]: from='client.? 172.21.15.52:0/960956854' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-04T09:30:54.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:30:54 smithi191 ceph-mon[36816]: from='client.? 172.21.15.52:0/378790348' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-04T09:30:54.609 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:54.612+0000 7fb1a7278640 1 -- 172.21.15.52:0/4142970534 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb1a002d1f0 con 0x7fb1a0101b80 2024-03-04T09:30:54.610 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:54.613+0000 7fb1967fc640 1 -- 172.21.15.52:0/4142970534 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+11406 (secure 0 0 0) 0x7fb1980bd420 con 0x7fb1a0101b80 2024-03-04T09:30:54.610 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-03-04T09:30:54.610 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6835/1007961019","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6834","nonce":1007961019},{"type":"v1","addr":"172.21.15.52:6835","nonce":1007961019}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6837/3753881283","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6836","nonce":3753881283},{"type":"v1","addr":"172.21.15.52:6837","nonce":3753881283}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14628,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6839/3467897808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6838","nonce":3467897808},{"type":"v1","addr":"172.21.15.52:6839","nonce":3467897808}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14640,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6841/2804292291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6840","nonce":2804292291},{"type":"v1","addr":"172.21.15.52:6841","nonce":2804292291}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24407,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6833/1818480323","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6832","nonce":1818480323},{"type":"v1","addr":"172.21.15.106:6833","nonce":1818480323}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24412,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6833/765768204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":765768204},{"type":"v1","addr":"172.21.15.191:6833","nonce":765768204}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-04T09:28:37.264970+0000","modified":"2024-03-04T09:29:04.001848+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24506,"mds_1":24490,"mds_2":24473},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24440":{"gid":24440,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.106:6835/1541069897","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6834","nonce":1541069897},{"type":"v1","addr":"172.21.15.106:6835","nonce":1541069897}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24451":{"gid":24451,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.191:6835/105048699","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":105048699},{"type":"v1","addr":"172.21.15.191:6835","nonce":105048699}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24463":{"gid":24463,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.191:6837/4071064225","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6836","nonce":4071064225},{"type":"v1","addr":"172.21.15.191:6837","nonce":4071064225}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24473":{"gid":24473,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.106:6837/2901755066","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6836","nonce":2901755066},{"type":"v1","addr":"172.21.15.106:6837","nonce":2901755066}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24490":{"gid":24490,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.191:6839/2194324891","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6838","nonce":2194324891},{"type":"v1","addr":"172.21.15.191:6839","nonce":2194324891}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24506":{"gid":24506,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.106:6839/1995853093","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6838","nonce":1995853093},{"type":"v1","addr":"172.21.15.106:6839","nonce":1995853093}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24506,"qdb_cluster":[24473,24490,24506]},"id":1}]} 2024-03-04T09:30:54.610 INFO:teuthology.orchestra.run.smithi052.stderr:dumped fsmap epoch 24 2024-03-04T09:30:54.612 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:54.616+0000 7fb1a7278640 1 -- 172.21.15.52:0/4142970534 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7fb18c0a1d80 msgr2=0x7fb18c0a4240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:30:54.613 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:54.616+0000 7fb1a7278640 1 --2- 172.21.15.52:0/4142970534 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7fb18c0a1d80 0x7fb18c0a4240 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fb19c009c90 tx=0x7fb19c005f90 comp rx=0 tx=0).stop 2024-03-04T09:30:54.613 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:54.616+0000 7fb1a7278640 1 -- 172.21.15.52:0/4142970534 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb1a0101b80 msgr2=0x7fb1a01a1000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:30:54.613 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:54.616+0000 7fb1a7278640 1 --2- 172.21.15.52:0/4142970534 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb1a0101b80 0x7fb1a01a1000 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fb19802c630 tx=0x7fb198032f20 comp rx=0 tx=0).stop 2024-03-04T09:30:54.613 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:54.616+0000 7fb1a7278640 1 -- 172.21.15.52:0/4142970534 shutdown_connections 2024-03-04T09:30:54.613 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:54.616+0000 7fb1a7278640 1 --2- 172.21.15.52:0/4142970534 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7fb18c0a1d80 0x7fb18c0a4240 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:54.613 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:54.616+0000 7fb1a7278640 1 --2- 172.21.15.52:0/4142970534 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb1a0101b80 0x7fb1a01a1000 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:54.613 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:54.616+0000 7fb1a7278640 1 --2- 172.21.15.52:0/4142970534 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb1a0100030 0x7fb1a0199f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:54.613 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:54.616+0000 7fb1a7278640 1 --2- 172.21.15.52:0/4142970534 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb1a00ff670 0x7fb1a0199a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:54.613 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:54.616+0000 7fb1a7278640 1 -- 172.21.15.52:0/4142970534 >> 172.21.15.52:0/4142970534 conn(0x7fb1a00fb1f0 msgr2=0x7fb1a00fc8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:30:54.613 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:54.616+0000 7fb1a7278640 1 -- 172.21.15.52:0/4142970534 shutdown_connections 2024-03-04T09:30:54.613 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:54.616+0000 7fb1a7278640 1 -- 172.21.15.52:0/4142970534 wait complete. 2024-03-04T09:30:54.624 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-04T09:30:54.631 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:30:54 smithi106 ceph-mon[36484]: from='client.? 172.21.15.52:0/24504963' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-04T09:30:54.631 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:30:54 smithi106 ceph-mon[36484]: from='client.? 172.21.15.52:0/714504996' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-04T09:30:54.631 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:30:54 smithi106 ceph-mon[36484]: from='client.? 172.21.15.52:0/960956854' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-04T09:30:54.631 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:30:54 smithi106 ceph-mon[36484]: from='client.? 172.21.15.52:0/378790348' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-04T09:30:54.729 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:54.729+0000 7f6d426c9640 1 -- 172.21.15.52:0/1049124574 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6d3c1016e0 msgr2=0x7f6d3c101b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:30:54.729 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:54.729+0000 7f6d426c9640 1 --2- 172.21.15.52:0/1049124574 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6d3c1016e0 0x7f6d3c101b60 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f6d30009900 tx=0x7f6d3002bc30 comp rx=0 tx=0).stop 2024-03-04T09:30:54.729 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:54.729+0000 7f6d426c9640 1 -- 172.21.15.52:0/1049124574 shutdown_connections 2024-03-04T09:30:54.729 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:54.729+0000 7f6d426c9640 1 --2- 172.21.15.52:0/1049124574 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6d3c1020a0 0x7f6d3c108930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:54.730 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:54.729+0000 7f6d426c9640 1 --2- 172.21.15.52:0/1049124574 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6d3c1016e0 0x7f6d3c101b60 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:54.730 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:54.729+0000 7f6d426c9640 1 --2- 172.21.15.52:0/1049124574 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6d3c1004f0 0x7f6d3c100910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:54.730 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:54.729+0000 7f6d426c9640 1 -- 172.21.15.52:0/1049124574 >> 172.21.15.52:0/1049124574 conn(0x7f6d3c0fba80 msgr2=0x7f6d3c0fdea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:30:54.730 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:54.729+0000 7f6d426c9640 1 -- 172.21.15.52:0/1049124574 shutdown_connections 2024-03-04T09:30:54.730 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:54.729+0000 7f6d426c9640 1 -- 172.21.15.52:0/1049124574 wait complete. 2024-03-04T09:30:54.730 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:54.730+0000 7f6d426c9640 1 Processor -- start 2024-03-04T09:30:54.730 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:54.730+0000 7f6d426c9640 1 -- start start 2024-03-04T09:30:54.730 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:54.730+0000 7f6d426c9640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6d3c1004f0 0x7f6d3c1967d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:30:54.730 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:54.730+0000 7f6d426c9640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6d3c1016e0 0x7f6d3c196d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:30:54.730 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:54.730+0000 7f6d426c9640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6d3c1020a0 0x7f6d3c19dd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:30:54.730 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:54.730+0000 7f6d426c9640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d3c10c180 con 0x7f6d3c1004f0 2024-03-04T09:30:54.731 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:54.730+0000 7f6d426c9640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d3c10c000 con 0x7f6d3c1016e0 2024-03-04T09:30:54.731 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:54.730+0000 7f6d426c9640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d3c10c300 con 0x7f6d3c1020a0 2024-03-04T09:30:54.731 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:54.730+0000 7f6d3bfff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6d3c1004f0 0x7f6d3c1967d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:30:54.731 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:54.730+0000 7f6d3bfff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6d3c1004f0 0x7f6d3c1967d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:46148/0 (socket says 172.21.15.52:46148) 2024-03-04T09:30:54.731 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:54.730+0000 7f6d3bfff640 1 -- 172.21.15.52:0/389816542 learned_addr learned my addr 172.21.15.52:0/389816542 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-04T09:30:54.731 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:54.731+0000 7f6d3b7fe640 1 --2- 172.21.15.52:0/389816542 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6d3c1016e0 0x7f6d3c196d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:30:54.731 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:54.731+0000 7f6d40c66640 1 --2- 172.21.15.52:0/389816542 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6d3c1020a0 0x7f6d3c19dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:30:54.731 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:54.731+0000 7f6d3bfff640 1 -- 172.21.15.52:0/389816542 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6d3c1020a0 msgr2=0x7f6d3c19dd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:30:54.731 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:54.731+0000 7f6d3bfff640 1 --2- 172.21.15.52:0/389816542 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6d3c1020a0 0x7f6d3c19dd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:54.731 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:54.731+0000 7f6d3bfff640 1 -- 172.21.15.52:0/389816542 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6d3c1016e0 msgr2=0x7f6d3c196d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:30:54.731 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:54.731+0000 7f6d3bfff640 1 --2- 172.21.15.52:0/389816542 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6d3c1016e0 0x7f6d3c196d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:54.731 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:54.731+0000 7f6d3bfff640 1 -- 172.21.15.52:0/389816542 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d3c19e3f0 con 0x7f6d3c1004f0 2024-03-04T09:30:54.731 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:54.731+0000 7f6d3bfff640 1 --2- 172.21.15.52:0/389816542 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6d3c1004f0 0x7f6d3c1967d0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f6d2c00e9c0 tx=0x7f6d2c00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:30:54.732 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:54.731+0000 7f6d397fa640 1 -- 172.21.15.52:0/389816542 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d2c00cd80 con 0x7f6d3c1004f0 2024-03-04T09:30:54.732 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:54.731+0000 7f6d397fa640 1 -- 172.21.15.52:0/389816542 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6d2c0035f0 con 0x7f6d3c1004f0 2024-03-04T09:30:54.732 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:54.731+0000 7f6d426c9640 1 -- 172.21.15.52:0/389816542 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d3c19e6e0 con 0x7f6d3c1004f0 2024-03-04T09:30:54.732 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:54.731+0000 7f6d397fa640 1 -- 172.21.15.52:0/389816542 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d2c010690 con 0x7f6d3c1004f0 2024-03-04T09:30:54.732 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:54.731+0000 7f6d426c9640 1 -- 172.21.15.52:0/389816542 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d3c19ec50 con 0x7f6d3c1004f0 2024-03-04T09:30:54.732 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:54.732+0000 7f6d426c9640 1 -- 172.21.15.52:0/389816542 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6d3c107890 con 0x7f6d3c1004f0 2024-03-04T09:30:54.733 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:54.733+0000 7f6d397fa640 1 -- 172.21.15.52:0/389816542 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) v1 ==== 136756+0+0 (secure 0 0 0) 0x7f6d2c0037a0 con 0x7f6d3c1004f0 2024-03-04T09:30:54.733 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:54.734+0000 7f6d397fa640 1 --2- 172.21.15.52:0/389816542 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f6d240a1e00 0x7f6d240a42c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:30:54.733 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:54.734+0000 7f6d3b7fe640 1 --2- 172.21.15.52:0/389816542 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f6d240a1e00 0x7f6d240a42c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:30:54.733 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:54.734+0000 7f6d397fa640 1 -- 172.21.15.52:0/389816542 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9158+0+0 (secure 0 0 0) 0x7f6d2c0ce520 con 0x7f6d3c1004f0 2024-03-04T09:30:54.733 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:54.734+0000 7f6d3b7fe640 1 --2- 172.21.15.52:0/389816542 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f6d240a1e00 0x7f6d240a42c0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f6d3002c7b0 tx=0x7f6d30005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:30:54.734 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:54.737+0000 7f6d397fa640 1 -- 172.21.15.52:0/389816542 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f6d2c09b640 con 0x7f6d3c1004f0 2024-03-04T09:30:54.918 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:54.921+0000 7f6d426c9640 1 -- 172.21.15.52:0/389816542 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6d3c0621d0 con 0x7f6d3c1004f0 2024-03-04T09:30:54.919 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:54.924+0000 7f6d397fa640 1 -- 172.21.15.52:0/389816542 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v93) v1 ==== 74+0+24804 (secure 0 0 0) 0x7f6d2c09af40 con 0x7f6d3c1004f0 2024-03-04T09:30:54.920 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-03-04T09:30:54.920 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":93,"fsid":"4e059f08-da08-11ee-95c6-87774f69a715","created":"2024-03-04T09:20:46.104963+0000","modified":"2024-03-04T09:30:41.237792+0000","last_up_change":"2024-03-04T09:27:49.261365+0000","last_in_change":"2024-03-04T09:27:30.024964+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-04T09:24:01.742461+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-04T09:28:35.835600+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-04T09:28:36.453099+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-04T09:28:40.168831+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"583172be-602f-4bf4-bfcb-42663cf5b277","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6802","nonce":1371221397},{"type":"v1","addr":"172.21.15.52:6803","nonce":1371221397}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6804","nonce":1371221397},{"type":"v1","addr":"172.21.15.52:6805","nonce":1371221397}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6808","nonce":1371221397},{"type":"v1","addr":"172.21.15.52:6809","nonce":1371221397}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6806","nonce":1371221397},{"type":"v1","addr":"172.21.15.52:6807","nonce":1371221397}]},"public_addr":"172.21.15.52:6803/1371221397","cluster_addr":"172.21.15.52:6805/1371221397","heartbeat_back_addr":"172.21.15.52:6809/1371221397","heartbeat_front_addr":"172.21.15.52:6807/1371221397","state":["exists","up"]},{"osd":1,"uuid":"5ae0081c-a2f6-46eb-ace0-0368208377b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":2540865537},{"type":"v1","addr":"172.21.15.106:6801","nonce":2540865537}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":2540865537},{"type":"v1","addr":"172.21.15.106:6803","nonce":2540865537}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":2540865537},{"type":"v1","addr":"172.21.15.106:6807","nonce":2540865537}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":2540865537},{"type":"v1","addr":"172.21.15.106:6805","nonce":2540865537}]},"public_addr":"172.21.15.106:6801/2540865537","cluster_addr":"172.21.15.106:6803/2540865537","heartbeat_back_addr":"172.21.15.106:6807/2540865537","heartbeat_front_addr":"172.21.15.106:6805/2540865537","state":["exists","up"]},{"osd":2,"uuid":"1948bac7-2bb2-4218-8b93-4e4743a0de04","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":1377083164},{"type":"v1","addr":"172.21.15.191:6801","nonce":1377083164}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":1377083164},{"type":"v1","addr":"172.21.15.191:6803","nonce":1377083164}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":1377083164},{"type":"v1","addr":"172.21.15.191:6807","nonce":1377083164}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":1377083164},{"type":"v1","addr":"172.21.15.191:6805","nonce":1377083164}]},"public_addr":"172.21.15.191:6801/1377083164","cluster_addr":"172.21.15.191:6803/1377083164","heartbeat_back_addr":"172.21.15.191:6807/1377083164","heartbeat_front_addr":"172.21.15.191:6805/1377083164","state":["exists","up"]},{"osd":3,"uuid":"16c9e9fe-7167-4992-bf12-1a7398e0498a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6810","nonce":2388571810},{"type":"v1","addr":"172.21.15.52:6811","nonce":2388571810}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6812","nonce":2388571810},{"type":"v1","addr":"172.21.15.52:6813","nonce":2388571810}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6816","nonce":2388571810},{"type":"v1","addr":"172.21.15.52:6817","nonce":2388571810}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6814","nonce":2388571810},{"type":"v1","addr":"172.21.15.52:6815","nonce":2388571810}]},"public_addr":"172.21.15.52:6811/2388571810","cluster_addr":"172.21.15.52:6813/2388571810","heartbeat_back_addr":"172.21.15.52:6817/2388571810","heartbeat_front_addr":"172.21.15.52:6815/2388571810","state":["exists","up"]},{"osd":4,"uuid":"c8ef48c7-7c78-44d3-a9e5-99ec750abd3d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6808","nonce":3465868015},{"type":"v1","addr":"172.21.15.106:6809","nonce":3465868015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6810","nonce":3465868015},{"type":"v1","addr":"172.21.15.106:6811","nonce":3465868015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6814","nonce":3465868015},{"type":"v1","addr":"172.21.15.106:6815","nonce":3465868015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6812","nonce":3465868015},{"type":"v1","addr":"172.21.15.106:6813","nonce":3465868015}]},"public_addr":"172.21.15.106:6809/3465868015","cluster_addr":"172.21.15.106:6811/3465868015","heartbeat_back_addr":"172.21.15.106:6815/3465868015","heartbeat_front_addr":"172.21.15.106:6813/3465868015","state":["exists","up"]},{"osd":5,"uuid":"9e41824a-c5a2-4f88-be50-2ba1dceb3220","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":1485092643},{"type":"v1","addr":"172.21.15.191:6809","nonce":1485092643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":1485092643},{"type":"v1","addr":"172.21.15.191:6811","nonce":1485092643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":1485092643},{"type":"v1","addr":"172.21.15.191:6815","nonce":1485092643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":1485092643},{"type":"v1","addr":"172.21.15.191:6813","nonce":1485092643}]},"public_addr":"172.21.15.191:6809/1485092643","cluster_addr":"172.21.15.191:6811/1485092643","heartbeat_back_addr":"172.21.15.191:6815/1485092643","heartbeat_front_addr":"172.21.15.191:6813/1485092643","state":["exists","up"]},{"osd":6,"uuid":"b35dccee-ec16-452e-ac69-3ac2fb7b27fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6818","nonce":1989797856},{"type":"v1","addr":"172.21.15.52:6819","nonce":1989797856}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6820","nonce":1989797856},{"type":"v1","addr":"172.21.15.52:6821","nonce":1989797856}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6824","nonce":1989797856},{"type":"v1","addr":"172.21.15.52:6825","nonce":1989797856}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6822","nonce":1989797856},{"type":"v1","addr":"172.21.15.52:6823","nonce":1989797856}]},"public_addr":"172.21.15.52:6819/1989797856","cluster_addr":"172.21.15.52:6821/1989797856","heartbeat_back_addr":"172.21.15.52:6825/1989797856","heartbeat_front_addr":"172.21.15.52:6823/1989797856","state":["exists","up"]},{"osd":7,"uuid":"71303e3d-14ec-4cbd-b19c-67645a1de3c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6816","nonce":1450870329},{"type":"v1","addr":"172.21.15.106:6817","nonce":1450870329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":1450870329},{"type":"v1","addr":"172.21.15.106:6819","nonce":1450870329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":1450870329},{"type":"v1","addr":"172.21.15.106:6823","nonce":1450870329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6820","nonce":1450870329},{"type":"v1","addr":"172.21.15.106:6821","nonce":1450870329}]},"public_addr":"172.21.15.106:6817/1450870329","cluster_addr":"172.21.15.106:6819/1450870329","heartbeat_back_addr":"172.21.15.106:6823/1450870329","heartbeat_front_addr":"172.21.15.106:6821/1450870329","state":["exists","up"]},{"osd":8,"uuid":"6f5786c4-4402-435a-bdad-adff46fe9473","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":1114119072},{"type":"v1","addr":"172.21.15.191:6817","nonce":1114119072}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6818","nonce":1114119072},{"type":"v1","addr":"172.21.15.191:6819","nonce":1114119072}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":1114119072},{"type":"v1","addr":"172.21.15.191:6823","nonce":1114119072}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":1114119072},{"type":"v1","addr":"172.21.15.191:6821","nonce":1114119072}]},"public_addr":"172.21.15.191:6817/1114119072","cluster_addr":"172.21.15.191:6819/1114119072","heartbeat_back_addr":"172.21.15.191:6823/1114119072","heartbeat_front_addr":"172.21.15.191:6821/1114119072","state":["exists","up"]},{"osd":9,"uuid":"062bbaa5-1227-4684-b859-450f5594a90c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6826","nonce":3485821914},{"type":"v1","addr":"172.21.15.52:6827","nonce":3485821914}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6828","nonce":3485821914},{"type":"v1","addr":"172.21.15.52:6829","nonce":3485821914}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6832","nonce":3485821914},{"type":"v1","addr":"172.21.15.52:6833","nonce":3485821914}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6830","nonce":3485821914},{"type":"v1","addr":"172.21.15.52:6831","nonce":3485821914}]},"public_addr":"172.21.15.52:6827/3485821914","cluster_addr":"172.21.15.52:6829/3485821914","heartbeat_back_addr":"172.21.15.52:6833/3485821914","heartbeat_front_addr":"172.21.15.52:6831/3485821914","state":["exists","up"]},{"osd":10,"uuid":"1a285a5a-f793-48da-a8eb-5326d9528cd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":3815586576},{"type":"v1","addr":"172.21.15.106:6825","nonce":3815586576}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6826","nonce":3815586576},{"type":"v1","addr":"172.21.15.106:6827","nonce":3815586576}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":3815586576},{"type":"v1","addr":"172.21.15.106:6831","nonce":3815586576}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":3815586576},{"type":"v1","addr":"172.21.15.106:6829","nonce":3815586576}]},"public_addr":"172.21.15.106:6825/3815586576","cluster_addr":"172.21.15.106:6827/3815586576","heartbeat_back_addr":"172.21.15.106:6831/3815586576","heartbeat_front_addr":"172.21.15.106:6829/3815586576","state":["exists","up"]},{"osd":11,"uuid":"1f8b9a43-57fa-41f4-9969-3b803a74bf45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":2009632843},{"type":"v1","addr":"172.21.15.191:6825","nonce":2009632843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":2009632843},{"type":"v1","addr":"172.21.15.191:6827","nonce":2009632843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":2009632843},{"type":"v1","addr":"172.21.15.191:6831","nonce":2009632843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":2009632843},{"type":"v1","addr":"172.21.15.191:6829","nonce":2009632843}]},"public_addr":"172.21.15.191:6825/2009632843","cluster_addr":"172.21.15.191:6827/2009632843","heartbeat_back_addr":"172.21.15.191:6831/2009632843","heartbeat_front_addr":"172.21.15.191:6829/2009632843","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T09:23:10.894763+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T09:23:34.420055+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T09:23:58.288324+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T09:24:23.673457+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T09:24:48.208561+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T09:25:13.405473+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T09:25:38.670915+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T09:26:04.451257+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T09:26:30.281380+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T09:26:56.543143+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T09:27:22.412930+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T09:27:48.003866+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.1d","mappings":[{"from":2,"to":1},{"from":8,"to":11}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.20","mappings":[{"from":5,"to":9}]},{"pgid":"3.21","mappings":[{"from":4,"to":6}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]},{"pgid":"4.9","mappings":[{"from":5,"to":8}]},{"pgid":"4.17","mappings":[{"from":9,"to":8}]},{"pgid":"4.1b","mappings":[{"from":9,"to":10}]},{"pgid":"4.1f","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.52:0/4082645718":"2024-03-05T09:21:39.401935+0000","172.21.15.52:0/1271949988":"2024-03-05T09:21:39.401935+0000","172.21.15.52:6801/2379361333":"2024-03-05T09:21:12.760938+0000","172.21.15.52:0/497075075":"2024-03-05T09:21:12.760938+0000","172.21.15.52:6800/2379361333":"2024-03-05T09:21:12.760938+0000","172.21.15.52:0/568242577":"2024-03-05T09:21:12.760938+0000","172.21.15.52:6801/2425730636":"2024-03-05T09:21:39.401935+0000","172.21.15.52:6800/2425730636":"2024-03-05T09:21:39.401935+0000","172.21.15.52:0/2743084243":"2024-03-05T09:21:39.401935+0000","172.21.15.52:0/1035878034":"2024-03-05T09:21:12.760938+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-04T09:30:54.922 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:54.925+0000 7f6d426c9640 1 -- 172.21.15.52:0/389816542 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f6d240a1e00 msgr2=0x7f6d240a42c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:30:54.922 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:54.925+0000 7f6d426c9640 1 --2- 172.21.15.52:0/389816542 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f6d240a1e00 0x7f6d240a42c0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f6d3002c7b0 tx=0x7f6d30005fd0 comp rx=0 tx=0).stop 2024-03-04T09:30:54.922 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:54.925+0000 7f6d426c9640 1 -- 172.21.15.52:0/389816542 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6d3c1004f0 msgr2=0x7f6d3c1967d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:30:54.922 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:54.925+0000 7f6d426c9640 1 --2- 172.21.15.52:0/389816542 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6d3c1004f0 0x7f6d3c1967d0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f6d2c00e9c0 tx=0x7f6d2c00ee80 comp rx=0 tx=0).stop 2024-03-04T09:30:54.922 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:54.925+0000 7f6d426c9640 1 -- 172.21.15.52:0/389816542 shutdown_connections 2024-03-04T09:30:54.922 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:54.925+0000 7f6d426c9640 1 --2- 172.21.15.52:0/389816542 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f6d240a1e00 0x7f6d240a42c0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:54.922 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:54.925+0000 7f6d426c9640 1 --2- 172.21.15.52:0/389816542 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6d3c1020a0 0x7f6d3c19dd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:54.922 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:54.925+0000 7f6d426c9640 1 --2- 172.21.15.52:0/389816542 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6d3c1016e0 0x7f6d3c196d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:54.922 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:54.925+0000 7f6d426c9640 1 --2- 172.21.15.52:0/389816542 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6d3c1004f0 0x7f6d3c1967d0 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:54.922 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:54.925+0000 7f6d426c9640 1 -- 172.21.15.52:0/389816542 >> 172.21.15.52:0/389816542 conn(0x7f6d3c0fba80 msgr2=0x7f6d3c104a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:30:54.922 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:54.925+0000 7f6d426c9640 1 -- 172.21.15.52:0/389816542 shutdown_connections 2024-03-04T09:30:54.922 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:54.925+0000 7f6d426c9640 1 -- 172.21.15.52:0/389816542 wait complete. 2024-03-04T09:30:54.934 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 24, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-03-04T09:28:37.264970+0000', 'modified': '2024-03-04T09:29:04.001848+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 24506, 'mds_1': 24490, 'mds_2': 24473}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24440': {'gid': 24440, 'name': 'e', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.106:6835/1541069897', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.106:6834', 'nonce': 1541069897}, {'type': 'v1', 'addr': '172.21.15.106:6835', 'nonce': 1541069897}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24451': {'gid': 24451, 'name': 'f', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.191:6835/105048699', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.191:6834', 'nonce': 105048699}, {'type': 'v1', 'addr': '172.21.15.191:6835', 'nonce': 105048699}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24463': {'gid': 24463, 'name': 'c', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.191:6837/4071064225', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.191:6836', 'nonce': 4071064225}, {'type': 'v1', 'addr': '172.21.15.191:6837', 'nonce': 4071064225}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24473': {'gid': 24473, 'name': 'h', 'rank': 2, 'incarnation': 22, 'state': 'up:active', 'state_seq': 12, 'addr': '172.21.15.106:6837/2901755066', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.106:6836', 'nonce': 2901755066}, {'type': 'v1', 'addr': '172.21.15.106:6837', 'nonce': 2901755066}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24490': {'gid': 24490, 'name': 'i', 'rank': 1, 'incarnation': 20, 'state': 'up:active', 'state_seq': 11, 'addr': '172.21.15.191:6839/2194324891', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.191:6838', 'nonce': 2194324891}, {'type': 'v1', 'addr': '172.21.15.191:6839', 'nonce': 2194324891}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24506': {'gid': 24506, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.106:6839/1995853093', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.106:6838', 'nonce': 1995853093}, {'type': 'v1', 'addr': '172.21.15.106:6839', 'nonce': 1995853093}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24506, 'qdb_cluster': [24473, 24490, 24506]} 2024-03-04T09:30:54.934 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-03-04T09:30:54.934 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.h status 2024-03-04T09:30:55.040 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.041+0000 7f976e5e9640 1 -- 172.21.15.52:0/2735539298 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f97681020a0 msgr2=0x7f9768108930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:30:55.040 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.041+0000 7f976e5e9640 1 --2- 172.21.15.52:0/2735539298 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f97681020a0 0x7f9768108930 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f9760009ca0 tx=0x7f976002c0f0 comp rx=0 tx=0).stop 2024-03-04T09:30:55.040 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.041+0000 7f976e5e9640 1 -- 172.21.15.52:0/2735539298 shutdown_connections 2024-03-04T09:30:55.040 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.041+0000 7f976e5e9640 1 --2- 172.21.15.52:0/2735539298 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f97681020a0 0x7f9768108930 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:55.040 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.041+0000 7f976e5e9640 1 --2- 172.21.15.52:0/2735539298 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f97681016e0 0x7f9768101b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:55.040 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.041+0000 7f976e5e9640 1 --2- 172.21.15.52:0/2735539298 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f97681004f0 0x7f9768100910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:55.040 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.041+0000 7f976e5e9640 1 -- 172.21.15.52:0/2735539298 >> 172.21.15.52:0/2735539298 conn(0x7f97680fba60 msgr2=0x7f97680fdea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:30:55.040 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.042+0000 7f976e5e9640 1 -- 172.21.15.52:0/2735539298 shutdown_connections 2024-03-04T09:30:55.040 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.042+0000 7f976e5e9640 1 -- 172.21.15.52:0/2735539298 wait complete. 2024-03-04T09:30:55.040 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.042+0000 7f976e5e9640 1 Processor -- start 2024-03-04T09:30:55.040 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.042+0000 7f976e5e9640 1 -- start start 2024-03-04T09:30:55.041 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.042+0000 7f976e5e9640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f97681004f0 0x7f976806a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:30:55.041 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.042+0000 7f976e5e9640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f97681016e0 0x7f976819c650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:30:55.041 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.042+0000 7f976e5e9640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f97681020a0 0x7f976819ec70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:30:55.041 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.042+0000 7f976e5e9640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f976810c180 con 0x7f97681016e0 2024-03-04T09:30:55.041 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.042+0000 7f976e5e9640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f976810c000 con 0x7f97681020a0 2024-03-04T09:30:55.041 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.042+0000 7f976e5e9640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f976810c300 con 0x7f97681004f0 2024-03-04T09:30:55.041 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.042+0000 7f97677fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f97681016e0 0x7f976819c650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:30:55.041 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.043+0000 7f97677fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f97681016e0 0x7f976819c650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:46164/0 (socket says 172.21.15.52:46164) 2024-03-04T09:30:55.041 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.043+0000 7f97677fe640 1 -- 172.21.15.52:0/2954183835 learned_addr learned my addr 172.21.15.52:0/2954183835 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-04T09:30:55.041 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.043+0000 7f9767fff640 1 --2- 172.21.15.52:0/2954183835 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f97681004f0 0x7f976806a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:30:55.041 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.043+0000 7f976cb86640 1 --2- 172.21.15.52:0/2954183835 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f97681020a0 0x7f976819ec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:30:55.041 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.043+0000 7f97677fe640 1 -- 172.21.15.52:0/2954183835 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f97681004f0 msgr2=0x7f976806a3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:30:55.041 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.043+0000 7f97677fe640 1 --2- 172.21.15.52:0/2954183835 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f97681004f0 0x7f976806a3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:55.041 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.043+0000 7f97677fe640 1 -- 172.21.15.52:0/2954183835 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f97681020a0 msgr2=0x7f976819ec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:30:55.041 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.043+0000 7f97677fe640 1 --2- 172.21.15.52:0/2954183835 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f97681020a0 0x7f976819ec70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:55.042 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.043+0000 7f97677fe640 1 -- 172.21.15.52:0/2954183835 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f976819f580 con 0x7f97681016e0 2024-03-04T09:30:55.042 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.043+0000 7f97677fe640 1 --2- 172.21.15.52:0/2954183835 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f97681016e0 0x7f976819c650 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f975c00e9f0 tx=0x7f975c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:30:55.044 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.043+0000 7f97657fa640 1 -- 172.21.15.52:0/2954183835 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f975c00cd80 con 0x7f97681016e0 2024-03-04T09:30:55.044 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.043+0000 7f97657fa640 1 -- 172.21.15.52:0/2954183835 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f975c010910 con 0x7f97681016e0 2024-03-04T09:30:55.044 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.043+0000 7f976e5e9640 1 -- 172.21.15.52:0/2954183835 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f976819f810 con 0x7f97681016e0 2024-03-04T09:30:55.044 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.043+0000 7f97657fa640 1 -- 172.21.15.52:0/2954183835 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f975c018990 con 0x7f97681016e0 2024-03-04T09:30:55.044 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.043+0000 7f976e5e9640 1 -- 172.21.15.52:0/2954183835 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f976819fd60 con 0x7f97681016e0 2024-03-04T09:30:55.051 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.045+0000 7f97657fa640 1 -- 172.21.15.52:0/2954183835 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) v1 ==== 136756+0+0 (secure 0 0 0) 0x7f975c010ac0 con 0x7f97681016e0 2024-03-04T09:30:55.051 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.046+0000 7f97711d0740 1 Processor -- start 2024-03-04T09:30:55.051 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.046+0000 7f97711d0740 1 -- start start 2024-03-04T09:30:55.051 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.046+0000 7f97711d0740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560d0eb1d7b0 0x560d0eb1dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:30:55.051 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.046+0000 7f97657fa640 1 --2- 172.21.15.52:0/2954183835 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f97500a1d80 0x7f97500a4240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:30:55.051 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.046+0000 7f97711d0740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560d0eb1e980 0x560d0eb1ee00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:30:55.051 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.046+0000 7f97711d0740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560d0eb1f340 0x560d0eb25bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:30:55.051 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.046+0000 7f97711d0740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x560d0ea826b0 con 0x560d0eb1f340 2024-03-04T09:30:55.051 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.046+0000 7f97711d0740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x560d0ea2ba60 con 0x560d0eb1e980 2024-03-04T09:30:55.051 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.046+0000 7f97711d0740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x560d0ea7e750 con 0x560d0eb1d7b0 2024-03-04T09:30:55.051 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.046+0000 7f9767fff640 1 --2- 172.21.15.52:0/2954183835 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f97500a1d80 0x7f97500a4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:30:55.052 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.046+0000 7f976cb86640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560d0eb1f340 0x560d0eb25bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:30:55.052 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.046+0000 7f976cb86640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560d0eb1d7b0 0x560d0eb1dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:30:55.052 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.046+0000 7f97677fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560d0eb1e980 0x560d0eb1ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:30:55.052 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.046+0000 7f976cb86640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560d0eb1d7b0 0x560d0eb1dbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.52:40774/0 (socket says 172.21.15.52:40774) 2024-03-04T09:30:55.052 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.046+0000 7f97657fa640 1 -- 172.21.15.52:0/2954183835 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9158+0+0 (secure 0 0 0) 0x7f975c0ce4c0 con 0x7f97681016e0 2024-03-04T09:30:55.052 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.046+0000 7f976cb86640 1 -- 172.21.15.52:0/1920764213 learned_addr learned my addr 172.21.15.52:0/1920764213 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-04T09:30:55.052 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.046+0000 7f976cb86640 1 -- 172.21.15.52:0/1920764213 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560d0eb1e980 msgr2=0x560d0eb1ee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T09:30:55.052 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.046+0000 7f976cb86640 1 --2- 172.21.15.52:0/1920764213 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560d0eb1e980 0x560d0eb1ee00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:55.052 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.046+0000 7f976cb86640 1 -- 172.21.15.52:0/1920764213 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560d0eb1f340 msgr2=0x560d0eb25bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T09:30:55.052 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.046+0000 7f976cb86640 1 --2- 172.21.15.52:0/1920764213 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560d0eb1f340 0x560d0eb25bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:55.052 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.046+0000 7f976cb86640 1 -- 172.21.15.52:0/1920764213 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560d0ea49f30 con 0x560d0eb1d7b0 2024-03-04T09:30:55.052 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.047+0000 7f9767fff640 1 --2- 172.21.15.52:0/2954183835 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f97500a1d80 0x7f97500a4240 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f9768101540 tx=0x7f9758005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:30:55.052 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.047+0000 7f976cb86640 1 --2- 172.21.15.52:0/1920764213 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560d0eb1d7b0 0x560d0eb1dbd0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f9760009ca0 tx=0x7f9760032590 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dbb698b872f200b1 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:30:55.053 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.047+0000 7f974e7fc640 1 -- 172.21.15.52:0/1920764213 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f976002ce60 con 0x560d0eb1d7b0 2024-03-04T09:30:55.053 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.047+0000 7f974e7fc640 1 -- 172.21.15.52:0/1920764213 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9760003970 con 0x560d0eb1d7b0 2024-03-04T09:30:55.053 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.047+0000 7f974e7fc640 1 -- 172.21.15.52:0/1920764213 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9760039040 con 0x560d0eb1d7b0 2024-03-04T09:30:55.053 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.047+0000 7f97711d0740 1 -- 172.21.15.52:0/1920764213 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560d0eb1d7b0 msgr2=0x560d0eb1dbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:30:55.053 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.047+0000 7f97711d0740 1 --2- 172.21.15.52:0/1920764213 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560d0eb1d7b0 0x560d0eb1dbd0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f9760009ca0 tx=0x7f9760032590 comp rx=0 tx=0).stop 2024-03-04T09:30:55.053 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.047+0000 7f97711d0740 1 -- 172.21.15.52:0/1920764213 shutdown_connections 2024-03-04T09:30:55.053 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.047+0000 7f97711d0740 1 --2- 172.21.15.52:0/1920764213 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560d0eb1f340 0x560d0eb25bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:55.053 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.047+0000 7f97711d0740 1 --2- 172.21.15.52:0/1920764213 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560d0eb1e980 0x560d0eb1ee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:55.053 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.047+0000 7f97711d0740 1 --2- 172.21.15.52:0/1920764213 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560d0eb1d7b0 0x560d0eb1dbd0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:55.053 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.047+0000 7f97711d0740 1 -- 172.21.15.52:0/1920764213 >> 172.21.15.52:0/1920764213 conn(0x560d0eb18ba0 msgr2=0x560d0eb18fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:30:55.053 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.047+0000 7f97711d0740 1 -- 172.21.15.52:0/1920764213 shutdown_connections 2024-03-04T09:30:55.053 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.047+0000 7f97711d0740 1 -- 172.21.15.52:0/1920764213 wait complete. 2024-03-04T09:30:55.053 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.048+0000 7f97711d0740 1 -- start start 2024-03-04T09:30:55.053 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.048+0000 7f97711d0740 1 Processor -- start 2024-03-04T09:30:55.054 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.048+0000 7f97711d0740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560d0eb1d7b0 0x560d0eb27c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:30:55.054 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.048+0000 7f97711d0740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560d0eb1e980 0x560d0eb281d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:30:55.054 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.048+0000 7f97711d0740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560d0eb1f340 0x560d0eb2f210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:30:55.054 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.048+0000 7f97711d0740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x560d0eb26290 con 0x560d0eb1e980 2024-03-04T09:30:55.054 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.048+0000 7f97711d0740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x560d0ea3eea0 con 0x560d0eb1d7b0 2024-03-04T09:30:55.054 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.048+0000 7f97711d0740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x560d0eb26410 con 0x560d0eb1f340 2024-03-04T09:30:55.054 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.048+0000 7f97711d0740 10 client.0 _ll_register_callbacks cb 0x560d0ea3e870 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-04T09:30:55.054 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.048+0000 7f97677fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560d0eb1e980 0x560d0eb281d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:30:55.054 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.048+0000 7f97677fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560d0eb1e980 0x560d0eb281d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:46194/0 (socket says 172.21.15.52:46194) 2024-03-04T09:30:55.054 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.048+0000 7f9767fff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560d0eb1d7b0 0x560d0eb27c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:30:55.054 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.048+0000 7f97677fe640 1 -- 172.21.15.52:0/1579081616 learned_addr learned my addr 172.21.15.52:0/1579081616 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-04T09:30:55.054 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.048+0000 7f976cb86640 1 --2- 172.21.15.52:0/1579081616 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560d0eb1f340 0x560d0eb2f210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:30:55.054 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.049+0000 7f97677fe640 1 -- 172.21.15.52:0/1579081616 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560d0eb1f340 msgr2=0x560d0eb2f210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T09:30:55.055 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.049+0000 7f97677fe640 1 --2- 172.21.15.52:0/1579081616 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560d0eb1f340 0x560d0eb2f210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:55.055 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.049+0000 7f97677fe640 1 -- 172.21.15.52:0/1579081616 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560d0eb1d7b0 msgr2=0x560d0eb27c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T09:30:55.055 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.049+0000 7f97677fe640 1 --2- 172.21.15.52:0/1579081616 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560d0eb1d7b0 0x560d0eb27c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:55.055 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.049+0000 7f97677fe640 1 -- 172.21.15.52:0/1579081616 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560d0ea3f270 con 0x560d0eb1e980 2024-03-04T09:30:55.055 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.049+0000 7f97677fe640 1 --2- 172.21.15.52:0/1579081616 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560d0eb1e980 0x560d0eb281d0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f975c00c6a0 tx=0x7f975c01f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74ef6fd05372548 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:30:55.055 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.049+0000 7f97477fe640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-03-04T09:30:55.055 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.049+0000 7f97477fe640 1 -- 172.21.15.52:0/1579081616 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f975c0a4450 con 0x560d0eb1e980 2024-03-04T09:30:55.055 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.049+0000 7f97477fe640 1 -- 172.21.15.52:0/1579081616 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9158+0+0 (secure 0 0 0) 0x7f975c01cd30 con 0x560d0eb1e980 2024-03-04T09:30:55.055 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.049+0000 7f97711d0740 1 -- 172.21.15.52:0/1579081616 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9760003970 con 0x560d0eb1e980 2024-03-04T09:30:55.055 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.049+0000 7f97477fe640 1 -- 172.21.15.52:0/1579081616 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f975c09aea0 con 0x560d0eb1e980 2024-03-04T09:30:55.055 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.049+0000 7f97711d0740 10 client.15399 fetch_fsmap learned FSMap version 24 2024-03-04T09:30:55.055 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.049+0000 7f97711d0740 1 -- 172.21.15.52:0/1579081616 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x560d0e7972d0 con 0x560d0eb1e980 2024-03-04T09:30:55.055 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.049+0000 7f97477fe640 1 -- 172.21.15.52:0/1579081616 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9760003970 con 0x560d0eb1e980 2024-03-04T09:30:55.055 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.049+0000 7f97477fe640 1 -- 172.21.15.52:0/1579081616 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f975c0aa790 con 0x560d0eb1e980 2024-03-04T09:30:55.056 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.049+0000 7f97711d0740 10 client.15399 fetch_fsmap finished waiting for FSMap version 24 2024-03-04T09:30:55.056 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.049+0000 7f97711d0740 10 client.15399 resolve_mds: resolved name 'h' to daemon mds.h 2024-03-04T09:30:55.056 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.049+0000 7f97711d0740 20 client.15399 populate_metadata read hostname 'smithi052' 2024-03-04T09:30:55.056 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.049+0000 7f97711d0740 1 --2- 172.21.15.52:0/1579081616 >> [v2:172.21.15.106:6836/2901755066,v1:172.21.15.106:6837/2901755066] conn(0x560d0eb34730 0x560d0eb33890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:30:55.056 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.049+0000 7f97711d0740 4 client.15399 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-04T09:30:55.056 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.049+0000 7f97711d0740 1 -- 172.21.15.52:0/1579081616 --> [v2:172.21.15.106:6836/2901755066,v1:172.21.15.106:6837/2901755066] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560d0ea8fbf0 con 0x560d0eb34730 2024-03-04T09:30:55.056 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.050+0000 7f9767fff640 1 --2- 172.21.15.52:0/1579081616 >> [v2:172.21.15.106:6836/2901755066,v1:172.21.15.106:6837/2901755066] conn(0x560d0eb34730 0x560d0eb33890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:30:55.056 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.050+0000 7f97477fe640 10 client.15399 ms_handle_connect on v2:172.21.15.106:6836/2901755066 2024-03-04T09:30:55.056 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.050+0000 7f9767fff640 1 --2- 172.21.15.52:0/1579081616 >> [v2:172.21.15.106:6836/2901755066,v1:172.21.15.106:6837/2901755066] conn(0x560d0eb34730 0x560d0eb33890 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=8654c24433eb8ab server_cookie=8e684ed40a7fa2e6 in_seq=0 out_seq=0 2024-03-04T09:30:55.056 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.051+0000 7f97477fe640 1 -- 172.21.15.52:0/1579081616 <== mds.2 v2:172.21.15.106:6836/2901755066 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x560d0ea8fbf0 con 0x560d0eb34730 2024-03-04T09:30:55.056 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.051+0000 7f97477fe640 10 client.15399 handle_command_reply: tid=0 2024-03-04T09:30:55.056 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.051+0000 7f97711d0740 1 client.15399 shutdown 2024-03-04T09:30:55.056 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.051+0000 7f97711d0740 1 -- 172.21.15.52:0/1579081616 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560d0eb1e980 msgr2=0x560d0eb281d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:30:55.056 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.051+0000 7f97711d0740 1 --2- 172.21.15.52:0/1579081616 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560d0eb1e980 0x560d0eb281d0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f975c00c6a0 tx=0x7f975c01f000 comp rx=0 tx=0).stop 2024-03-04T09:30:55.057 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.051+0000 7f97711d0740 1 -- 172.21.15.52:0/1579081616 shutdown_connections 2024-03-04T09:30:55.057 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.051+0000 7f97711d0740 1 --2- 172.21.15.52:0/1579081616 >> [v2:172.21.15.106:6836/2901755066,v1:172.21.15.106:6837/2901755066] conn(0x560d0eb34730 0x560d0eb33890 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:55.057 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.051+0000 7f97711d0740 1 --2- 172.21.15.52:0/1579081616 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560d0eb1f340 0x560d0eb2f210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:55.057 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.051+0000 7f97711d0740 1 --2- 172.21.15.52:0/1579081616 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560d0eb1e980 0x560d0eb281d0 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:55.057 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.051+0000 7f97711d0740 1 --2- 172.21.15.52:0/1579081616 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560d0eb1d7b0 0x560d0eb27c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:55.057 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.051+0000 7f97477fe640 0 client.15399 ms_handle_reset on v2:172.21.15.106:6836/2901755066 2024-03-04T09:30:55.057 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.051+0000 7f97711d0740 1 -- 172.21.15.52:0/1579081616 >> 172.21.15.52:0/1579081616 conn(0x560d0eb18ba0 msgr2=0x560d0eb17da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:30:55.057 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.051+0000 7f97711d0740 1 -- 172.21.15.52:0/1579081616 shutdown_connections 2024-03-04T09:30:55.057 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.051+0000 7f97711d0740 1 -- 172.21.15.52:0/1579081616 wait complete. 2024-03-04T09:30:55.057 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.051+0000 7f97711d0740 20 client.15399 trim_cache size 0 max 16384 2024-03-04T09:30:55.068 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.067+0000 7f97711d0740 1 Processor -- start 2024-03-04T09:30:55.068 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.067+0000 7f97711d0740 1 -- start start 2024-03-04T09:30:55.068 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.068+0000 7f97711d0740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560d0eb1d7b0 0x560d0eb278a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:30:55.069 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.068+0000 7f97711d0740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560d0eb1e980 0x560d0eb27de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:30:55.069 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.068+0000 7f97711d0740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560d0eb1f340 0x560d0eb3cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:30:55.069 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.068+0000 7f97711d0740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x560d0eb33f50 con 0x560d0eb1e980 2024-03-04T09:30:55.069 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.068+0000 7f97711d0740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x560d0eb345b0 con 0x560d0eb1d7b0 2024-03-04T09:30:55.069 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.068+0000 7f97711d0740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x560d0eb34c10 con 0x560d0eb1f340 2024-03-04T09:30:55.069 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.068+0000 7f9767fff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560d0eb1d7b0 0x560d0eb278a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:30:55.069 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.068+0000 7f97677fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560d0eb1e980 0x560d0eb27de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:30:55.069 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.068+0000 7f9767fff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560d0eb1d7b0 0x560d0eb278a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.52:36112/0 (socket says 172.21.15.52:36112) 2024-03-04T09:30:55.069 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.068+0000 7f9767fff640 1 -- 172.21.15.52:0/3594631042 learned_addr learned my addr 172.21.15.52:0/3594631042 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-04T09:30:55.069 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.068+0000 7f976cb86640 1 --2- 172.21.15.52:0/3594631042 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560d0eb1f340 0x560d0eb3cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:30:55.069 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.068+0000 7f97677fe640 1 -- 172.21.15.52:0/3594631042 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560d0eb1f340 msgr2=0x560d0eb3cbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T09:30:55.069 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.068+0000 7f97677fe640 1 --2- 172.21.15.52:0/3594631042 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560d0eb1f340 0x560d0eb3cbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:55.069 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.068+0000 7f97677fe640 1 -- 172.21.15.52:0/3594631042 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560d0eb1d7b0 msgr2=0x560d0eb278a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T09:30:55.070 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.068+0000 7f97677fe640 1 --2- 172.21.15.52:0/3594631042 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560d0eb1d7b0 0x560d0eb278a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:55.070 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.068+0000 7f97677fe640 1 -- 172.21.15.52:0/3594631042 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560d0e8616b0 con 0x560d0eb1e980 2024-03-04T09:30:55.070 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.068+0000 7f9767fff640 1 --2- 172.21.15.52:0/3594631042 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560d0eb1d7b0 0x560d0eb278a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-04T09:30:55.070 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.068+0000 7f97677fe640 1 --2- 172.21.15.52:0/3594631042 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560d0eb1e980 0x560d0eb27de0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f975c09aba0 tx=0x7f975c000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=398f7f43919b125a server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:30:55.070 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.068+0000 7f974dffb640 1 -- 172.21.15.52:0/3594631042 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f975c0a3590 con 0x560d0eb1e980 2024-03-04T09:30:55.070 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.068+0000 7f974dffb640 1 -- 172.21.15.52:0/3594631042 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f975c09e070 con 0x560d0eb1e980 2024-03-04T09:30:55.070 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.068+0000 7f974dffb640 1 -- 172.21.15.52:0/3594631042 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f975c09a2f0 con 0x560d0eb1e980 2024-03-04T09:30:55.070 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.069+0000 7f97711d0740 1 -- 172.21.15.52:0/3594631042 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560d0eb1e980 msgr2=0x560d0eb27de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:30:55.070 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.069+0000 7f97711d0740 1 --2- 172.21.15.52:0/3594631042 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560d0eb1e980 0x560d0eb27de0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f975c09aba0 tx=0x7f975c000bc0 comp rx=0 tx=0).stop 2024-03-04T09:30:55.070 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.069+0000 7f97711d0740 1 -- 172.21.15.52:0/3594631042 shutdown_connections 2024-03-04T09:30:55.070 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.069+0000 7f97711d0740 1 --2- 172.21.15.52:0/3594631042 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560d0eb1f340 0x560d0eb3cbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:55.070 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.069+0000 7f97711d0740 1 --2- 172.21.15.52:0/3594631042 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560d0eb1e980 0x560d0eb27de0 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:55.070 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.069+0000 7f97711d0740 1 --2- 172.21.15.52:0/3594631042 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560d0eb1d7b0 0x560d0eb278a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:55.070 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.069+0000 7f97711d0740 1 -- 172.21.15.52:0/3594631042 >> 172.21.15.52:0/3594631042 conn(0x560d0eb18ba0 msgr2=0x560d0eb22560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:30:55.071 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.069+0000 7f97711d0740 1 -- 172.21.15.52:0/3594631042 shutdown_connections 2024-03-04T09:30:55.071 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.069+0000 7f97711d0740 1 -- 172.21.15.52:0/3594631042 wait complete. 2024-03-04T09:30:55.071 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.069+0000 7f97711d0740 1 -- start start 2024-03-04T09:30:55.071 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.069+0000 7f97711d0740 1 Processor -- start 2024-03-04T09:30:55.071 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.070+0000 7f97711d0740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560d0eb1d7b0 0x560d0eb20300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:30:55.071 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.070+0000 7f97711d0740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560d0eb1e980 0x560d0eb20840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:30:55.071 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.070+0000 7f97711d0740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560d0eb1f340 0x560d0eb20d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:30:55.071 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.070+0000 7f97711d0740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x560d0eb358d0 con 0x560d0eb1d7b0 2024-03-04T09:30:55.071 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.070+0000 7f97711d0740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x560d0eb35270 con 0x560d0eb1f340 2024-03-04T09:30:55.071 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.070+0000 7f97711d0740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x560d0eb35f30 con 0x560d0eb1e980 2024-03-04T09:30:55.071 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.070+0000 7f97711d0740 10 client.0 _ll_register_callbacks cb 0x7f975c09aa00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-04T09:30:55.071 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.070+0000 7f9767fff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560d0eb1d7b0 0x560d0eb20300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:30:55.071 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.070+0000 7f9767fff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560d0eb1d7b0 0x560d0eb20300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:46220/0 (socket says 172.21.15.52:46220) 2024-03-04T09:30:55.071 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.070+0000 7f9767fff640 1 -- 172.21.15.52:0/3426125808 learned_addr learned my addr 172.21.15.52:0/3426125808 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-04T09:30:55.072 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.070+0000 7f976cb86640 1 --2- 172.21.15.52:0/3426125808 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560d0eb1f340 0x560d0eb20d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:30:55.072 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.070+0000 7f97677fe640 1 --2- 172.21.15.52:0/3426125808 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560d0eb1e980 0x560d0eb20840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:30:55.072 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.070+0000 7f9767fff640 1 -- 172.21.15.52:0/3426125808 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560d0eb1e980 msgr2=0x560d0eb20840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T09:30:55.072 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.070+0000 7f9767fff640 1 --2- 172.21.15.52:0/3426125808 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560d0eb1e980 0x560d0eb20840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:55.072 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.070+0000 7f9767fff640 1 -- 172.21.15.52:0/3426125808 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560d0eb1f340 msgr2=0x560d0eb20d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T09:30:55.072 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.070+0000 7f9767fff640 1 --2- 172.21.15.52:0/3426125808 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560d0eb1f340 0x560d0eb20d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:55.072 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.070+0000 7f9767fff640 1 -- 172.21.15.52:0/3426125808 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560d0eb937b0 con 0x560d0eb1d7b0 2024-03-04T09:30:55.072 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.070+0000 7f9767fff640 1 --2- 172.21.15.52:0/3426125808 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560d0eb1d7b0 0x560d0eb20300 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f975800e6b0 tx=0x7f9758004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac362437fc31de06 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:30:55.072 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.070+0000 7f97477fe640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-03-04T09:30:55.072 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.070+0000 7f97477fe640 1 -- 172.21.15.52:0/3426125808 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9758029900 con 0x560d0eb1d7b0 2024-03-04T09:30:55.072 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.070+0000 7f97477fe640 1 -- 172.21.15.52:0/3426125808 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9158+0+0 (secure 0 0 0) 0x7f9758029aa0 con 0x560d0eb1d7b0 2024-03-04T09:30:55.072 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.070+0000 7f97711d0740 1 -- 172.21.15.52:0/3426125808 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f975c09e070 con 0x560d0eb1d7b0 2024-03-04T09:30:55.072 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.071+0000 7f97477fe640 1 -- 172.21.15.52:0/3426125808 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f975803b050 con 0x560d0eb1d7b0 2024-03-04T09:30:55.072 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.071+0000 7f97711d0740 10 client.15405 fetch_fsmap learned FSMap version 24 2024-03-04T09:30:55.073 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.071+0000 7f97711d0740 1 -- 172.21.15.52:0/3426125808 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x560d0eb949e0 con 0x560d0eb1d7b0 2024-03-04T09:30:55.073 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.071+0000 7f97477fe640 1 -- 172.21.15.52:0/3426125808 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f975c09e070 con 0x560d0eb1d7b0 2024-03-04T09:30:55.073 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.071+0000 7f97477fe640 1 -- 172.21.15.52:0/3426125808 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f97580049d0 con 0x560d0eb1d7b0 2024-03-04T09:30:55.073 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.071+0000 7f97711d0740 10 client.15405 fetch_fsmap finished waiting for FSMap version 24 2024-03-04T09:30:55.073 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.071+0000 7f97711d0740 10 client.15405 resolve_mds: resolved name 'h' to daemon mds.h 2024-03-04T09:30:55.073 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.071+0000 7f97711d0740 20 client.15405 populate_metadata read hostname 'smithi052' 2024-03-04T09:30:55.073 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.071+0000 7f97711d0740 1 --2- 172.21.15.52:0/3426125808 >> [v2:172.21.15.106:6836/2901755066,v1:172.21.15.106:6837/2901755066] conn(0x560d0eb990a0 0x560d0eb94cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:30:55.073 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.071+0000 7f97711d0740 4 client.15405 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-04T09:30:55.073 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.071+0000 7f97711d0740 1 -- 172.21.15.52:0/3426125808 --> [v2:172.21.15.106:6836/2901755066,v1:172.21.15.106:6837/2901755066] -- command(tid 0: {"prefix": "status"}) v1 -- 0x560d0ea89f40 con 0x560d0eb990a0 2024-03-04T09:30:55.073 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.071+0000 7f97677fe640 1 --2- 172.21.15.52:0/3426125808 >> [v2:172.21.15.106:6836/2901755066,v1:172.21.15.106:6837/2901755066] conn(0x560d0eb990a0 0x560d0eb94cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:30:55.074 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.071+0000 7f97477fe640 10 client.15405 ms_handle_connect on v2:172.21.15.106:6836/2901755066 2024-03-04T09:30:55.074 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.071+0000 7f97677fe640 1 --2- 172.21.15.52:0/3426125808 >> [v2:172.21.15.106:6836/2901755066,v1:172.21.15.106:6837/2901755066] conn(0x560d0eb990a0 0x560d0eb94cd0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=2a3397301947f276 server_cookie=a30ab3371acfa754 in_seq=0 out_seq=0 2024-03-04T09:30:55.074 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.072+0000 7f97477fe640 1 -- 172.21.15.52:0/3426125808 <== mds.2 v2:172.21.15.106:6836/2901755066 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+322 (crc 0 0 0) 0x560d0ea89f40 con 0x560d0eb990a0 2024-03-04T09:30:55.074 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.072+0000 7f97477fe640 10 client.15405 handle_command_reply: tid=0 2024-03-04T09:30:55.074 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.072+0000 7f97711d0740 1 client.15405 shutdown 2024-03-04T09:30:55.074 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.072+0000 7f97711d0740 1 -- 172.21.15.52:0/3426125808 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560d0eb1d7b0 msgr2=0x560d0eb20300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:30:55.074 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.072+0000 7f97711d0740 1 --2- 172.21.15.52:0/3426125808 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560d0eb1d7b0 0x560d0eb20300 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f975800e6b0 tx=0x7f9758004730 comp rx=0 tx=0).stop 2024-03-04T09:30:55.074 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.072+0000 7f97711d0740 1 -- 172.21.15.52:0/3426125808 shutdown_connections 2024-03-04T09:30:55.074 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.072+0000 7f97711d0740 1 --2- 172.21.15.52:0/3426125808 >> [v2:172.21.15.106:6836/2901755066,v1:172.21.15.106:6837/2901755066] conn(0x560d0eb990a0 0x560d0eb94cd0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:55.074 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.072+0000 7f97711d0740 1 --2- 172.21.15.52:0/3426125808 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560d0eb1f340 0x560d0eb20d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:55.074 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.072+0000 7f97711d0740 1 --2- 172.21.15.52:0/3426125808 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560d0eb1e980 0x560d0eb20840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:55.074 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.072+0000 7f97477fe640 0 client.15405 ms_handle_reset on v2:172.21.15.106:6836/2901755066 2024-03-04T09:30:55.074 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.072+0000 7f97711d0740 1 --2- 172.21.15.52:0/3426125808 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560d0eb1d7b0 0x560d0eb20300 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:55.075 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.072+0000 7f97711d0740 1 -- 172.21.15.52:0/3426125808 >> 172.21.15.52:0/3426125808 conn(0x560d0eb18ba0 msgr2=0x560d0eb1be00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:30:55.075 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.072+0000 7f97711d0740 1 -- 172.21.15.52:0/3426125808 shutdown_connections 2024-03-04T09:30:55.075 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.072+0000 7f97711d0740 1 -- 172.21.15.52:0/3426125808 wait complete. 2024-03-04T09:30:55.075 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.072+0000 7f97711d0740 20 client.15405 trim_cache size 0 max 16384 2024-03-04T09:30:55.075 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2024-03-04T09:30:55.075 INFO:teuthology.orchestra.run.smithi052.stdout: "cluster_fsid": "4e059f08-da08-11ee-95c6-87774f69a715", 2024-03-04T09:30:55.075 INFO:teuthology.orchestra.run.smithi052.stdout: "whoami": 2, 2024-03-04T09:30:55.075 INFO:teuthology.orchestra.run.smithi052.stdout: "id": 24473, 2024-03-04T09:30:55.075 INFO:teuthology.orchestra.run.smithi052.stdout: "want_state": "up:active", 2024-03-04T09:30:55.075 INFO:teuthology.orchestra.run.smithi052.stdout: "state": "up:active", 2024-03-04T09:30:55.076 INFO:teuthology.orchestra.run.smithi052.stdout: "fs_name": "cephfs", 2024-03-04T09:30:55.076 INFO:teuthology.orchestra.run.smithi052.stdout: "rank_uptime": 112.080118617, 2024-03-04T09:30:55.076 INFO:teuthology.orchestra.run.smithi052.stdout: "mdsmap_epoch": 24, 2024-03-04T09:30:55.076 INFO:teuthology.orchestra.run.smithi052.stdout: "osdmap_epoch": 93, 2024-03-04T09:30:55.076 INFO:teuthology.orchestra.run.smithi052.stdout: "osdmap_epoch_barrier": 89, 2024-03-04T09:30:55.076 INFO:teuthology.orchestra.run.smithi052.stdout: "uptime": 152.749203679 2024-03-04T09:30:55.076 INFO:teuthology.orchestra.run.smithi052.stdout:} 2024-03-04T09:30:55.077 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.072+0000 7f976e5e9640 1 -- 172.21.15.52:0/2954183835 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f97500a1d80 msgr2=0x7f97500a4240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:30:55.077 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.072+0000 7f976e5e9640 1 --2- 172.21.15.52:0/2954183835 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f97500a1d80 0x7f97500a4240 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f9768101540 tx=0x7f9758005f90 comp rx=0 tx=0).stop 2024-03-04T09:30:55.077 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.072+0000 7f976e5e9640 1 -- 172.21.15.52:0/2954183835 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f97681016e0 msgr2=0x7f976819c650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:30:55.077 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.072+0000 7f976e5e9640 1 --2- 172.21.15.52:0/2954183835 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f97681016e0 0x7f976819c650 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f975c00e9f0 tx=0x7f975c00eeb0 comp rx=0 tx=0).stop 2024-03-04T09:30:55.077 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.073+0000 7f976e5e9640 1 -- 172.21.15.52:0/2954183835 shutdown_connections 2024-03-04T09:30:55.077 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.073+0000 7f976e5e9640 1 --2- 172.21.15.52:0/2954183835 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f97500a1d80 0x7f97500a4240 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:55.077 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.073+0000 7f976e5e9640 1 --2- 172.21.15.52:0/2954183835 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f97681020a0 0x7f976819ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:55.077 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.073+0000 7f976e5e9640 1 --2- 172.21.15.52:0/2954183835 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f97681016e0 0x7f976819c650 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:55.077 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.073+0000 7f976e5e9640 1 --2- 172.21.15.52:0/2954183835 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f97681004f0 0x7f976806a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:55.077 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.073+0000 7f976e5e9640 1 -- 172.21.15.52:0/2954183835 >> 172.21.15.52:0/2954183835 conn(0x7f97680fba60 msgr2=0x7f9768104b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:30:55.078 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.073+0000 7f976e5e9640 1 -- 172.21.15.52:0/2954183835 shutdown_connections 2024-03-04T09:30:55.078 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.073+0000 7f976e5e9640 1 -- 172.21.15.52:0/2954183835 wait complete. 2024-03-04T09:30:55.079 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.i status 2024-03-04T09:30:55.185 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.187+0000 7f3402c2a640 1 -- 172.21.15.52:0/638669990 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f33fc1020d0 msgr2=0x7f33fc1044c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:30:55.185 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.187+0000 7f3402c2a640 1 --2- 172.21.15.52:0/638669990 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f33fc1020d0 0x7f33fc1044c0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f33f0009900 tx=0x7f33f002bc30 comp rx=0 tx=0).stop 2024-03-04T09:30:55.185 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.187+0000 7f3402c2a640 1 -- 172.21.15.52:0/638669990 shutdown_connections 2024-03-04T09:30:55.185 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.187+0000 7f3402c2a640 1 --2- 172.21.15.52:0/638669990 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f33fc107330 0x7f33fc109720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:55.185 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.187+0000 7f3402c2a640 1 --2- 172.21.15.52:0/638669990 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f33fc104a00 0x7f33fc106df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:55.186 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.187+0000 7f3402c2a640 1 --2- 172.21.15.52:0/638669990 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f33fc1020d0 0x7f33fc1044c0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:55.186 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.187+0000 7f3402c2a640 1 -- 172.21.15.52:0/638669990 >> 172.21.15.52:0/638669990 conn(0x7f33fc0fba80 msgr2=0x7f33fc0fdea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:30:55.186 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.187+0000 7f3402c2a640 1 -- 172.21.15.52:0/638669990 shutdown_connections 2024-03-04T09:30:55.186 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.187+0000 7f3402c2a640 1 -- 172.21.15.52:0/638669990 wait complete. 2024-03-04T09:30:55.186 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.187+0000 7f3402c2a640 1 Processor -- start 2024-03-04T09:30:55.186 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.187+0000 7f3402c2a640 1 -- start start 2024-03-04T09:30:55.186 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.187+0000 7f3402c2a640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f33fc1020d0 0x7f33fc197980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:30:55.186 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.187+0000 7f3402c2a640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f33fc104a00 0x7f33fc197ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:30:55.186 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.187+0000 7f3402c2a640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f33fc107330 0x7f33fc19eee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:30:55.186 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.187+0000 7f3402c2a640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33fc10d260 con 0x7f33fc104a00 2024-03-04T09:30:55.186 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.187+0000 7f3402c2a640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33fc10d0e0 con 0x7f33fc1020d0 2024-03-04T09:30:55.186 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.187+0000 7f3402c2a640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33fc10d3e0 con 0x7f33fc107330 2024-03-04T09:30:55.186 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.188+0000 7f33fbfff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f33fc104a00 0x7f33fc197ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:30:55.186 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.188+0000 7f33fbfff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f33fc104a00 0x7f33fc197ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:46234/0 (socket says 172.21.15.52:46234) 2024-03-04T09:30:55.187 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.188+0000 7f33fbfff640 1 -- 172.21.15.52:0/126783164 learned_addr learned my addr 172.21.15.52:0/126783164 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-04T09:30:55.187 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.188+0000 7f34009c6640 1 --2- 172.21.15.52:0/126783164 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f33fc1020d0 0x7f33fc197980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:30:55.187 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.188+0000 7f34011c7640 1 --2- 172.21.15.52:0/126783164 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f33fc107330 0x7f33fc19eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:30:55.187 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.188+0000 7f33fbfff640 1 -- 172.21.15.52:0/126783164 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f33fc107330 msgr2=0x7f33fc19eee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:30:55.187 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.188+0000 7f33fbfff640 1 --2- 172.21.15.52:0/126783164 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f33fc107330 0x7f33fc19eee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:55.187 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.188+0000 7f33fbfff640 1 -- 172.21.15.52:0/126783164 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f33fc1020d0 msgr2=0x7f33fc197980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:30:55.187 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.188+0000 7f33fbfff640 1 --2- 172.21.15.52:0/126783164 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f33fc1020d0 0x7f33fc197980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:55.187 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.188+0000 7f33fbfff640 1 -- 172.21.15.52:0/126783164 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33fc19f550 con 0x7f33fc104a00 2024-03-04T09:30:55.187 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.188+0000 7f34009c6640 1 --2- 172.21.15.52:0/126783164 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f33fc1020d0 0x7f33fc197980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-04T09:30:55.187 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.188+0000 7f33fbfff640 1 --2- 172.21.15.52:0/126783164 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f33fc104a00 0x7f33fc197ec0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f33e800e9c0 tx=0x7f33e800edc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:30:55.187 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.188+0000 7f33f9ffb640 1 -- 172.21.15.52:0/126783164 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33e800ccc0 con 0x7f33fc104a00 2024-03-04T09:30:55.187 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.188+0000 7f33f9ffb640 1 -- 172.21.15.52:0/126783164 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f33e8005500 con 0x7f33fc104a00 2024-03-04T09:30:55.187 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.188+0000 7f3402c2a640 1 -- 172.21.15.52:0/126783164 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33fc19f840 con 0x7f33fc104a00 2024-03-04T09:30:55.187 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.188+0000 7f33f9ffb640 1 -- 172.21.15.52:0/126783164 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33e8010430 con 0x7f33fc104a00 2024-03-04T09:30:55.187 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.189+0000 7f3402c2a640 1 -- 172.21.15.52:0/126783164 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33fc19fd90 con 0x7f33fc104a00 2024-03-04T09:30:55.194 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.190+0000 7f33f9ffb640 1 -- 172.21.15.52:0/126783164 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) v1 ==== 136756+0+0 (secure 0 0 0) 0x7f33e8005000 con 0x7f33fc104a00 2024-03-04T09:30:55.194 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.191+0000 7f3405814740 1 Processor -- start 2024-03-04T09:30:55.194 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.191+0000 7f3405814740 1 -- start start 2024-03-04T09:30:55.194 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.191+0000 7f33f9ffb640 1 --2- 172.21.15.52:0/126783164 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f33ec0a1d80 0x7f33ec0a4240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:30:55.194 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.191+0000 7f3405814740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557027f3f7b0 0x557027f3fbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:30:55.195 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.191+0000 7f3405814740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557027f40980 0x557027f40e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:30:55.195 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.191+0000 7f3405814740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557027f41340 0x557027f47bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:30:55.195 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.191+0000 7f3405814740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x557027ea46b0 con 0x557027f3f7b0 2024-03-04T09:30:55.195 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.191+0000 7f3405814740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x557027e4da60 con 0x557027f41340 2024-03-04T09:30:55.195 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.191+0000 7f3405814740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x557027ea0750 con 0x557027f40980 2024-03-04T09:30:55.195 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.191+0000 7f34011c7640 1 --2- 172.21.15.52:0/126783164 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f33ec0a1d80 0x7f33ec0a4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:30:55.195 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.191+0000 7f33f9ffb640 1 -- 172.21.15.52:0/126783164 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9158+0+0 (secure 0 0 0) 0x7f33e80ce590 con 0x7f33fc104a00 2024-03-04T09:30:55.195 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.191+0000 7f34009c6640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557027f3f7b0 0x557027f3fbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:30:55.195 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.191+0000 7f34009c6640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557027f3f7b0 0x557027f3fbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:46242/0 (socket says 172.21.15.52:46242) 2024-03-04T09:30:55.195 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.191+0000 7f34009c6640 1 -- 172.21.15.52:0/1206359323 learned_addr learned my addr 172.21.15.52:0/1206359323 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-04T09:30:55.195 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.191+0000 7f33fbfff640 1 --2- 172.21.15.52:0/1206359323 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557027f40980 0x557027f40e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:30:55.195 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.191+0000 7f34011c7640 1 --2- 172.21.15.52:0/1206359323 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557027f41340 0x557027f47bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:30:55.195 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.192+0000 7f34009c6640 1 -- 172.21.15.52:0/1206359323 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557027f40980 msgr2=0x557027f40e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T09:30:55.195 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.192+0000 7f34009c6640 1 --2- 172.21.15.52:0/1206359323 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557027f40980 0x557027f40e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:55.195 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.192+0000 7f34009c6640 1 -- 172.21.15.52:0/1206359323 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557027f41340 msgr2=0x557027f47bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T09:30:55.196 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.192+0000 7f34009c6640 1 --2- 172.21.15.52:0/1206359323 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557027f41340 0x557027f47bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:55.196 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.192+0000 7f34009c6640 1 -- 172.21.15.52:0/1206359323 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557027e6bf30 con 0x557027f3f7b0 2024-03-04T09:30:55.196 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.192+0000 7f34011c7640 1 --2- 172.21.15.52:0/126783164 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f33ec0a1d80 0x7f33ec0a4240 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f33f4005fd0 tx=0x7f33f4013b80 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:30:55.196 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.192+0000 7f34009c6640 1 --2- 172.21.15.52:0/1206359323 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557027f3f7b0 0x557027f3fbd0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f33f0036040 tx=0x7f33f002cd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f72db51bd6e6b8e server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:30:55.196 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.192+0000 7f33e6ffd640 1 -- 172.21.15.52:0/1206359323 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33f0005d50 con 0x557027f3f7b0 2024-03-04T09:30:55.196 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.192+0000 7f33e6ffd640 1 -- 172.21.15.52:0/1206359323 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f33f0005520 con 0x557027f3f7b0 2024-03-04T09:30:55.196 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.192+0000 7f33e6ffd640 1 -- 172.21.15.52:0/1206359323 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33f0004c40 con 0x557027f3f7b0 2024-03-04T09:30:55.196 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.193+0000 7f3405814740 1 -- 172.21.15.52:0/1206359323 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557027f3f7b0 msgr2=0x557027f3fbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:30:55.196 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.193+0000 7f3405814740 1 --2- 172.21.15.52:0/1206359323 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557027f3f7b0 0x557027f3fbd0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f33f0036040 tx=0x7f33f002cd30 comp rx=0 tx=0).stop 2024-03-04T09:30:55.196 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.193+0000 7f3405814740 1 -- 172.21.15.52:0/1206359323 shutdown_connections 2024-03-04T09:30:55.196 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.193+0000 7f3405814740 1 --2- 172.21.15.52:0/1206359323 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557027f41340 0x557027f47bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:55.196 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.193+0000 7f3405814740 1 --2- 172.21.15.52:0/1206359323 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557027f40980 0x557027f40e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:55.196 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.193+0000 7f3405814740 1 --2- 172.21.15.52:0/1206359323 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557027f3f7b0 0x557027f3fbd0 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:55.197 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.193+0000 7f3405814740 1 -- 172.21.15.52:0/1206359323 >> 172.21.15.52:0/1206359323 conn(0x557027f3aba0 msgr2=0x557027f3afa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:30:55.197 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.193+0000 7f3405814740 1 -- 172.21.15.52:0/1206359323 shutdown_connections 2024-03-04T09:30:55.197 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.193+0000 7f3405814740 1 -- 172.21.15.52:0/1206359323 wait complete. 2024-03-04T09:30:55.197 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.193+0000 7f3405814740 1 -- start start 2024-03-04T09:30:55.197 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.193+0000 7f3405814740 1 Processor -- start 2024-03-04T09:30:55.197 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.193+0000 7f3405814740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557027f3f7b0 0x557027f49cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:30:55.197 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.193+0000 7f3405814740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557027f40980 0x557027f4a200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:30:55.197 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.193+0000 7f3405814740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557027f41340 0x557027f51240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:30:55.197 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.193+0000 7f3405814740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x557027f482f0 con 0x557027f41340 2024-03-04T09:30:55.197 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.193+0000 7f3405814740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x557027e60ea0 con 0x557027f3f7b0 2024-03-04T09:30:55.197 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.193+0000 7f3405814740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x557027f48470 con 0x557027f40980 2024-03-04T09:30:55.197 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.194+0000 7f3405814740 10 client.0 _ll_register_callbacks cb 0x557027e60870 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-04T09:30:55.197 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.194+0000 7f34011c7640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557027f41340 0x557027f51240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:30:55.198 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.194+0000 7f34009c6640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557027f3f7b0 0x557027f49cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:30:55.198 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.194+0000 7f34011c7640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557027f41340 0x557027f51240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:46244/0 (socket says 172.21.15.52:46244) 2024-03-04T09:30:55.198 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.194+0000 7f34011c7640 1 -- 172.21.15.52:0/3016710358 learned_addr learned my addr 172.21.15.52:0/3016710358 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-04T09:30:55.198 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.194+0000 7f33fbfff640 1 --2- 172.21.15.52:0/3016710358 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557027f40980 0x557027f4a200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:30:55.198 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.194+0000 7f34011c7640 1 -- 172.21.15.52:0/3016710358 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557027f40980 msgr2=0x557027f4a200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T09:30:55.198 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.194+0000 7f34011c7640 1 --2- 172.21.15.52:0/3016710358 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557027f40980 0x557027f4a200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:55.198 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.194+0000 7f34011c7640 1 -- 172.21.15.52:0/3016710358 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557027f3f7b0 msgr2=0x557027f49cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T09:30:55.198 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.194+0000 7f34011c7640 1 --2- 172.21.15.52:0/3016710358 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557027f3f7b0 0x557027f49cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:55.198 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.194+0000 7f34011c7640 1 -- 172.21.15.52:0/3016710358 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557027e61270 con 0x557027f41340 2024-03-04T09:30:55.198 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.194+0000 7f34009c6640 1 --2- 172.21.15.52:0/3016710358 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557027f3f7b0 0x557027f49cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-04T09:30:55.198 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.194+0000 7f34011c7640 1 --2- 172.21.15.52:0/3016710358 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557027f41340 0x557027f51240 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f33f400ff20 tx=0x7f33f4004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61c727af2ce1b220 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:30:55.198 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.194+0000 7f33dbfff640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-03-04T09:30:55.198 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.194+0000 7f33dbfff640 1 -- 172.21.15.52:0/3016710358 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33f4024030 con 0x557027f41340 2024-03-04T09:30:55.198 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.194+0000 7f33dbfff640 1 -- 172.21.15.52:0/3016710358 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9158+0+0 (secure 0 0 0) 0x7f33f4004740 con 0x557027f41340 2024-03-04T09:30:55.199 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.194+0000 7f3405814740 1 -- 172.21.15.52:0/3016710358 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f33f0005520 con 0x557027f41340 2024-03-04T09:30:55.199 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.194+0000 7f33dbfff640 1 -- 172.21.15.52:0/3016710358 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f33f4031050 con 0x557027f41340 2024-03-04T09:30:55.199 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.194+0000 7f3405814740 10 client.15417 fetch_fsmap learned FSMap version 24 2024-03-04T09:30:55.199 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.194+0000 7f3405814740 1 -- 172.21.15.52:0/3016710358 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x557027bb92d0 con 0x557027f41340 2024-03-04T09:30:55.199 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.195+0000 7f33dbfff640 1 -- 172.21.15.52:0/3016710358 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f33f0005520 con 0x557027f41340 2024-03-04T09:30:55.199 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.195+0000 7f33dbfff640 1 -- 172.21.15.52:0/3016710358 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f33f401f070 con 0x557027f41340 2024-03-04T09:30:55.199 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.195+0000 7f3405814740 10 client.15417 fetch_fsmap finished waiting for FSMap version 24 2024-03-04T09:30:55.199 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.195+0000 7f3405814740 10 client.15417 resolve_mds: resolved name 'i' to daemon mds.i 2024-03-04T09:30:55.199 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.195+0000 7f3405814740 20 client.15417 populate_metadata read hostname 'smithi052' 2024-03-04T09:30:55.199 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.195+0000 7f3405814740 1 --2- 172.21.15.52:0/3016710358 >> [v2:172.21.15.191:6838/2194324891,v1:172.21.15.191:6839/2194324891] conn(0x557027f56950 0x557027f55960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:30:55.199 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.195+0000 7f3405814740 4 client.15417 mds_command: new command op to 24490 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-04T09:30:55.199 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.195+0000 7f3405814740 1 -- 172.21.15.52:0/3016710358 --> [v2:172.21.15.191:6838/2194324891,v1:172.21.15.191:6839/2194324891] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557027eb1bf0 con 0x557027f56950 2024-03-04T09:30:55.199 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.195+0000 7f34009c6640 1 --2- 172.21.15.52:0/3016710358 >> [v2:172.21.15.191:6838/2194324891,v1:172.21.15.191:6839/2194324891] conn(0x557027f56950 0x557027f55960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:30:55.199 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.195+0000 7f34009c6640 1 --2- 172.21.15.52:0/3016710358 >> [v2:172.21.15.191:6838/2194324891,v1:172.21.15.191:6839/2194324891] conn(0x557027f56950 0x557027f55960 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=ddf1d33766ffe8b1 server_cookie=d90aa2752e388596 in_seq=0 out_seq=0 2024-03-04T09:30:55.200 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.195+0000 7f33dbfff640 10 client.15417 ms_handle_connect on v2:172.21.15.191:6838/2194324891 2024-03-04T09:30:55.200 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.196+0000 7f33dbfff640 1 -- 172.21.15.52:0/3016710358 <== mds.1 v2:172.21.15.191:6838/2194324891 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x557027eb1bf0 con 0x557027f56950 2024-03-04T09:30:55.200 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.196+0000 7f33dbfff640 10 client.15417 handle_command_reply: tid=0 2024-03-04T09:30:55.200 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.196+0000 7f3405814740 1 client.15417 shutdown 2024-03-04T09:30:55.200 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.196+0000 7f3405814740 1 -- 172.21.15.52:0/3016710358 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557027f41340 msgr2=0x557027f51240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:30:55.200 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.196+0000 7f3405814740 1 --2- 172.21.15.52:0/3016710358 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557027f41340 0x557027f51240 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f33f400ff20 tx=0x7f33f4004ab0 comp rx=0 tx=0).stop 2024-03-04T09:30:55.200 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.196+0000 7f3405814740 1 -- 172.21.15.52:0/3016710358 shutdown_connections 2024-03-04T09:30:55.200 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.196+0000 7f3405814740 1 --2- 172.21.15.52:0/3016710358 >> [v2:172.21.15.191:6838/2194324891,v1:172.21.15.191:6839/2194324891] conn(0x557027f56950 0x557027f55960 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:55.200 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.196+0000 7f3405814740 1 --2- 172.21.15.52:0/3016710358 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557027f41340 0x557027f51240 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:55.200 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.196+0000 7f3405814740 1 --2- 172.21.15.52:0/3016710358 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557027f40980 0x557027f4a200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:55.200 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.196+0000 7f33dbfff640 0 client.15417 ms_handle_reset on v2:172.21.15.191:6838/2194324891 2024-03-04T09:30:55.200 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.196+0000 7f3405814740 1 --2- 172.21.15.52:0/3016710358 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557027f3f7b0 0x557027f49cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:55.200 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.196+0000 7f3405814740 1 -- 172.21.15.52:0/3016710358 >> 172.21.15.52:0/3016710358 conn(0x557027f3aba0 msgr2=0x557027f39da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:30:55.201 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.196+0000 7f3405814740 1 -- 172.21.15.52:0/3016710358 shutdown_connections 2024-03-04T09:30:55.201 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.196+0000 7f3405814740 1 -- 172.21.15.52:0/3016710358 wait complete. 2024-03-04T09:30:55.201 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.196+0000 7f3405814740 20 client.15417 trim_cache size 0 max 16384 2024-03-04T09:30:55.214 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.213+0000 7f3405814740 1 Processor -- start 2024-03-04T09:30:55.214 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.213+0000 7f3405814740 1 -- start start 2024-03-04T09:30:55.214 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.213+0000 7f3405814740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557027f3f7b0 0x557027f47b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:30:55.214 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.213+0000 7f3405814740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557027f40980 0x557027f3a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:30:55.214 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.213+0000 7f3405814740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557027f41340 0x557027f55960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:30:55.215 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.213+0000 7f3405814740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x557027f56170 con 0x557027f41340 2024-03-04T09:30:55.215 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.213+0000 7f3405814740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x557027f567d0 con 0x557027f3f7b0 2024-03-04T09:30:55.215 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.213+0000 7f3405814740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x557027f56e30 con 0x557027f40980 2024-03-04T09:30:55.215 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.213+0000 7f34011c7640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557027f41340 0x557027f55960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:30:55.215 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.213+0000 7f34011c7640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557027f41340 0x557027f55960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:46248/0 (socket says 172.21.15.52:46248) 2024-03-04T09:30:55.215 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.213+0000 7f34011c7640 1 -- 172.21.15.52:0/2526865911 learned_addr learned my addr 172.21.15.52:0/2526865911 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-04T09:30:55.215 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.213+0000 7f34009c6640 1 --2- 172.21.15.52:0/2526865911 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557027f3f7b0 0x557027f47b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:30:55.215 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.213+0000 7f33fbfff640 1 --2- 172.21.15.52:0/2526865911 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557027f40980 0x557027f3a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:30:55.215 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.214+0000 7f34011c7640 1 -- 172.21.15.52:0/2526865911 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557027f40980 msgr2=0x557027f3a4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T09:30:55.215 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.214+0000 7f34011c7640 1 --2- 172.21.15.52:0/2526865911 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557027f40980 0x557027f3a4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:55.215 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.214+0000 7f34011c7640 1 -- 172.21.15.52:0/2526865911 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557027f3f7b0 msgr2=0x557027f47b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T09:30:55.215 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.214+0000 7f34011c7640 1 --2- 172.21.15.52:0/2526865911 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557027f3f7b0 0x557027f47b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:55.215 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.214+0000 7f34011c7640 1 -- 172.21.15.52:0/2526865911 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557027c836b0 con 0x557027f41340 2024-03-04T09:30:55.215 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.214+0000 7f34011c7640 1 --2- 172.21.15.52:0/2526865911 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557027f41340 0x557027f55960 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f33f4010500 tx=0x7f33f400fd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e926e430cc0688cf server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:30:55.216 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.214+0000 7f33e67fc640 1 -- 172.21.15.52:0/2526865911 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33f4024070 con 0x557027f41340 2024-03-04T09:30:55.216 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.214+0000 7f33e67fc640 1 -- 172.21.15.52:0/2526865911 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f33f400f810 con 0x557027f41340 2024-03-04T09:30:55.216 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.214+0000 7f33e67fc640 1 -- 172.21.15.52:0/2526865911 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33f4035790 con 0x557027f41340 2024-03-04T09:30:55.216 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.214+0000 7f3405814740 1 -- 172.21.15.52:0/2526865911 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557027f41340 msgr2=0x557027f55960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:30:55.216 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.214+0000 7f3405814740 1 --2- 172.21.15.52:0/2526865911 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557027f41340 0x557027f55960 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f33f4010500 tx=0x7f33f400fd50 comp rx=0 tx=0).stop 2024-03-04T09:30:55.216 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.214+0000 7f3405814740 1 -- 172.21.15.52:0/2526865911 shutdown_connections 2024-03-04T09:30:55.216 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.214+0000 7f3405814740 1 --2- 172.21.15.52:0/2526865911 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557027f41340 0x557027f55960 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:55.216 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.214+0000 7f3405814740 1 --2- 172.21.15.52:0/2526865911 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557027f40980 0x557027f3a4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:55.216 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.214+0000 7f3405814740 1 --2- 172.21.15.52:0/2526865911 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557027f3f7b0 0x557027f47b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:55.216 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.214+0000 7f3405814740 1 -- 172.21.15.52:0/2526865911 >> 172.21.15.52:0/2526865911 conn(0x557027f3aba0 msgr2=0x557027f35e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:30:55.216 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.215+0000 7f3405814740 1 -- 172.21.15.52:0/2526865911 shutdown_connections 2024-03-04T09:30:55.216 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.215+0000 7f3405814740 1 -- 172.21.15.52:0/2526865911 wait complete. 2024-03-04T09:30:55.216 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.215+0000 7f3405814740 1 -- start start 2024-03-04T09:30:55.216 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.215+0000 7f3405814740 1 Processor -- start 2024-03-04T09:30:55.217 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.215+0000 7f3405814740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557027f3f7b0 0x557027fadbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:30:55.217 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.215+0000 7f3405814740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557027f40980 0x557027fae120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:30:55.217 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.215+0000 7f3405814740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557027f41340 0x557027fb4c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:30:55.217 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.215+0000 7f3405814740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x557027f57af0 con 0x557027f3f7b0 2024-03-04T09:30:55.217 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.215+0000 7f3405814740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x557027f57490 con 0x557027f41340 2024-03-04T09:30:55.217 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.215+0000 7f3405814740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x557027f58150 con 0x557027f40980 2024-03-04T09:30:55.217 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.215+0000 7f3405814740 10 client.0 _ll_register_callbacks cb 0x7f33f400fe20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-04T09:30:55.217 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.215+0000 7f34009c6640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557027f3f7b0 0x557027fadbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:30:55.217 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.215+0000 7f34009c6640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557027f3f7b0 0x557027fadbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:46254/0 (socket says 172.21.15.52:46254) 2024-03-04T09:30:55.217 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.215+0000 7f34009c6640 1 -- 172.21.15.52:0/577131807 learned_addr learned my addr 172.21.15.52:0/577131807 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-04T09:30:55.217 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.215+0000 7f33fbfff640 1 --2- 172.21.15.52:0/577131807 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557027f40980 0x557027fae120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:30:55.217 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.215+0000 7f34011c7640 1 --2- 172.21.15.52:0/577131807 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557027f41340 0x557027fb4c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:30:55.217 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.216+0000 7f34009c6640 1 -- 172.21.15.52:0/577131807 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557027f40980 msgr2=0x557027fae120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T09:30:55.217 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.216+0000 7f34009c6640 1 --2- 172.21.15.52:0/577131807 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557027f40980 0x557027fae120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:55.218 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.216+0000 7f34009c6640 1 -- 172.21.15.52:0/577131807 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557027f41340 msgr2=0x557027fb4c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T09:30:55.218 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.216+0000 7f34009c6640 1 --2- 172.21.15.52:0/577131807 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557027f41340 0x557027fb4c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:55.218 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.216+0000 7f34009c6640 1 -- 172.21.15.52:0/577131807 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557027fb5220 con 0x557027f3f7b0 2024-03-04T09:30:55.218 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.216+0000 7f34011c7640 1 --2- 172.21.15.52:0/577131807 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557027f41340 0x557027fb4c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T09:30:55.218 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.216+0000 7f34009c6640 1 --2- 172.21.15.52:0/577131807 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557027f3f7b0 0x557027fadbe0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f33f0000c00 tx=0x7f33f0005a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91b1e02fa3257d38 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:30:55.218 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.216+0000 7f33dbfff640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-03-04T09:30:55.218 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.216+0000 7f33dbfff640 1 -- 172.21.15.52:0/577131807 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33f0054070 con 0x557027f3f7b0 2024-03-04T09:30:55.218 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.216+0000 7f33dbfff640 1 -- 172.21.15.52:0/577131807 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9158+0+0 (secure 0 0 0) 0x7f33f004f870 con 0x557027f3f7b0 2024-03-04T09:30:55.218 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.216+0000 7f3405814740 1 -- 172.21.15.52:0/577131807 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f33f400f810 con 0x557027f3f7b0 2024-03-04T09:30:55.218 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.216+0000 7f33dbfff640 1 -- 172.21.15.52:0/577131807 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f33f0060050 con 0x557027f3f7b0 2024-03-04T09:30:55.218 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.216+0000 7f3405814740 10 client.15423 fetch_fsmap learned FSMap version 24 2024-03-04T09:30:55.218 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.216+0000 7f3405814740 1 -- 172.21.15.52:0/577131807 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x557027fb6520 con 0x557027f3f7b0 2024-03-04T09:30:55.218 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.216+0000 7f33dbfff640 1 -- 172.21.15.52:0/577131807 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f33f400f810 con 0x557027f3f7b0 2024-03-04T09:30:55.218 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.216+0000 7f33dbfff640 1 -- 172.21.15.52:0/577131807 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f33f0003b10 con 0x557027f3f7b0 2024-03-04T09:30:55.219 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.216+0000 7f3405814740 10 client.15423 fetch_fsmap finished waiting for FSMap version 24 2024-03-04T09:30:55.219 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.216+0000 7f3405814740 10 client.15423 resolve_mds: resolved name 'i' to daemon mds.i 2024-03-04T09:30:55.219 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.216+0000 7f3405814740 20 client.15423 populate_metadata read hostname 'smithi052' 2024-03-04T09:30:55.219 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.216+0000 7f3405814740 1 --2- 172.21.15.52:0/577131807 >> [v2:172.21.15.191:6838/2194324891,v1:172.21.15.191:6839/2194324891] conn(0x557027fba6c0 0x557027fb66f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:30:55.219 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.216+0000 7f3405814740 4 client.15423 mds_command: new command op to 24490 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-04T09:30:55.219 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.216+0000 7f3405814740 1 -- 172.21.15.52:0/577131807 --> [v2:172.21.15.191:6838/2194324891,v1:172.21.15.191:6839/2194324891] -- command(tid 0: {"prefix": "status"}) v1 -- 0x557027eabf40 con 0x557027fba6c0 2024-03-04T09:30:55.219 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.217+0000 7f33fbfff640 1 --2- 172.21.15.52:0/577131807 >> [v2:172.21.15.191:6838/2194324891,v1:172.21.15.191:6839/2194324891] conn(0x557027fba6c0 0x557027fb66f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:30:55.219 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.217+0000 7f33dbfff640 10 client.15423 ms_handle_connect on v2:172.21.15.191:6838/2194324891 2024-03-04T09:30:55.219 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.217+0000 7f33fbfff640 1 --2- 172.21.15.52:0/577131807 >> [v2:172.21.15.191:6838/2194324891,v1:172.21.15.191:6839/2194324891] conn(0x557027fba6c0 0x557027fb66f0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=ddeddfc8e049ff3c server_cookie=58bef3fb5e0d9ad4 in_seq=0 out_seq=0 2024-03-04T09:30:55.219 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.217+0000 7f33dbfff640 1 -- 172.21.15.52:0/577131807 <== mds.1 v2:172.21.15.191:6838/2194324891 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x557027eabf40 con 0x557027fba6c0 2024-03-04T09:30:55.219 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.217+0000 7f33dbfff640 10 client.15423 handle_command_reply: tid=0 2024-03-04T09:30:55.219 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.217+0000 7f3405814740 1 client.15423 shutdown 2024-03-04T09:30:55.219 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.217+0000 7f3405814740 1 -- 172.21.15.52:0/577131807 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557027f3f7b0 msgr2=0x557027fadbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:30:55.219 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.217+0000 7f3405814740 1 --2- 172.21.15.52:0/577131807 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557027f3f7b0 0x557027fadbe0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f33f0000c00 tx=0x7f33f0005a20 comp rx=0 tx=0).stop 2024-03-04T09:30:55.220 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.217+0000 7f3405814740 1 -- 172.21.15.52:0/577131807 shutdown_connections 2024-03-04T09:30:55.220 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.217+0000 7f3405814740 1 --2- 172.21.15.52:0/577131807 >> [v2:172.21.15.191:6838/2194324891,v1:172.21.15.191:6839/2194324891] conn(0x557027fba6c0 0x557027fb66f0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:55.220 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.217+0000 7f3405814740 1 --2- 172.21.15.52:0/577131807 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557027f41340 0x557027fb4c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:55.220 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.217+0000 7f3405814740 1 --2- 172.21.15.52:0/577131807 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557027f40980 0x557027fae120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:55.220 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.217+0000 7f33dbfff640 0 client.15423 ms_handle_reset on v2:172.21.15.191:6838/2194324891 2024-03-04T09:30:55.220 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.217+0000 7f3405814740 1 --2- 172.21.15.52:0/577131807 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557027f3f7b0 0x557027fadbe0 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:55.220 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.217+0000 7f3405814740 1 -- 172.21.15.52:0/577131807 >> 172.21.15.52:0/577131807 conn(0x557027f3aba0 msgr2=0x557027f55960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:30:55.220 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.217+0000 7f3405814740 1 -- 172.21.15.52:0/577131807 shutdown_connections 2024-03-04T09:30:55.220 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.217+0000 7f3405814740 1 -- 172.21.15.52:0/577131807 wait complete. 2024-03-04T09:30:55.220 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.218+0000 7f3405814740 20 client.15423 trim_cache size 0 max 16384 2024-03-04T09:30:55.220 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2024-03-04T09:30:55.221 INFO:teuthology.orchestra.run.smithi052.stdout: "cluster_fsid": "4e059f08-da08-11ee-95c6-87774f69a715", 2024-03-04T09:30:55.221 INFO:teuthology.orchestra.run.smithi052.stdout: "whoami": 1, 2024-03-04T09:30:55.221 INFO:teuthology.orchestra.run.smithi052.stdout: "id": 24490, 2024-03-04T09:30:55.221 INFO:teuthology.orchestra.run.smithi052.stdout: "want_state": "up:active", 2024-03-04T09:30:55.221 INFO:teuthology.orchestra.run.smithi052.stdout: "state": "up:active", 2024-03-04T09:30:55.221 INFO:teuthology.orchestra.run.smithi052.stdout: "fs_name": "cephfs", 2024-03-04T09:30:55.221 INFO:teuthology.orchestra.run.smithi052.stdout: "rank_uptime": 113.22788809399999, 2024-03-04T09:30:55.221 INFO:teuthology.orchestra.run.smithi052.stdout: "mdsmap_epoch": 24, 2024-03-04T09:30:55.221 INFO:teuthology.orchestra.run.smithi052.stdout: "osdmap_epoch": 93, 2024-03-04T09:30:55.221 INFO:teuthology.orchestra.run.smithi052.stdout: "osdmap_epoch_barrier": 89, 2024-03-04T09:30:55.221 INFO:teuthology.orchestra.run.smithi052.stdout: "uptime": 150.05275432600001 2024-03-04T09:30:55.221 INFO:teuthology.orchestra.run.smithi052.stdout:} 2024-03-04T09:30:55.222 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.218+0000 7f3402c2a640 1 -- 172.21.15.52:0/126783164 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f33ec0a1d80 msgr2=0x7f33ec0a4240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:30:55.222 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.218+0000 7f3402c2a640 1 --2- 172.21.15.52:0/126783164 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f33ec0a1d80 0x7f33ec0a4240 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f33f4005fd0 tx=0x7f33f4013b80 comp rx=0 tx=0).stop 2024-03-04T09:30:55.222 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.218+0000 7f3402c2a640 1 -- 172.21.15.52:0/126783164 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f33fc104a00 msgr2=0x7f33fc197ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:30:55.222 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.218+0000 7f3402c2a640 1 --2- 172.21.15.52:0/126783164 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f33fc104a00 0x7f33fc197ec0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f33e800e9c0 tx=0x7f33e800edc0 comp rx=0 tx=0).stop 2024-03-04T09:30:55.222 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.218+0000 7f3402c2a640 1 -- 172.21.15.52:0/126783164 shutdown_connections 2024-03-04T09:30:55.223 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.218+0000 7f3402c2a640 1 --2- 172.21.15.52:0/126783164 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f33ec0a1d80 0x7f33ec0a4240 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:55.223 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.218+0000 7f3402c2a640 1 --2- 172.21.15.52:0/126783164 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f33fc107330 0x7f33fc19eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:55.223 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.218+0000 7f3402c2a640 1 --2- 172.21.15.52:0/126783164 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f33fc104a00 0x7f33fc197ec0 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:55.223 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.218+0000 7f3402c2a640 1 --2- 172.21.15.52:0/126783164 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f33fc1020d0 0x7f33fc197980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:55.223 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.218+0000 7f3402c2a640 1 -- 172.21.15.52:0/126783164 >> 172.21.15.52:0/126783164 conn(0x7f33fc0fba80 msgr2=0x7f33fc102d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:30:55.223 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.218+0000 7f3402c2a640 1 -- 172.21.15.52:0/126783164 shutdown_connections 2024-03-04T09:30:55.223 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.218+0000 7f3402c2a640 1 -- 172.21.15.52:0/126783164 wait complete. 2024-03-04T09:30:55.225 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-03-04T09:30:55.331 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.333+0000 7fbb6b4eb640 1 -- 172.21.15.52:0/2765360357 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbb641004e0 msgr2=0x7fbb64100900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:30:55.331 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.333+0000 7fbb6b4eb640 1 --2- 172.21.15.52:0/2765360357 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbb641004e0 0x7fbb64100900 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fbb5c009860 tx=0x7fbb5c02bb70 comp rx=0 tx=0).stop 2024-03-04T09:30:55.331 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.333+0000 7fbb6b4eb640 1 -- 172.21.15.52:0/2765360357 shutdown_connections 2024-03-04T09:30:55.331 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.333+0000 7fbb6b4eb640 1 --2- 172.21.15.52:0/2765360357 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbb64102090 0x7fbb64108920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:55.331 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.333+0000 7fbb6b4eb640 1 --2- 172.21.15.52:0/2765360357 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbb641016d0 0x7fbb64101b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:55.332 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.333+0000 7fbb6b4eb640 1 --2- 172.21.15.52:0/2765360357 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbb641004e0 0x7fbb64100900 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:55.332 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.333+0000 7fbb6b4eb640 1 -- 172.21.15.52:0/2765360357 >> 172.21.15.52:0/2765360357 conn(0x7fbb640fba70 msgr2=0x7fbb640fde90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:30:55.332 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.333+0000 7fbb6b4eb640 1 -- 172.21.15.52:0/2765360357 shutdown_connections 2024-03-04T09:30:55.332 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.333+0000 7fbb6b4eb640 1 -- 172.21.15.52:0/2765360357 wait complete. 2024-03-04T09:30:55.332 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.334+0000 7fbb6b4eb640 1 Processor -- start 2024-03-04T09:30:55.332 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.334+0000 7fbb6b4eb640 1 -- start start 2024-03-04T09:30:55.332 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.334+0000 7fbb6b4eb640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbb641004e0 0x7fbb6406a730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:30:55.332 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.334+0000 7fbb6b4eb640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbb641016d0 0x7fbb6419ea50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:30:55.332 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.334+0000 7fbb6b4eb640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbb64102090 0x7fbb641a0fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:30:55.332 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.334+0000 7fbb6b4eb640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb6410c170 con 0x7fbb641016d0 2024-03-04T09:30:55.332 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.334+0000 7fbb6b4eb640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb6410bff0 con 0x7fbb64102090 2024-03-04T09:30:55.333 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.334+0000 7fbb6b4eb640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb6410c2f0 con 0x7fbb641004e0 2024-03-04T09:30:55.333 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.334+0000 7fbb68a86640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbb641016d0 0x7fbb6419ea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:30:55.333 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.334+0000 7fbb69287640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbb641004e0 0x7fbb6406a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:30:55.333 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.334+0000 7fbb68a86640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbb641016d0 0x7fbb6419ea50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:46280/0 (socket says 172.21.15.52:46280) 2024-03-04T09:30:55.333 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.334+0000 7fbb68a86640 1 -- 172.21.15.52:0/2581870803 learned_addr learned my addr 172.21.15.52:0/2581870803 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-04T09:30:55.333 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.334+0000 7fbb69287640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbb641004e0 0x7fbb6406a730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.52:40880/0 (socket says 172.21.15.52:40880) 2024-03-04T09:30:55.333 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.334+0000 7fbb69a88640 1 --2- 172.21.15.52:0/2581870803 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbb64102090 0x7fbb641a0fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:30:55.333 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.334+0000 7fbb68a86640 1 -- 172.21.15.52:0/2581870803 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbb641004e0 msgr2=0x7fbb6406a730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:30:55.333 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.334+0000 7fbb68a86640 1 --2- 172.21.15.52:0/2581870803 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbb641004e0 0x7fbb6406a730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:55.333 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.334+0000 7fbb68a86640 1 -- 172.21.15.52:0/2581870803 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbb64102090 msgr2=0x7fbb641a0fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:30:55.333 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.334+0000 7fbb68a86640 1 --2- 172.21.15.52:0/2581870803 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbb64102090 0x7fbb641a0fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:55.333 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.334+0000 7fbb68a86640 1 -- 172.21.15.52:0/2581870803 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb641a1790 con 0x7fbb641016d0 2024-03-04T09:30:55.334 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.334+0000 7fbb69a88640 1 --2- 172.21.15.52:0/2581870803 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbb64102090 0x7fbb641a0fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-04T09:30:55.334 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.334+0000 7fbb69287640 1 --2- 172.21.15.52:0/2581870803 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbb641004e0 0x7fbb6406a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-04T09:30:55.334 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.334+0000 7fbb68a86640 1 --2- 172.21.15.52:0/2581870803 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbb641016d0 0x7fbb6419ea50 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fbb5800e9c0 tx=0x7fbb5800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:30:55.334 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.335+0000 7fbb627fc640 1 -- 172.21.15.52:0/2581870803 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb5800ccc0 con 0x7fbb641016d0 2024-03-04T09:30:55.334 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.335+0000 7fbb627fc640 1 -- 172.21.15.52:0/2581870803 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fbb58005500 con 0x7fbb641016d0 2024-03-04T09:30:55.334 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.335+0000 7fbb6b4eb640 1 -- 172.21.15.52:0/2581870803 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb641a1a20 con 0x7fbb641016d0 2024-03-04T09:30:55.334 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.335+0000 7fbb627fc640 1 -- 172.21.15.52:0/2581870803 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb58010430 con 0x7fbb641016d0 2024-03-04T09:30:55.334 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.335+0000 7fbb6b4eb640 1 -- 172.21.15.52:0/2581870803 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb641a1f70 con 0x7fbb641016d0 2024-03-04T09:30:55.341 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.337+0000 7fbb627fc640 1 -- 172.21.15.52:0/2581870803 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) v1 ==== 136756+0+0 (secure 0 0 0) 0x7fbb580046c0 con 0x7fbb641016d0 2024-03-04T09:30:55.341 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.337+0000 7fbb6e1aa740 1 Processor -- start 2024-03-04T09:30:55.341 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.337+0000 7fbb6e1aa740 1 -- start start 2024-03-04T09:30:55.342 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.337+0000 7fbb6e1aa740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563808f3a7b0 0x563808f3abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:30:55.342 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.337+0000 7fbb6e1aa740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563808f3b980 0x563808f3be00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:30:55.342 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.337+0000 7fbb6e1aa740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563808f3c340 0x563808f42bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:30:55.342 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.337+0000 7fbb6e1aa740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x563808e9f6b0 con 0x563808f3a7b0 2024-03-04T09:30:55.342 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.337+0000 7fbb6e1aa740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x563808e48a60 con 0x563808f3b980 2024-03-04T09:30:55.342 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.337+0000 7fbb6e1aa740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x563808e9b750 con 0x563808f3c340 2024-03-04T09:30:55.342 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.337+0000 7fbb627fc640 1 --2- 172.21.15.52:0/2581870803 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7fbb4c0a1db0 0x7fbb4c0a4270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:30:55.342 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.337+0000 7fbb69a88640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563808f3a7b0 0x563808f3abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:30:55.342 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.338+0000 7fbb69a88640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563808f3a7b0 0x563808f3abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:46290/0 (socket says 172.21.15.52:46290) 2024-03-04T09:30:55.342 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.338+0000 7fbb69a88640 1 -- 172.21.15.52:0/2573330055 learned_addr learned my addr 172.21.15.52:0/2573330055 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-04T09:30:55.342 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.338+0000 7fbb69287640 1 --2- 172.21.15.52:0/2573330055 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563808f3b980 0x563808f3be00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:30:55.342 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.338+0000 7fbb68a86640 1 --2- 172.21.15.52:0/2573330055 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563808f3c340 0x563808f42bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:30:55.342 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.338+0000 7fbb69a88640 1 --2- 172.21.15.52:0/2581870803 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7fbb4c0a1db0 0x7fbb4c0a4270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:30:55.343 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.338+0000 7fbb627fc640 1 -- 172.21.15.52:0/2581870803 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9158+0+0 (secure 0 0 0) 0x7fbb580ce3d0 con 0x7fbb641016d0 2024-03-04T09:30:55.343 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.338+0000 7fbb69287640 1 -- 172.21.15.52:0/2573330055 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563808f3c340 msgr2=0x563808f42bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T09:30:55.343 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.338+0000 7fbb69287640 1 --2- 172.21.15.52:0/2573330055 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563808f3c340 0x563808f42bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:55.343 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.338+0000 7fbb69287640 1 -- 172.21.15.52:0/2573330055 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563808f3a7b0 msgr2=0x563808f3abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T09:30:55.343 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.338+0000 7fbb69287640 1 --2- 172.21.15.52:0/2573330055 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563808f3a7b0 0x563808f3abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:55.343 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.338+0000 7fbb69287640 1 -- 172.21.15.52:0/2573330055 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563808e66f30 con 0x563808f3b980 2024-03-04T09:30:55.343 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.338+0000 7fbb68a86640 1 --2- 172.21.15.52:0/2573330055 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563808f3c340 0x563808f42bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-04T09:30:55.343 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.338+0000 7fbb69a88640 1 --2- 172.21.15.52:0/2581870803 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7fbb4c0a1db0 0x7fbb4c0a4270 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fbb540099e0 tx=0x7fbb54009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:30:55.343 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.338+0000 7fbb69287640 1 --2- 172.21.15.52:0/2573330055 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563808f3b980 0x563808f3be00 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fbb5c02c0b0 tx=0x7fbb5c034ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5003c1990d2899e6 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:30:55.343 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.338+0000 7fbb4b7fe640 1 -- 172.21.15.52:0/2573330055 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb5c044070 con 0x563808f3b980 2024-03-04T09:30:55.343 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.338+0000 7fbb4b7fe640 1 -- 172.21.15.52:0/2573330055 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fbb5c034640 con 0x563808f3b980 2024-03-04T09:30:55.343 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.338+0000 7fbb4b7fe640 1 -- 172.21.15.52:0/2573330055 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb5c039040 con 0x563808f3b980 2024-03-04T09:30:55.343 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.339+0000 7fbb6e1aa740 1 -- 172.21.15.52:0/2573330055 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563808f3b980 msgr2=0x563808f3be00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:30:55.343 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.339+0000 7fbb6e1aa740 1 --2- 172.21.15.52:0/2573330055 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563808f3b980 0x563808f3be00 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fbb5c02c0b0 tx=0x7fbb5c034ad0 comp rx=0 tx=0).stop 2024-03-04T09:30:55.344 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.339+0000 7fbb6e1aa740 1 -- 172.21.15.52:0/2573330055 shutdown_connections 2024-03-04T09:30:55.344 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.339+0000 7fbb6e1aa740 1 --2- 172.21.15.52:0/2573330055 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563808f3c340 0x563808f42bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:55.344 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.339+0000 7fbb6e1aa740 1 --2- 172.21.15.52:0/2573330055 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563808f3b980 0x563808f3be00 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:55.344 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.339+0000 7fbb6e1aa740 1 --2- 172.21.15.52:0/2573330055 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563808f3a7b0 0x563808f3abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:55.344 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.339+0000 7fbb6e1aa740 1 -- 172.21.15.52:0/2573330055 >> 172.21.15.52:0/2573330055 conn(0x563808f35ba0 msgr2=0x563808f35fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:30:55.344 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.339+0000 7fbb6e1aa740 1 -- 172.21.15.52:0/2573330055 shutdown_connections 2024-03-04T09:30:55.344 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.339+0000 7fbb6e1aa740 1 -- 172.21.15.52:0/2573330055 wait complete. 2024-03-04T09:30:55.344 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.339+0000 7fbb6e1aa740 1 -- start start 2024-03-04T09:30:55.344 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.339+0000 7fbb6e1aa740 1 Processor -- start 2024-03-04T09:30:55.344 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.340+0000 7fbb6e1aa740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563808f3a7b0 0x563808f44cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:30:55.344 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.340+0000 7fbb6e1aa740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563808f3b980 0x563808f45200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:30:55.344 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.340+0000 7fbb6e1aa740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563808f3c340 0x563808f4c240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:30:55.345 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.340+0000 7fbb6e1aa740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x563808f432f0 con 0x563808f3c340 2024-03-04T09:30:55.345 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.340+0000 7fbb6e1aa740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x563808e5bea0 con 0x563808f3b980 2024-03-04T09:30:55.345 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.340+0000 7fbb6e1aa740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x563808f43470 con 0x563808f3a7b0 2024-03-04T09:30:55.345 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.340+0000 7fbb6e1aa740 10 client.0 _ll_register_callbacks cb 0x563808e5b870 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-04T09:30:55.345 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.340+0000 7fbb69a88640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563808f3c340 0x563808f4c240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:30:55.345 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.340+0000 7fbb69a88640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563808f3c340 0x563808f4c240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:46292/0 (socket says 172.21.15.52:46292) 2024-03-04T09:30:55.345 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.340+0000 7fbb69a88640 1 -- 172.21.15.52:0/3925126974 learned_addr learned my addr 172.21.15.52:0/3925126974 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-04T09:30:55.345 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.340+0000 7fbb69287640 1 --2- 172.21.15.52:0/3925126974 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563808f3a7b0 0x563808f44cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:30:55.345 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.340+0000 7fbb68a86640 1 --2- 172.21.15.52:0/3925126974 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563808f3b980 0x563808f45200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:30:55.345 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.340+0000 7fbb69a88640 1 -- 172.21.15.52:0/3925126974 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563808f3a7b0 msgr2=0x563808f44cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T09:30:55.345 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.340+0000 7fbb69a88640 1 --2- 172.21.15.52:0/3925126974 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563808f3a7b0 0x563808f44cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:55.345 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.340+0000 7fbb69a88640 1 -- 172.21.15.52:0/3925126974 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563808f3b980 msgr2=0x563808f45200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T09:30:55.345 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.340+0000 7fbb69a88640 1 --2- 172.21.15.52:0/3925126974 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563808f3b980 0x563808f45200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:55.345 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.340+0000 7fbb69a88640 1 -- 172.21.15.52:0/3925126974 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563808e5c270 con 0x563808f3c340 2024-03-04T09:30:55.346 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.340+0000 7fbb69a88640 1 --2- 172.21.15.52:0/3925126974 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563808f3c340 0x563808f4c240 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fbb54003e70 tx=0x7fbb54004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a961a7f1ef2e1ce3 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:30:55.346 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.340+0000 7fbb48ff9640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-03-04T09:30:55.346 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.340+0000 7fbb48ff9640 1 -- 172.21.15.52:0/3925126974 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb54004d20 con 0x563808f3c340 2024-03-04T09:30:55.346 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.340+0000 7fbb48ff9640 1 -- 172.21.15.52:0/3925126974 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9158+0+0 (secure 0 0 0) 0x7fbb54020670 con 0x563808f3c340 2024-03-04T09:30:55.346 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.340+0000 7fbb6e1aa740 1 -- 172.21.15.52:0/3925126974 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fbb5c034640 con 0x563808f3c340 2024-03-04T09:30:55.346 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.341+0000 7fbb48ff9640 1 -- 172.21.15.52:0/3925126974 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbb54033050 con 0x563808f3c340 2024-03-04T09:30:55.346 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.341+0000 7fbb6e1aa740 10 client.15435 fetch_fsmap learned FSMap version 24 2024-03-04T09:30:55.346 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.341+0000 7fbb6e1aa740 1 -- 172.21.15.52:0/3925126974 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x563808bb42d0 con 0x563808f3c340 2024-03-04T09:30:55.346 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.341+0000 7fbb48ff9640 1 -- 172.21.15.52:0/3925126974 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fbb5c034640 con 0x563808f3c340 2024-03-04T09:30:55.346 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.341+0000 7fbb48ff9640 1 -- 172.21.15.52:0/3925126974 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7fbb54027030 con 0x563808f3c340 2024-03-04T09:30:55.346 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.341+0000 7fbb6e1aa740 10 client.15435 fetch_fsmap finished waiting for FSMap version 24 2024-03-04T09:30:55.347 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.341+0000 7fbb6e1aa740 10 client.15435 resolve_mds: resolved name 'b' to daemon mds.b 2024-03-04T09:30:55.347 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.341+0000 7fbb6e1aa740 20 client.15435 populate_metadata read hostname 'smithi052' 2024-03-04T09:30:55.347 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.341+0000 7fbb6e1aa740 1 --2- 172.21.15.52:0/3925126974 >> [v2:172.21.15.106:6838/1995853093,v1:172.21.15.106:6839/1995853093] conn(0x563808f51900 0x563808f508d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:30:55.347 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.341+0000 7fbb6e1aa740 4 client.15435 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-04T09:30:55.347 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.341+0000 7fbb6e1aa740 1 -- 172.21.15.52:0/3925126974 --> [v2:172.21.15.106:6838/1995853093,v1:172.21.15.106:6839/1995853093] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563808eacbf0 con 0x563808f51900 2024-03-04T09:30:55.347 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.341+0000 7fbb69287640 1 --2- 172.21.15.52:0/3925126974 >> [v2:172.21.15.106:6838/1995853093,v1:172.21.15.106:6839/1995853093] conn(0x563808f51900 0x563808f508d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:30:55.347 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.342+0000 7fbb69287640 1 --2- 172.21.15.52:0/3925126974 >> [v2:172.21.15.106:6838/1995853093,v1:172.21.15.106:6839/1995853093] conn(0x563808f51900 0x563808f508d0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3d813dd46a005f50 server_cookie=d95680a816da97fc in_seq=0 out_seq=0 2024-03-04T09:30:55.347 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.342+0000 7fbb48ff9640 10 client.15435 ms_handle_connect on v2:172.21.15.106:6838/1995853093 2024-03-04T09:30:55.347 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.342+0000 7fbb48ff9640 1 -- 172.21.15.52:0/3925126974 <== mds.0 v2:172.21.15.106:6838/1995853093 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x563808eacbf0 con 0x563808f51900 2024-03-04T09:30:55.347 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.342+0000 7fbb48ff9640 10 client.15435 handle_command_reply: tid=0 2024-03-04T09:30:55.347 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.342+0000 7fbb6e1aa740 1 client.15435 shutdown 2024-03-04T09:30:55.348 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.342+0000 7fbb6e1aa740 1 -- 172.21.15.52:0/3925126974 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563808f3c340 msgr2=0x563808f4c240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:30:55.348 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.342+0000 7fbb6e1aa740 1 --2- 172.21.15.52:0/3925126974 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563808f3c340 0x563808f4c240 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fbb54003e70 tx=0x7fbb54004a40 comp rx=0 tx=0).stop 2024-03-04T09:30:55.348 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.342+0000 7fbb6e1aa740 1 -- 172.21.15.52:0/3925126974 shutdown_connections 2024-03-04T09:30:55.348 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.342+0000 7fbb6e1aa740 1 --2- 172.21.15.52:0/3925126974 >> [v2:172.21.15.106:6838/1995853093,v1:172.21.15.106:6839/1995853093] conn(0x563808f51900 0x563808f508d0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:55.348 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.342+0000 7fbb6e1aa740 1 --2- 172.21.15.52:0/3925126974 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563808f3c340 0x563808f4c240 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:55.348 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.342+0000 7fbb6e1aa740 1 --2- 172.21.15.52:0/3925126974 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563808f3b980 0x563808f45200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:55.348 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.342+0000 7fbb48ff9640 0 client.15435 ms_handle_reset on v2:172.21.15.106:6838/1995853093 2024-03-04T09:30:55.348 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.342+0000 7fbb6e1aa740 1 --2- 172.21.15.52:0/3925126974 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563808f3a7b0 0x563808f44cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:55.348 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.342+0000 7fbb6e1aa740 1 -- 172.21.15.52:0/3925126974 >> 172.21.15.52:0/3925126974 conn(0x563808f35ba0 msgr2=0x563808f34da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:30:55.348 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.343+0000 7fbb6e1aa740 1 -- 172.21.15.52:0/3925126974 shutdown_connections 2024-03-04T09:30:55.349 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.343+0000 7fbb6e1aa740 1 -- 172.21.15.52:0/3925126974 wait complete. 2024-03-04T09:30:55.349 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.343+0000 7fbb6e1aa740 20 client.15435 trim_cache size 0 max 16384 2024-03-04T09:30:55.361 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.359+0000 7fbb6e1aa740 1 Processor -- start 2024-03-04T09:30:55.361 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.359+0000 7fbb6e1aa740 1 -- start start 2024-03-04T09:30:55.361 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.359+0000 7fbb6e1aa740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563808f3a7b0 0x563808f43870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:30:55.361 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.359+0000 7fbb6e1aa740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563808f3b980 0x563808f4c160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:30:55.362 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.359+0000 7fbb6e1aa740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563808f3c340 0x563808f43db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:30:55.362 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.359+0000 7fbb6e1aa740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x563808f51120 con 0x563808f3a7b0 2024-03-04T09:30:55.362 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.359+0000 7fbb6e1aa740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x563808f51780 con 0x563808f3c340 2024-03-04T09:30:55.362 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.359+0000 7fbb6e1aa740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x563808f51de0 con 0x563808f3b980 2024-03-04T09:30:55.362 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.359+0000 7fbb69287640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563808f3a7b0 0x563808f43870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:30:55.362 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.359+0000 7fbb69287640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563808f3a7b0 0x563808f43870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:46306/0 (socket says 172.21.15.52:46306) 2024-03-04T09:30:55.362 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.360+0000 7fbb69287640 1 -- 172.21.15.52:0/718279952 learned_addr learned my addr 172.21.15.52:0/718279952 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-04T09:30:55.362 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.360+0000 7fbb68a86640 1 --2- 172.21.15.52:0/718279952 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563808f3b980 0x563808f4c160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:30:55.362 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.360+0000 7fbb69a88640 1 --2- 172.21.15.52:0/718279952 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563808f3c340 0x563808f43db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:30:55.362 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.360+0000 7fbb69287640 1 -- 172.21.15.52:0/718279952 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563808f3b980 msgr2=0x563808f4c160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T09:30:55.362 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.360+0000 7fbb69287640 1 --2- 172.21.15.52:0/718279952 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563808f3b980 0x563808f4c160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:55.362 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.360+0000 7fbb69287640 1 -- 172.21.15.52:0/718279952 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563808f3c340 msgr2=0x563808f43db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T09:30:55.363 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.360+0000 7fbb69287640 1 --2- 172.21.15.52:0/718279952 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563808f3c340 0x563808f43db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:55.363 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.360+0000 7fbb69287640 1 -- 172.21.15.52:0/718279952 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563808c7e6b0 con 0x563808f3a7b0 2024-03-04T09:30:55.363 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.360+0000 7fbb69a88640 1 --2- 172.21.15.52:0/718279952 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563808f3c340 0x563808f43db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T09:30:55.363 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.360+0000 7fbb68a86640 1 --2- 172.21.15.52:0/718279952 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563808f3b980 0x563808f4c160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T09:30:55.363 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.360+0000 7fbb69287640 1 --2- 172.21.15.52:0/718279952 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563808f3a7b0 0x563808f43870 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fbb5c0349e0 tx=0x7fbb5c02bb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44243ee43eaa2666 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:30:55.363 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.360+0000 7fbb4affd640 1 -- 172.21.15.52:0/718279952 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb5c054070 con 0x563808f3a7b0 2024-03-04T09:30:55.363 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.360+0000 7fbb4affd640 1 -- 172.21.15.52:0/718279952 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fbb5c004040 con 0x563808f3a7b0 2024-03-04T09:30:55.363 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.360+0000 7fbb4affd640 1 -- 172.21.15.52:0/718279952 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb5c039040 con 0x563808f3a7b0 2024-03-04T09:30:55.363 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.361+0000 7fbb6e1aa740 1 -- 172.21.15.52:0/718279952 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563808f3a7b0 msgr2=0x563808f43870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:30:55.363 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.361+0000 7fbb6e1aa740 1 --2- 172.21.15.52:0/718279952 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563808f3a7b0 0x563808f43870 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fbb5c0349e0 tx=0x7fbb5c02bb70 comp rx=0 tx=0).stop 2024-03-04T09:30:55.363 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.361+0000 7fbb6e1aa740 1 -- 172.21.15.52:0/718279952 shutdown_connections 2024-03-04T09:30:55.364 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.361+0000 7fbb6e1aa740 1 --2- 172.21.15.52:0/718279952 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563808f3c340 0x563808f43db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:55.364 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.361+0000 7fbb6e1aa740 1 --2- 172.21.15.52:0/718279952 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563808f3b980 0x563808f4c160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:55.364 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.361+0000 7fbb6e1aa740 1 --2- 172.21.15.52:0/718279952 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563808f3a7b0 0x563808f43870 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:55.364 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.361+0000 7fbb6e1aa740 1 -- 172.21.15.52:0/718279952 >> 172.21.15.52:0/718279952 conn(0x563808f35ba0 msgr2=0x563808f44cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:30:55.364 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.361+0000 7fbb6e1aa740 1 -- 172.21.15.52:0/718279952 shutdown_connections 2024-03-04T09:30:55.364 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.361+0000 7fbb6e1aa740 1 -- 172.21.15.52:0/718279952 wait complete. 2024-03-04T09:30:55.364 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.361+0000 7fbb6e1aa740 1 -- start start 2024-03-04T09:30:55.364 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.361+0000 7fbb6e1aa740 1 Processor -- start 2024-03-04T09:30:55.364 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.361+0000 7fbb6e1aa740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563808f3a7b0 0x563808f46380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:30:55.364 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.361+0000 7fbb6e1aa740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563808f3b980 0x563808f46900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:30:55.364 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.361+0000 7fbb6e1aa740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563808f3c340 0x563808f4b660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:30:55.364 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.361+0000 7fbb6e1aa740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x563808f52aa0 con 0x563808f3c340 2024-03-04T09:30:55.364 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.361+0000 7fbb6e1aa740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x563808f52440 con 0x563808f3b980 2024-03-04T09:30:55.364 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.361+0000 7fbb6e1aa740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x563808f53100 con 0x563808f3a7b0 2024-03-04T09:30:55.365 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.361+0000 7fbb6e1aa740 10 client.0 _ll_register_callbacks cb 0x7fbb54003c80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-04T09:30:55.365 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.362+0000 7fbb69a88640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563808f3c340 0x563808f4b660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:30:55.365 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.362+0000 7fbb69a88640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563808f3c340 0x563808f4b660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:46322/0 (socket says 172.21.15.52:46322) 2024-03-04T09:30:55.365 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.362+0000 7fbb69a88640 1 -- 172.21.15.52:0/2198080662 learned_addr learned my addr 172.21.15.52:0/2198080662 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-04T09:30:55.365 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.362+0000 7fbb69287640 1 --2- 172.21.15.52:0/2198080662 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563808f3a7b0 0x563808f46380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:30:55.365 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.362+0000 7fbb68a86640 1 --2- 172.21.15.52:0/2198080662 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563808f3b980 0x563808f46900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:30:55.365 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.362+0000 7fbb69a88640 1 -- 172.21.15.52:0/2198080662 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563808f3a7b0 msgr2=0x563808f46380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T09:30:55.365 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.362+0000 7fbb69a88640 1 --2- 172.21.15.52:0/2198080662 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563808f3a7b0 0x563808f46380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:55.365 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.362+0000 7fbb69a88640 1 -- 172.21.15.52:0/2198080662 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563808f3b980 msgr2=0x563808f46900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T09:30:55.365 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.362+0000 7fbb69a88640 1 --2- 172.21.15.52:0/2198080662 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563808f3b980 0x563808f46900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:55.365 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.362+0000 7fbb69a88640 1 -- 172.21.15.52:0/2198080662 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563808f4bc50 con 0x563808f3c340 2024-03-04T09:30:55.365 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.362+0000 7fbb69a88640 1 --2- 172.21.15.52:0/2198080662 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563808f3c340 0x563808f4b660 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fbb54005f90 tx=0x7fbb54005fc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa2d22fd69f7a888 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:30:55.366 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.362+0000 7fbb48ff9640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-03-04T09:30:55.366 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.362+0000 7fbb48ff9640 1 -- 172.21.15.52:0/2198080662 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb5403c060 con 0x563808f3c340 2024-03-04T09:30:55.366 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.362+0000 7fbb48ff9640 1 -- 172.21.15.52:0/2198080662 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9158+0+0 (secure 0 0 0) 0x7fbb54004ab0 con 0x563808f3c340 2024-03-04T09:30:55.366 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.362+0000 7fbb6e1aa740 1 -- 172.21.15.52:0/2198080662 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fbb5c004040 con 0x563808f3c340 2024-03-04T09:30:55.366 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.362+0000 7fbb48ff9640 1 -- 172.21.15.52:0/2198080662 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbb54043050 con 0x563808f3c340 2024-03-04T09:30:55.366 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.362+0000 7fbb6e1aa740 10 client.15441 fetch_fsmap learned FSMap version 24 2024-03-04T09:30:55.366 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.362+0000 7fbb6e1aa740 1 -- 172.21.15.52:0/2198080662 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x563808fad080 con 0x563808f3c340 2024-03-04T09:30:55.366 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.362+0000 7fbb48ff9640 1 -- 172.21.15.52:0/2198080662 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fbb5c004040 con 0x563808f3c340 2024-03-04T09:30:55.366 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.363+0000 7fbb48ff9640 1 -- 172.21.15.52:0/2198080662 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7fbb54027070 con 0x563808f3c340 2024-03-04T09:30:55.366 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.363+0000 7fbb6e1aa740 10 client.15441 fetch_fsmap finished waiting for FSMap version 24 2024-03-04T09:30:55.366 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.363+0000 7fbb6e1aa740 10 client.15441 resolve_mds: resolved name 'b' to daemon mds.b 2024-03-04T09:30:55.366 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.363+0000 7fbb6e1aa740 20 client.15441 populate_metadata read hostname 'smithi052' 2024-03-04T09:30:55.366 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.363+0000 7fbb6e1aa740 1 --2- 172.21.15.52:0/2198080662 >> [v2:172.21.15.106:6838/1995853093,v1:172.21.15.106:6839/1995853093] conn(0x563808fb1720 0x563808fad310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:30:55.366 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.363+0000 7fbb6e1aa740 4 client.15441 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-04T09:30:55.367 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.363+0000 7fbb6e1aa740 1 -- 172.21.15.52:0/2198080662 --> [v2:172.21.15.106:6838/1995853093,v1:172.21.15.106:6839/1995853093] -- command(tid 0: {"prefix": "status"}) v1 -- 0x563808ea6f40 con 0x563808fb1720 2024-03-04T09:30:55.367 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.363+0000 7fbb69287640 1 --2- 172.21.15.52:0/2198080662 >> [v2:172.21.15.106:6838/1995853093,v1:172.21.15.106:6839/1995853093] conn(0x563808fb1720 0x563808fad310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:30:55.367 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.363+0000 7fbb48ff9640 10 client.15441 ms_handle_connect on v2:172.21.15.106:6838/1995853093 2024-03-04T09:30:55.367 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.363+0000 7fbb69287640 1 --2- 172.21.15.52:0/2198080662 >> [v2:172.21.15.106:6838/1995853093,v1:172.21.15.106:6839/1995853093] conn(0x563808fb1720 0x563808fad310 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a30939d4d944e7c server_cookie=d8ab0e49a9c56e21 in_seq=0 out_seq=0 2024-03-04T09:30:55.367 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.363+0000 7fbb48ff9640 1 -- 172.21.15.52:0/2198080662 <== mds.0 v2:172.21.15.106:6838/1995853093 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x563808ea6f40 con 0x563808fb1720 2024-03-04T09:30:55.367 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.363+0000 7fbb48ff9640 10 client.15441 handle_command_reply: tid=0 2024-03-04T09:30:55.367 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.363+0000 7fbb6e1aa740 1 client.15441 shutdown 2024-03-04T09:30:55.367 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.364+0000 7fbb6e1aa740 1 -- 172.21.15.52:0/2198080662 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563808f3c340 msgr2=0x563808f4b660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:30:55.367 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.364+0000 7fbb6e1aa740 1 --2- 172.21.15.52:0/2198080662 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563808f3c340 0x563808f4b660 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fbb54005f90 tx=0x7fbb54005fc0 comp rx=0 tx=0).stop 2024-03-04T09:30:55.367 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.364+0000 7fbb6e1aa740 1 -- 172.21.15.52:0/2198080662 shutdown_connections 2024-03-04T09:30:55.367 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.364+0000 7fbb6e1aa740 1 --2- 172.21.15.52:0/2198080662 >> [v2:172.21.15.106:6838/1995853093,v1:172.21.15.106:6839/1995853093] conn(0x563808fb1720 0x563808fad310 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:55.367 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.364+0000 7fbb6e1aa740 1 --2- 172.21.15.52:0/2198080662 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563808f3c340 0x563808f4b660 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:55.367 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.364+0000 7fbb6e1aa740 1 --2- 172.21.15.52:0/2198080662 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563808f3b980 0x563808f46900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:55.368 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.364+0000 7fbb6e1aa740 1 --2- 172.21.15.52:0/2198080662 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563808f3a7b0 0x563808f46380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:55.368 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.364+0000 7fbb6e1aa740 1 -- 172.21.15.52:0/2198080662 >> 172.21.15.52:0/2198080662 conn(0x563808f35ba0 msgr2=0x563808f39690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:30:55.368 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.364+0000 7fbb48ff9640 0 client.15441 ms_handle_reset on v2:172.21.15.106:6838/1995853093 2024-03-04T09:30:55.368 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.364+0000 7fbb6e1aa740 1 -- 172.21.15.52:0/2198080662 shutdown_connections 2024-03-04T09:30:55.368 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.364+0000 7fbb6e1aa740 1 -- 172.21.15.52:0/2198080662 wait complete. 2024-03-04T09:30:55.368 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.364+0000 7fbb6e1aa740 20 client.15441 trim_cache size 0 max 16384 2024-03-04T09:30:55.368 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2024-03-04T09:30:55.368 INFO:teuthology.orchestra.run.smithi052.stdout: "cluster_fsid": "4e059f08-da08-11ee-95c6-87774f69a715", 2024-03-04T09:30:55.368 INFO:teuthology.orchestra.run.smithi052.stdout: "whoami": 0, 2024-03-04T09:30:55.369 INFO:teuthology.orchestra.run.smithi052.stdout: "id": 24506, 2024-03-04T09:30:55.369 INFO:teuthology.orchestra.run.smithi052.stdout: "want_state": "up:active", 2024-03-04T09:30:55.369 INFO:teuthology.orchestra.run.smithi052.stdout: "state": "up:active", 2024-03-04T09:30:55.369 INFO:teuthology.orchestra.run.smithi052.stdout: "fs_name": "cephfs", 2024-03-04T09:30:55.369 INFO:teuthology.orchestra.run.smithi052.stdout: "rank_uptime": 137.45791067499999, 2024-03-04T09:30:55.369 INFO:teuthology.orchestra.run.smithi052.stdout: "mdsmap_epoch": 24, 2024-03-04T09:30:55.369 INFO:teuthology.orchestra.run.smithi052.stdout: "osdmap_epoch": 93, 2024-03-04T09:30:55.369 INFO:teuthology.orchestra.run.smithi052.stdout: "osdmap_epoch_barrier": 89, 2024-03-04T09:30:55.369 INFO:teuthology.orchestra.run.smithi052.stdout: "uptime": 141.70195834099999 2024-03-04T09:30:55.369 INFO:teuthology.orchestra.run.smithi052.stdout:} 2024-03-04T09:30:55.370 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.364+0000 7fbb6b4eb640 1 -- 172.21.15.52:0/2581870803 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7fbb4c0a1db0 msgr2=0x7fbb4c0a4270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:30:55.370 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.364+0000 7fbb6b4eb640 1 --2- 172.21.15.52:0/2581870803 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7fbb4c0a1db0 0x7fbb4c0a4270 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fbb540099e0 tx=0x7fbb54009500 comp rx=0 tx=0).stop 2024-03-04T09:30:55.370 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.364+0000 7fbb6b4eb640 1 -- 172.21.15.52:0/2581870803 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbb641016d0 msgr2=0x7fbb6419ea50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:30:55.370 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.364+0000 7fbb6b4eb640 1 --2- 172.21.15.52:0/2581870803 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbb641016d0 0x7fbb6419ea50 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fbb5800e9c0 tx=0x7fbb5800ee80 comp rx=0 tx=0).stop 2024-03-04T09:30:55.370 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.364+0000 7fbb6b4eb640 1 -- 172.21.15.52:0/2581870803 shutdown_connections 2024-03-04T09:30:55.370 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.364+0000 7fbb6b4eb640 1 --2- 172.21.15.52:0/2581870803 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7fbb4c0a1db0 0x7fbb4c0a4270 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:55.371 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.364+0000 7fbb6b4eb640 1 --2- 172.21.15.52:0/2581870803 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbb64102090 0x7fbb641a0fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:55.371 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.364+0000 7fbb6b4eb640 1 --2- 172.21.15.52:0/2581870803 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbb641016d0 0x7fbb6419ea50 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:55.371 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.364+0000 7fbb6b4eb640 1 --2- 172.21.15.52:0/2581870803 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbb641004e0 0x7fbb6406a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:55.371 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.364+0000 7fbb6b4eb640 1 -- 172.21.15.52:0/2581870803 >> 172.21.15.52:0/2581870803 conn(0x7fbb640fba70 msgr2=0x7fbb641049d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:30:55.371 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.364+0000 7fbb6b4eb640 1 -- 172.21.15.52:0/2581870803 shutdown_connections 2024-03-04T09:30:55.371 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:55.364+0000 7fbb6b4eb640 1 -- 172.21.15.52:0/2581870803 wait complete. 2024-03-04T09:30:55.372 INFO:teuthology.run_tasks:Running task kclient... 2024-03-04T09:30:55.386 INFO:tasks.kclient:Mounting kernel clients... 2024-03-04T09:30:55.386 INFO:tasks.kclient:config is {'client.0': {}} 2024-03-04T09:30:55.386 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-04T09:30:55.387 DEBUG:teuthology.orchestra.run.smithi052:> ip netns list 2024-03-04T09:30:55.402 INFO:teuthology.orchestra.run.smithi052.stdout:ceph-ns--home-ubuntu-cephtest-mnt.admin (id: 0) 2024-03-04T09:30:55.402 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-04T09:30:55.402 DEBUG:teuthology.orchestra.run.smithi052:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-03-04T09:30:55.469 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-04T09:30:55.470 DEBUG:teuthology.orchestra.run.smithi052:> sudo ip link delete ceph-brx 2024-03-04T09:30:55.476 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:30:55 smithi052 ceph-mon[32473]: pgmap v360: 129 pgs: 129 active+clean; 641 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s; 1 B/s, 0 keys/s, 0 objects/s recovering 2024-03-04T09:30:55.477 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:30:55 smithi052 ceph-mon[32473]: from='client.? 172.21.15.52:0/4142970534' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-04T09:30:55.477 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:30:55 smithi052 ceph-mon[32473]: from='client.? 172.21.15.52:0/389816542' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-04T09:30:55.533 INFO:tasks.kclient:client.0 config is {'mntopts': ['ms_mode=legacy', 'nowsync'], 'syntax': 'v1', 'mount_subvol_num': 1} 2024-03-04T09:30:55.533 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-03-04T09:30:55.533 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-03-04T09:30:55.533 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd shell --fsid 4e059f08-da08-11ee-95c6-87774f69a715 -- ceph fs ls 2024-03-04T09:30:55.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:30:55 smithi191 ceph-mon[36816]: pgmap v360: 129 pgs: 129 active+clean; 641 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s; 1 B/s, 0 keys/s, 0 objects/s recovering 2024-03-04T09:30:55.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:30:55 smithi191 ceph-mon[36816]: from='client.? 172.21.15.52:0/4142970534' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-04T09:30:55.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:30:55 smithi191 ceph-mon[36816]: from='client.? 172.21.15.52:0/389816542' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-04T09:30:55.631 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:30:55 smithi106 ceph-mon[36484]: pgmap v360: 129 pgs: 129 active+clean; 641 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s; 1 B/s, 0 keys/s, 0 objects/s recovering 2024-03-04T09:30:55.631 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:30:55 smithi106 ceph-mon[36484]: from='client.? 172.21.15.52:0/4142970534' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-04T09:30:55.631 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:30:55 smithi106 ceph-mon[36484]: from='client.? 172.21.15.52:0/389816542' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-04T09:30:55.775 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/4e059f08-da08-11ee-95c6-87774f69a715/mon.a/config 2024-03-04T09:30:57.430 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:57.431+0000 7f0dd238b700 1 -- 172.21.15.52:0/1073401269 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0dcc101960 msgr2=0x7f0dcc10de40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:30:57.430 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:57.431+0000 7f0dd238b700 1 --2- 172.21.15.52:0/1073401269 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0dcc101960 0x7f0dcc10de40 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f0dc400b0a0 tx=0x7f0dc400b3b0 comp rx=0 tx=0).stop 2024-03-04T09:30:57.430 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:57.431+0000 7f0dd238b700 1 -- 172.21.15.52:0/1073401269 shutdown_connections 2024-03-04T09:30:57.430 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:57.431+0000 7f0dd238b700 1 --2- 172.21.15.52:0/1073401269 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0dcc107000 0x7f0dcc1073e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:57.430 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:57.431+0000 7f0dd238b700 1 --2- 172.21.15.52:0/1073401269 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0dcc101960 0x7f0dcc10de40 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:57.430 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:57.431+0000 7f0dd238b700 1 --2- 172.21.15.52:0/1073401269 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0dcc100fc0 0x7f0dcc101420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:57.430 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:57.431+0000 7f0dd238b700 1 -- 172.21.15.52:0/1073401269 >> 172.21.15.52:0/1073401269 conn(0x7f0dcc077ef0 msgr2=0x7f0dcc078300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:30:57.431 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:57.431+0000 7f0dd238b700 1 -- 172.21.15.52:0/1073401269 shutdown_connections 2024-03-04T09:30:57.431 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:57.431+0000 7f0dd238b700 1 -- 172.21.15.52:0/1073401269 wait complete. 2024-03-04T09:30:57.431 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:57.432+0000 7f0dd238b700 1 Processor -- start 2024-03-04T09:30:57.431 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:57.432+0000 7f0dd238b700 1 -- start start 2024-03-04T09:30:57.431 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:57.432+0000 7f0dd238b700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0dcc100fc0 0x7f0dcc102c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:30:57.431 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:57.432+0000 7f0dd238b700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0dcc101960 0x7f0dcc1031b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:30:57.431 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:57.432+0000 7f0dd238b700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0dcc107000 0x7f0dcc1036f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:30:57.431 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:57.432+0000 7f0dd238b700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0dcc1112d0 con 0x7f0dcc107000 2024-03-04T09:30:57.431 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:57.432+0000 7f0dd238b700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0dcc111150 con 0x7f0dcc101960 2024-03-04T09:30:57.431 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:57.432+0000 7f0dd238b700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0dcc111450 con 0x7f0dcc100fc0 2024-03-04T09:30:57.431 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:57.432+0000 7f0dd0928700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0dcc107000 0x7f0dcc1036f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:30:57.432 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:57.432+0000 7f0dd0928700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0dcc107000 0x7f0dcc1036f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:46342/0 (socket says 172.21.15.52:46342) 2024-03-04T09:30:57.432 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:57.432+0000 7f0dd0928700 1 -- 172.21.15.52:0/324805421 learned_addr learned my addr 172.21.15.52:0/324805421 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-04T09:30:57.432 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:57.432+0000 7f0dcb7fe700 1 --2- 172.21.15.52:0/324805421 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0dcc101960 0x7f0dcc1031b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:30:57.432 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:57.432+0000 7f0dcbfff700 1 --2- 172.21.15.52:0/324805421 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0dcc100fc0 0x7f0dcc102c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:30:57.432 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:57.432+0000 7f0dcbfff700 1 -- 172.21.15.52:0/324805421 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0dcc101960 msgr2=0x7f0dcc1031b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:30:57.432 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:57.432+0000 7f0dcbfff700 1 --2- 172.21.15.52:0/324805421 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0dcc101960 0x7f0dcc1031b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:57.432 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:57.432+0000 7f0dcbfff700 1 -- 172.21.15.52:0/324805421 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0dcc107000 msgr2=0x7f0dcc1036f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:30:57.432 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:57.432+0000 7f0dcbfff700 1 --2- 172.21.15.52:0/324805421 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0dcc107000 0x7f0dcc1036f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:57.432 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:57.432+0000 7f0dcbfff700 1 -- 172.21.15.52:0/324805421 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0dcc1a6e00 con 0x7f0dcc100fc0 2024-03-04T09:30:57.432 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:57.433+0000 7f0dcb7fe700 1 --2- 172.21.15.52:0/324805421 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0dcc101960 0x7f0dcc1031b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-04T09:30:57.432 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:57.433+0000 7f0dd0928700 1 --2- 172.21.15.52:0/324805421 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0dcc107000 0x7f0dcc1036f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-04T09:30:57.433 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:57.433+0000 7f0dcbfff700 1 --2- 172.21.15.52:0/324805421 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0dcc100fc0 0x7f0dcc102c70 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f0dbc00e9f0 tx=0x7f0dbc00ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:30:57.433 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:57.433+0000 7f0dc97fa700 1 -- 172.21.15.52:0/324805421 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0dbc004220 con 0x7f0dcc100fc0 2024-03-04T09:30:57.433 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:57.433+0000 7f0dd238b700 1 -- 172.21.15.52:0/324805421 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0dcc1a70f0 con 0x7f0dcc100fc0 2024-03-04T09:30:57.433 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:57.433+0000 7f0dc97fa700 1 -- 172.21.15.52:0/324805421 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0dbc004d10 con 0x7f0dcc100fc0 2024-03-04T09:30:57.433 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:57.433+0000 7f0dc97fa700 1 -- 172.21.15.52:0/324805421 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0dbc005020 con 0x7f0dcc100fc0 2024-03-04T09:30:57.433 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:57.433+0000 7f0dd238b700 1 -- 172.21.15.52:0/324805421 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0dcc1a75e0 con 0x7f0dcc100fc0 2024-03-04T09:30:57.434 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:57.434+0000 7f0dd238b700 1 -- 172.21.15.52:0/324805421 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0dcc068d80 con 0x7f0dcc100fc0 2024-03-04T09:30:57.434 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:57.435+0000 7f0dc97fa700 1 -- 172.21.15.52:0/324805421 <== mon.1 v2:172.21.15.191:3300/0 4 ==== mgrmap(e 18) v1 ==== 136756+0+0 (secure 0 0 0) 0x7f0dbc004750 con 0x7f0dcc100fc0 2024-03-04T09:30:57.434 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:57.435+0000 7f0dc97fa700 1 --2- 172.21.15.52:0/324805421 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f0db40a1e10 0x7f0db40a42d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:30:57.434 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:57.436+0000 7f0dcb7fe700 1 --2- 172.21.15.52:0/324805421 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f0db40a1e10 0x7f0db40a42d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:30:57.434 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:57.436+0000 7f0dc97fa700 1 -- 172.21.15.52:0/324805421 <== mon.1 v2:172.21.15.191:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9158+0+0 (secure 0 0 0) 0x7f0dbc0cdf80 con 0x7f0dcc100fc0 2024-03-04T09:30:57.434 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:57.436+0000 7f0dcb7fe700 1 --2- 172.21.15.52:0/324805421 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f0db40a1e10 0x7f0db40a42d0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f0dcc0784f0 tx=0x7f0dc0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:30:57.440 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:57.443+0000 7f0dc97fa700 1 -- 172.21.15.52:0/324805421 <== mon.1 v2:172.21.15.191:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f0dbc09afb0 con 0x7f0dcc100fc0 2024-03-04T09:30:57.584 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:30:57 smithi052 ceph-mon[32473]: pgmap v361: 129 pgs: 129 active+clean; 641 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s; 1 B/s, 0 keys/s, 0 objects/s recovering 2024-03-04T09:30:57.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:30:57 smithi191 ceph-mon[36816]: pgmap v361: 129 pgs: 129 active+clean; 641 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s; 1 B/s, 0 keys/s, 0 objects/s recovering 2024-03-04T09:30:57.631 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:30:57 smithi106 ceph-mon[36484]: pgmap v361: 129 pgs: 129 active+clean; 641 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s; 1 B/s, 0 keys/s, 0 objects/s recovering 2024-03-04T09:30:57.692 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:57.695+0000 7f0dd238b700 1 -- 172.21.15.52:0/324805421 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f0dcc1041a0 con 0x7f0dcc100fc0 2024-03-04T09:30:57.692 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:57.695+0000 7f0dc97fa700 1 -- 172.21.15.52:0/324805421 <== mon.1 v2:172.21.15.191:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v24) v1 ==== 53+0+88 (secure 0 0 0) 0x7f0dcc1041a0 con 0x7f0dcc100fc0 2024-03-04T09:30:57.693 INFO:teuthology.orchestra.run.smithi052.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-03-04T09:30:57.695 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:57.697+0000 7f0dd238b700 1 -- 172.21.15.52:0/324805421 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f0db40a1e10 msgr2=0x7f0db40a42d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:30:57.695 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:57.697+0000 7f0dd238b700 1 --2- 172.21.15.52:0/324805421 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f0db40a1e10 0x7f0db40a42d0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f0dcc0784f0 tx=0x7f0dc0009260 comp rx=0 tx=0).stop 2024-03-04T09:30:57.695 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:57.697+0000 7f0dd238b700 1 -- 172.21.15.52:0/324805421 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0dcc100fc0 msgr2=0x7f0dcc102c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:30:57.695 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:57.697+0000 7f0dd238b700 1 --2- 172.21.15.52:0/324805421 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0dcc100fc0 0x7f0dcc102c70 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f0dbc00e9f0 tx=0x7f0dbc00ed00 comp rx=0 tx=0).stop 2024-03-04T09:30:57.695 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:57.698+0000 7f0dd238b700 1 -- 172.21.15.52:0/324805421 shutdown_connections 2024-03-04T09:30:57.695 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:57.698+0000 7f0dd238b700 1 --2- 172.21.15.52:0/324805421 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f0db40a1e10 0x7f0db40a42d0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:57.696 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:57.698+0000 7f0dd238b700 1 --2- 172.21.15.52:0/324805421 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0dcc100fc0 0x7f0dcc102c70 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:57.696 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:57.698+0000 7f0dd238b700 1 --2- 172.21.15.52:0/324805421 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0dcc107000 0x7f0dcc1036f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:57.696 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:57.698+0000 7f0dd238b700 1 --2- 172.21.15.52:0/324805421 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0dcc101960 0x7f0dcc1031b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:57.696 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:57.698+0000 7f0dd238b700 1 -- 172.21.15.52:0/324805421 >> 172.21.15.52:0/324805421 conn(0x7f0dcc077ef0 msgr2=0x7f0dcc100380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:30:57.696 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:57.698+0000 7f0dd238b700 1 -- 172.21.15.52:0/324805421 shutdown_connections 2024-03-04T09:30:57.696 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:57.698+0000 7f0dd238b700 1 -- 172.21.15.52:0/324805421 wait complete. 2024-03-04T09:30:58.370 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-03-04T09:30:58.370 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-03-04T09:30:58.370 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi052.front.sepia.ceph.com 2024-03-04T09:30:58.370 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-03-04T09:30:58.370 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-03-04T09:30:58.370 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-03-04T09:30:58.370 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-03-04T09:30:58.370 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-03-04T09:30:58.370 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-03-04T09:30:58.371 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-04T09:30:58.371 DEBUG:teuthology.orchestra.run.smithi052:> ip addr 2024-03-04T09:30:58.387 INFO:teuthology.orchestra.run.smithi052.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-04T09:30:58.387 INFO:teuthology.orchestra.run.smithi052.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-04T09:30:58.388 INFO:teuthology.orchestra.run.smithi052.stdout: inet 127.0.0.1/8 scope host lo 2024-03-04T09:30:58.388 INFO:teuthology.orchestra.run.smithi052.stdout: valid_lft forever preferred_lft forever 2024-03-04T09:30:58.388 INFO:teuthology.orchestra.run.smithi052.stdout: inet6 ::1/128 scope host 2024-03-04T09:30:58.388 INFO:teuthology.orchestra.run.smithi052.stdout: valid_lft forever preferred_lft forever 2024-03-04T09:30:58.388 INFO:teuthology.orchestra.run.smithi052.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-04T09:30:58.388 INFO:teuthology.orchestra.run.smithi052.stdout: link/ether 0c:c4:7a:6b:fd:d2 brd ff:ff:ff:ff:ff:ff 2024-03-04T09:30:58.388 INFO:teuthology.orchestra.run.smithi052.stdout: altname enp7s0f0 2024-03-04T09:30:58.388 INFO:teuthology.orchestra.run.smithi052.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-04T09:30:58.388 INFO:teuthology.orchestra.run.smithi052.stdout: link/ether 0c:c4:7a:6b:fd:d3 brd ff:ff:ff:ff:ff:ff 2024-03-04T09:30:58.388 INFO:teuthology.orchestra.run.smithi052.stdout: altname enp7s0f1 2024-03-04T09:30:58.388 INFO:teuthology.orchestra.run.smithi052.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-03-04T09:30:58.388 INFO:teuthology.orchestra.run.smithi052.stdout: link/ether 0c:c4:7a:bd:15:de brd ff:ff:ff:ff:ff:ff 2024-03-04T09:30:58.388 INFO:teuthology.orchestra.run.smithi052.stdout: altname enp1s0f0 2024-03-04T09:30:58.388 INFO:teuthology.orchestra.run.smithi052.stdout: inet 172.21.15.52/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-03-04T09:30:58.389 INFO:teuthology.orchestra.run.smithi052.stdout: valid_lft 42073sec preferred_lft 42073sec 2024-03-04T09:30:58.389 INFO:teuthology.orchestra.run.smithi052.stdout: inet6 fe80::ec4:7aff:febd:15de/64 scope link noprefixroute 2024-03-04T09:30:58.389 INFO:teuthology.orchestra.run.smithi052.stdout: valid_lft forever preferred_lft forever 2024-03-04T09:30:58.389 INFO:teuthology.orchestra.run.smithi052.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-04T09:30:58.389 INFO:teuthology.orchestra.run.smithi052.stdout: link/ether 0c:c4:7a:bd:15:df brd ff:ff:ff:ff:ff:ff 2024-03-04T09:30:58.389 INFO:teuthology.orchestra.run.smithi052.stdout: altname enp1s0f1 2024-03-04T09:30:58.389 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-03-04T09:30:58.389 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-04T09:30:58.389 DEBUG:teuthology.orchestra.run.smithi052:> (cd / && exec bash -c ' 2024-03-04T09:30:58.390 DEBUG:teuthology.orchestra.run.smithi052:> set -e 2024-03-04T09:30:58.390 DEBUG:teuthology.orchestra.run.smithi052:> sudo ip link add name ceph-brx type bridge 2024-03-04T09:30:58.390 DEBUG:teuthology.orchestra.run.smithi052:> sudo ip addr flush dev ceph-brx 2024-03-04T09:30:58.390 DEBUG:teuthology.orchestra.run.smithi052:> sudo ip link set ceph-brx up 2024-03-04T09:30:58.390 DEBUG:teuthology.orchestra.run.smithi052:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-03-04T09:30:58.390 DEBUG:teuthology.orchestra.run.smithi052:> ') 2024-03-04T09:30:58.500 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-04T09:30:58.501 DEBUG:teuthology.orchestra.run.smithi052:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-03-04T09:30:58.527 INFO:teuthology.orchestra.run.smithi052.stdout:1 2024-03-04T09:30:58.528 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-04T09:30:58.529 DEBUG:teuthology.orchestra.run.smithi052:> route 2024-03-04T09:30:58.584 INFO:teuthology.orchestra.run.smithi052.stdout:Kernel IP routing table 2024-03-04T09:30:58.584 INFO:teuthology.orchestra.run.smithi052.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-03-04T09:30:58.584 INFO:teuthology.orchestra.run.smithi052.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-03-04T09:30:58.584 INFO:teuthology.orchestra.run.smithi052.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-03-04T09:30:58.584 INFO:teuthology.orchestra.run.smithi052.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-03-04T09:30:58.584 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-04T09:30:58.585 DEBUG:teuthology.orchestra.run.smithi052:> (cd / && exec bash -c ' 2024-03-04T09:30:58.585 DEBUG:teuthology.orchestra.run.smithi052:> set -e 2024-03-04T09:30:58.585 DEBUG:teuthology.orchestra.run.smithi052:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-03-04T09:30:58.585 DEBUG:teuthology.orchestra.run.smithi052:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-03-04T09:30:58.585 DEBUG:teuthology.orchestra.run.smithi052:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-03-04T09:30:58.585 DEBUG:teuthology.orchestra.run.smithi052:> ') 2024-03-04T09:30:58.586 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:30:58 smithi052 ceph-mon[32473]: from='client.? 172.21.15.52:0/324805421' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-03-04T09:30:58.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:30:58 smithi191 ceph-mon[36816]: from='client.? 172.21.15.52:0/324805421' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-03-04T09:30:58.631 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:30:58 smithi106 ceph-mon[36484]: from='client.? 172.21.15.52:0/324805421' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-03-04T09:30:58.640 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-04T09:30:58.640 DEBUG:teuthology.orchestra.run.smithi052:> ip netns list 2024-03-04T09:30:58.655 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-04T09:30:58.655 DEBUG:teuthology.orchestra.run.smithi052:> ip netns list-id 2024-03-04T09:30:58.709 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-04T09:30:58.709 DEBUG:teuthology.orchestra.run.smithi052:> (cd / && exec bash -c ' 2024-03-04T09:30:58.709 DEBUG:teuthology.orchestra.run.smithi052:> set -e 2024-03-04T09:30:58.709 DEBUG:teuthology.orchestra.run.smithi052:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-04T09:30:58.709 DEBUG:teuthology.orchestra.run.smithi052:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-03-04T09:30:58.709 DEBUG:teuthology.orchestra.run.smithi052:> ') 2024-03-04T09:30:58.791 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-03-04T09:30:58.791 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-04T09:30:58.792 DEBUG:teuthology.orchestra.run.smithi052:> (cd / && exec bash -c ' 2024-03-04T09:30:58.792 DEBUG:teuthology.orchestra.run.smithi052:> set -e 2024-03-04T09:30:58.792 DEBUG:teuthology.orchestra.run.smithi052:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-03-04T09:30:58.792 DEBUG:teuthology.orchestra.run.smithi052:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-03-04T09:30:58.792 DEBUG:teuthology.orchestra.run.smithi052:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-03-04T09:30:58.792 DEBUG:teuthology.orchestra.run.smithi052:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-03-04T09:30:58.792 DEBUG:teuthology.orchestra.run.smithi052:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-03-04T09:30:58.792 DEBUG:teuthology.orchestra.run.smithi052:> ') 2024-03-04T09:30:58.918 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-04T09:30:58.918 DEBUG:teuthology.orchestra.run.smithi052:> (cd / && exec bash -c ' 2024-03-04T09:30:58.918 DEBUG:teuthology.orchestra.run.smithi052:> set -e 2024-03-04T09:30:58.918 DEBUG:teuthology.orchestra.run.smithi052:> sudo ip link set brx.0 up 2024-03-04T09:30:58.918 DEBUG:teuthology.orchestra.run.smithi052:> sudo ip link set dev brx.0 master ceph-brx 2024-03-04T09:30:58.918 DEBUG:teuthology.orchestra.run.smithi052:> ') 2024-03-04T09:30:58.959 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-04T09:30:58.959 DEBUG:teuthology.orchestra.run.smithi052:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-03-04T09:30:59.014 INFO:teuthology.orchestra.run.smithi052.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-03-04T09:30:59.014 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-04T09:30:59.014 DEBUG:teuthology.orchestra.run.smithi052:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-03-04T09:30:59.068 WARNING:tasks.cephfs.mount:You cannot specify both: cephfs_mntpt and mount_subvol_num 2024-03-04T09:30:59.069 INFO:tasks.cephfs.mount:Mounting subvol 1 for now 2024-03-04T09:30:59.069 INFO:tasks.cephfs.kernel_mount:mounting using device: :/volumes/qa/sv_1/9260c598-0246-4b15-9bc1-639e0bc67ba7 2024-03-04T09:30:59.069 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-04T09:30:59.069 DEBUG:teuthology.orchestra.run.smithi052:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/volumes/qa/sv_1/9260c598-0246-4b15-9bc1-639e0bc67ba7 /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=legacy,nowsync 2024-03-04T09:30:59.152 INFO:teuthology.orchestra.run.smithi052.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=legacy,nowsync 2024-03-04T09:30:59.152 INFO:teuthology.orchestra.run.smithi052.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=legacy,nowsync". 2024-03-04T09:30:59.152 INFO:teuthology.orchestra.run.smithi052.stdout:invalid new device string format 2024-03-04T09:30:59.271 INFO:teuthology.orchestra.run.smithi052.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=legacy,nowsync 2024-03-04T09:30:59.271 INFO:teuthology.orchestra.run.smithi052.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=legacy,nowsync". 2024-03-04T09:30:59.271 INFO:teuthology.orchestra.run.smithi052.stdout:invalid new device string format 2024-03-04T09:30:59.271 INFO:teuthology.orchestra.run.smithi052.stdout:mount.ceph: resolved to: "172.21.15.52:6789,172.21.15.106:6789,172.21.15.191:6789" 2024-03-04T09:30:59.271 INFO:teuthology.orchestra.run.smithi052.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.52:6789,172.21.15.106:6789,172.21.15.191:6789:/volumes/qa/sv_1/9260c598-0246-4b15-9bc1-639e0bc67ba7 2024-03-04T09:30:59.271 INFO:teuthology.orchestra.run.smithi052.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=legacy,nowsync,key=0,fsid=4e059f08-da08-11ee-95c6-87774f69a715" will pass to kernel 2024-03-04T09:30:59.272 INFO:teuthology.orchestra.run.smithi052.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-03-04T09:30:59.272 INFO:teuthology.orchestra.run.smithi052.stdout: You just mounted a file system that supports labels which does not 2024-03-04T09:30:59.272 INFO:teuthology.orchestra.run.smithi052.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-03-04T09:30:59.272 INFO:teuthology.orchestra.run.smithi052.stdout: applications will generate AVC messages and not be allowed access to 2024-03-04T09:30:59.272 INFO:teuthology.orchestra.run.smithi052.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-03-04T09:30:59.274 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-03-04T09:30:59.274 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-04T09:30:59.274 DEBUG:teuthology.orchestra.run.smithi052:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-03-04T09:30:59.347 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-04T09:30:59.347 DEBUG:teuthology.orchestra.run.smithi052:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-04T09:30:59.403 INFO:teuthology.orchestra.run.smithi052.stdout:client15450 2024-03-04T09:30:59.403 DEBUG:teuthology.orchestra.run.smithi052:> set -ex 2024-03-04T09:30:59.403 DEBUG:teuthology.orchestra.run.smithi052:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-04T09:30:59.460 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-04T09:30:59.461 DEBUG:teuthology.orchestra.run.smithi052:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-04T09:30:59.517 INFO:teuthology.orchestra.run.smithi052.stdout:client15450 2024-03-04T09:30:59.517 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-04T09:30:59.517 DEBUG:teuthology.orchestra.run.smithi052:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-04T09:30:59.574 INFO:teuthology.orchestra.run.smithi052.stdout:client15450 2024-03-04T09:30:59.574 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-04T09:30:59.574 DEBUG:teuthology.orchestra.run.smithi052:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/4e059f08-da08-11ee-95c6-87774f69a715.client15450/status') 2024-03-04T09:30:59.584 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:30:59 smithi052 ceph-mon[32473]: pgmap v362: 129 pgs: 129 active+clean; 641 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-03-04T09:30:59.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:30:59 smithi191 ceph-mon[36816]: pgmap v362: 129 pgs: 129 active+clean; 641 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-03-04T09:30:59.610 INFO:teuthology.orchestra.run.smithi052.stdout:instance: client.15450 (3)192.168.0.1:0/2728263021 2024-03-04T09:30:59.610 INFO:teuthology.orchestra.run.smithi052.stdout:blocklisted: false 2024-03-04T09:30:59.610 INFO:teuthology.orchestra.run.smithi052.stderr:0+1 records in 2024-03-04T09:30:59.610 INFO:teuthology.orchestra.run.smithi052.stderr:0+1 records out 2024-03-04T09:30:59.610 INFO:teuthology.orchestra.run.smithi052.stderr:70 bytes copied, 2.0615e-05 s, 3.4 MB/s 2024-03-04T09:30:59.612 INFO:teuthology.run_tasks:Running task exec... 2024-03-04T09:30:59.622 INFO:teuthology.task.exec:Executing custom commands... 2024-03-04T09:30:59.622 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi052.front.sepia.ceph.com 2024-03-04T09:30:59.622 DEBUG:teuthology.orchestra.run.smithi052:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs set cephfs balance_automate false' 2024-03-04T09:30:59.631 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:30:59 smithi106 ceph-mon[36484]: pgmap v362: 129 pgs: 129 active+clean; 641 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-03-04T09:30:59.758 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:59.759+0000 7f67c130c640 1 -- 172.21.15.52:0/2478745375 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f67bc100540 msgr2=0x7f67bc100960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:30:59.758 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:59.759+0000 7f67c130c640 1 --2- 172.21.15.52:0/2478745375 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f67bc100540 0x7f67bc100960 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f67b4009900 tx=0x7f67b402bc30 comp rx=0 tx=0).stop 2024-03-04T09:30:59.758 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:59.759+0000 7f67c130c640 1 -- 172.21.15.52:0/2478745375 shutdown_connections 2024-03-04T09:30:59.759 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:59.759+0000 7f67c130c640 1 --2- 172.21.15.52:0/2478745375 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f67bc1020f0 0x7f67bc108980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:59.759 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:59.759+0000 7f67c130c640 1 --2- 172.21.15.52:0/2478745375 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f67bc101730 0x7f67bc101bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:59.759 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:59.759+0000 7f67c130c640 1 --2- 172.21.15.52:0/2478745375 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f67bc100540 0x7f67bc100960 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:59.759 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:59.759+0000 7f67c130c640 1 -- 172.21.15.52:0/2478745375 >> 172.21.15.52:0/2478745375 conn(0x7f67bc0fbab0 msgr2=0x7f67bc0fdef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:30:59.759 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:59.759+0000 7f67c130c640 1 -- 172.21.15.52:0/2478745375 shutdown_connections 2024-03-04T09:30:59.759 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:59.759+0000 7f67c130c640 1 -- 172.21.15.52:0/2478745375 wait complete. 2024-03-04T09:30:59.759 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:59.760+0000 7f67c130c640 1 Processor -- start 2024-03-04T09:30:59.759 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:59.760+0000 7f67c130c640 1 -- start start 2024-03-04T09:30:59.759 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:59.760+0000 7f67c130c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f67bc100540 0x7f67bc196760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:30:59.759 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:59.760+0000 7f67c130c640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f67bc101730 0x7f67bc196ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:30:59.759 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:59.760+0000 7f67c130c640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f67bc1020f0 0x7f67bc19dd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:30:59.760 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:59.760+0000 7f67c130c640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67bc10c1d0 con 0x7f67bc101730 2024-03-04T09:30:59.760 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:59.760+0000 7f67c130c640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67bc10c050 con 0x7f67bc100540 2024-03-04T09:30:59.760 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:59.760+0000 7f67c130c640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67bc10c350 con 0x7f67bc1020f0 2024-03-04T09:30:59.760 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:59.760+0000 7f67ba7fc640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f67bc101730 0x7f67bc196ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:30:59.760 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:59.760+0000 7f67baffd640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f67bc100540 0x7f67bc196760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:30:59.760 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:59.760+0000 7f67baffd640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f67bc100540 0x7f67bc196760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.52:36282/0 (socket says 172.21.15.52:36282) 2024-03-04T09:30:59.760 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:59.760+0000 7f67ba7fc640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f67bc101730 0x7f67bc196ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:46362/0 (socket says 172.21.15.52:46362) 2024-03-04T09:30:59.760 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:59.760+0000 7f67baffd640 1 -- 172.21.15.52:0/2539810728 learned_addr learned my addr 172.21.15.52:0/2539810728 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-04T09:30:59.760 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:59.760+0000 7f67bb7fe640 1 --2- 172.21.15.52:0/2539810728 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f67bc1020f0 0x7f67bc19dd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:30:59.760 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:59.760+0000 7f67baffd640 1 -- 172.21.15.52:0/2539810728 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f67bc1020f0 msgr2=0x7f67bc19dd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:30:59.760 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:59.760+0000 7f67baffd640 1 --2- 172.21.15.52:0/2539810728 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f67bc1020f0 0x7f67bc19dd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:59.760 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:59.760+0000 7f67baffd640 1 -- 172.21.15.52:0/2539810728 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f67bc101730 msgr2=0x7f67bc196ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:30:59.761 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:59.760+0000 7f67baffd640 1 --2- 172.21.15.52:0/2539810728 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f67bc101730 0x7f67bc196ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:30:59.761 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:59.760+0000 7f67baffd640 1 -- 172.21.15.52:0/2539810728 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67bc19e380 con 0x7f67bc100540 2024-03-04T09:30:59.761 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:59.760+0000 7f67ba7fc640 1 --2- 172.21.15.52:0/2539810728 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f67bc101730 0x7f67bc196ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-04T09:30:59.761 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:59.761+0000 7f67baffd640 1 --2- 172.21.15.52:0/2539810728 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f67bc100540 0x7f67bc196760 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f67b4000c00 tx=0x7f67b402c640 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:30:59.761 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:59.761+0000 7f67b3fff640 1 -- 172.21.15.52:0/2539810728 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67b4041070 con 0x7f67bc100540 2024-03-04T09:30:59.761 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:59.761+0000 7f67c130c640 1 -- 172.21.15.52:0/2539810728 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67bc19e610 con 0x7f67bc100540 2024-03-04T09:30:59.761 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:59.761+0000 7f67b3fff640 1 -- 172.21.15.52:0/2539810728 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f67b4004450 con 0x7f67bc100540 2024-03-04T09:30:59.761 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:59.761+0000 7f67b3fff640 1 -- 172.21.15.52:0/2539810728 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67b4034440 con 0x7f67bc100540 2024-03-04T09:30:59.761 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:59.761+0000 7f67c130c640 1 -- 172.21.15.52:0/2539810728 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67bc19eb50 con 0x7f67bc100540 2024-03-04T09:30:59.762 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:59.762+0000 7f67c130c640 1 -- 172.21.15.52:0/2539810728 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f67bc107860 con 0x7f67bc100540 2024-03-04T09:30:59.762 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:59.763+0000 7f67b3fff640 1 -- 172.21.15.52:0/2539810728 <== mon.2 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 18) v1 ==== 136756+0+0 (secure 0 0 0) 0x7f67b4044070 con 0x7f67bc100540 2024-03-04T09:30:59.762 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:59.763+0000 7f67b3fff640 1 --2- 172.21.15.52:0/2539810728 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f67a80a1e20 0x7f67a80a42e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:30:59.762 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:59.763+0000 7f67ba7fc640 1 --2- 172.21.15.52:0/2539810728 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f67a80a1e20 0x7f67a80a42e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:30:59.763 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:59.763+0000 7f67b3fff640 1 -- 172.21.15.52:0/2539810728 <== mon.2 v2:172.21.15.106:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9158+0+0 (secure 0 0 0) 0x7f67b40efc20 con 0x7f67bc100540 2024-03-04T09:30:59.763 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:59.764+0000 7f67ba7fc640 1 --2- 172.21.15.52:0/2539810728 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f67a80a1e20 0x7f67a80a42e0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f67bc197c70 tx=0x7f67a4005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:30:59.764 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:59.768+0000 7f67b3fff640 1 -- 172.21.15.52:0/2539810728 <== mon.2 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f67b40bc690 con 0x7f67bc100540 2024-03-04T09:30:59.988 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:30:59.991+0000 7f67c130c640 1 -- 172.21.15.52:0/2539810728 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"} v 0) v1 -- 0x7f67bc02cff0 con 0x7f67bc100540 2024-03-04T09:31:00.331 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:00.334+0000 7f67b3fff640 1 -- 172.21.15.52:0/2539810728 <== mon.2 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]=0 v25) v1 ==== 118+0+0 (secure 0 0 0) 0x7f67b40c0730 con 0x7f67bc100540 2024-03-04T09:31:00.333 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:00.336+0000 7f67c130c640 1 -- 172.21.15.52:0/2539810728 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f67a80a1e20 msgr2=0x7f67a80a42e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:31:00.333 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:00.336+0000 7f67c130c640 1 --2- 172.21.15.52:0/2539810728 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f67a80a1e20 0x7f67a80a42e0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f67bc197c70 tx=0x7f67a4005f90 comp rx=0 tx=0).stop 2024-03-04T09:31:00.333 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:00.336+0000 7f67c130c640 1 -- 172.21.15.52:0/2539810728 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f67bc100540 msgr2=0x7f67bc196760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:31:00.333 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:00.336+0000 7f67c130c640 1 --2- 172.21.15.52:0/2539810728 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f67bc100540 0x7f67bc196760 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f67b4000c00 tx=0x7f67b402c640 comp rx=0 tx=0).stop 2024-03-04T09:31:00.333 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:00.336+0000 7f67c130c640 1 -- 172.21.15.52:0/2539810728 shutdown_connections 2024-03-04T09:31:00.334 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:00.336+0000 7f67c130c640 1 --2- 172.21.15.52:0/2539810728 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f67a80a1e20 0x7f67a80a42e0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:31:00.334 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:00.336+0000 7f67c130c640 1 --2- 172.21.15.52:0/2539810728 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f67bc1020f0 0x7f67bc19dd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:31:00.334 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:00.336+0000 7f67c130c640 1 --2- 172.21.15.52:0/2539810728 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f67bc101730 0x7f67bc196ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:31:00.334 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:00.336+0000 7f67c130c640 1 --2- 172.21.15.52:0/2539810728 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f67bc100540 0x7f67bc196760 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:31:00.334 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:00.336+0000 7f67c130c640 1 -- 172.21.15.52:0/2539810728 >> 172.21.15.52:0/2539810728 conn(0x7f67bc0fbab0 msgr2=0x7f67bc104a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:31:00.334 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:00.336+0000 7f67c130c640 1 -- 172.21.15.52:0/2539810728 shutdown_connections 2024-03-04T09:31:00.334 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:00.336+0000 7f67c130c640 1 -- 172.21.15.52:0/2539810728 wait complete. 2024-03-04T09:31:00.345 DEBUG:teuthology.orchestra.run.smithi052:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs subvolumegroup pin cephfs qa random 0.10' 2024-03-04T09:31:00.455 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:00.455+0000 7fc59fd7a640 1 -- 172.21.15.52:0/2108546425 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc5981007e0 msgr2=0x7fc598100c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:31:00.455 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:00.455+0000 7fc59fd7a640 1 --2- 172.21.15.52:0/2108546425 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc5981007e0 0x7fc598100c40 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fc588009900 tx=0x7fc58802bc30 comp rx=0 tx=0).stop 2024-03-04T09:31:00.455 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:00.455+0000 7fc59fd7a640 1 -- 172.21.15.52:0/2108546425 shutdown_connections 2024-03-04T09:31:00.455 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:00.455+0000 7fc59fd7a640 1 --2- 172.21.15.52:0/2108546425 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc598101180 0x7fc598101600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:31:00.455 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:00.455+0000 7fc59fd7a640 1 --2- 172.21.15.52:0/2108546425 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc5981007e0 0x7fc598100c40 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:31:00.456 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:00.455+0000 7fc59fd7a640 1 --2- 172.21.15.52:0/2108546425 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc5980ffe80 0x7fc5981002a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:31:00.456 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:00.455+0000 7fc59fd7a640 1 -- 172.21.15.52:0/2108546425 >> 172.21.15.52:0/2108546425 conn(0x7fc5980fba80 msgr2=0x7fc5980fdec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:31:00.456 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:00.455+0000 7fc59fd7a640 1 -- 172.21.15.52:0/2108546425 shutdown_connections 2024-03-04T09:31:00.456 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:00.455+0000 7fc59fd7a640 1 -- 172.21.15.52:0/2108546425 wait complete. 2024-03-04T09:31:00.456 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:00.455+0000 7fc59fd7a640 1 Processor -- start 2024-03-04T09:31:00.456 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:00.455+0000 7fc59fd7a640 1 -- start start 2024-03-04T09:31:00.456 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:00.455+0000 7fc59fd7a640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc5980ffe80 0x7fc598199a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:31:00.456 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:00.455+0000 7fc59fd7a640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc5981007e0 0x7fc598199f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:31:00.456 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:00.455+0000 7fc59fd7a640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc598101180 0x7fc5981a0fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:31:00.456 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:00.455+0000 7fc59fd7a640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc59810f4a0 con 0x7fc5980ffe80 2024-03-04T09:31:00.456 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:00.455+0000 7fc59fd7a640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc59810f320 con 0x7fc598101180 2024-03-04T09:31:00.457 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:00.455+0000 7fc59fd7a640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc59810f620 con 0x7fc5981007e0 2024-03-04T09:31:00.457 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:00.456+0000 7fc59db16640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc5980ffe80 0x7fc598199a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:31:00.457 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:00.456+0000 7fc59db16640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc5980ffe80 0x7fc598199a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:46376/0 (socket says 172.21.15.52:46376) 2024-03-04T09:31:00.457 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:00.456+0000 7fc59db16640 1 -- 172.21.15.52:0/2677559835 learned_addr learned my addr 172.21.15.52:0/2677559835 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-04T09:31:00.457 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:00.456+0000 7fc59d315640 1 --2- 172.21.15.52:0/2677559835 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc5981007e0 0x7fc598199f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:31:00.457 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:00.456+0000 7fc59e317640 1 --2- 172.21.15.52:0/2677559835 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc598101180 0x7fc5981a0fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:31:00.457 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:00.456+0000 7fc59db16640 1 -- 172.21.15.52:0/2677559835 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc5981007e0 msgr2=0x7fc598199f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:31:00.457 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:00.456+0000 7fc59db16640 1 --2- 172.21.15.52:0/2677559835 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc5981007e0 0x7fc598199f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:31:00.457 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:00.456+0000 7fc59db16640 1 -- 172.21.15.52:0/2677559835 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc598101180 msgr2=0x7fc5981a0fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:31:00.457 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:00.456+0000 7fc59db16640 1 --2- 172.21.15.52:0/2677559835 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc598101180 0x7fc5981a0fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:31:00.458 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:00.456+0000 7fc59db16640 1 -- 172.21.15.52:0/2677559835 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5981a1640 con 0x7fc5980ffe80 2024-03-04T09:31:00.458 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:00.456+0000 7fc59db16640 1 --2- 172.21.15.52:0/2677559835 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc5980ffe80 0x7fc598199a20 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fc59400d950 tx=0x7fc59400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:31:00.458 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:00.456+0000 7fc58effd640 1 -- 172.21.15.52:0/2677559835 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc594017030 con 0x7fc5980ffe80 2024-03-04T09:31:00.458 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:00.456+0000 7fc58effd640 1 -- 172.21.15.52:0/2677559835 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc594004020 con 0x7fc5980ffe80 2024-03-04T09:31:00.458 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:00.456+0000 7fc59fd7a640 1 -- 172.21.15.52:0/2677559835 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5981a1930 con 0x7fc5980ffe80 2024-03-04T09:31:00.458 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:00.456+0000 7fc58effd640 1 -- 172.21.15.52:0/2677559835 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5940047a0 con 0x7fc5980ffe80 2024-03-04T09:31:00.458 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:00.456+0000 7fc59fd7a640 1 -- 172.21.15.52:0/2677559835 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5981a1e50 con 0x7fc5980ffe80 2024-03-04T09:31:00.458 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:00.457+0000 7fc59fd7a640 1 -- 172.21.15.52:0/2677559835 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc59810aca0 con 0x7fc5980ffe80 2024-03-04T09:31:00.458 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:00.458+0000 7fc58effd640 1 -- 172.21.15.52:0/2677559835 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) v1 ==== 136756+0+0 (secure 0 0 0) 0x7fc5940041d0 con 0x7fc5980ffe80 2024-03-04T09:31:00.459 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:00.459+0000 7fc58effd640 1 --2- 172.21.15.52:0/2677559835 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7fc5840a1e20 0x7fc5840a42e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:31:00.459 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:00.459+0000 7fc59d315640 1 --2- 172.21.15.52:0/2677559835 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7fc5840a1e20 0x7fc5840a42e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:31:00.459 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:00.459+0000 7fc58effd640 1 -- 172.21.15.52:0/2677559835 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9158+0+0 (secure 0 0 0) 0x7fc5940cdb00 con 0x7fc5980ffe80 2024-03-04T09:31:00.459 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:00.459+0000 7fc59d315640 1 --2- 172.21.15.52:0/2677559835 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7fc5840a1e20 0x7fc5840a42e0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fc58802c240 tx=0x7fc58802df90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:31:00.460 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:00.464+0000 7fc58effd640 1 -- 172.21.15.52:0/2677559835 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fc59409ab90 con 0x7fc5980ffe80 2024-03-04T09:31:00.584 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:31:00 smithi052 ceph-mon[32473]: from='client.? 172.21.15.52:0/2539810728' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-03-04T09:31:00.585 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:31:00 smithi052 ceph-mon[32473]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-03-04T09:31:00.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:31:00 smithi191 ceph-mon[36816]: from='client.? 172.21.15.52:0/2539810728' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-03-04T09:31:00.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:31:00 smithi191 ceph-mon[36816]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-03-04T09:31:00.631 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:31:00 smithi106 ceph-mon[36484]: from='client.? 172.21.15.52:0/2539810728' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-03-04T09:31:00.631 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:31:00 smithi106 ceph-mon[36484]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-03-04T09:31:00.669 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:00.672+0000 7fc59fd7a640 1 -- 172.21.15.52:0/2677559835 --> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}) v1 -- 0x7fc598061ab0 con 0x7fc5840a1e20 2024-03-04T09:31:00.688 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:00.691+0000 7fc58effd640 1 -- 172.21.15.52:0/2677559835 <== mgr.14152 v2:172.21.15.52:6800/3036033923 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3 (secure 0 0 0) 0x7fc598061ab0 con 0x7fc5840a1e20 2024-03-04T09:31:00.688 INFO:teuthology.orchestra.run.smithi052.stdout:{} 2024-03-04T09:31:00.690 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:00.693+0000 7fc59fd7a640 1 -- 172.21.15.52:0/2677559835 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7fc5840a1e20 msgr2=0x7fc5840a42e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:31:00.690 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:00.693+0000 7fc59fd7a640 1 --2- 172.21.15.52:0/2677559835 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7fc5840a1e20 0x7fc5840a42e0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fc58802c240 tx=0x7fc58802df90 comp rx=0 tx=0).stop 2024-03-04T09:31:00.690 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:00.693+0000 7fc59fd7a640 1 -- 172.21.15.52:0/2677559835 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc5980ffe80 msgr2=0x7fc598199a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:31:00.690 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:00.693+0000 7fc59fd7a640 1 --2- 172.21.15.52:0/2677559835 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc5980ffe80 0x7fc598199a20 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fc59400d950 tx=0x7fc59400de10 comp rx=0 tx=0).stop 2024-03-04T09:31:00.690 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:00.693+0000 7fc59fd7a640 1 -- 172.21.15.52:0/2677559835 shutdown_connections 2024-03-04T09:31:00.690 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:00.693+0000 7fc59fd7a640 1 --2- 172.21.15.52:0/2677559835 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7fc5840a1e20 0x7fc5840a42e0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:31:00.690 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:00.693+0000 7fc59fd7a640 1 --2- 172.21.15.52:0/2677559835 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc598101180 0x7fc5981a0fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:31:00.690 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:00.693+0000 7fc59fd7a640 1 --2- 172.21.15.52:0/2677559835 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc5981007e0 0x7fc598199f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:31:00.691 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:00.693+0000 7fc59fd7a640 1 --2- 172.21.15.52:0/2677559835 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc5980ffe80 0x7fc598199a20 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:31:00.691 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:00.693+0000 7fc59fd7a640 1 -- 172.21.15.52:0/2677559835 >> 172.21.15.52:0/2677559835 conn(0x7fc5980fba80 msgr2=0x7fc5980fd0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:31:00.691 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:00.693+0000 7fc59fd7a640 1 -- 172.21.15.52:0/2677559835 shutdown_connections 2024-03-04T09:31:00.691 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:00.693+0000 7fc59fd7a640 1 -- 172.21.15.52:0/2677559835 wait complete. 2024-03-04T09:31:00.701 INFO:teuthology.run_tasks:Running task check-counter... 2024-03-04T09:31:00.713 INFO:teuthology.run_tasks:Running task exec... 2024-03-04T09:31:00.723 INFO:teuthology.task.exec:Executing custom commands... 2024-03-04T09:31:00.723 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi052.front.sepia.ceph.com 2024-03-04T09:31:00.723 DEBUG:teuthology.orchestra.run.smithi052:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph mgr module enable snap_schedule' 2024-03-04T09:31:00.831 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:00.830+0000 7f6913972640 1 -- 172.21.15.52:0/2790912635 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f690c0ff8e0 msgr2=0x7f690c0ffd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:31:00.831 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:00.830+0000 7f6913972640 1 --2- 172.21.15.52:0/2790912635 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f690c0ff8e0 0x7f690c0ffd00 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f6908009930 tx=0x7f690802bc30 comp rx=0 tx=0).stop 2024-03-04T09:31:00.831 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:00.830+0000 7f6913972640 1 -- 172.21.15.52:0/2790912635 shutdown_connections 2024-03-04T09:31:00.831 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:00.830+0000 7f6913972640 1 --2- 172.21.15.52:0/2790912635 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f690c103270 0x7f690c109b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:31:00.831 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:00.830+0000 7f6913972640 1 --2- 172.21.15.52:0/2790912635 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f690c1028b0 0x7f690c102d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:31:00.831 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:00.830+0000 7f6913972640 1 --2- 172.21.15.52:0/2790912635 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f690c0ff8e0 0x7f690c0ffd00 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:31:00.831 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:00.830+0000 7f6913972640 1 -- 172.21.15.52:0/2790912635 >> 172.21.15.52:0/2790912635 conn(0x7f690c0fb210 msgr2=0x7f690c0fd650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:31:00.832 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:00.831+0000 7f6913972640 1 -- 172.21.15.52:0/2790912635 shutdown_connections 2024-03-04T09:31:00.832 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:00.831+0000 7f6913972640 1 -- 172.21.15.52:0/2790912635 wait complete. 2024-03-04T09:31:00.832 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:00.831+0000 7f6913972640 1 Processor -- start 2024-03-04T09:31:00.832 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:00.831+0000 7f6913972640 1 -- start start 2024-03-04T09:31:00.832 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:00.831+0000 7f6913972640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f690c0ff8e0 0x7f690c1978d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:31:00.832 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:00.831+0000 7f6913972640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f690c1028b0 0x7f690c197e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:31:00.832 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:00.831+0000 7f6913972640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f690c103270 0x7f690c19ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:31:00.832 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:00.831+0000 7f6913972640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f690c10d2c0 con 0x7f690c103270 2024-03-04T09:31:00.832 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:00.831+0000 7f6913972640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f690c10d140 con 0x7f690c0ff8e0 2024-03-04T09:31:00.832 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:00.831+0000 7f6913972640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f690c10d440 con 0x7f690c1028b0 2024-03-04T09:31:00.832 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:00.831+0000 7f6911f0f640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f690c103270 0x7f690c19ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:31:00.833 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:00.831+0000 7f6911f0f640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f690c103270 0x7f690c19ee80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:46414/0 (socket says 172.21.15.52:46414) 2024-03-04T09:31:00.833 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:00.831+0000 7f6911f0f640 1 -- 172.21.15.52:0/2596070772 learned_addr learned my addr 172.21.15.52:0/2596070772 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-04T09:31:00.833 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:00.831+0000 7f6910f0d640 1 --2- 172.21.15.52:0/2596070772 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f690c1028b0 0x7f690c197e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:31:00.833 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:00.831+0000 7f691170e640 1 --2- 172.21.15.52:0/2596070772 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f690c0ff8e0 0x7f690c1978d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:31:00.833 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:00.832+0000 7f6911f0f640 1 -- 172.21.15.52:0/2596070772 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f690c1028b0 msgr2=0x7f690c197e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:31:00.833 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:00.832+0000 7f6911f0f640 1 --2- 172.21.15.52:0/2596070772 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f690c1028b0 0x7f690c197e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:31:00.833 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:00.832+0000 7f6911f0f640 1 -- 172.21.15.52:0/2596070772 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f690c0ff8e0 msgr2=0x7f690c1978d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:31:00.833 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:00.832+0000 7f6911f0f640 1 --2- 172.21.15.52:0/2596070772 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f690c0ff8e0 0x7f690c1978d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:31:00.833 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:00.832+0000 7f6911f0f640 1 -- 172.21.15.52:0/2596070772 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f690c19f4f0 con 0x7f690c103270 2024-03-04T09:31:00.833 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:00.832+0000 7f6911f0f640 1 --2- 172.21.15.52:0/2596070772 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f690c103270 0x7f690c19ee80 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f690400eb10 tx=0x7f690400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:31:00.833 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:00.832+0000 7f69027fc640 1 -- 172.21.15.52:0/2596070772 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6904004b20 con 0x7f690c103270 2024-03-04T09:31:00.833 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:00.832+0000 7f69027fc640 1 -- 172.21.15.52:0/2596070772 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6904004cc0 con 0x7f690c103270 2024-03-04T09:31:00.834 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:00.832+0000 7f6913972640 1 -- 172.21.15.52:0/2596070772 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f690c19f7e0 con 0x7f690c103270 2024-03-04T09:31:00.834 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:00.832+0000 7f69027fc640 1 -- 172.21.15.52:0/2596070772 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6904010760 con 0x7f690c103270 2024-03-04T09:31:00.834 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:00.832+0000 7f6913972640 1 -- 172.21.15.52:0/2596070772 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f690c19fd30 con 0x7f690c103270 2024-03-04T09:31:00.834 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:00.833+0000 7f6913972640 1 -- 172.21.15.52:0/2596070772 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f690c1089e0 con 0x7f690c103270 2024-03-04T09:31:00.834 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:00.834+0000 7f69027fc640 1 -- 172.21.15.52:0/2596070772 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) v1 ==== 136756+0+0 (secure 0 0 0) 0x7f6904005020 con 0x7f690c103270 2024-03-04T09:31:00.835 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:00.835+0000 7f69027fc640 1 --2- 172.21.15.52:0/2596070772 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f68f80a1e20 0x7f68f80a42e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:31:00.835 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:00.835+0000 7f691170e640 1 --2- 172.21.15.52:0/2596070772 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f68f80a1e20 0x7f68f80a42e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:31:00.835 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:00.835+0000 7f69027fc640 1 -- 172.21.15.52:0/2596070772 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9158+0+0 (secure 0 0 0) 0x7f69040ce8b0 con 0x7f690c103270 2024-03-04T09:31:00.835 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:00.835+0000 7f691170e640 1 --2- 172.21.15.52:0/2596070772 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f68f80a1e20 0x7f68f80a42e0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f690802c7b0 tx=0x7f6908000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:31:00.836 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:00.839+0000 7f69027fc640 1 -- 172.21.15.52:0/2596070772 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f690409b8e0 con 0x7f690c103270 2024-03-04T09:31:01.069 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:01.072+0000 7f6913972640 1 -- 172.21.15.52:0/2596070772 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) v1 -- 0x7f690c0621d0 con 0x7f690c103270 2024-03-04T09:31:01.085 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Mar 04 09:31:00 smithi052 ceph-4e059f08-da08-11ee-95c6-87774f69a715-mgr-x[32705]: 2024-03-04T09:31:00.677+0000 7f9c12c24700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-04T09:31:01.085 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Mar 04 09:31:00 smithi052 ceph-4e059f08-da08-11ee-95c6-87774f69a715-mgr-x[32705]: 2024-03-04T09:31:00.677+0000 7f9c12c24700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-04T09:31:01.085 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Mar 04 09:31:00 smithi052 ceph-4e059f08-da08-11ee-95c6-87774f69a715-mgr-x[32705]: 2024-03-04T09:31:00.677+0000 7f9c12c24700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-04T09:31:01.085 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Mar 04 09:31:00 smithi052 ceph-4e059f08-da08-11ee-95c6-87774f69a715-mgr-x[32705]: 2024-03-04T09:31:00.677+0000 7f9c12c24700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-04T09:31:01.085 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Mar 04 09:31:00 smithi052 ceph-4e059f08-da08-11ee-95c6-87774f69a715-mgr-x[32705]: 2024-03-04T09:31:00.677+0000 7f9c12c24700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-04T09:31:01.334 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:01.337+0000 7f69027fc640 1 -- 172.21.15.52:0/2596070772 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v19) v1 ==== 92+0+0 (secure 0 0 0) 0x7f690409b1e0 con 0x7f690c103270 2024-03-04T09:31:01.336 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:01.339+0000 7f6913972640 1 -- 172.21.15.52:0/2596070772 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f68f80a1e20 msgr2=0x7f68f80a42e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:31:01.337 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:01.339+0000 7f6913972640 1 --2- 172.21.15.52:0/2596070772 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f68f80a1e20 0x7f68f80a42e0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f690802c7b0 tx=0x7f6908000bc0 comp rx=0 tx=0).stop 2024-03-04T09:31:01.337 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:01.339+0000 7f6913972640 1 -- 172.21.15.52:0/2596070772 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f690c103270 msgr2=0x7f690c19ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:31:01.337 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:01.339+0000 7f6913972640 1 --2- 172.21.15.52:0/2596070772 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f690c103270 0x7f690c19ee80 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f690400eb10 tx=0x7f690400c2d0 comp rx=0 tx=0).stop 2024-03-04T09:31:01.337 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:01.340+0000 7f6913972640 1 -- 172.21.15.52:0/2596070772 shutdown_connections 2024-03-04T09:31:01.337 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:01.340+0000 7f6913972640 1 --2- 172.21.15.52:0/2596070772 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f68f80a1e20 0x7f68f80a42e0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:31:01.337 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:01.340+0000 7f6913972640 1 --2- 172.21.15.52:0/2596070772 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f690c103270 0x7f690c19ee80 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:31:01.337 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:01.340+0000 7f6913972640 1 --2- 172.21.15.52:0/2596070772 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f690c1028b0 0x7f690c197e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:31:01.337 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:01.340+0000 7f6913972640 1 --2- 172.21.15.52:0/2596070772 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f690c0ff8e0 0x7f690c1978d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:31:01.337 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:01.340+0000 7f6913972640 1 -- 172.21.15.52:0/2596070772 >> 172.21.15.52:0/2596070772 conn(0x7f690c0fb210 msgr2=0x7f690c105bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:31:01.337 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:01.340+0000 7f6913972640 1 -- 172.21.15.52:0/2596070772 shutdown_connections 2024-03-04T09:31:01.337 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:01.340+0000 7f6913972640 1 -- 172.21.15.52:0/2596070772 wait complete. 2024-03-04T09:31:01.354 DEBUG:teuthology.orchestra.run.smithi052:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/allow_m_granularity true' 2024-03-04T09:31:01.470 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:01.472+0000 7fe1c542a640 1 -- 172.21.15.52:0/92189841 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe1c01049a0 msgr2=0x7fe1c0106d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:31:01.470 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:01.472+0000 7fe1c542a640 1 --2- 172.21.15.52:0/92189841 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe1c01049a0 0x7fe1c0106d90 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fe1a8009900 tx=0x7fe1a802bc30 comp rx=0 tx=0).stop 2024-03-04T09:31:01.470 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:01.473+0000 7fe1c542a640 1 -- 172.21.15.52:0/92189841 shutdown_connections 2024-03-04T09:31:01.470 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:01.473+0000 7fe1c542a640 1 --2- 172.21.15.52:0/92189841 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe1c01072d0 0x7fe1c0109720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:31:01.470 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:01.473+0000 7fe1c542a640 1 --2- 172.21.15.52:0/92189841 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe1c01049a0 0x7fe1c0106d90 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:31:01.470 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:01.473+0000 7fe1c542a640 1 --2- 172.21.15.52:0/92189841 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe1c0102070 0x7fe1c0104460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:31:01.470 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:01.473+0000 7fe1c542a640 1 -- 172.21.15.52:0/92189841 >> 172.21.15.52:0/92189841 conn(0x7fe1c00fba60 msgr2=0x7fe1c00fdea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:31:01.470 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:01.473+0000 7fe1c542a640 1 -- 172.21.15.52:0/92189841 shutdown_connections 2024-03-04T09:31:01.471 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:01.473+0000 7fe1c542a640 1 -- 172.21.15.52:0/92189841 wait complete. 2024-03-04T09:31:01.471 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:01.473+0000 7fe1c542a640 1 Processor -- start 2024-03-04T09:31:01.471 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:01.473+0000 7fe1c542a640 1 -- start start 2024-03-04T09:31:01.472 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:01.473+0000 7fe1c542a640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe1c0102070 0x7fe1c0197880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:31:01.473 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:01.474+0000 7fe1c542a640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe1c01049a0 0x7fe1c0197dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:31:01.473 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:01.474+0000 7fe1c542a640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe1c01072d0 0x7fe1c019ee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:31:01.473 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:01.474+0000 7fe1c542a640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1c010d270 con 0x7fe1c01072d0 2024-03-04T09:31:01.473 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:01.474+0000 7fe1c542a640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1c010d0f0 con 0x7fe1c0102070 2024-03-04T09:31:01.473 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:01.474+0000 7fe1c542a640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1c010d3f0 con 0x7fe1c01049a0 2024-03-04T09:31:01.473 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:01.474+0000 7fe1bf7fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe1c01072d0 0x7fe1c019ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:31:01.473 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:01.474+0000 7fe1bf7fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe1c01072d0 0x7fe1c019ee30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:46466/0 (socket says 172.21.15.52:46466) 2024-03-04T09:31:01.473 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:01.474+0000 7fe1bf7fe640 1 -- 172.21.15.52:0/522270054 learned_addr learned my addr 172.21.15.52:0/522270054 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-04T09:31:01.473 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:01.474+0000 7fe1beffd640 1 --2- 172.21.15.52:0/522270054 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe1c0102070 0x7fe1c0197880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:31:01.473 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:01.474+0000 7fe1b67fc640 1 --2- 172.21.15.52:0/522270054 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe1c01049a0 0x7fe1c0197dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:31:01.473 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:01.474+0000 7fe1bf7fe640 1 -- 172.21.15.52:0/522270054 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe1c01049a0 msgr2=0x7fe1c0197dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:31:01.473 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:01.474+0000 7fe1bf7fe640 1 --2- 172.21.15.52:0/522270054 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe1c01049a0 0x7fe1c0197dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:31:01.473 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:01.474+0000 7fe1bf7fe640 1 -- 172.21.15.52:0/522270054 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe1c0102070 msgr2=0x7fe1c0197880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:31:01.474 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:01.474+0000 7fe1bf7fe640 1 --2- 172.21.15.52:0/522270054 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe1c0102070 0x7fe1c0197880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:31:01.474 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:01.474+0000 7fe1bf7fe640 1 -- 172.21.15.52:0/522270054 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1c019f4a0 con 0x7fe1c01072d0 2024-03-04T09:31:01.474 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:01.474+0000 7fe1bf7fe640 1 --2- 172.21.15.52:0/522270054 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe1c01072d0 0x7fe1c019ee30 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fe1b800eb10 tx=0x7fe1b800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:31:01.474 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:01.474+0000 7fe1bcff9640 1 -- 172.21.15.52:0/522270054 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1b8004b20 con 0x7fe1c01072d0 2024-03-04T09:31:01.474 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:01.474+0000 7fe1c542a640 1 -- 172.21.15.52:0/522270054 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1c019f790 con 0x7fe1c01072d0 2024-03-04T09:31:01.474 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:01.474+0000 7fe1bcff9640 1 -- 172.21.15.52:0/522270054 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe1b8004cc0 con 0x7fe1c01072d0 2024-03-04T09:31:01.474 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:01.474+0000 7fe1bcff9640 1 -- 172.21.15.52:0/522270054 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1b8010760 con 0x7fe1c01072d0 2024-03-04T09:31:01.474 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:01.475+0000 7fe1c542a640 1 -- 172.21.15.52:0/522270054 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1c019fcb0 con 0x7fe1c01072d0 2024-03-04T09:31:01.474 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:01.476+0000 7fe1c542a640 1 -- 172.21.15.52:0/522270054 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe1c00fd550 con 0x7fe1c01072d0 2024-03-04T09:31:01.475 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:01.477+0000 7fe1bcff9640 1 -- 172.21.15.52:0/522270054 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 19) v1 ==== 136824+0+0 (secure 0 0 0) 0x7fe1b8005020 con 0x7fe1c01072d0 2024-03-04T09:31:01.475 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:01.477+0000 7fe1bcff9640 1 --2- 172.21.15.52:0/522270054 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7fe1ac0a1f10 0x7fe1ac0a43d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:31:01.475 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:01.478+0000 7fe1beffd640 1 -- 172.21.15.52:0/522270054 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7fe1ac0a1f10 msgr2=0x7fe1ac0a43d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.52:6800/3036033923 2024-03-04T09:31:01.475 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:01.478+0000 7fe1bcff9640 1 -- 172.21.15.52:0/522270054 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9158+0+0 (secure 0 0 0) 0x7fe1b80ce8f0 con 0x7fe1c01072d0 2024-03-04T09:31:01.475 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:01.478+0000 7fe1beffd640 1 --2- 172.21.15.52:0/522270054 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7fe1ac0a1f10 0x7fe1ac0a43d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T09:31:01.479 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:01.482+0000 7fe1bcff9640 1 -- 172.21.15.52:0/522270054 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fe1b809b820 con 0x7fe1c01072d0 2024-03-04T09:31:01.584 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Mar 04 09:31:01 smithi052 ceph-4e059f08-da08-11ee-95c6-87774f69a715-mgr-x[32705]: ignoring --setuser ceph since I am not root 2024-03-04T09:31:01.585 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Mar 04 09:31:01 smithi052 ceph-4e059f08-da08-11ee-95c6-87774f69a715-mgr-x[32705]: ignoring --setgroup ceph since I am not root 2024-03-04T09:31:01.585 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:31:01 smithi052 ceph-mon[32473]: pgmap v363: 129 pgs: 129 active+clean; 641 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-04T09:31:01.585 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:31:01 smithi052 ceph-mon[32473]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-03-04T09:31:01.585 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:31:01 smithi052 ceph-mon[32473]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-03-04T09:31:01.585 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:31:01 smithi052 ceph-mon[32473]: from='client.? 172.21.15.52:0/2596070772' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-03-04T09:31:01.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:31:01 smithi191 ceph-mon[36816]: pgmap v363: 129 pgs: 129 active+clean; 641 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-04T09:31:01.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:31:01 smithi191 ceph-mon[36816]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-03-04T09:31:01.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:31:01 smithi191 ceph-mon[36816]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-03-04T09:31:01.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:31:01 smithi191 ceph-mon[36816]: from='client.? 172.21.15.52:0/2596070772' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-03-04T09:31:01.586 INFO:journalctl@ceph.mgr.z.smithi191.stdout:Mar 04 09:31:01 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-mgr-z[38667]: ignoring --setuser ceph since I am not root 2024-03-04T09:31:01.586 INFO:journalctl@ceph.mgr.z.smithi191.stdout:Mar 04 09:31:01 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-mgr-z[38667]: ignoring --setgroup ceph since I am not root 2024-03-04T09:31:01.587 INFO:journalctl@ceph.mgr.z.smithi191.stdout:Mar 04 09:31:01 smithi191 ceph-mgr[38671]: -- 172.21.15.191:0/1139145164 <== mon.1 v2:172.21.15.191:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55b24e71e5a0 con 0x55b24e6f2c00 2024-03-04T09:31:01.631 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:31:01 smithi106 ceph-mon[36484]: pgmap v363: 129 pgs: 129 active+clean; 641 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-04T09:31:01.632 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:31:01 smithi106 ceph-mon[36484]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-03-04T09:31:01.632 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:31:01 smithi106 ceph-mon[36484]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-03-04T09:31:01.632 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:31:01 smithi106 ceph-mon[36484]: from='client.? 172.21.15.52:0/2596070772' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-03-04T09:31:01.632 INFO:journalctl@ceph.mgr.y.smithi106.stdout:Mar 04 09:31:01 smithi106 ceph-4e059f08-da08-11ee-95c6-87774f69a715-mgr-y[37865]: ignoring --setuser ceph since I am not root 2024-03-04T09:31:01.632 INFO:journalctl@ceph.mgr.y.smithi106.stdout:Mar 04 09:31:01 smithi106 ceph-4e059f08-da08-11ee-95c6-87774f69a715-mgr-y[37865]: ignoring --setgroup ceph since I am not root 2024-03-04T09:31:01.632 INFO:journalctl@ceph.mgr.y.smithi106.stdout:Mar 04 09:31:01 smithi106 ceph-mgr[37869]: -- 172.21.15.106:0/522439028 <== mon.2 v2:172.21.15.106:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55754a2d65a0 con 0x55754a2aac00 2024-03-04T09:31:01.663 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:01.666+0000 7fe1c542a640 1 -- 172.21.15.52:0/522270054 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) v1 -- 0x7fe1c00621d0 con 0x7fe1c01072d0 2024-03-04T09:31:01.666 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:01.670+0000 7fe1bcff9640 1 -- 172.21.15.52:0/522270054 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v27)=0 v27) v1 ==== 138+0+0 (secure 0 0 0) 0x7fe1b809b120 con 0x7fe1c01072d0 2024-03-04T09:31:01.669 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:01.672+0000 7fe1c542a640 1 -- 172.21.15.52:0/522270054 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7fe1ac0a1f10 msgr2=0x7fe1ac0a43d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-04T09:31:01.670 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:01.672+0000 7fe1c542a640 1 --2- 172.21.15.52:0/522270054 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7fe1ac0a1f10 0x7fe1ac0a43d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:31:01.670 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:01.672+0000 7fe1c542a640 1 -- 172.21.15.52:0/522270054 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe1c01072d0 msgr2=0x7fe1c019ee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:31:01.670 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:01.672+0000 7fe1c542a640 1 --2- 172.21.15.52:0/522270054 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe1c01072d0 0x7fe1c019ee30 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fe1b800eb10 tx=0x7fe1b800c2d0 comp rx=0 tx=0).stop 2024-03-04T09:31:01.670 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:01.672+0000 7fe1c542a640 1 -- 172.21.15.52:0/522270054 shutdown_connections 2024-03-04T09:31:01.670 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:01.672+0000 7fe1c542a640 1 --2- 172.21.15.52:0/522270054 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7fe1ac0a1f10 0x7fe1ac0a43d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:31:01.670 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:01.672+0000 7fe1c542a640 1 --2- 172.21.15.52:0/522270054 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe1c01072d0 0x7fe1c019ee30 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:31:01.670 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:01.672+0000 7fe1c542a640 1 --2- 172.21.15.52:0/522270054 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe1c01049a0 0x7fe1c0197dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:31:01.670 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:01.672+0000 7fe1c542a640 1 --2- 172.21.15.52:0/522270054 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe1c0102070 0x7fe1c0197880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:31:01.670 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:01.672+0000 7fe1c542a640 1 -- 172.21.15.52:0/522270054 >> 172.21.15.52:0/522270054 conn(0x7fe1c00fba60 msgr2=0x7fe1c01029b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:31:01.670 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:01.673+0000 7fe1c542a640 1 -- 172.21.15.52:0/522270054 shutdown_connections 2024-03-04T09:31:01.670 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:01.673+0000 7fe1c542a640 1 -- 172.21.15.52:0/522270054 wait complete. 2024-03-04T09:31:01.681 DEBUG:teuthology.orchestra.run.smithi052:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/dump_on_update true' 2024-03-04T09:31:01.791 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:01.789+0000 7f8a3f73a640 1 -- 172.21.15.52:0/2232428596 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8a381020a0 msgr2=0x7f8a38108930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:31:01.792 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:01.789+0000 7f8a3f73a640 1 --2- 172.21.15.52:0/2232428596 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8a381020a0 0x7f8a38108930 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f8a30009ca0 tx=0x7f8a3002c0f0 comp rx=0 tx=0).stop 2024-03-04T09:31:01.792 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:01.789+0000 7f8a3f73a640 1 -- 172.21.15.52:0/2232428596 shutdown_connections 2024-03-04T09:31:01.792 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:01.789+0000 7f8a3f73a640 1 --2- 172.21.15.52:0/2232428596 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8a381020a0 0x7f8a38108930 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:31:01.792 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:01.789+0000 7f8a3f73a640 1 --2- 172.21.15.52:0/2232428596 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8a381016e0 0x7f8a38101b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:31:01.792 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:01.789+0000 7f8a3f73a640 1 --2- 172.21.15.52:0/2232428596 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8a381004f0 0x7f8a38100910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:31:01.792 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:01.789+0000 7f8a3f73a640 1 -- 172.21.15.52:0/2232428596 >> 172.21.15.52:0/2232428596 conn(0x7f8a380fba60 msgr2=0x7f8a380fdea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:31:01.792 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:01.789+0000 7f8a3f73a640 1 -- 172.21.15.52:0/2232428596 shutdown_connections 2024-03-04T09:31:01.792 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:01.789+0000 7f8a3f73a640 1 -- 172.21.15.52:0/2232428596 wait complete. 2024-03-04T09:31:01.792 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:01.790+0000 7f8a3f73a640 1 Processor -- start 2024-03-04T09:31:01.792 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:01.790+0000 7f8a3f73a640 1 -- start start 2024-03-04T09:31:01.792 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:01.790+0000 7f8a3f73a640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8a381004f0 0x7f8a38104480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:31:01.793 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:01.790+0000 7f8a3f73a640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8a381016e0 0x7f8a38102ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:31:01.793 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:01.790+0000 7f8a3f73a640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8a381020a0 0x7f8a38103000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:31:01.793 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:01.790+0000 7f8a3f73a640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a3810c180 con 0x7f8a381004f0 2024-03-04T09:31:01.793 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:01.790+0000 7f8a3f73a640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a3810c000 con 0x7f8a381016e0 2024-03-04T09:31:01.793 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:01.790+0000 7f8a3f73a640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a3810c300 con 0x7f8a381020a0 2024-03-04T09:31:01.793 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:01.790+0000 7f8a3d4d6640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8a381004f0 0x7f8a38104480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:31:01.793 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:01.790+0000 7f8a3d4d6640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8a381004f0 0x7f8a38104480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:46488/0 (socket says 172.21.15.52:46488) 2024-03-04T09:31:01.793 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:01.790+0000 7f8a3d4d6640 1 -- 172.21.15.52:0/4250967846 learned_addr learned my addr 172.21.15.52:0/4250967846 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-04T09:31:01.793 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:01.790+0000 7f8a3ccd5640 1 --2- 172.21.15.52:0/4250967846 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8a381016e0 0x7f8a38102ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:31:01.793 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:01.790+0000 7f8a3dcd7640 1 --2- 172.21.15.52:0/4250967846 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8a381020a0 0x7f8a38103000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:31:01.793 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:01.790+0000 7f8a3d4d6640 1 -- 172.21.15.52:0/4250967846 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8a381020a0 msgr2=0x7f8a38103000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:31:01.793 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:01.790+0000 7f8a3d4d6640 1 --2- 172.21.15.52:0/4250967846 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8a381020a0 0x7f8a38103000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:31:01.794 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:01.790+0000 7f8a3d4d6640 1 -- 172.21.15.52:0/4250967846 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8a381016e0 msgr2=0x7f8a38102ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:31:01.794 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:01.790+0000 7f8a3d4d6640 1 --2- 172.21.15.52:0/4250967846 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8a381016e0 0x7f8a38102ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:31:01.794 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:01.790+0000 7f8a3d4d6640 1 -- 172.21.15.52:0/4250967846 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a38103890 con 0x7f8a381004f0 2024-03-04T09:31:01.794 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:01.790+0000 7f8a3ccd5640 1 --2- 172.21.15.52:0/4250967846 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8a381016e0 0x7f8a38102ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-04T09:31:01.794 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:01.791+0000 7f8a3d4d6640 1 --2- 172.21.15.52:0/4250967846 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8a381004f0 0x7f8a38104480 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f8a2800e9c0 tx=0x7f8a2800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:31:01.794 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:01.791+0000 7f8a2e7fc640 1 -- 172.21.15.52:0/4250967846 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a2800cd80 con 0x7f8a381004f0 2024-03-04T09:31:01.794 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:01.791+0000 7f8a2e7fc640 1 -- 172.21.15.52:0/4250967846 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8a280035f0 con 0x7f8a381004f0 2024-03-04T09:31:01.794 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:01.791+0000 7f8a3f73a640 1 -- 172.21.15.52:0/4250967846 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a381a1920 con 0x7f8a381004f0 2024-03-04T09:31:01.794 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:01.791+0000 7f8a2e7fc640 1 -- 172.21.15.52:0/4250967846 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a28010690 con 0x7f8a381004f0 2024-03-04T09:31:01.794 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:01.791+0000 7f8a3f73a640 1 -- 172.21.15.52:0/4250967846 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a381a1e20 con 0x7f8a381004f0 2024-03-04T09:31:01.794 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:01.792+0000 7f8a3f73a640 1 -- 172.21.15.52:0/4250967846 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a381a20b0 con 0x7f8a381004f0 2024-03-04T09:31:01.794 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:01.793+0000 7f8a2e7fc640 1 -- 172.21.15.52:0/4250967846 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 19) v1 ==== 136824+0+0 (secure 0 0 0) 0x7f8a280037a0 con 0x7f8a381004f0 2024-03-04T09:31:01.794 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:01.793+0000 7f8a2e7fc640 1 --2- 172.21.15.52:0/4250967846 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f8a240a1f10 0x7f8a240a43d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:31:01.795 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:01.793+0000 7f8a3ccd5640 1 -- 172.21.15.52:0/4250967846 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f8a240a1f10 msgr2=0x7f8a240a43d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.52:6800/3036033923 2024-03-04T09:31:01.795 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:01.794+0000 7f8a2e7fc640 1 -- 172.21.15.52:0/4250967846 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9158+0+0 (secure 0 0 0) 0x7f8a280ce770 con 0x7f8a381004f0 2024-03-04T09:31:01.795 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:01.794+0000 7f8a3ccd5640 1 --2- 172.21.15.52:0/4250967846 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f8a240a1f10 0x7f8a240a43d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T09:31:01.795 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:01.798+0000 7f8a2e7fc640 1 -- 172.21.15.52:0/4250967846 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f8a2809b640 con 0x7f8a381004f0 2024-03-04T09:31:01.991 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:01.995+0000 7f8a3ccd5640 1 -- 172.21.15.52:0/4250967846 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f8a240a1f10 msgr2=0x7f8a240a43d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.52:6800/3036033923 2024-03-04T09:31:01.992 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:01.995+0000 7f8a3ccd5640 1 --2- 172.21.15.52:0/4250967846 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f8a240a1f10 0x7f8a240a43d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T09:31:01.998 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:02.001+0000 7f8a3f73a640 1 -- 172.21.15.52:0/4250967846 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) v1 -- 0x7f8a381a24b0 con 0x7f8a381004f0 2024-03-04T09:31:02.002 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:02.006+0000 7f8a2e7fc640 1 -- 172.21.15.52:0/4250967846 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v28)=0 v28) v1 ==== 133+0+0 (secure 0 0 0) 0x7f8a2809af40 con 0x7f8a381004f0 2024-03-04T09:31:02.005 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:02.008+0000 7f8a3f73a640 1 -- 172.21.15.52:0/4250967846 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f8a240a1f10 msgr2=0x7f8a240a43d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-04T09:31:02.005 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:02.008+0000 7f8a3f73a640 1 --2- 172.21.15.52:0/4250967846 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f8a240a1f10 0x7f8a240a43d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:31:02.005 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:02.008+0000 7f8a3f73a640 1 -- 172.21.15.52:0/4250967846 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8a381004f0 msgr2=0x7f8a38104480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:31:02.005 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:02.008+0000 7f8a3f73a640 1 --2- 172.21.15.52:0/4250967846 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8a381004f0 0x7f8a38104480 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f8a2800e9c0 tx=0x7f8a2800ee80 comp rx=0 tx=0).stop 2024-03-04T09:31:02.005 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:02.008+0000 7f8a3f73a640 1 -- 172.21.15.52:0/4250967846 shutdown_connections 2024-03-04T09:31:02.005 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:02.008+0000 7f8a3f73a640 1 --2- 172.21.15.52:0/4250967846 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f8a240a1f10 0x7f8a240a43d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:31:02.005 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:02.008+0000 7f8a3f73a640 1 --2- 172.21.15.52:0/4250967846 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8a381020a0 0x7f8a38103000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:31:02.006 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:02.008+0000 7f8a3f73a640 1 --2- 172.21.15.52:0/4250967846 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8a381016e0 0x7f8a38102ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:31:02.006 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:02.008+0000 7f8a3f73a640 1 --2- 172.21.15.52:0/4250967846 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8a381004f0 0x7f8a38104480 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:31:02.006 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:02.008+0000 7f8a3f73a640 1 -- 172.21.15.52:0/4250967846 >> 172.21.15.52:0/4250967846 conn(0x7f8a380fba60 msgr2=0x7f8a380fd420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:31:02.006 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:02.008+0000 7f8a3f73a640 1 -- 172.21.15.52:0/4250967846 shutdown_connections 2024-03-04T09:31:02.006 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:02.008+0000 7f8a3f73a640 1 -- 172.21.15.52:0/4250967846 wait complete. 2024-03-04T09:31:02.016 DEBUG:teuthology.orchestra.run.smithi052:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m' 2024-03-04T09:31:02.125 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:02.126+0000 7f1d1c615640 1 -- 172.21.15.52:0/1026200351 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1d14101620 msgr2=0x7f1d14101a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:31:02.125 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:02.126+0000 7f1d1c615640 1 --2- 172.21.15.52:0/1026200351 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1d14101620 0x7f1d14101a40 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f1d10009900 tx=0x7f1d1002bc30 comp rx=0 tx=0).stop 2024-03-04T09:31:02.125 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:02.126+0000 7f1d1c615640 1 -- 172.21.15.52:0/1026200351 shutdown_connections 2024-03-04T09:31:02.125 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:02.126+0000 7f1d1c615640 1 --2- 172.21.15.52:0/1026200351 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1d141031d0 0x7f1d14109a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:31:02.126 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:02.126+0000 7f1d1c615640 1 --2- 172.21.15.52:0/1026200351 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1d14102810 0x7f1d14102c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:31:02.126 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:02.126+0000 7f1d1c615640 1 --2- 172.21.15.52:0/1026200351 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1d14101620 0x7f1d14101a40 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:31:02.126 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:02.126+0000 7f1d1c615640 1 -- 172.21.15.52:0/1026200351 >> 172.21.15.52:0/1026200351 conn(0x7f1d140fcb90 msgr2=0x7f1d140fefd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:31:02.126 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:02.126+0000 7f1d1c615640 1 -- 172.21.15.52:0/1026200351 shutdown_connections 2024-03-04T09:31:02.126 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:02.126+0000 7f1d1c615640 1 -- 172.21.15.52:0/1026200351 wait complete. 2024-03-04T09:31:02.126 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:02.127+0000 7f1d1c615640 1 Processor -- start 2024-03-04T09:31:02.126 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:02.127+0000 7f1d1c615640 1 -- start start 2024-03-04T09:31:02.126 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:02.127+0000 7f1d1c615640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1d14101620 0x7f1d14197910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:31:02.126 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:02.127+0000 7f1d1c615640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1d14102810 0x7f1d14197e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:31:02.126 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:02.127+0000 7f1d1c615640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1d141031d0 0x7f1d1419eec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:31:02.126 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:02.127+0000 7f1d1c615640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d1410d2a0 con 0x7f1d14102810 2024-03-04T09:31:02.126 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:02.127+0000 7f1d1c615640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d1410d120 con 0x7f1d141031d0 2024-03-04T09:31:02.126 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:02.127+0000 7f1d1c615640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d1410d420 con 0x7f1d14101620 2024-03-04T09:31:02.126 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:02.127+0000 7f1d19bb0640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1d14102810 0x7f1d14197e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:31:02.127 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:02.127+0000 7f1d19bb0640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1d14102810 0x7f1d14197e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:46504/0 (socket says 172.21.15.52:46504) 2024-03-04T09:31:02.127 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:02.127+0000 7f1d19bb0640 1 -- 172.21.15.52:0/2176133942 learned_addr learned my addr 172.21.15.52:0/2176133942 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-04T09:31:02.127 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:02.127+0000 7f1d1a3b1640 1 --2- 172.21.15.52:0/2176133942 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1d14101620 0x7f1d14197910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:31:02.127 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:02.127+0000 7f1d1abb2640 1 --2- 172.21.15.52:0/2176133942 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1d141031d0 0x7f1d1419eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:31:02.127 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:02.127+0000 7f1d19bb0640 1 -- 172.21.15.52:0/2176133942 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1d14101620 msgr2=0x7f1d14197910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:31:02.127 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:02.127+0000 7f1d19bb0640 1 --2- 172.21.15.52:0/2176133942 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1d14101620 0x7f1d14197910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:31:02.127 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:02.127+0000 7f1d19bb0640 1 -- 172.21.15.52:0/2176133942 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1d141031d0 msgr2=0x7f1d1419eec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:31:02.127 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:02.127+0000 7f1d19bb0640 1 --2- 172.21.15.52:0/2176133942 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1d141031d0 0x7f1d1419eec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:31:02.127 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:02.127+0000 7f1d19bb0640 1 -- 172.21.15.52:0/2176133942 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d1419f530 con 0x7f1d14102810 2024-03-04T09:31:02.127 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:02.127+0000 7f1d19bb0640 1 --2- 172.21.15.52:0/2176133942 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1d14102810 0x7f1d14197e50 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f1d0400e9f0 tx=0x7f1d0400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:31:02.127 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:02.127+0000 7f1d0b7fe640 1 -- 172.21.15.52:0/2176133942 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d0400cd80 con 0x7f1d14102810 2024-03-04T09:31:02.127 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:02.127+0000 7f1d0b7fe640 1 -- 172.21.15.52:0/2176133942 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1d04005500 con 0x7f1d14102810 2024-03-04T09:31:02.127 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:02.127+0000 7f1d1c615640 1 -- 172.21.15.52:0/2176133942 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d14073860 con 0x7f1d14102810 2024-03-04T09:31:02.128 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:02.127+0000 7f1d0b7fe640 1 -- 172.21.15.52:0/2176133942 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d04010690 con 0x7f1d14102810 2024-03-04T09:31:02.128 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:02.128+0000 7f1d1c615640 1 -- 172.21.15.52:0/2176133942 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d14073d60 con 0x7f1d14102810 2024-03-04T09:31:02.128 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:02.129+0000 7f1d1c615640 1 -- 172.21.15.52:0/2176133942 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d14108940 con 0x7f1d14102810 2024-03-04T09:31:02.129 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:02.130+0000 7f1d0b7fe640 1 -- 172.21.15.52:0/2176133942 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 19) v1 ==== 136824+0+0 (secure 0 0 0) 0x7f1d040046c0 con 0x7f1d14102810 2024-03-04T09:31:02.129 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:02.130+0000 7f1d0b7fe640 1 --2- 172.21.15.52:0/2176133942 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f1d000a1f10 0x7f1d000a43d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:31:02.129 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:02.130+0000 7f1d1a3b1640 1 -- 172.21.15.52:0/2176133942 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f1d000a1f10 msgr2=0x7f1d000a43d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.52:6800/3036033923 2024-03-04T09:31:02.129 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:02.131+0000 7f1d0b7fe640 1 -- 172.21.15.52:0/2176133942 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9158+0+0 (secure 0 0 0) 0x7f1d040ce730 con 0x7f1d14102810 2024-03-04T09:31:02.129 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:02.131+0000 7f1d1a3b1640 1 --2- 172.21.15.52:0/2176133942 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f1d000a1f10 0x7f1d000a43d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T09:31:02.131 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:02.135+0000 7f1d0b7fe640 1 -- 172.21.15.52:0/2176133942 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f1d0409b660 con 0x7f1d14102810 2024-03-04T09:31:02.329 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:02.332+0000 7f1d1a3b1640 1 -- 172.21.15.52:0/2176133942 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f1d000a1f10 msgr2=0x7f1d000a43d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.52:6800/3036033923 2024-03-04T09:31:02.329 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:02.332+0000 7f1d1a3b1640 1 --2- 172.21.15.52:0/2176133942 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f1d000a1f10 0x7f1d000a43d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T09:31:02.334 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Mar 04 09:31:01 smithi052 ceph-4e059f08-da08-11ee-95c6-87774f69a715-mgr-x[32705]: 2024-03-04T09:31:01.951+0000 7f600290f200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-04T09:31:02.334 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Mar 04 09:31:02 smithi052 ceph-4e059f08-da08-11ee-95c6-87774f69a715-mgr-x[32705]: 2024-03-04T09:31:02.053+0000 7f600290f200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-04T09:31:02.335 INFO:journalctl@ceph.mgr.z.smithi191.stdout:Mar 04 09:31:01 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-mgr-z[38667]: 2024-03-04T09:31:01.903+0000 7f8d8a385200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-04T09:31:02.335 INFO:journalctl@ceph.mgr.z.smithi191.stdout:Mar 04 09:31:02 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-mgr-z[38667]: 2024-03-04T09:31:02.000+0000 7f8d8a385200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-04T09:31:02.337 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:02.341+0000 7f1d1c615640 1 -- 172.21.15.52:0/2176133942 --> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) v1 -- 0x7f1d14061a10 con 0x7f1d000a1f10 2024-03-04T09:31:02.338 INFO:journalctl@ceph.mgr.y.smithi106.stdout:Mar 04 09:31:01 smithi106 ceph-4e059f08-da08-11ee-95c6-87774f69a715-mgr-y[37865]: 2024-03-04T09:31:01.971+0000 7faa9a554200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-04T09:31:02.338 INFO:journalctl@ceph.mgr.y.smithi106.stdout:Mar 04 09:31:02 smithi106 ceph-4e059f08-da08-11ee-95c6-87774f69a715-mgr-y[37865]: 2024-03-04T09:31:02.069+0000 7faa9a554200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-04T09:31:02.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:31:02 smithi191 ceph-mon[36816]: from='client.? 172.21.15.52:0/2596070772' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-03-04T09:31:02.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:31:02 smithi191 ceph-mon[36816]: mgrmap e19: x(active, since 9m), standbys: z, y 2024-03-04T09:31:02.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:31:02 smithi191 ceph-mon[36816]: from='client.? 172.21.15.52:0/522270054' entity='client.admin' 2024-03-04T09:31:02.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:31:02 smithi191 ceph-mon[36816]: from='client.? 172.21.15.52:0/4250967846' entity='client.admin' 2024-03-04T09:31:02.631 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:31:02 smithi106 ceph-mon[36484]: from='client.? 172.21.15.52:0/2596070772' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-03-04T09:31:02.631 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:31:02 smithi106 ceph-mon[36484]: mgrmap e19: x(active, since 9m), standbys: z, y 2024-03-04T09:31:02.631 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:31:02 smithi106 ceph-mon[36484]: from='client.? 172.21.15.52:0/522270054' entity='client.admin' 2024-03-04T09:31:02.631 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:31:02 smithi106 ceph-mon[36484]: from='client.? 172.21.15.52:0/4250967846' entity='client.admin' 2024-03-04T09:31:02.716 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:31:02 smithi052 ceph-mon[32473]: from='client.? 172.21.15.52:0/2596070772' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-03-04T09:31:02.716 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:31:02 smithi052 ceph-mon[32473]: mgrmap e19: x(active, since 9m), standbys: z, y 2024-03-04T09:31:02.716 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:31:02 smithi052 ceph-mon[32473]: from='client.? 172.21.15.52:0/522270054' entity='client.admin' 2024-03-04T09:31:02.716 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:31:02 smithi052 ceph-mon[32473]: from='client.? 172.21.15.52:0/4250967846' entity='client.admin' 2024-03-04T09:31:02.729 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:02.733+0000 7f1d1a3b1640 1 -- 172.21.15.52:0/2176133942 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f1d000a1f10 msgr2=0x7f1d000a43d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.52:6800/3036033923 2024-03-04T09:31:02.730 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:02.733+0000 7f1d1a3b1640 1 --2- 172.21.15.52:0/2176133942 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f1d000a1f10 0x7f1d000a43d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T09:31:02.904 INFO:journalctl@ceph.mgr.z.smithi191.stdout:Mar 04 09:31:02 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-mgr-z[38667]: 2024-03-04T09:31:02.648+0000 7f8d8a385200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-04T09:31:02.905 INFO:journalctl@ceph.mgr.z.smithi191.stdout:Mar 04 09:31:02 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-mgr-z[38667]: 2024-03-04T09:31:02.741+0000 7f8d8a385200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-04T09:31:02.905 INFO:journalctl@ceph.mgr.z.smithi191.stdout:Mar 04 09:31:02 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-mgr-z[38667]: 2024-03-04T09:31:02.907+0000 7f8d8a385200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-04T09:31:03.054 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Mar 04 09:31:02 smithi052 ceph-4e059f08-da08-11ee-95c6-87774f69a715-mgr-x[32705]: 2024-03-04T09:31:02.719+0000 7f600290f200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-04T09:31:03.054 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Mar 04 09:31:02 smithi052 ceph-4e059f08-da08-11ee-95c6-87774f69a715-mgr-x[32705]: 2024-03-04T09:31:02.815+0000 7f600290f200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-04T09:31:03.054 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Mar 04 09:31:02 smithi052 ceph-4e059f08-da08-11ee-95c6-87774f69a715-mgr-x[32705]: 2024-03-04T09:31:02.959+0000 7f600290f200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-04T09:31:03.054 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Mar 04 09:31:03 smithi052 ceph-4e059f08-da08-11ee-95c6-87774f69a715-mgr-x[32705]: 2024-03-04T09:31:03.057+0000 7f600290f200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-04T09:31:03.119 INFO:journalctl@ceph.mgr.y.smithi106.stdout:Mar 04 09:31:02 smithi106 ceph-4e059f08-da08-11ee-95c6-87774f69a715-mgr-y[37865]: 2024-03-04T09:31:02.726+0000 7faa9a554200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-04T09:31:03.119 INFO:journalctl@ceph.mgr.y.smithi106.stdout:Mar 04 09:31:02 smithi106 ceph-4e059f08-da08-11ee-95c6-87774f69a715-mgr-y[37865]: 2024-03-04T09:31:02.820+0000 7faa9a554200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-04T09:31:03.119 INFO:journalctl@ceph.mgr.y.smithi106.stdout:Mar 04 09:31:02 smithi106 ceph-4e059f08-da08-11ee-95c6-87774f69a715-mgr-y[37865]: 2024-03-04T09:31:02.972+0000 7faa9a554200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-04T09:31:03.120 INFO:journalctl@ceph.mgr.y.smithi106.stdout:Mar 04 09:31:03 smithi106 ceph-4e059f08-da08-11ee-95c6-87774f69a715-mgr-y[37865]: 2024-03-04T09:31:03.122+0000 7faa9a554200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-04T09:31:03.335 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Mar 04 09:31:03 smithi052 ceph-4e059f08-da08-11ee-95c6-87774f69a715-mgr-x[32705]: 2024-03-04T09:31:03.143+0000 7f600290f200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-04T09:31:03.336 INFO:journalctl@ceph.mgr.z.smithi191.stdout:Mar 04 09:31:03 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-mgr-z[38667]: 2024-03-04T09:31:03.034+0000 7f8d8a385200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-04T09:31:03.336 INFO:journalctl@ceph.mgr.z.smithi191.stdout:Mar 04 09:31:03 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-mgr-z[38667]: 2024-03-04T09:31:03.145+0000 7f8d8a385200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-04T09:31:03.381 INFO:journalctl@ceph.mgr.y.smithi106.stdout:Mar 04 09:31:03 smithi106 ceph-4e059f08-da08-11ee-95c6-87774f69a715-mgr-y[37865]: 2024-03-04T09:31:03.239+0000 7faa9a554200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-04T09:31:03.530 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:03.534+0000 7f1d1a3b1640 1 -- 172.21.15.52:0/2176133942 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f1d000a1f10 msgr2=0x7f1d000a43d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.52:6800/3036033923 2024-03-04T09:31:03.531 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:03.534+0000 7f1d1a3b1640 1 --2- 172.21.15.52:0/2176133942 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f1d000a1f10 0x7f1d000a43d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T09:31:03.704 INFO:journalctl@ceph.mgr.z.smithi191.stdout:Mar 04 09:31:03 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-mgr-z[38667]: 2024-03-04T09:31:03.411+0000 7f8d8a385200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-04T09:31:03.704 INFO:journalctl@ceph.mgr.z.smithi191.stdout:Mar 04 09:31:03 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-mgr-z[38667]: 2024-03-04T09:31:03.522+0000 7f8d8a385200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-04T09:31:03.704 INFO:journalctl@ceph.mgr.z.smithi191.stdout:Mar 04 09:31:03 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-mgr-z[38667]: 2024-03-04T09:31:03.610+0000 7f8d8a385200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-04T09:31:03.707 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Mar 04 09:31:03 smithi052 ceph-4e059f08-da08-11ee-95c6-87774f69a715-mgr-x[32705]: 2024-03-04T09:31:03.411+0000 7f600290f200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-04T09:31:03.707 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Mar 04 09:31:03 smithi052 ceph-4e059f08-da08-11ee-95c6-87774f69a715-mgr-x[32705]: 2024-03-04T09:31:03.523+0000 7f600290f200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-04T09:31:03.707 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Mar 04 09:31:03 smithi052 ceph-4e059f08-da08-11ee-95c6-87774f69a715-mgr-x[32705]: 2024-03-04T09:31:03.612+0000 7f600290f200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-04T09:31:03.707 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Mar 04 09:31:03 smithi052 ceph-4e059f08-da08-11ee-95c6-87774f69a715-mgr-x[32705]: 2024-03-04T09:31:03.710+0000 7f600290f200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-04T09:31:03.881 INFO:journalctl@ceph.mgr.y.smithi106.stdout:Mar 04 09:31:03 smithi106 ceph-4e059f08-da08-11ee-95c6-87774f69a715-mgr-y[37865]: 2024-03-04T09:31:03.545+0000 7faa9a554200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-04T09:31:03.881 INFO:journalctl@ceph.mgr.y.smithi106.stdout:Mar 04 09:31:03 smithi106 ceph-4e059f08-da08-11ee-95c6-87774f69a715-mgr-y[37865]: 2024-03-04T09:31:03.656+0000 7faa9a554200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-04T09:31:03.881 INFO:journalctl@ceph.mgr.y.smithi106.stdout:Mar 04 09:31:03 smithi106 ceph-4e059f08-da08-11ee-95c6-87774f69a715-mgr-y[37865]: 2024-03-04T09:31:03.765+0000 7faa9a554200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-04T09:31:04.084 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Mar 04 09:31:03 smithi052 ceph-4e059f08-da08-11ee-95c6-87774f69a715-mgr-x[32705]: 2024-03-04T09:31:03.888+0000 7f600290f200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-04T09:31:04.086 INFO:journalctl@ceph.mgr.z.smithi191.stdout:Mar 04 09:31:03 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-mgr-z[38667]: 2024-03-04T09:31:03.706+0000 7f8d8a385200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-04T09:31:04.086 INFO:journalctl@ceph.mgr.z.smithi191.stdout:Mar 04 09:31:03 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-mgr-z[38667]: 2024-03-04T09:31:03.886+0000 7f8d8a385200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-04T09:31:04.381 INFO:journalctl@ceph.mgr.y.smithi106.stdout:Mar 04 09:31:03 smithi106 ceph-4e059f08-da08-11ee-95c6-87774f69a715-mgr-y[37865]: 2024-03-04T09:31:03.908+0000 7faa9a554200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-04T09:31:04.381 INFO:journalctl@ceph.mgr.y.smithi106.stdout:Mar 04 09:31:04 smithi106 ceph-4e059f08-da08-11ee-95c6-87774f69a715-mgr-y[37865]: 2024-03-04T09:31:04.143+0000 7faa9a554200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-04T09:31:05.131 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:05.135+0000 7f1d1a3b1640 1 -- 172.21.15.52:0/2176133942 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f1d000a1f10 msgr2=0x7f1d000a43d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.52:6800/3036033923 2024-03-04T09:31:05.132 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:05.135+0000 7f1d1a3b1640 1 --2- 172.21.15.52:0/2176133942 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f1d000a1f10 0x7f1d000a43d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-03-04T09:31:05.334 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Mar 04 09:31:04 smithi052 ceph-4e059f08-da08-11ee-95c6-87774f69a715-mgr-x[32705]: 2024-03-04T09:31:04.881+0000 7f600290f200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-04T09:31:05.335 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Mar 04 09:31:05 smithi052 ceph-4e059f08-da08-11ee-95c6-87774f69a715-mgr-x[32705]: 2024-03-04T09:31:05.015+0000 7f600290f200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-04T09:31:05.336 INFO:journalctl@ceph.mgr.z.smithi191.stdout:Mar 04 09:31:04 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-mgr-z[38667]: 2024-03-04T09:31:04.880+0000 7f8d8a385200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-04T09:31:05.336 INFO:journalctl@ceph.mgr.z.smithi191.stdout:Mar 04 09:31:05 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-mgr-z[38667]: 2024-03-04T09:31:05.012+0000 7f8d8a385200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-04T09:31:05.631 INFO:journalctl@ceph.mgr.y.smithi106.stdout:Mar 04 09:31:05 smithi106 ceph-4e059f08-da08-11ee-95c6-87774f69a715-mgr-y[37865]: 2024-03-04T09:31:05.244+0000 7faa9a554200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-04T09:31:05.631 INFO:journalctl@ceph.mgr.y.smithi106.stdout:Mar 04 09:31:05 smithi106 ceph-4e059f08-da08-11ee-95c6-87774f69a715-mgr-y[37865]: 2024-03-04T09:31:05.377+0000 7faa9a554200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-04T09:31:05.649 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Mar 04 09:31:05 smithi052 ceph-4e059f08-da08-11ee-95c6-87774f69a715-mgr-x[32705]: 2024-03-04T09:31:05.355+0000 7f600290f200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-04T09:31:05.649 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Mar 04 09:31:05 smithi052 ceph-4e059f08-da08-11ee-95c6-87774f69a715-mgr-x[32705]: 2024-03-04T09:31:05.448+0000 7f600290f200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-04T09:31:05.664 INFO:journalctl@ceph.mgr.z.smithi191.stdout:Mar 04 09:31:05 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-mgr-z[38667]: 2024-03-04T09:31:05.353+0000 7f8d8a385200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-04T09:31:05.664 INFO:journalctl@ceph.mgr.z.smithi191.stdout:Mar 04 09:31:05 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-mgr-z[38667]: 2024-03-04T09:31:05.444+0000 7f8d8a385200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-04T09:31:05.665 INFO:journalctl@ceph.mgr.z.smithi191.stdout:Mar 04 09:31:05 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-mgr-z[38667]: 2024-03-04T09:31:05.667+0000 7f8d8a385200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-04T09:31:06.003 INFO:journalctl@ceph.mgr.y.smithi106.stdout:Mar 04 09:31:05 smithi106 ceph-4e059f08-da08-11ee-95c6-87774f69a715-mgr-y[37865]: 2024-03-04T09:31:05.714+0000 7faa9a554200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-04T09:31:06.003 INFO:journalctl@ceph.mgr.y.smithi106.stdout:Mar 04 09:31:05 smithi106 ceph-4e059f08-da08-11ee-95c6-87774f69a715-mgr-y[37865]: 2024-03-04T09:31:05.804+0000 7faa9a554200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-04T09:31:06.084 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Mar 04 09:31:05 smithi052 ceph-4e059f08-da08-11ee-95c6-87774f69a715-mgr-x[32705]: 2024-03-04T09:31:05.652+0000 7f600290f200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-04T09:31:06.085 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Mar 04 09:31:05 smithi052 ceph-4e059f08-da08-11ee-95c6-87774f69a715-mgr-x[32705]: 2024-03-04T09:31:05.845+0000 7f600290f200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-04T09:31:06.086 INFO:journalctl@ceph.mgr.z.smithi191.stdout:Mar 04 09:31:05 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-mgr-z[38667]: 2024-03-04T09:31:05.852+0000 7f8d8a385200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-04T09:31:06.381 INFO:journalctl@ceph.mgr.y.smithi106.stdout:Mar 04 09:31:06 smithi106 ceph-4e059f08-da08-11ee-95c6-87774f69a715-mgr-y[37865]: 2024-03-04T09:31:06.006+0000 7faa9a554200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-04T09:31:06.381 INFO:journalctl@ceph.mgr.y.smithi106.stdout:Mar 04 09:31:06 smithi106 ceph-4e059f08-da08-11ee-95c6-87774f69a715-mgr-y[37865]: 2024-03-04T09:31:06.194+0000 7faa9a554200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-04T09:31:06.404 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Mar 04 09:31:06 smithi052 ceph-4e059f08-da08-11ee-95c6-87774f69a715-mgr-x[32705]: 2024-03-04T09:31:06.144+0000 7f600290f200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-04T09:31:06.461 INFO:journalctl@ceph.mgr.z.smithi191.stdout:Mar 04 09:31:06 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-mgr-z[38667]: 2024-03-04T09:31:06.182+0000 7f8d8a385200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-04T09:31:06.461 INFO:journalctl@ceph.mgr.z.smithi191.stdout:Mar 04 09:31:06 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-mgr-z[38667]: 2024-03-04T09:31:06.464+0000 7f8d8a385200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-04T09:31:06.741 INFO:journalctl@ceph.mgr.y.smithi106.stdout:Mar 04 09:31:06 smithi106 ceph-4e059f08-da08-11ee-95c6-87774f69a715-mgr-y[37865]: 2024-03-04T09:31:06.483+0000 7faa9a554200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-04T09:31:06.741 INFO:journalctl@ceph.mgr.y.smithi106.stdout:Mar 04 09:31:06 smithi106 ceph-4e059f08-da08-11ee-95c6-87774f69a715-mgr-y[37865]: 2024-03-04T09:31:06.743+0000 7faa9a554200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-04T09:31:06.834 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Mar 04 09:31:06 smithi052 ceph-4e059f08-da08-11ee-95c6-87774f69a715-mgr-x[32705]: 2024-03-04T09:31:06.407+0000 7f600290f200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-04T09:31:06.835 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Mar 04 09:31:06 smithi052 ceph-4e059f08-da08-11ee-95c6-87774f69a715-mgr-x[32705]: 2024-03-04T09:31:06.654+0000 7f600290f200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-04T09:31:07.086 INFO:journalctl@ceph.mgr.z.smithi191.stdout:Mar 04 09:31:06 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-mgr-z[38667]: 2024-03-04T09:31:06.775+0000 7f8d8a385200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-04T09:31:07.131 INFO:journalctl@ceph.mgr.y.smithi106.stdout:Mar 04 09:31:06 smithi106 ceph-4e059f08-da08-11ee-95c6-87774f69a715-mgr-y[37865]: 2024-03-04T09:31:06.982+0000 7faa9a554200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-04T09:31:07.834 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Mar 04 09:31:07 smithi052 ceph-4e059f08-da08-11ee-95c6-87774f69a715-mgr-x[32705]: 2024-03-04T09:31:07.493+0000 7f600290f200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-04T09:31:08.086 INFO:journalctl@ceph.mgr.z.smithi191.stdout:Mar 04 09:31:07 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-mgr-z[38667]: 2024-03-04T09:31:07.621+0000 7f8d8a385200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-04T09:31:08.334 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Mar 04 09:31:08 smithi052 ceph-4e059f08-da08-11ee-95c6-87774f69a715-mgr-x[32705]: 2024-03-04T09:31:08.076+0000 7f600290f200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-04T09:31:08.335 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Mar 04 09:31:08 smithi052 ceph-4e059f08-da08-11ee-95c6-87774f69a715-mgr-x[32705]: 2024-03-04T09:31:08.180+0000 7f600290f200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-04T09:31:08.335 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:08.338+0000 7f1d1a3b1640 1 -- 172.21.15.52:0/2176133942 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f1d000a1f10 msgr2=0x7f1d000a43d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.52:6800/3036033923 2024-03-04T09:31:08.335 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:08.338+0000 7f1d1a3b1640 1 --2- 172.21.15.52:0/2176133942 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f1d000a1f10 0x7f1d000a43d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 6.400000 2024-03-04T09:31:08.381 INFO:journalctl@ceph.mgr.y.smithi106.stdout:Mar 04 09:31:07 smithi106 ceph-4e059f08-da08-11ee-95c6-87774f69a715-mgr-y[37865]: 2024-03-04T09:31:07.972+0000 7faa9a554200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-04T09:31:08.428 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:08.431+0000 7f1d0b7fe640 1 -- 172.21.15.52:0/2176133942 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mgrmap(e 20) v1 ==== 136441+0+0 (secure 0 0 0) 0x7f1d040975d0 con 0x7f1d14102810 2024-03-04T09:31:08.428 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:08.432+0000 7f1d0b7fe640 1 -- 172.21.15.52:0/2176133942 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f1d000a1f10 msgr2=0x7f1d000a43d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-04T09:31:08.429 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:08.432+0000 7f1d0b7fe640 1 --2- 172.21.15.52:0/2176133942 >> [v2:172.21.15.52:6800/3036033923,v1:172.21.15.52:6801/3036033923] conn(0x7f1d000a1f10 0x7f1d000a43d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:31:08.471 INFO:journalctl@ceph.mgr.z.smithi191.stdout:Mar 04 09:31:08 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-mgr-z[38667]: 2024-03-04T09:31:08.205+0000 7f8d8a385200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-04T09:31:08.471 INFO:journalctl@ceph.mgr.z.smithi191.stdout:Mar 04 09:31:08 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-mgr-z[38667]: 2024-03-04T09:31:08.303+0000 7f8d8a385200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-04T09:31:08.619 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Mar 04 09:31:08 smithi052 ceph-4e059f08-da08-11ee-95c6-87774f69a715-mgr-x[32705]: 2024-03-04T09:31:08.408+0000 7f600290f200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-04T09:31:08.619 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:31:08 smithi052 ceph-mon[32473]: Active manager daemon x restarted 2024-03-04T09:31:08.619 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:31:08 smithi052 ceph-mon[32473]: Activating manager daemon x 2024-03-04T09:31:08.619 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:31:08 smithi052 ceph-mon[32473]: osdmap e94: 12 total, 12 up, 12 in 2024-03-04T09:31:08.619 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:31:08 smithi052 ceph-mon[32473]: mgrmap e20: x(active, starting, since 0.0117708s), standbys: z, y 2024-03-04T09:31:08.619 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:31:08 smithi052 ceph-mon[32473]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-04T09:31:08.620 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:31:08 smithi052 ceph-mon[32473]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-04T09:31:08.620 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:31:08 smithi052 ceph-mon[32473]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-04T09:31:08.620 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:31:08 smithi052 ceph-mon[32473]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-03-04T09:31:08.620 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:31:08 smithi052 ceph-mon[32473]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-03-04T09:31:08.620 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:31:08 smithi052 ceph-mon[32473]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-03-04T09:31:08.620 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:31:08 smithi052 ceph-mon[32473]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-03-04T09:31:08.620 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:31:08 smithi052 ceph-mon[32473]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-03-04T09:31:08.620 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:31:08 smithi052 ceph-mon[32473]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-03-04T09:31:08.620 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:31:08 smithi052 ceph-mon[32473]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-03-04T09:31:08.620 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:31:08 smithi052 ceph-mon[32473]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-03-04T09:31:08.620 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:31:08 smithi052 ceph-mon[32473]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-04T09:31:08.620 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:31:08 smithi052 ceph-mon[32473]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-03-04T09:31:08.620 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:31:08 smithi052 ceph-mon[32473]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-03-04T09:31:08.620 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:31:08 smithi052 ceph-mon[32473]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-03-04T09:31:08.620 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:31:08 smithi052 ceph-mon[32473]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-03-04T09:31:08.621 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:31:08 smithi052 ceph-mon[32473]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-03-04T09:31:08.621 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:31:08 smithi052 ceph-mon[32473]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-03-04T09:31:08.621 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:31:08 smithi052 ceph-mon[32473]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-04T09:31:08.621 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:31:08 smithi052 ceph-mon[32473]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-04T09:31:08.621 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:31:08 smithi052 ceph-mon[32473]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-04T09:31:08.621 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:31:08 smithi052 ceph-mon[32473]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-04T09:31:08.621 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:31:08 smithi052 ceph-mon[32473]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-04T09:31:08.621 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:31:08 smithi052 ceph-mon[32473]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-04T09:31:08.621 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:31:08 smithi052 ceph-mon[32473]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-04T09:31:08.621 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:31:08 smithi052 ceph-mon[32473]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-04T09:31:08.621 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:31:08 smithi052 ceph-mon[32473]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-04T09:31:08.621 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:31:08 smithi052 ceph-mon[32473]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-04T09:31:08.621 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:31:08 smithi052 ceph-mon[32473]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-04T09:31:08.621 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:31:08 smithi052 ceph-mon[32473]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-04T09:31:08.621 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:31:08 smithi052 ceph-mon[32473]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-04T09:31:08.621 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:31:08 smithi052 ceph-mon[32473]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-04T09:31:08.621 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:31:08 smithi052 ceph-mon[32473]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-04T09:31:08.622 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:31:08 smithi052 ceph-mon[32473]: Manager daemon x is now available 2024-03-04T09:31:08.747 INFO:journalctl@ceph.mgr.y.smithi106.stdout:Mar 04 09:31:08 smithi106 ceph-4e059f08-da08-11ee-95c6-87774f69a715-mgr-y[37865]: 2024-03-04T09:31:08.643+0000 7faa9a554200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-04T09:31:08.747 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:31:08 smithi106 ceph-mon[36484]: Active manager daemon x restarted 2024-03-04T09:31:08.747 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:31:08 smithi106 ceph-mon[36484]: Activating manager daemon x 2024-03-04T09:31:08.747 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:31:08 smithi106 ceph-mon[36484]: osdmap e94: 12 total, 12 up, 12 in 2024-03-04T09:31:08.747 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:31:08 smithi106 ceph-mon[36484]: mgrmap e20: x(active, starting, since 0.0117708s), standbys: z, y 2024-03-04T09:31:08.747 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:31:08 smithi106 ceph-mon[36484]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-04T09:31:08.747 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:31:08 smithi106 ceph-mon[36484]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-04T09:31:08.748 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:31:08 smithi106 ceph-mon[36484]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-04T09:31:08.748 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:31:08 smithi106 ceph-mon[36484]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-03-04T09:31:08.748 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:31:08 smithi106 ceph-mon[36484]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-03-04T09:31:08.748 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:31:08 smithi106 ceph-mon[36484]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-03-04T09:31:08.748 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:31:08 smithi106 ceph-mon[36484]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-03-04T09:31:08.748 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:31:08 smithi106 ceph-mon[36484]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-03-04T09:31:08.748 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:31:08 smithi106 ceph-mon[36484]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-03-04T09:31:08.748 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:31:08 smithi106 ceph-mon[36484]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-03-04T09:31:08.748 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:31:08 smithi106 ceph-mon[36484]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-03-04T09:31:08.748 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:31:08 smithi106 ceph-mon[36484]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-04T09:31:08.748 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:31:08 smithi106 ceph-mon[36484]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-03-04T09:31:08.748 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:31:08 smithi106 ceph-mon[36484]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-03-04T09:31:08.748 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:31:08 smithi106 ceph-mon[36484]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-03-04T09:31:08.749 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:31:08 smithi106 ceph-mon[36484]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-03-04T09:31:08.749 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:31:08 smithi106 ceph-mon[36484]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-03-04T09:31:08.749 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:31:08 smithi106 ceph-mon[36484]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-03-04T09:31:08.749 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:31:08 smithi106 ceph-mon[36484]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-04T09:31:08.749 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:31:08 smithi106 ceph-mon[36484]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-04T09:31:08.749 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:31:08 smithi106 ceph-mon[36484]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-04T09:31:08.749 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:31:08 smithi106 ceph-mon[36484]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-04T09:31:08.749 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:31:08 smithi106 ceph-mon[36484]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-04T09:31:08.749 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:31:08 smithi106 ceph-mon[36484]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-04T09:31:08.749 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:31:08 smithi106 ceph-mon[36484]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-04T09:31:08.749 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:31:08 smithi106 ceph-mon[36484]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-04T09:31:08.749 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:31:08 smithi106 ceph-mon[36484]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-04T09:31:08.749 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:31:08 smithi106 ceph-mon[36484]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-04T09:31:08.749 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:31:08 smithi106 ceph-mon[36484]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-04T09:31:08.749 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:31:08 smithi106 ceph-mon[36484]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-04T09:31:08.750 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:31:08 smithi106 ceph-mon[36484]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-04T09:31:08.750 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:31:08 smithi106 ceph-mon[36484]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-04T09:31:08.750 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:31:08 smithi106 ceph-mon[36484]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-04T09:31:08.750 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:31:08 smithi106 ceph-mon[36484]: Manager daemon x is now available 2024-03-04T09:31:08.837 INFO:journalctl@ceph.mgr.z.smithi191.stdout:Mar 04 09:31:08 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-mgr-z[38667]: 2024-03-04T09:31:08.530+0000 7f8d8a385200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-04T09:31:08.837 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:31:08 smithi191 ceph-mon[36816]: Active manager daemon x restarted 2024-03-04T09:31:08.837 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:31:08 smithi191 ceph-mon[36816]: Activating manager daemon x 2024-03-04T09:31:08.837 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:31:08 smithi191 ceph-mon[36816]: osdmap e94: 12 total, 12 up, 12 in 2024-03-04T09:31:08.837 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:31:08 smithi191 ceph-mon[36816]: mgrmap e20: x(active, starting, since 0.0117708s), standbys: z, y 2024-03-04T09:31:08.838 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:31:08 smithi191 ceph-mon[36816]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-04T09:31:08.838 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:31:08 smithi191 ceph-mon[36816]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-04T09:31:08.838 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:31:08 smithi191 ceph-mon[36816]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-04T09:31:08.838 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:31:08 smithi191 ceph-mon[36816]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-03-04T09:31:08.838 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:31:08 smithi191 ceph-mon[36816]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-03-04T09:31:08.838 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:31:08 smithi191 ceph-mon[36816]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-03-04T09:31:08.838 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:31:08 smithi191 ceph-mon[36816]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-03-04T09:31:08.838 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:31:08 smithi191 ceph-mon[36816]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-03-04T09:31:08.838 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:31:08 smithi191 ceph-mon[36816]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-03-04T09:31:08.838 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:31:08 smithi191 ceph-mon[36816]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-03-04T09:31:08.838 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:31:08 smithi191 ceph-mon[36816]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-03-04T09:31:08.838 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:31:08 smithi191 ceph-mon[36816]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-04T09:31:08.838 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:31:08 smithi191 ceph-mon[36816]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-03-04T09:31:08.838 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:31:08 smithi191 ceph-mon[36816]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-03-04T09:31:08.839 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:31:08 smithi191 ceph-mon[36816]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-03-04T09:31:08.839 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:31:08 smithi191 ceph-mon[36816]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-03-04T09:31:08.839 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:31:08 smithi191 ceph-mon[36816]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-03-04T09:31:08.839 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:31:08 smithi191 ceph-mon[36816]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-03-04T09:31:08.839 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:31:08 smithi191 ceph-mon[36816]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-04T09:31:08.839 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:31:08 smithi191 ceph-mon[36816]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-04T09:31:08.839 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:31:08 smithi191 ceph-mon[36816]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-04T09:31:08.839 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:31:08 smithi191 ceph-mon[36816]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-04T09:31:08.839 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:31:08 smithi191 ceph-mon[36816]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-04T09:31:08.839 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:31:08 smithi191 ceph-mon[36816]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-04T09:31:08.839 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:31:08 smithi191 ceph-mon[36816]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-04T09:31:08.839 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:31:08 smithi191 ceph-mon[36816]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-04T09:31:08.839 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:31:08 smithi191 ceph-mon[36816]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-04T09:31:08.839 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:31:08 smithi191 ceph-mon[36816]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-04T09:31:08.840 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:31:08 smithi191 ceph-mon[36816]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-04T09:31:08.840 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:31:08 smithi191 ceph-mon[36816]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-04T09:31:08.840 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:31:08 smithi191 ceph-mon[36816]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-04T09:31:08.840 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:31:08 smithi191 ceph-mon[36816]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-04T09:31:08.840 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:31:08 smithi191 ceph-mon[36816]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-04T09:31:08.840 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:31:08 smithi191 ceph-mon[36816]: Manager daemon x is now available 2024-03-04T09:31:08.899 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Mar 04 09:31:08 smithi052 ceph-4e059f08-da08-11ee-95c6-87774f69a715-mgr-x[32705]: 2024-03-04T09:31:08.726+0000 7f5f642e8700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-04T09:31:08.900 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Mar 04 09:31:08 smithi052 ceph-4e059f08-da08-11ee-95c6-87774f69a715-mgr-x[32705]: 2024-03-04T09:31:08.726+0000 7f5f642e8700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-04T09:31:08.900 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Mar 04 09:31:08 smithi052 ceph-4e059f08-da08-11ee-95c6-87774f69a715-mgr-x[32705]: 2024-03-04T09:31:08.726+0000 7f5f642e8700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-04T09:31:08.900 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Mar 04 09:31:08 smithi052 ceph-4e059f08-da08-11ee-95c6-87774f69a715-mgr-x[32705]: 2024-03-04T09:31:08.726+0000 7f5f642e8700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-04T09:31:08.900 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Mar 04 09:31:08 smithi052 ceph-4e059f08-da08-11ee-95c6-87774f69a715-mgr-x[32705]: 2024-03-04T09:31:08.726+0000 7f5f642e8700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-04T09:31:08.900 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Mar 04 09:31:08 smithi052 ceph-4e059f08-da08-11ee-95c6-87774f69a715-mgr-x[32705]: 2024-03-04T09:31:08.727+0000 7f5f60ae1700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-04T09:31:08.900 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Mar 04 09:31:08 smithi052 ceph-4e059f08-da08-11ee-95c6-87774f69a715-mgr-x[32705]: 2024-03-04T09:31:08.727+0000 7f5f60ae1700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-04T09:31:08.900 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Mar 04 09:31:08 smithi052 ceph-4e059f08-da08-11ee-95c6-87774f69a715-mgr-x[32705]: 2024-03-04T09:31:08.727+0000 7f5f60ae1700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-04T09:31:08.900 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Mar 04 09:31:08 smithi052 ceph-4e059f08-da08-11ee-95c6-87774f69a715-mgr-x[32705]: 2024-03-04T09:31:08.727+0000 7f5f60ae1700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-04T09:31:08.900 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Mar 04 09:31:08 smithi052 ceph-4e059f08-da08-11ee-95c6-87774f69a715-mgr-x[32705]: 2024-03-04T09:31:08.727+0000 7f5f60ae1700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-04T09:31:09.131 INFO:journalctl@ceph.mgr.y.smithi106.stdout:Mar 04 09:31:08 smithi106 ceph-4e059f08-da08-11ee-95c6-87774f69a715-mgr-y[37865]: 2024-03-04T09:31:08.749+0000 7faa9a554200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-04T09:31:09.131 INFO:journalctl@ceph.mgr.y.smithi106.stdout:Mar 04 09:31:08 smithi106 ceph-4e059f08-da08-11ee-95c6-87774f69a715-mgr-y[37865]: 2024-03-04T09:31:08.980+0000 7faa9a554200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-04T09:31:09.437 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:09.439+0000 7f1d0b7fe640 1 -- 172.21.15.52:0/2176133942 <== mon.0 v2:172.21.15.52:3300/0 8 ==== mgrmap(e 21) v1 ==== 136568+0+0 (secure 0 0 0) 0x7f1d0409b1c0 con 0x7f1d14102810 2024-03-04T09:31:09.437 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:09.439+0000 7f1d0b7fe640 1 --2- 172.21.15.52:0/2176133942 >> [v2:172.21.15.52:6800/478031757,v1:172.21.15.52:6801/478031757] conn(0x7f1d000515d0 0x7f1d000519b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:31:09.437 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:09.439+0000 7f1d0b7fe640 1 -- 172.21.15.52:0/2176133942 --> [v2:172.21.15.52:6800/478031757,v1:172.21.15.52:6801/478031757] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) v1 -- 0x7f1d0003c760 con 0x7f1d000515d0 2024-03-04T09:31:09.438 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:09.442+0000 7f1d1a3b1640 1 --2- 172.21.15.52:0/2176133942 >> [v2:172.21.15.52:6800/478031757,v1:172.21.15.52:6801/478031757] conn(0x7f1d000515d0 0x7f1d000519b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:31:09.441 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:09.444+0000 7f1d1a3b1640 1 --2- 172.21.15.52:0/2176133942 >> [v2:172.21.15.52:6800/478031757,v1:172.21.15.52:6801/478031757] conn(0x7f1d000515d0 0x7f1d000519b0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f1d10000c00 tx=0x7f1d10005fd0 comp rx=0 tx=0).ready entity=mgr.15486 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:31:09.515 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:09.518+0000 7f1d0b7fe640 1 -- 172.21.15.52:0/2176133942 <== mgr.15486 v2:172.21.15.52:6800/478031757 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f1d0003c760 con 0x7f1d000515d0 2024-03-04T09:31:09.516 INFO:teuthology.orchestra.run.smithi052.stdout:Schedule set for path / 2024-03-04T09:31:09.517 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:09.520+0000 7f1d1c615640 1 -- 172.21.15.52:0/2176133942 >> [v2:172.21.15.52:6800/478031757,v1:172.21.15.52:6801/478031757] conn(0x7f1d000515d0 msgr2=0x7f1d000519b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:31:09.517 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:09.520+0000 7f1d1c615640 1 --2- 172.21.15.52:0/2176133942 >> [v2:172.21.15.52:6800/478031757,v1:172.21.15.52:6801/478031757] conn(0x7f1d000515d0 0x7f1d000519b0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f1d10000c00 tx=0x7f1d10005fd0 comp rx=0 tx=0).stop 2024-03-04T09:31:09.517 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:09.520+0000 7f1d1c615640 1 -- 172.21.15.52:0/2176133942 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1d14102810 msgr2=0x7f1d14197e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:31:09.518 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:09.520+0000 7f1d1c615640 1 --2- 172.21.15.52:0/2176133942 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1d14102810 0x7f1d14197e50 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f1d0400e9f0 tx=0x7f1d0400edf0 comp rx=0 tx=0).stop 2024-03-04T09:31:09.518 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:09.520+0000 7f1d1abb2640 1 -- 172.21.15.52:0/2176133942 reap_dead start 2024-03-04T09:31:09.518 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:09.520+0000 7f1d1c615640 1 -- 172.21.15.52:0/2176133942 shutdown_connections 2024-03-04T09:31:09.518 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:09.520+0000 7f1d1c615640 1 -- 172.21.15.52:0/2176133942 >> 172.21.15.52:0/2176133942 conn(0x7f1d140fcb90 msgr2=0x7f1d14105b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:31:09.518 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:09.521+0000 7f1d1c615640 1 -- 172.21.15.52:0/2176133942 shutdown_connections 2024-03-04T09:31:09.518 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:09.521+0000 7f1d1c615640 1 -- 172.21.15.52:0/2176133942 wait complete. 2024-03-04T09:31:09.530 DEBUG:teuthology.orchestra.run.smithi052:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h' 2024-03-04T09:31:09.647 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:09.650+0000 7f49eb91e640 1 -- 172.21.15.52:0/3291276997 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f49e41020f0 msgr2=0x7f49e41044e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:31:09.647 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:09.650+0000 7f49eb91e640 1 --2- 172.21.15.52:0/3291276997 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f49e41020f0 0x7f49e41044e0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f49d4009900 tx=0x7f49d402bc30 comp rx=0 tx=0).stop 2024-03-04T09:31:09.647 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:09.650+0000 7f49eb91e640 1 -- 172.21.15.52:0/3291276997 shutdown_connections 2024-03-04T09:31:09.648 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:09.650+0000 7f49eb91e640 1 --2- 172.21.15.52:0/3291276997 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f49e4107350 0x7f49e4109740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:31:09.648 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:09.650+0000 7f49eb91e640 1 --2- 172.21.15.52:0/3291276997 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f49e4104a20 0x7f49e4106e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:31:09.648 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:09.650+0000 7f49eb91e640 1 --2- 172.21.15.52:0/3291276997 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f49e41020f0 0x7f49e41044e0 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:31:09.648 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:09.650+0000 7f49eb91e640 1 -- 172.21.15.52:0/3291276997 >> 172.21.15.52:0/3291276997 conn(0x7f49e40fba60 msgr2=0x7f49e40fdea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:31:09.648 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:09.650+0000 7f49eb91e640 1 -- 172.21.15.52:0/3291276997 shutdown_connections 2024-03-04T09:31:09.648 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:09.650+0000 7f49eb91e640 1 -- 172.21.15.52:0/3291276997 wait complete. 2024-03-04T09:31:09.648 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:09.651+0000 7f49eb91e640 1 Processor -- start 2024-03-04T09:31:09.648 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:09.651+0000 7f49eb91e640 1 -- start start 2024-03-04T09:31:09.650 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:09.651+0000 7f49eb91e640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f49e41020f0 0x7f49e4197880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:31:09.650 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:09.651+0000 7f49eb91e640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f49e4104a20 0x7f49e4197dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:31:09.650 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:09.651+0000 7f49eb91e640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f49e4107350 0x7f49e419ee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:31:09.650 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:09.651+0000 7f49eb91e640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49e410d260 con 0x7f49e4104a20 2024-03-04T09:31:09.650 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:09.651+0000 7f49eb91e640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49e410d0e0 con 0x7f49e41020f0 2024-03-04T09:31:09.650 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:09.651+0000 7f49eb91e640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49e410d3e0 con 0x7f49e4107350 2024-03-04T09:31:09.650 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:09.651+0000 7f49e8eb9640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f49e4104a20 0x7f49e4197dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:31:09.650 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:09.651+0000 7f49e8eb9640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f49e4104a20 0x7f49e4197dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:57004/0 (socket says 172.21.15.52:57004) 2024-03-04T09:31:09.651 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:09.651+0000 7f49e8eb9640 1 -- 172.21.15.52:0/467409428 learned_addr learned my addr 172.21.15.52:0/467409428 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-04T09:31:09.651 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:09.651+0000 7f49e9ebb640 1 --2- 172.21.15.52:0/467409428 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f49e4107350 0x7f49e419ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:31:09.651 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:09.651+0000 7f49e96ba640 1 --2- 172.21.15.52:0/467409428 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f49e41020f0 0x7f49e4197880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:31:09.651 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:09.651+0000 7f49e8eb9640 1 -- 172.21.15.52:0/467409428 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f49e4107350 msgr2=0x7f49e419ee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:31:09.651 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:09.651+0000 7f49e8eb9640 1 --2- 172.21.15.52:0/467409428 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f49e4107350 0x7f49e419ee30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:31:09.651 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:09.651+0000 7f49e8eb9640 1 -- 172.21.15.52:0/467409428 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f49e41020f0 msgr2=0x7f49e4197880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:31:09.651 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:09.651+0000 7f49e8eb9640 1 --2- 172.21.15.52:0/467409428 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f49e41020f0 0x7f49e4197880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:31:09.651 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:09.651+0000 7f49e8eb9640 1 -- 172.21.15.52:0/467409428 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49e419f4a0 con 0x7f49e4104a20 2024-03-04T09:31:09.651 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:09.651+0000 7f49e8eb9640 1 --2- 172.21.15.52:0/467409428 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f49e4104a20 0x7f49e4197dc0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f49e000e9c0 tx=0x7f49e000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:31:09.651 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:09.652+0000 7f49da7fc640 1 -- 172.21.15.52:0/467409428 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49e000cd80 con 0x7f49e4104a20 2024-03-04T09:31:09.651 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:09.652+0000 7f49eb91e640 1 -- 172.21.15.52:0/467409428 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49e419f790 con 0x7f49e4104a20 2024-03-04T09:31:09.651 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:09.652+0000 7f49da7fc640 1 -- 172.21.15.52:0/467409428 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f49e0005500 con 0x7f49e4104a20 2024-03-04T09:31:09.651 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:09.652+0000 7f49da7fc640 1 -- 172.21.15.52:0/467409428 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49e0010690 con 0x7f49e4104a20 2024-03-04T09:31:09.652 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:09.652+0000 7f49eb91e640 1 -- 172.21.15.52:0/467409428 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49e419fce0 con 0x7f49e4104a20 2024-03-04T09:31:09.652 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:09.653+0000 7f49eb91e640 1 -- 172.21.15.52:0/467409428 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f49e40fd550 con 0x7f49e4104a20 2024-03-04T09:31:09.652 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:09.655+0000 7f49da7fc640 1 -- 172.21.15.52:0/467409428 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 21) v1 ==== 136568+0+0 (secure 0 0 0) 0x7f49e00046c0 con 0x7f49e4104a20 2024-03-04T09:31:09.652 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:09.655+0000 7f49da7fc640 1 --2- 172.21.15.52:0/467409428 >> [v2:172.21.15.52:6800/478031757,v1:172.21.15.52:6801/478031757] conn(0x7f49d00a1c30 0x7f49d00a40f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:31:09.652 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:09.656+0000 7f49e96ba640 1 --2- 172.21.15.52:0/467409428 >> [v2:172.21.15.52:6800/478031757,v1:172.21.15.52:6801/478031757] conn(0x7f49d00a1c30 0x7f49d00a40f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:31:09.653 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:09.656+0000 7f49da7fc640 1 -- 172.21.15.52:0/467409428 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9459+0+0 (secure 0 0 0) 0x7f49e00ce3d0 con 0x7f49e4104a20 2024-03-04T09:31:09.653 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:09.656+0000 7f49e96ba640 1 --2- 172.21.15.52:0/467409428 >> [v2:172.21.15.52:6800/478031757,v1:172.21.15.52:6801/478031757] conn(0x7f49d00a1c30 0x7f49d00a40f0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f49d402c7b0 tx=0x7f49d4005fd0 comp rx=0 tx=0).ready entity=mgr.15486 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:31:09.655 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:09.659+0000 7f49da7fc640 1 -- 172.21.15.52:0/467409428 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f49e0053280 con 0x7f49e4104a20 2024-03-04T09:31:09.786 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:31:09 smithi191 ceph-mon[36816]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T09:31:09.786 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:31:09 smithi191 ceph-mon[36816]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T09:31:09.786 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:31:09 smithi191 ceph-mon[36816]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-03-04T09:31:09.786 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:31:09 smithi191 ceph-mon[36816]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-03-04T09:31:09.786 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:31:09 smithi191 ceph-mon[36816]: Standby manager daemon z restarted 2024-03-04T09:31:09.786 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:31:09 smithi191 ceph-mon[36816]: Standby manager daemon z started 2024-03-04T09:31:09.786 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:31:09 smithi191 ceph-mon[36816]: from='mgr.? 172.21.15.191:0/198984678' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-03-04T09:31:09.786 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:31:09 smithi191 ceph-mon[36816]: from='mgr.? 172.21.15.191:0/198984678' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-04T09:31:09.787 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:31:09 smithi191 ceph-mon[36816]: from='mgr.? 172.21.15.191:0/198984678' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-03-04T09:31:09.787 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:31:09 smithi191 ceph-mon[36816]: from='mgr.? 172.21.15.191:0/198984678' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-04T09:31:09.787 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:31:09 smithi191 ceph-mon[36816]: Standby manager daemon y restarted 2024-03-04T09:31:09.787 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:31:09 smithi191 ceph-mon[36816]: Standby manager daemon y started 2024-03-04T09:31:09.787 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:31:09 smithi191 ceph-mon[36816]: from='mgr.? 172.21.15.106:0/2869438767' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-03-04T09:31:09.787 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:31:09 smithi191 ceph-mon[36816]: from='mgr.? 172.21.15.106:0/2869438767' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-04T09:31:09.787 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:31:09 smithi191 ceph-mon[36816]: from='mgr.? 172.21.15.106:0/2869438767' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-03-04T09:31:09.787 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:31:09 smithi191 ceph-mon[36816]: from='mgr.? 172.21.15.106:0/2869438767' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-04T09:31:09.787 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:31:09 smithi191 ceph-mon[36816]: [04/Mar/2024:09:31:09] ENGINE Bus STARTING 2024-03-04T09:31:09.787 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:31:09 smithi191 ceph-mon[36816]: [04/Mar/2024:09:31:09] ENGINE Serving on http://172.21.15.52:8765 2024-03-04T09:31:09.787 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:31:09 smithi191 ceph-mon[36816]: [04/Mar/2024:09:31:09] ENGINE Serving on https://172.21.15.52:7150 2024-03-04T09:31:09.787 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:31:09 smithi191 ceph-mon[36816]: [04/Mar/2024:09:31:09] ENGINE Bus STARTED 2024-03-04T09:31:09.787 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:31:09 smithi191 ceph-mon[36816]: mgrmap e21: x(active, since 1.01961s), standbys: z, y 2024-03-04T09:31:09.835 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:31:09 smithi052 ceph-mon[32473]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T09:31:09.835 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:31:09 smithi052 ceph-mon[32473]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T09:31:09.835 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:31:09 smithi052 ceph-mon[32473]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-03-04T09:31:09.835 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:31:09 smithi052 ceph-mon[32473]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-03-04T09:31:09.835 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:31:09 smithi052 ceph-mon[32473]: Standby manager daemon z restarted 2024-03-04T09:31:09.835 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:31:09 smithi052 ceph-mon[32473]: Standby manager daemon z started 2024-03-04T09:31:09.836 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:31:09 smithi052 ceph-mon[32473]: from='mgr.? 172.21.15.191:0/198984678' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-03-04T09:31:09.836 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:31:09 smithi052 ceph-mon[32473]: from='mgr.? 172.21.15.191:0/198984678' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-04T09:31:09.836 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:31:09 smithi052 ceph-mon[32473]: from='mgr.? 172.21.15.191:0/198984678' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-03-04T09:31:09.836 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:31:09 smithi052 ceph-mon[32473]: from='mgr.? 172.21.15.191:0/198984678' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-04T09:31:09.836 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:31:09 smithi052 ceph-mon[32473]: Standby manager daemon y restarted 2024-03-04T09:31:09.836 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:31:09 smithi052 ceph-mon[32473]: Standby manager daemon y started 2024-03-04T09:31:09.836 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:31:09 smithi052 ceph-mon[32473]: from='mgr.? 172.21.15.106:0/2869438767' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-03-04T09:31:09.836 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:31:09 smithi052 ceph-mon[32473]: from='mgr.? 172.21.15.106:0/2869438767' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-04T09:31:09.836 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:31:09 smithi052 ceph-mon[32473]: from='mgr.? 172.21.15.106:0/2869438767' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-03-04T09:31:09.836 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:31:09 smithi052 ceph-mon[32473]: from='mgr.? 172.21.15.106:0/2869438767' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-04T09:31:09.836 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:31:09 smithi052 ceph-mon[32473]: [04/Mar/2024:09:31:09] ENGINE Bus STARTING 2024-03-04T09:31:09.836 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:31:09 smithi052 ceph-mon[32473]: [04/Mar/2024:09:31:09] ENGINE Serving on http://172.21.15.52:8765 2024-03-04T09:31:09.836 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:31:09 smithi052 ceph-mon[32473]: [04/Mar/2024:09:31:09] ENGINE Serving on https://172.21.15.52:7150 2024-03-04T09:31:09.837 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:31:09 smithi052 ceph-mon[32473]: [04/Mar/2024:09:31:09] ENGINE Bus STARTED 2024-03-04T09:31:09.837 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:31:09 smithi052 ceph-mon[32473]: mgrmap e21: x(active, since 1.01961s), standbys: z, y 2024-03-04T09:31:09.863 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:09.866+0000 7f49eb91e640 1 -- 172.21.15.52:0/467409428 --> [v2:172.21.15.52:6800/478031757,v1:172.21.15.52:6801/478031757] -- mgr_command(tid 0: {"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}) v1 -- 0x7f49e4061ab0 con 0x7f49d00a1c30 2024-03-04T09:31:09.881 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:31:09 smithi106 ceph-mon[36484]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T09:31:09.882 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:31:09 smithi106 ceph-mon[36484]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T09:31:09.882 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:31:09 smithi106 ceph-mon[36484]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-03-04T09:31:09.882 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:31:09 smithi106 ceph-mon[36484]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-03-04T09:31:09.882 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:31:09 smithi106 ceph-mon[36484]: Standby manager daemon z restarted 2024-03-04T09:31:09.882 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:31:09 smithi106 ceph-mon[36484]: Standby manager daemon z started 2024-03-04T09:31:09.882 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:31:09 smithi106 ceph-mon[36484]: from='mgr.? 172.21.15.191:0/198984678' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-03-04T09:31:09.882 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:31:09 smithi106 ceph-mon[36484]: from='mgr.? 172.21.15.191:0/198984678' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-04T09:31:09.882 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:31:09 smithi106 ceph-mon[36484]: from='mgr.? 172.21.15.191:0/198984678' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-03-04T09:31:09.882 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:31:09 smithi106 ceph-mon[36484]: from='mgr.? 172.21.15.191:0/198984678' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-04T09:31:09.882 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:31:09 smithi106 ceph-mon[36484]: Standby manager daemon y restarted 2024-03-04T09:31:09.882 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:31:09 smithi106 ceph-mon[36484]: Standby manager daemon y started 2024-03-04T09:31:09.882 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:31:09 smithi106 ceph-mon[36484]: from='mgr.? 172.21.15.106:0/2869438767' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-03-04T09:31:09.882 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:31:09 smithi106 ceph-mon[36484]: from='mgr.? 172.21.15.106:0/2869438767' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-04T09:31:09.883 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:31:09 smithi106 ceph-mon[36484]: from='mgr.? 172.21.15.106:0/2869438767' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-03-04T09:31:09.883 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:31:09 smithi106 ceph-mon[36484]: from='mgr.? 172.21.15.106:0/2869438767' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-04T09:31:09.883 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:31:09 smithi106 ceph-mon[36484]: [04/Mar/2024:09:31:09] ENGINE Bus STARTING 2024-03-04T09:31:09.883 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:31:09 smithi106 ceph-mon[36484]: [04/Mar/2024:09:31:09] ENGINE Serving on http://172.21.15.52:8765 2024-03-04T09:31:09.883 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:31:09 smithi106 ceph-mon[36484]: [04/Mar/2024:09:31:09] ENGINE Serving on https://172.21.15.52:7150 2024-03-04T09:31:09.883 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:31:09 smithi106 ceph-mon[36484]: [04/Mar/2024:09:31:09] ENGINE Bus STARTED 2024-03-04T09:31:09.883 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:31:09 smithi106 ceph-mon[36484]: mgrmap e21: x(active, since 1.01961s), standbys: z, y 2024-03-04T09:31:09.925 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:09.928+0000 7f49da7fc640 1 -- 172.21.15.52:0/467409428 <== mgr.15486 v2:172.21.15.52:6800/478031757 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+26 (secure 0 0 0) 0x7f49e4061ab0 con 0x7f49d00a1c30 2024-03-04T09:31:09.925 INFO:teuthology.orchestra.run.smithi052.stdout:Retention added to path / 2024-03-04T09:31:09.928 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:09.930+0000 7f49eb91e640 1 -- 172.21.15.52:0/467409428 >> [v2:172.21.15.52:6800/478031757,v1:172.21.15.52:6801/478031757] conn(0x7f49d00a1c30 msgr2=0x7f49d00a40f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:31:09.928 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:09.930+0000 7f49eb91e640 1 --2- 172.21.15.52:0/467409428 >> [v2:172.21.15.52:6800/478031757,v1:172.21.15.52:6801/478031757] conn(0x7f49d00a1c30 0x7f49d00a40f0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f49d402c7b0 tx=0x7f49d4005fd0 comp rx=0 tx=0).stop 2024-03-04T09:31:09.928 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:09.930+0000 7f49eb91e640 1 -- 172.21.15.52:0/467409428 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f49e4104a20 msgr2=0x7f49e4197dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:31:09.928 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:09.930+0000 7f49eb91e640 1 --2- 172.21.15.52:0/467409428 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f49e4104a20 0x7f49e4197dc0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f49e000e9c0 tx=0x7f49e000ee80 comp rx=0 tx=0).stop 2024-03-04T09:31:09.928 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:09.930+0000 7f49eb91e640 1 -- 172.21.15.52:0/467409428 shutdown_connections 2024-03-04T09:31:09.928 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:09.930+0000 7f49eb91e640 1 --2- 172.21.15.52:0/467409428 >> [v2:172.21.15.52:6800/478031757,v1:172.21.15.52:6801/478031757] conn(0x7f49d00a1c30 0x7f49d00a40f0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:31:09.928 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:09.930+0000 7f49eb91e640 1 --2- 172.21.15.52:0/467409428 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f49e4107350 0x7f49e419ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:31:09.928 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:09.930+0000 7f49eb91e640 1 --2- 172.21.15.52:0/467409428 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f49e4104a20 0x7f49e4197dc0 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:31:09.928 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:09.930+0000 7f49eb91e640 1 --2- 172.21.15.52:0/467409428 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f49e41020f0 0x7f49e4197880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:31:09.928 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:09.930+0000 7f49eb91e640 1 -- 172.21.15.52:0/467409428 >> 172.21.15.52:0/467409428 conn(0x7f49e40fba60 msgr2=0x7f49e4102b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:31:09.928 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:09.930+0000 7f49eb91e640 1 -- 172.21.15.52:0/467409428 shutdown_connections 2024-03-04T09:31:09.928 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:09.930+0000 7f49eb91e640 1 -- 172.21.15.52:0/467409428 wait complete. 2024-03-04T09:31:09.939 DEBUG:teuthology.orchestra.run.smithi052:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2024-03-04T09:31:10.052 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:10.051+0000 7f53aba55640 1 -- 172.21.15.52:0/4082502178 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f53a4101700 msgr2=0x7f53a4101b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:31:10.052 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:10.051+0000 7f53aba55640 1 --2- 172.21.15.52:0/4082502178 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f53a4101700 0x7f53a4101b80 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f5394009900 tx=0x7f539402bc30 comp rx=0 tx=0).stop 2024-03-04T09:31:10.052 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:10.051+0000 7f53aba55640 1 -- 172.21.15.52:0/4082502178 shutdown_connections 2024-03-04T09:31:10.052 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:10.051+0000 7f53aba55640 1 --2- 172.21.15.52:0/4082502178 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f53a41020c0 0x7f53a4108950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:31:10.052 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:10.051+0000 7f53aba55640 1 --2- 172.21.15.52:0/4082502178 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f53a4101700 0x7f53a4101b80 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:31:10.052 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:10.051+0000 7f53aba55640 1 --2- 172.21.15.52:0/4082502178 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f53a4100510 0x7f53a4100930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:31:10.052 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:10.051+0000 7f53aba55640 1 -- 172.21.15.52:0/4082502178 >> 172.21.15.52:0/4082502178 conn(0x7f53a40fba80 msgr2=0x7f53a40fdec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:31:10.052 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:10.051+0000 7f53aba55640 1 -- 172.21.15.52:0/4082502178 shutdown_connections 2024-03-04T09:31:10.052 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:10.051+0000 7f53aba55640 1 -- 172.21.15.52:0/4082502178 wait complete. 2024-03-04T09:31:10.053 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:10.052+0000 7f53aba55640 1 Processor -- start 2024-03-04T09:31:10.053 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:10.052+0000 7f53aba55640 1 -- start start 2024-03-04T09:31:10.053 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:10.052+0000 7f53aba55640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f53a4100510 0x7f53a4197850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:31:10.053 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:10.052+0000 7f53aba55640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f53a4101700 0x7f53a4197d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:31:10.053 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:10.052+0000 7f53aba55640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f53a41020c0 0x7f53a419ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:31:10.053 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:10.052+0000 7f53aba55640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53a410c1a0 con 0x7f53a4100510 2024-03-04T09:31:10.053 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:10.052+0000 7f53aba55640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53a410c020 con 0x7f53a41020c0 2024-03-04T09:31:10.053 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:10.052+0000 7f53aba55640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53a410c320 con 0x7f53a4101700 2024-03-04T09:31:10.053 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:10.052+0000 7f53a97f1640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f53a4100510 0x7f53a4197850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:31:10.053 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:10.052+0000 7f53a97f1640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f53a4100510 0x7f53a4197850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:57012/0 (socket says 172.21.15.52:57012) 2024-03-04T09:31:10.053 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:10.052+0000 7f53a97f1640 1 -- 172.21.15.52:0/1082164722 learned_addr learned my addr 172.21.15.52:0/1082164722 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-04T09:31:10.053 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:10.052+0000 7f53a8ff0640 1 --2- 172.21.15.52:0/1082164722 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f53a4101700 0x7f53a4197d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:31:10.053 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:10.052+0000 7f53a9ff2640 1 --2- 172.21.15.52:0/1082164722 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f53a41020c0 0x7f53a419ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:31:10.053 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:10.052+0000 7f53a97f1640 1 -- 172.21.15.52:0/1082164722 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f53a4101700 msgr2=0x7f53a4197d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:31:10.054 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:10.052+0000 7f53a97f1640 1 --2- 172.21.15.52:0/1082164722 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f53a4101700 0x7f53a4197d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:31:10.054 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:10.052+0000 7f53a97f1640 1 -- 172.21.15.52:0/1082164722 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f53a41020c0 msgr2=0x7f53a419ee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:31:10.054 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:10.052+0000 7f53a97f1640 1 --2- 172.21.15.52:0/1082164722 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f53a41020c0 0x7f53a419ee00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:31:10.054 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:10.052+0000 7f53a97f1640 1 -- 172.21.15.52:0/1082164722 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53a419f470 con 0x7f53a4100510 2024-03-04T09:31:10.054 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:10.053+0000 7f53a97f1640 1 --2- 172.21.15.52:0/1082164722 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f53a4100510 0x7f53a4197850 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f53a000e9c0 tx=0x7f53a000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:31:10.054 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:10.053+0000 7f539a7fc640 1 -- 172.21.15.52:0/1082164722 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53a000cd80 con 0x7f53a4100510 2024-03-04T09:31:10.054 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:10.053+0000 7f539a7fc640 1 -- 172.21.15.52:0/1082164722 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f53a00035f0 con 0x7f53a4100510 2024-03-04T09:31:10.054 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:10.053+0000 7f53aba55640 1 -- 172.21.15.52:0/1082164722 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53a419f760 con 0x7f53a4100510 2024-03-04T09:31:10.054 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:10.053+0000 7f539a7fc640 1 -- 172.21.15.52:0/1082164722 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53a0010690 con 0x7f53a4100510 2024-03-04T09:31:10.054 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:10.053+0000 7f53aba55640 1 -- 172.21.15.52:0/1082164722 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53a419fc30 con 0x7f53a4100510 2024-03-04T09:31:10.054 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:10.054+0000 7f53aba55640 1 -- 172.21.15.52:0/1082164722 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f53a419fef0 con 0x7f53a4100510 2024-03-04T09:31:10.054 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:10.055+0000 7f539a7fc640 1 -- 172.21.15.52:0/1082164722 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 21) v1 ==== 136568+0+0 (secure 0 0 0) 0x7f53a00037a0 con 0x7f53a4100510 2024-03-04T09:31:10.055 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:10.055+0000 7f539a7fc640 1 --2- 172.21.15.52:0/1082164722 >> [v2:172.21.15.52:6800/478031757,v1:172.21.15.52:6801/478031757] conn(0x7f53900a1c30 0x7f53900a40f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:31:10.055 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:10.056+0000 7f53a8ff0640 1 --2- 172.21.15.52:0/1082164722 >> [v2:172.21.15.52:6800/478031757,v1:172.21.15.52:6801/478031757] conn(0x7f53900a1c30 0x7f53900a40f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:31:10.055 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:10.056+0000 7f539a7fc640 1 -- 172.21.15.52:0/1082164722 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9459+0+0 (secure 0 0 0) 0x7f53a00ce1d0 con 0x7f53a4100510 2024-03-04T09:31:10.055 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:10.056+0000 7f53a8ff0640 1 --2- 172.21.15.52:0/1082164722 >> [v2:172.21.15.52:6800/478031757,v1:172.21.15.52:6801/478031757] conn(0x7f53900a1c30 0x7f53900a40f0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f539402c7b0 tx=0x7f5394005fd0 comp rx=0 tx=0).ready entity=mgr.15486 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:31:10.056 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:10.060+0000 7f539a7fc640 1 -- 172.21.15.52:0/1082164722 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f53a009b420 con 0x7f53a4100510 2024-03-04T09:31:10.261 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:10.264+0000 7f53aba55640 1 -- 172.21.15.52:0/1082164722 --> [v2:172.21.15.52:6800/478031757,v1:172.21.15.52:6801/478031757] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) v1 -- 0x7f53a40fdbc0 con 0x7f53900a1c30 2024-03-04T09:31:10.268 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:10.271+0000 7f539a7fc640 1 -- 172.21.15.52:0/1082164722 <== mgr.15486 v2:172.21.15.52:6800/478031757 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+298 (secure 0 0 0) 0x7f53a40fdbc0 con 0x7f53900a1c30 2024-03-04T09:31:10.268 INFO:teuthology.orchestra.run.smithi052.stdout:{"fs": "cephfs", "subvol": null, "group": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2024-03-04T00:00:00", "created": "2024-03-04T09:31:09", "first": null, "last": null, "last_pruned": null, "created_count": 0, "pruned_count": 0, "active": true} 2024-03-04T09:31:10.270 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:10.273+0000 7f53aba55640 1 -- 172.21.15.52:0/1082164722 >> [v2:172.21.15.52:6800/478031757,v1:172.21.15.52:6801/478031757] conn(0x7f53900a1c30 msgr2=0x7f53900a40f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:31:10.271 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:10.273+0000 7f53aba55640 1 --2- 172.21.15.52:0/1082164722 >> [v2:172.21.15.52:6800/478031757,v1:172.21.15.52:6801/478031757] conn(0x7f53900a1c30 0x7f53900a40f0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f539402c7b0 tx=0x7f5394005fd0 comp rx=0 tx=0).stop 2024-03-04T09:31:10.271 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:10.273+0000 7f53aba55640 1 -- 172.21.15.52:0/1082164722 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f53a4100510 msgr2=0x7f53a4197850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:31:10.271 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:10.273+0000 7f53aba55640 1 --2- 172.21.15.52:0/1082164722 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f53a4100510 0x7f53a4197850 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f53a000e9c0 tx=0x7f53a000ee80 comp rx=0 tx=0).stop 2024-03-04T09:31:10.271 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:10.273+0000 7f53aba55640 1 -- 172.21.15.52:0/1082164722 shutdown_connections 2024-03-04T09:31:10.271 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:10.273+0000 7f53aba55640 1 --2- 172.21.15.52:0/1082164722 >> [v2:172.21.15.52:6800/478031757,v1:172.21.15.52:6801/478031757] conn(0x7f53900a1c30 0x7f53900a40f0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:31:10.271 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:10.273+0000 7f53aba55640 1 --2- 172.21.15.52:0/1082164722 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f53a41020c0 0x7f53a419ee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:31:10.271 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:10.273+0000 7f53aba55640 1 --2- 172.21.15.52:0/1082164722 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f53a4101700 0x7f53a4197d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:31:10.271 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:10.273+0000 7f53aba55640 1 --2- 172.21.15.52:0/1082164722 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f53a4100510 0x7f53a4197850 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:31:10.271 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:10.273+0000 7f53aba55640 1 -- 172.21.15.52:0/1082164722 >> 172.21.15.52:0/1082164722 conn(0x7f53a40fba80 msgr2=0x7f53a40fd4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:31:10.271 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:10.273+0000 7f53aba55640 1 -- 172.21.15.52:0/1082164722 shutdown_connections 2024-03-04T09:31:10.271 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:10.273+0000 7f53aba55640 1 -- 172.21.15.52:0/1082164722 wait complete. 2024-03-04T09:31:10.281 DEBUG:teuthology.orchestra.run.smithi052:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2024-03-04T09:31:10.395 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:10.396+0000 7f441ffbc640 1 -- 172.21.15.52:0/3481192497 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f44181031b0 msgr2=0x7f4418109a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:31:10.395 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:10.396+0000 7f441ffbc640 1 --2- 172.21.15.52:0/3481192497 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f44181031b0 0x7f4418109a40 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f4410009ca0 tx=0x7f441002c0f0 comp rx=0 tx=0).stop 2024-03-04T09:31:10.396 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:10.397+0000 7f441ffbc640 1 -- 172.21.15.52:0/3481192497 shutdown_connections 2024-03-04T09:31:10.396 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:10.397+0000 7f441ffbc640 1 --2- 172.21.15.52:0/3481192497 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f44181031b0 0x7f4418109a40 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:31:10.396 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:10.397+0000 7f441ffbc640 1 --2- 172.21.15.52:0/3481192497 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f44181027f0 0x7f4418102c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:31:10.396 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:10.397+0000 7f441ffbc640 1 --2- 172.21.15.52:0/3481192497 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4418101600 0x7f4418101a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:31:10.396 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:10.397+0000 7f441ffbc640 1 -- 172.21.15.52:0/3481192497 >> 172.21.15.52:0/3481192497 conn(0x7f44180fcb90 msgr2=0x7f44180fefb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:31:10.396 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:10.397+0000 7f441ffbc640 1 -- 172.21.15.52:0/3481192497 shutdown_connections 2024-03-04T09:31:10.396 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:10.397+0000 7f441ffbc640 1 -- 172.21.15.52:0/3481192497 wait complete. 2024-03-04T09:31:10.396 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:10.397+0000 7f441ffbc640 1 Processor -- start 2024-03-04T09:31:10.396 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:10.397+0000 7f441ffbc640 1 -- start start 2024-03-04T09:31:10.396 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:10.397+0000 7f441ffbc640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4418101600 0x7f44181977e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:31:10.396 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:10.397+0000 7f441ffbc640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f44181027f0 0x7f4418197d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:31:10.396 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:10.397+0000 7f441ffbc640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f44181031b0 0x7f441819ed90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:31:10.397 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:10.397+0000 7f441ffbc640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f441810d280 con 0x7f44181031b0 2024-03-04T09:31:10.397 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:10.397+0000 7f441ffbc640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f441810d100 con 0x7f4418101600 2024-03-04T09:31:10.397 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:10.397+0000 7f441ffbc640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f441810d400 con 0x7f44181027f0 2024-03-04T09:31:10.397 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:10.397+0000 7f441e559640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f44181031b0 0x7f441819ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:31:10.397 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:10.397+0000 7f441e559640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f44181031b0 0x7f441819ed90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:57032/0 (socket says 172.21.15.52:57032) 2024-03-04T09:31:10.397 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:10.397+0000 7f441e559640 1 -- 172.21.15.52:0/3102077195 learned_addr learned my addr 172.21.15.52:0/3102077195 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-04T09:31:10.397 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:10.398+0000 7f441d557640 1 --2- 172.21.15.52:0/3102077195 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f44181027f0 0x7f4418197d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:31:10.397 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:10.398+0000 7f441dd58640 1 --2- 172.21.15.52:0/3102077195 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4418101600 0x7f44181977e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:31:10.397 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:10.398+0000 7f441e559640 1 -- 172.21.15.52:0/3102077195 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f44181027f0 msgr2=0x7f4418197d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:31:10.397 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:10.398+0000 7f441e559640 1 --2- 172.21.15.52:0/3102077195 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f44181027f0 0x7f4418197d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:31:10.397 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:10.398+0000 7f441e559640 1 -- 172.21.15.52:0/3102077195 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4418101600 msgr2=0x7f44181977e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:31:10.397 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:10.398+0000 7f441e559640 1 --2- 172.21.15.52:0/3102077195 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4418101600 0x7f44181977e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:31:10.397 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:10.398+0000 7f441e559640 1 -- 172.21.15.52:0/3102077195 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f441819f400 con 0x7f44181031b0 2024-03-04T09:31:10.398 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:10.398+0000 7f441e559640 1 --2- 172.21.15.52:0/3102077195 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f44181031b0 0x7f441819ed90 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f4410037040 tx=0x7f4410032f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:31:10.398 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:10.398+0000 7f440effd640 1 -- 172.21.15.52:0/3102077195 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4410041020 con 0x7f44181031b0 2024-03-04T09:31:10.398 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:10.398+0000 7f440effd640 1 -- 172.21.15.52:0/3102077195 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4410034d90 con 0x7f44181031b0 2024-03-04T09:31:10.398 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:10.398+0000 7f441ffbc640 1 -- 172.21.15.52:0/3102077195 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f441819f690 con 0x7f44181031b0 2024-03-04T09:31:10.398 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:10.398+0000 7f440effd640 1 -- 172.21.15.52:0/3102077195 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4410039040 con 0x7f44181031b0 2024-03-04T09:31:10.398 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:10.398+0000 7f441ffbc640 1 -- 172.21.15.52:0/3102077195 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f441819fb80 con 0x7f44181031b0 2024-03-04T09:31:10.399 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:10.399+0000 7f441ffbc640 1 -- 172.21.15.52:0/3102077195 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f44181088c0 con 0x7f44181031b0 2024-03-04T09:31:10.399 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:10.400+0000 7f440effd640 1 -- 172.21.15.52:0/3102077195 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 21) v1 ==== 136568+0+0 (secure 0 0 0) 0x7f4410044020 con 0x7f44181031b0 2024-03-04T09:31:10.399 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:10.401+0000 7f440effd640 1 --2- 172.21.15.52:0/3102077195 >> [v2:172.21.15.52:6800/478031757,v1:172.21.15.52:6801/478031757] conn(0x7f44040a1c30 0x7f44040a40f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:31:10.399 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:10.401+0000 7f441dd58640 1 --2- 172.21.15.52:0/3102077195 >> [v2:172.21.15.52:6800/478031757,v1:172.21.15.52:6801/478031757] conn(0x7f44040a1c30 0x7f44040a40f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:31:10.399 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:10.401+0000 7f440effd640 1 -- 172.21.15.52:0/3102077195 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9459+0+0 (secure 0 0 0) 0x7f44100f0170 con 0x7f44181031b0 2024-03-04T09:31:10.399 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:10.402+0000 7f441dd58640 1 --2- 172.21.15.52:0/3102077195 >> [v2:172.21.15.52:6800/478031757,v1:172.21.15.52:6801/478031757] conn(0x7f44040a1c30 0x7f44040a40f0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f44080058c0 tx=0x7f440800a2d0 comp rx=0 tx=0).ready entity=mgr.15486 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:31:10.402 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:10.405+0000 7f440effd640 1 -- 172.21.15.52:0/3102077195 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f441004b020 con 0x7f44181031b0 2024-03-04T09:31:10.632 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:10.635+0000 7f441ffbc640 1 -- 172.21.15.52:0/3102077195 --> [v2:172.21.15.52:6800/478031757,v1:172.21.15.52:6801/478031757] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) v1 -- 0x7f4418061a10 con 0x7f44040a1c30 2024-03-04T09:31:10.643 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:10.646+0000 7f440effd640 1 -- 172.21.15.52:0/3102077195 <== mgr.15486 v2:172.21.15.52:6800/478031757 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+10 (secure 0 0 0) 0x7f4418061a10 con 0x7f44040a1c30 2024-03-04T09:31:10.643 INFO:teuthology.orchestra.run.smithi052.stdout:/ 1m 6m3h 2024-03-04T09:31:10.645 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:10.648+0000 7f441ffbc640 1 -- 172.21.15.52:0/3102077195 >> [v2:172.21.15.52:6800/478031757,v1:172.21.15.52:6801/478031757] conn(0x7f44040a1c30 msgr2=0x7f44040a40f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:31:10.645 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:10.648+0000 7f441ffbc640 1 --2- 172.21.15.52:0/3102077195 >> [v2:172.21.15.52:6800/478031757,v1:172.21.15.52:6801/478031757] conn(0x7f44040a1c30 0x7f44040a40f0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f44080058c0 tx=0x7f440800a2d0 comp rx=0 tx=0).stop 2024-03-04T09:31:10.645 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:10.648+0000 7f441ffbc640 1 -- 172.21.15.52:0/3102077195 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f44181031b0 msgr2=0x7f441819ed90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:31:10.645 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:10.648+0000 7f441ffbc640 1 --2- 172.21.15.52:0/3102077195 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f44181031b0 0x7f441819ed90 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f4410037040 tx=0x7f4410032f20 comp rx=0 tx=0).stop 2024-03-04T09:31:10.645 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:10.648+0000 7f441ffbc640 1 -- 172.21.15.52:0/3102077195 shutdown_connections 2024-03-04T09:31:10.645 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:10.648+0000 7f441ffbc640 1 --2- 172.21.15.52:0/3102077195 >> [v2:172.21.15.52:6800/478031757,v1:172.21.15.52:6801/478031757] conn(0x7f44040a1c30 0x7f44040a40f0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:31:10.645 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:10.648+0000 7f441ffbc640 1 --2- 172.21.15.52:0/3102077195 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f44181031b0 0x7f441819ed90 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:31:10.645 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:10.648+0000 7f441ffbc640 1 --2- 172.21.15.52:0/3102077195 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f44181027f0 0x7f4418197d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:31:10.645 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:10.648+0000 7f441ffbc640 1 --2- 172.21.15.52:0/3102077195 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4418101600 0x7f44181977e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:31:10.646 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:10.648+0000 7f441ffbc640 1 -- 172.21.15.52:0/3102077195 >> 172.21.15.52:0/3102077195 conn(0x7f44180fcb90 msgr2=0x7f4418105a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:31:10.646 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:10.649+0000 7f441ffbc640 1 -- 172.21.15.52:0/3102077195 shutdown_connections 2024-03-04T09:31:10.646 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:31:10.649+0000 7f441ffbc640 1 -- 172.21.15.52:0/3102077195 wait complete. 2024-03-04T09:31:10.658 DEBUG:teuthology.orchestra.run.smithi052:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > START_TIME' 2024-03-04T09:31:10.686 INFO:teuthology.run_tasks:Running task full_sequential_finally... 2024-03-04T09:31:10.697 INFO:teuthology.run_tasks:Running task workunit... 2024-03-04T09:31:10.708 INFO:tasks.workunit:Pulling workunits from ref 1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd 2024-03-04T09:31:10.708 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-03-04T09:31:10.708 INFO:tasks.workunit:timeout=3h 2024-03-04T09:31:10.708 INFO:tasks.workunit:cleanup=True 2024-03-04T09:31:10.709 DEBUG:teuthology.orchestra.run.smithi052:> stat -- /home/ubuntu/cephtest/mnt.0 2024-03-04T09:31:10.741 INFO:teuthology.orchestra.run.smithi052.stdout: File: /home/ubuntu/cephtest/mnt.0 2024-03-04T09:31:10.741 INFO:teuthology.orchestra.run.smithi052.stdout: Size: 0 Blocks: 0 IO Block: 65536 directory 2024-03-04T09:31:10.741 INFO:teuthology.orchestra.run.smithi052.stdout:Device: 77h/119d Inode: 1099511628283 Links: 2 2024-03-04T09:31:10.742 INFO:teuthology.orchestra.run.smithi052.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-04T09:31:10.742 INFO:teuthology.orchestra.run.smithi052.stdout:Context: system_u:object_r:unlabeled_t:s0 2024-03-04T09:31:10.742 INFO:teuthology.orchestra.run.smithi052.stdout:Access: 2024-03-04 09:29:05.523096248 +0000 2024-03-04T09:31:10.742 INFO:teuthology.orchestra.run.smithi052.stdout:Modify: 2024-03-04 09:29:05.523096248 +0000 2024-03-04T09:31:10.742 INFO:teuthology.orchestra.run.smithi052.stdout:Change: 2024-03-04 09:30:59.344950140 +0000 2024-03-04T09:31:10.742 INFO:teuthology.orchestra.run.smithi052.stdout: Birth: 2024-03-04 09:29:05.523096248 +0000 2024-03-04T09:31:10.742 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2024-03-04T09:31:10.742 DEBUG:teuthology.orchestra.run.smithi052:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2024-03-04T09:31:10.805 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:31:10 smithi052 ceph-mon[32473]: from='client.15510 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T09:31:10.805 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:31:10 smithi052 ceph-mon[32473]: pgmap v3: 129 pgs: 129 active+clean; 1.1 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-04T09:31:10.805 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:31:10 smithi052 ceph-mon[32473]: from='client.15552 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T09:31:10.805 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:31:10 smithi052 ceph-mon[32473]: from='client.15558 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T09:31:10.815 DEBUG:teuthology.orchestra.run.smithi052:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph-ci.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout 1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd 2024-03-04T09:31:10.836 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:31:10 smithi191 ceph-mon[36816]: from='client.15510 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T09:31:10.836 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:31:10 smithi191 ceph-mon[36816]: pgmap v3: 129 pgs: 129 active+clean; 1.1 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-04T09:31:10.836 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:31:10 smithi191 ceph-mon[36816]: from='client.15552 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T09:31:10.836 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:31:10 smithi191 ceph-mon[36816]: from='client.15558 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T09:31:10.871 INFO:tasks.workunit.client.0.smithi052.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-03-04T09:31:10.881 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:31:10 smithi106 ceph-mon[36484]: from='client.15510 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T09:31:10.881 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:31:10 smithi106 ceph-mon[36484]: pgmap v3: 129 pgs: 129 active+clean; 1.1 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-04T09:31:10.881 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:31:10 smithi106 ceph-mon[36484]: from='client.15552 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T09:31:10.881 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:31:10 smithi106 ceph-mon[36484]: from='client.15558 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T09:31:12.084 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:31:11 smithi052 ceph-mon[32473]: pgmap v4: 129 pgs: 129 active+clean; 1.1 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-04T09:31:12.085 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:31:11 smithi052 ceph-mon[32473]: from='client.15564 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T09:31:12.085 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:31:11 smithi052 ceph-mon[32473]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' 2024-03-04T09:31:12.085 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:31:11 smithi052 ceph-mon[32473]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' 2024-03-04T09:31:12.085 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:31:11 smithi052 ceph-mon[32473]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' 2024-03-04T09:31:12.085 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:31:11 smithi052 ceph-mon[32473]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' 2024-03-04T09:31:12.085 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:31:11 smithi052 ceph-mon[32473]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' 2024-03-04T09:31:12.085 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:31:11 smithi052 ceph-mon[32473]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' 2024-03-04T09:31:12.085 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:31:11 smithi052 ceph-mon[32473]: mgrmap e22: x(active, since 3s), standbys: z, y 2024-03-04T09:31:12.086 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:31:11 smithi191 ceph-mon[36816]: pgmap v4: 129 pgs: 129 active+clean; 1.1 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-04T09:31:12.086 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:31:11 smithi191 ceph-mon[36816]: from='client.15564 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T09:31:12.086 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:31:11 smithi191 ceph-mon[36816]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' 2024-03-04T09:31:12.086 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:31:11 smithi191 ceph-mon[36816]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' 2024-03-04T09:31:12.086 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:31:11 smithi191 ceph-mon[36816]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' 2024-03-04T09:31:12.086 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:31:11 smithi191 ceph-mon[36816]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' 2024-03-04T09:31:12.086 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:31:11 smithi191 ceph-mon[36816]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' 2024-03-04T09:31:12.086 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:31:11 smithi191 ceph-mon[36816]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' 2024-03-04T09:31:12.086 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:31:11 smithi191 ceph-mon[36816]: mgrmap e22: x(active, since 3s), standbys: z, y 2024-03-04T09:31:12.131 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:31:11 smithi106 ceph-mon[36484]: pgmap v4: 129 pgs: 129 active+clean; 1.1 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-04T09:31:12.131 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:31:11 smithi106 ceph-mon[36484]: from='client.15564 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T09:31:12.131 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:31:11 smithi106 ceph-mon[36484]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' 2024-03-04T09:31:12.131 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:31:11 smithi106 ceph-mon[36484]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' 2024-03-04T09:31:12.131 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:31:11 smithi106 ceph-mon[36484]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' 2024-03-04T09:31:12.131 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:31:11 smithi106 ceph-mon[36484]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' 2024-03-04T09:31:12.131 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:31:11 smithi106 ceph-mon[36484]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' 2024-03-04T09:31:12.132 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:31:11 smithi106 ceph-mon[36484]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' 2024-03-04T09:31:12.132 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:31:11 smithi106 ceph-mon[36484]: mgrmap e22: x(active, since 3s), standbys: z, y 2024-03-04T09:31:13.133 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:31:12 smithi106 ceph-mon[36484]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' 2024-03-04T09:31:13.133 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:31:12 smithi106 ceph-mon[36484]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' 2024-03-04T09:31:13.133 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:31:12 smithi106 ceph-mon[36484]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-04T09:31:13.133 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:31:12 smithi106 ceph-mon[36484]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-04T09:31:13.133 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:31:12 smithi106 ceph-mon[36484]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-04T09:31:13.133 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:31:12 smithi106 ceph-mon[36484]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-04T09:31:13.133 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:31:12 smithi106 ceph-mon[36484]: Adjusting osd_memory_target on smithi191 to 174.1M 2024-03-04T09:31:13.133 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:31:12 smithi106 ceph-mon[36484]: Unable to set osd_memory_target on smithi191 to 182640640: error parsing value: Value '182640640' is below minimum 939524096 2024-03-04T09:31:13.134 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:31:12 smithi106 ceph-mon[36484]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' 2024-03-04T09:31:13.134 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:31:12 smithi106 ceph-mon[36484]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' 2024-03-04T09:31:13.134 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:31:12 smithi106 ceph-mon[36484]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-04T09:31:13.134 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:31:12 smithi106 ceph-mon[36484]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-04T09:31:13.134 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:31:12 smithi106 ceph-mon[36484]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-04T09:31:13.134 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:31:12 smithi106 ceph-mon[36484]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-04T09:31:13.134 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:31:12 smithi106 ceph-mon[36484]: Adjusting osd_memory_target on smithi052 to 177.2M 2024-03-04T09:31:13.134 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:31:12 smithi106 ceph-mon[36484]: Unable to set osd_memory_target on smithi052 to 185897062: error parsing value: Value '185897062' is below minimum 939524096 2024-03-04T09:31:13.134 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:31:12 smithi106 ceph-mon[36484]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' 2024-03-04T09:31:13.134 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:31:12 smithi106 ceph-mon[36484]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' 2024-03-04T09:31:13.134 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:31:12 smithi106 ceph-mon[36484]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-04T09:31:13.134 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:31:12 smithi106 ceph-mon[36484]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-04T09:31:13.134 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:31:12 smithi106 ceph-mon[36484]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-04T09:31:13.134 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:31:12 smithi106 ceph-mon[36484]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-04T09:31:13.134 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:31:12 smithi106 ceph-mon[36484]: Adjusting osd_memory_target on smithi106 to 174.1M 2024-03-04T09:31:13.135 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:31:12 smithi106 ceph-mon[36484]: Unable to set osd_memory_target on smithi106 to 182633472: error parsing value: Value '182633472' is below minimum 939524096 2024-03-04T09:31:13.135 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:31:12 smithi106 ceph-mon[36484]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T09:31:13.135 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:31:12 smithi106 ceph-mon[36484]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T09:31:13.135 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:31:12 smithi106 ceph-mon[36484]: Updating smithi052:/etc/ceph/ceph.conf 2024-03-04T09:31:13.135 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:31:12 smithi106 ceph-mon[36484]: Updating smithi106:/etc/ceph/ceph.conf 2024-03-04T09:31:13.135 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:31:12 smithi106 ceph-mon[36484]: Updating smithi191:/etc/ceph/ceph.conf 2024-03-04T09:31:13.135 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:31:12 smithi106 ceph-mon[36484]: Updating smithi052:/var/lib/ceph/4e059f08-da08-11ee-95c6-87774f69a715/config/ceph.conf 2024-03-04T09:31:13.135 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:31:12 smithi106 ceph-mon[36484]: Updating smithi106:/var/lib/ceph/4e059f08-da08-11ee-95c6-87774f69a715/config/ceph.conf 2024-03-04T09:31:13.135 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:31:12 smithi106 ceph-mon[36484]: Updating smithi191:/var/lib/ceph/4e059f08-da08-11ee-95c6-87774f69a715/config/ceph.conf 2024-03-04T09:31:13.335 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:31:12 smithi052 ceph-mon[32473]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' 2024-03-04T09:31:13.335 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:31:12 smithi052 ceph-mon[32473]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' 2024-03-04T09:31:13.335 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:31:12 smithi052 ceph-mon[32473]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-04T09:31:13.335 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:31:12 smithi052 ceph-mon[32473]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-04T09:31:13.335 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:31:12 smithi052 ceph-mon[32473]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-04T09:31:13.335 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:31:12 smithi052 ceph-mon[32473]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-04T09:31:13.336 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:31:12 smithi052 ceph-mon[32473]: Adjusting osd_memory_target on smithi191 to 174.1M 2024-03-04T09:31:13.336 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:31:12 smithi052 ceph-mon[32473]: Unable to set osd_memory_target on smithi191 to 182640640: error parsing value: Value '182640640' is below minimum 939524096 2024-03-04T09:31:13.336 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:31:12 smithi052 ceph-mon[32473]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' 2024-03-04T09:31:13.336 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:31:12 smithi052 ceph-mon[32473]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' 2024-03-04T09:31:13.336 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:31:12 smithi052 ceph-mon[32473]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-04T09:31:13.336 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:31:12 smithi052 ceph-mon[32473]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-04T09:31:13.336 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:31:12 smithi052 ceph-mon[32473]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-04T09:31:13.336 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:31:12 smithi052 ceph-mon[32473]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-04T09:31:13.336 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:31:12 smithi052 ceph-mon[32473]: Adjusting osd_memory_target on smithi052 to 177.2M 2024-03-04T09:31:13.336 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:31:12 smithi052 ceph-mon[32473]: Unable to set osd_memory_target on smithi052 to 185897062: error parsing value: Value '185897062' is below minimum 939524096 2024-03-04T09:31:13.336 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:31:12 smithi052 ceph-mon[32473]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' 2024-03-04T09:31:13.336 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:31:12 smithi052 ceph-mon[32473]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' 2024-03-04T09:31:13.336 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:31:12 smithi052 ceph-mon[32473]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-04T09:31:13.337 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:31:12 smithi052 ceph-mon[32473]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-04T09:31:13.337 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:31:12 smithi052 ceph-mon[32473]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-04T09:31:13.337 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:31:12 smithi052 ceph-mon[32473]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-04T09:31:13.337 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:31:12 smithi052 ceph-mon[32473]: Adjusting osd_memory_target on smithi106 to 174.1M 2024-03-04T09:31:13.337 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:31:12 smithi052 ceph-mon[32473]: Unable to set osd_memory_target on smithi106 to 182633472: error parsing value: Value '182633472' is below minimum 939524096 2024-03-04T09:31:13.337 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:31:12 smithi052 ceph-mon[32473]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T09:31:13.337 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:31:12 smithi052 ceph-mon[32473]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T09:31:13.337 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:31:12 smithi052 ceph-mon[32473]: Updating smithi052:/etc/ceph/ceph.conf 2024-03-04T09:31:13.337 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:31:12 smithi052 ceph-mon[32473]: Updating smithi106:/etc/ceph/ceph.conf 2024-03-04T09:31:13.337 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:31:12 smithi052 ceph-mon[32473]: Updating smithi191:/etc/ceph/ceph.conf 2024-03-04T09:31:13.337 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:31:12 smithi052 ceph-mon[32473]: Updating smithi052:/var/lib/ceph/4e059f08-da08-11ee-95c6-87774f69a715/config/ceph.conf 2024-03-04T09:31:13.337 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:31:12 smithi052 ceph-mon[32473]: Updating smithi106:/var/lib/ceph/4e059f08-da08-11ee-95c6-87774f69a715/config/ceph.conf 2024-03-04T09:31:13.337 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:31:12 smithi052 ceph-mon[32473]: Updating smithi191:/var/lib/ceph/4e059f08-da08-11ee-95c6-87774f69a715/config/ceph.conf 2024-03-04T09:31:13.338 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:31:12 smithi191 ceph-mon[36816]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' 2024-03-04T09:31:13.338 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:31:12 smithi191 ceph-mon[36816]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' 2024-03-04T09:31:13.338 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:31:12 smithi191 ceph-mon[36816]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-04T09:31:13.338 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:31:12 smithi191 ceph-mon[36816]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-04T09:31:13.338 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:31:12 smithi191 ceph-mon[36816]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-04T09:31:13.338 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:31:12 smithi191 ceph-mon[36816]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-04T09:31:13.338 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:31:12 smithi191 ceph-mon[36816]: Adjusting osd_memory_target on smithi191 to 174.1M 2024-03-04T09:31:13.338 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:31:12 smithi191 ceph-mon[36816]: Unable to set osd_memory_target on smithi191 to 182640640: error parsing value: Value '182640640' is below minimum 939524096 2024-03-04T09:31:13.339 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:31:12 smithi191 ceph-mon[36816]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' 2024-03-04T09:31:13.339 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:31:12 smithi191 ceph-mon[36816]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' 2024-03-04T09:31:13.339 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:31:12 smithi191 ceph-mon[36816]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-04T09:31:13.339 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:31:12 smithi191 ceph-mon[36816]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-04T09:31:13.339 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:31:12 smithi191 ceph-mon[36816]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-04T09:31:13.339 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:31:12 smithi191 ceph-mon[36816]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-04T09:31:13.339 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:31:12 smithi191 ceph-mon[36816]: Adjusting osd_memory_target on smithi052 to 177.2M 2024-03-04T09:31:13.340 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:31:12 smithi191 ceph-mon[36816]: Unable to set osd_memory_target on smithi052 to 185897062: error parsing value: Value '185897062' is below minimum 939524096 2024-03-04T09:31:13.340 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:31:12 smithi191 ceph-mon[36816]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' 2024-03-04T09:31:13.340 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:31:12 smithi191 ceph-mon[36816]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' 2024-03-04T09:31:13.340 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:31:12 smithi191 ceph-mon[36816]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-04T09:31:13.340 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:31:12 smithi191 ceph-mon[36816]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-04T09:31:13.340 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:31:12 smithi191 ceph-mon[36816]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-04T09:31:13.340 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:31:12 smithi191 ceph-mon[36816]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-04T09:31:13.340 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:31:12 smithi191 ceph-mon[36816]: Adjusting osd_memory_target on smithi106 to 174.1M 2024-03-04T09:31:13.340 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:31:12 smithi191 ceph-mon[36816]: Unable to set osd_memory_target on smithi106 to 182633472: error parsing value: Value '182633472' is below minimum 939524096 2024-03-04T09:31:13.340 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:31:12 smithi191 ceph-mon[36816]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T09:31:13.341 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:31:12 smithi191 ceph-mon[36816]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T09:31:13.341 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:31:12 smithi191 ceph-mon[36816]: Updating smithi052:/etc/ceph/ceph.conf 2024-03-04T09:31:13.341 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:31:12 smithi191 ceph-mon[36816]: Updating smithi106:/etc/ceph/ceph.conf 2024-03-04T09:31:13.341 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:31:12 smithi191 ceph-mon[36816]: Updating smithi191:/etc/ceph/ceph.conf 2024-03-04T09:31:13.341 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:31:12 smithi191 ceph-mon[36816]: Updating smithi052:/var/lib/ceph/4e059f08-da08-11ee-95c6-87774f69a715/config/ceph.conf 2024-03-04T09:31:13.341 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:31:12 smithi191 ceph-mon[36816]: Updating smithi106:/var/lib/ceph/4e059f08-da08-11ee-95c6-87774f69a715/config/ceph.conf 2024-03-04T09:31:13.341 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:31:12 smithi191 ceph-mon[36816]: Updating smithi191:/var/lib/ceph/4e059f08-da08-11ee-95c6-87774f69a715/config/ceph.conf 2024-03-04T09:31:14.335 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:31:14 smithi052 ceph-mon[32473]: pgmap v5: 129 pgs: 129 active+clean; 1.1 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-04T09:31:14.335 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:31:14 smithi052 ceph-mon[32473]: Updating smithi052:/etc/ceph/ceph.client.admin.keyring 2024-03-04T09:31:14.335 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:31:14 smithi052 ceph-mon[32473]: Updating smithi106:/etc/ceph/ceph.client.admin.keyring 2024-03-04T09:31:14.335 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:31:14 smithi052 ceph-mon[32473]: Updating smithi191:/etc/ceph/ceph.client.admin.keyring 2024-03-04T09:31:14.335 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:31:14 smithi052 ceph-mon[32473]: Updating smithi052:/var/lib/ceph/4e059f08-da08-11ee-95c6-87774f69a715/config/ceph.client.admin.keyring 2024-03-04T09:31:14.335 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:31:14 smithi052 ceph-mon[32473]: Updating smithi106:/var/lib/ceph/4e059f08-da08-11ee-95c6-87774f69a715/config/ceph.client.admin.keyring 2024-03-04T09:31:14.335 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:31:14 smithi052 ceph-mon[32473]: Updating smithi191:/var/lib/ceph/4e059f08-da08-11ee-95c6-87774f69a715/config/ceph.client.admin.keyring 2024-03-04T09:31:14.335 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:31:14 smithi052 ceph-mon[32473]: mgrmap e23: x(active, since 4s), standbys: z, y 2024-03-04T09:31:14.336 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:31:14 smithi052 ceph-mon[32473]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' 2024-03-04T09:31:14.336 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:31:14 smithi052 ceph-mon[32473]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' 2024-03-04T09:31:14.336 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:31:14 smithi052 ceph-mon[32473]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' 2024-03-04T09:31:14.336 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:31:14 smithi052 ceph-mon[32473]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' 2024-03-04T09:31:14.336 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:31:14 smithi052 ceph-mon[32473]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' 2024-03-04T09:31:14.336 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:31:14 smithi052 ceph-mon[32473]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' 2024-03-04T09:31:14.336 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:31:14 smithi052 ceph-mon[32473]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' 2024-03-04T09:31:14.336 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:31:14 smithi052 ceph-mon[32473]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T09:31:14.336 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:31:14 smithi052 ceph-mon[32473]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T09:31:14.336 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:31:14 smithi052 ceph-mon[32473]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T09:31:14.336 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:31:14 smithi052 ceph-mon[32473]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' 2024-03-04T09:31:14.337 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:31:14 smithi191 ceph-mon[36816]: pgmap v5: 129 pgs: 129 active+clean; 1.1 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-04T09:31:14.337 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:31:14 smithi191 ceph-mon[36816]: Updating smithi052:/etc/ceph/ceph.client.admin.keyring 2024-03-04T09:31:14.337 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:31:14 smithi191 ceph-mon[36816]: Updating smithi106:/etc/ceph/ceph.client.admin.keyring 2024-03-04T09:31:14.337 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:31:14 smithi191 ceph-mon[36816]: Updating smithi191:/etc/ceph/ceph.client.admin.keyring 2024-03-04T09:31:14.337 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:31:14 smithi191 ceph-mon[36816]: Updating smithi052:/var/lib/ceph/4e059f08-da08-11ee-95c6-87774f69a715/config/ceph.client.admin.keyring 2024-03-04T09:31:14.337 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:31:14 smithi191 ceph-mon[36816]: Updating smithi106:/var/lib/ceph/4e059f08-da08-11ee-95c6-87774f69a715/config/ceph.client.admin.keyring 2024-03-04T09:31:14.337 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:31:14 smithi191 ceph-mon[36816]: Updating smithi191:/var/lib/ceph/4e059f08-da08-11ee-95c6-87774f69a715/config/ceph.client.admin.keyring 2024-03-04T09:31:14.337 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:31:14 smithi191 ceph-mon[36816]: mgrmap e23: x(active, since 4s), standbys: z, y 2024-03-04T09:31:14.337 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:31:14 smithi191 ceph-mon[36816]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' 2024-03-04T09:31:14.337 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:31:14 smithi191 ceph-mon[36816]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' 2024-03-04T09:31:14.337 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:31:14 smithi191 ceph-mon[36816]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' 2024-03-04T09:31:14.338 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:31:14 smithi191 ceph-mon[36816]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' 2024-03-04T09:31:14.338 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:31:14 smithi191 ceph-mon[36816]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' 2024-03-04T09:31:14.338 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:31:14 smithi191 ceph-mon[36816]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' 2024-03-04T09:31:14.338 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:31:14 smithi191 ceph-mon[36816]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' 2024-03-04T09:31:14.338 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:31:14 smithi191 ceph-mon[36816]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T09:31:14.338 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:31:14 smithi191 ceph-mon[36816]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T09:31:14.338 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:31:14 smithi191 ceph-mon[36816]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T09:31:14.338 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:31:14 smithi191 ceph-mon[36816]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' 2024-03-04T09:31:14.381 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:31:14 smithi106 ceph-mon[36484]: pgmap v5: 129 pgs: 129 active+clean; 1.1 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-04T09:31:14.381 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:31:14 smithi106 ceph-mon[36484]: Updating smithi052:/etc/ceph/ceph.client.admin.keyring 2024-03-04T09:31:14.381 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:31:14 smithi106 ceph-mon[36484]: Updating smithi106:/etc/ceph/ceph.client.admin.keyring 2024-03-04T09:31:14.381 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:31:14 smithi106 ceph-mon[36484]: Updating smithi191:/etc/ceph/ceph.client.admin.keyring 2024-03-04T09:31:14.381 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:31:14 smithi106 ceph-mon[36484]: Updating smithi052:/var/lib/ceph/4e059f08-da08-11ee-95c6-87774f69a715/config/ceph.client.admin.keyring 2024-03-04T09:31:14.381 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:31:14 smithi106 ceph-mon[36484]: Updating smithi106:/var/lib/ceph/4e059f08-da08-11ee-95c6-87774f69a715/config/ceph.client.admin.keyring 2024-03-04T09:31:14.382 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:31:14 smithi106 ceph-mon[36484]: Updating smithi191:/var/lib/ceph/4e059f08-da08-11ee-95c6-87774f69a715/config/ceph.client.admin.keyring 2024-03-04T09:31:14.382 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:31:14 smithi106 ceph-mon[36484]: mgrmap e23: x(active, since 4s), standbys: z, y 2024-03-04T09:31:14.382 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:31:14 smithi106 ceph-mon[36484]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' 2024-03-04T09:31:14.382 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:31:14 smithi106 ceph-mon[36484]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' 2024-03-04T09:31:14.382 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:31:14 smithi106 ceph-mon[36484]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' 2024-03-04T09:31:14.382 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:31:14 smithi106 ceph-mon[36484]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' 2024-03-04T09:31:14.382 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:31:14 smithi106 ceph-mon[36484]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' 2024-03-04T09:31:14.382 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:31:14 smithi106 ceph-mon[36484]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' 2024-03-04T09:31:14.382 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:31:14 smithi106 ceph-mon[36484]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' 2024-03-04T09:31:14.382 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:31:14 smithi106 ceph-mon[36484]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T09:31:14.382 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:31:14 smithi106 ceph-mon[36484]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T09:31:14.382 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:31:14 smithi106 ceph-mon[36484]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T09:31:14.382 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:31:14 smithi106 ceph-mon[36484]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' 2024-03-04T09:31:16.335 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:31:16 smithi052 ceph-mon[32473]: pgmap v6: 129 pgs: 129 active+clean; 1.1 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-04T09:31:16.657 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:31:16 smithi106 ceph-mon[36484]: pgmap v6: 129 pgs: 129 active+clean; 1.1 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-04T09:31:16.657 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:31:16 smithi191 ceph-mon[36816]: pgmap v6: 129 pgs: 129 active+clean; 1.1 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-04T09:31:18.335 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:31:18 smithi052 ceph-mon[32473]: pgmap v7: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 132 KiB/s wr, 76 op/s 2024-03-04T09:31:18.336 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:31:18 smithi191 ceph-mon[36816]: pgmap v7: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 132 KiB/s wr, 76 op/s 2024-03-04T09:31:18.381 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:31:18 smithi106 ceph-mon[36484]: pgmap v7: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 132 KiB/s wr, 76 op/s 2024-03-04T09:31:20.335 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:31:20 smithi052 ceph-mon[32473]: pgmap v8: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 103 KiB/s wr, 59 op/s 2024-03-04T09:31:20.336 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:31:20 smithi191 ceph-mon[36816]: pgmap v8: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 103 KiB/s wr, 59 op/s 2024-03-04T09:31:20.381 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:31:20 smithi106 ceph-mon[36484]: pgmap v8: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 103 KiB/s wr, 59 op/s 2024-03-04T09:31:22.335 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:31:22 smithi052 ceph-mon[32473]: pgmap v9: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 84 KiB/s wr, 51 op/s 2024-03-04T09:31:22.336 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:31:22 smithi191 ceph-mon[36816]: pgmap v9: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 84 KiB/s wr, 51 op/s 2024-03-04T09:31:22.381 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:31:22 smithi106 ceph-mon[36484]: pgmap v9: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 84 KiB/s wr, 51 op/s 2024-03-04T09:31:24.335 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:31:24 smithi052 ceph-mon[32473]: pgmap v10: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 77 KiB/s wr, 47 op/s 2024-03-04T09:31:24.336 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:31:24 smithi191 ceph-mon[36816]: pgmap v10: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 77 KiB/s wr, 47 op/s 2024-03-04T09:31:24.381 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:31:24 smithi106 ceph-mon[36484]: pgmap v10: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 77 KiB/s wr, 47 op/s 2024-03-04T09:31:26.335 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:31:26 smithi052 ceph-mon[32473]: pgmap v11: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 77 KiB/s wr, 47 op/s 2024-03-04T09:31:26.336 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:31:26 smithi191 ceph-mon[36816]: pgmap v11: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 77 KiB/s wr, 47 op/s 2024-03-04T09:31:26.381 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:31:26 smithi106 ceph-mon[36484]: pgmap v11: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 77 KiB/s wr, 47 op/s 2024-03-04T09:31:28.335 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:31:28 smithi052 ceph-mon[32473]: pgmap v12: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 77 KiB/s wr, 49 op/s 2024-03-04T09:31:28.336 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:31:28 smithi191 ceph-mon[36816]: pgmap v12: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 77 KiB/s wr, 49 op/s 2024-03-04T09:31:28.381 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:31:28 smithi106 ceph-mon[36484]: pgmap v12: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 77 KiB/s wr, 49 op/s 2024-03-04T09:31:30.335 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:31:30 smithi052 ceph-mon[32473]: pgmap v13: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-04T09:31:30.336 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:31:30 smithi191 ceph-mon[36816]: pgmap v13: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-04T09:31:30.381 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:31:30 smithi106 ceph-mon[36484]: pgmap v13: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-04T09:31:32.335 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:31:32 smithi052 ceph-mon[32473]: pgmap v14: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-04T09:31:32.336 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:31:32 smithi191 ceph-mon[36816]: pgmap v14: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-04T09:31:32.381 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:31:32 smithi106 ceph-mon[36484]: pgmap v14: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-04T09:31:34.335 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:31:34 smithi052 ceph-mon[32473]: pgmap v15: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-04T09:31:34.336 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:31:34 smithi191 ceph-mon[36816]: pgmap v15: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-04T09:31:34.381 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:31:34 smithi106 ceph-mon[36484]: pgmap v15: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-04T09:31:36.335 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:31:36 smithi052 ceph-mon[32473]: pgmap v16: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-04T09:31:36.336 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:31:36 smithi191 ceph-mon[36816]: pgmap v16: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-04T09:31:36.381 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:31:36 smithi106 ceph-mon[36484]: pgmap v16: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-04T09:31:38.335 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:31:38 smithi052 ceph-mon[32473]: pgmap v17: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-04T09:31:38.336 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:31:38 smithi191 ceph-mon[36816]: pgmap v17: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-04T09:31:38.381 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:31:38 smithi106 ceph-mon[36484]: pgmap v17: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-04T09:31:40.335 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:31:40 smithi052 ceph-mon[32473]: pgmap v18: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-04T09:31:40.336 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:31:40 smithi191 ceph-mon[36816]: pgmap v18: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-04T09:31:40.381 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:31:40 smithi106 ceph-mon[36484]: pgmap v18: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-04T09:31:42.335 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:31:42 smithi052 ceph-mon[32473]: pgmap v19: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-04T09:31:42.336 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:31:42 smithi191 ceph-mon[36816]: pgmap v19: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-04T09:31:42.381 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:31:42 smithi106 ceph-mon[36484]: pgmap v19: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-04T09:31:44.335 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:31:44 smithi052 ceph-mon[32473]: pgmap v20: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-04T09:31:44.336 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:31:44 smithi191 ceph-mon[36816]: pgmap v20: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-04T09:31:44.381 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:31:44 smithi106 ceph-mon[36484]: pgmap v20: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-04T09:31:46.335 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:31:46 smithi052 ceph-mon[32473]: pgmap v21: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-04T09:31:46.336 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:31:46 smithi191 ceph-mon[36816]: pgmap v21: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-04T09:31:46.381 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:31:46 smithi106 ceph-mon[36484]: pgmap v21: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-04T09:31:48.335 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:31:48 smithi052 ceph-mon[32473]: pgmap v22: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 682 B/s wr, 7 op/s 2024-03-04T09:31:48.336 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:31:48 smithi191 ceph-mon[36816]: pgmap v22: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 682 B/s wr, 7 op/s 2024-03-04T09:31:48.381 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:31:48 smithi106 ceph-mon[36484]: pgmap v22: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 682 B/s wr, 7 op/s 2024-03-04T09:31:50.335 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:31:50 smithi052 ceph-mon[32473]: pgmap v23: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 682 B/s wr, 5 op/s 2024-03-04T09:31:50.336 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:31:50 smithi191 ceph-mon[36816]: pgmap v23: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 682 B/s wr, 5 op/s 2024-03-04T09:31:50.381 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:31:50 smithi106 ceph-mon[36484]: pgmap v23: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 682 B/s wr, 5 op/s 2024-03-04T09:31:52.334 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:31:52 smithi052 ceph-mon[32473]: pgmap v24: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 682 B/s wr, 7 op/s 2024-03-04T09:31:52.336 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:31:52 smithi191 ceph-mon[36816]: pgmap v24: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 682 B/s wr, 7 op/s 2024-03-04T09:31:52.381 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:31:52 smithi106 ceph-mon[36484]: pgmap v24: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 682 B/s wr, 7 op/s 2024-03-04T09:31:54.334 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:31:54 smithi052 ceph-mon[32473]: pgmap v25: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 682 B/s wr, 5 op/s 2024-03-04T09:31:54.336 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:31:54 smithi191 ceph-mon[36816]: pgmap v25: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 682 B/s wr, 5 op/s 2024-03-04T09:31:54.381 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:31:54 smithi106 ceph-mon[36484]: pgmap v25: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 682 B/s wr, 5 op/s 2024-03-04T09:31:56.334 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:31:56 smithi052 ceph-mon[32473]: pgmap v26: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 682 B/s wr, 6 op/s 2024-03-04T09:31:56.336 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:31:56 smithi191 ceph-mon[36816]: pgmap v26: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 682 B/s wr, 6 op/s 2024-03-04T09:31:56.381 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:31:56 smithi106 ceph-mon[36484]: pgmap v26: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 682 B/s wr, 6 op/s 2024-03-04T09:31:58.334 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:31:58 smithi052 ceph-mon[32473]: pgmap v27: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 682 B/s wr, 7 op/s 2024-03-04T09:31:58.336 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:31:58 smithi191 ceph-mon[36816]: pgmap v27: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 682 B/s wr, 7 op/s 2024-03-04T09:31:58.381 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:31:58 smithi106 ceph-mon[36484]: pgmap v27: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 682 B/s wr, 7 op/s 2024-03-04T09:32:00.334 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:32:00 smithi052 ceph-mon[32473]: pgmap v28: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-04T09:32:00.336 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:32:00 smithi191 ceph-mon[36816]: pgmap v28: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-04T09:32:00.381 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:32:00 smithi106 ceph-mon[36484]: pgmap v28: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-04T09:32:01.335 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Mar 04 09:32:00 smithi052 ceph-4e059f08-da08-11ee-95c6-87774f69a715-mgr-x[32705]: 2024-03-04T09:32:00.938+0000 7f5f42f26700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-04T09:32:01.335 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Mar 04 09:32:00 smithi052 ceph-4e059f08-da08-11ee-95c6-87774f69a715-mgr-x[32705]: 2024-03-04T09:32:00.938+0000 7f5f42f26700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-04T09:32:01.335 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Mar 04 09:32:00 smithi052 ceph-4e059f08-da08-11ee-95c6-87774f69a715-mgr-x[32705]: 2024-03-04T09:32:00.938+0000 7f5f42f26700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-04T09:32:01.335 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Mar 04 09:32:00 smithi052 ceph-4e059f08-da08-11ee-95c6-87774f69a715-mgr-x[32705]: 2024-03-04T09:32:00.938+0000 7f5f42f26700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-04T09:32:01.335 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Mar 04 09:32:00 smithi052 ceph-4e059f08-da08-11ee-95c6-87774f69a715-mgr-x[32705]: 2024-03-04T09:32:00.938+0000 7f5f42f26700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-04T09:32:02.334 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:32:02 smithi052 ceph-mon[32473]: pgmap v29: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-04T09:32:02.336 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:32:02 smithi191 ceph-mon[36816]: pgmap v29: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-04T09:32:02.381 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:32:02 smithi106 ceph-mon[36484]: pgmap v29: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-04T09:32:04.335 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:32:04 smithi052 ceph-mon[32473]: pgmap v30: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-04T09:32:04.335 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:32:04 smithi052 ceph-mon[32473]: mgrmap e24: x(active, since 54s), standbys: z, y 2024-03-04T09:32:04.336 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:32:04 smithi191 ceph-mon[36816]: pgmap v30: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-04T09:32:04.336 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:32:04 smithi191 ceph-mon[36816]: mgrmap e24: x(active, since 54s), standbys: z, y 2024-03-04T09:32:04.381 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:32:04 smithi106 ceph-mon[36484]: pgmap v30: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-04T09:32:04.381 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:32:04 smithi106 ceph-mon[36484]: mgrmap e24: x(active, since 54s), standbys: z, y 2024-03-04T09:32:06.381 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:32:06 smithi106 ceph-mon[36484]: pgmap v31: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 12 KiB/s wr, 9 op/s 2024-03-04T09:32:06.585 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:32:06 smithi052 ceph-mon[32473]: pgmap v31: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 12 KiB/s wr, 9 op/s 2024-03-04T09:32:06.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:32:06 smithi191 ceph-mon[36816]: pgmap v31: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 12 KiB/s wr, 9 op/s 2024-03-04T09:32:08.381 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:32:08 smithi106 ceph-mon[36484]: pgmap v32: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 21 op/s 2024-03-04T09:32:08.386 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:32:08 smithi052 ceph-mon[32473]: pgmap v32: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 21 op/s 2024-03-04T09:32:08.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:32:08 smithi191 ceph-mon[36816]: pgmap v32: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 21 op/s 2024-03-04T09:32:10.381 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:32:10 smithi106 ceph-mon[36484]: pgmap v33: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-03-04T09:32:10.584 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:32:10 smithi052 ceph-mon[32473]: pgmap v33: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-03-04T09:32:10.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:32:10 smithi191 ceph-mon[36816]: pgmap v33: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-03-04T09:32:12.381 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:32:12 smithi106 ceph-mon[36484]: pgmap v34: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-03-04T09:32:12.584 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:32:12 smithi052 ceph-mon[32473]: pgmap v34: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-03-04T09:32:12.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:32:12 smithi191 ceph-mon[36816]: pgmap v34: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-03-04T09:32:14.381 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:32:14 smithi106 ceph-mon[36484]: pgmap v35: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-03-04T09:32:14.381 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:32:14 smithi106 ceph-mon[36484]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T09:32:14.381 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:32:14 smithi106 ceph-mon[36484]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' 2024-03-04T09:32:14.381 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:32:14 smithi106 ceph-mon[36484]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' 2024-03-04T09:32:14.382 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:32:14 smithi106 ceph-mon[36484]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' 2024-03-04T09:32:14.382 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:32:14 smithi106 ceph-mon[36484]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' 2024-03-04T09:32:14.382 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:32:14 smithi106 ceph-mon[36484]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' 2024-03-04T09:32:14.382 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:32:14 smithi106 ceph-mon[36484]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' 2024-03-04T09:32:14.585 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:32:14 smithi052 ceph-mon[32473]: pgmap v35: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-03-04T09:32:14.585 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:32:14 smithi052 ceph-mon[32473]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T09:32:14.585 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:32:14 smithi052 ceph-mon[32473]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' 2024-03-04T09:32:14.585 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:32:14 smithi052 ceph-mon[32473]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' 2024-03-04T09:32:14.585 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:32:14 smithi052 ceph-mon[32473]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' 2024-03-04T09:32:14.585 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:32:14 smithi052 ceph-mon[32473]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' 2024-03-04T09:32:14.585 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:32:14 smithi052 ceph-mon[32473]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' 2024-03-04T09:32:14.585 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:32:14 smithi052 ceph-mon[32473]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' 2024-03-04T09:32:14.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:32:14 smithi191 ceph-mon[36816]: pgmap v35: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-03-04T09:32:14.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:32:14 smithi191 ceph-mon[36816]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T09:32:14.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:32:14 smithi191 ceph-mon[36816]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' 2024-03-04T09:32:14.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:32:14 smithi191 ceph-mon[36816]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' 2024-03-04T09:32:14.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:32:14 smithi191 ceph-mon[36816]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' 2024-03-04T09:32:14.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:32:14 smithi191 ceph-mon[36816]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' 2024-03-04T09:32:14.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:32:14 smithi191 ceph-mon[36816]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' 2024-03-04T09:32:14.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:32:14 smithi191 ceph-mon[36816]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' 2024-03-04T09:32:15.381 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:32:15 smithi106 ceph-mon[36484]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T09:32:15.381 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:32:15 smithi106 ceph-mon[36484]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T09:32:15.381 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:32:15 smithi106 ceph-mon[36484]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' 2024-03-04T09:32:15.584 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:32:15 smithi052 ceph-mon[32473]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T09:32:15.585 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:32:15 smithi052 ceph-mon[32473]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T09:32:15.585 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:32:15 smithi052 ceph-mon[32473]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' 2024-03-04T09:32:15.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:32:15 smithi191 ceph-mon[36816]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T09:32:15.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:32:15 smithi191 ceph-mon[36816]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T09:32:15.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:32:15 smithi191 ceph-mon[36816]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' 2024-03-04T09:32:16.381 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:32:16 smithi106 ceph-mon[36484]: pgmap v36: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-03-04T09:32:16.584 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:32:16 smithi052 ceph-mon[32473]: pgmap v36: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-03-04T09:32:16.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:32:16 smithi191 ceph-mon[36816]: pgmap v36: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-03-04T09:32:18.381 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:32:18 smithi106 ceph-mon[36484]: pgmap v37: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 11 KiB/s wr, 18 op/s 2024-03-04T09:32:18.584 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:32:18 smithi052 ceph-mon[32473]: pgmap v37: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 11 KiB/s wr, 18 op/s 2024-03-04T09:32:18.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:32:18 smithi191 ceph-mon[36816]: pgmap v37: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 11 KiB/s wr, 18 op/s 2024-03-04T09:32:20.381 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:32:20 smithi106 ceph-mon[36484]: pgmap v38: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-03-04T09:32:20.584 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:32:20 smithi052 ceph-mon[32473]: pgmap v38: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-03-04T09:32:20.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:32:20 smithi191 ceph-mon[36816]: pgmap v38: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-03-04T09:32:22.381 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:32:22 smithi106 ceph-mon[36484]: pgmap v39: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 85 B/s wr, 7 op/s 2024-03-04T09:32:22.584 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:32:22 smithi052 ceph-mon[32473]: pgmap v39: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 85 B/s wr, 7 op/s 2024-03-04T09:32:22.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:32:22 smithi191 ceph-mon[36816]: pgmap v39: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 85 B/s wr, 7 op/s 2024-03-04T09:32:24.381 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:32:24 smithi106 ceph-mon[36484]: pgmap v40: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-04T09:32:24.389 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:32:24 smithi191 ceph-mon[36816]: pgmap v40: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-04T09:32:24.584 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:32:24 smithi052 ceph-mon[32473]: pgmap v40: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-04T09:32:26.381 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:32:26 smithi106 ceph-mon[36484]: pgmap v41: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-04T09:32:26.584 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:32:26 smithi052 ceph-mon[32473]: pgmap v41: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-04T09:32:26.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:32:26 smithi191 ceph-mon[36816]: pgmap v41: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-04T09:32:28.188 INFO:tasks.workunit.client.0.smithi052.stderr:Updating files: 95% (11684/12271) Updating files: 96% (11781/12271) Updating files: 97% (11903/12271) Updating files: 98% (12026/12271) Updating files: 99% (12149/12271) Updating files: 100% (12271/12271) Updating files: 100% (12271/12271), done. 2024-03-04T09:32:28.381 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:32:28 smithi106 ceph-mon[36484]: pgmap v42: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-04T09:32:28.387 INFO:tasks.workunit.client.0.smithi052.stderr:Note: switching to '1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd'. 2024-03-04T09:32:28.387 INFO:tasks.workunit.client.0.smithi052.stderr: 2024-03-04T09:32:28.387 INFO:tasks.workunit.client.0.smithi052.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-03-04T09:32:28.388 INFO:tasks.workunit.client.0.smithi052.stderr:changes and commit them, and you can discard any commits you make in this 2024-03-04T09:32:28.388 INFO:tasks.workunit.client.0.smithi052.stderr:state without impacting any branches by switching back to a branch. 2024-03-04T09:32:28.388 INFO:tasks.workunit.client.0.smithi052.stderr: 2024-03-04T09:32:28.388 INFO:tasks.workunit.client.0.smithi052.stderr:If you want to create a new branch to retain commits you create, you may 2024-03-04T09:32:28.388 INFO:tasks.workunit.client.0.smithi052.stderr:do so (now or later) by using -c with the switch command. Example: 2024-03-04T09:32:28.388 INFO:tasks.workunit.client.0.smithi052.stderr: 2024-03-04T09:32:28.388 INFO:tasks.workunit.client.0.smithi052.stderr: git switch -c 2024-03-04T09:32:28.388 INFO:tasks.workunit.client.0.smithi052.stderr: 2024-03-04T09:32:28.388 INFO:tasks.workunit.client.0.smithi052.stderr:Or undo this operation with: 2024-03-04T09:32:28.388 INFO:tasks.workunit.client.0.smithi052.stderr: 2024-03-04T09:32:28.388 INFO:tasks.workunit.client.0.smithi052.stderr: git switch - 2024-03-04T09:32:28.388 INFO:tasks.workunit.client.0.smithi052.stderr: 2024-03-04T09:32:28.389 INFO:tasks.workunit.client.0.smithi052.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-03-04T09:32:28.389 INFO:tasks.workunit.client.0.smithi052.stderr: 2024-03-04T09:32:28.389 INFO:tasks.workunit.client.0.smithi052.stderr:HEAD is now at 1bc250c7129 mds/quiesce-db: incorporate review comments 2024-03-04T09:32:28.390 DEBUG:teuthology.orchestra.run.smithi052:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-03-04T09:32:28.451 INFO:tasks.workunit.client.0.smithi052.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-03-04T09:32:28.453 INFO:tasks.workunit.client.0.smithi052.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-03-04T09:32:28.453 INFO:tasks.workunit.client.0.smithi052.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-03-04T09:32:28.584 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:32:28 smithi052 ceph-mon[32473]: pgmap v42: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-04T09:32:28.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:32:28 smithi191 ceph-mon[36816]: pgmap v42: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-04T09:32:29.144 INFO:tasks.workunit.client.0.smithi052.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-03-04T09:32:29.207 INFO:tasks.workunit.client.0.smithi052.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-03-04T09:32:29.247 INFO:tasks.workunit.client.0.smithi052.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-03-04T09:32:29.249 INFO:tasks.workunit.client.0.smithi052.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-03-04T09:32:29.250 INFO:tasks.workunit.client.0.smithi052.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-03-04T09:32:29.287 INFO:tasks.workunit.client.0.smithi052.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-03-04T09:32:29.290 DEBUG:teuthology.orchestra.run.smithi052:> set -ex 2024-03-04T09:32:29.291 DEBUG:teuthology.orchestra.run.smithi052:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-03-04T09:32:29.346 INFO:tasks.workunit:Running workunits matching suites/fsx.sh on client.0... 2024-03-04T09:32:29.347 INFO:tasks.workunit:Running workunit suites/fsx.sh... 2024-03-04T09:32:29.347 DEBUG:teuthology.orchestra.run.smithi052:workunit test suites/fsx.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/fsx.sh 2024-03-04T09:32:29.406 INFO:tasks.workunit.client.0.smithi052.stderr:+ set -e 2024-03-04T09:32:29.406 INFO:tasks.workunit.client.0.smithi052.stderr:+ git clone https://git.ceph.com/xfstests-dev.git 2024-03-04T09:32:29.408 INFO:tasks.workunit.client.0.smithi052.stderr:Cloning into 'xfstests-dev'... 2024-03-04T09:32:30.381 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:32:30 smithi106 ceph-mon[36484]: pgmap v43: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-04T09:32:30.584 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:32:30 smithi052 ceph-mon[32473]: pgmap v43: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-04T09:32:30.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:32:30 smithi191 ceph-mon[36816]: pgmap v43: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-04T09:32:32.381 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:32:32 smithi106 ceph-mon[36484]: pgmap v44: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6.4 KiB/s wr, 7 op/s 2024-03-04T09:32:32.584 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:32:32 smithi052 ceph-mon[32473]: pgmap v44: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6.4 KiB/s wr, 7 op/s 2024-03-04T09:32:32.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:32:32 smithi191 ceph-mon[36816]: pgmap v44: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6.4 KiB/s wr, 7 op/s 2024-03-04T09:32:34.381 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:32:34 smithi106 ceph-mon[36484]: pgmap v45: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 6.4 KiB/s wr, 5 op/s 2024-03-04T09:32:34.584 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:32:34 smithi052 ceph-mon[32473]: pgmap v45: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 6.4 KiB/s wr, 5 op/s 2024-03-04T09:32:34.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:32:34 smithi191 ceph-mon[36816]: pgmap v45: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 6.4 KiB/s wr, 5 op/s 2024-03-04T09:32:36.381 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:32:36 smithi106 ceph-mon[36484]: pgmap v46: 129 pgs: 129 active+clean; 15 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 195 KiB/s rd, 1.1 MiB/s wr, 12 op/s 2024-03-04T09:32:36.585 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:32:36 smithi052 ceph-mon[32473]: pgmap v46: 129 pgs: 129 active+clean; 15 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 195 KiB/s rd, 1.1 MiB/s wr, 12 op/s 2024-03-04T09:32:36.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:32:36 smithi191 ceph-mon[36816]: pgmap v46: 129 pgs: 129 active+clean; 15 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 195 KiB/s rd, 1.1 MiB/s wr, 12 op/s 2024-03-04T09:32:38.381 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:32:38 smithi106 ceph-mon[36484]: pgmap v47: 129 pgs: 129 active+clean; 15 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 194 KiB/s rd, 1.1 MiB/s wr, 10 op/s 2024-03-04T09:32:38.387 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:32:38 smithi052 ceph-mon[32473]: pgmap v47: 129 pgs: 129 active+clean; 15 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 194 KiB/s rd, 1.1 MiB/s wr, 10 op/s 2024-03-04T09:32:38.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:32:38 smithi191 ceph-mon[36816]: pgmap v47: 129 pgs: 129 active+clean; 15 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 194 KiB/s rd, 1.1 MiB/s wr, 10 op/s 2024-03-04T09:32:40.585 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:32:40 smithi052 ceph-mon[32473]: pgmap v48: 129 pgs: 129 active+clean; 15 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 194 KiB/s rd, 1.1 MiB/s wr, 10 op/s 2024-03-04T09:32:40.585 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:32:40 smithi052 ceph-mon[32473]: mgrmap e25: x(active, since 90s), standbys: z, y 2024-03-04T09:32:40.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:32:40 smithi191 ceph-mon[36816]: pgmap v48: 129 pgs: 129 active+clean; 15 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 194 KiB/s rd, 1.1 MiB/s wr, 10 op/s 2024-03-04T09:32:40.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:32:40 smithi191 ceph-mon[36816]: mgrmap e25: x(active, since 90s), standbys: z, y 2024-03-04T09:32:40.631 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:32:40 smithi106 ceph-mon[36484]: pgmap v48: 129 pgs: 129 active+clean; 15 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 194 KiB/s rd, 1.1 MiB/s wr, 10 op/s 2024-03-04T09:32:40.631 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:32:40 smithi106 ceph-mon[36484]: mgrmap e25: x(active, since 90s), standbys: z, y 2024-03-04T09:32:42.585 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:32:42 smithi052 ceph-mon[32473]: pgmap v49: 129 pgs: 129 active+clean; 16 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 272 KiB/s rd, 1.2 MiB/s wr, 17 op/s 2024-03-04T09:32:42.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:32:42 smithi191 ceph-mon[36816]: pgmap v49: 129 pgs: 129 active+clean; 16 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 272 KiB/s rd, 1.2 MiB/s wr, 17 op/s 2024-03-04T09:32:42.631 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:32:42 smithi106 ceph-mon[36484]: pgmap v49: 129 pgs: 129 active+clean; 16 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 272 KiB/s rd, 1.2 MiB/s wr, 17 op/s 2024-03-04T09:32:44.588 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:32:44 smithi052 ceph-mon[32473]: pgmap v50: 129 pgs: 129 active+clean; 16 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 271 KiB/s rd, 1.2 MiB/s wr, 14 op/s 2024-03-04T09:32:44.588 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:32:44 smithi191 ceph-mon[36816]: pgmap v50: 129 pgs: 129 active+clean; 16 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 271 KiB/s rd, 1.2 MiB/s wr, 14 op/s 2024-03-04T09:32:44.631 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:32:44 smithi106 ceph-mon[36484]: pgmap v50: 129 pgs: 129 active+clean; 16 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 271 KiB/s rd, 1.2 MiB/s wr, 14 op/s 2024-03-04T09:32:46.585 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:32:46 smithi052 ceph-mon[32473]: pgmap v51: 129 pgs: 129 active+clean; 18 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 448 KiB/s rd, 1.4 MiB/s wr, 21 op/s 2024-03-04T09:32:46.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:32:46 smithi191 ceph-mon[36816]: pgmap v51: 129 pgs: 129 active+clean; 18 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 448 KiB/s rd, 1.4 MiB/s wr, 21 op/s 2024-03-04T09:32:46.631 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:32:46 smithi106 ceph-mon[36484]: pgmap v51: 129 pgs: 129 active+clean; 18 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 448 KiB/s rd, 1.4 MiB/s wr, 21 op/s 2024-03-04T09:32:48.585 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:32:48 smithi052 ceph-mon[32473]: pgmap v52: 129 pgs: 129 active+clean; 18 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 255 KiB/s rd, 320 KiB/s wr, 14 op/s 2024-03-04T09:32:48.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:32:48 smithi191 ceph-mon[36816]: pgmap v52: 129 pgs: 129 active+clean; 18 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 255 KiB/s rd, 320 KiB/s wr, 14 op/s 2024-03-04T09:32:48.631 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:32:48 smithi106 ceph-mon[36484]: pgmap v52: 129 pgs: 129 active+clean; 18 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 255 KiB/s rd, 320 KiB/s wr, 14 op/s 2024-03-04T09:32:50.585 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:32:50 smithi052 ceph-mon[32473]: pgmap v53: 129 pgs: 129 active+clean; 18 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 255 KiB/s rd, 320 KiB/s wr, 14 op/s 2024-03-04T09:32:50.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:32:50 smithi191 ceph-mon[36816]: pgmap v53: 129 pgs: 129 active+clean; 18 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 255 KiB/s rd, 320 KiB/s wr, 14 op/s 2024-03-04T09:32:50.631 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:32:50 smithi106 ceph-mon[36484]: pgmap v53: 129 pgs: 129 active+clean; 18 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 255 KiB/s rd, 320 KiB/s wr, 14 op/s 2024-03-04T09:32:52.585 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:32:52 smithi052 ceph-mon[32473]: pgmap v54: 129 pgs: 129 active+clean; 19 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 339 KiB/s rd, 472 KiB/s wr, 21 op/s 2024-03-04T09:32:52.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:32:52 smithi191 ceph-mon[36816]: pgmap v54: 129 pgs: 129 active+clean; 19 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 339 KiB/s rd, 472 KiB/s wr, 21 op/s 2024-03-04T09:32:52.631 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:32:52 smithi106 ceph-mon[36484]: pgmap v54: 129 pgs: 129 active+clean; 19 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 339 KiB/s rd, 472 KiB/s wr, 21 op/s 2024-03-04T09:32:54.584 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:32:54 smithi052 ceph-mon[32473]: pgmap v55: 129 pgs: 129 active+clean; 19 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 261 KiB/s rd, 370 KiB/s wr, 14 op/s 2024-03-04T09:32:54.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:32:54 smithi191 ceph-mon[36816]: pgmap v55: 129 pgs: 129 active+clean; 19 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 261 KiB/s rd, 370 KiB/s wr, 14 op/s 2024-03-04T09:32:54.631 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:32:54 smithi106 ceph-mon[36484]: pgmap v55: 129 pgs: 129 active+clean; 19 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 261 KiB/s rd, 370 KiB/s wr, 14 op/s 2024-03-04T09:32:55.215 INFO:tasks.workunit.client.0.smithi052.stderr:Updating files: 7% (321/4261) Updating files: 8% (341/4261) Updating files: 9% (384/4261) Updating files: 10% (427/4261) Updating files: 11% (469/4261) Updating files: 12% (512/4261) Updating files: 13% (554/4261) Updating files: 14% (597/4261) Updating files: 15% (640/4261) Updating files: 16% (682/4261) Updating files: 17% (725/4261) Updating files: 17% (741/4261) Updating files: 18% (767/4261) Updating files: 19% (810/4261) Updating files: 20% (853/4261) Updating files: 21% (895/4261) Updating files: 22% (938/4261) Updating files: 23% (981/4261) Updating files: 24% (1023/4261) Updating files: 25% (1066/4261) Updating files: 26% (1108/4261) Updating files: 27% (1151/4261) Updating files: 28% (1194/4261) Updating files: 29% (1236/4261) Updating files: 30% (1279/4261) Updating files: 31% (1321/4261) Updating files: 32% (1364/4261) Updating files: 33% (1407/4261) Updating files: 34% (1449/4261) Updating files: 35% (1492/4261) Updating files: 35% (1501/4261) Updating files: 36% (1534/4261) Updating files: 37% (1577/4261) Updating files: 38% (1620/4261) Updating files: 39% (1662/4261) Updating files: 40% (1705/4261) Updating files: 41% (1748/4261) Updating files: 42% (1790/4261) Updating files: 43% (1833/4261) Updating files: 44% (1875/4261) Updating files: 45% (1918/4261) Updating files: 46% (1961/4261) Updating files: 47% (2003/4261) Updating files: 48% (2046/4261) Updating files: 49% (2088/4261) Updating files: 50% (2131/4261) Updating files: 51% (2174/4261) Updating files: 52% (2216/4261) Updating files: 52% (2256/4261) Updating files: 53% (2259/4261) Updating files: 53% (2261/4261) Updating files: 54% (2301/4261) Updating files: 55% (2344/4261) Updating files: 56% (2387/4261) Updating files: 57% (2429/4261) Updating files: 58% (2472/4261) Updating files: 59% (2514/4261) Updating files: 59% (2515/4261) Updating files: 60% (2557/4261) Updating files: 61% (2600/4261) Updating files: 62% (2642/4261) Updating files: 63% (2685/4261) Updating files: 64% (2728/4261) Updating files: 65% (2770/4261) Updating files: 66% (2813/4261) Updating files: 67% (2855/4261) Updating files: 68% (2898/4261) Updating files: 69% (2941/4261) Updating files: 70% (2983/4261) Updating files: 71% (3026/4261) Updating files: 72% (3068/4261) Updating files: 73% (3111/4261) Updating files: 74% (3154/4261) Updating files: 75% (3196/4261) Updating files: 76% (3239/4261) Updating files: 77% (3281/4261) Updating files: 78% (3324/4261) Updating files: 79% (3367/4261) Updating files: 80% (3409/4261) Updating files: 81% (3452/4261) Updating files: 82% (3495/4261) Updating files: 83% (3537/4261) Updating files: 84% (3580/4261) Updating files: 85% (3622/4261) Updating files: 86% (3665/4261) Updating files: 87% (3708/4261) Updating files: 88% (3750/4261) Updating files: 88% (3783/4261) Updating files: 89% (3793/4261) Updating files: 90% (3835/4261) Updating files: 91% (3878/4261) Updating files: 92% (3921/4261) Updating files: 93% (3963/4261) Updating files: 94% (4006/4261) Updating files: 95% (4048/4261) Updating files: 96% (4091/4261) Updating files: 97% (4134/4261) Updating files: 98% (4176/4261) Updating files: 99% (4219/4261) Updating files: 100% (4261/4261) Updating files: 100% (4261/4261), done. 2024-03-04T09:32:55.226 INFO:tasks.workunit.client.0.smithi052.stderr:+ cd xfstests-dev 2024-03-04T09:32:55.226 INFO:tasks.workunit.client.0.smithi052.stderr:+ git checkout 12973fc04fd10d4af086901e10ffa8e48866b735 2024-03-04T09:32:56.584 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:32:56 smithi052 ceph-mon[32473]: pgmap v56: 129 pgs: 129 active+clean; 21 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 310 KiB/s rd, 600 KiB/s wr, 29 op/s 2024-03-04T09:32:56.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:32:56 smithi191 ceph-mon[36816]: pgmap v56: 129 pgs: 129 active+clean; 21 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 310 KiB/s rd, 600 KiB/s wr, 29 op/s 2024-03-04T09:32:56.631 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:32:56 smithi106 ceph-mon[36484]: pgmap v56: 129 pgs: 129 active+clean; 21 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 310 KiB/s rd, 600 KiB/s wr, 29 op/s 2024-03-04T09:32:58.585 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:32:58 smithi052 ceph-mon[32473]: pgmap v57: 129 pgs: 129 active+clean; 21 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 133 KiB/s rd, 382 KiB/s wr, 22 op/s 2024-03-04T09:32:58.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:32:58 smithi191 ceph-mon[36816]: pgmap v57: 129 pgs: 129 active+clean; 21 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 133 KiB/s rd, 382 KiB/s wr, 22 op/s 2024-03-04T09:32:58.631 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:32:58 smithi106 ceph-mon[36484]: pgmap v57: 129 pgs: 129 active+clean; 21 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 133 KiB/s rd, 382 KiB/s wr, 22 op/s 2024-03-04T09:33:00.433 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:33:00 smithi106 ceph-mon[36484]: pgmap v58: 129 pgs: 129 active+clean; 21 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 133 KiB/s rd, 382 KiB/s wr, 22 op/s 2024-03-04T09:33:00.584 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:33:00 smithi052 ceph-mon[32473]: pgmap v58: 129 pgs: 129 active+clean; 21 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 133 KiB/s rd, 382 KiB/s wr, 22 op/s 2024-03-04T09:33:00.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:33:00 smithi191 ceph-mon[36816]: pgmap v58: 129 pgs: 129 active+clean; 21 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 133 KiB/s rd, 382 KiB/s wr, 22 op/s 2024-03-04T09:33:02.585 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:33:02 smithi052 ceph-mon[32473]: pgmap v59: 129 pgs: 129 active+clean; 40 MiB data, 540 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.3 MiB/s wr, 354 op/s 2024-03-04T09:33:02.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:33:02 smithi191 ceph-mon[36816]: pgmap v59: 129 pgs: 129 active+clean; 40 MiB data, 540 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.3 MiB/s wr, 354 op/s 2024-03-04T09:33:02.631 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:33:02 smithi106 ceph-mon[36484]: pgmap v59: 129 pgs: 129 active+clean; 40 MiB data, 540 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.3 MiB/s wr, 354 op/s 2024-03-04T09:33:04.584 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:33:04 smithi052 ceph-mon[32473]: pgmap v60: 129 pgs: 129 active+clean; 40 MiB data, 540 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.2 MiB/s wr, 347 op/s 2024-03-04T09:33:04.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:33:04 smithi191 ceph-mon[36816]: pgmap v60: 129 pgs: 129 active+clean; 40 MiB data, 540 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.2 MiB/s wr, 347 op/s 2024-03-04T09:33:04.631 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:33:04 smithi106 ceph-mon[36484]: pgmap v60: 129 pgs: 129 active+clean; 40 MiB data, 540 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.2 MiB/s wr, 347 op/s 2024-03-04T09:33:05.585 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:33:05 smithi052 ceph-mon[32473]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-03-04T09:33:05.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:33:05 smithi191 ceph-mon[36816]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-03-04T09:33:05.631 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:33:05 smithi106 ceph-mon[36484]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-03-04T09:33:06.584 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:33:06 smithi052 ceph-mon[32473]: pgmap v61: 129 pgs: 129 active+clean; 53 MiB data, 572 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.4 MiB/s wr, 509 op/s 2024-03-04T09:33:06.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:33:06 smithi191 ceph-mon[36816]: pgmap v61: 129 pgs: 129 active+clean; 53 MiB data, 572 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.4 MiB/s wr, 509 op/s 2024-03-04T09:33:06.631 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:33:06 smithi106 ceph-mon[36484]: pgmap v61: 129 pgs: 129 active+clean; 53 MiB data, 572 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.4 MiB/s wr, 509 op/s 2024-03-04T09:33:07.775 INFO:tasks.workunit.client.0.smithi052.stderr:Note: switching to '12973fc04fd10d4af086901e10ffa8e48866b735'. 2024-03-04T09:33:07.775 INFO:tasks.workunit.client.0.smithi052.stderr: 2024-03-04T09:33:07.775 INFO:tasks.workunit.client.0.smithi052.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-03-04T09:33:07.775 INFO:tasks.workunit.client.0.smithi052.stderr:changes and commit them, and you can discard any commits you make in this 2024-03-04T09:33:07.776 INFO:tasks.workunit.client.0.smithi052.stderr:state without impacting any branches by switching back to a branch. 2024-03-04T09:33:07.776 INFO:tasks.workunit.client.0.smithi052.stderr: 2024-03-04T09:33:07.776 INFO:tasks.workunit.client.0.smithi052.stderr:If you want to create a new branch to retain commits you create, you may 2024-03-04T09:33:07.776 INFO:tasks.workunit.client.0.smithi052.stderr:do so (now or later) by using -c with the switch command. Example: 2024-03-04T09:33:07.776 INFO:tasks.workunit.client.0.smithi052.stderr: 2024-03-04T09:33:07.776 INFO:tasks.workunit.client.0.smithi052.stderr: git switch -c 2024-03-04T09:33:07.776 INFO:tasks.workunit.client.0.smithi052.stderr: 2024-03-04T09:33:07.776 INFO:tasks.workunit.client.0.smithi052.stderr:Or undo this operation with: 2024-03-04T09:33:07.776 INFO:tasks.workunit.client.0.smithi052.stderr: 2024-03-04T09:33:07.776 INFO:tasks.workunit.client.0.smithi052.stderr: git switch - 2024-03-04T09:33:07.776 INFO:tasks.workunit.client.0.smithi052.stderr: 2024-03-04T09:33:07.776 INFO:tasks.workunit.client.0.smithi052.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-03-04T09:33:07.777 INFO:tasks.workunit.client.0.smithi052.stderr: 2024-03-04T09:33:07.777 INFO:tasks.workunit.client.0.smithi052.stderr:HEAD is now at 12973fc0 btrfs/022: Correct seq number in golden output 2024-03-04T09:33:07.779 INFO:tasks.workunit.client.0.smithi052.stderr:+ make -j4 2024-03-04T09:33:07.792 INFO:tasks.workunit.client.0.smithi052.stdout:libtoolize -cfi 2024-03-04T09:33:08.235 INFO:tasks.workunit.client.0.smithi052.stdout:libtoolize: putting auxiliary files in '.'. 2024-03-04T09:33:08.236 INFO:tasks.workunit.client.0.smithi052.stdout:libtoolize: copying file './config.guess' 2024-03-04T09:33:08.238 INFO:tasks.workunit.client.0.smithi052.stdout:libtoolize: copying file './config.sub' 2024-03-04T09:33:08.242 INFO:tasks.workunit.client.0.smithi052.stdout:libtoolize: copying file './install-sh' 2024-03-04T09:33:08.246 INFO:tasks.workunit.client.0.smithi052.stdout:libtoolize: copying file './ltmain.sh' 2024-03-04T09:33:08.268 INFO:tasks.workunit.client.0.smithi052.stdout:libtoolize: putting macros in AC_CONFIG_MACRO_DIRS, 'm4'. 2024-03-04T09:33:08.268 INFO:tasks.workunit.client.0.smithi052.stdout:libtoolize: copying file 'm4/libtool.m4' 2024-03-04T09:33:08.272 INFO:tasks.workunit.client.0.smithi052.stdout:libtoolize: copying file 'm4/ltoptions.m4' 2024-03-04T09:33:08.276 INFO:tasks.workunit.client.0.smithi052.stdout:libtoolize: copying file 'm4/ltsugar.m4' 2024-03-04T09:33:08.281 INFO:tasks.workunit.client.0.smithi052.stdout:libtoolize: copying file 'm4/ltversion.m4' 2024-03-04T09:33:08.285 INFO:tasks.workunit.client.0.smithi052.stdout:libtoolize: copying file 'm4/lt~obsolete.m4' 2024-03-04T09:33:08.292 INFO:tasks.workunit.client.0.smithi052.stdout:libtoolize: Consider adding '-I m4' to ACLOCAL_AMFLAGS in Makefile.am. 2024-03-04T09:33:08.293 INFO:tasks.workunit.client.0.smithi052.stdout:cp include/install-sh . 2024-03-04T09:33:08.305 INFO:tasks.workunit.client.0.smithi052.stdout:aclocal -I m4 2024-03-04T09:33:08.585 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:33:08 smithi052 ceph-mon[32473]: pgmap v62: 129 pgs: 129 active+clean; 53 MiB data, 572 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.2 MiB/s wr, 494 op/s 2024-03-04T09:33:08.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:33:08 smithi191 ceph-mon[36816]: pgmap v62: 129 pgs: 129 active+clean; 53 MiB data, 572 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.2 MiB/s wr, 494 op/s 2024-03-04T09:33:08.631 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:33:08 smithi106 ceph-mon[36484]: pgmap v62: 129 pgs: 129 active+clean; 53 MiB data, 572 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.2 MiB/s wr, 494 op/s 2024-03-04T09:33:09.387 INFO:tasks.workunit.client.0.smithi052.stdout:autoheader 2024-03-04T09:33:09.699 INFO:tasks.workunit.client.0.smithi052.stdout:autoconf 2024-03-04T09:33:09.878 INFO:tasks.workunit.client.0.smithi052.stdout:./configure \ 2024-03-04T09:33:09.878 INFO:tasks.workunit.client.0.smithi052.stdout: --libexecdir=/usr/lib \ 2024-03-04T09:33:09.878 INFO:tasks.workunit.client.0.smithi052.stdout: --exec_prefix=/var/lib 2024-03-04T09:33:09.996 INFO:tasks.workunit.client.0.smithi052.stdout:checking build system type... x86_64-pc-linux-gnu 2024-03-04T09:33:09.996 INFO:tasks.workunit.client.0.smithi052.stdout:checking host system type... x86_64-pc-linux-gnu 2024-03-04T09:33:09.997 INFO:tasks.workunit.client.0.smithi052.stdout:checking how to print strings... printf 2024-03-04T09:33:09.997 INFO:tasks.workunit.client.0.smithi052.stdout:checking for gcc... gcc 2024-03-04T09:33:10.068 INFO:tasks.workunit.client.0.smithi052.stdout:checking whether the C compiler works... yes 2024-03-04T09:33:10.068 INFO:tasks.workunit.client.0.smithi052.stdout:checking for C compiler default output file name... a.out 2024-03-04T09:33:10.106 INFO:tasks.workunit.client.0.smithi052.stdout:checking for suffix of executables... 2024-03-04T09:33:10.140 INFO:tasks.workunit.client.0.smithi052.stdout:checking whether we are cross compiling... no 2024-03-04T09:33:10.177 INFO:tasks.workunit.client.0.smithi052.stdout:checking for suffix of object files... o 2024-03-04T09:33:10.199 INFO:tasks.workunit.client.0.smithi052.stdout:checking whether we are using the GNU C compiler... yes 2024-03-04T09:33:10.219 INFO:tasks.workunit.client.0.smithi052.stdout:checking whether gcc accepts -g... yes 2024-03-04T09:33:10.302 INFO:tasks.workunit.client.0.smithi052.stdout:checking for gcc option to accept ISO C89... none needed 2024-03-04T09:33:10.305 INFO:tasks.workunit.client.0.smithi052.stdout:checking for a sed that does not truncate output... /usr/bin/sed 2024-03-04T09:33:10.309 INFO:tasks.workunit.client.0.smithi052.stdout:checking for grep that handles long lines and -e... /usr/bin/grep 2024-03-04T09:33:10.310 INFO:tasks.workunit.client.0.smithi052.stdout:checking for egrep... /usr/bin/grep -E 2024-03-04T09:33:10.311 INFO:tasks.workunit.client.0.smithi052.stdout:checking for fgrep... /usr/bin/grep -F 2024-03-04T09:33:10.314 INFO:tasks.workunit.client.0.smithi052.stdout:checking for ld used by gcc... /usr/bin/ld 2024-03-04T09:33:10.316 INFO:tasks.workunit.client.0.smithi052.stdout:checking if the linker (/usr/bin/ld) is GNU ld... yes 2024-03-04T09:33:10.327 INFO:tasks.workunit.client.0.smithi052.stdout:checking for BSD- or MS-compatible name lister (nm)... /usr/bin/nm -B 2024-03-04T09:33:10.584 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:33:10 smithi052 ceph-mon[32473]: pgmap v63: 129 pgs: 129 active+clean; 53 MiB data, 572 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.2 MiB/s wr, 494 op/s 2024-03-04T09:33:10.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:33:10 smithi191 ceph-mon[36816]: pgmap v63: 129 pgs: 129 active+clean; 53 MiB data, 572 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.2 MiB/s wr, 494 op/s 2024-03-04T09:33:10.631 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:33:10 smithi106 ceph-mon[36484]: pgmap v63: 129 pgs: 129 active+clean; 53 MiB data, 572 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.2 MiB/s wr, 494 op/s 2024-03-04T09:33:11.323 INFO:tasks.workunit.client.0.smithi052.stdout:checking the name lister (/usr/bin/nm -B) interface... BSD nm 2024-03-04T09:33:11.323 INFO:tasks.workunit.client.0.smithi052.stdout:checking whether ln -s works... yes 2024-03-04T09:33:11.344 INFO:tasks.workunit.client.0.smithi052.stdout:checking the maximum length of command line arguments... 1572864 2024-03-04T09:33:11.349 INFO:tasks.workunit.client.0.smithi052.stdout:checking how to convert x86_64-pc-linux-gnu file names to x86_64-pc-linux-gnu format... func_convert_file_noop 2024-03-04T09:33:11.349 INFO:tasks.workunit.client.0.smithi052.stdout:checking how to convert x86_64-pc-linux-gnu file names to toolchain format... func_convert_file_noop 2024-03-04T09:33:11.349 INFO:tasks.workunit.client.0.smithi052.stdout:checking for /usr/bin/ld option to reload object files... -r 2024-03-04T09:33:11.349 INFO:tasks.workunit.client.0.smithi052.stdout:checking for objdump... objdump 2024-03-04T09:33:11.349 INFO:tasks.workunit.client.0.smithi052.stdout:checking how to recognize dependent libraries... pass_all 2024-03-04T09:33:11.349 INFO:tasks.workunit.client.0.smithi052.stdout:checking for dlltool... no 2024-03-04T09:33:11.349 INFO:tasks.workunit.client.0.smithi052.stdout:checking how to associate runtime and link libraries... printf %s\n 2024-03-04T09:33:11.349 INFO:tasks.workunit.client.0.smithi052.stdout:checking for ar... ar 2024-03-04T09:33:11.428 INFO:tasks.workunit.client.0.smithi052.stdout:checking for archiver @FILE support... @ 2024-03-04T09:33:11.428 INFO:tasks.workunit.client.0.smithi052.stdout:checking for strip... strip 2024-03-04T09:33:11.428 INFO:tasks.workunit.client.0.smithi052.stdout:checking for ranlib... ranlib 2024-03-04T09:33:11.428 INFO:tasks.workunit.client.0.smithi052.stdout:checking for gawk... gawk 2024-03-04T09:33:11.532 INFO:tasks.workunit.client.0.smithi052.stdout:checking command to parse /usr/bin/nm -B output from gcc object... ok 2024-03-04T09:33:11.533 INFO:tasks.workunit.client.0.smithi052.stdout:checking for sysroot... no 2024-03-04T09:33:11.544 INFO:tasks.workunit.client.0.smithi052.stdout:checking for a working dd... /usr/bin/dd 2024-03-04T09:33:11.554 INFO:tasks.workunit.client.0.smithi052.stdout:checking how to truncate binary pipes... /usr/bin/dd bs=4096 count=1 2024-03-04T09:33:11.591 INFO:tasks.workunit.client.0.smithi052.stdout:checking for mt... no 2024-03-04T09:33:11.604 INFO:tasks.workunit.client.0.smithi052.stdout:checking if : is a manifest tool... no 2024-03-04T09:33:11.703 INFO:tasks.workunit.client.0.smithi052.stdout:checking how to run the C preprocessor... gcc -E 2024-03-04T09:33:11.885 INFO:tasks.workunit.client.0.smithi052.stdout:checking for ANSI C header files... yes 2024-03-04T09:33:11.916 INFO:tasks.workunit.client.0.smithi052.stdout:checking for sys/types.h... yes 2024-03-04T09:33:11.952 INFO:tasks.workunit.client.0.smithi052.stdout:checking for sys/stat.h... yes 2024-03-04T09:33:11.986 INFO:tasks.workunit.client.0.smithi052.stdout:checking for stdlib.h... yes 2024-03-04T09:33:12.025 INFO:tasks.workunit.client.0.smithi052.stdout:checking for string.h... yes 2024-03-04T09:33:12.071 INFO:tasks.workunit.client.0.smithi052.stdout:checking for memory.h... yes 2024-03-04T09:33:12.110 INFO:tasks.workunit.client.0.smithi052.stdout:checking for strings.h... yes 2024-03-04T09:33:12.148 INFO:tasks.workunit.client.0.smithi052.stdout:checking for inttypes.h... yes 2024-03-04T09:33:12.186 INFO:tasks.workunit.client.0.smithi052.stdout:checking for stdint.h... yes 2024-03-04T09:33:12.226 INFO:tasks.workunit.client.0.smithi052.stdout:checking for unistd.h... yes 2024-03-04T09:33:12.275 INFO:tasks.workunit.client.0.smithi052.stdout:checking for dlfcn.h... yes 2024-03-04T09:33:12.280 INFO:tasks.workunit.client.0.smithi052.stdout:checking for objdir... .libs 2024-03-04T09:33:12.385 INFO:tasks.workunit.client.0.smithi052.stdout:checking if gcc supports -fno-rtti -fno-exceptions... no 2024-03-04T09:33:12.385 INFO:tasks.workunit.client.0.smithi052.stdout:checking for gcc option to produce PIC... -fPIC -DPIC 2024-03-04T09:33:12.421 INFO:tasks.workunit.client.0.smithi052.stdout:checking if gcc PIC flag -fPIC -DPIC works... yes 2024-03-04T09:33:12.461 INFO:tasks.workunit.client.0.smithi052.stdout:checking if gcc static flag -static works... no 2024-03-04T09:33:12.520 INFO:tasks.workunit.client.0.smithi052.stdout:checking if gcc supports -c -o file.o... yes 2024-03-04T09:33:12.520 INFO:tasks.workunit.client.0.smithi052.stdout:checking if gcc supports -c -o file.o... (cached) yes 2024-03-04T09:33:12.526 INFO:tasks.workunit.client.0.smithi052.stdout:checking whether the gcc linker (/usr/bin/ld -m elf_x86_64) supports shared libraries... yes 2024-03-04T09:33:12.575 INFO:tasks.workunit.client.0.smithi052.stdout:checking whether -lc should be explicitly linked in... no 2024-03-04T09:33:12.584 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:33:12 smithi052 ceph-mon[32473]: pgmap v64: 129 pgs: 129 active+clean; 61 MiB data, 630 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.8 MiB/s wr, 646 op/s 2024-03-04T09:33:12.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:33:12 smithi191 ceph-mon[36816]: pgmap v64: 129 pgs: 129 active+clean; 61 MiB data, 630 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.8 MiB/s wr, 646 op/s 2024-03-04T09:33:12.631 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:33:12 smithi106 ceph-mon[36484]: pgmap v64: 129 pgs: 129 active+clean; 61 MiB data, 630 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.8 MiB/s wr, 646 op/s 2024-03-04T09:33:12.636 INFO:tasks.workunit.client.0.smithi052.stdout:checking dynamic linker characteristics... GNU/Linux ld.so 2024-03-04T09:33:12.636 INFO:tasks.workunit.client.0.smithi052.stdout:checking how to hardcode library paths into programs... immediate 2024-03-04T09:33:12.638 INFO:tasks.workunit.client.0.smithi052.stdout:checking whether stripping libraries is possible... yes 2024-03-04T09:33:12.638 INFO:tasks.workunit.client.0.smithi052.stdout:checking if libtool supports shared libraries... yes 2024-03-04T09:33:12.638 INFO:tasks.workunit.client.0.smithi052.stdout:checking whether to build shared libraries... yes 2024-03-04T09:33:12.638 INFO:tasks.workunit.client.0.smithi052.stdout:checking whether to build static libraries... yes 2024-03-04T09:33:12.644 INFO:tasks.workunit.client.0.smithi052.stdout:checking for gcc... (cached) gcc 2024-03-04T09:33:12.670 INFO:tasks.workunit.client.0.smithi052.stdout:checking whether we are using the GNU C compiler... (cached) yes 2024-03-04T09:33:12.670 INFO:tasks.workunit.client.0.smithi052.stdout:checking whether gcc accepts -g... (cached) yes 2024-03-04T09:33:12.670 INFO:tasks.workunit.client.0.smithi052.stdout:checking for gcc option to accept ISO C89... (cached) none needed 2024-03-04T09:33:12.670 INFO:tasks.workunit.client.0.smithi052.stdout:checking for gmake... /usr/bin/gmake 2024-03-04T09:33:12.671 INFO:tasks.workunit.client.0.smithi052.stderr:./configure: line 11968: test: $(SHELL): binary operator expected 2024-03-04T09:33:12.671 INFO:tasks.workunit.client.0.smithi052.stdout:checking for tar... /bin/tar 2024-03-04T09:33:12.672 INFO:tasks.workunit.client.0.smithi052.stdout:checking for gzip... /bin/gzip 2024-03-04T09:33:12.689 INFO:tasks.workunit.client.0.smithi052.stdout:checking whether gcc -MM is supported... yes 2024-03-04T09:33:12.689 INFO:tasks.workunit.client.0.smithi052.stdout:checking for sort... /bin/sort 2024-03-04T09:33:12.689 INFO:tasks.workunit.client.0.smithi052.stdout:checking whether ln -s works... yes 2024-03-04T09:33:12.689 INFO:tasks.workunit.client.0.smithi052.stdout:checking for rpm... /bin/rpm 2024-03-04T09:33:12.692 INFO:tasks.workunit.client.0.smithi052.stdout:checking for rpmbuild... no 2024-03-04T09:33:12.692 INFO:tasks.workunit.client.0.smithi052.stdout:checking for ANSI C header files... (cached) yes 2024-03-04T09:33:12.755 INFO:tasks.workunit.client.0.smithi052.stdout:checking assert.h usability... yes 2024-03-04T09:33:12.768 INFO:tasks.workunit.client.0.smithi052.stdout:checking assert.h presence... yes 2024-03-04T09:33:12.769 INFO:tasks.workunit.client.0.smithi052.stdout:checking for assert.h... yes 2024-03-04T09:33:12.824 INFO:tasks.workunit.client.0.smithi052.stdout:checking bstring.h usability... no 2024-03-04T09:33:12.848 INFO:tasks.workunit.client.0.smithi052.stdout:checking bstring.h presence... no 2024-03-04T09:33:12.848 INFO:tasks.workunit.client.0.smithi052.stdout:checking for bstring.h... no 2024-03-04T09:33:12.889 INFO:tasks.workunit.client.0.smithi052.stdout:checking libgen.h usability... yes 2024-03-04T09:33:12.902 INFO:tasks.workunit.client.0.smithi052.stdout:checking libgen.h presence... yes 2024-03-04T09:33:12.903 INFO:tasks.workunit.client.0.smithi052.stdout:checking for libgen.h... yes 2024-03-04T09:33:12.953 INFO:tasks.workunit.client.0.smithi052.stdout:checking dirent.h usability... yes 2024-03-04T09:33:12.967 INFO:tasks.workunit.client.0.smithi052.stdout:checking dirent.h presence... yes 2024-03-04T09:33:12.967 INFO:tasks.workunit.client.0.smithi052.stdout:checking for dirent.h... yes 2024-03-04T09:33:13.011 INFO:tasks.workunit.client.0.smithi052.stdout:checking errno.h usability... yes 2024-03-04T09:33:13.024 INFO:tasks.workunit.client.0.smithi052.stdout:checking errno.h presence... yes 2024-03-04T09:33:13.024 INFO:tasks.workunit.client.0.smithi052.stdout:checking for errno.h... yes 2024-03-04T09:33:13.104 INFO:tasks.workunit.client.0.smithi052.stdout:checking malloc.h usability... yes 2024-03-04T09:33:13.121 INFO:tasks.workunit.client.0.smithi052.stdout:checking malloc.h presence... yes 2024-03-04T09:33:13.121 INFO:tasks.workunit.client.0.smithi052.stdout:checking for malloc.h... yes 2024-03-04T09:33:13.171 INFO:tasks.workunit.client.0.smithi052.stdout:checking uuid.h usability... no 2024-03-04T09:33:13.198 INFO:tasks.workunit.client.0.smithi052.stdout:checking uuid.h presence... no 2024-03-04T09:33:13.198 INFO:tasks.workunit.client.0.smithi052.stdout:checking for uuid.h... no 2024-03-04T09:33:13.241 INFO:tasks.workunit.client.0.smithi052.stdout:checking uuid/uuid.h usability... yes 2024-03-04T09:33:13.258 INFO:tasks.workunit.client.0.smithi052.stdout:checking uuid/uuid.h presence... yes 2024-03-04T09:33:13.258 INFO:tasks.workunit.client.0.smithi052.stdout:checking for uuid/uuid.h... yes 2024-03-04T09:33:13.313 INFO:tasks.workunit.client.0.smithi052.stdout:checking sys/uuid.h usability... no 2024-03-04T09:33:13.338 INFO:tasks.workunit.client.0.smithi052.stdout:checking sys/uuid.h presence... no 2024-03-04T09:33:13.338 INFO:tasks.workunit.client.0.smithi052.stdout:checking for sys/uuid.h... no 2024-03-04T09:33:13.377 INFO:tasks.workunit.client.0.smithi052.stdout:checking sys/file.h usability... yes 2024-03-04T09:33:13.393 INFO:tasks.workunit.client.0.smithi052.stdout:checking sys/file.h presence... yes 2024-03-04T09:33:13.393 INFO:tasks.workunit.client.0.smithi052.stdout:checking for sys/file.h... yes 2024-03-04T09:33:13.447 INFO:tasks.workunit.client.0.smithi052.stdout:checking sys/fcntl.h usability... yes 2024-03-04T09:33:13.462 INFO:tasks.workunit.client.0.smithi052.stdout:checking sys/fcntl.h presence... yes 2024-03-04T09:33:13.462 INFO:tasks.workunit.client.0.smithi052.stdout:checking for sys/fcntl.h... yes 2024-03-04T09:33:13.512 INFO:tasks.workunit.client.0.smithi052.stdout:checking sys/syssgi.h usability... no 2024-03-04T09:33:13.536 INFO:tasks.workunit.client.0.smithi052.stdout:checking sys/syssgi.h presence... no 2024-03-04T09:33:13.537 INFO:tasks.workunit.client.0.smithi052.stdout:checking for sys/syssgi.h... no 2024-03-04T09:33:13.629 INFO:tasks.workunit.client.0.smithi052.stdout:checking sys/param.h usability... yes 2024-03-04T09:33:13.644 INFO:tasks.workunit.client.0.smithi052.stdout:checking sys/param.h presence... yes 2024-03-04T09:33:13.644 INFO:tasks.workunit.client.0.smithi052.stdout:checking for sys/param.h... yes 2024-03-04T09:33:13.657 INFO:tasks.workunit.client.0.smithi052.stdout:checking for sys/stat.h... (cached) yes 2024-03-04T09:33:13.701 INFO:tasks.workunit.client.0.smithi052.stdout:checking sys/statvfs.h usability... yes 2024-03-04T09:33:13.717 INFO:tasks.workunit.client.0.smithi052.stdout:checking sys/statvfs.h presence... yes 2024-03-04T09:33:13.717 INFO:tasks.workunit.client.0.smithi052.stdout:checking for sys/statvfs.h... yes 2024-03-04T09:33:13.759 INFO:tasks.workunit.client.0.smithi052.stdout:checking sys/time.h usability... yes 2024-03-04T09:33:13.772 INFO:tasks.workunit.client.0.smithi052.stdout:checking sys/time.h presence... yes 2024-03-04T09:33:13.772 INFO:tasks.workunit.client.0.smithi052.stdout:checking for sys/time.h... yes 2024-03-04T09:33:13.814 INFO:tasks.workunit.client.0.smithi052.stdout:checking sys/ioctl.h usability... yes 2024-03-04T09:33:13.827 INFO:tasks.workunit.client.0.smithi052.stdout:checking sys/ioctl.h presence... yes 2024-03-04T09:33:13.827 INFO:tasks.workunit.client.0.smithi052.stdout:checking for sys/ioctl.h... yes 2024-03-04T09:33:13.879 INFO:tasks.workunit.client.0.smithi052.stdout:checking sys/wait.h usability... yes 2024-03-04T09:33:13.897 INFO:tasks.workunit.client.0.smithi052.stdout:checking sys/wait.h presence... yes 2024-03-04T09:33:13.897 INFO:tasks.workunit.client.0.smithi052.stdout:checking for sys/wait.h... yes 2024-03-04T09:33:13.915 INFO:tasks.workunit.client.0.smithi052.stdout:checking for sys/types.h... (cached) yes 2024-03-04T09:33:13.919 INFO:tasks.workunit.client.0.smithi052.stdout:checking for strings.h... (cached) yes 2024-03-04T09:33:13.953 INFO:tasks.workunit.client.0.smithi052.stdout:checking err.h usability... yes 2024-03-04T09:33:13.967 INFO:tasks.workunit.client.0.smithi052.stdout:checking err.h presence... yes 2024-03-04T09:33:13.967 INFO:tasks.workunit.client.0.smithi052.stdout:checking for err.h... yes 2024-03-04T09:33:14.015 INFO:tasks.workunit.client.0.smithi052.stdout:checking linux/falloc.h usability... yes 2024-03-04T09:33:14.029 INFO:tasks.workunit.client.0.smithi052.stdout:checking linux/falloc.h presence... yes 2024-03-04T09:33:14.029 INFO:tasks.workunit.client.0.smithi052.stdout:checking for linux/falloc.h... yes 2024-03-04T09:33:14.079 INFO:tasks.workunit.client.0.smithi052.stdout:checking sys/fs/xfs_fsops.h usability... no 2024-03-04T09:33:14.106 INFO:tasks.workunit.client.0.smithi052.stdout:checking sys/fs/xfs_fsops.h presence... no 2024-03-04T09:33:14.106 INFO:tasks.workunit.client.0.smithi052.stdout:checking for sys/fs/xfs_fsops.h... no 2024-03-04T09:33:14.150 INFO:tasks.workunit.client.0.smithi052.stdout:checking sys/fs/xfs_itable.h usability... no 2024-03-04T09:33:14.173 INFO:tasks.workunit.client.0.smithi052.stdout:checking sys/fs/xfs_itable.h presence... no 2024-03-04T09:33:14.173 INFO:tasks.workunit.client.0.smithi052.stdout:checking for sys/fs/xfs_itable.h... no 2024-03-04T09:33:14.217 INFO:tasks.workunit.client.0.smithi052.stdout:checking xfs/platform_defs.h usability... no 2024-03-04T09:33:14.243 INFO:tasks.workunit.client.0.smithi052.stdout:checking xfs/platform_defs.h presence... no 2024-03-04T09:33:14.243 INFO:tasks.workunit.client.0.smithi052.stdout:checking for xfs/platform_defs.h... no 2024-03-04T09:33:14.287 INFO:tasks.workunit.client.0.smithi052.stdout:checking btrfs/ioctl.h usability... no 2024-03-04T09:33:14.313 INFO:tasks.workunit.client.0.smithi052.stdout:checking btrfs/ioctl.h presence... no 2024-03-04T09:33:14.313 INFO:tasks.workunit.client.0.smithi052.stdout:checking for btrfs/ioctl.h... no 2024-03-04T09:33:14.354 INFO:tasks.workunit.client.0.smithi052.stdout:checking cifs/ioctl.h usability... no 2024-03-04T09:33:14.379 INFO:tasks.workunit.client.0.smithi052.stdout:checking cifs/ioctl.h presence... no 2024-03-04T09:33:14.379 INFO:tasks.workunit.client.0.smithi052.stdout:checking for cifs/ioctl.h... no 2024-03-04T09:33:14.419 INFO:tasks.workunit.client.0.smithi052.stdout:checking sys/mman.h usability... yes 2024-03-04T09:33:14.435 INFO:tasks.workunit.client.0.smithi052.stdout:checking sys/mman.h presence... yes 2024-03-04T09:33:14.435 INFO:tasks.workunit.client.0.smithi052.stdout:checking for sys/mman.h... yes 2024-03-04T09:33:14.475 INFO:tasks.workunit.client.0.smithi052.stdout:checking for xfs/xfs_log_format.h... no 2024-03-04T09:33:14.523 INFO:tasks.workunit.client.0.smithi052.stdout:checking for uuid_compare... no 2024-03-04T09:33:14.561 INFO:tasks.workunit.client.0.smithi052.stdout:checking for uuid_compare in -luuid... yes 2024-03-04T09:33:14.584 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:33:14 smithi052 ceph-mon[32473]: pgmap v65: 129 pgs: 129 active+clean; 61 MiB data, 630 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.9 MiB/s wr, 314 op/s 2024-03-04T09:33:14.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:33:14 smithi191 ceph-mon[36816]: pgmap v65: 129 pgs: 129 active+clean; 61 MiB data, 630 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.9 MiB/s wr, 314 op/s 2024-03-04T09:33:14.621 INFO:tasks.workunit.client.0.smithi052.stdout:checking for xfs/xfs.h... yes 2024-03-04T09:33:14.631 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:33:14 smithi106 ceph-mon[36484]: pgmap v65: 129 pgs: 129 active+clean; 61 MiB data, 630 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.9 MiB/s wr, 314 op/s 2024-03-04T09:33:14.657 INFO:tasks.workunit.client.0.smithi052.stdout:checking for xfs/libxfs.h... no 2024-03-04T09:33:14.690 INFO:tasks.workunit.client.0.smithi052.stdout:checking whether xlog_assign_lsn is declared... no 2024-03-04T09:33:14.726 INFO:tasks.workunit.client.0.smithi052.stdout:checking for xfs/xqm.h... yes 2024-03-04T09:33:14.775 INFO:tasks.workunit.client.0.smithi052.stdout:checking xfsctl from xfs/xfs.h... ok 2024-03-04T09:33:14.807 INFO:tasks.workunit.client.0.smithi052.stdout:checking xfs/handle.h usability... yes 2024-03-04T09:33:14.819 INFO:tasks.workunit.client.0.smithi052.stdout:checking xfs/handle.h presence... yes 2024-03-04T09:33:14.819 INFO:tasks.workunit.client.0.smithi052.stdout:checking for xfs/handle.h... yes 2024-03-04T09:33:14.855 INFO:tasks.workunit.client.0.smithi052.stdout:checking for attr_list_by_handle in -lhandle... yes 2024-03-04T09:33:14.929 INFO:tasks.workunit.client.0.smithi052.stdout:checking attr/attributes.h usability... yes 2024-03-04T09:33:14.944 INFO:tasks.workunit.client.0.smithi052.stdout:checking attr/attributes.h presence... yes 2024-03-04T09:33:14.944 INFO:tasks.workunit.client.0.smithi052.stdout:checking for attr/attributes.h... yes 2024-03-04T09:33:15.002 INFO:tasks.workunit.client.0.smithi052.stdout:checking sys/attributes.h usability... no 2024-03-04T09:33:15.031 INFO:tasks.workunit.client.0.smithi052.stdout:checking sys/attributes.h presence... no 2024-03-04T09:33:15.031 INFO:tasks.workunit.client.0.smithi052.stdout:checking for sys/attributes.h... no 2024-03-04T09:33:15.069 INFO:tasks.workunit.client.0.smithi052.stdout:checking for attr_list in -lattr... yes 2024-03-04T09:33:15.105 INFO:tasks.workunit.client.0.smithi052.stdout:checking for attr_set in -lattr... yes 2024-03-04T09:33:15.132 INFO:tasks.workunit.client.0.smithi052.stdout:checking sys/acl.h usability... yes 2024-03-04T09:33:15.146 INFO:tasks.workunit.client.0.smithi052.stdout:checking sys/acl.h presence... yes 2024-03-04T09:33:15.146 INFO:tasks.workunit.client.0.smithi052.stdout:checking for sys/acl.h... yes 2024-03-04T09:33:15.173 INFO:tasks.workunit.client.0.smithi052.stdout:checking acl/libacl.h usability... yes 2024-03-04T09:33:15.189 INFO:tasks.workunit.client.0.smithi052.stdout:checking acl/libacl.h presence... yes 2024-03-04T09:33:15.189 INFO:tasks.workunit.client.0.smithi052.stdout:checking for acl/libacl.h... yes 2024-03-04T09:33:15.229 INFO:tasks.workunit.client.0.smithi052.stdout:checking for acl_init in -lacl... yes 2024-03-04T09:33:15.299 INFO:tasks.workunit.client.0.smithi052.stdout:checking gdbm-ndbm.h usability... no 2024-03-04T09:33:15.324 INFO:tasks.workunit.client.0.smithi052.stdout:checking gdbm-ndbm.h presence... no 2024-03-04T09:33:15.324 INFO:tasks.workunit.client.0.smithi052.stdout:checking for gdbm-ndbm.h... no 2024-03-04T09:33:15.362 INFO:tasks.workunit.client.0.smithi052.stdout:checking gdbm/ndbm.h usability... no 2024-03-04T09:33:15.390 INFO:tasks.workunit.client.0.smithi052.stdout:checking gdbm/ndbm.h presence... no 2024-03-04T09:33:15.390 INFO:tasks.workunit.client.0.smithi052.stdout:checking for gdbm/ndbm.h... no 2024-03-04T09:33:15.424 INFO:tasks.workunit.client.0.smithi052.stdout:checking ndbm.h usability... no 2024-03-04T09:33:15.451 INFO:tasks.workunit.client.0.smithi052.stdout:checking ndbm.h presence... no 2024-03-04T09:33:15.452 INFO:tasks.workunit.client.0.smithi052.stdout:checking for ndbm.h... no 2024-03-04T09:33:15.484 INFO:tasks.workunit.client.0.smithi052.stdout:checking libaio.h usability... yes 2024-03-04T09:33:15.501 INFO:tasks.workunit.client.0.smithi052.stdout:checking libaio.h presence... yes 2024-03-04T09:33:15.501 INFO:tasks.workunit.client.0.smithi052.stdout:checking for libaio.h... yes 2024-03-04T09:33:15.546 INFO:tasks.workunit.client.0.smithi052.stdout:checking for dm_make_handle in -ldm... no 2024-03-04T09:33:15.546 INFO:tasks.workunit.client.0.smithi052.stdout: 2024-03-04T09:33:15.546 INFO:tasks.workunit.client.0.smithi052.stdout:WARNING: could not find a valid DMAPI base library. 2024-03-04T09:33:15.546 INFO:tasks.workunit.client.0.smithi052.stdout:If you want DMAPI support please install the data migration 2024-03-04T09:33:15.546 INFO:tasks.workunit.client.0.smithi052.stdout:API (dmapi) library package. Alternatively, run "make install" 2024-03-04T09:33:15.546 INFO:tasks.workunit.client.0.smithi052.stdout:from the dmapi source. 2024-03-04T09:33:15.546 INFO:tasks.workunit.client.0.smithi052.stdout: 2024-03-04T09:33:15.578 INFO:tasks.workunit.client.0.smithi052.stdout:checking linux/fiemap.h usability... yes 2024-03-04T09:33:15.584 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:33:15 smithi052 ceph-mon[32473]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T09:33:15.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:33:15 smithi191 ceph-mon[36816]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T09:33:15.592 INFO:tasks.workunit.client.0.smithi052.stdout:checking linux/fiemap.h presence... yes 2024-03-04T09:33:15.592 INFO:tasks.workunit.client.0.smithi052.stdout:checking for linux/fiemap.h... yes 2024-03-04T09:33:15.625 INFO:tasks.workunit.client.0.smithi052.stdout:checking for fallocate... yes 2024-03-04T09:33:15.631 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:33:15 smithi106 ceph-mon[36484]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T09:33:15.670 INFO:tasks.workunit.client.0.smithi052.stdout:checking for open_by_handle_at... yes 2024-03-04T09:33:15.717 INFO:tasks.workunit.client.0.smithi052.stdout:checking sys/prctl.h usability... yes 2024-03-04T09:33:15.733 INFO:tasks.workunit.client.0.smithi052.stdout:checking sys/prctl.h presence... yes 2024-03-04T09:33:15.733 INFO:tasks.workunit.client.0.smithi052.stdout:checking for sys/prctl.h... yes 2024-03-04T09:33:15.763 INFO:tasks.workunit.client.0.smithi052.stdout:checking linux/fs.h usability... yes 2024-03-04T09:33:15.775 INFO:tasks.workunit.client.0.smithi052.stdout:checking linux/fs.h presence... yes 2024-03-04T09:33:15.775 INFO:tasks.workunit.client.0.smithi052.stdout:checking for linux/fs.h... yes 2024-03-04T09:33:15.830 INFO:tasks.workunit.client.0.smithi052.stdout:checking btrfsutil.h usability... no 2024-03-04T09:33:15.859 INFO:tasks.workunit.client.0.smithi052.stdout:checking btrfsutil.h presence... no 2024-03-04T09:33:15.860 INFO:tasks.workunit.client.0.smithi052.stdout:checking for btrfsutil.h... no 2024-03-04T09:33:15.904 INFO:tasks.workunit.client.0.smithi052.stdout:checking for copy_file_range... yes 2024-03-04T09:33:15.947 INFO:tasks.workunit.client.0.smithi052.stdout:checking for renameat2... yes 2024-03-04T09:33:15.964 INFO:tasks.workunit.client.0.smithi052.stdout:configure: creating ./config.status 2024-03-04T09:33:16.289 INFO:tasks.workunit.client.0.smithi052.stdout:config.status: creating include/builddefs 2024-03-04T09:33:16.300 INFO:tasks.workunit.client.0.smithi052.stdout:config.status: creating include/config.h 2024-03-04T09:33:16.309 INFO:tasks.workunit.client.0.smithi052.stdout:config.status: executing libtool commands 2024-03-04T09:33:16.385 INFO:tasks.workunit.client.0.smithi052.stdout:Building include 2024-03-04T09:33:16.385 INFO:tasks.workunit.client.0.smithi052.stdout:Building lib 2024-03-04T09:33:16.386 INFO:tasks.workunit.client.0.smithi052.stdout:Building tests 2024-03-04T09:33:16.389 INFO:tasks.workunit.client.0.smithi052.stdout: [LTDEP] 2024-03-04T09:33:16.541 INFO:tasks.workunit.client.0.smithi052.stdout: [CC] dataascii.lo 2024-03-04T09:33:16.541 INFO:tasks.workunit.client.0.smithi052.stdout: [CC] databin.lo 2024-03-04T09:33:16.541 INFO:tasks.workunit.client.0.smithi052.stdout: [CC] datapid.lo 2024-03-04T09:33:16.541 INFO:tasks.workunit.client.0.smithi052.stdout: [CC] file_lock.lo 2024-03-04T09:33:16.584 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:33:16 smithi052 ceph-mon[32473]: pgmap v66: 129 pgs: 129 active+clean; 64 MiB data, 637 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.1 MiB/s wr, 329 op/s 2024-03-04T09:33:16.585 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:33:16 smithi052 ceph-mon[32473]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T09:33:16.585 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:33:16 smithi052 ceph-mon[32473]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T09:33:16.585 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:33:16 smithi052 ceph-mon[32473]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' 2024-03-04T09:33:16.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:33:16 smithi191 ceph-mon[36816]: pgmap v66: 129 pgs: 129 active+clean; 64 MiB data, 637 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.1 MiB/s wr, 329 op/s 2024-03-04T09:33:16.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:33:16 smithi191 ceph-mon[36816]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T09:33:16.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:33:16 smithi191 ceph-mon[36816]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T09:33:16.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:33:16 smithi191 ceph-mon[36816]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' 2024-03-04T09:33:16.631 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:33:16 smithi106 ceph-mon[36484]: pgmap v66: 129 pgs: 129 active+clean; 64 MiB data, 637 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.1 MiB/s wr, 329 op/s 2024-03-04T09:33:16.631 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:33:16 smithi106 ceph-mon[36484]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T09:33:16.631 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:33:16 smithi106 ceph-mon[36484]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T09:33:16.631 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:33:16 smithi106 ceph-mon[36484]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' 2024-03-04T09:33:16.650 INFO:tasks.workunit.client.0.smithi052.stdout: [CC] forker.lo 2024-03-04T09:33:16.667 INFO:tasks.workunit.client.0.smithi052.stdout: [CC] pattern.lo 2024-03-04T09:33:16.708 INFO:tasks.workunit.client.0.smithi052.stdout: [CC] open_flags.lo 2024-03-04T09:33:16.739 INFO:tasks.workunit.client.0.smithi052.stdout: [CC] random_range.lo 2024-03-04T09:33:16.797 INFO:tasks.workunit.client.0.smithi052.stdout: [CC] string_to_tokens.lo 2024-03-04T09:33:16.803 INFO:tasks.workunit.client.0.smithi052.stdout: [CC] str_to_bytes.lo 2024-03-04T09:33:16.861 INFO:tasks.workunit.client.0.smithi052.stdout: [CC] tlibio.lo 2024-03-04T09:33:16.908 INFO:tasks.workunit.client.0.smithi052.stdout: [CC] write_log.lo 2024-03-04T09:33:16.962 INFO:tasks.workunit.client.0.smithi052.stdout: [CC] random.lo 2024-03-04T09:33:17.149 INFO:tasks.workunit.client.0.smithi052.stdout: [LD] 2024-03-04T09:33:17.262 INFO:tasks.workunit.client.0.smithi052.stdout:Building ltp 2024-03-04T09:33:17.262 INFO:tasks.workunit.client.0.smithi052.stdout:Building src 2024-03-04T09:33:17.262 INFO:tasks.workunit.client.0.smithi052.stdout:Building m4 2024-03-04T09:33:17.262 INFO:tasks.workunit.client.0.smithi052.stdout:Building common 2024-03-04T09:33:17.296 INFO:tasks.workunit.client.0.smithi052.stdout: [DEP] 2024-03-04T09:33:17.345 INFO:tasks.workunit.client.0.smithi052.stdout: [DEP] 2024-03-04T09:33:18.587 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:33:18 smithi052 ceph-mon[32473]: pgmap v67: 129 pgs: 129 active+clean; 64 MiB data, 637 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 911 KiB/s wr, 167 op/s 2024-03-04T09:33:18.588 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:33:18 smithi191 ceph-mon[36816]: pgmap v67: 129 pgs: 129 active+clean; 64 MiB data, 637 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 911 KiB/s wr, 167 op/s 2024-03-04T09:33:18.631 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:33:18 smithi106 ceph-mon[36484]: pgmap v67: 129 pgs: 129 active+clean; 64 MiB data, 637 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 911 KiB/s wr, 167 op/s 2024-03-04T09:33:18.799 INFO:tasks.workunit.client.0.smithi052.stdout: [CC] doio 2024-03-04T09:33:18.799 INFO:tasks.workunit.client.0.smithi052.stdout: [CC] fsstress 2024-03-04T09:33:18.799 INFO:tasks.workunit.client.0.smithi052.stdout: [CC] fsx 2024-03-04T09:33:18.945 INFO:tasks.workunit.client.0.smithi052.stderr:fsstress.c: In function 'attr_list_path': 2024-03-04T09:33:18.945 INFO:tasks.workunit.client.0.smithi052.stderr:fsstress.c:859:9: warning: 'attr_list' is deprecated: Use listxattr or llistxattr instead [-Wdeprecated-declarations] 2024-03-04T09:33:18.945 INFO:tasks.workunit.client.0.smithi052.stderr: 859 | rval = attr_list(name->path, buffer, buffersize, flags, cursor); 2024-03-04T09:33:18.945 INFO:tasks.workunit.client.0.smithi052.stderr: | ^~~~ 2024-03-04T09:33:18.945 INFO:tasks.workunit.client.0.smithi052.stderr:In file included from ../src/global.h:25, 2024-03-04T09:33:18.945 INFO:tasks.workunit.client.0.smithi052.stderr: from fsstress.c:12: 2024-03-04T09:33:18.945 INFO:tasks.workunit.client.0.smithi052.stderr:/usr/include/attr/attributes.h:165:12: note: declared here 2024-03-04T09:33:18.945 INFO:tasks.workunit.client.0.smithi052.stderr: 165 | extern int attr_list(const char *__path, char *__buffer, const int __buffersize, 2024-03-04T09:33:18.946 INFO:tasks.workunit.client.0.smithi052.stderr: | ^~~~~~~~~ 2024-03-04T09:33:18.946 INFO:tasks.workunit.client.0.smithi052.stderr:fsstress.c: In function 'attr_remove_path': 2024-03-04T09:33:18.946 INFO:tasks.workunit.client.0.smithi052.stderr:fsstress.c:878:9: warning: 'attr_remove' is deprecated: Use removexattr or lremovexattr instead [-Wdeprecated-declarations] 2024-03-04T09:33:18.946 INFO:tasks.workunit.client.0.smithi052.stderr: 878 | rval = attr_remove(name->path, attrname, flags); 2024-03-04T09:33:18.946 INFO:tasks.workunit.client.0.smithi052.stderr: | ^~~~ 2024-03-04T09:33:18.946 INFO:tasks.workunit.client.0.smithi052.stderr:In file included from ../src/global.h:25, 2024-03-04T09:33:18.946 INFO:tasks.workunit.client.0.smithi052.stderr: from fsstress.c:12: 2024-03-04T09:33:18.946 INFO:tasks.workunit.client.0.smithi052.stderr:/usr/include/attr/attributes.h:152:12: note: declared here 2024-03-04T09:33:18.946 INFO:tasks.workunit.client.0.smithi052.stderr: 152 | extern int attr_remove (const char *__path, const char *__attrname, 2024-03-04T09:33:18.946 INFO:tasks.workunit.client.0.smithi052.stderr: | ^~~~~~~~~~~ 2024-03-04T09:33:18.946 INFO:tasks.workunit.client.0.smithi052.stderr:fsstress.c: In function 'attr_set_path': 2024-03-04T09:33:18.946 INFO:tasks.workunit.client.0.smithi052.stderr:fsstress.c:898:9: warning: 'attr_set' is deprecated: Use setxattr or lsetxattr instead [-Wdeprecated-declarations] 2024-03-04T09:33:18.946 INFO:tasks.workunit.client.0.smithi052.stderr: 898 | rval = attr_set(name->path, attrname, attrvalue, valuelength, flags); 2024-03-04T09:33:18.946 INFO:tasks.workunit.client.0.smithi052.stderr: | ^~~~ 2024-03-04T09:33:18.946 INFO:tasks.workunit.client.0.smithi052.stderr:In file included from ../src/global.h:25, 2024-03-04T09:33:18.947 INFO:tasks.workunit.client.0.smithi052.stderr: from fsstress.c:12: 2024-03-04T09:33:18.947 INFO:tasks.workunit.client.0.smithi052.stderr:/usr/include/attr/attributes.h:139:12: note: declared here 2024-03-04T09:33:18.947 INFO:tasks.workunit.client.0.smithi052.stderr: 139 | extern int attr_set (const char *__path, const char *__attrname, 2024-03-04T09:33:18.947 INFO:tasks.workunit.client.0.smithi052.stderr: | ^~~~~~~~ 2024-03-04T09:33:18.956 INFO:tasks.workunit.client.0.smithi052.stderr:fsstress.c: In function 'allocsp_f': 2024-03-04T09:33:18.956 INFO:tasks.workunit.client.0.smithi052.stderr:fsstress.c:2047:32: error: 'XFS_IOC_ALLOCSP64' undeclared (first use in this function); did you mean 'XFS_IOC_RESVSP64'? 2024-03-04T09:33:18.956 INFO:tasks.workunit.client.0.smithi052.stderr: 2047 | e = xfsctl(f.path, fd, XFS_IOC_ALLOCSP64, &fl) < 0 ? errno : 0; 2024-03-04T09:33:18.956 INFO:tasks.workunit.client.0.smithi052.stderr: | ^~~~~~~~~~~~~~~~~ 2024-03-04T09:33:18.956 INFO:tasks.workunit.client.0.smithi052.stderr: | XFS_IOC_RESVSP64 2024-03-04T09:33:18.956 INFO:tasks.workunit.client.0.smithi052.stderr:fsstress.c:2047:32: note: each undeclared identifier is reported only once for each function it appears in 2024-03-04T09:33:18.978 INFO:tasks.workunit.client.0.smithi052.stderr:fsstress.c: In function 'freesp_f': 2024-03-04T09:33:18.979 INFO:tasks.workunit.client.0.smithi052.stderr:fsstress.c:3640:32: error: 'XFS_IOC_FREESP64' undeclared (first use in this function); did you mean 'XFS_IOC_RESVSP64'? 2024-03-04T09:33:18.979 INFO:tasks.workunit.client.0.smithi052.stderr: 3640 | e = xfsctl(f.path, fd, XFS_IOC_FREESP64, &fl) < 0 ? errno : 0; 2024-03-04T09:33:18.979 INFO:tasks.workunit.client.0.smithi052.stderr: | ^~~~~~~~~~~~~~~~ 2024-03-04T09:33:18.979 INFO:tasks.workunit.client.0.smithi052.stderr: | XFS_IOC_RESVSP64 2024-03-04T09:33:19.012 INFO:tasks.workunit.client.0.smithi052.stderr:gmake[3]: *** [Makefile:47: fsstress] Error 1 2024-03-04T09:33:19.012 INFO:tasks.workunit.client.0.smithi052.stderr:gmake[3]: *** Waiting for unfinished jobs.... 2024-03-04T09:33:19.499 INFO:tasks.workunit.client.0.smithi052.stderr:doio.c: In function 'parse_cmdline': 2024-03-04T09:33:19.499 INFO:tasks.workunit.client.0.smithi052.stderr:doio.c:3113:33: warning: '%s' directive argument is null [-Wformat-overflow=] 2024-03-04T09:33:19.499 INFO:tasks.workunit.client.0.smithi052.stderr: 3113 | fprintf(stderr, 2024-03-04T09:33:19.499 INFO:tasks.workunit.client.0.smithi052.stderr: | ^~~~~~~~~~~~~~~ 2024-03-04T09:33:19.499 INFO:tasks.workunit.client.0.smithi052.stderr: 3114 | "%s%s: Illegal -C arg (%s). Must be one of: ", 2024-03-04T09:33:19.499 INFO:tasks.workunit.client.0.smithi052.stderr: | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-03-04T09:33:19.499 INFO:tasks.workunit.client.0.smithi052.stderr: 3115 | Prog, TagName, tok); 2024-03-04T09:33:19.499 INFO:tasks.workunit.client.0.smithi052.stderr: | ~~~~~~~~~~~~~~~~~~~ 2024-03-04T09:33:19.521 INFO:tasks.workunit.client.0.smithi052.stderr:fsx.c: In function 'main': 2024-03-04T09:33:19.521 INFO:tasks.workunit.client.0.smithi052.stderr:fsx.c:2662:33: warning: 'strncpy' specified bound 4096 equals destination size [-Wstringop-truncation] 2024-03-04T09:33:19.521 INFO:tasks.workunit.client.0.smithi052.stderr: 2662 | strncpy(opsfile, optarg, sizeof(opsfile)); 2024-03-04T09:33:19.521 INFO:tasks.workunit.client.0.smithi052.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-03-04T09:33:19.521 INFO:tasks.workunit.client.0.smithi052.stderr:fsx.c:2632:25: warning: 'strncpy' specified bound 1024 equals destination size [-Wstringop-truncation] 2024-03-04T09:33:19.521 INFO:tasks.workunit.client.0.smithi052.stderr: 2632 | strncpy(dname, optarg, sizeof(dname)); 2024-03-04T09:33:19.521 INFO:tasks.workunit.client.0.smithi052.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-03-04T09:33:19.618 INFO:tasks.workunit.client.0.smithi052.stderr:gmake[2]: *** [include/buildrules:30: ltp] Error 2 2024-03-04T09:33:19.618 INFO:tasks.workunit.client.0.smithi052.stderr:gmake[2]: *** Waiting for unfinished jobs.... 2024-03-04T09:33:20.202 INFO:tasks.workunit.client.0.smithi052.stdout: [DEP] 2024-03-04T09:33:20.202 INFO:tasks.workunit.client.0.smithi052.stdout: [DEP] 2024-03-04T09:33:20.204 INFO:tasks.workunit.client.0.smithi052.stdout: [DEP] 2024-03-04T09:33:20.422 INFO:tasks.workunit.client.0.smithi052.stdout: [CC] dirstress 2024-03-04T09:33:20.424 INFO:tasks.workunit.client.0.smithi052.stdout: [CC] fill 2024-03-04T09:33:20.424 INFO:tasks.workunit.client.0.smithi052.stdout: [CC] fill2 2024-03-04T09:33:20.425 INFO:tasks.workunit.client.0.smithi052.stdout: [CC] getpagesize 2024-03-04T09:33:20.568 INFO:tasks.workunit.client.0.smithi052.stdout: [CC] holes 2024-03-04T09:33:20.570 INFO:tasks.workunit.client.0.smithi052.stdout: [CC] lstat64 2024-03-04T09:33:20.584 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:33:20 smithi052 ceph-mon[32473]: pgmap v68: 129 pgs: 129 active+clean; 64 MiB data, 637 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 911 KiB/s wr, 167 op/s 2024-03-04T09:33:20.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:33:20 smithi191 ceph-mon[36816]: pgmap v68: 129 pgs: 129 active+clean; 64 MiB data, 637 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 911 KiB/s wr, 167 op/s 2024-03-04T09:33:20.631 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:33:20 smithi106 ceph-mon[36484]: pgmap v68: 129 pgs: 129 active+clean; 64 MiB data, 637 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 911 KiB/s wr, 167 op/s 2024-03-04T09:33:20.664 INFO:tasks.workunit.client.0.smithi052.stdout: [CC] nametest 2024-03-04T09:33:20.681 INFO:tasks.workunit.client.0.smithi052.stdout: [CC] permname 2024-03-04T09:33:20.765 INFO:tasks.workunit.client.0.smithi052.stdout: [CC] randholes 2024-03-04T09:33:20.787 INFO:tasks.workunit.client.0.smithi052.stdout: [CC] runas 2024-03-04T09:33:20.875 INFO:tasks.workunit.client.0.smithi052.stdout: [CC] truncfile 2024-03-04T09:33:20.925 INFO:tasks.workunit.client.0.smithi052.stdout: [CC] usemem 2024-03-04T09:33:20.964 INFO:tasks.workunit.client.0.smithi052.stdout: [CC] mmapcat 2024-03-04T09:33:21.015 INFO:tasks.workunit.client.0.smithi052.stdout: [CC] append_reader 2024-03-04T09:33:21.051 INFO:tasks.workunit.client.0.smithi052.stdout: [CC] append_writer 2024-03-04T09:33:21.067 INFO:tasks.workunit.client.0.smithi052.stdout: [CC] dirperf 2024-03-04T09:33:21.122 INFO:tasks.workunit.client.0.smithi052.stdout: [CC] metaperf 2024-03-04T09:33:21.178 INFO:tasks.workunit.client.0.smithi052.stdout: [CC] devzero 2024-03-04T09:33:21.192 INFO:tasks.workunit.client.0.smithi052.stdout: [CC] feature 2024-03-04T09:33:21.316 INFO:tasks.workunit.client.0.smithi052.stdout: [CC] alloc 2024-03-04T09:33:21.346 INFO:tasks.workunit.client.0.smithi052.stdout: [CC] fault 2024-03-04T09:33:21.388 INFO:tasks.workunit.client.0.smithi052.stdout: [CC] fstest 2024-03-04T09:33:21.441 INFO:tasks.workunit.client.0.smithi052.stderr:alloc.c: In function 'main': 2024-03-04T09:33:21.441 INFO:tasks.workunit.client.0.smithi052.stderr:alloc.c:154:1: error: 'bozo' undeclared (first use in this function) 2024-03-04T09:33:21.441 INFO:tasks.workunit.client.0.smithi052.stderr: 154 | bozo! 2024-03-04T09:33:21.441 INFO:tasks.workunit.client.0.smithi052.stderr: | ^~~~ 2024-03-04T09:33:21.441 INFO:tasks.workunit.client.0.smithi052.stderr:alloc.c:154:1: note: each undeclared identifier is reported only once for each function it appears in 2024-03-04T09:33:21.441 INFO:tasks.workunit.client.0.smithi052.stderr:alloc.c:154:5: error: expected ';' before '!' token 2024-03-04T09:33:21.442 INFO:tasks.workunit.client.0.smithi052.stderr: 154 | bozo! 2024-03-04T09:33:21.442 INFO:tasks.workunit.client.0.smithi052.stderr: | ^ 2024-03-04T09:33:21.442 INFO:tasks.workunit.client.0.smithi052.stderr: | ; 2024-03-04T09:33:21.442 INFO:tasks.workunit.client.0.smithi052.stderr:alloc.c:184:25: error: 'rflag' undeclared (first use in this function); did you mean 'nflag'? 2024-03-04T09:33:21.442 INFO:tasks.workunit.client.0.smithi052.stderr: 184 | rflag = 1; 2024-03-04T09:33:21.442 INFO:tasks.workunit.client.0.smithi052.stderr: | ^~~~~ 2024-03-04T09:33:21.442 INFO:tasks.workunit.client.0.smithi052.stderr: | nflag 2024-03-04T09:33:21.443 INFO:tasks.workunit.client.0.smithi052.stderr:alloc.c:320:5: error: expected ';' before '!' token 2024-03-04T09:33:21.443 INFO:tasks.workunit.client.0.smithi052.stderr: 320 | bozo! 2024-03-04T09:33:21.443 INFO:tasks.workunit.client.0.smithi052.stderr: | ^ 2024-03-04T09:33:21.443 INFO:tasks.workunit.client.0.smithi052.stderr: | ; 2024-03-04T09:33:21.443 INFO:tasks.workunit.client.0.smithi052.stderr:alloc.c:128:25: warning: variable 'f' set but not used [-Wunused-but-set-variable] 2024-03-04T09:33:21.443 INFO:tasks.workunit.client.0.smithi052.stderr: 128 | struct flock64 f; 2024-03-04T09:33:21.443 INFO:tasks.workunit.client.0.smithi052.stderr: | ^ 2024-03-04T09:33:21.445 INFO:tasks.workunit.client.0.smithi052.stderr:gmake[3]: *** [Makefile:83: alloc] Error 1 2024-03-04T09:33:21.445 INFO:tasks.workunit.client.0.smithi052.stderr:gmake[3]: *** Waiting for unfinished jobs.... 2024-03-04T09:33:21.535 INFO:tasks.workunit.client.0.smithi052.stderr:fstest.c: In function 'run_child': 2024-03-04T09:33:21.535 INFO:tasks.workunit.client.0.smithi052.stderr:fstest.c:95:27: warning: '/file' directive writing 5 bytes into a region of size between 1 and 1024 [-Wformat-overflow=] 2024-03-04T09:33:21.535 INFO:tasks.workunit.client.0.smithi052.stderr: 95 | sprintf(fname, "%s/file%d", dir, fnum); 2024-03-04T09:33:21.535 INFO:tasks.workunit.client.0.smithi052.stderr: | ^~~~~ 2024-03-04T09:33:21.535 INFO:tasks.workunit.client.0.smithi052.stderr:fstest.c:95:24: note: directive argument in the range [0, 2147483647] 2024-03-04T09:33:21.535 INFO:tasks.workunit.client.0.smithi052.stderr: 95 | sprintf(fname, "%s/file%d", dir, fnum); 2024-03-04T09:33:21.536 INFO:tasks.workunit.client.0.smithi052.stderr: | ^~~~~~~~~~~ 2024-03-04T09:33:21.536 INFO:tasks.workunit.client.0.smithi052.stderr:fstest.c:95:9: note: 'sprintf' output between 7 and 1039 bytes into a destination of size 1024 2024-03-04T09:33:21.536 INFO:tasks.workunit.client.0.smithi052.stderr: 95 | sprintf(fname, "%s/file%d", dir, fnum); 2024-03-04T09:33:21.536 INFO:tasks.workunit.client.0.smithi052.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-03-04T09:33:21.536 INFO:tasks.workunit.client.0.smithi052.stderr:fstest.c:166:27: warning: '/file' directive writing 5 bytes into a region of size between 1 and 1024 [-Wformat-overflow=] 2024-03-04T09:33:21.536 INFO:tasks.workunit.client.0.smithi052.stderr: 166 | sprintf(fname, "%s/file%d", dir, fnum); 2024-03-04T09:33:21.536 INFO:tasks.workunit.client.0.smithi052.stderr: | ^~~~~ 2024-03-04T09:33:21.536 INFO:tasks.workunit.client.0.smithi052.stderr:fstest.c:166:24: note: directive argument in the range [0, 2147483647] 2024-03-04T09:33:21.536 INFO:tasks.workunit.client.0.smithi052.stderr: 166 | sprintf(fname, "%s/file%d", dir, fnum); 2024-03-04T09:33:21.536 INFO:tasks.workunit.client.0.smithi052.stderr: | ^~~~~~~~~~~ 2024-03-04T09:33:21.536 INFO:tasks.workunit.client.0.smithi052.stderr:fstest.c:166:9: note: 'sprintf' output between 7 and 1039 bytes into a destination of size 1024 2024-03-04T09:33:21.536 INFO:tasks.workunit.client.0.smithi052.stderr: 166 | sprintf(fname, "%s/file%d", dir, fnum); 2024-03-04T09:33:21.536 INFO:tasks.workunit.client.0.smithi052.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-03-04T09:33:21.603 INFO:tasks.workunit.client.0.smithi052.stderr:gmake[2]: *** [include/buildrules:30: src] Error 2 2024-03-04T09:33:21.604 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-03-04T09:33:21.605 INFO:tasks.workunit.client.0.smithi052.stderr:make[1]: *** [Makefile:53: default] Error 2 2024-03-04T09:33:21.605 INFO:tasks.workunit.client.0.smithi052.stderr:make: *** [Makefile:51: default] Error 2 2024-03-04T09:33:21.605 INFO:tasks.workunit:Stopping ['suites/fsx.sh'] on client.0... 2024-03-04T09:33:21.605 DEBUG:teuthology.orchestra.run.smithi052:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-03-04T09:33:21.929 ERROR:teuthology.run_tasks:Saw exception from tasks. Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/run_tasks.py", line 105, in run_tasks manager = run_one_task(taskname, ctx=ctx, config=config) File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/run_tasks.py", line 83, in run_one_task return task(**kwargs) File "/home/teuthworker/src/git.ceph.com_ceph-c_1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd/qa/tasks/workunit.py", line 145, in task _spawn_on_all_clients(ctx, refspec, all_tasks, config.get('env'), File "/home/teuthworker/src/git.ceph.com_ceph-c_1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd/qa/tasks/workunit.py", line 295, in _spawn_on_all_clients p.spawn(_run_tests, ctx, refspec, role, [unit], env, File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/parallel.py", line 84, in __exit__ for result in self: File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/parallel.py", line 98, in __next__ resurrect_traceback(result) File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/parallel.py", line 30, in resurrect_traceback raise exc.exc_info[1] File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/parallel.py", line 23, in capture_traceback return func(*args, **kwargs) File "/home/teuthworker/src/git.ceph.com_ceph-c_1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd/qa/tasks/workunit.py", line 434, in _run_tests remote.run( File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/orchestra/remote.py", line 523, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed (workunit test suites/fsx.sh) on smithi052 with status 2: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/fsx.sh' 2024-03-04T09:33:22.185 ERROR:teuthology.util.sentry: Sentry event: https://sentry.ceph.com/organizations/ceph/?query=8b3bd6f693264883bb0f493cf8aa771f Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/run_tasks.py", line 105, in run_tasks manager = run_one_task(taskname, ctx=ctx, config=config) File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/run_tasks.py", line 83, in run_one_task return task(**kwargs) File "/home/teuthworker/src/git.ceph.com_ceph-c_1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd/qa/tasks/workunit.py", line 145, in task _spawn_on_all_clients(ctx, refspec, all_tasks, config.get('env'), File "/home/teuthworker/src/git.ceph.com_ceph-c_1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd/qa/tasks/workunit.py", line 295, in _spawn_on_all_clients p.spawn(_run_tests, ctx, refspec, role, [unit], env, File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/parallel.py", line 84, in __exit__ for result in self: File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/parallel.py", line 98, in __next__ resurrect_traceback(result) File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/parallel.py", line 30, in resurrect_traceback raise exc.exc_info[1] File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/parallel.py", line 23, in capture_traceback return func(*args, **kwargs) File "/home/teuthworker/src/git.ceph.com_ceph-c_1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd/qa/tasks/workunit.py", line 434, in _run_tests remote.run( File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/orchestra/remote.py", line 523, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed (workunit test suites/fsx.sh) on smithi052 with status 2: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/fsx.sh' 2024-03-04T09:33:22.188 DEBUG:teuthology.run_tasks:Unwinding manager full_sequential_finally 2024-03-04T09:33:22.197 INFO:teuthology.task.full_sequential_finally:In full_sequential_finally, running task exec... 2024-03-04T09:33:22.198 INFO:teuthology.task.exec:Executing custom commands... 2024-03-04T09:33:22.198 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi052.front.sepia.ceph.com 2024-03-04T09:33:22.198 DEBUG:teuthology.orchestra.run.smithi052:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > END_TIME' 2024-03-04T09:33:22.243 DEBUG:teuthology.orchestra.run.smithi052:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi' 2024-03-04T09:33:22.585 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:33:22 smithi052 ceph-mon[32473]: pgmap v69: 129 pgs: 129 active+clean; 80 MiB data, 726 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.5 MiB/s wr, 390 op/s 2024-03-04T09:33:22.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:33:22 smithi191 ceph-mon[36816]: pgmap v69: 129 pgs: 129 active+clean; 80 MiB data, 726 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.5 MiB/s wr, 390 op/s 2024-03-04T09:33:22.631 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:33:22 smithi106 ceph-mon[36484]: pgmap v69: 129 pgs: 129 active+clean; 80 MiB data, 726 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.5 MiB/s wr, 390 op/s 2024-03-04T09:33:24.585 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:33:24 smithi052 ceph-mon[32473]: pgmap v70: 129 pgs: 129 active+clean; 80 MiB data, 726 MiB used, 1.0 TiB / 1.0 TiB avail; 800 KiB/s rd, 1.8 MiB/s wr, 238 op/s 2024-03-04T09:33:24.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:33:24 smithi191 ceph-mon[36816]: pgmap v70: 129 pgs: 129 active+clean; 80 MiB data, 726 MiB used, 1.0 TiB / 1.0 TiB avail; 800 KiB/s rd, 1.8 MiB/s wr, 238 op/s 2024-03-04T09:33:24.631 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:33:24 smithi106 ceph-mon[36484]: pgmap v70: 129 pgs: 129 active+clean; 80 MiB data, 726 MiB used, 1.0 TiB / 1.0 TiB avail; 800 KiB/s rd, 1.8 MiB/s wr, 238 op/s 2024-03-04T09:33:26.585 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:33:26 smithi052 ceph-mon[32473]: pgmap v71: 129 pgs: 129 active+clean; 80 MiB data, 724 MiB used, 1.0 TiB / 1.0 TiB avail; 834 KiB/s rd, 1.8 MiB/s wr, 243 op/s 2024-03-04T09:33:26.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:33:26 smithi191 ceph-mon[36816]: pgmap v71: 129 pgs: 129 active+clean; 80 MiB data, 724 MiB used, 1.0 TiB / 1.0 TiB avail; 834 KiB/s rd, 1.8 MiB/s wr, 243 op/s 2024-03-04T09:33:26.631 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:33:26 smithi106 ceph-mon[36484]: pgmap v71: 129 pgs: 129 active+clean; 80 MiB data, 724 MiB used, 1.0 TiB / 1.0 TiB avail; 834 KiB/s rd, 1.8 MiB/s wr, 243 op/s 2024-03-04T09:33:28.585 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:33:28 smithi052 ceph-mon[32473]: pgmap v72: 129 pgs: 129 active+clean; 80 MiB data, 724 MiB used, 1.0 TiB / 1.0 TiB avail; 602 KiB/s rd, 1.6 MiB/s wr, 228 op/s 2024-03-04T09:33:28.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:33:28 smithi191 ceph-mon[36816]: pgmap v72: 129 pgs: 129 active+clean; 80 MiB data, 724 MiB used, 1.0 TiB / 1.0 TiB avail; 602 KiB/s rd, 1.6 MiB/s wr, 228 op/s 2024-03-04T09:33:28.631 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:33:28 smithi106 ceph-mon[36484]: pgmap v72: 129 pgs: 129 active+clean; 80 MiB data, 724 MiB used, 1.0 TiB / 1.0 TiB avail; 602 KiB/s rd, 1.6 MiB/s wr, 228 op/s 2024-03-04T09:33:30.585 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:33:30 smithi052 ceph-mon[32473]: pgmap v73: 129 pgs: 129 active+clean; 80 MiB data, 724 MiB used, 1.0 TiB / 1.0 TiB avail; 602 KiB/s rd, 1.6 MiB/s wr, 228 op/s 2024-03-04T09:33:30.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:33:30 smithi191 ceph-mon[36816]: pgmap v73: 129 pgs: 129 active+clean; 80 MiB data, 724 MiB used, 1.0 TiB / 1.0 TiB avail; 602 KiB/s rd, 1.6 MiB/s wr, 228 op/s 2024-03-04T09:33:30.631 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:33:30 smithi106 ceph-mon[36484]: pgmap v73: 129 pgs: 129 active+clean; 80 MiB data, 724 MiB used, 1.0 TiB / 1.0 TiB avail; 602 KiB/s rd, 1.6 MiB/s wr, 228 op/s 2024-03-04T09:33:32.585 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:33:32 smithi052 ceph-mon[32473]: pgmap v74: 129 pgs: 129 active+clean; 80 MiB data, 724 MiB used, 1.0 TiB / 1.0 TiB avail; 606 KiB/s rd, 1.6 MiB/s wr, 233 op/s 2024-03-04T09:33:32.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:33:32 smithi191 ceph-mon[36816]: pgmap v74: 129 pgs: 129 active+clean; 80 MiB data, 724 MiB used, 1.0 TiB / 1.0 TiB avail; 606 KiB/s rd, 1.6 MiB/s wr, 233 op/s 2024-03-04T09:33:32.631 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:33:32 smithi106 ceph-mon[36484]: pgmap v74: 129 pgs: 129 active+clean; 80 MiB data, 724 MiB used, 1.0 TiB / 1.0 TiB avail; 606 KiB/s rd, 1.6 MiB/s wr, 233 op/s 2024-03-04T09:33:34.585 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:33:34 smithi052 ceph-mon[32473]: pgmap v75: 129 pgs: 129 active+clean; 80 MiB data, 724 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 37 KiB/s wr, 9 op/s 2024-03-04T09:33:34.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:33:34 smithi191 ceph-mon[36816]: pgmap v75: 129 pgs: 129 active+clean; 80 MiB data, 724 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 37 KiB/s wr, 9 op/s 2024-03-04T09:33:34.631 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:33:34 smithi106 ceph-mon[36484]: pgmap v75: 129 pgs: 129 active+clean; 80 MiB data, 724 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 37 KiB/s wr, 9 op/s 2024-03-04T09:33:36.585 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:33:36 smithi052 ceph-mon[32473]: pgmap v76: 129 pgs: 129 active+clean; 80 MiB data, 724 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 38 KiB/s wr, 13 op/s 2024-03-04T09:33:36.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:33:36 smithi191 ceph-mon[36816]: pgmap v76: 129 pgs: 129 active+clean; 80 MiB data, 724 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 38 KiB/s wr, 13 op/s 2024-03-04T09:33:36.631 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:33:36 smithi106 ceph-mon[36484]: pgmap v76: 129 pgs: 129 active+clean; 80 MiB data, 724 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 38 KiB/s wr, 13 op/s 2024-03-04T09:33:38.585 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:33:38 smithi052 ceph-mon[32473]: pgmap v77: 129 pgs: 129 active+clean; 80 MiB data, 724 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 5.1 KiB/s wr, 8 op/s 2024-03-04T09:33:38.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:33:38 smithi191 ceph-mon[36816]: pgmap v77: 129 pgs: 129 active+clean; 80 MiB data, 724 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 5.1 KiB/s wr, 8 op/s 2024-03-04T09:33:38.631 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:33:38 smithi106 ceph-mon[36484]: pgmap v77: 129 pgs: 129 active+clean; 80 MiB data, 724 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 5.1 KiB/s wr, 8 op/s 2024-03-04T09:33:40.585 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:33:40 smithi052 ceph-mon[32473]: pgmap v78: 129 pgs: 129 active+clean; 80 MiB data, 724 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 5.1 KiB/s wr, 8 op/s 2024-03-04T09:33:40.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:33:40 smithi191 ceph-mon[36816]: pgmap v78: 129 pgs: 129 active+clean; 80 MiB data, 724 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 5.1 KiB/s wr, 8 op/s 2024-03-04T09:33:40.631 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:33:40 smithi106 ceph-mon[36484]: pgmap v78: 129 pgs: 129 active+clean; 80 MiB data, 724 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 5.1 KiB/s wr, 8 op/s 2024-03-04T09:33:42.585 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:33:42 smithi052 ceph-mon[32473]: pgmap v79: 129 pgs: 129 active+clean; 81 MiB data, 724 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 101 KiB/s wr, 13 op/s 2024-03-04T09:33:42.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:33:42 smithi191 ceph-mon[36816]: pgmap v79: 129 pgs: 129 active+clean; 81 MiB data, 724 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 101 KiB/s wr, 13 op/s 2024-03-04T09:33:42.631 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:33:42 smithi106 ceph-mon[36484]: pgmap v79: 129 pgs: 129 active+clean; 81 MiB data, 724 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 101 KiB/s wr, 13 op/s 2024-03-04T09:33:44.585 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:33:44 smithi052 ceph-mon[32473]: pgmap v80: 129 pgs: 129 active+clean; 81 MiB data, 724 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 98 KiB/s wr, 8 op/s 2024-03-04T09:33:44.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:33:44 smithi191 ceph-mon[36816]: pgmap v80: 129 pgs: 129 active+clean; 81 MiB data, 724 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 98 KiB/s wr, 8 op/s 2024-03-04T09:33:44.631 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:33:44 smithi106 ceph-mon[36484]: pgmap v80: 129 pgs: 129 active+clean; 81 MiB data, 724 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 98 KiB/s wr, 8 op/s 2024-03-04T09:33:46.585 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:33:46 smithi052 ceph-mon[32473]: pgmap v81: 129 pgs: 129 active+clean; 81 MiB data, 726 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 98 KiB/s wr, 11 op/s 2024-03-04T09:33:46.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:33:46 smithi191 ceph-mon[36816]: pgmap v81: 129 pgs: 129 active+clean; 81 MiB data, 726 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 98 KiB/s wr, 11 op/s 2024-03-04T09:33:46.631 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:33:46 smithi106 ceph-mon[36484]: pgmap v81: 129 pgs: 129 active+clean; 81 MiB data, 726 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 98 KiB/s wr, 11 op/s 2024-03-04T09:33:48.585 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:33:48 smithi052 ceph-mon[32473]: pgmap v82: 129 pgs: 129 active+clean; 81 MiB data, 726 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 96 KiB/s wr, 8 op/s 2024-03-04T09:33:48.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:33:48 smithi191 ceph-mon[36816]: pgmap v82: 129 pgs: 129 active+clean; 81 MiB data, 726 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 96 KiB/s wr, 8 op/s 2024-03-04T09:33:48.631 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:33:48 smithi106 ceph-mon[36484]: pgmap v82: 129 pgs: 129 active+clean; 81 MiB data, 726 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 96 KiB/s wr, 8 op/s 2024-03-04T09:33:50.585 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:33:50 smithi052 ceph-mon[32473]: pgmap v83: 129 pgs: 129 active+clean; 81 MiB data, 726 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 96 KiB/s wr, 8 op/s 2024-03-04T09:33:50.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:33:50 smithi191 ceph-mon[36816]: pgmap v83: 129 pgs: 129 active+clean; 81 MiB data, 726 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 96 KiB/s wr, 8 op/s 2024-03-04T09:33:50.631 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:33:50 smithi106 ceph-mon[36484]: pgmap v83: 129 pgs: 129 active+clean; 81 MiB data, 726 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 96 KiB/s wr, 8 op/s 2024-03-04T09:33:52.585 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:33:52 smithi052 ceph-mon[32473]: pgmap v84: 129 pgs: 129 active+clean; 81 MiB data, 726 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 99 KiB/s wr, 11 op/s 2024-03-04T09:33:52.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:33:52 smithi191 ceph-mon[36816]: pgmap v84: 129 pgs: 129 active+clean; 81 MiB data, 726 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 99 KiB/s wr, 11 op/s 2024-03-04T09:33:52.631 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:33:52 smithi106 ceph-mon[36484]: pgmap v84: 129 pgs: 129 active+clean; 81 MiB data, 726 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 99 KiB/s wr, 11 op/s 2024-03-04T09:33:54.585 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:33:54 smithi052 ceph-mon[32473]: pgmap v85: 129 pgs: 129 active+clean; 81 MiB data, 726 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 3.4 KiB/s wr, 6 op/s 2024-03-04T09:33:54.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:33:54 smithi191 ceph-mon[36816]: pgmap v85: 129 pgs: 129 active+clean; 81 MiB data, 726 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 3.4 KiB/s wr, 6 op/s 2024-03-04T09:33:54.631 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:33:54 smithi106 ceph-mon[36484]: pgmap v85: 129 pgs: 129 active+clean; 81 MiB data, 726 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 3.4 KiB/s wr, 6 op/s 2024-03-04T09:33:56.585 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:33:56 smithi052 ceph-mon[32473]: pgmap v86: 129 pgs: 129 active+clean; 81 MiB data, 726 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 3.6 KiB/s wr, 8 op/s 2024-03-04T09:33:56.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:33:56 smithi191 ceph-mon[36816]: pgmap v86: 129 pgs: 129 active+clean; 81 MiB data, 726 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 3.6 KiB/s wr, 8 op/s 2024-03-04T09:33:56.631 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:33:56 smithi106 ceph-mon[36484]: pgmap v86: 129 pgs: 129 active+clean; 81 MiB data, 726 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 3.6 KiB/s wr, 8 op/s 2024-03-04T09:33:58.585 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:33:58 smithi052 ceph-mon[32473]: pgmap v87: 129 pgs: 129 active+clean; 81 MiB data, 726 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 3.3 KiB/s wr, 5 op/s 2024-03-04T09:33:58.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:33:58 smithi191 ceph-mon[36816]: pgmap v87: 129 pgs: 129 active+clean; 81 MiB data, 726 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 3.3 KiB/s wr, 5 op/s 2024-03-04T09:33:58.631 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:33:58 smithi106 ceph-mon[36484]: pgmap v87: 129 pgs: 129 active+clean; 81 MiB data, 726 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 3.3 KiB/s wr, 5 op/s 2024-03-04T09:34:00.585 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:34:00 smithi052 ceph-mon[32473]: pgmap v88: 129 pgs: 129 active+clean; 81 MiB data, 726 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 3.3 KiB/s wr, 5 op/s 2024-03-04T09:34:00.592 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:34:00 smithi191 ceph-mon[36816]: pgmap v88: 129 pgs: 129 active+clean; 81 MiB data, 726 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 3.3 KiB/s wr, 5 op/s 2024-03-04T09:34:00.631 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:34:00 smithi106 ceph-mon[36484]: pgmap v88: 129 pgs: 129 active+clean; 81 MiB data, 726 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 3.3 KiB/s wr, 5 op/s 2024-03-04T09:34:02.585 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:34:02 smithi052 ceph-mon[32473]: pgmap v89: 129 pgs: 129 active+clean; 81 MiB data, 726 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 4.0 KiB/s wr, 8 op/s 2024-03-04T09:34:02.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:34:02 smithi191 ceph-mon[36816]: pgmap v89: 129 pgs: 129 active+clean; 81 MiB data, 726 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 4.0 KiB/s wr, 8 op/s 2024-03-04T09:34:02.631 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:34:02 smithi106 ceph-mon[36484]: pgmap v89: 129 pgs: 129 active+clean; 81 MiB data, 726 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 4.0 KiB/s wr, 8 op/s 2024-03-04T09:34:04.585 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:34:04 smithi052 ceph-mon[32473]: pgmap v90: 129 pgs: 129 active+clean; 81 MiB data, 726 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 853 B/s wr, 5 op/s 2024-03-04T09:34:04.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:34:04 smithi191 ceph-mon[36816]: pgmap v90: 129 pgs: 129 active+clean; 81 MiB data, 726 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 853 B/s wr, 5 op/s 2024-03-04T09:34:04.631 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:34:04 smithi106 ceph-mon[36484]: pgmap v90: 129 pgs: 129 active+clean; 81 MiB data, 726 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 853 B/s wr, 5 op/s 2024-03-04T09:34:06.585 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:34:06 smithi052 ceph-mon[32473]: pgmap v91: 129 pgs: 129 active+clean; 81 MiB data, 731 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 26 KiB/s wr, 22 op/s 2024-03-04T09:34:06.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:34:06 smithi191 ceph-mon[36816]: pgmap v91: 129 pgs: 129 active+clean; 81 MiB data, 731 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 26 KiB/s wr, 22 op/s 2024-03-04T09:34:06.631 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:34:06 smithi106 ceph-mon[36484]: pgmap v91: 129 pgs: 129 active+clean; 81 MiB data, 731 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 26 KiB/s wr, 22 op/s 2024-03-04T09:34:08.585 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:34:08 smithi052 ceph-mon[32473]: pgmap v92: 129 pgs: 129 active+clean; 81 MiB data, 731 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 26 KiB/s wr, 20 op/s 2024-03-04T09:34:08.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:34:08 smithi191 ceph-mon[36816]: pgmap v92: 129 pgs: 129 active+clean; 81 MiB data, 731 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 26 KiB/s wr, 20 op/s 2024-03-04T09:34:08.631 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:34:08 smithi106 ceph-mon[36484]: pgmap v92: 129 pgs: 129 active+clean; 81 MiB data, 731 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 26 KiB/s wr, 20 op/s 2024-03-04T09:34:10.585 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:34:10 smithi052 ceph-mon[32473]: pgmap v93: 129 pgs: 129 active+clean; 81 MiB data, 731 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 26 KiB/s wr, 20 op/s 2024-03-04T09:34:10.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:34:10 smithi191 ceph-mon[36816]: pgmap v93: 129 pgs: 129 active+clean; 81 MiB data, 731 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 26 KiB/s wr, 20 op/s 2024-03-04T09:34:10.631 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:34:10 smithi106 ceph-mon[36484]: pgmap v93: 129 pgs: 129 active+clean; 81 MiB data, 731 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 26 KiB/s wr, 20 op/s 2024-03-04T09:34:12.585 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:34:12 smithi052 ceph-mon[32473]: pgmap v94: 129 pgs: 129 active+clean; 81 MiB data, 731 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 27 KiB/s wr, 23 op/s 2024-03-04T09:34:12.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:34:12 smithi191 ceph-mon[36816]: pgmap v94: 129 pgs: 129 active+clean; 81 MiB data, 731 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 27 KiB/s wr, 23 op/s 2024-03-04T09:34:12.631 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:34:12 smithi106 ceph-mon[36484]: pgmap v94: 129 pgs: 129 active+clean; 81 MiB data, 731 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 27 KiB/s wr, 23 op/s 2024-03-04T09:34:14.585 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:34:14 smithi052 ceph-mon[32473]: pgmap v95: 129 pgs: 129 active+clean; 81 MiB data, 731 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 27 KiB/s wr, 20 op/s 2024-03-04T09:34:14.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:34:14 smithi191 ceph-mon[36816]: pgmap v95: 129 pgs: 129 active+clean; 81 MiB data, 731 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 27 KiB/s wr, 20 op/s 2024-03-04T09:34:14.631 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:34:14 smithi106 ceph-mon[36484]: pgmap v95: 129 pgs: 129 active+clean; 81 MiB data, 731 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 27 KiB/s wr, 20 op/s 2024-03-04T09:34:16.585 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:34:16 smithi052 ceph-mon[32473]: pgmap v96: 129 pgs: 129 active+clean; 81 MiB data, 731 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 27 KiB/s wr, 23 op/s 2024-03-04T09:34:16.585 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:34:16 smithi052 ceph-mon[32473]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T09:34:16.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:34:16 smithi191 ceph-mon[36816]: pgmap v96: 129 pgs: 129 active+clean; 81 MiB data, 731 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 27 KiB/s wr, 23 op/s 2024-03-04T09:34:16.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:34:16 smithi191 ceph-mon[36816]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T09:34:16.631 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:34:16 smithi106 ceph-mon[36484]: pgmap v96: 129 pgs: 129 active+clean; 81 MiB data, 731 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 27 KiB/s wr, 23 op/s 2024-03-04T09:34:16.631 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:34:16 smithi106 ceph-mon[36484]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T09:34:17.585 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:34:17 smithi052 ceph-mon[32473]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T09:34:17.585 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:34:17 smithi052 ceph-mon[32473]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T09:34:17.585 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:34:17 smithi052 ceph-mon[32473]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' 2024-03-04T09:34:17.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:34:17 smithi191 ceph-mon[36816]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T09:34:17.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:34:17 smithi191 ceph-mon[36816]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T09:34:17.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:34:17 smithi191 ceph-mon[36816]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' 2024-03-04T09:34:17.631 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:34:17 smithi106 ceph-mon[36484]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T09:34:17.631 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:34:17 smithi106 ceph-mon[36484]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T09:34:17.631 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:34:17 smithi106 ceph-mon[36484]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' 2024-03-04T09:34:18.631 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:34:18 smithi106 ceph-mon[36484]: pgmap v97: 129 pgs: 129 active+clean; 81 MiB data, 731 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 2.0 KiB/s wr, 5 op/s 2024-03-04T09:34:18.834 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:34:18 smithi052 ceph-mon[32473]: pgmap v97: 129 pgs: 129 active+clean; 81 MiB data, 731 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 2.0 KiB/s wr, 5 op/s 2024-03-04T09:34:18.836 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:34:18 smithi191 ceph-mon[36816]: pgmap v97: 129 pgs: 129 active+clean; 81 MiB data, 731 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 2.0 KiB/s wr, 5 op/s 2024-03-04T09:34:20.631 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:34:20 smithi106 ceph-mon[36484]: pgmap v98: 129 pgs: 129 active+clean; 81 MiB data, 731 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 2.0 KiB/s wr, 5 op/s 2024-03-04T09:34:20.835 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:34:20 smithi052 ceph-mon[32473]: pgmap v98: 129 pgs: 129 active+clean; 81 MiB data, 731 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 2.0 KiB/s wr, 5 op/s 2024-03-04T09:34:20.836 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:34:20 smithi191 ceph-mon[36816]: pgmap v98: 129 pgs: 129 active+clean; 81 MiB data, 731 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 2.0 KiB/s wr, 5 op/s 2024-03-04T09:34:22.631 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:34:22 smithi106 ceph-mon[36484]: pgmap v99: 129 pgs: 129 active+clean; 81 MiB data, 731 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 2.0 KiB/s wr, 8 op/s 2024-03-04T09:34:22.834 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:34:22 smithi052 ceph-mon[32473]: pgmap v99: 129 pgs: 129 active+clean; 81 MiB data, 731 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 2.0 KiB/s wr, 8 op/s 2024-03-04T09:34:22.836 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:34:22 smithi191 ceph-mon[36816]: pgmap v99: 129 pgs: 129 active+clean; 81 MiB data, 731 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 2.0 KiB/s wr, 8 op/s 2024-03-04T09:34:24.631 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:34:24 smithi106 ceph-mon[36484]: pgmap v100: 129 pgs: 129 active+clean; 81 MiB data, 731 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 170 B/s wr, 5 op/s 2024-03-04T09:34:24.638 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:34:24 smithi191 ceph-mon[36816]: pgmap v100: 129 pgs: 129 active+clean; 81 MiB data, 731 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 170 B/s wr, 5 op/s 2024-03-04T09:34:24.835 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:34:24 smithi052 ceph-mon[32473]: pgmap v100: 129 pgs: 129 active+clean; 81 MiB data, 731 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 170 B/s wr, 5 op/s 2024-03-04T09:34:26.631 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:34:26 smithi106 ceph-mon[36484]: pgmap v101: 129 pgs: 129 active+clean; 81 MiB data, 731 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 170 B/s wr, 8 op/s 2024-03-04T09:34:26.835 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:34:26 smithi052 ceph-mon[32473]: pgmap v101: 129 pgs: 129 active+clean; 81 MiB data, 731 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 170 B/s wr, 8 op/s 2024-03-04T09:34:26.836 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:34:26 smithi191 ceph-mon[36816]: pgmap v101: 129 pgs: 129 active+clean; 81 MiB data, 731 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 170 B/s wr, 8 op/s 2024-03-04T09:34:28.631 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:34:28 smithi106 ceph-mon[36484]: pgmap v102: 129 pgs: 129 active+clean; 81 MiB data, 731 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 5 op/s 2024-03-04T09:34:28.835 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:34:28 smithi052 ceph-mon[32473]: pgmap v102: 129 pgs: 129 active+clean; 81 MiB data, 731 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 5 op/s 2024-03-04T09:34:28.836 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:34:28 smithi191 ceph-mon[36816]: pgmap v102: 129 pgs: 129 active+clean; 81 MiB data, 731 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 5 op/s 2024-03-04T09:34:30.631 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:34:30 smithi106 ceph-mon[36484]: pgmap v103: 129 pgs: 129 active+clean; 81 MiB data, 731 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 5 op/s 2024-03-04T09:34:30.835 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:34:30 smithi052 ceph-mon[32473]: pgmap v103: 129 pgs: 129 active+clean; 81 MiB data, 731 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 5 op/s 2024-03-04T09:34:30.836 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:34:30 smithi191 ceph-mon[36816]: pgmap v103: 129 pgs: 129 active+clean; 81 MiB data, 731 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 5 op/s 2024-03-04T09:34:32.631 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:34:32 smithi106 ceph-mon[36484]: pgmap v104: 129 pgs: 129 active+clean; 82 MiB data, 733 MiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 108 KiB/s wr, 8 op/s 2024-03-04T09:34:32.835 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:34:32 smithi052 ceph-mon[32473]: pgmap v104: 129 pgs: 129 active+clean; 82 MiB data, 733 MiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 108 KiB/s wr, 8 op/s 2024-03-04T09:34:32.836 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:34:32 smithi191 ceph-mon[36816]: pgmap v104: 129 pgs: 129 active+clean; 82 MiB data, 733 MiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 108 KiB/s wr, 8 op/s 2024-03-04T09:34:34.631 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:34:34 smithi106 ceph-mon[36484]: pgmap v105: 129 pgs: 129 active+clean; 82 MiB data, 733 MiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 108 KiB/s wr, 5 op/s 2024-03-04T09:34:34.835 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:34:34 smithi052 ceph-mon[32473]: pgmap v105: 129 pgs: 129 active+clean; 82 MiB data, 733 MiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 108 KiB/s wr, 5 op/s 2024-03-04T09:34:34.836 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:34:34 smithi191 ceph-mon[36816]: pgmap v105: 129 pgs: 129 active+clean; 82 MiB data, 733 MiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 108 KiB/s wr, 5 op/s 2024-03-04T09:34:36.631 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:34:36 smithi106 ceph-mon[36484]: pgmap v106: 129 pgs: 129 active+clean; 82 MiB data, 735 MiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 112 KiB/s wr, 9 op/s 2024-03-04T09:34:36.834 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:34:36 smithi052 ceph-mon[32473]: pgmap v106: 129 pgs: 129 active+clean; 82 MiB data, 735 MiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 112 KiB/s wr, 9 op/s 2024-03-04T09:34:36.836 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:34:36 smithi191 ceph-mon[36816]: pgmap v106: 129 pgs: 129 active+clean; 82 MiB data, 735 MiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 112 KiB/s wr, 9 op/s 2024-03-04T09:34:38.631 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:34:38 smithi106 ceph-mon[36484]: pgmap v107: 129 pgs: 129 active+clean; 82 MiB data, 735 MiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 112 KiB/s wr, 6 op/s 2024-03-04T09:34:38.637 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:34:38 smithi052 ceph-mon[32473]: pgmap v107: 129 pgs: 129 active+clean; 82 MiB data, 735 MiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 112 KiB/s wr, 6 op/s 2024-03-04T09:34:38.836 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:34:38 smithi191 ceph-mon[36816]: pgmap v107: 129 pgs: 129 active+clean; 82 MiB data, 735 MiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 112 KiB/s wr, 6 op/s 2024-03-04T09:34:40.631 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:34:40 smithi106 ceph-mon[36484]: pgmap v108: 129 pgs: 129 active+clean; 82 MiB data, 735 MiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 112 KiB/s wr, 6 op/s 2024-03-04T09:34:40.835 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:34:40 smithi052 ceph-mon[32473]: pgmap v108: 129 pgs: 129 active+clean; 82 MiB data, 735 MiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 112 KiB/s wr, 6 op/s 2024-03-04T09:34:40.836 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:34:40 smithi191 ceph-mon[36816]: pgmap v108: 129 pgs: 129 active+clean; 82 MiB data, 735 MiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 112 KiB/s wr, 6 op/s 2024-03-04T09:34:42.631 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:34:42 smithi106 ceph-mon[36484]: pgmap v109: 129 pgs: 129 active+clean; 82 MiB data, 735 MiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 112 KiB/s wr, 8 op/s 2024-03-04T09:34:42.835 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:34:42 smithi052 ceph-mon[32473]: pgmap v109: 129 pgs: 129 active+clean; 82 MiB data, 735 MiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 112 KiB/s wr, 8 op/s 2024-03-04T09:34:42.836 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:34:42 smithi191 ceph-mon[36816]: pgmap v109: 129 pgs: 129 active+clean; 82 MiB data, 735 MiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 112 KiB/s wr, 8 op/s 2024-03-04T09:34:44.631 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:34:44 smithi106 ceph-mon[36484]: pgmap v110: 129 pgs: 129 active+clean; 82 MiB data, 735 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 3.9 KiB/s wr, 5 op/s 2024-03-04T09:34:44.835 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:34:44 smithi052 ceph-mon[32473]: pgmap v110: 129 pgs: 129 active+clean; 82 MiB data, 735 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 3.9 KiB/s wr, 5 op/s 2024-03-04T09:34:44.836 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:34:44 smithi191 ceph-mon[36816]: pgmap v110: 129 pgs: 129 active+clean; 82 MiB data, 735 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 3.9 KiB/s wr, 5 op/s 2024-03-04T09:34:46.835 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:34:46 smithi052 ceph-mon[32473]: pgmap v111: 129 pgs: 129 active+clean; 82 MiB data, 735 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 5.7 KiB/s wr, 8 op/s 2024-03-04T09:34:46.836 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:34:46 smithi191 ceph-mon[36816]: pgmap v111: 129 pgs: 129 active+clean; 82 MiB data, 735 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 5.7 KiB/s wr, 8 op/s 2024-03-04T09:34:46.881 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:34:46 smithi106 ceph-mon[36484]: pgmap v111: 129 pgs: 129 active+clean; 82 MiB data, 735 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 5.7 KiB/s wr, 8 op/s 2024-03-04T09:34:48.835 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:34:48 smithi052 ceph-mon[32473]: pgmap v112: 129 pgs: 129 active+clean; 82 MiB data, 735 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.7 KiB/s wr, 5 op/s 2024-03-04T09:34:48.836 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:34:48 smithi191 ceph-mon[36816]: pgmap v112: 129 pgs: 129 active+clean; 82 MiB data, 735 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.7 KiB/s wr, 5 op/s 2024-03-04T09:34:48.881 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:34:48 smithi106 ceph-mon[36484]: pgmap v112: 129 pgs: 129 active+clean; 82 MiB data, 735 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.7 KiB/s wr, 5 op/s 2024-03-04T09:34:50.835 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:34:50 smithi052 ceph-mon[32473]: pgmap v113: 129 pgs: 129 active+clean; 82 MiB data, 735 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.7 KiB/s wr, 5 op/s 2024-03-04T09:34:50.836 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:34:50 smithi191 ceph-mon[36816]: pgmap v113: 129 pgs: 129 active+clean; 82 MiB data, 735 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.7 KiB/s wr, 5 op/s 2024-03-04T09:34:50.881 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:34:50 smithi106 ceph-mon[36484]: pgmap v113: 129 pgs: 129 active+clean; 82 MiB data, 735 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.7 KiB/s wr, 5 op/s 2024-03-04T09:34:52.835 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:34:52 smithi052 ceph-mon[32473]: pgmap v114: 129 pgs: 129 active+clean; 82 MiB data, 735 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1.7 KiB/s wr, 7 op/s 2024-03-04T09:34:52.836 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:34:52 smithi191 ceph-mon[36816]: pgmap v114: 129 pgs: 129 active+clean; 82 MiB data, 735 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1.7 KiB/s wr, 7 op/s 2024-03-04T09:34:52.881 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:34:52 smithi106 ceph-mon[36484]: pgmap v114: 129 pgs: 129 active+clean; 82 MiB data, 735 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1.7 KiB/s wr, 7 op/s 2024-03-04T09:34:54.835 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:34:54 smithi052 ceph-mon[32473]: pgmap v115: 129 pgs: 129 active+clean; 82 MiB data, 735 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.7 KiB/s wr, 5 op/s 2024-03-04T09:34:54.836 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:34:54 smithi191 ceph-mon[36816]: pgmap v115: 129 pgs: 129 active+clean; 82 MiB data, 735 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.7 KiB/s wr, 5 op/s 2024-03-04T09:34:54.881 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:34:54 smithi106 ceph-mon[36484]: pgmap v115: 129 pgs: 129 active+clean; 82 MiB data, 735 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.7 KiB/s wr, 5 op/s 2024-03-04T09:34:56.835 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:34:56 smithi052 ceph-mon[32473]: pgmap v116: 129 pgs: 129 active+clean; 82 MiB data, 735 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1.7 KiB/s wr, 7 op/s 2024-03-04T09:34:56.910 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:34:56 smithi106 ceph-mon[36484]: pgmap v116: 129 pgs: 129 active+clean; 82 MiB data, 735 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1.7 KiB/s wr, 7 op/s 2024-03-04T09:34:56.910 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:34:56 smithi191 ceph-mon[36816]: pgmap v116: 129 pgs: 129 active+clean; 82 MiB data, 735 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1.7 KiB/s wr, 7 op/s 2024-03-04T09:34:58.835 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:34:58 smithi052 ceph-mon[32473]: pgmap v117: 129 pgs: 129 active+clean; 82 MiB data, 735 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-04T09:34:58.836 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:34:58 smithi191 ceph-mon[36816]: pgmap v117: 129 pgs: 129 active+clean; 82 MiB data, 735 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-04T09:34:58.881 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:34:58 smithi106 ceph-mon[36484]: pgmap v117: 129 pgs: 129 active+clean; 82 MiB data, 735 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-04T09:35:00.683 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:35:00 smithi106 ceph-mon[36484]: pgmap v118: 129 pgs: 129 active+clean; 82 MiB data, 735 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-04T09:35:00.835 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:35:00 smithi052 ceph-mon[32473]: pgmap v118: 129 pgs: 129 active+clean; 82 MiB data, 735 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-04T09:35:00.836 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:35:00 smithi191 ceph-mon[36816]: pgmap v118: 129 pgs: 129 active+clean; 82 MiB data, 735 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-04T09:35:02.835 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:35:02 smithi052 ceph-mon[32473]: pgmap v119: 129 pgs: 129 active+clean; 82 MiB data, 735 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-04T09:35:02.836 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:35:02 smithi191 ceph-mon[36816]: pgmap v119: 129 pgs: 129 active+clean; 82 MiB data, 735 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-04T09:35:02.881 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:35:02 smithi106 ceph-mon[36484]: pgmap v119: 129 pgs: 129 active+clean; 82 MiB data, 735 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-04T09:35:04.835 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:35:04 smithi052 ceph-mon[32473]: pgmap v120: 129 pgs: 129 active+clean; 82 MiB data, 735 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-04T09:35:04.836 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:35:04 smithi191 ceph-mon[36816]: pgmap v120: 129 pgs: 129 active+clean; 82 MiB data, 735 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-04T09:35:04.881 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:35:04 smithi106 ceph-mon[36484]: pgmap v120: 129 pgs: 129 active+clean; 82 MiB data, 735 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-04T09:35:06.835 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:35:06 smithi052 ceph-mon[32473]: pgmap v121: 129 pgs: 129 active+clean; 82 MiB data, 735 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-03-04T09:35:06.836 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:35:06 smithi191 ceph-mon[36816]: pgmap v121: 129 pgs: 129 active+clean; 82 MiB data, 735 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-03-04T09:35:06.881 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:35:06 smithi106 ceph-mon[36484]: pgmap v121: 129 pgs: 129 active+clean; 82 MiB data, 735 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-03-04T09:35:08.835 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:35:08 smithi052 ceph-mon[32473]: pgmap v122: 129 pgs: 129 active+clean; 82 MiB data, 735 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-03-04T09:35:08.836 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:35:08 smithi191 ceph-mon[36816]: pgmap v122: 129 pgs: 129 active+clean; 82 MiB data, 735 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-03-04T09:35:08.881 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:35:08 smithi106 ceph-mon[36484]: pgmap v122: 129 pgs: 129 active+clean; 82 MiB data, 735 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-03-04T09:35:10.835 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:35:10 smithi052 ceph-mon[32473]: pgmap v123: 129 pgs: 129 active+clean; 82 MiB data, 735 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-03-04T09:35:10.836 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:35:10 smithi191 ceph-mon[36816]: pgmap v123: 129 pgs: 129 active+clean; 82 MiB data, 735 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-03-04T09:35:10.881 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:35:10 smithi106 ceph-mon[36484]: pgmap v123: 129 pgs: 129 active+clean; 82 MiB data, 735 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-03-04T09:35:12.835 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:35:12 smithi052 ceph-mon[32473]: pgmap v124: 129 pgs: 129 active+clean; 82 MiB data, 735 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-03-04T09:35:12.836 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:35:12 smithi191 ceph-mon[36816]: pgmap v124: 129 pgs: 129 active+clean; 82 MiB data, 735 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-03-04T09:35:12.881 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:35:12 smithi106 ceph-mon[36484]: pgmap v124: 129 pgs: 129 active+clean; 82 MiB data, 735 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-03-04T09:35:14.835 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:35:14 smithi052 ceph-mon[32473]: pgmap v125: 129 pgs: 129 active+clean; 82 MiB data, 735 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-03-04T09:35:14.836 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:35:14 smithi191 ceph-mon[36816]: pgmap v125: 129 pgs: 129 active+clean; 82 MiB data, 735 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-03-04T09:35:14.881 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:35:14 smithi106 ceph-mon[36484]: pgmap v125: 129 pgs: 129 active+clean; 82 MiB data, 735 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-03-04T09:35:16.835 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:35:16 smithi052 ceph-mon[32473]: pgmap v126: 129 pgs: 129 active+clean; 82 MiB data, 735 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-03-04T09:35:16.836 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:35:16 smithi191 ceph-mon[36816]: pgmap v126: 129 pgs: 129 active+clean; 82 MiB data, 735 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-03-04T09:35:16.881 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:35:16 smithi106 ceph-mon[36484]: pgmap v126: 129 pgs: 129 active+clean; 82 MiB data, 735 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-03-04T09:35:17.835 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:35:17 smithi052 ceph-mon[32473]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T09:35:17.835 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:35:17 smithi052 ceph-mon[32473]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T09:35:17.835 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:35:17 smithi052 ceph-mon[32473]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T09:35:17.835 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:35:17 smithi052 ceph-mon[32473]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' 2024-03-04T09:35:17.836 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:35:17 smithi191 ceph-mon[36816]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T09:35:17.836 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:35:17 smithi191 ceph-mon[36816]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T09:35:17.836 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:35:17 smithi191 ceph-mon[36816]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T09:35:17.836 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:35:17 smithi191 ceph-mon[36816]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' 2024-03-04T09:35:17.881 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:35:17 smithi106 ceph-mon[36484]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T09:35:17.881 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:35:17 smithi106 ceph-mon[36484]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T09:35:17.881 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:35:17 smithi106 ceph-mon[36484]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T09:35:17.881 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:35:17 smithi106 ceph-mon[36484]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' 2024-03-04T09:35:18.835 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:35:18 smithi052 ceph-mon[32473]: pgmap v127: 129 pgs: 129 active+clean; 82 MiB data, 735 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-03-04T09:35:18.836 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:35:18 smithi191 ceph-mon[36816]: pgmap v127: 129 pgs: 129 active+clean; 82 MiB data, 735 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-03-04T09:35:18.881 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:35:18 smithi106 ceph-mon[36484]: pgmap v127: 129 pgs: 129 active+clean; 82 MiB data, 735 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-03-04T09:35:20.835 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:35:20 smithi052 ceph-mon[32473]: pgmap v128: 129 pgs: 129 active+clean; 82 MiB data, 735 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-03-04T09:35:20.836 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:35:20 smithi191 ceph-mon[36816]: pgmap v128: 129 pgs: 129 active+clean; 82 MiB data, 735 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-03-04T09:35:20.881 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:35:20 smithi106 ceph-mon[36484]: pgmap v128: 129 pgs: 129 active+clean; 82 MiB data, 735 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-03-04T09:35:22.835 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:35:22 smithi052 ceph-mon[32473]: pgmap v129: 129 pgs: 129 active+clean; 82 MiB data, 735 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 85 B/s wr, 7 op/s 2024-03-04T09:35:22.836 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:35:22 smithi191 ceph-mon[36816]: pgmap v129: 129 pgs: 129 active+clean; 82 MiB data, 735 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 85 B/s wr, 7 op/s 2024-03-04T09:35:22.881 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:35:22 smithi106 ceph-mon[36484]: pgmap v129: 129 pgs: 129 active+clean; 82 MiB data, 735 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 85 B/s wr, 7 op/s 2024-03-04T09:35:24.835 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:35:24 smithi052 ceph-mon[32473]: pgmap v130: 129 pgs: 129 active+clean; 82 MiB data, 735 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-04T09:35:24.836 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:35:24 smithi191 ceph-mon[36816]: pgmap v130: 129 pgs: 129 active+clean; 82 MiB data, 735 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-04T09:35:24.881 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:35:24 smithi106 ceph-mon[36484]: pgmap v130: 129 pgs: 129 active+clean; 82 MiB data, 735 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-04T09:35:26.835 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:35:26 smithi052 ceph-mon[32473]: pgmap v131: 129 pgs: 129 active+clean; 82 MiB data, 735 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-04T09:35:26.836 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:35:26 smithi191 ceph-mon[36816]: pgmap v131: 129 pgs: 129 active+clean; 82 MiB data, 735 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-04T09:35:26.881 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:35:26 smithi106 ceph-mon[36484]: pgmap v131: 129 pgs: 129 active+clean; 82 MiB data, 735 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-04T09:35:28.835 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:35:28 smithi052 ceph-mon[32473]: pgmap v132: 129 pgs: 129 active+clean; 82 MiB data, 735 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-04T09:35:28.836 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:35:28 smithi191 ceph-mon[36816]: pgmap v132: 129 pgs: 129 active+clean; 82 MiB data, 735 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-04T09:35:28.881 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:35:28 smithi106 ceph-mon[36484]: pgmap v132: 129 pgs: 129 active+clean; 82 MiB data, 735 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-04T09:35:29.835 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:35:29 smithi052 ceph-mon[32473]: pgmap v133: 129 pgs: 129 active+clean; 82 MiB data, 735 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-04T09:35:29.836 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:35:29 smithi191 ceph-mon[36816]: pgmap v133: 129 pgs: 129 active+clean; 82 MiB data, 735 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-04T09:35:29.881 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:35:29 smithi106 ceph-mon[36484]: pgmap v133: 129 pgs: 129 active+clean; 82 MiB data, 735 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-04T09:35:31.835 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:35:31 smithi052 ceph-mon[32473]: pgmap v134: 129 pgs: 129 active+clean; 82 MiB data, 735 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-04T09:35:31.836 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:35:31 smithi191 ceph-mon[36816]: pgmap v134: 129 pgs: 129 active+clean; 82 MiB data, 735 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-04T09:35:31.881 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:35:31 smithi106 ceph-mon[36484]: pgmap v134: 129 pgs: 129 active+clean; 82 MiB data, 735 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-04T09:35:33.835 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:35:33 smithi052 ceph-mon[32473]: pgmap v135: 129 pgs: 129 active+clean; 82 MiB data, 735 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-04T09:35:33.836 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:35:33 smithi191 ceph-mon[36816]: pgmap v135: 129 pgs: 129 active+clean; 82 MiB data, 735 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-04T09:35:33.881 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:35:33 smithi106 ceph-mon[36484]: pgmap v135: 129 pgs: 129 active+clean; 82 MiB data, 735 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-04T09:35:35.835 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:35:35 smithi052 ceph-mon[32473]: pgmap v136: 129 pgs: 129 active+clean; 82 MiB data, 735 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-04T09:35:35.835 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:35:35 smithi052 ceph-mon[32473]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-03-04T09:35:35.836 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:35:35 smithi191 ceph-mon[36816]: pgmap v136: 129 pgs: 129 active+clean; 82 MiB data, 735 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-04T09:35:35.836 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:35:35 smithi191 ceph-mon[36816]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-03-04T09:35:35.881 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:35:35 smithi106 ceph-mon[36484]: pgmap v136: 129 pgs: 129 active+clean; 82 MiB data, 735 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-04T09:35:35.881 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:35:35 smithi106 ceph-mon[36484]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-03-04T09:35:37.835 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:35:37 smithi052 ceph-mon[32473]: pgmap v137: 129 pgs: 129 active+clean; 82 MiB data, 735 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-04T09:35:37.836 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:35:37 smithi191 ceph-mon[36816]: pgmap v137: 129 pgs: 129 active+clean; 82 MiB data, 735 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-04T09:35:37.883 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:35:37 smithi106 ceph-mon[36484]: pgmap v137: 129 pgs: 129 active+clean; 82 MiB data, 735 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-04T09:35:39.835 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:35:39 smithi052 ceph-mon[32473]: pgmap v138: 129 pgs: 129 active+clean; 82 MiB data, 735 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-04T09:35:39.836 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:35:39 smithi191 ceph-mon[36816]: pgmap v138: 129 pgs: 129 active+clean; 82 MiB data, 735 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-04T09:35:39.881 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:35:39 smithi106 ceph-mon[36484]: pgmap v138: 129 pgs: 129 active+clean; 82 MiB data, 735 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-04T09:35:40.882 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:35:40 smithi106 ceph-mon[36484]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-03-04T09:35:41.085 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:35:40 smithi052 ceph-mon[32473]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-03-04T09:35:41.086 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:35:40 smithi191 ceph-mon[36816]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-03-04T09:35:41.881 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:35:41 smithi106 ceph-mon[36484]: pgmap v139: 129 pgs: 129 active+clean; 82 MiB data, 735 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-04T09:35:42.085 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:35:41 smithi052 ceph-mon[32473]: pgmap v139: 129 pgs: 129 active+clean; 82 MiB data, 735 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-04T09:35:42.086 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:35:41 smithi191 ceph-mon[36816]: pgmap v139: 129 pgs: 129 active+clean; 82 MiB data, 735 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-04T09:35:43.881 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:35:43 smithi106 ceph-mon[36484]: pgmap v140: 129 pgs: 129 active+clean; 82 MiB data, 735 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-04T09:35:44.085 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:35:43 smithi052 ceph-mon[32473]: pgmap v140: 129 pgs: 129 active+clean; 82 MiB data, 735 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-04T09:35:44.086 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:35:43 smithi191 ceph-mon[36816]: pgmap v140: 129 pgs: 129 active+clean; 82 MiB data, 735 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-04T09:35:45.881 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:35:45 smithi106 ceph-mon[36484]: pgmap v141: 129 pgs: 129 active+clean; 82 MiB data, 735 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-04T09:35:46.085 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:35:45 smithi052 ceph-mon[32473]: pgmap v141: 129 pgs: 129 active+clean; 82 MiB data, 735 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-04T09:35:46.086 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:35:45 smithi191 ceph-mon[36816]: pgmap v141: 129 pgs: 129 active+clean; 82 MiB data, 735 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-04T09:35:47.881 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:35:47 smithi106 ceph-mon[36484]: pgmap v142: 129 pgs: 129 active+clean; 82 MiB data, 735 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-04T09:35:48.085 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:35:47 smithi052 ceph-mon[32473]: pgmap v142: 129 pgs: 129 active+clean; 82 MiB data, 735 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-04T09:35:48.086 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:35:47 smithi191 ceph-mon[36816]: pgmap v142: 129 pgs: 129 active+clean; 82 MiB data, 735 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-04T09:35:49.881 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:35:49 smithi106 ceph-mon[36484]: pgmap v143: 129 pgs: 129 active+clean; 82 MiB data, 735 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-04T09:35:50.084 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:35:49 smithi052 ceph-mon[32473]: pgmap v143: 129 pgs: 129 active+clean; 82 MiB data, 735 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-04T09:35:50.086 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:35:49 smithi191 ceph-mon[36816]: pgmap v143: 129 pgs: 129 active+clean; 82 MiB data, 735 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-04T09:35:51.881 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:35:51 smithi106 ceph-mon[36484]: pgmap v144: 129 pgs: 129 active+clean; 82 MiB data, 735 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-04T09:35:52.084 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:35:51 smithi052 ceph-mon[32473]: pgmap v144: 129 pgs: 129 active+clean; 82 MiB data, 735 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-04T09:35:52.086 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:35:51 smithi191 ceph-mon[36816]: pgmap v144: 129 pgs: 129 active+clean; 82 MiB data, 735 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-04T09:35:53.881 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:35:53 smithi106 ceph-mon[36484]: pgmap v145: 129 pgs: 129 active+clean; 82 MiB data, 735 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-04T09:35:54.085 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:35:53 smithi052 ceph-mon[32473]: pgmap v145: 129 pgs: 129 active+clean; 82 MiB data, 735 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-04T09:35:54.086 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:35:53 smithi191 ceph-mon[36816]: pgmap v145: 129 pgs: 129 active+clean; 82 MiB data, 735 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-04T09:35:55.881 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:35:55 smithi106 ceph-mon[36484]: pgmap v146: 129 pgs: 129 active+clean; 82 MiB data, 735 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-04T09:35:55.888 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:35:55 smithi191 ceph-mon[36816]: pgmap v146: 129 pgs: 129 active+clean; 82 MiB data, 735 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-04T09:35:56.085 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:35:55 smithi052 ceph-mon[32473]: pgmap v146: 129 pgs: 129 active+clean; 82 MiB data, 735 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-04T09:35:57.881 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:35:57 smithi106 ceph-mon[36484]: pgmap v147: 129 pgs: 129 active+clean; 82 MiB data, 735 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-04T09:35:58.085 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:35:57 smithi052 ceph-mon[32473]: pgmap v147: 129 pgs: 129 active+clean; 82 MiB data, 735 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-04T09:35:58.087 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:35:57 smithi191 ceph-mon[36816]: pgmap v147: 129 pgs: 129 active+clean; 82 MiB data, 735 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-04T09:35:59.881 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:35:59 smithi106 ceph-mon[36484]: pgmap v148: 129 pgs: 129 active+clean; 82 MiB data, 735 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-04T09:36:00.085 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:35:59 smithi052 ceph-mon[32473]: pgmap v148: 129 pgs: 129 active+clean; 82 MiB data, 735 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-04T09:36:00.086 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:35:59 smithi191 ceph-mon[36816]: pgmap v148: 129 pgs: 129 active+clean; 82 MiB data, 735 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-04T09:36:01.881 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:36:01 smithi106 ceph-mon[36484]: pgmap v149: 129 pgs: 129 active+clean; 82 MiB data, 735 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-04T09:36:02.085 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:36:01 smithi052 ceph-mon[32473]: pgmap v149: 129 pgs: 129 active+clean; 82 MiB data, 735 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-04T09:36:02.086 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:36:01 smithi191 ceph-mon[36816]: pgmap v149: 129 pgs: 129 active+clean; 82 MiB data, 735 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-04T09:36:03.881 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:36:03 smithi106 ceph-mon[36484]: pgmap v150: 129 pgs: 129 active+clean; 82 MiB data, 735 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-04T09:36:04.085 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:36:03 smithi052 ceph-mon[32473]: pgmap v150: 129 pgs: 129 active+clean; 82 MiB data, 735 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-04T09:36:04.086 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:36:03 smithi191 ceph-mon[36816]: pgmap v150: 129 pgs: 129 active+clean; 82 MiB data, 735 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-04T09:36:05.881 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:36:05 smithi106 ceph-mon[36484]: pgmap v151: 129 pgs: 129 active+clean; 82 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-03-04T09:36:06.085 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:36:05 smithi052 ceph-mon[32473]: pgmap v151: 129 pgs: 129 active+clean; 82 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-03-04T09:36:06.086 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:36:05 smithi191 ceph-mon[36816]: pgmap v151: 129 pgs: 129 active+clean; 82 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-03-04T09:36:07.881 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:36:07 smithi106 ceph-mon[36484]: pgmap v152: 129 pgs: 129 active+clean; 82 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-03-04T09:36:08.085 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:36:07 smithi052 ceph-mon[32473]: pgmap v152: 129 pgs: 129 active+clean; 82 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-03-04T09:36:08.086 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:36:07 smithi191 ceph-mon[36816]: pgmap v152: 129 pgs: 129 active+clean; 82 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-03-04T09:36:09.881 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:36:09 smithi106 ceph-mon[36484]: pgmap v153: 129 pgs: 129 active+clean; 82 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-03-04T09:36:09.887 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:36:09 smithi052 ceph-mon[32473]: pgmap v153: 129 pgs: 129 active+clean; 82 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-03-04T09:36:10.086 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:36:09 smithi191 ceph-mon[36816]: pgmap v153: 129 pgs: 129 active+clean; 82 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-03-04T09:36:11.881 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:36:11 smithi106 ceph-mon[36484]: pgmap v154: 129 pgs: 129 active+clean; 82 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-03-04T09:36:12.085 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:36:11 smithi052 ceph-mon[32473]: pgmap v154: 129 pgs: 129 active+clean; 82 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-03-04T09:36:12.086 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:36:11 smithi191 ceph-mon[36816]: pgmap v154: 129 pgs: 129 active+clean; 82 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-03-04T09:36:14.085 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:36:13 smithi052 ceph-mon[32473]: pgmap v155: 129 pgs: 129 active+clean; 82 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-03-04T09:36:14.086 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:36:13 smithi191 ceph-mon[36816]: pgmap v155: 129 pgs: 129 active+clean; 82 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-03-04T09:36:14.131 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:36:13 smithi106 ceph-mon[36484]: pgmap v155: 129 pgs: 129 active+clean; 82 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-03-04T09:36:16.085 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:36:15 smithi052 ceph-mon[32473]: pgmap v156: 129 pgs: 129 active+clean; 82 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-03-04T09:36:16.086 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:36:15 smithi191 ceph-mon[36816]: pgmap v156: 129 pgs: 129 active+clean; 82 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-03-04T09:36:16.131 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:36:15 smithi106 ceph-mon[36484]: pgmap v156: 129 pgs: 129 active+clean; 82 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-03-04T09:36:18.085 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:36:17 smithi052 ceph-mon[32473]: pgmap v157: 129 pgs: 129 active+clean; 82 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-03-04T09:36:18.085 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:36:17 smithi052 ceph-mon[32473]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T09:36:18.086 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:36:17 smithi191 ceph-mon[36816]: pgmap v157: 129 pgs: 129 active+clean; 82 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-03-04T09:36:18.086 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:36:17 smithi191 ceph-mon[36816]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T09:36:18.131 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:36:17 smithi106 ceph-mon[36484]: pgmap v157: 129 pgs: 129 active+clean; 82 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-03-04T09:36:18.131 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:36:17 smithi106 ceph-mon[36484]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T09:36:19.085 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:36:18 smithi052 ceph-mon[32473]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T09:36:19.085 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:36:18 smithi052 ceph-mon[32473]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T09:36:19.085 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:36:18 smithi052 ceph-mon[32473]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' 2024-03-04T09:36:19.086 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:36:18 smithi191 ceph-mon[36816]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T09:36:19.086 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:36:18 smithi191 ceph-mon[36816]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T09:36:19.086 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:36:18 smithi191 ceph-mon[36816]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' 2024-03-04T09:36:19.131 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:36:18 smithi106 ceph-mon[36484]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T09:36:19.131 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:36:18 smithi106 ceph-mon[36484]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T09:36:19.131 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:36:18 smithi106 ceph-mon[36484]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' 2024-03-04T09:36:20.335 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:36:19 smithi052 ceph-mon[32473]: pgmap v158: 129 pgs: 129 active+clean; 82 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-03-04T09:36:20.336 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:36:19 smithi191 ceph-mon[36816]: pgmap v158: 129 pgs: 129 active+clean; 82 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-03-04T09:36:20.381 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:36:19 smithi106 ceph-mon[36484]: pgmap v158: 129 pgs: 129 active+clean; 82 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-03-04T09:36:22.335 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:36:21 smithi052 ceph-mon[32473]: pgmap v159: 129 pgs: 129 active+clean; 82 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 85 B/s wr, 7 op/s 2024-03-04T09:36:22.336 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:36:21 smithi191 ceph-mon[36816]: pgmap v159: 129 pgs: 129 active+clean; 82 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 85 B/s wr, 7 op/s 2024-03-04T09:36:22.381 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:36:21 smithi106 ceph-mon[36484]: pgmap v159: 129 pgs: 129 active+clean; 82 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 85 B/s wr, 7 op/s 2024-03-04T09:36:24.335 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:36:23 smithi052 ceph-mon[32473]: pgmap v160: 129 pgs: 129 active+clean; 82 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-04T09:36:24.336 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:36:23 smithi191 ceph-mon[36816]: pgmap v160: 129 pgs: 129 active+clean; 82 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-04T09:36:24.381 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:36:23 smithi106 ceph-mon[36484]: pgmap v160: 129 pgs: 129 active+clean; 82 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-04T09:36:26.335 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:36:25 smithi052 ceph-mon[32473]: pgmap v161: 129 pgs: 129 active+clean; 82 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-04T09:36:26.336 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:36:25 smithi191 ceph-mon[36816]: pgmap v161: 129 pgs: 129 active+clean; 82 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-04T09:36:26.381 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:36:25 smithi106 ceph-mon[36484]: pgmap v161: 129 pgs: 129 active+clean; 82 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-04T09:36:28.335 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:36:27 smithi052 ceph-mon[32473]: pgmap v162: 129 pgs: 129 active+clean; 82 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-04T09:36:28.336 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:36:27 smithi191 ceph-mon[36816]: pgmap v162: 129 pgs: 129 active+clean; 82 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-04T09:36:28.381 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:36:27 smithi106 ceph-mon[36484]: pgmap v162: 129 pgs: 129 active+clean; 82 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-04T09:36:30.335 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:36:29 smithi052 ceph-mon[32473]: pgmap v163: 129 pgs: 129 active+clean; 82 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-04T09:36:30.336 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:36:29 smithi191 ceph-mon[36816]: pgmap v163: 129 pgs: 129 active+clean; 82 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-04T09:36:30.381 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:36:29 smithi106 ceph-mon[36484]: pgmap v163: 129 pgs: 129 active+clean; 82 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-04T09:36:32.183 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:36:31 smithi106 ceph-mon[36484]: pgmap v164: 129 pgs: 129 active+clean; 82 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-04T09:36:32.334 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:36:31 smithi052 ceph-mon[32473]: pgmap v164: 129 pgs: 129 active+clean; 82 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-04T09:36:32.336 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:36:31 smithi191 ceph-mon[36816]: pgmap v164: 129 pgs: 129 active+clean; 82 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-04T09:36:34.335 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:36:33 smithi052 ceph-mon[32473]: pgmap v165: 129 pgs: 129 active+clean; 82 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-04T09:36:34.336 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:36:33 smithi191 ceph-mon[36816]: pgmap v165: 129 pgs: 129 active+clean; 82 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-04T09:36:34.381 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:36:33 smithi106 ceph-mon[36484]: pgmap v165: 129 pgs: 129 active+clean; 82 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-04T09:36:36.335 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:36:35 smithi052 ceph-mon[32473]: pgmap v166: 129 pgs: 129 active+clean; 82 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-04T09:36:36.336 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:36:35 smithi191 ceph-mon[36816]: pgmap v166: 129 pgs: 129 active+clean; 82 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-04T09:36:36.381 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:36:35 smithi106 ceph-mon[36484]: pgmap v166: 129 pgs: 129 active+clean; 82 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-04T09:36:38.335 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:36:37 smithi052 ceph-mon[32473]: pgmap v167: 129 pgs: 129 active+clean; 82 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-04T09:36:38.336 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:36:37 smithi191 ceph-mon[36816]: pgmap v167: 129 pgs: 129 active+clean; 82 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-04T09:36:38.381 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:36:37 smithi106 ceph-mon[36484]: pgmap v167: 129 pgs: 129 active+clean; 82 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-04T09:36:40.335 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:36:39 smithi052 ceph-mon[32473]: pgmap v168: 129 pgs: 129 active+clean; 82 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-04T09:36:40.336 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:36:39 smithi191 ceph-mon[36816]: pgmap v168: 129 pgs: 129 active+clean; 82 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-04T09:36:40.381 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:36:39 smithi106 ceph-mon[36484]: pgmap v168: 129 pgs: 129 active+clean; 82 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-04T09:36:42.335 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:36:41 smithi052 ceph-mon[32473]: pgmap v169: 129 pgs: 129 active+clean; 82 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-04T09:36:42.336 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:36:41 smithi191 ceph-mon[36816]: pgmap v169: 129 pgs: 129 active+clean; 82 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-04T09:36:42.381 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:36:41 smithi106 ceph-mon[36484]: pgmap v169: 129 pgs: 129 active+clean; 82 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-04T09:36:44.335 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:36:43 smithi052 ceph-mon[32473]: pgmap v170: 129 pgs: 129 active+clean; 82 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-04T09:36:44.336 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:36:43 smithi191 ceph-mon[36816]: pgmap v170: 129 pgs: 129 active+clean; 82 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-04T09:36:44.381 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:36:43 smithi106 ceph-mon[36484]: pgmap v170: 129 pgs: 129 active+clean; 82 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-04T09:36:46.335 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:36:45 smithi052 ceph-mon[32473]: pgmap v171: 129 pgs: 129 active+clean; 82 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-04T09:36:46.336 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:36:45 smithi191 ceph-mon[36816]: pgmap v171: 129 pgs: 129 active+clean; 82 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-04T09:36:46.381 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:36:45 smithi106 ceph-mon[36484]: pgmap v171: 129 pgs: 129 active+clean; 82 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-04T09:36:48.335 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:36:47 smithi052 ceph-mon[32473]: pgmap v172: 129 pgs: 129 active+clean; 82 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-04T09:36:48.336 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:36:47 smithi191 ceph-mon[36816]: pgmap v172: 129 pgs: 129 active+clean; 82 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-04T09:36:48.381 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:36:47 smithi106 ceph-mon[36484]: pgmap v172: 129 pgs: 129 active+clean; 82 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-04T09:36:50.335 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:36:49 smithi052 ceph-mon[32473]: pgmap v173: 129 pgs: 129 active+clean; 82 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-04T09:36:50.336 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:36:49 smithi191 ceph-mon[36816]: pgmap v173: 129 pgs: 129 active+clean; 82 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-04T09:36:50.381 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:36:49 smithi106 ceph-mon[36484]: pgmap v173: 129 pgs: 129 active+clean; 82 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-04T09:36:52.335 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:36:51 smithi052 ceph-mon[32473]: pgmap v174: 129 pgs: 129 active+clean; 82 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-04T09:36:52.336 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:36:51 smithi191 ceph-mon[36816]: pgmap v174: 129 pgs: 129 active+clean; 82 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-04T09:36:52.381 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:36:51 smithi106 ceph-mon[36484]: pgmap v174: 129 pgs: 129 active+clean; 82 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-04T09:36:54.335 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:36:53 smithi052 ceph-mon[32473]: pgmap v175: 129 pgs: 129 active+clean; 82 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-04T09:36:54.336 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:36:53 smithi191 ceph-mon[36816]: pgmap v175: 129 pgs: 129 active+clean; 82 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-04T09:36:54.381 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:36:53 smithi106 ceph-mon[36484]: pgmap v175: 129 pgs: 129 active+clean; 82 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-04T09:36:56.335 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:36:55 smithi052 ceph-mon[32473]: pgmap v176: 129 pgs: 129 active+clean; 82 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-04T09:36:56.336 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:36:55 smithi191 ceph-mon[36816]: pgmap v176: 129 pgs: 129 active+clean; 82 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-04T09:36:56.381 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:36:55 smithi106 ceph-mon[36484]: pgmap v176: 129 pgs: 129 active+clean; 82 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-04T09:36:58.335 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:36:57 smithi052 ceph-mon[32473]: pgmap v177: 129 pgs: 129 active+clean; 82 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-04T09:36:58.336 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:36:57 smithi191 ceph-mon[36816]: pgmap v177: 129 pgs: 129 active+clean; 82 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-04T09:36:58.381 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:36:57 smithi106 ceph-mon[36484]: pgmap v177: 129 pgs: 129 active+clean; 82 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-04T09:37:00.335 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:36:59 smithi052 ceph-mon[32473]: pgmap v178: 129 pgs: 129 active+clean; 82 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-04T09:37:00.336 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:36:59 smithi191 ceph-mon[36816]: pgmap v178: 129 pgs: 129 active+clean; 82 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-04T09:37:00.381 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:36:59 smithi106 ceph-mon[36484]: pgmap v178: 129 pgs: 129 active+clean; 82 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-04T09:37:02.335 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:37:01 smithi052 ceph-mon[32473]: pgmap v179: 129 pgs: 129 active+clean; 82 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-04T09:37:02.336 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:37:01 smithi191 ceph-mon[36816]: pgmap v179: 129 pgs: 129 active+clean; 82 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-04T09:37:02.381 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:37:01 smithi106 ceph-mon[36484]: pgmap v179: 129 pgs: 129 active+clean; 82 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-04T09:37:04.335 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:37:03 smithi052 ceph-mon[32473]: pgmap v180: 129 pgs: 129 active+clean; 82 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-04T09:37:04.336 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:37:03 smithi191 ceph-mon[36816]: pgmap v180: 129 pgs: 129 active+clean; 82 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-04T09:37:04.381 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:37:03 smithi106 ceph-mon[36484]: pgmap v180: 129 pgs: 129 active+clean; 82 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-04T09:37:06.335 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:37:05 smithi052 ceph-mon[32473]: pgmap v181: 129 pgs: 129 active+clean; 82 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-03-04T09:37:06.336 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:37:05 smithi191 ceph-mon[36816]: pgmap v181: 129 pgs: 129 active+clean; 82 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-03-04T09:37:06.381 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:37:05 smithi106 ceph-mon[36484]: pgmap v181: 129 pgs: 129 active+clean; 82 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-03-04T09:37:08.335 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:37:07 smithi052 ceph-mon[32473]: pgmap v182: 129 pgs: 129 active+clean; 82 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-03-04T09:37:08.336 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:37:07 smithi191 ceph-mon[36816]: pgmap v182: 129 pgs: 129 active+clean; 82 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-03-04T09:37:08.381 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:37:07 smithi106 ceph-mon[36484]: pgmap v182: 129 pgs: 129 active+clean; 82 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-03-04T09:37:10.335 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:37:09 smithi052 ceph-mon[32473]: pgmap v183: 129 pgs: 129 active+clean; 82 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-03-04T09:37:10.336 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:37:09 smithi191 ceph-mon[36816]: pgmap v183: 129 pgs: 129 active+clean; 82 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-03-04T09:37:10.381 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:37:09 smithi106 ceph-mon[36484]: pgmap v183: 129 pgs: 129 active+clean; 82 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-03-04T09:37:12.335 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:37:11 smithi052 ceph-mon[32473]: pgmap v184: 129 pgs: 129 active+clean; 82 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-03-04T09:37:12.336 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:37:11 smithi191 ceph-mon[36816]: pgmap v184: 129 pgs: 129 active+clean; 82 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-03-04T09:37:12.381 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:37:11 smithi106 ceph-mon[36484]: pgmap v184: 129 pgs: 129 active+clean; 82 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-03-04T09:37:14.335 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:37:13 smithi052 ceph-mon[32473]: pgmap v185: 129 pgs: 129 active+clean; 82 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-03-04T09:37:14.336 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:37:13 smithi191 ceph-mon[36816]: pgmap v185: 129 pgs: 129 active+clean; 82 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-03-04T09:37:14.381 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:37:13 smithi106 ceph-mon[36484]: pgmap v185: 129 pgs: 129 active+clean; 82 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-03-04T09:37:16.335 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:37:15 smithi052 ceph-mon[32473]: pgmap v186: 129 pgs: 129 active+clean; 82 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-03-04T09:37:16.336 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:37:15 smithi191 ceph-mon[36816]: pgmap v186: 129 pgs: 129 active+clean; 82 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-03-04T09:37:16.381 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:37:15 smithi106 ceph-mon[36484]: pgmap v186: 129 pgs: 129 active+clean; 82 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-03-04T09:37:18.335 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:37:17 smithi052 ceph-mon[32473]: pgmap v187: 129 pgs: 129 active+clean; 82 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-03-04T09:37:18.336 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:37:17 smithi191 ceph-mon[36816]: pgmap v187: 129 pgs: 129 active+clean; 82 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-03-04T09:37:18.381 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:37:17 smithi106 ceph-mon[36484]: pgmap v187: 129 pgs: 129 active+clean; 82 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-03-04T09:37:19.335 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:37:18 smithi052 ceph-mon[32473]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T09:37:19.335 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:37:18 smithi052 ceph-mon[32473]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T09:37:19.335 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:37:18 smithi052 ceph-mon[32473]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T09:37:19.335 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:37:18 smithi052 ceph-mon[32473]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' 2024-03-04T09:37:19.336 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:37:18 smithi191 ceph-mon[36816]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T09:37:19.336 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:37:18 smithi191 ceph-mon[36816]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T09:37:19.336 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:37:18 smithi191 ceph-mon[36816]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T09:37:19.336 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:37:18 smithi191 ceph-mon[36816]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' 2024-03-04T09:37:19.381 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:37:18 smithi106 ceph-mon[36484]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T09:37:19.381 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:37:18 smithi106 ceph-mon[36484]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T09:37:19.381 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:37:18 smithi106 ceph-mon[36484]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T09:37:19.381 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:37:18 smithi106 ceph-mon[36484]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' 2024-03-04T09:37:20.335 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:37:19 smithi052 ceph-mon[32473]: pgmap v188: 129 pgs: 129 active+clean; 82 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-03-04T09:37:20.336 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:37:19 smithi191 ceph-mon[36816]: pgmap v188: 129 pgs: 129 active+clean; 82 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-03-04T09:37:20.381 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:37:19 smithi106 ceph-mon[36484]: pgmap v188: 129 pgs: 129 active+clean; 82 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-03-04T09:37:22.335 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:37:21 smithi052 ceph-mon[32473]: pgmap v189: 129 pgs: 129 active+clean; 82 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 85 B/s wr, 7 op/s 2024-03-04T09:37:22.336 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:37:21 smithi191 ceph-mon[36816]: pgmap v189: 129 pgs: 129 active+clean; 82 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 85 B/s wr, 7 op/s 2024-03-04T09:37:22.381 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:37:21 smithi106 ceph-mon[36484]: pgmap v189: 129 pgs: 129 active+clean; 82 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 85 B/s wr, 7 op/s 2024-03-04T09:37:24.335 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:37:23 smithi052 ceph-mon[32473]: pgmap v190: 129 pgs: 129 active+clean; 82 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-04T09:37:24.336 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:37:23 smithi191 ceph-mon[36816]: pgmap v190: 129 pgs: 129 active+clean; 82 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-04T09:37:24.381 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:37:23 smithi106 ceph-mon[36484]: pgmap v190: 129 pgs: 129 active+clean; 82 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-04T09:37:26.335 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:37:25 smithi052 ceph-mon[32473]: pgmap v191: 129 pgs: 129 active+clean; 82 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-04T09:37:26.336 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:37:25 smithi191 ceph-mon[36816]: pgmap v191: 129 pgs: 129 active+clean; 82 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-04T09:37:26.381 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:37:25 smithi106 ceph-mon[36484]: pgmap v191: 129 pgs: 129 active+clean; 82 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-04T09:37:28.335 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:37:27 smithi052 ceph-mon[32473]: pgmap v192: 129 pgs: 129 active+clean; 82 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-04T09:37:28.336 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:37:27 smithi191 ceph-mon[36816]: pgmap v192: 129 pgs: 129 active+clean; 82 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-04T09:37:28.381 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:37:27 smithi106 ceph-mon[36484]: pgmap v192: 129 pgs: 129 active+clean; 82 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-04T09:37:30.335 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:37:29 smithi052 ceph-mon[32473]: pgmap v193: 129 pgs: 129 active+clean; 82 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-04T09:37:30.336 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:37:30 smithi191 ceph-mon[36816]: pgmap v193: 129 pgs: 129 active+clean; 82 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-04T09:37:30.381 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:37:29 smithi106 ceph-mon[36484]: pgmap v193: 129 pgs: 129 active+clean; 82 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-04T09:37:32.335 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:37:32 smithi052 ceph-mon[32473]: pgmap v194: 129 pgs: 129 active+clean; 82 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-04T09:37:32.336 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:37:32 smithi191 ceph-mon[36816]: pgmap v194: 129 pgs: 129 active+clean; 82 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-04T09:37:32.381 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:37:32 smithi106 ceph-mon[36484]: pgmap v194: 129 pgs: 129 active+clean; 82 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-04T09:37:34.335 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:37:34 smithi052 ceph-mon[32473]: pgmap v195: 129 pgs: 129 active+clean; 82 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-04T09:37:34.336 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:37:34 smithi191 ceph-mon[36816]: pgmap v195: 129 pgs: 129 active+clean; 82 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-04T09:37:34.381 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:37:34 smithi106 ceph-mon[36484]: pgmap v195: 129 pgs: 129 active+clean; 82 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-04T09:37:36.335 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:37:36 smithi052 ceph-mon[32473]: pgmap v196: 129 pgs: 129 active+clean; 82 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-04T09:37:36.336 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:37:36 smithi191 ceph-mon[36816]: pgmap v196: 129 pgs: 129 active+clean; 82 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-04T09:37:36.381 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:37:36 smithi106 ceph-mon[36484]: pgmap v196: 129 pgs: 129 active+clean; 82 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-04T09:37:38.335 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:37:38 smithi052 ceph-mon[32473]: pgmap v197: 129 pgs: 129 active+clean; 82 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-04T09:37:38.336 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:37:38 smithi191 ceph-mon[36816]: pgmap v197: 129 pgs: 129 active+clean; 82 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-04T09:37:38.381 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:37:38 smithi106 ceph-mon[36484]: pgmap v197: 129 pgs: 129 active+clean; 82 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-04T09:37:40.335 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:37:40 smithi052 ceph-mon[32473]: pgmap v198: 129 pgs: 129 active+clean; 82 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-04T09:37:40.336 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:37:40 smithi191 ceph-mon[36816]: pgmap v198: 129 pgs: 129 active+clean; 82 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-04T09:37:40.381 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:37:40 smithi106 ceph-mon[36484]: pgmap v198: 129 pgs: 129 active+clean; 82 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-04T09:37:42.335 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:37:42 smithi052 ceph-mon[32473]: pgmap v199: 129 pgs: 129 active+clean; 82 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-04T09:37:42.336 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:37:42 smithi191 ceph-mon[36816]: pgmap v199: 129 pgs: 129 active+clean; 82 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-04T09:37:42.381 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:37:42 smithi106 ceph-mon[36484]: pgmap v199: 129 pgs: 129 active+clean; 82 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-04T09:37:44.335 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:37:44 smithi052 ceph-mon[32473]: pgmap v200: 129 pgs: 129 active+clean; 82 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-04T09:37:44.336 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:37:44 smithi191 ceph-mon[36816]: pgmap v200: 129 pgs: 129 active+clean; 82 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-04T09:37:44.381 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:37:44 smithi106 ceph-mon[36484]: pgmap v200: 129 pgs: 129 active+clean; 82 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-04T09:37:46.335 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:37:46 smithi052 ceph-mon[32473]: pgmap v201: 129 pgs: 129 active+clean; 82 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-04T09:37:46.336 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:37:46 smithi191 ceph-mon[36816]: pgmap v201: 129 pgs: 129 active+clean; 82 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-04T09:37:46.381 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:37:46 smithi106 ceph-mon[36484]: pgmap v201: 129 pgs: 129 active+clean; 82 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-04T09:37:48.335 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:37:48 smithi052 ceph-mon[32473]: pgmap v202: 129 pgs: 129 active+clean; 82 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-04T09:37:48.336 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:37:48 smithi191 ceph-mon[36816]: pgmap v202: 129 pgs: 129 active+clean; 82 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-04T09:37:48.381 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:37:48 smithi106 ceph-mon[36484]: pgmap v202: 129 pgs: 129 active+clean; 82 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-04T09:37:50.335 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:37:50 smithi052 ceph-mon[32473]: pgmap v203: 129 pgs: 129 active+clean; 82 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-04T09:37:50.336 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:37:50 smithi191 ceph-mon[36816]: pgmap v203: 129 pgs: 129 active+clean; 82 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-04T09:37:50.381 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:37:50 smithi106 ceph-mon[36484]: pgmap v203: 129 pgs: 129 active+clean; 82 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-04T09:37:52.335 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:37:52 smithi052 ceph-mon[32473]: pgmap v204: 129 pgs: 129 active+clean; 82 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-04T09:37:52.336 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:37:52 smithi191 ceph-mon[36816]: pgmap v204: 129 pgs: 129 active+clean; 82 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-04T09:37:52.381 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:37:52 smithi106 ceph-mon[36484]: pgmap v204: 129 pgs: 129 active+clean; 82 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-04T09:37:54.335 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:37:54 smithi052 ceph-mon[32473]: pgmap v205: 129 pgs: 129 active+clean; 82 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-04T09:37:54.336 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:37:54 smithi191 ceph-mon[36816]: pgmap v205: 129 pgs: 129 active+clean; 82 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-04T09:37:54.381 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:37:54 smithi106 ceph-mon[36484]: pgmap v205: 129 pgs: 129 active+clean; 82 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-04T09:37:56.335 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:37:56 smithi052 ceph-mon[32473]: pgmap v206: 129 pgs: 129 active+clean; 82 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-04T09:37:56.336 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:37:56 smithi191 ceph-mon[36816]: pgmap v206: 129 pgs: 129 active+clean; 82 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-04T09:37:56.381 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:37:56 smithi106 ceph-mon[36484]: pgmap v206: 129 pgs: 129 active+clean; 82 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-04T09:37:58.335 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:37:58 smithi052 ceph-mon[32473]: pgmap v207: 129 pgs: 129 active+clean; 82 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-04T09:37:58.336 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:37:58 smithi191 ceph-mon[36816]: pgmap v207: 129 pgs: 129 active+clean; 82 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-04T09:37:58.381 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:37:58 smithi106 ceph-mon[36484]: pgmap v207: 129 pgs: 129 active+clean; 82 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-04T09:38:00.335 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:38:00 smithi052 ceph-mon[32473]: pgmap v208: 129 pgs: 129 active+clean; 82 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-04T09:38:00.336 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:38:00 smithi191 ceph-mon[36816]: pgmap v208: 129 pgs: 129 active+clean; 82 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-04T09:38:00.381 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:38:00 smithi106 ceph-mon[36484]: pgmap v208: 129 pgs: 129 active+clean; 82 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-04T09:38:02.335 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:38:02 smithi052 ceph-mon[32473]: pgmap v209: 129 pgs: 129 active+clean; 82 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-04T09:38:02.336 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:38:02 smithi191 ceph-mon[36816]: pgmap v209: 129 pgs: 129 active+clean; 82 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-04T09:38:02.381 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:38:02 smithi106 ceph-mon[36484]: pgmap v209: 129 pgs: 129 active+clean; 82 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-04T09:38:04.335 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:38:04 smithi052 ceph-mon[32473]: pgmap v210: 129 pgs: 129 active+clean; 82 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-04T09:38:04.336 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:38:04 smithi191 ceph-mon[36816]: pgmap v210: 129 pgs: 129 active+clean; 82 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-04T09:38:04.381 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:38:04 smithi106 ceph-mon[36484]: pgmap v210: 129 pgs: 129 active+clean; 82 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-04T09:38:06.335 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:38:06 smithi052 ceph-mon[32473]: pgmap v211: 129 pgs: 129 active+clean; 82 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-03-04T09:38:06.336 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:38:06 smithi191 ceph-mon[36816]: pgmap v211: 129 pgs: 129 active+clean; 82 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-03-04T09:38:06.381 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:38:06 smithi106 ceph-mon[36484]: pgmap v211: 129 pgs: 129 active+clean; 82 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-03-04T09:38:08.335 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:38:08 smithi052 ceph-mon[32473]: pgmap v212: 129 pgs: 129 active+clean; 82 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-03-04T09:38:08.336 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:38:08 smithi191 ceph-mon[36816]: pgmap v212: 129 pgs: 129 active+clean; 82 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-03-04T09:38:08.381 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:38:08 smithi106 ceph-mon[36484]: pgmap v212: 129 pgs: 129 active+clean; 82 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-03-04T09:38:10.335 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:38:10 smithi052 ceph-mon[32473]: pgmap v213: 129 pgs: 129 active+clean; 82 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-03-04T09:38:10.335 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:38:10 smithi052 ceph-mon[32473]: osdmap e95: 12 total, 12 up, 12 in 2024-03-04T09:38:10.336 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:38:10 smithi191 ceph-mon[36816]: pgmap v213: 129 pgs: 129 active+clean; 82 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-03-04T09:38:10.336 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:38:10 smithi191 ceph-mon[36816]: osdmap e95: 12 total, 12 up, 12 in 2024-03-04T09:38:10.381 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:38:10 smithi106 ceph-mon[36484]: pgmap v213: 129 pgs: 129 active+clean; 82 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-03-04T09:38:10.381 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:38:10 smithi106 ceph-mon[36484]: osdmap e95: 12 total, 12 up, 12 in 2024-03-04T09:38:12.335 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:38:12 smithi052 ceph-mon[32473]: pgmap v215: 129 pgs: 129 active+clean; 82 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-03-04T09:38:12.336 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:38:12 smithi191 ceph-mon[36816]: pgmap v215: 129 pgs: 129 active+clean; 82 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-03-04T09:38:12.381 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:38:12 smithi106 ceph-mon[36484]: pgmap v215: 129 pgs: 129 active+clean; 82 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-03-04T09:38:14.335 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:38:14 smithi052 ceph-mon[32473]: pgmap v216: 129 pgs: 129 active+clean; 82 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-03-04T09:38:14.336 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:38:14 smithi191 ceph-mon[36816]: pgmap v216: 129 pgs: 129 active+clean; 82 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-03-04T09:38:14.381 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:38:14 smithi106 ceph-mon[36484]: pgmap v216: 129 pgs: 129 active+clean; 82 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-03-04T09:38:16.335 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:38:16 smithi052 ceph-mon[32473]: pgmap v217: 129 pgs: 129 active+clean; 83 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-03-04T09:38:16.335 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:38:16 smithi052 ceph-mon[32473]: osdmap e96: 12 total, 12 up, 12 in 2024-03-04T09:38:16.336 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:38:16 smithi191 ceph-mon[36816]: pgmap v217: 129 pgs: 129 active+clean; 83 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-03-04T09:38:16.336 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:38:16 smithi191 ceph-mon[36816]: osdmap e96: 12 total, 12 up, 12 in 2024-03-04T09:38:16.381 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:38:16 smithi106 ceph-mon[36484]: pgmap v217: 129 pgs: 129 active+clean; 83 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-03-04T09:38:16.381 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:38:16 smithi106 ceph-mon[36484]: osdmap e96: 12 total, 12 up, 12 in 2024-03-04T09:38:18.335 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:38:18 smithi052 ceph-mon[32473]: pgmap v219: 129 pgs: 129 active+clean; 83 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 255 B/s wr, 7 op/s 2024-03-04T09:38:18.336 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:38:18 smithi191 ceph-mon[36816]: pgmap v219: 129 pgs: 129 active+clean; 83 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 255 B/s wr, 7 op/s 2024-03-04T09:38:18.381 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:38:18 smithi106 ceph-mon[36484]: pgmap v219: 129 pgs: 129 active+clean; 83 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 255 B/s wr, 7 op/s 2024-03-04T09:38:19.335 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:38:19 smithi052 ceph-mon[32473]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T09:38:19.336 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:38:19 smithi191 ceph-mon[36816]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T09:38:19.381 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:38:19 smithi106 ceph-mon[36484]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T09:38:20.335 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:38:20 smithi052 ceph-mon[32473]: pgmap v220: 129 pgs: 129 active+clean; 83 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 215 B/s wr, 6 op/s 2024-03-04T09:38:20.335 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:38:20 smithi052 ceph-mon[32473]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T09:38:20.335 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:38:20 smithi052 ceph-mon[32473]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T09:38:20.335 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:38:20 smithi052 ceph-mon[32473]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' 2024-03-04T09:38:20.336 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:38:20 smithi191 ceph-mon[36816]: pgmap v220: 129 pgs: 129 active+clean; 83 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 215 B/s wr, 6 op/s 2024-03-04T09:38:20.336 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:38:20 smithi191 ceph-mon[36816]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T09:38:20.336 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:38:20 smithi191 ceph-mon[36816]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T09:38:20.336 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:38:20 smithi191 ceph-mon[36816]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' 2024-03-04T09:38:20.381 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:38:20 smithi106 ceph-mon[36484]: pgmap v220: 129 pgs: 129 active+clean; 83 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 215 B/s wr, 6 op/s 2024-03-04T09:38:20.381 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:38:20 smithi106 ceph-mon[36484]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T09:38:20.381 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:38:20 smithi106 ceph-mon[36484]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T09:38:20.381 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:38:20 smithi106 ceph-mon[36484]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' 2024-03-04T09:38:22.335 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:38:22 smithi052 ceph-mon[32473]: pgmap v221: 129 pgs: 129 active+clean; 83 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-03-04T09:38:22.336 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:38:22 smithi191 ceph-mon[36816]: pgmap v221: 129 pgs: 129 active+clean; 83 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-03-04T09:38:22.381 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:38:22 smithi106 ceph-mon[36484]: pgmap v221: 129 pgs: 129 active+clean; 83 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-03-04T09:38:24.335 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:38:24 smithi052 ceph-mon[32473]: pgmap v222: 129 pgs: 129 active+clean; 83 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-03-04T09:38:24.336 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:38:24 smithi191 ceph-mon[36816]: pgmap v222: 129 pgs: 129 active+clean; 83 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-03-04T09:38:24.381 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:38:24 smithi106 ceph-mon[36484]: pgmap v222: 129 pgs: 129 active+clean; 83 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-03-04T09:38:26.335 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:38:26 smithi052 ceph-mon[32473]: pgmap v223: 129 pgs: 129 active+clean; 83 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-04T09:38:26.336 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:38:26 smithi191 ceph-mon[36816]: pgmap v223: 129 pgs: 129 active+clean; 83 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-04T09:38:26.381 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:38:26 smithi106 ceph-mon[36484]: pgmap v223: 129 pgs: 129 active+clean; 83 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-04T09:38:28.335 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:38:28 smithi052 ceph-mon[32473]: pgmap v224: 129 pgs: 129 active+clean; 83 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-03-04T09:38:28.336 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:38:28 smithi191 ceph-mon[36816]: pgmap v224: 129 pgs: 129 active+clean; 83 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-03-04T09:38:28.381 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:38:28 smithi106 ceph-mon[36484]: pgmap v224: 129 pgs: 129 active+clean; 83 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-03-04T09:38:30.335 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:38:30 smithi052 ceph-mon[32473]: pgmap v225: 129 pgs: 129 active+clean; 83 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-04T09:38:30.336 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:38:30 smithi191 ceph-mon[36816]: pgmap v225: 129 pgs: 129 active+clean; 83 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-04T09:38:30.381 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:38:30 smithi106 ceph-mon[36484]: pgmap v225: 129 pgs: 129 active+clean; 83 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-04T09:38:32.335 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:38:32 smithi052 ceph-mon[32473]: pgmap v226: 129 pgs: 129 active+clean; 83 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-04T09:38:32.336 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:38:32 smithi191 ceph-mon[36816]: pgmap v226: 129 pgs: 129 active+clean; 83 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-04T09:38:32.381 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:38:32 smithi106 ceph-mon[36484]: pgmap v226: 129 pgs: 129 active+clean; 83 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-04T09:38:34.335 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:38:34 smithi052 ceph-mon[32473]: pgmap v227: 129 pgs: 129 active+clean; 83 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-04T09:38:34.336 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:38:34 smithi191 ceph-mon[36816]: pgmap v227: 129 pgs: 129 active+clean; 83 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-04T09:38:34.381 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:38:34 smithi106 ceph-mon[36484]: pgmap v227: 129 pgs: 129 active+clean; 83 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-04T09:38:36.335 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:38:36 smithi052 ceph-mon[32473]: pgmap v228: 129 pgs: 129 active+clean; 83 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-04T09:38:36.336 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:38:36 smithi191 ceph-mon[36816]: pgmap v228: 129 pgs: 129 active+clean; 83 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-04T09:38:36.381 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:38:36 smithi106 ceph-mon[36484]: pgmap v228: 129 pgs: 129 active+clean; 83 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-04T09:38:38.335 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:38:38 smithi052 ceph-mon[32473]: pgmap v229: 129 pgs: 129 active+clean; 83 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-04T09:38:38.336 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:38:38 smithi191 ceph-mon[36816]: pgmap v229: 129 pgs: 129 active+clean; 83 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-04T09:38:38.381 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:38:38 smithi106 ceph-mon[36484]: pgmap v229: 129 pgs: 129 active+clean; 83 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-04T09:38:40.381 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:38:40 smithi106 ceph-mon[36484]: pgmap v230: 129 pgs: 129 active+clean; 83 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-04T09:38:40.585 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:38:40 smithi052 ceph-mon[32473]: pgmap v230: 129 pgs: 129 active+clean; 83 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-04T09:38:40.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:38:40 smithi191 ceph-mon[36816]: pgmap v230: 129 pgs: 129 active+clean; 83 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-04T09:38:42.381 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:38:42 smithi106 ceph-mon[36484]: pgmap v231: 129 pgs: 129 active+clean; 83 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-04T09:38:42.585 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:38:42 smithi052 ceph-mon[32473]: pgmap v231: 129 pgs: 129 active+clean; 83 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-04T09:38:42.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:38:42 smithi191 ceph-mon[36816]: pgmap v231: 129 pgs: 129 active+clean; 83 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-04T09:38:44.381 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:38:44 smithi106 ceph-mon[36484]: pgmap v232: 129 pgs: 129 active+clean; 83 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-04T09:38:44.585 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:38:44 smithi052 ceph-mon[32473]: pgmap v232: 129 pgs: 129 active+clean; 83 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-04T09:38:44.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:38:44 smithi191 ceph-mon[36816]: pgmap v232: 129 pgs: 129 active+clean; 83 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-04T09:38:46.381 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:38:46 smithi106 ceph-mon[36484]: pgmap v233: 129 pgs: 129 active+clean; 83 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-04T09:38:46.585 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:38:46 smithi052 ceph-mon[32473]: pgmap v233: 129 pgs: 129 active+clean; 83 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-04T09:38:46.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:38:46 smithi191 ceph-mon[36816]: pgmap v233: 129 pgs: 129 active+clean; 83 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-04T09:38:48.381 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:38:48 smithi106 ceph-mon[36484]: pgmap v234: 129 pgs: 129 active+clean; 83 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-04T09:38:48.585 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:38:48 smithi052 ceph-mon[32473]: pgmap v234: 129 pgs: 129 active+clean; 83 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-04T09:38:48.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:38:48 smithi191 ceph-mon[36816]: pgmap v234: 129 pgs: 129 active+clean; 83 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-04T09:38:50.381 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:38:50 smithi106 ceph-mon[36484]: pgmap v235: 129 pgs: 129 active+clean; 83 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-04T09:38:50.585 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:38:50 smithi052 ceph-mon[32473]: pgmap v235: 129 pgs: 129 active+clean; 83 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-04T09:38:50.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:38:50 smithi191 ceph-mon[36816]: pgmap v235: 129 pgs: 129 active+clean; 83 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-04T09:38:52.381 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:38:52 smithi106 ceph-mon[36484]: pgmap v236: 129 pgs: 129 active+clean; 83 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-04T09:38:52.585 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:38:52 smithi052 ceph-mon[32473]: pgmap v236: 129 pgs: 129 active+clean; 83 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-04T09:38:52.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:38:52 smithi191 ceph-mon[36816]: pgmap v236: 129 pgs: 129 active+clean; 83 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-04T09:38:54.381 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:38:54 smithi106 ceph-mon[36484]: pgmap v237: 129 pgs: 129 active+clean; 83 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-04T09:38:54.585 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:38:54 smithi052 ceph-mon[32473]: pgmap v237: 129 pgs: 129 active+clean; 83 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-04T09:38:54.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:38:54 smithi191 ceph-mon[36816]: pgmap v237: 129 pgs: 129 active+clean; 83 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-04T09:38:56.381 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:38:56 smithi106 ceph-mon[36484]: pgmap v238: 129 pgs: 129 active+clean; 83 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-04T09:38:56.388 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:38:56 smithi191 ceph-mon[36816]: pgmap v238: 129 pgs: 129 active+clean; 83 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-04T09:38:56.585 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:38:56 smithi052 ceph-mon[32473]: pgmap v238: 129 pgs: 129 active+clean; 83 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-04T09:38:58.381 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:38:58 smithi106 ceph-mon[36484]: pgmap v239: 129 pgs: 129 active+clean; 83 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-04T09:38:58.585 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:38:58 smithi052 ceph-mon[32473]: pgmap v239: 129 pgs: 129 active+clean; 83 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-04T09:38:58.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:38:58 smithi191 ceph-mon[36816]: pgmap v239: 129 pgs: 129 active+clean; 83 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-04T09:39:00.381 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:39:00 smithi106 ceph-mon[36484]: pgmap v240: 129 pgs: 129 active+clean; 83 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-04T09:39:00.585 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:39:00 smithi052 ceph-mon[32473]: pgmap v240: 129 pgs: 129 active+clean; 83 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-04T09:39:00.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:39:00 smithi191 ceph-mon[36816]: pgmap v240: 129 pgs: 129 active+clean; 83 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-04T09:39:02.381 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:39:02 smithi106 ceph-mon[36484]: pgmap v241: 129 pgs: 129 active+clean; 83 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-04T09:39:02.585 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:39:02 smithi052 ceph-mon[32473]: pgmap v241: 129 pgs: 129 active+clean; 83 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-04T09:39:02.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:39:02 smithi191 ceph-mon[36816]: pgmap v241: 129 pgs: 129 active+clean; 83 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-04T09:39:04.381 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:39:04 smithi106 ceph-mon[36484]: pgmap v242: 129 pgs: 129 active+clean; 83 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-04T09:39:04.585 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:39:04 smithi052 ceph-mon[32473]: pgmap v242: 129 pgs: 129 active+clean; 83 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-04T09:39:04.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:39:04 smithi191 ceph-mon[36816]: pgmap v242: 129 pgs: 129 active+clean; 83 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-04T09:39:06.381 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:39:06 smithi106 ceph-mon[36484]: pgmap v243: 129 pgs: 129 active+clean; 83 MiB data, 740 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-03-04T09:39:06.584 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:39:06 smithi052 ceph-mon[32473]: pgmap v243: 129 pgs: 129 active+clean; 83 MiB data, 740 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-03-04T09:39:06.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:39:06 smithi191 ceph-mon[36816]: pgmap v243: 129 pgs: 129 active+clean; 83 MiB data, 740 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-03-04T09:39:08.388 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:39:08 smithi106 ceph-mon[36484]: pgmap v244: 129 pgs: 129 active+clean; 83 MiB data, 740 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-03-04T09:39:08.585 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:39:08 smithi052 ceph-mon[32473]: pgmap v244: 129 pgs: 129 active+clean; 83 MiB data, 740 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-03-04T09:39:08.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:39:08 smithi191 ceph-mon[36816]: pgmap v244: 129 pgs: 129 active+clean; 83 MiB data, 740 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-03-04T09:39:10.381 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:39:10 smithi106 ceph-mon[36484]: pgmap v245: 129 pgs: 129 active+clean; 83 MiB data, 740 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-03-04T09:39:10.381 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:39:10 smithi106 ceph-mon[36484]: osdmap e97: 12 total, 12 up, 12 in 2024-03-04T09:39:10.387 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:39:10 smithi052 ceph-mon[32473]: pgmap v245: 129 pgs: 129 active+clean; 83 MiB data, 740 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-03-04T09:39:10.387 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:39:10 smithi052 ceph-mon[32473]: osdmap e97: 12 total, 12 up, 12 in 2024-03-04T09:39:10.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:39:10 smithi191 ceph-mon[36816]: pgmap v245: 129 pgs: 129 active+clean; 83 MiB data, 740 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-03-04T09:39:10.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:39:10 smithi191 ceph-mon[36816]: osdmap e97: 12 total, 12 up, 12 in 2024-03-04T09:39:12.381 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:39:12 smithi106 ceph-mon[36484]: pgmap v247: 129 pgs: 18 active+clean+snaptrim, 55 active+clean, 56 active+clean+snaptrim_wait; 80 MiB data, 724 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-03-04T09:39:12.585 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:39:12 smithi052 ceph-mon[32473]: pgmap v247: 129 pgs: 18 active+clean+snaptrim, 55 active+clean, 56 active+clean+snaptrim_wait; 80 MiB data, 724 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-03-04T09:39:12.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:39:12 smithi191 ceph-mon[36816]: pgmap v247: 129 pgs: 18 active+clean+snaptrim, 55 active+clean, 56 active+clean+snaptrim_wait; 80 MiB data, 724 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-03-04T09:39:14.585 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:39:14 smithi052 ceph-mon[32473]: pgmap v248: 129 pgs: 18 active+clean+snaptrim, 55 active+clean, 56 active+clean+snaptrim_wait; 80 MiB data, 724 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-03-04T09:39:14.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:39:14 smithi191 ceph-mon[36816]: pgmap v248: 129 pgs: 18 active+clean+snaptrim, 55 active+clean, 56 active+clean+snaptrim_wait; 80 MiB data, 724 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-03-04T09:39:14.631 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:39:14 smithi106 ceph-mon[36484]: pgmap v248: 129 pgs: 18 active+clean+snaptrim, 55 active+clean, 56 active+clean+snaptrim_wait; 80 MiB data, 724 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-03-04T09:39:16.585 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:39:16 smithi052 ceph-mon[32473]: pgmap v249: 129 pgs: 129 active+clean; 78 MiB data, 718 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-03-04T09:39:16.586 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:39:16 smithi052 ceph-mon[32473]: osdmap e98: 12 total, 12 up, 12 in 2024-03-04T09:39:16.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:39:16 smithi191 ceph-mon[36816]: pgmap v249: 129 pgs: 129 active+clean; 78 MiB data, 718 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-03-04T09:39:16.587 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:39:16 smithi191 ceph-mon[36816]: osdmap e98: 12 total, 12 up, 12 in 2024-03-04T09:39:16.631 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:39:16 smithi106 ceph-mon[36484]: pgmap v249: 129 pgs: 129 active+clean; 78 MiB data, 718 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-03-04T09:39:16.631 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:39:16 smithi106 ceph-mon[36484]: osdmap e98: 12 total, 12 up, 12 in 2024-03-04T09:39:18.585 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:39:18 smithi052 ceph-mon[32473]: pgmap v251: 129 pgs: 129 active+clean; 78 MiB data, 718 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 255 B/s wr, 7 op/s 2024-03-04T09:39:18.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:39:18 smithi191 ceph-mon[36816]: pgmap v251: 129 pgs: 129 active+clean; 78 MiB data, 718 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 255 B/s wr, 7 op/s 2024-03-04T09:39:18.631 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:39:18 smithi106 ceph-mon[36484]: pgmap v251: 129 pgs: 129 active+clean; 78 MiB data, 718 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 255 B/s wr, 7 op/s 2024-03-04T09:39:20.585 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:39:20 smithi052 ceph-mon[32473]: pgmap v252: 129 pgs: 129 active+clean; 78 MiB data, 718 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 216 B/s wr, 6 op/s 2024-03-04T09:39:20.585 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:39:20 smithi052 ceph-mon[32473]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T09:39:20.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:39:20 smithi191 ceph-mon[36816]: pgmap v252: 129 pgs: 129 active+clean; 78 MiB data, 718 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 216 B/s wr, 6 op/s 2024-03-04T09:39:20.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:39:20 smithi191 ceph-mon[36816]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T09:39:20.631 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:39:20 smithi106 ceph-mon[36484]: pgmap v252: 129 pgs: 129 active+clean; 78 MiB data, 718 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 216 B/s wr, 6 op/s 2024-03-04T09:39:20.631 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:39:20 smithi106 ceph-mon[36484]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T09:39:21.585 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:39:21 smithi052 ceph-mon[32473]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T09:39:21.585 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:39:21 smithi052 ceph-mon[32473]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T09:39:21.585 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:39:21 smithi052 ceph-mon[32473]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' 2024-03-04T09:39:21.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:39:21 smithi191 ceph-mon[36816]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T09:39:21.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:39:21 smithi191 ceph-mon[36816]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T09:39:21.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:39:21 smithi191 ceph-mon[36816]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' 2024-03-04T09:39:21.631 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:39:21 smithi106 ceph-mon[36484]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T09:39:21.631 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:39:21 smithi106 ceph-mon[36484]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T09:39:21.632 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:39:21 smithi106 ceph-mon[36484]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' 2024-03-04T09:39:22.585 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:39:22 smithi052 ceph-mon[32473]: pgmap v253: 129 pgs: 129 active+clean; 78 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-03-04T09:39:22.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:39:22 smithi191 ceph-mon[36816]: pgmap v253: 129 pgs: 129 active+clean; 78 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-03-04T09:39:22.631 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:39:22 smithi106 ceph-mon[36484]: pgmap v253: 129 pgs: 129 active+clean; 78 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-03-04T09:39:24.585 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:39:24 smithi052 ceph-mon[32473]: pgmap v254: 129 pgs: 129 active+clean; 78 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-03-04T09:39:24.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:39:24 smithi191 ceph-mon[36816]: pgmap v254: 129 pgs: 129 active+clean; 78 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-03-04T09:39:24.631 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:39:24 smithi106 ceph-mon[36484]: pgmap v254: 129 pgs: 129 active+clean; 78 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-03-04T09:39:26.585 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:39:26 smithi052 ceph-mon[32473]: pgmap v255: 129 pgs: 129 active+clean; 78 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-04T09:39:26.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:39:26 smithi191 ceph-mon[36816]: pgmap v255: 129 pgs: 129 active+clean; 78 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-04T09:39:26.631 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:39:26 smithi106 ceph-mon[36484]: pgmap v255: 129 pgs: 129 active+clean; 78 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-04T09:39:28.585 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:39:28 smithi052 ceph-mon[32473]: pgmap v256: 129 pgs: 129 active+clean; 78 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-03-04T09:39:28.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:39:28 smithi191 ceph-mon[36816]: pgmap v256: 129 pgs: 129 active+clean; 78 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-03-04T09:39:28.631 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:39:28 smithi106 ceph-mon[36484]: pgmap v256: 129 pgs: 129 active+clean; 78 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-03-04T09:39:30.585 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:39:30 smithi052 ceph-mon[32473]: pgmap v257: 129 pgs: 129 active+clean; 78 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-04T09:39:30.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:39:30 smithi191 ceph-mon[36816]: pgmap v257: 129 pgs: 129 active+clean; 78 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-04T09:39:30.631 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:39:30 smithi106 ceph-mon[36484]: pgmap v257: 129 pgs: 129 active+clean; 78 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-04T09:39:32.585 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:39:32 smithi052 ceph-mon[32473]: pgmap v258: 129 pgs: 129 active+clean; 78 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-04T09:39:32.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:39:32 smithi191 ceph-mon[36816]: pgmap v258: 129 pgs: 129 active+clean; 78 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-04T09:39:32.631 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:39:32 smithi106 ceph-mon[36484]: pgmap v258: 129 pgs: 129 active+clean; 78 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-04T09:39:34.585 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:39:34 smithi052 ceph-mon[32473]: pgmap v259: 129 pgs: 129 active+clean; 78 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-04T09:39:34.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:39:34 smithi191 ceph-mon[36816]: pgmap v259: 129 pgs: 129 active+clean; 78 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-04T09:39:34.631 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:39:34 smithi106 ceph-mon[36484]: pgmap v259: 129 pgs: 129 active+clean; 78 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-04T09:39:36.585 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:39:36 smithi052 ceph-mon[32473]: pgmap v260: 129 pgs: 129 active+clean; 78 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-04T09:39:36.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:39:36 smithi191 ceph-mon[36816]: pgmap v260: 129 pgs: 129 active+clean; 78 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-04T09:39:36.631 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:39:36 smithi106 ceph-mon[36484]: pgmap v260: 129 pgs: 129 active+clean; 78 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-04T09:39:38.585 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:39:38 smithi052 ceph-mon[32473]: pgmap v261: 129 pgs: 129 active+clean; 78 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-04T09:39:38.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:39:38 smithi191 ceph-mon[36816]: pgmap v261: 129 pgs: 129 active+clean; 78 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-04T09:39:38.631 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:39:38 smithi106 ceph-mon[36484]: pgmap v261: 129 pgs: 129 active+clean; 78 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-04T09:39:40.585 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:39:40 smithi052 ceph-mon[32473]: pgmap v262: 129 pgs: 129 active+clean; 78 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-04T09:39:40.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:39:40 smithi191 ceph-mon[36816]: pgmap v262: 129 pgs: 129 active+clean; 78 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-04T09:39:40.631 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:39:40 smithi106 ceph-mon[36484]: pgmap v262: 129 pgs: 129 active+clean; 78 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-04T09:39:42.586 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:39:42 smithi191 ceph-mon[36816]: pgmap v263: 129 pgs: 129 active+clean; 78 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-04T09:39:42.631 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:39:42 smithi106 ceph-mon[36484]: pgmap v263: 129 pgs: 129 active+clean; 78 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-04T09:39:42.835 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:39:42 smithi052 ceph-mon[32473]: pgmap v263: 129 pgs: 129 active+clean; 78 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-04T09:39:44.631 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:39:44 smithi106 ceph-mon[36484]: pgmap v264: 129 pgs: 129 active+clean; 78 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-04T09:39:44.835 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:39:44 smithi052 ceph-mon[32473]: pgmap v264: 129 pgs: 129 active+clean; 78 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-04T09:39:44.836 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:39:44 smithi191 ceph-mon[36816]: pgmap v264: 129 pgs: 129 active+clean; 78 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-04T09:39:46.631 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:39:46 smithi106 ceph-mon[36484]: pgmap v265: 129 pgs: 129 active+clean; 78 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-04T09:39:46.835 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:39:46 smithi052 ceph-mon[32473]: pgmap v265: 129 pgs: 129 active+clean; 78 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-04T09:39:46.836 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:39:46 smithi191 ceph-mon[36816]: pgmap v265: 129 pgs: 129 active+clean; 78 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-04T09:39:48.631 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:39:48 smithi106 ceph-mon[36484]: pgmap v266: 129 pgs: 129 active+clean; 78 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-04T09:39:48.835 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:39:48 smithi052 ceph-mon[32473]: pgmap v266: 129 pgs: 129 active+clean; 78 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-04T09:39:48.836 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:39:48 smithi191 ceph-mon[36816]: pgmap v266: 129 pgs: 129 active+clean; 78 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-04T09:39:50.631 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:39:50 smithi106 ceph-mon[36484]: pgmap v267: 129 pgs: 129 active+clean; 78 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-04T09:39:50.835 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:39:50 smithi052 ceph-mon[32473]: pgmap v267: 129 pgs: 129 active+clean; 78 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-04T09:39:50.836 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:39:50 smithi191 ceph-mon[36816]: pgmap v267: 129 pgs: 129 active+clean; 78 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-04T09:39:52.631 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:39:52 smithi106 ceph-mon[36484]: pgmap v268: 129 pgs: 129 active+clean; 78 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-04T09:39:52.835 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:39:52 smithi052 ceph-mon[32473]: pgmap v268: 129 pgs: 129 active+clean; 78 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-04T09:39:52.836 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:39:52 smithi191 ceph-mon[36816]: pgmap v268: 129 pgs: 129 active+clean; 78 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-04T09:39:54.632 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:39:54 smithi106 ceph-mon[36484]: pgmap v269: 129 pgs: 129 active+clean; 78 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-04T09:39:54.835 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:39:54 smithi052 ceph-mon[32473]: pgmap v269: 129 pgs: 129 active+clean; 78 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-04T09:39:54.836 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:39:54 smithi191 ceph-mon[36816]: pgmap v269: 129 pgs: 129 active+clean; 78 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-04T09:39:56.631 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:39:56 smithi106 ceph-mon[36484]: pgmap v270: 129 pgs: 129 active+clean; 78 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-04T09:39:56.638 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:39:56 smithi191 ceph-mon[36816]: pgmap v270: 129 pgs: 129 active+clean; 78 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-04T09:39:56.835 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:39:56 smithi052 ceph-mon[32473]: pgmap v270: 129 pgs: 129 active+clean; 78 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-04T09:39:58.631 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:39:58 smithi106 ceph-mon[36484]: pgmap v271: 129 pgs: 129 active+clean; 78 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-04T09:39:58.835 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:39:58 smithi052 ceph-mon[32473]: pgmap v271: 129 pgs: 129 active+clean; 78 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-04T09:39:58.836 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:39:58 smithi191 ceph-mon[36816]: pgmap v271: 129 pgs: 129 active+clean; 78 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-04T09:40:00.631 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:40:00 smithi106 ceph-mon[36484]: pgmap v272: 129 pgs: 129 active+clean; 78 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-04T09:40:00.632 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:40:00 smithi106 ceph-mon[36484]: overall HEALTH_OK 2024-03-04T09:40:00.835 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:40:00 smithi052 ceph-mon[32473]: pgmap v272: 129 pgs: 129 active+clean; 78 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-04T09:40:00.835 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:40:00 smithi052 ceph-mon[32473]: overall HEALTH_OK 2024-03-04T09:40:00.836 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:40:00 smithi191 ceph-mon[36816]: pgmap v272: 129 pgs: 129 active+clean; 78 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-04T09:40:00.836 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:40:00 smithi191 ceph-mon[36816]: overall HEALTH_OK 2024-03-04T09:40:02.632 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:40:02 smithi106 ceph-mon[36484]: pgmap v273: 129 pgs: 129 active+clean; 78 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-04T09:40:02.835 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:40:02 smithi052 ceph-mon[32473]: pgmap v273: 129 pgs: 129 active+clean; 78 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-04T09:40:02.836 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:40:02 smithi191 ceph-mon[36816]: pgmap v273: 129 pgs: 129 active+clean; 78 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-04T09:40:04.631 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:40:04 smithi106 ceph-mon[36484]: pgmap v274: 129 pgs: 129 active+clean; 78 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-04T09:40:04.835 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:40:04 smithi052 ceph-mon[32473]: pgmap v274: 129 pgs: 129 active+clean; 78 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-04T09:40:04.836 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:40:04 smithi191 ceph-mon[36816]: pgmap v274: 129 pgs: 129 active+clean; 78 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-04T09:40:06.631 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:40:06 smithi106 ceph-mon[36484]: pgmap v275: 129 pgs: 129 active+clean; 78 MiB data, 686 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-03-04T09:40:06.835 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:40:06 smithi052 ceph-mon[32473]: pgmap v275: 129 pgs: 129 active+clean; 78 MiB data, 686 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-03-04T09:40:06.836 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:40:06 smithi191 ceph-mon[36816]: pgmap v275: 129 pgs: 129 active+clean; 78 MiB data, 686 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-03-04T09:40:08.631 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:40:08 smithi106 ceph-mon[36484]: pgmap v276: 129 pgs: 129 active+clean; 78 MiB data, 686 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-03-04T09:40:08.835 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:40:08 smithi052 ceph-mon[32473]: pgmap v276: 129 pgs: 129 active+clean; 78 MiB data, 686 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-03-04T09:40:08.836 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:40:08 smithi191 ceph-mon[36816]: pgmap v276: 129 pgs: 129 active+clean; 78 MiB data, 686 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-03-04T09:40:10.631 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:40:10 smithi106 ceph-mon[36484]: pgmap v277: 129 pgs: 129 active+clean; 78 MiB data, 686 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-03-04T09:40:10.632 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:40:10 smithi106 ceph-mon[36484]: osdmap e99: 12 total, 12 up, 12 in 2024-03-04T09:40:10.637 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:40:10 smithi052 ceph-mon[32473]: pgmap v277: 129 pgs: 129 active+clean; 78 MiB data, 686 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-03-04T09:40:10.637 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:40:10 smithi052 ceph-mon[32473]: osdmap e99: 12 total, 12 up, 12 in 2024-03-04T09:40:10.836 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:40:10 smithi191 ceph-mon[36816]: pgmap v277: 129 pgs: 129 active+clean; 78 MiB data, 686 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-03-04T09:40:10.837 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:40:10 smithi191 ceph-mon[36816]: osdmap e99: 12 total, 12 up, 12 in 2024-03-04T09:40:12.835 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:40:12 smithi052 ceph-mon[32473]: pgmap v279: 129 pgs: 129 active+clean; 78 MiB data, 687 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-03-04T09:40:12.836 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:40:12 smithi191 ceph-mon[36816]: pgmap v279: 129 pgs: 129 active+clean; 78 MiB data, 687 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-03-04T09:40:12.881 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:40:12 smithi106 ceph-mon[36484]: pgmap v279: 129 pgs: 129 active+clean; 78 MiB data, 687 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-03-04T09:40:14.835 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:40:14 smithi052 ceph-mon[32473]: pgmap v280: 129 pgs: 129 active+clean; 78 MiB data, 687 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-03-04T09:40:14.836 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:40:14 smithi191 ceph-mon[36816]: pgmap v280: 129 pgs: 129 active+clean; 78 MiB data, 687 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-03-04T09:40:14.881 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:40:14 smithi106 ceph-mon[36484]: pgmap v280: 129 pgs: 129 active+clean; 78 MiB data, 687 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-03-04T09:40:16.835 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:40:16 smithi052 ceph-mon[32473]: pgmap v281: 129 pgs: 129 active+clean; 78 MiB data, 686 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-03-04T09:40:16.835 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:40:16 smithi052 ceph-mon[32473]: osdmap e100: 12 total, 12 up, 12 in 2024-03-04T09:40:16.836 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:40:16 smithi191 ceph-mon[36816]: pgmap v281: 129 pgs: 129 active+clean; 78 MiB data, 686 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-03-04T09:40:16.836 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:40:16 smithi191 ceph-mon[36816]: osdmap e100: 12 total, 12 up, 12 in 2024-03-04T09:40:16.881 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:40:16 smithi106 ceph-mon[36484]: pgmap v281: 129 pgs: 129 active+clean; 78 MiB data, 686 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-03-04T09:40:16.882 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:40:16 smithi106 ceph-mon[36484]: osdmap e100: 12 total, 12 up, 12 in 2024-03-04T09:40:18.835 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:40:18 smithi052 ceph-mon[32473]: pgmap v283: 129 pgs: 129 active+clean; 78 MiB data, 686 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 255 B/s wr, 7 op/s 2024-03-04T09:40:18.836 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:40:18 smithi191 ceph-mon[36816]: pgmap v283: 129 pgs: 129 active+clean; 78 MiB data, 686 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 255 B/s wr, 7 op/s 2024-03-04T09:40:18.881 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:40:18 smithi106 ceph-mon[36484]: pgmap v283: 129 pgs: 129 active+clean; 78 MiB data, 686 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 255 B/s wr, 7 op/s 2024-03-04T09:40:20.835 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:40:20 smithi052 ceph-mon[32473]: pgmap v284: 129 pgs: 129 active+clean; 78 MiB data, 686 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 222 B/s wr, 6 op/s 2024-03-04T09:40:20.836 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:40:20 smithi191 ceph-mon[36816]: pgmap v284: 129 pgs: 129 active+clean; 78 MiB data, 686 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 222 B/s wr, 6 op/s 2024-03-04T09:40:20.881 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:40:20 smithi106 ceph-mon[36484]: pgmap v284: 129 pgs: 129 active+clean; 78 MiB data, 686 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 222 B/s wr, 6 op/s 2024-03-04T09:40:21.835 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:40:21 smithi052 ceph-mon[32473]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T09:40:21.835 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:40:21 smithi052 ceph-mon[32473]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T09:40:21.835 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:40:21 smithi052 ceph-mon[32473]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T09:40:21.836 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:40:21 smithi052 ceph-mon[32473]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' 2024-03-04T09:40:21.836 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:40:21 smithi191 ceph-mon[36816]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T09:40:21.836 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:40:21 smithi191 ceph-mon[36816]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T09:40:21.836 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:40:21 smithi191 ceph-mon[36816]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T09:40:21.837 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:40:21 smithi191 ceph-mon[36816]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' 2024-03-04T09:40:21.881 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:40:21 smithi106 ceph-mon[36484]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T09:40:21.882 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:40:21 smithi106 ceph-mon[36484]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T09:40:21.882 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:40:21 smithi106 ceph-mon[36484]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T09:40:21.882 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:40:21 smithi106 ceph-mon[36484]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' 2024-03-04T09:40:22.835 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:40:22 smithi052 ceph-mon[32473]: pgmap v285: 129 pgs: 129 active+clean; 78 MiB data, 687 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-03-04T09:40:22.836 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:40:22 smithi191 ceph-mon[36816]: pgmap v285: 129 pgs: 129 active+clean; 78 MiB data, 687 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-03-04T09:40:22.881 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:40:22 smithi106 ceph-mon[36484]: pgmap v285: 129 pgs: 129 active+clean; 78 MiB data, 687 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-03-04T09:40:24.835 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:40:24 smithi052 ceph-mon[32473]: pgmap v286: 129 pgs: 129 active+clean; 78 MiB data, 687 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-03-04T09:40:24.836 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:40:24 smithi191 ceph-mon[36816]: pgmap v286: 129 pgs: 129 active+clean; 78 MiB data, 687 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-03-04T09:40:24.881 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:40:24 smithi106 ceph-mon[36484]: pgmap v286: 129 pgs: 129 active+clean; 78 MiB data, 687 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-03-04T09:40:26.835 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:40:26 smithi052 ceph-mon[32473]: pgmap v287: 129 pgs: 129 active+clean; 78 MiB data, 687 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-04T09:40:26.836 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:40:26 smithi191 ceph-mon[36816]: pgmap v287: 129 pgs: 129 active+clean; 78 MiB data, 687 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-04T09:40:26.881 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:40:26 smithi106 ceph-mon[36484]: pgmap v287: 129 pgs: 129 active+clean; 78 MiB data, 687 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-04T09:40:28.835 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:40:28 smithi052 ceph-mon[32473]: pgmap v288: 129 pgs: 129 active+clean; 78 MiB data, 687 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-03-04T09:40:28.836 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:40:28 smithi191 ceph-mon[36816]: pgmap v288: 129 pgs: 129 active+clean; 78 MiB data, 687 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-03-04T09:40:28.881 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:40:28 smithi106 ceph-mon[36484]: pgmap v288: 129 pgs: 129 active+clean; 78 MiB data, 687 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-03-04T09:40:30.835 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:40:30 smithi052 ceph-mon[32473]: pgmap v289: 129 pgs: 129 active+clean; 78 MiB data, 687 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-04T09:40:30.836 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:40:30 smithi191 ceph-mon[36816]: pgmap v289: 129 pgs: 129 active+clean; 78 MiB data, 687 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-04T09:40:30.881 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:40:30 smithi106 ceph-mon[36484]: pgmap v289: 129 pgs: 129 active+clean; 78 MiB data, 687 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-04T09:40:32.684 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:40:32 smithi106 ceph-mon[36484]: pgmap v290: 129 pgs: 129 active+clean; 78 MiB data, 687 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-04T09:40:32.835 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:40:32 smithi052 ceph-mon[32473]: pgmap v290: 129 pgs: 129 active+clean; 78 MiB data, 687 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-04T09:40:32.836 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:40:32 smithi191 ceph-mon[36816]: pgmap v290: 129 pgs: 129 active+clean; 78 MiB data, 687 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-04T09:40:34.631 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:40:34 smithi106 ceph-mon[36484]: pgmap v291: 129 pgs: 129 active+clean; 78 MiB data, 687 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-04T09:40:34.835 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:40:34 smithi052 ceph-mon[32473]: pgmap v291: 129 pgs: 129 active+clean; 78 MiB data, 687 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-04T09:40:34.836 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:40:34 smithi191 ceph-mon[36816]: pgmap v291: 129 pgs: 129 active+clean; 78 MiB data, 687 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-04T09:40:36.835 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:40:36 smithi052 ceph-mon[32473]: pgmap v292: 129 pgs: 129 active+clean; 78 MiB data, 687 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-04T09:40:36.836 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:40:36 smithi191 ceph-mon[36816]: pgmap v292: 129 pgs: 129 active+clean; 78 MiB data, 687 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-04T09:40:36.882 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:40:36 smithi106 ceph-mon[36484]: pgmap v292: 129 pgs: 129 active+clean; 78 MiB data, 687 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-04T09:40:38.835 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:40:38 smithi052 ceph-mon[32473]: pgmap v293: 129 pgs: 129 active+clean; 78 MiB data, 687 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-04T09:40:38.836 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:40:38 smithi191 ceph-mon[36816]: pgmap v293: 129 pgs: 129 active+clean; 78 MiB data, 687 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-04T09:40:38.881 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:40:38 smithi106 ceph-mon[36484]: pgmap v293: 129 pgs: 129 active+clean; 78 MiB data, 687 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-04T09:40:40.835 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:40:40 smithi052 ceph-mon[32473]: pgmap v294: 129 pgs: 129 active+clean; 78 MiB data, 687 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-04T09:40:40.836 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:40:40 smithi191 ceph-mon[36816]: pgmap v294: 129 pgs: 129 active+clean; 78 MiB data, 687 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-04T09:40:40.881 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:40:40 smithi106 ceph-mon[36484]: pgmap v294: 129 pgs: 129 active+clean; 78 MiB data, 687 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-04T09:40:42.835 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:40:42 smithi052 ceph-mon[32473]: pgmap v295: 129 pgs: 129 active+clean; 78 MiB data, 687 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-04T09:40:42.836 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:40:42 smithi191 ceph-mon[36816]: pgmap v295: 129 pgs: 129 active+clean; 78 MiB data, 687 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-04T09:40:42.881 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:40:42 smithi106 ceph-mon[36484]: pgmap v295: 129 pgs: 129 active+clean; 78 MiB data, 687 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-04T09:40:44.835 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:40:44 smithi052 ceph-mon[32473]: pgmap v296: 129 pgs: 129 active+clean; 78 MiB data, 687 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-04T09:40:44.836 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:40:44 smithi191 ceph-mon[36816]: pgmap v296: 129 pgs: 129 active+clean; 78 MiB data, 687 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-04T09:40:44.881 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:40:44 smithi106 ceph-mon[36484]: pgmap v296: 129 pgs: 129 active+clean; 78 MiB data, 687 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-04T09:40:46.835 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:40:46 smithi052 ceph-mon[32473]: pgmap v297: 129 pgs: 129 active+clean; 78 MiB data, 687 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-04T09:40:46.837 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:40:46 smithi191 ceph-mon[36816]: pgmap v297: 129 pgs: 129 active+clean; 78 MiB data, 687 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-04T09:40:46.883 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:40:46 smithi106 ceph-mon[36484]: pgmap v297: 129 pgs: 129 active+clean; 78 MiB data, 687 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-04T09:40:48.835 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:40:48 smithi052 ceph-mon[32473]: pgmap v298: 129 pgs: 129 active+clean; 78 MiB data, 687 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-04T09:40:48.836 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:40:48 smithi191 ceph-mon[36816]: pgmap v298: 129 pgs: 129 active+clean; 78 MiB data, 687 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-04T09:40:48.882 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:40:48 smithi106 ceph-mon[36484]: pgmap v298: 129 pgs: 129 active+clean; 78 MiB data, 687 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-04T09:40:50.835 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:40:50 smithi052 ceph-mon[32473]: pgmap v299: 129 pgs: 129 active+clean; 78 MiB data, 687 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-04T09:40:50.836 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:40:50 smithi191 ceph-mon[36816]: pgmap v299: 129 pgs: 129 active+clean; 78 MiB data, 687 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-04T09:40:50.881 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:40:50 smithi106 ceph-mon[36484]: pgmap v299: 129 pgs: 129 active+clean; 78 MiB data, 687 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-04T09:40:52.835 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:40:52 smithi052 ceph-mon[32473]: pgmap v300: 129 pgs: 129 active+clean; 78 MiB data, 687 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-04T09:40:52.836 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:40:52 smithi191 ceph-mon[36816]: pgmap v300: 129 pgs: 129 active+clean; 78 MiB data, 687 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-04T09:40:52.881 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:40:52 smithi106 ceph-mon[36484]: pgmap v300: 129 pgs: 129 active+clean; 78 MiB data, 687 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-04T09:40:54.835 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:40:54 smithi052 ceph-mon[32473]: pgmap v301: 129 pgs: 129 active+clean; 78 MiB data, 687 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-04T09:40:54.837 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:40:54 smithi191 ceph-mon[36816]: pgmap v301: 129 pgs: 129 active+clean; 78 MiB data, 687 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-04T09:40:54.882 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:40:54 smithi106 ceph-mon[36484]: pgmap v301: 129 pgs: 129 active+clean; 78 MiB data, 687 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-04T09:40:56.835 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:40:56 smithi052 ceph-mon[32473]: pgmap v302: 129 pgs: 129 active+clean; 78 MiB data, 687 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-04T09:40:56.836 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:40:56 smithi191 ceph-mon[36816]: pgmap v302: 129 pgs: 129 active+clean; 78 MiB data, 687 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-04T09:40:56.881 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:40:56 smithi106 ceph-mon[36484]: pgmap v302: 129 pgs: 129 active+clean; 78 MiB data, 687 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-04T09:40:58.835 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:40:58 smithi052 ceph-mon[32473]: pgmap v303: 129 pgs: 129 active+clean; 78 MiB data, 687 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-04T09:40:58.837 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:40:58 smithi191 ceph-mon[36816]: pgmap v303: 129 pgs: 129 active+clean; 78 MiB data, 687 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-04T09:40:58.882 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:40:58 smithi106 ceph-mon[36484]: pgmap v303: 129 pgs: 129 active+clean; 78 MiB data, 687 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-04T09:41:00.835 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:41:00 smithi052 ceph-mon[32473]: pgmap v304: 129 pgs: 129 active+clean; 78 MiB data, 687 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-04T09:41:00.837 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:41:00 smithi191 ceph-mon[36816]: pgmap v304: 129 pgs: 129 active+clean; 78 MiB data, 687 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-04T09:41:00.882 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:41:00 smithi106 ceph-mon[36484]: pgmap v304: 129 pgs: 129 active+clean; 78 MiB data, 687 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-04T09:41:02.836 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:41:02 smithi052 ceph-mon[32473]: pgmap v305: 129 pgs: 129 active+clean; 78 MiB data, 687 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-04T09:41:02.837 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:41:02 smithi191 ceph-mon[36816]: pgmap v305: 129 pgs: 129 active+clean; 78 MiB data, 687 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-04T09:41:02.882 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:41:02 smithi106 ceph-mon[36484]: pgmap v305: 129 pgs: 129 active+clean; 78 MiB data, 687 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-04T09:41:04.836 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:41:04 smithi052 ceph-mon[32473]: pgmap v306: 129 pgs: 129 active+clean; 78 MiB data, 687 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-04T09:41:04.837 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:41:04 smithi191 ceph-mon[36816]: pgmap v306: 129 pgs: 129 active+clean; 78 MiB data, 687 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-04T09:41:04.882 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:41:04 smithi106 ceph-mon[36484]: pgmap v306: 129 pgs: 129 active+clean; 78 MiB data, 687 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-04T09:41:06.835 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:41:06 smithi052 ceph-mon[32473]: pgmap v307: 129 pgs: 129 active+clean; 78 MiB data, 687 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-03-04T09:41:06.837 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:41:06 smithi191 ceph-mon[36816]: pgmap v307: 129 pgs: 129 active+clean; 78 MiB data, 687 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-03-04T09:41:06.881 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:41:06 smithi106 ceph-mon[36484]: pgmap v307: 129 pgs: 129 active+clean; 78 MiB data, 687 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-03-04T09:41:08.835 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:41:08 smithi052 ceph-mon[32473]: pgmap v308: 129 pgs: 129 active+clean; 78 MiB data, 687 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-03-04T09:41:08.837 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:41:08 smithi191 ceph-mon[36816]: pgmap v308: 129 pgs: 129 active+clean; 78 MiB data, 687 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-03-04T09:41:08.882 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:41:08 smithi106 ceph-mon[36484]: pgmap v308: 129 pgs: 129 active+clean; 78 MiB data, 687 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-03-04T09:41:10.315 DEBUG:teuthology.orchestra.run.smithi052:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2024-03-04T09:41:10.434 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:10.433+0000 7fcb334d6640 1 -- 172.21.15.52:0/2718296848 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcb2c073970 msgr2=0x7fcb2c073d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:41:10.434 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:10.433+0000 7fcb334d6640 1 --2- 172.21.15.52:0/2718296848 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcb2c073970 0x7fcb2c073d50 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fcb28009900 tx=0x7fcb2802bcd0 comp rx=0 tx=0).stop 2024-03-04T09:41:10.434 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:10.434+0000 7fcb334d6640 1 -- 172.21.15.52:0/2718296848 shutdown_connections 2024-03-04T09:41:10.434 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:10.434+0000 7fcb334d6640 1 --2- 172.21.15.52:0/2718296848 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcb2c112590 0x7fcb2c114980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:41:10.435 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:10.434+0000 7fcb334d6640 1 --2- 172.21.15.52:0/2718296848 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcb2c074290 0x7fcb2c112050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:41:10.435 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:10.434+0000 7fcb334d6640 1 --2- 172.21.15.52:0/2718296848 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcb2c073970 0x7fcb2c073d50 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:41:10.435 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:10.434+0000 7fcb334d6640 1 -- 172.21.15.52:0/2718296848 >> 172.21.15.52:0/2718296848 conn(0x7fcb2c0fd090 msgr2=0x7fcb2c0ff4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:41:10.435 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:10.434+0000 7fcb334d6640 1 -- 172.21.15.52:0/2718296848 shutdown_connections 2024-03-04T09:41:10.435 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:10.434+0000 7fcb334d6640 1 -- 172.21.15.52:0/2718296848 wait complete. 2024-03-04T09:41:10.435 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:10.434+0000 7fcb334d6640 1 Processor -- start 2024-03-04T09:41:10.435 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:10.434+0000 7fcb334d6640 1 -- start start 2024-03-04T09:41:10.435 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:10.434+0000 7fcb334d6640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcb2c073970 0x7fcb2c19e470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:41:10.436 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:10.434+0000 7fcb334d6640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcb2c074290 0x7fcb2c19e9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:41:10.436 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:10.434+0000 7fcb334d6640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcb2c112590 0x7fcb2c1a2ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:41:10.436 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:10.434+0000 7fcb334d6640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb2c117f90 con 0x7fcb2c112590 2024-03-04T09:41:10.436 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:10.434+0000 7fcb334d6640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb2c117e10 con 0x7fcb2c073970 2024-03-04T09:41:10.436 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:10.434+0000 7fcb334d6640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb2c118110 con 0x7fcb2c074290 2024-03-04T09:41:10.436 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:10.435+0000 7fcb31a73640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcb2c112590 0x7fcb2c1a2ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:41:10.436 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:10.435+0000 7fcb31a73640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcb2c112590 0x7fcb2c1a2ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:36866/0 (socket says 172.21.15.52:36866) 2024-03-04T09:41:10.437 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:10.435+0000 7fcb31a73640 1 -- 172.21.15.52:0/2836189030 learned_addr learned my addr 172.21.15.52:0/2836189030 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-04T09:41:10.437 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:10.435+0000 7fcb30a71640 1 --2- 172.21.15.52:0/2836189030 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcb2c074290 0x7fcb2c19e9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:41:10.437 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:10.435+0000 7fcb31272640 1 --2- 172.21.15.52:0/2836189030 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcb2c073970 0x7fcb2c19e470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:41:10.437 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:10.435+0000 7fcb31a73640 1 -- 172.21.15.52:0/2836189030 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcb2c074290 msgr2=0x7fcb2c19e9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:41:10.437 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:10.435+0000 7fcb31a73640 1 --2- 172.21.15.52:0/2836189030 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcb2c074290 0x7fcb2c19e9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:41:10.437 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:10.435+0000 7fcb31a73640 1 -- 172.21.15.52:0/2836189030 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcb2c073970 msgr2=0x7fcb2c19e470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:41:10.437 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:10.435+0000 7fcb31a73640 1 --2- 172.21.15.52:0/2836189030 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcb2c073970 0x7fcb2c19e470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:41:10.438 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:10.435+0000 7fcb31a73640 1 -- 172.21.15.52:0/2836189030 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb2c1a3410 con 0x7fcb2c112590 2024-03-04T09:41:10.438 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:10.435+0000 7fcb31a73640 1 --2- 172.21.15.52:0/2836189030 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcb2c112590 0x7fcb2c1a2ce0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fcb24009c90 tx=0x7fcb2400c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:41:10.438 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:10.435+0000 7fcb227fc640 1 -- 172.21.15.52:0/2836189030 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb24003b70 con 0x7fcb2c112590 2024-03-04T09:41:10.438 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:10.435+0000 7fcb227fc640 1 -- 172.21.15.52:0/2836189030 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fcb24003d10 con 0x7fcb2c112590 2024-03-04T09:41:10.438 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:10.435+0000 7fcb334d6640 1 -- 172.21.15.52:0/2836189030 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb2c1a3700 con 0x7fcb2c112590 2024-03-04T09:41:10.438 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:10.435+0000 7fcb227fc640 1 -- 172.21.15.52:0/2836189030 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb24010960 con 0x7fcb2c112590 2024-03-04T09:41:10.439 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:10.435+0000 7fcb334d6640 1 -- 172.21.15.52:0/2836189030 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb2c1aaf90 con 0x7fcb2c112590 2024-03-04T09:41:10.439 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:10.436+0000 7fcb334d6640 1 -- 172.21.15.52:0/2836189030 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb2c04f330 con 0x7fcb2c112590 2024-03-04T09:41:10.440 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:10.437+0000 7fcb227fc640 1 -- 172.21.15.52:0/2836189030 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 25) v1 ==== 136887+0+0 (secure 0 0 0) 0x7fcb24004020 con 0x7fcb2c112590 2024-03-04T09:41:10.440 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:10.438+0000 7fcb227fc640 1 --2- 172.21.15.52:0/2836189030 >> [v2:172.21.15.52:6800/478031757,v1:172.21.15.52:6801/478031757] conn(0x7fcb180a2010 0x7fcb180a44d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:41:10.440 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:10.438+0000 7fcb227fc640 1 -- 172.21.15.52:0/2836189030 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 9635+0+0 (secure 0 0 0) 0x7fcb240ceb90 con 0x7fcb2c112590 2024-03-04T09:41:10.440 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:10.438+0000 7fcb31272640 1 --2- 172.21.15.52:0/2836189030 >> [v2:172.21.15.52:6800/478031757,v1:172.21.15.52:6801/478031757] conn(0x7fcb180a2010 0x7fcb180a44d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:41:10.440 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:10.439+0000 7fcb31272640 1 --2- 172.21.15.52:0/2836189030 >> [v2:172.21.15.52:6800/478031757,v1:172.21.15.52:6801/478031757] conn(0x7fcb180a2010 0x7fcb180a44d0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fcb2802efd0 tx=0x7fcb2802cf90 comp rx=0 tx=0).ready entity=mgr.15486 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:41:10.441 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:10.443+0000 7fcb227fc640 1 -- 172.21.15.52:0/2836189030 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fcb2409b7c0 con 0x7fcb2c112590 2024-03-04T09:41:10.585 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:41:10 smithi052 ceph-mon[32473]: pgmap v309: 129 pgs: 129 active+clean; 78 MiB data, 687 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-03-04T09:41:10.586 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:41:10 smithi052 ceph-mon[32473]: osdmap e101: 12 total, 12 up, 12 in 2024-03-04T09:41:10.647 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:10.649+0000 7fcb334d6640 1 -- 172.21.15.52:0/2836189030 --> [v2:172.21.15.52:6800/478031757,v1:172.21.15.52:6801/478031757] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) v1 -- 0x7fcb2c0feed0 con 0x7fcb180a2010 2024-03-04T09:41:10.653 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:10.656+0000 7fcb227fc640 1 -- 172.21.15.52:0/2836189030 <== mgr.15486 v2:172.21.15.52:6800/478031757 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+350 (secure 0 0 0) 0x7fcb2c0feed0 con 0x7fcb180a2010 2024-03-04T09:41:10.654 INFO:teuthology.orchestra.run.smithi052.stdout:{"fs": "cephfs", "subvol": null, "group": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2024-03-04T00:00:00", "created": "2024-03-04T09:31:09", "first": "2024-03-04T09:32:00", "last": "2024-03-04T09:41:00", "last_pruned": "2024-03-04T09:41:00", "created_count": 10, "pruned_count": 4, "active": true} 2024-03-04T09:41:10.656 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:10.658+0000 7fcb334d6640 1 -- 172.21.15.52:0/2836189030 >> [v2:172.21.15.52:6800/478031757,v1:172.21.15.52:6801/478031757] conn(0x7fcb180a2010 msgr2=0x7fcb180a44d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:41:10.656 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:10.658+0000 7fcb334d6640 1 --2- 172.21.15.52:0/2836189030 >> [v2:172.21.15.52:6800/478031757,v1:172.21.15.52:6801/478031757] conn(0x7fcb180a2010 0x7fcb180a44d0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fcb2802efd0 tx=0x7fcb2802cf90 comp rx=0 tx=0).stop 2024-03-04T09:41:10.656 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:10.658+0000 7fcb334d6640 1 -- 172.21.15.52:0/2836189030 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcb2c112590 msgr2=0x7fcb2c1a2ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:41:10.656 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:10.658+0000 7fcb334d6640 1 --2- 172.21.15.52:0/2836189030 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcb2c112590 0x7fcb2c1a2ce0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fcb24009c90 tx=0x7fcb2400c6a0 comp rx=0 tx=0).stop 2024-03-04T09:41:10.656 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:10.658+0000 7fcb334d6640 1 -- 172.21.15.52:0/2836189030 shutdown_connections 2024-03-04T09:41:10.657 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:10.658+0000 7fcb334d6640 1 --2- 172.21.15.52:0/2836189030 >> [v2:172.21.15.52:6800/478031757,v1:172.21.15.52:6801/478031757] conn(0x7fcb180a2010 0x7fcb180a44d0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:41:10.657 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:10.658+0000 7fcb334d6640 1 --2- 172.21.15.52:0/2836189030 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcb2c112590 0x7fcb2c1a2ce0 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:41:10.657 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:10.658+0000 7fcb334d6640 1 --2- 172.21.15.52:0/2836189030 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcb2c074290 0x7fcb2c19e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:41:10.657 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:10.658+0000 7fcb334d6640 1 --2- 172.21.15.52:0/2836189030 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcb2c073970 0x7fcb2c19e470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:41:10.657 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:10.658+0000 7fcb334d6640 1 -- 172.21.15.52:0/2836189030 >> 172.21.15.52:0/2836189030 conn(0x7fcb2c0fd090 msgr2=0x7fcb2c0fe7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:41:10.657 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:10.658+0000 7fcb334d6640 1 -- 172.21.15.52:0/2836189030 shutdown_connections 2024-03-04T09:41:10.658 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:10.658+0000 7fcb334d6640 1 -- 172.21.15.52:0/2836189030 wait complete. 2024-03-04T09:41:10.666 DEBUG:teuthology.orchestra.run.smithi052:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2024-03-04T09:41:10.770 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:10.772+0000 7f6d625f6640 1 -- 172.21.15.52:0/113159669 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6d5c1010c0 msgr2=0x7f6d5c10d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:41:10.771 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:10.772+0000 7f6d625f6640 1 --2- 172.21.15.52:0/113159669 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6d5c1010c0 0x7f6d5c10d300 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f6d54009ca0 tx=0x7f6d5402c0f0 comp rx=0 tx=0).stop 2024-03-04T09:41:10.771 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:10.772+0000 7f6d625f6640 1 -- 172.21.15.52:0/113159669 shutdown_connections 2024-03-04T09:41:10.771 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:10.772+0000 7f6d625f6640 1 --2- 172.21.15.52:0/113159669 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6d5c1010c0 0x7f6d5c10d300 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:41:10.771 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:10.772+0000 7f6d625f6640 1 --2- 172.21.15.52:0/113159669 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6d5c100720 0x7f6d5c100b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:41:10.771 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:10.772+0000 7f6d625f6640 1 --2- 172.21.15.52:0/113159669 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6d5c105450 0x7f6d5c105830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:41:10.771 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:10.772+0000 7f6d625f6640 1 -- 172.21.15.52:0/113159669 >> 172.21.15.52:0/113159669 conn(0x7f6d5c0772a0 msgr2=0x7f6d5c0776b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:41:10.771 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:10.772+0000 7f6d625f6640 1 -- 172.21.15.52:0/113159669 shutdown_connections 2024-03-04T09:41:10.771 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:10.772+0000 7f6d625f6640 1 -- 172.21.15.52:0/113159669 wait complete. 2024-03-04T09:41:10.772 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:10.773+0000 7f6d625f6640 1 Processor -- start 2024-03-04T09:41:10.772 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:10.773+0000 7f6d625f6640 1 -- start start 2024-03-04T09:41:10.774 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:10.773+0000 7f6d625f6640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6d5c100720 0x7f6d5c198c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:41:10.774 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:10.773+0000 7f6d625f6640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6d5c1010c0 0x7f6d5c199180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:41:10.774 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:10.773+0000 7f6d625f6640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6d5c105450 0x7f6d5c1a3cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:41:10.774 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:10.773+0000 7f6d625f6640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d5c110900 con 0x7f6d5c100720 2024-03-04T09:41:10.775 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:10.773+0000 7f6d625f6640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d5c110780 con 0x7f6d5c105450 2024-03-04T09:41:10.775 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:10.773+0000 7f6d625f6640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d5c110a80 con 0x7f6d5c1010c0 2024-03-04T09:41:10.775 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:10.773+0000 7f6d5bfff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6d5c100720 0x7f6d5c198c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:41:10.775 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:10.773+0000 7f6d5bfff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6d5c100720 0x7f6d5c198c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:36888/0 (socket says 172.21.15.52:36888) 2024-03-04T09:41:10.775 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:10.773+0000 7f6d5bfff640 1 -- 172.21.15.52:0/759737660 learned_addr learned my addr 172.21.15.52:0/759737660 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-04T09:41:10.775 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:10.773+0000 7f6d60b93640 1 --2- 172.21.15.52:0/759737660 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6d5c105450 0x7f6d5c1a3cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:41:10.775 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:10.773+0000 7f6d5b7fe640 1 --2- 172.21.15.52:0/759737660 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6d5c1010c0 0x7f6d5c199180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:41:10.775 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:10.774+0000 7f6d5bfff640 1 -- 172.21.15.52:0/759737660 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6d5c1010c0 msgr2=0x7f6d5c199180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:41:10.776 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:10.774+0000 7f6d5bfff640 1 --2- 172.21.15.52:0/759737660 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6d5c1010c0 0x7f6d5c199180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:41:10.776 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:10.774+0000 7f6d5bfff640 1 -- 172.21.15.52:0/759737660 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6d5c105450 msgr2=0x7f6d5c1a3cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:41:10.776 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:10.774+0000 7f6d5bfff640 1 --2- 172.21.15.52:0/759737660 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6d5c105450 0x7f6d5c1a3cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:41:10.776 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:10.774+0000 7f6d5bfff640 1 -- 172.21.15.52:0/759737660 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d5c1a4200 con 0x7f6d5c100720 2024-03-04T09:41:10.776 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:10.774+0000 7f6d5bfff640 1 --2- 172.21.15.52:0/759737660 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6d5c100720 0x7f6d5c198c40 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f6d4c00e9f0 tx=0x7f6d4c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:41:10.776 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:10.774+0000 7f6d597fa640 1 -- 172.21.15.52:0/759737660 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d4c00cd80 con 0x7f6d5c100720 2024-03-04T09:41:10.776 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:10.774+0000 7f6d625f6640 1 -- 172.21.15.52:0/759737660 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d5c1a43d0 con 0x7f6d5c100720 2024-03-04T09:41:10.776 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:10.774+0000 7f6d597fa640 1 -- 172.21.15.52:0/759737660 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6d4c010910 con 0x7f6d5c100720 2024-03-04T09:41:10.777 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:10.774+0000 7f6d597fa640 1 -- 172.21.15.52:0/759737660 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d4c0188f0 con 0x7f6d5c100720 2024-03-04T09:41:10.777 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:10.774+0000 7f6d625f6640 1 -- 172.21.15.52:0/759737660 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d5c1a4940 con 0x7f6d5c100720 2024-03-04T09:41:10.777 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:10.775+0000 7f6d625f6640 1 -- 172.21.15.52:0/759737660 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6d5c101be0 con 0x7f6d5c100720 2024-03-04T09:41:10.777 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:10.776+0000 7f6d597fa640 1 -- 172.21.15.52:0/759737660 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 25) v1 ==== 136887+0+0 (secure 0 0 0) 0x7f6d4c010ac0 con 0x7f6d5c100720 2024-03-04T09:41:10.777 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:10.776+0000 7f6d597fa640 1 --2- 172.21.15.52:0/759737660 >> [v2:172.21.15.52:6800/478031757,v1:172.21.15.52:6801/478031757] conn(0x7f6d440a1f90 0x7f6d440a4450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:41:10.777 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:10.777+0000 7f6d597fa640 1 -- 172.21.15.52:0/759737660 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 9635+0+0 (secure 0 0 0) 0x7f6d4c0ce970 con 0x7f6d5c100720 2024-03-04T09:41:10.777 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:10.777+0000 7f6d5b7fe640 1 --2- 172.21.15.52:0/759737660 >> [v2:172.21.15.52:6800/478031757,v1:172.21.15.52:6801/478031757] conn(0x7f6d440a1f90 0x7f6d440a4450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:41:10.778 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:10.777+0000 7f6d5b7fe640 1 --2- 172.21.15.52:0/759737660 >> [v2:172.21.15.52:6800/478031757,v1:172.21.15.52:6801/478031757] conn(0x7f6d440a1f90 0x7f6d440a4450 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f6d500095f0 tx=0x7f6d50005f90 comp rx=0 tx=0).ready entity=mgr.15486 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:41:10.778 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:10.781+0000 7f6d597fa640 1 -- 172.21.15.52:0/759737660 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f6d4c09af50 con 0x7f6d5c100720 2024-03-04T09:41:10.837 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:41:10 smithi191 ceph-mon[36816]: pgmap v309: 129 pgs: 129 active+clean; 78 MiB data, 687 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-03-04T09:41:10.837 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:41:10 smithi191 ceph-mon[36816]: osdmap e101: 12 total, 12 up, 12 in 2024-03-04T09:41:10.882 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:41:10 smithi106 ceph-mon[36484]: pgmap v309: 129 pgs: 129 active+clean; 78 MiB data, 687 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-03-04T09:41:10.882 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:41:10 smithi106 ceph-mon[36484]: osdmap e101: 12 total, 12 up, 12 in 2024-03-04T09:41:10.983 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:10.986+0000 7f6d625f6640 1 -- 172.21.15.52:0/759737660 --> [v2:172.21.15.52:6800/478031757,v1:172.21.15.52:6801/478031757] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) v1 -- 0x7f6d5c0fe840 con 0x7f6d440a1f90 2024-03-04T09:41:10.990 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:10.992+0000 7f6d597fa640 1 -- 172.21.15.52:0/759737660 <== mgr.15486 v2:172.21.15.52:6800/478031757 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+10 (secure 0 0 0) 0x7f6d5c0fe840 con 0x7f6d440a1f90 2024-03-04T09:41:10.990 INFO:teuthology.orchestra.run.smithi052.stdout:/ 1m 6m3h 2024-03-04T09:41:10.992 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:10.994+0000 7f6d625f6640 1 -- 172.21.15.52:0/759737660 >> [v2:172.21.15.52:6800/478031757,v1:172.21.15.52:6801/478031757] conn(0x7f6d440a1f90 msgr2=0x7f6d440a4450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:41:10.992 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:10.994+0000 7f6d625f6640 1 --2- 172.21.15.52:0/759737660 >> [v2:172.21.15.52:6800/478031757,v1:172.21.15.52:6801/478031757] conn(0x7f6d440a1f90 0x7f6d440a4450 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f6d500095f0 tx=0x7f6d50005f90 comp rx=0 tx=0).stop 2024-03-04T09:41:10.992 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:10.994+0000 7f6d625f6640 1 -- 172.21.15.52:0/759737660 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6d5c100720 msgr2=0x7f6d5c198c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:41:10.992 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:10.994+0000 7f6d625f6640 1 --2- 172.21.15.52:0/759737660 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6d5c100720 0x7f6d5c198c40 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f6d4c00e9f0 tx=0x7f6d4c00eeb0 comp rx=0 tx=0).stop 2024-03-04T09:41:10.992 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:10.994+0000 7f6d625f6640 1 -- 172.21.15.52:0/759737660 shutdown_connections 2024-03-04T09:41:10.993 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:10.994+0000 7f6d625f6640 1 --2- 172.21.15.52:0/759737660 >> [v2:172.21.15.52:6800/478031757,v1:172.21.15.52:6801/478031757] conn(0x7f6d440a1f90 0x7f6d440a4450 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:41:10.993 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:10.994+0000 7f6d625f6640 1 --2- 172.21.15.52:0/759737660 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6d5c105450 0x7f6d5c1a3cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:41:10.993 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:10.994+0000 7f6d625f6640 1 --2- 172.21.15.52:0/759737660 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6d5c1010c0 0x7f6d5c199180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:41:10.993 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:10.994+0000 7f6d625f6640 1 --2- 172.21.15.52:0/759737660 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6d5c100720 0x7f6d5c198c40 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:41:10.993 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:10.994+0000 7f6d625f6640 1 -- 172.21.15.52:0/759737660 >> 172.21.15.52:0/759737660 conn(0x7f6d5c0772a0 msgr2=0x7f6d5c0fe130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:41:10.993 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:10.994+0000 7f6d625f6640 1 -- 172.21.15.52:0/759737660 shutdown_connections 2024-03-04T09:41:10.993 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:10.994+0000 7f6d625f6640 1 -- 172.21.15.52:0/759737660 wait complete. 2024-03-04T09:41:11.003 DEBUG:teuthology.run_tasks:Unwinding manager check-counter 2024-03-04T09:41:11.015 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-04T09:41:11.120 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:11.120+0000 7f4f1305c640 1 -- 172.21.15.52:0/2193690409 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4f0c101f40 msgr2=0x7f4f0c10e400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:41:11.120 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:11.120+0000 7f4f1305c640 1 --2- 172.21.15.52:0/2193690409 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4f0c101f40 0x7f4f0c10e400 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f4f04009ca0 tx=0x7f4f0402c0f0 comp rx=0 tx=0).stop 2024-03-04T09:41:11.120 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:11.120+0000 7f4f1305c640 1 -- 172.21.15.52:0/2193690409 shutdown_connections 2024-03-04T09:41:11.120 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:11.120+0000 7f4f1305c640 1 --2- 172.21.15.52:0/2193690409 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4f0c101f40 0x7f4f0c10e400 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:41:11.120 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:11.120+0000 7f4f1305c640 1 --2- 172.21.15.52:0/2193690409 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4f0c101580 0x7f4f0c101a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:41:11.121 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:11.120+0000 7f4f1305c640 1 --2- 172.21.15.52:0/2193690409 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4f0c107640 0x7f4f0c107a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:41:11.121 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:11.120+0000 7f4f1305c640 1 -- 172.21.15.52:0/2193690409 >> 172.21.15.52:0/2193690409 conn(0x7f4f0c0fd070 msgr2=0x7f4f0c0ff490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:41:11.121 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:11.120+0000 7f4f1305c640 1 -- 172.21.15.52:0/2193690409 shutdown_connections 2024-03-04T09:41:11.121 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:11.120+0000 7f4f1305c640 1 -- 172.21.15.52:0/2193690409 wait complete. 2024-03-04T09:41:11.121 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:11.121+0000 7f4f1305c640 1 Processor -- start 2024-03-04T09:41:11.121 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:11.121+0000 7f4f1305c640 1 -- start start 2024-03-04T09:41:11.121 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:11.121+0000 7f4f1305c640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4f0c101580 0x7f4f0c197d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:41:11.121 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:11.121+0000 7f4f1305c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4f0c101f40 0x7f4f0c1982a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:41:11.122 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:11.121+0000 7f4f1305c640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4f0c107640 0x7f4f0c19c620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:41:11.122 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:11.121+0000 7f4f1305c640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f0c111920 con 0x7f4f0c107640 2024-03-04T09:41:11.122 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:11.121+0000 7f4f1305c640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f0c1117a0 con 0x7f4f0c101f40 2024-03-04T09:41:11.122 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:11.121+0000 7f4f1305c640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f0c111aa0 con 0x7f4f0c101580 2024-03-04T09:41:11.122 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:11.121+0000 7f4f115f9640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4f0c107640 0x7f4f0c19c620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:41:11.122 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:11.121+0000 7f4f115f9640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4f0c107640 0x7f4f0c19c620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:36906/0 (socket says 172.21.15.52:36906) 2024-03-04T09:41:11.122 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:11.121+0000 7f4f115f9640 1 -- 172.21.15.52:0/4185430684 learned_addr learned my addr 172.21.15.52:0/4185430684 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-04T09:41:11.122 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:11.121+0000 7f4f10df8640 1 --2- 172.21.15.52:0/4185430684 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4f0c101580 0x7f4f0c197d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:41:11.123 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:11.121+0000 7f4f0bfff640 1 --2- 172.21.15.52:0/4185430684 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4f0c101f40 0x7f4f0c1982a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:41:11.123 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:11.121+0000 7f4f115f9640 1 -- 172.21.15.52:0/4185430684 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4f0c101580 msgr2=0x7f4f0c197d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:41:11.123 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:11.121+0000 7f4f115f9640 1 --2- 172.21.15.52:0/4185430684 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4f0c101580 0x7f4f0c197d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:41:11.123 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:11.121+0000 7f4f115f9640 1 -- 172.21.15.52:0/4185430684 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4f0c101f40 msgr2=0x7f4f0c1982a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:41:11.123 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:11.121+0000 7f4f115f9640 1 --2- 172.21.15.52:0/4185430684 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4f0c101f40 0x7f4f0c1982a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:41:11.123 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:11.121+0000 7f4f115f9640 1 -- 172.21.15.52:0/4185430684 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f0c19cd50 con 0x7f4f0c107640 2024-03-04T09:41:11.123 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:11.121+0000 7f4f0bfff640 1 --2- 172.21.15.52:0/4185430684 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4f0c101f40 0x7f4f0c1982a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-04T09:41:11.123 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:11.121+0000 7f4f115f9640 1 --2- 172.21.15.52:0/4185430684 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4f0c107640 0x7f4f0c19c620 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f4f0402c700 tx=0x7f4f04032f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:41:11.123 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:11.122+0000 7f4f09ffb640 1 -- 172.21.15.52:0/4185430684 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f04042020 con 0x7f4f0c107640 2024-03-04T09:41:11.124 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:11.122+0000 7f4f09ffb640 1 -- 172.21.15.52:0/4185430684 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4f04034640 con 0x7f4f0c107640 2024-03-04T09:41:11.124 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:11.122+0000 7f4f1305c640 1 -- 172.21.15.52:0/4185430684 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f0c19cfe0 con 0x7f4f0c107640 2024-03-04T09:41:11.124 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:11.122+0000 7f4f09ffb640 1 -- 172.21.15.52:0/4185430684 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f04039040 con 0x7f4f0c107640 2024-03-04T09:41:11.124 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:11.122+0000 7f4f1305c640 1 -- 172.21.15.52:0/4185430684 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f0c1a48d0 con 0x7f4f0c107640 2024-03-04T09:41:11.125 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:11.123+0000 7f4f1305c640 1 -- 172.21.15.52:0/4185430684 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4f0c068810 con 0x7f4f0c107640 2024-03-04T09:41:11.125 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:11.124+0000 7f4f09ffb640 1 -- 172.21.15.52:0/4185430684 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 25) v1 ==== 136887+0+0 (secure 0 0 0) 0x7f4f04045020 con 0x7f4f0c107640 2024-03-04T09:41:11.125 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:11.125+0000 7f4f09ffb640 1 --2- 172.21.15.52:0/4185430684 >> [v2:172.21.15.52:6800/478031757,v1:172.21.15.52:6801/478031757] conn(0x7f4efc0a1ef0 0x7f4efc0a43b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:41:11.125 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:11.125+0000 7f4f10df8640 1 --2- 172.21.15.52:0/4185430684 >> [v2:172.21.15.52:6800/478031757,v1:172.21.15.52:6801/478031757] conn(0x7f4efc0a1ef0 0x7f4efc0a43b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:41:11.126 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:11.125+0000 7f4f09ffb640 1 -- 172.21.15.52:0/4185430684 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 9635+0+0 (secure 0 0 0) 0x7f4f040f0840 con 0x7f4f0c107640 2024-03-04T09:41:11.126 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:11.125+0000 7f4f10df8640 1 --2- 172.21.15.52:0/4185430684 >> [v2:172.21.15.52:6800/478031757,v1:172.21.15.52:6801/478031757] conn(0x7f4efc0a1ef0 0x7f4efc0a43b0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f4f000095f0 tx=0x7f4f00005f90 comp rx=0 tx=0).ready entity=mgr.15486 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:41:11.126 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:11.128+0000 7f4f09ffb640 1 -- 172.21.15.52:0/4185430684 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f4f0404c020 con 0x7f4f0c107640 2024-03-04T09:41:11.311 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:11.314+0000 7f4f1305c640 1 -- 172.21.15.52:0/4185430684 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4f0c109990 con 0x7f4f0c107640 2024-03-04T09:41:11.313 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:11.315+0000 7f4f09ffb640 1 -- 172.21.15.52:0/4185430684 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v101) v1 ==== 74+0+25562 (secure 0 0 0) 0x7f4f040bd600 con 0x7f4f0c107640 2024-03-04T09:41:11.314 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-03-04T09:41:11.314 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":101,"fsid":"4e059f08-da08-11ee-95c6-87774f69a715","created":"2024-03-04T09:20:46.104963+0000","modified":"2024-03-04T09:41:09.457752+0000","last_up_change":"2024-03-04T09:27:49.261365+0000","last_in_change":"2024-03-04T09:27:30.024964+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-04T09:24:01.742461+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-04T09:28:35.835600+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-04T09:28:36.453099+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"101","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":15,"snap_epoch":101,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-04T09:28:40.168831+0000","flags":8197,"flags_names":"hashpspool,ec_overwrites,selfmanaged_snaps","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"101","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":15,"snap_epoch":101,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"583172be-602f-4bf4-bfcb-42663cf5b277","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6802","nonce":1371221397},{"type":"v1","addr":"172.21.15.52:6803","nonce":1371221397}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6804","nonce":1371221397},{"type":"v1","addr":"172.21.15.52:6805","nonce":1371221397}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6808","nonce":1371221397},{"type":"v1","addr":"172.21.15.52:6809","nonce":1371221397}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6806","nonce":1371221397},{"type":"v1","addr":"172.21.15.52:6807","nonce":1371221397}]},"public_addr":"172.21.15.52:6803/1371221397","cluster_addr":"172.21.15.52:6805/1371221397","heartbeat_back_addr":"172.21.15.52:6809/1371221397","heartbeat_front_addr":"172.21.15.52:6807/1371221397","state":["exists","up"]},{"osd":1,"uuid":"5ae0081c-a2f6-46eb-ace0-0368208377b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":2540865537},{"type":"v1","addr":"172.21.15.106:6801","nonce":2540865537}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":2540865537},{"type":"v1","addr":"172.21.15.106:6803","nonce":2540865537}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":2540865537},{"type":"v1","addr":"172.21.15.106:6807","nonce":2540865537}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":2540865537},{"type":"v1","addr":"172.21.15.106:6805","nonce":2540865537}]},"public_addr":"172.21.15.106:6801/2540865537","cluster_addr":"172.21.15.106:6803/2540865537","heartbeat_back_addr":"172.21.15.106:6807/2540865537","heartbeat_front_addr":"172.21.15.106:6805/2540865537","state":["exists","up"]},{"osd":2,"uuid":"1948bac7-2bb2-4218-8b93-4e4743a0de04","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":1377083164},{"type":"v1","addr":"172.21.15.191:6801","nonce":1377083164}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":1377083164},{"type":"v1","addr":"172.21.15.191:6803","nonce":1377083164}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":1377083164},{"type":"v1","addr":"172.21.15.191:6807","nonce":1377083164}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":1377083164},{"type":"v1","addr":"172.21.15.191:6805","nonce":1377083164}]},"public_addr":"172.21.15.191:6801/1377083164","cluster_addr":"172.21.15.191:6803/1377083164","heartbeat_back_addr":"172.21.15.191:6807/1377083164","heartbeat_front_addr":"172.21.15.191:6805/1377083164","state":["exists","up"]},{"osd":3,"uuid":"16c9e9fe-7167-4992-bf12-1a7398e0498a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6810","nonce":2388571810},{"type":"v1","addr":"172.21.15.52:6811","nonce":2388571810}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6812","nonce":2388571810},{"type":"v1","addr":"172.21.15.52:6813","nonce":2388571810}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6816","nonce":2388571810},{"type":"v1","addr":"172.21.15.52:6817","nonce":2388571810}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6814","nonce":2388571810},{"type":"v1","addr":"172.21.15.52:6815","nonce":2388571810}]},"public_addr":"172.21.15.52:6811/2388571810","cluster_addr":"172.21.15.52:6813/2388571810","heartbeat_back_addr":"172.21.15.52:6817/2388571810","heartbeat_front_addr":"172.21.15.52:6815/2388571810","state":["exists","up"]},{"osd":4,"uuid":"c8ef48c7-7c78-44d3-a9e5-99ec750abd3d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6808","nonce":3465868015},{"type":"v1","addr":"172.21.15.106:6809","nonce":3465868015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6810","nonce":3465868015},{"type":"v1","addr":"172.21.15.106:6811","nonce":3465868015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6814","nonce":3465868015},{"type":"v1","addr":"172.21.15.106:6815","nonce":3465868015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6812","nonce":3465868015},{"type":"v1","addr":"172.21.15.106:6813","nonce":3465868015}]},"public_addr":"172.21.15.106:6809/3465868015","cluster_addr":"172.21.15.106:6811/3465868015","heartbeat_back_addr":"172.21.15.106:6815/3465868015","heartbeat_front_addr":"172.21.15.106:6813/3465868015","state":["exists","up"]},{"osd":5,"uuid":"9e41824a-c5a2-4f88-be50-2ba1dceb3220","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":1485092643},{"type":"v1","addr":"172.21.15.191:6809","nonce":1485092643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":1485092643},{"type":"v1","addr":"172.21.15.191:6811","nonce":1485092643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":1485092643},{"type":"v1","addr":"172.21.15.191:6815","nonce":1485092643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":1485092643},{"type":"v1","addr":"172.21.15.191:6813","nonce":1485092643}]},"public_addr":"172.21.15.191:6809/1485092643","cluster_addr":"172.21.15.191:6811/1485092643","heartbeat_back_addr":"172.21.15.191:6815/1485092643","heartbeat_front_addr":"172.21.15.191:6813/1485092643","state":["exists","up"]},{"osd":6,"uuid":"b35dccee-ec16-452e-ac69-3ac2fb7b27fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6818","nonce":1989797856},{"type":"v1","addr":"172.21.15.52:6819","nonce":1989797856}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6820","nonce":1989797856},{"type":"v1","addr":"172.21.15.52:6821","nonce":1989797856}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6824","nonce":1989797856},{"type":"v1","addr":"172.21.15.52:6825","nonce":1989797856}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6822","nonce":1989797856},{"type":"v1","addr":"172.21.15.52:6823","nonce":1989797856}]},"public_addr":"172.21.15.52:6819/1989797856","cluster_addr":"172.21.15.52:6821/1989797856","heartbeat_back_addr":"172.21.15.52:6825/1989797856","heartbeat_front_addr":"172.21.15.52:6823/1989797856","state":["exists","up"]},{"osd":7,"uuid":"71303e3d-14ec-4cbd-b19c-67645a1de3c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6816","nonce":1450870329},{"type":"v1","addr":"172.21.15.106:6817","nonce":1450870329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":1450870329},{"type":"v1","addr":"172.21.15.106:6819","nonce":1450870329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":1450870329},{"type":"v1","addr":"172.21.15.106:6823","nonce":1450870329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6820","nonce":1450870329},{"type":"v1","addr":"172.21.15.106:6821","nonce":1450870329}]},"public_addr":"172.21.15.106:6817/1450870329","cluster_addr":"172.21.15.106:6819/1450870329","heartbeat_back_addr":"172.21.15.106:6823/1450870329","heartbeat_front_addr":"172.21.15.106:6821/1450870329","state":["exists","up"]},{"osd":8,"uuid":"6f5786c4-4402-435a-bdad-adff46fe9473","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":1114119072},{"type":"v1","addr":"172.21.15.191:6817","nonce":1114119072}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6818","nonce":1114119072},{"type":"v1","addr":"172.21.15.191:6819","nonce":1114119072}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":1114119072},{"type":"v1","addr":"172.21.15.191:6823","nonce":1114119072}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":1114119072},{"type":"v1","addr":"172.21.15.191:6821","nonce":1114119072}]},"public_addr":"172.21.15.191:6817/1114119072","cluster_addr":"172.21.15.191:6819/1114119072","heartbeat_back_addr":"172.21.15.191:6823/1114119072","heartbeat_front_addr":"172.21.15.191:6821/1114119072","state":["exists","up"]},{"osd":9,"uuid":"062bbaa5-1227-4684-b859-450f5594a90c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6826","nonce":3485821914},{"type":"v1","addr":"172.21.15.52:6827","nonce":3485821914}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6828","nonce":3485821914},{"type":"v1","addr":"172.21.15.52:6829","nonce":3485821914}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6832","nonce":3485821914},{"type":"v1","addr":"172.21.15.52:6833","nonce":3485821914}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6830","nonce":3485821914},{"type":"v1","addr":"172.21.15.52:6831","nonce":3485821914}]},"public_addr":"172.21.15.52:6827/3485821914","cluster_addr":"172.21.15.52:6829/3485821914","heartbeat_back_addr":"172.21.15.52:6833/3485821914","heartbeat_front_addr":"172.21.15.52:6831/3485821914","state":["exists","up"]},{"osd":10,"uuid":"1a285a5a-f793-48da-a8eb-5326d9528cd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":3815586576},{"type":"v1","addr":"172.21.15.106:6825","nonce":3815586576}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6826","nonce":3815586576},{"type":"v1","addr":"172.21.15.106:6827","nonce":3815586576}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":3815586576},{"type":"v1","addr":"172.21.15.106:6831","nonce":3815586576}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":3815586576},{"type":"v1","addr":"172.21.15.106:6829","nonce":3815586576}]},"public_addr":"172.21.15.106:6825/3815586576","cluster_addr":"172.21.15.106:6827/3815586576","heartbeat_back_addr":"172.21.15.106:6831/3815586576","heartbeat_front_addr":"172.21.15.106:6829/3815586576","state":["exists","up"]},{"osd":11,"uuid":"1f8b9a43-57fa-41f4-9969-3b803a74bf45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":2009632843},{"type":"v1","addr":"172.21.15.191:6825","nonce":2009632843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":2009632843},{"type":"v1","addr":"172.21.15.191:6827","nonce":2009632843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":2009632843},{"type":"v1","addr":"172.21.15.191:6831","nonce":2009632843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":2009632843},{"type":"v1","addr":"172.21.15.191:6829","nonce":2009632843}]},"public_addr":"172.21.15.191:6825/2009632843","cluster_addr":"172.21.15.191:6827/2009632843","heartbeat_back_addr":"172.21.15.191:6831/2009632843","heartbeat_front_addr":"172.21.15.191:6829/2009632843","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T09:23:10.894763+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T09:23:34.420055+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T09:23:58.288324+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T09:24:23.673457+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T09:24:48.208561+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T09:25:13.405473+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T09:25:38.670915+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T09:26:04.451257+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T09:26:30.281380+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T09:26:56.543143+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T09:27:22.412930+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T09:27:48.003866+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.1d","mappings":[{"from":2,"to":1},{"from":8,"to":11}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.20","mappings":[{"from":5,"to":9}]},{"pgid":"3.21","mappings":[{"from":4,"to":6}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]},{"pgid":"4.9","mappings":[{"from":5,"to":8}]},{"pgid":"4.17","mappings":[{"from":9,"to":8}]},{"pgid":"4.1b","mappings":[{"from":9,"to":10}]},{"pgid":"4.1f","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.52:0/1035878034":"2024-03-05T09:21:12.760938+0000","172.21.15.52:0/2743084243":"2024-03-05T09:21:39.401935+0000","172.21.15.52:0/1094295190":"2024-03-05T09:31:08.422018+0000","172.21.15.52:0/568242577":"2024-03-05T09:21:12.760938+0000","172.21.15.52:0/2080168931":"2024-03-05T09:31:08.422018+0000","172.21.15.52:6801/2425730636":"2024-03-05T09:21:39.401935+0000","172.21.15.52:6800/2379361333":"2024-03-05T09:21:12.760938+0000","172.21.15.52:6800/3036033923":"2024-03-05T09:31:08.422018+0000","172.21.15.52:0/497075075":"2024-03-05T09:21:12.760938+0000","172.21.15.52:6800/2425730636":"2024-03-05T09:21:39.401935+0000","172.21.15.52:6801/2379361333":"2024-03-05T09:21:12.760938+0000","172.21.15.52:0/1271949988":"2024-03-05T09:21:39.401935+0000","172.21.15.52:0/2491451071":"2024-03-05T09:31:08.422018+0000","172.21.15.52:0/2466768":"2024-03-05T09:31:08.422018+0000","172.21.15.52:0/4082645718":"2024-03-05T09:21:39.401935+0000","172.21.15.52:0/2935914357":"2024-03-05T09:31:08.422018+0000","172.21.15.52:6801/3036033923":"2024-03-05T09:31:08.422018+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":3,"snaps":[{"begin":5,"length":1},{"begin":15,"length":1}]},{"pool":4,"snaps":[{"begin":5,"length":1},{"begin":15,"length":1}]}],"new_removed_snaps":[{"pool":3,"snaps":[{"begin":5,"length":1},{"begin":15,"length":1}]},{"pool":4,"snaps":[{"begin":5,"length":1},{"begin":15,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-04T09:41:11.315 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:11.317+0000 7f4f1305c640 1 -- 172.21.15.52:0/4185430684 >> [v2:172.21.15.52:6800/478031757,v1:172.21.15.52:6801/478031757] conn(0x7f4efc0a1ef0 msgr2=0x7f4efc0a43b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:41:11.315 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:11.317+0000 7f4f1305c640 1 --2- 172.21.15.52:0/4185430684 >> [v2:172.21.15.52:6800/478031757,v1:172.21.15.52:6801/478031757] conn(0x7f4efc0a1ef0 0x7f4efc0a43b0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f4f000095f0 tx=0x7f4f00005f90 comp rx=0 tx=0).stop 2024-03-04T09:41:11.315 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:11.317+0000 7f4f1305c640 1 -- 172.21.15.52:0/4185430684 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4f0c107640 msgr2=0x7f4f0c19c620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:41:11.315 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:11.317+0000 7f4f1305c640 1 --2- 172.21.15.52:0/4185430684 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4f0c107640 0x7f4f0c19c620 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f4f0402c700 tx=0x7f4f04032f20 comp rx=0 tx=0).stop 2024-03-04T09:41:11.316 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:11.317+0000 7f4f1305c640 1 -- 172.21.15.52:0/4185430684 shutdown_connections 2024-03-04T09:41:11.316 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:11.317+0000 7f4f1305c640 1 --2- 172.21.15.52:0/4185430684 >> [v2:172.21.15.52:6800/478031757,v1:172.21.15.52:6801/478031757] conn(0x7f4efc0a1ef0 0x7f4efc0a43b0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:41:11.316 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:11.317+0000 7f4f1305c640 1 --2- 172.21.15.52:0/4185430684 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4f0c107640 0x7f4f0c19c620 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:41:11.316 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:11.317+0000 7f4f1305c640 1 --2- 172.21.15.52:0/4185430684 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4f0c101f40 0x7f4f0c1982a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:41:11.316 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:11.317+0000 7f4f1305c640 1 --2- 172.21.15.52:0/4185430684 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4f0c101580 0x7f4f0c197d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:41:11.316 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:11.317+0000 7f4f1305c640 1 -- 172.21.15.52:0/4185430684 >> 172.21.15.52:0/4185430684 conn(0x7f4f0c0fd070 msgr2=0x7f4f0c0fe830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:41:11.316 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:11.317+0000 7f4f1305c640 1 -- 172.21.15.52:0/4185430684 shutdown_connections 2024-03-04T09:41:11.317 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:11.317+0000 7f4f1305c640 1 -- 172.21.15.52:0/4185430684 wait complete. 2024-03-04T09:41:11.326 INFO:tasks.check_counter.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-04T09:24:01.742461+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-04T09:28:35.835600+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-04T09:28:36.453099+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '101', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 15, 'snap_epoch': 101, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-04T09:28:40.168831+0000', 'flags': 8197, 'flags_names': 'hashpspool,ec_overwrites,selfmanaged_snaps', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '101', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 15, 'snap_epoch': 101, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-04T09:41:11.327 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-04T09:41:11.434 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:11.434+0000 7f185077a640 1 -- 172.21.15.52:0/2668141589 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f18480687d0 msgr2=0x7f1848068c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:41:11.434 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:11.434+0000 7f185077a640 1 --2- 172.21.15.52:0/2668141589 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f18480687d0 0x7f1848068c50 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f183c009930 tx=0x7f183c02bc30 comp rx=0 tx=0).stop 2024-03-04T09:41:11.435 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:11.435+0000 7f185077a640 1 -- 172.21.15.52:0/2668141589 shutdown_connections 2024-03-04T09:41:11.435 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:11.435+0000 7f185077a640 1 --2- 172.21.15.52:0/2668141589 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1848069190 0x7f184810e400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:41:11.435 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:11.435+0000 7f185077a640 1 --2- 172.21.15.52:0/2668141589 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f18480687d0 0x7f1848068c50 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:41:11.435 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:11.435+0000 7f185077a640 1 --2- 172.21.15.52:0/2668141589 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f18481075d0 0x7f18481079b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:41:11.435 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:11.435+0000 7f185077a640 1 -- 172.21.15.52:0/2668141589 >> 172.21.15.52:0/2668141589 conn(0x7f18480775d0 msgr2=0x7f18480779e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:41:11.435 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:11.435+0000 7f185077a640 1 -- 172.21.15.52:0/2668141589 shutdown_connections 2024-03-04T09:41:11.435 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:11.435+0000 7f185077a640 1 -- 172.21.15.52:0/2668141589 wait complete. 2024-03-04T09:41:11.435 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:11.435+0000 7f185077a640 1 Processor -- start 2024-03-04T09:41:11.436 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:11.435+0000 7f185077a640 1 -- start start 2024-03-04T09:41:11.436 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:11.435+0000 7f185077a640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f18480687d0 0x7f1848100490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:41:11.436 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:11.435+0000 7f185077a640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1848069190 0x7f18481009d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:41:11.436 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:11.435+0000 7f185077a640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f18481075d0 0x7f1848100f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:41:11.436 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:11.435+0000 7f185077a640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1848111970 con 0x7f1848069190 2024-03-04T09:41:11.436 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:11.435+0000 7f185077a640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18481117f0 con 0x7f18481075d0 2024-03-04T09:41:11.436 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:11.435+0000 7f185077a640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1848111af0 con 0x7f18480687d0 2024-03-04T09:41:11.437 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:11.435+0000 7f184dd15640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1848069190 0x7f18481009d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:41:11.437 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:11.435+0000 7f184dd15640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1848069190 0x7f18481009d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:36930/0 (socket says 172.21.15.52:36930) 2024-03-04T09:41:11.437 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:11.435+0000 7f184dd15640 1 -- 172.21.15.52:0/3236477509 learned_addr learned my addr 172.21.15.52:0/3236477509 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-04T09:41:11.437 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:11.436+0000 7f184e516640 1 --2- 172.21.15.52:0/3236477509 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f18480687d0 0x7f1848100490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:41:11.437 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:11.436+0000 7f184ed17640 1 --2- 172.21.15.52:0/3236477509 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f18481075d0 0x7f1848100f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:41:11.437 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:11.436+0000 7f184dd15640 1 -- 172.21.15.52:0/3236477509 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f18480687d0 msgr2=0x7f1848100490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:41:11.438 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:11.436+0000 7f184dd15640 1 --2- 172.21.15.52:0/3236477509 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f18480687d0 0x7f1848100490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:41:11.438 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:11.436+0000 7f184dd15640 1 -- 172.21.15.52:0/3236477509 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f18481075d0 msgr2=0x7f1848100f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:41:11.438 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:11.436+0000 7f184dd15640 1 --2- 172.21.15.52:0/3236477509 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f18481075d0 0x7f1848100f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:41:11.438 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:11.436+0000 7f184dd15640 1 -- 172.21.15.52:0/3236477509 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18481a6310 con 0x7f1848069190 2024-03-04T09:41:11.438 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:11.436+0000 7f184dd15640 1 --2- 172.21.15.52:0/3236477509 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1848069190 0x7f18481009d0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f183c009900 tx=0x7f183c004360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:41:11.438 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:11.436+0000 7f18477fe640 1 -- 172.21.15.52:0/3236477509 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f183c043020 con 0x7f1848069190 2024-03-04T09:41:11.438 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:11.436+0000 7f18477fe640 1 -- 172.21.15.52:0/3236477509 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f183c0054e0 con 0x7f1848069190 2024-03-04T09:41:11.438 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:11.436+0000 7f18477fe640 1 -- 172.21.15.52:0/3236477509 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f183c003bb0 con 0x7f1848069190 2024-03-04T09:41:11.439 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:11.436+0000 7f185077a640 1 -- 172.21.15.52:0/3236477509 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18481a65a0 con 0x7f1848069190 2024-03-04T09:41:11.439 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:11.436+0000 7f185077a640 1 -- 172.21.15.52:0/3236477509 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18481a6b40 con 0x7f1848069190 2024-03-04T09:41:11.440 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:11.437+0000 7f185077a640 1 -- 172.21.15.52:0/3236477509 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f18480683c0 con 0x7f1848069190 2024-03-04T09:41:11.440 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:11.438+0000 7f18477fe640 1 -- 172.21.15.52:0/3236477509 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 25) v1 ==== 136887+0+0 (secure 0 0 0) 0x7f183c046020 con 0x7f1848069190 2024-03-04T09:41:11.440 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:11.439+0000 7f18477fe640 1 --2- 172.21.15.52:0/3236477509 >> [v2:172.21.15.52:6800/478031757,v1:172.21.15.52:6801/478031757] conn(0x7f18300a1fa0 0x7f18300a4460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:41:11.441 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:11.439+0000 7f184e516640 1 --2- 172.21.15.52:0/3236477509 >> [v2:172.21.15.52:6800/478031757,v1:172.21.15.52:6801/478031757] conn(0x7f18300a1fa0 0x7f18300a4460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:41:11.441 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:11.439+0000 7f18477fe640 1 -- 172.21.15.52:0/3236477509 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 9635+0+0 (secure 0 0 0) 0x7f183c0f07e0 con 0x7f1848069190 2024-03-04T09:41:11.441 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:11.439+0000 7f184e516640 1 --2- 172.21.15.52:0/3236477509 >> [v2:172.21.15.52:6800/478031757,v1:172.21.15.52:6801/478031757] conn(0x7f18300a1fa0 0x7f18300a4460 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f18400095f0 tx=0x7f1840005f90 comp rx=0 tx=0).ready entity=mgr.15486 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:41:11.441 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:11.442+0000 7f18477fe640 1 -- 172.21.15.52:0/3236477509 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f183c04d020 con 0x7f1848069190 2024-03-04T09:41:11.585 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:41:11 smithi052 ceph-mon[32473]: from='client.? 172.21.15.52:0/4185430684' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-04T09:41:11.622 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:11.624+0000 7f185077a640 1 -- 172.21.15.52:0/3236477509 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f184802cff0 con 0x7f1848069190 2024-03-04T09:41:11.622 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:11.624+0000 7f18477fe640 1 -- 172.21.15.52:0/3236477509 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v101) v1 ==== 93+0+10 (secure 0 0 0) 0x7f183c0bd580 con 0x7f1848069190 2024-03-04T09:41:11.622 INFO:teuthology.orchestra.run.smithi052.stdout:pg_num: 1 2024-03-04T09:41:11.624 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:11.626+0000 7f185077a640 1 -- 172.21.15.52:0/3236477509 >> [v2:172.21.15.52:6800/478031757,v1:172.21.15.52:6801/478031757] conn(0x7f18300a1fa0 msgr2=0x7f18300a4460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:41:11.624 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:11.626+0000 7f185077a640 1 --2- 172.21.15.52:0/3236477509 >> [v2:172.21.15.52:6800/478031757,v1:172.21.15.52:6801/478031757] conn(0x7f18300a1fa0 0x7f18300a4460 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f18400095f0 tx=0x7f1840005f90 comp rx=0 tx=0).stop 2024-03-04T09:41:11.625 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:11.626+0000 7f185077a640 1 -- 172.21.15.52:0/3236477509 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1848069190 msgr2=0x7f18481009d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:41:11.625 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:11.626+0000 7f185077a640 1 --2- 172.21.15.52:0/3236477509 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1848069190 0x7f18481009d0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f183c009900 tx=0x7f183c004360 comp rx=0 tx=0).stop 2024-03-04T09:41:11.625 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:11.626+0000 7f185077a640 1 -- 172.21.15.52:0/3236477509 shutdown_connections 2024-03-04T09:41:11.625 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:11.626+0000 7f185077a640 1 --2- 172.21.15.52:0/3236477509 >> [v2:172.21.15.52:6800/478031757,v1:172.21.15.52:6801/478031757] conn(0x7f18300a1fa0 0x7f18300a4460 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:41:11.625 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:11.626+0000 7f185077a640 1 --2- 172.21.15.52:0/3236477509 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f18481075d0 0x7f1848100f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:41:11.625 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:11.626+0000 7f185077a640 1 --2- 172.21.15.52:0/3236477509 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1848069190 0x7f18481009d0 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:41:11.625 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:11.626+0000 7f185077a640 1 --2- 172.21.15.52:0/3236477509 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f18480687d0 0x7f1848100490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:41:11.625 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:11.626+0000 7f185077a640 1 -- 172.21.15.52:0/3236477509 >> 172.21.15.52:0/3236477509 conn(0x7f18480775d0 msgr2=0x7f18480ff0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:41:11.625 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:11.626+0000 7f185077a640 1 -- 172.21.15.52:0/3236477509 shutdown_connections 2024-03-04T09:41:11.626 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:11.626+0000 7f185077a640 1 -- 172.21.15.52:0/3236477509 wait complete. 2024-03-04T09:41:11.635 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-04T09:41:11.743 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:11.743+0000 7fb5853bd640 1 -- 172.21.15.52:0/2707747337 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb580100e60 msgr2=0x7fb58010d320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:41:11.743 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:11.743+0000 7fb5853bd640 1 --2- 172.21.15.52:0/2707747337 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb580100e60 0x7fb58010d320 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fb578009ca0 tx=0x7fb57802c0f0 comp rx=0 tx=0).stop 2024-03-04T09:41:11.743 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:11.743+0000 7fb5853bd640 1 -- 172.21.15.52:0/2707747337 shutdown_connections 2024-03-04T09:41:11.743 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:11.743+0000 7fb5853bd640 1 --2- 172.21.15.52:0/2707747337 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb580100e60 0x7fb58010d320 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:41:11.744 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:11.743+0000 7fb5853bd640 1 --2- 172.21.15.52:0/2707747337 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb5801004a0 0x7fb580100920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:41:11.744 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:11.743+0000 7fb5853bd640 1 --2- 172.21.15.52:0/2707747337 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb580106560 0x7fb580106940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:41:11.744 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:11.743+0000 7fb5853bd640 1 -- 172.21.15.52:0/2707747337 >> 172.21.15.52:0/2707747337 conn(0x7fb580076850 msgr2=0x7fb580076c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:41:11.744 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:11.743+0000 7fb5853bd640 1 -- 172.21.15.52:0/2707747337 shutdown_connections 2024-03-04T09:41:11.744 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:11.743+0000 7fb5853bd640 1 -- 172.21.15.52:0/2707747337 wait complete. 2024-03-04T09:41:11.744 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:11.743+0000 7fb5853bd640 1 Processor -- start 2024-03-04T09:41:11.744 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:11.743+0000 7fb5853bd640 1 -- start start 2024-03-04T09:41:11.744 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:11.744+0000 7fb5853bd640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb5801004a0 0x7fb580197d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:41:11.744 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:11.744+0000 7fb5853bd640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb580100e60 0x7fb5801982a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:41:11.745 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:11.744+0000 7fb5853bd640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb580106560 0x7fb58019c620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:41:11.745 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:11.744+0000 7fb5853bd640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5801119c0 con 0x7fb5801004a0 2024-03-04T09:41:11.745 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:11.744+0000 7fb5853bd640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb580111840 con 0x7fb580100e60 2024-03-04T09:41:11.745 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:11.744+0000 7fb5853bd640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb580111b40 con 0x7fb580106560 2024-03-04T09:41:11.745 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:11.744+0000 7fb57effd640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb5801004a0 0x7fb580197d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:41:11.745 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:11.744+0000 7fb57effd640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb5801004a0 0x7fb580197d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:36950/0 (socket says 172.21.15.52:36950) 2024-03-04T09:41:11.745 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:11.744+0000 7fb57effd640 1 -- 172.21.15.52:0/4095129042 learned_addr learned my addr 172.21.15.52:0/4095129042 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-04T09:41:11.745 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:11.744+0000 7fb57e7fc640 1 --2- 172.21.15.52:0/4095129042 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb580100e60 0x7fb5801982a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:41:11.746 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:11.744+0000 7fb57f7fe640 1 --2- 172.21.15.52:0/4095129042 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb580106560 0x7fb58019c620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:41:11.746 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:11.744+0000 7fb57effd640 1 -- 172.21.15.52:0/4095129042 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb580106560 msgr2=0x7fb58019c620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:41:11.746 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:11.744+0000 7fb57effd640 1 --2- 172.21.15.52:0/4095129042 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb580106560 0x7fb58019c620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:41:11.746 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:11.744+0000 7fb57effd640 1 -- 172.21.15.52:0/4095129042 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb580100e60 msgr2=0x7fb5801982a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:41:11.746 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:11.744+0000 7fb57effd640 1 --2- 172.21.15.52:0/4095129042 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb580100e60 0x7fb5801982a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:41:11.746 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:11.744+0000 7fb57effd640 1 -- 172.21.15.52:0/4095129042 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb58019cd50 con 0x7fb5801004a0 2024-03-04T09:41:11.746 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:11.744+0000 7fb57effd640 1 --2- 172.21.15.52:0/4095129042 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb5801004a0 0x7fb580197d60 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fb57000e9f0 tx=0x7fb57000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:41:11.747 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:11.744+0000 7fb567fff640 1 -- 172.21.15.52:0/4095129042 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb57000cd80 con 0x7fb5801004a0 2024-03-04T09:41:11.747 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:11.744+0000 7fb567fff640 1 -- 172.21.15.52:0/4095129042 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb570010910 con 0x7fb5801004a0 2024-03-04T09:41:11.747 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:11.744+0000 7fb5853bd640 1 -- 172.21.15.52:0/4095129042 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb58019d040 con 0x7fb5801004a0 2024-03-04T09:41:11.747 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:11.744+0000 7fb567fff640 1 -- 172.21.15.52:0/4095129042 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb570018990 con 0x7fb5801004a0 2024-03-04T09:41:11.747 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:11.744+0000 7fb5853bd640 1 -- 172.21.15.52:0/4095129042 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5801a48d0 con 0x7fb5801004a0 2024-03-04T09:41:11.747 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:11.745+0000 7fb5853bd640 1 -- 172.21.15.52:0/4095129042 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb580101bc0 con 0x7fb5801004a0 2024-03-04T09:41:11.748 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:11.746+0000 7fb567fff640 1 -- 172.21.15.52:0/4095129042 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 25) v1 ==== 136887+0+0 (secure 0 0 0) 0x7fb570010ac0 con 0x7fb5801004a0 2024-03-04T09:41:11.748 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:11.747+0000 7fb567fff640 1 --2- 172.21.15.52:0/4095129042 >> [v2:172.21.15.52:6800/478031757,v1:172.21.15.52:6801/478031757] conn(0x7fb5680a1ef0 0x7fb5680a43b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:41:11.748 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:11.747+0000 7fb57e7fc640 1 --2- 172.21.15.52:0/4095129042 >> [v2:172.21.15.52:6800/478031757,v1:172.21.15.52:6801/478031757] conn(0x7fb5680a1ef0 0x7fb5680a43b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:41:11.749 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:11.747+0000 7fb567fff640 1 -- 172.21.15.52:0/4095129042 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 9635+0+0 (secure 0 0 0) 0x7fb5700ce8f0 con 0x7fb5801004a0 2024-03-04T09:41:11.749 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:11.748+0000 7fb57e7fc640 1 --2- 172.21.15.52:0/4095129042 >> [v2:172.21.15.52:6800/478031757,v1:172.21.15.52:6801/478031757] conn(0x7fb5680a1ef0 0x7fb5680a43b0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fb580105b10 tx=0x7fb574005f90 comp rx=0 tx=0).ready entity=mgr.15486 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:41:11.749 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:11.752+0000 7fb567fff640 1 -- 172.21.15.52:0/4095129042 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fb57009af30 con 0x7fb5801004a0 2024-03-04T09:41:11.837 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:41:11 smithi191 ceph-mon[36816]: from='client.? 172.21.15.52:0/4185430684' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-04T09:41:11.882 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:41:11 smithi106 ceph-mon[36484]: from='client.? 172.21.15.52:0/4185430684' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-04T09:41:11.931 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:11.934+0000 7fb5853bd640 1 -- 172.21.15.52:0/4095129042 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fb5801083e0 con 0x7fb5801004a0 2024-03-04T09:41:11.932 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:11.934+0000 7fb567fff640 1 -- 172.21.15.52:0/4095129042 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v101) v1 ==== 104+0+11 (secure 0 0 0) 0x7fb57009efd0 con 0x7fb5801004a0 2024-03-04T09:41:11.932 INFO:teuthology.orchestra.run.smithi052.stdout:pg_num: 32 2024-03-04T09:41:11.934 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:11.936+0000 7fb5853bd640 1 -- 172.21.15.52:0/4095129042 >> [v2:172.21.15.52:6800/478031757,v1:172.21.15.52:6801/478031757] conn(0x7fb5680a1ef0 msgr2=0x7fb5680a43b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:41:11.934 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:11.936+0000 7fb5853bd640 1 --2- 172.21.15.52:0/4095129042 >> [v2:172.21.15.52:6800/478031757,v1:172.21.15.52:6801/478031757] conn(0x7fb5680a1ef0 0x7fb5680a43b0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fb580105b10 tx=0x7fb574005f90 comp rx=0 tx=0).stop 2024-03-04T09:41:11.934 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:11.936+0000 7fb5853bd640 1 -- 172.21.15.52:0/4095129042 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb5801004a0 msgr2=0x7fb580197d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:41:11.934 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:11.936+0000 7fb5853bd640 1 --2- 172.21.15.52:0/4095129042 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb5801004a0 0x7fb580197d60 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fb57000e9f0 tx=0x7fb57000eeb0 comp rx=0 tx=0).stop 2024-03-04T09:41:11.935 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:11.936+0000 7fb5853bd640 1 -- 172.21.15.52:0/4095129042 shutdown_connections 2024-03-04T09:41:11.935 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:11.936+0000 7fb5853bd640 1 --2- 172.21.15.52:0/4095129042 >> [v2:172.21.15.52:6800/478031757,v1:172.21.15.52:6801/478031757] conn(0x7fb5680a1ef0 0x7fb5680a43b0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:41:11.935 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:11.936+0000 7fb5853bd640 1 --2- 172.21.15.52:0/4095129042 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb580106560 0x7fb58019c620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:41:11.935 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:11.936+0000 7fb5853bd640 1 --2- 172.21.15.52:0/4095129042 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb580100e60 0x7fb5801982a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:41:11.935 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:11.936+0000 7fb5853bd640 1 --2- 172.21.15.52:0/4095129042 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb5801004a0 0x7fb580197d60 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:41:11.935 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:11.936+0000 7fb5853bd640 1 -- 172.21.15.52:0/4095129042 >> 172.21.15.52:0/4095129042 conn(0x7fb580076850 msgr2=0x7fb5800fec80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:41:11.935 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:11.936+0000 7fb5853bd640 1 -- 172.21.15.52:0/4095129042 shutdown_connections 2024-03-04T09:41:11.935 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:11.936+0000 7fb5853bd640 1 -- 172.21.15.52:0/4095129042 wait complete. 2024-03-04T09:41:11.945 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-04T09:41:12.054 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:12.054+0000 7f0d1ded4640 1 -- 172.21.15.52:0/3807659137 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0d18100eb0 msgr2=0x7f0d1810d320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:41:12.054 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:12.054+0000 7f0d1ded4640 1 --2- 172.21.15.52:0/3807659137 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0d18100eb0 0x7f0d1810d320 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f0d10009c10 tx=0x7f0d1002bef0 comp rx=0 tx=0).stop 2024-03-04T09:41:12.055 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:12.054+0000 7f0d1ded4640 1 -- 172.21.15.52:0/3807659137 shutdown_connections 2024-03-04T09:41:12.055 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:12.054+0000 7f0d1ded4640 1 --2- 172.21.15.52:0/3807659137 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0d18100eb0 0x7f0d1810d320 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:41:12.055 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:12.054+0000 7f0d1ded4640 1 --2- 172.21.15.52:0/3807659137 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0d181004f0 0x7f0d18100970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:41:12.055 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:12.054+0000 7f0d1ded4640 1 --2- 172.21.15.52:0/3807659137 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0d18106560 0x7f0d18106940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:41:12.055 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:12.054+0000 7f0d1ded4640 1 -- 172.21.15.52:0/3807659137 >> 172.21.15.52:0/3807659137 conn(0x7f0d18077630 msgr2=0x7f0d18077a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:41:12.055 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:12.054+0000 7f0d1ded4640 1 -- 172.21.15.52:0/3807659137 shutdown_connections 2024-03-04T09:41:12.055 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:12.054+0000 7f0d1ded4640 1 -- 172.21.15.52:0/3807659137 wait complete. 2024-03-04T09:41:12.055 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:12.054+0000 7f0d1ded4640 1 Processor -- start 2024-03-04T09:41:12.056 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:12.054+0000 7f0d1ded4640 1 -- start start 2024-03-04T09:41:12.056 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:12.055+0000 7f0d1ded4640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0d181004f0 0x7f0d1810fb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:41:12.056 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:12.055+0000 7f0d1ded4640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0d18100eb0 0x7f0d18110090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:41:12.056 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:12.055+0000 7f0d1ded4640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0d18106560 0x7f0d181105d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:41:12.056 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:12.055+0000 7f0d1ded4640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d1806a630 con 0x7f0d18100eb0 2024-03-04T09:41:12.056 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:12.055+0000 7f0d1ded4640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d1806a4b0 con 0x7f0d18106560 2024-03-04T09:41:12.056 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:12.055+0000 7f0d1ded4640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d1806a7b0 con 0x7f0d181004f0 2024-03-04T09:41:12.056 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:12.055+0000 7f0d16ffd640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0d18100eb0 0x7f0d18110090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:41:12.057 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:12.055+0000 7f0d16ffd640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0d18100eb0 0x7f0d18110090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:36972/0 (socket says 172.21.15.52:36972) 2024-03-04T09:41:12.057 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:12.055+0000 7f0d16ffd640 1 -- 172.21.15.52:0/1524903704 learned_addr learned my addr 172.21.15.52:0/1524903704 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-04T09:41:12.057 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:12.055+0000 7f0d17fff640 1 --2- 172.21.15.52:0/1524903704 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0d18106560 0x7f0d181105d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:41:12.057 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:12.055+0000 7f0d177fe640 1 --2- 172.21.15.52:0/1524903704 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0d181004f0 0x7f0d1810fb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:41:12.057 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:12.055+0000 7f0d16ffd640 1 -- 172.21.15.52:0/1524903704 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0d181004f0 msgr2=0x7f0d1810fb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:41:12.057 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:12.055+0000 7f0d16ffd640 1 --2- 172.21.15.52:0/1524903704 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0d181004f0 0x7f0d1810fb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:41:12.057 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:12.055+0000 7f0d16ffd640 1 -- 172.21.15.52:0/1524903704 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0d18106560 msgr2=0x7f0d181105d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:41:12.057 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:12.055+0000 7f0d16ffd640 1 --2- 172.21.15.52:0/1524903704 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0d18106560 0x7f0d181105d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:41:12.057 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:12.055+0000 7f0d16ffd640 1 -- 172.21.15.52:0/1524903704 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d181a6280 con 0x7f0d18100eb0 2024-03-04T09:41:12.058 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:12.055+0000 7f0d177fe640 1 --2- 172.21.15.52:0/1524903704 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0d181004f0 0x7f0d1810fb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T09:41:12.058 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:12.055+0000 7f0d16ffd640 1 --2- 172.21.15.52:0/1524903704 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0d18100eb0 0x7f0d18110090 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f0d0c00e9f0 tx=0x7f0d0c00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:41:12.058 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:12.055+0000 7f0d14ff9640 1 -- 172.21.15.52:0/1524903704 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d0c00cd80 con 0x7f0d18100eb0 2024-03-04T09:41:12.058 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:12.055+0000 7f0d14ff9640 1 -- 172.21.15.52:0/1524903704 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0d0c005500 con 0x7f0d18100eb0 2024-03-04T09:41:12.058 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:12.055+0000 7f0d1ded4640 1 -- 172.21.15.52:0/1524903704 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d181a6570 con 0x7f0d18100eb0 2024-03-04T09:41:12.058 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:12.055+0000 7f0d14ff9640 1 -- 172.21.15.52:0/1524903704 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d0c010690 con 0x7f0d18100eb0 2024-03-04T09:41:12.058 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:12.056+0000 7f0d1ded4640 1 -- 172.21.15.52:0/1524903704 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d181a6a70 con 0x7f0d18100eb0 2024-03-04T09:41:12.058 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:12.056+0000 7f0d1ded4640 1 -- 172.21.15.52:0/1524903704 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d18067b30 con 0x7f0d18100eb0 2024-03-04T09:41:12.059 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:12.057+0000 7f0d14ff9640 1 -- 172.21.15.52:0/1524903704 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 25) v1 ==== 136887+0+0 (secure 0 0 0) 0x7f0d0c0046c0 con 0x7f0d18100eb0 2024-03-04T09:41:12.059 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:12.058+0000 7f0d14ff9640 1 --2- 172.21.15.52:0/1524903704 >> [v2:172.21.15.52:6800/478031757,v1:172.21.15.52:6801/478031757] conn(0x7f0d000a1ef0 0x7f0d000a43b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:41:12.060 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:12.058+0000 7f0d14ff9640 1 -- 172.21.15.52:0/1524903704 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 9635+0+0 (secure 0 0 0) 0x7f0d0c0ceac0 con 0x7f0d18100eb0 2024-03-04T09:41:12.060 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:12.058+0000 7f0d177fe640 1 --2- 172.21.15.52:0/1524903704 >> [v2:172.21.15.52:6800/478031757,v1:172.21.15.52:6801/478031757] conn(0x7f0d000a1ef0 0x7f0d000a43b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:41:12.060 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:12.058+0000 7f0d177fe640 1 --2- 172.21.15.52:0/1524903704 >> [v2:172.21.15.52:6800/478031757,v1:172.21.15.52:6801/478031757] conn(0x7f0d000a1ef0 0x7f0d000a43b0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f0d08005fd0 tx=0x7f0d08009260 comp rx=0 tx=0).ready entity=mgr.15486 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:41:12.060 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:12.062+0000 7f0d14ff9640 1 -- 172.21.15.52:0/1524903704 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f0d0c09b750 con 0x7f0d18100eb0 2024-03-04T09:41:12.244 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:12.247+0000 7f0d1ded4640 1 -- 172.21.15.52:0/1524903704 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f0d1802cff0 con 0x7f0d18100eb0 2024-03-04T09:41:12.245 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:12.247+0000 7f0d14ff9640 1 -- 172.21.15.52:0/1524903704 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v101) v1 ==== 100+0+11 (secure 0 0 0) 0x7f0d0c09b050 con 0x7f0d18100eb0 2024-03-04T09:41:12.245 INFO:teuthology.orchestra.run.smithi052.stdout:pg_num: 64 2024-03-04T09:41:12.247 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:12.249+0000 7f0d1ded4640 1 -- 172.21.15.52:0/1524903704 >> [v2:172.21.15.52:6800/478031757,v1:172.21.15.52:6801/478031757] conn(0x7f0d000a1ef0 msgr2=0x7f0d000a43b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:41:12.247 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:12.249+0000 7f0d1ded4640 1 --2- 172.21.15.52:0/1524903704 >> [v2:172.21.15.52:6800/478031757,v1:172.21.15.52:6801/478031757] conn(0x7f0d000a1ef0 0x7f0d000a43b0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f0d08005fd0 tx=0x7f0d08009260 comp rx=0 tx=0).stop 2024-03-04T09:41:12.247 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:12.249+0000 7f0d1ded4640 1 -- 172.21.15.52:0/1524903704 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0d18100eb0 msgr2=0x7f0d18110090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:41:12.247 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:12.249+0000 7f0d1ded4640 1 --2- 172.21.15.52:0/1524903704 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0d18100eb0 0x7f0d18110090 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f0d0c00e9f0 tx=0x7f0d0c00edf0 comp rx=0 tx=0).stop 2024-03-04T09:41:12.247 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:12.249+0000 7f0d1ded4640 1 -- 172.21.15.52:0/1524903704 shutdown_connections 2024-03-04T09:41:12.248 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:12.249+0000 7f0d1ded4640 1 --2- 172.21.15.52:0/1524903704 >> [v2:172.21.15.52:6800/478031757,v1:172.21.15.52:6801/478031757] conn(0x7f0d000a1ef0 0x7f0d000a43b0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:41:12.248 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:12.249+0000 7f0d1ded4640 1 --2- 172.21.15.52:0/1524903704 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0d18106560 0x7f0d181105d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:41:12.248 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:12.249+0000 7f0d1ded4640 1 --2- 172.21.15.52:0/1524903704 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0d18100eb0 0x7f0d18110090 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:41:12.248 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:12.249+0000 7f0d1ded4640 1 --2- 172.21.15.52:0/1524903704 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0d181004f0 0x7f0d1810fb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:41:12.248 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:12.249+0000 7f0d1ded4640 1 -- 172.21.15.52:0/1524903704 >> 172.21.15.52:0/1524903704 conn(0x7f0d18077630 msgr2=0x7f0d180ff0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:41:12.248 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:12.249+0000 7f0d1ded4640 1 -- 172.21.15.52:0/1524903704 shutdown_connections 2024-03-04T09:41:12.248 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:12.249+0000 7f0d1ded4640 1 -- 172.21.15.52:0/1524903704 wait complete. 2024-03-04T09:41:12.258 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-04T09:41:12.365 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:12.365+0000 7faad74af640 1 -- 172.21.15.52:0/2274752485 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7faad0100830 msgr2=0x7faad010cb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:41:12.366 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:12.365+0000 7faad74af640 1 --2- 172.21.15.52:0/2274752485 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7faad0100830 0x7faad010cb00 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7faac8009ca0 tx=0x7faac802c0f0 comp rx=0 tx=0).stop 2024-03-04T09:41:12.366 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:12.365+0000 7faad74af640 1 -- 172.21.15.52:0/2274752485 shutdown_connections 2024-03-04T09:41:12.366 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:12.365+0000 7faad74af640 1 --2- 172.21.15.52:0/2274752485 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7faad0100830 0x7faad010cb00 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:41:12.366 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:12.365+0000 7faad74af640 1 --2- 172.21.15.52:0/2274752485 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7faad00ffe90 0x7faad01002f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:41:12.366 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:12.365+0000 7faad74af640 1 --2- 172.21.15.52:0/2274752485 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7faad0104c00 0x7faad0104fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:41:12.366 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:12.365+0000 7faad74af640 1 -- 172.21.15.52:0/2274752485 >> 172.21.15.52:0/2274752485 conn(0x7faad0075810 msgr2=0x7faad0075c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:41:12.366 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:12.365+0000 7faad74af640 1 -- 172.21.15.52:0/2274752485 shutdown_connections 2024-03-04T09:41:12.367 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:12.365+0000 7faad74af640 1 -- 172.21.15.52:0/2274752485 wait complete. 2024-03-04T09:41:12.367 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:12.366+0000 7faad74af640 1 Processor -- start 2024-03-04T09:41:12.367 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:12.366+0000 7faad74af640 1 -- start start 2024-03-04T09:41:12.367 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:12.366+0000 7faad74af640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7faad00ffe90 0x7faad0198b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:41:12.367 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:12.366+0000 7faad74af640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7faad0100830 0x7faad0199070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:41:12.367 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:12.366+0000 7faad74af640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7faad0104c00 0x7faad01a3bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:41:12.367 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:12.366+0000 7faad74af640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faad01108b0 con 0x7faad0104c00 2024-03-04T09:41:12.367 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:12.366+0000 7faad74af640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faad0110730 con 0x7faad00ffe90 2024-03-04T09:41:12.368 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:12.366+0000 7faad74af640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faad0110a30 con 0x7faad0100830 2024-03-04T09:41:12.368 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:12.366+0000 7faad5a4c640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7faad0104c00 0x7faad01a3bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:41:12.368 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:12.366+0000 7faad5a4c640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7faad0104c00 0x7faad01a3bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:36978/0 (socket says 172.21.15.52:36978) 2024-03-04T09:41:12.368 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:12.366+0000 7faad5a4c640 1 -- 172.21.15.52:0/1656086961 learned_addr learned my addr 172.21.15.52:0/1656086961 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-04T09:41:12.368 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:12.366+0000 7faad4a4a640 1 --2- 172.21.15.52:0/1656086961 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7faad0100830 0x7faad0199070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:41:12.368 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:12.366+0000 7faad524b640 1 --2- 172.21.15.52:0/1656086961 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7faad00ffe90 0x7faad0198b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:41:12.368 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:12.366+0000 7faad5a4c640 1 -- 172.21.15.52:0/1656086961 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7faad0100830 msgr2=0x7faad0199070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:41:12.368 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:12.366+0000 7faad5a4c640 1 --2- 172.21.15.52:0/1656086961 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7faad0100830 0x7faad0199070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:41:12.369 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:12.366+0000 7faad5a4c640 1 -- 172.21.15.52:0/1656086961 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7faad00ffe90 msgr2=0x7faad0198b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:41:12.369 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:12.366+0000 7faad5a4c640 1 --2- 172.21.15.52:0/1656086961 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7faad00ffe90 0x7faad0198b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:41:12.369 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:12.366+0000 7faad5a4c640 1 -- 172.21.15.52:0/1656086961 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faad01a40f0 con 0x7faad0104c00 2024-03-04T09:41:12.369 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:12.366+0000 7faad4a4a640 1 --2- 172.21.15.52:0/1656086961 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7faad0100830 0x7faad0199070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-04T09:41:12.369 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:12.367+0000 7faad5a4c640 1 --2- 172.21.15.52:0/1656086961 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7faad0104c00 0x7faad01a3bb0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7faac802c700 tx=0x7faac8032f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:41:12.369 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:12.367+0000 7faac67fc640 1 -- 172.21.15.52:0/1656086961 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faac8042020 con 0x7faad0104c00 2024-03-04T09:41:12.369 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:12.367+0000 7faac67fc640 1 -- 172.21.15.52:0/1656086961 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7faac8034640 con 0x7faad0104c00 2024-03-04T09:41:12.369 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:12.367+0000 7faad74af640 1 -- 172.21.15.52:0/1656086961 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faad01a42c0 con 0x7faad0104c00 2024-03-04T09:41:12.370 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:12.367+0000 7faac67fc640 1 -- 172.21.15.52:0/1656086961 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faac8039040 con 0x7faad0104c00 2024-03-04T09:41:12.370 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:12.367+0000 7faad74af640 1 -- 172.21.15.52:0/1656086961 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faad01a4780 con 0x7faad0104c00 2024-03-04T09:41:12.370 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:12.368+0000 7faad74af640 1 -- 172.21.15.52:0/1656086961 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faad0068810 con 0x7faad0104c00 2024-03-04T09:41:12.371 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:12.369+0000 7faac67fc640 1 -- 172.21.15.52:0/1656086961 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 25) v1 ==== 136887+0+0 (secure 0 0 0) 0x7faac8045020 con 0x7faad0104c00 2024-03-04T09:41:12.371 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:12.369+0000 7faac67fc640 1 --2- 172.21.15.52:0/1656086961 >> [v2:172.21.15.52:6800/478031757,v1:172.21.15.52:6801/478031757] conn(0x7faabc0a1ef0 0x7faabc0a43b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:41:12.371 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:12.370+0000 7faac67fc640 1 -- 172.21.15.52:0/1656086961 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 9635+0+0 (secure 0 0 0) 0x7faac80f0870 con 0x7faad0104c00 2024-03-04T09:41:12.371 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:12.370+0000 7faad524b640 1 --2- 172.21.15.52:0/1656086961 >> [v2:172.21.15.52:6800/478031757,v1:172.21.15.52:6801/478031757] conn(0x7faabc0a1ef0 0x7faabc0a43b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:41:12.371 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:12.370+0000 7faad524b640 1 --2- 172.21.15.52:0/1656086961 >> [v2:172.21.15.52:6800/478031757,v1:172.21.15.52:6801/478031757] conn(0x7faabc0a1ef0 0x7faabc0a43b0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7faacc0095f0 tx=0x7faacc005f90 comp rx=0 tx=0).ready entity=mgr.15486 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:41:12.371 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:12.373+0000 7faac67fc640 1 -- 172.21.15.52:0/1656086961 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7faac804c020 con 0x7faad0104c00 2024-03-04T09:41:12.552 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:12.555+0000 7faad74af640 1 -- 172.21.15.52:0/1656086961 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7faad0107bc0 con 0x7faad0104c00 2024-03-04T09:41:12.553 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:12.555+0000 7faac67fc640 1 -- 172.21.15.52:0/1656086961 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v101) v1 ==== 103+0+11 (secure 0 0 0) 0x7faac80bcfc0 con 0x7faad0104c00 2024-03-04T09:41:12.553 INFO:teuthology.orchestra.run.smithi052.stdout:pg_num: 32 2024-03-04T09:41:12.555 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:12.557+0000 7faad74af640 1 -- 172.21.15.52:0/1656086961 >> [v2:172.21.15.52:6800/478031757,v1:172.21.15.52:6801/478031757] conn(0x7faabc0a1ef0 msgr2=0x7faabc0a43b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:41:12.555 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:12.557+0000 7faad74af640 1 --2- 172.21.15.52:0/1656086961 >> [v2:172.21.15.52:6800/478031757,v1:172.21.15.52:6801/478031757] conn(0x7faabc0a1ef0 0x7faabc0a43b0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7faacc0095f0 tx=0x7faacc005f90 comp rx=0 tx=0).stop 2024-03-04T09:41:12.555 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:12.557+0000 7faad74af640 1 -- 172.21.15.52:0/1656086961 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7faad0104c00 msgr2=0x7faad01a3bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:41:12.555 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:12.557+0000 7faad74af640 1 --2- 172.21.15.52:0/1656086961 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7faad0104c00 0x7faad01a3bb0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7faac802c700 tx=0x7faac8032f20 comp rx=0 tx=0).stop 2024-03-04T09:41:12.555 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:12.557+0000 7faad74af640 1 -- 172.21.15.52:0/1656086961 shutdown_connections 2024-03-04T09:41:12.556 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:12.557+0000 7faad74af640 1 --2- 172.21.15.52:0/1656086961 >> [v2:172.21.15.52:6800/478031757,v1:172.21.15.52:6801/478031757] conn(0x7faabc0a1ef0 0x7faabc0a43b0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:41:12.556 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:12.557+0000 7faad74af640 1 --2- 172.21.15.52:0/1656086961 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7faad0104c00 0x7faad01a3bb0 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:41:12.556 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:12.557+0000 7faad74af640 1 --2- 172.21.15.52:0/1656086961 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7faad0100830 0x7faad0199070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:41:12.556 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:12.557+0000 7faad74af640 1 --2- 172.21.15.52:0/1656086961 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7faad00ffe90 0x7faad0198b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:41:12.556 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:12.557+0000 7faad74af640 1 -- 172.21.15.52:0/1656086961 >> 172.21.15.52:0/1656086961 conn(0x7faad0075810 msgr2=0x7faad00fdea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:41:12.556 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:12.557+0000 7faad74af640 1 -- 172.21.15.52:0/1656086961 shutdown_connections 2024-03-04T09:41:12.556 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:12.557+0000 7faad74af640 1 -- 172.21.15.52:0/1656086961 wait complete. 2024-03-04T09:41:12.564 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:41:12 smithi052 ceph-mon[32473]: pgmap v311: 129 pgs: 2 active+clean+snaptrim, 7 active+clean+snaptrim_wait, 120 active+clean; 78 MiB data, 687 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-03-04T09:41:12.564 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:41:12 smithi052 ceph-mon[32473]: from='client.15582 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T09:41:12.565 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:41:12 smithi052 ceph-mon[32473]: from='client.15588 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T09:41:12.565 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:41:12 smithi052 ceph-mon[32473]: from='client.? 172.21.15.52:0/3236477509' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-04T09:41:12.565 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:41:12 smithi052 ceph-mon[32473]: from='client.? 172.21.15.52:0/4095129042' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-04T09:41:12.565 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:41:12 smithi052 ceph-mon[32473]: from='client.? 172.21.15.52:0/1524903704' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-04T09:41:12.566 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2024-03-04T09:41:12.672 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:12.672+0000 7f945ab7a640 1 -- 172.21.15.52:0/3728915906 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9454101f40 msgr2=0x7f945410e400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:41:12.672 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:12.672+0000 7f945ab7a640 1 --2- 172.21.15.52:0/3728915906 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9454101f40 0x7f945410e400 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f944c009ca0 tx=0x7f944c02c0f0 comp rx=0 tx=0).stop 2024-03-04T09:41:12.672 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:12.672+0000 7f945ab7a640 1 -- 172.21.15.52:0/3728915906 shutdown_connections 2024-03-04T09:41:12.672 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:12.672+0000 7f945ab7a640 1 --2- 172.21.15.52:0/3728915906 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9454101f40 0x7f945410e400 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:41:12.672 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:12.672+0000 7f945ab7a640 1 --2- 172.21.15.52:0/3728915906 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9454101580 0x7f9454101a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:41:12.672 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:12.672+0000 7f945ab7a640 1 --2- 172.21.15.52:0/3728915906 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9454107640 0x7f9454107a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:41:12.673 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:12.672+0000 7f945ab7a640 1 -- 172.21.15.52:0/3728915906 >> 172.21.15.52:0/3728915906 conn(0x7f94540fd070 msgr2=0x7f94540ff490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:41:12.673 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:12.672+0000 7f945ab7a640 1 -- 172.21.15.52:0/3728915906 shutdown_connections 2024-03-04T09:41:12.673 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:12.672+0000 7f945ab7a640 1 -- 172.21.15.52:0/3728915906 wait complete. 2024-03-04T09:41:12.673 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:12.672+0000 7f945ab7a640 1 Processor -- start 2024-03-04T09:41:12.673 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:12.672+0000 7f945ab7a640 1 -- start start 2024-03-04T09:41:12.673 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:12.673+0000 7f945ab7a640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9454101580 0x7f94540738d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:41:12.673 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:12.673+0000 7f945ab7a640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9454101f40 0x7f9454073e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:41:12.674 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:12.673+0000 7f945ab7a640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9454107640 0x7f9454074350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:41:12.674 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:12.673+0000 7f945ab7a640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9454111920 con 0x7f9454101f40 2024-03-04T09:41:12.674 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:12.673+0000 7f945ab7a640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94541117a0 con 0x7f9454101580 2024-03-04T09:41:12.674 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:12.673+0000 7f945ab7a640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9454111aa0 con 0x7f9454107640 2024-03-04T09:41:12.674 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:12.673+0000 7f9453fff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9454101f40 0x7f9454073e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:41:12.674 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:12.673+0000 7f9453fff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9454101f40 0x7f9454073e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:36998/0 (socket says 172.21.15.52:36998) 2024-03-04T09:41:12.674 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:12.673+0000 7f9453fff640 1 -- 172.21.15.52:0/2700761962 learned_addr learned my addr 172.21.15.52:0/2700761962 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-04T09:41:12.674 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:12.673+0000 7f9458916640 1 --2- 172.21.15.52:0/2700761962 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9454101580 0x7f94540738d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:41:12.674 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:12.673+0000 7f9459117640 1 --2- 172.21.15.52:0/2700761962 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9454107640 0x7f9454074350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:41:12.675 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:12.673+0000 7f9453fff640 1 -- 172.21.15.52:0/2700761962 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9454107640 msgr2=0x7f9454074350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:41:12.675 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:12.673+0000 7f9453fff640 1 --2- 172.21.15.52:0/2700761962 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9454107640 0x7f9454074350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:41:12.675 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:12.673+0000 7f9453fff640 1 -- 172.21.15.52:0/2700761962 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9454101580 msgr2=0x7f94540738d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:41:12.675 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:12.673+0000 7f9453fff640 1 --2- 172.21.15.52:0/2700761962 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9454101580 0x7f94540738d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:41:12.675 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:12.673+0000 7f9453fff640 1 -- 172.21.15.52:0/2700761962 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94541a6260 con 0x7f9454101f40 2024-03-04T09:41:12.675 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:12.673+0000 7f9453fff640 1 --2- 172.21.15.52:0/2700761962 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9454101f40 0x7f9454073e10 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f944000e9f0 tx=0x7f944000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:41:12.676 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:12.673+0000 7f9451ffb640 1 -- 172.21.15.52:0/2700761962 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f944000cd80 con 0x7f9454101f40 2024-03-04T09:41:12.676 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:12.673+0000 7f945ab7a640 1 -- 172.21.15.52:0/2700761962 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94541a6550 con 0x7f9454101f40 2024-03-04T09:41:12.676 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:12.673+0000 7f9451ffb640 1 -- 172.21.15.52:0/2700761962 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9440005500 con 0x7f9454101f40 2024-03-04T09:41:12.676 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:12.673+0000 7f9451ffb640 1 -- 172.21.15.52:0/2700761962 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9440010690 con 0x7f9454101f40 2024-03-04T09:41:12.676 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:12.674+0000 7f945ab7a640 1 -- 172.21.15.52:0/2700761962 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94541a6aa0 con 0x7f9454101f40 2024-03-04T09:41:12.677 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:12.674+0000 7f945ab7a640 1 -- 172.21.15.52:0/2700761962 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9454068810 con 0x7f9454101f40 2024-03-04T09:41:12.677 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:12.675+0000 7f9451ffb640 1 -- 172.21.15.52:0/2700761962 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 25) v1 ==== 136887+0+0 (secure 0 0 0) 0x7f94400046c0 con 0x7f9454101f40 2024-03-04T09:41:12.677 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:12.676+0000 7f9451ffb640 1 --2- 172.21.15.52:0/2700761962 >> [v2:172.21.15.52:6800/478031757,v1:172.21.15.52:6801/478031757] conn(0x7f94440a1ef0 0x7f94440a43b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:41:12.678 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:12.676+0000 7f9458916640 1 --2- 172.21.15.52:0/2700761962 >> [v2:172.21.15.52:6800/478031757,v1:172.21.15.52:6801/478031757] conn(0x7f94440a1ef0 0x7f94440a43b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:41:12.678 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:12.676+0000 7f9451ffb640 1 -- 172.21.15.52:0/2700761962 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 9635+0+0 (secure 0 0 0) 0x7f94400cea10 con 0x7f9454101f40 2024-03-04T09:41:12.678 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:12.676+0000 7f9458916640 1 --2- 172.21.15.52:0/2700761962 >> [v2:172.21.15.52:6800/478031757,v1:172.21.15.52:6801/478031757] conn(0x7f94440a1ef0 0x7f94440a43b0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f9448009b50 tx=0x7f9448005f90 comp rx=0 tx=0).ready entity=mgr.15486 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:41:12.678 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:12.680+0000 7f9451ffb640 1 -- 172.21.15.52:0/2700761962 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f944009b6a0 con 0x7f9454101f40 2024-03-04T09:41:12.837 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:41:12 smithi191 ceph-mon[36816]: pgmap v311: 129 pgs: 2 active+clean+snaptrim, 7 active+clean+snaptrim_wait, 120 active+clean; 78 MiB data, 687 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-03-04T09:41:12.837 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:41:12 smithi191 ceph-mon[36816]: from='client.15582 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T09:41:12.837 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:41:12 smithi191 ceph-mon[36816]: from='client.15588 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T09:41:12.837 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:41:12 smithi191 ceph-mon[36816]: from='client.? 172.21.15.52:0/3236477509' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-04T09:41:12.838 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:41:12 smithi191 ceph-mon[36816]: from='client.? 172.21.15.52:0/4095129042' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-04T09:41:12.838 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:41:12 smithi191 ceph-mon[36816]: from='client.? 172.21.15.52:0/1524903704' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-04T09:41:12.882 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:41:12 smithi106 ceph-mon[36484]: pgmap v311: 129 pgs: 2 active+clean+snaptrim, 7 active+clean+snaptrim_wait, 120 active+clean; 78 MiB data, 687 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-03-04T09:41:12.882 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:41:12 smithi106 ceph-mon[36484]: from='client.15582 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T09:41:12.882 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:41:12 smithi106 ceph-mon[36484]: from='client.15588 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T09:41:12.882 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:41:12 smithi106 ceph-mon[36484]: from='client.? 172.21.15.52:0/3236477509' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-04T09:41:12.882 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:41:12 smithi106 ceph-mon[36484]: from='client.? 172.21.15.52:0/4095129042' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-04T09:41:12.882 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:41:12 smithi106 ceph-mon[36484]: from='client.? 172.21.15.52:0/1524903704' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-04T09:41:12.911 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:12.913+0000 7f945ab7a640 1 -- 172.21.15.52:0/2700761962 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) v1 -- 0x7f94541099b0 con 0x7f9454101f40 2024-03-04T09:41:12.919 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:12.921+0000 7f9451ffb640 1 -- 172.21.15.52:0/2700761962 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v25) v1 ==== 81+0+736925 (secure 0 0 0) 0x7f944009afa0 con 0x7f9454101f40 2024-03-04T09:41:12.930 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-03-04T09:41:12.930 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2024-03-04T09:41:12.930 INFO:teuthology.orchestra.run.smithi052.stdout: "epoch": 25, 2024-03-04T09:41:12.931 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:12.931 INFO:teuthology.orchestra.run.smithi052.stdout: "active_gid": 15486, 2024-03-04T09:41:12.931 INFO:teuthology.orchestra.run.smithi052.stdout: "active_name": "x", 2024-03-04T09:41:12.931 INFO:teuthology.orchestra.run.smithi052.stdout: "active_addrs": { 2024-03-04T09:41:12.931 INFO:teuthology.orchestra.run.smithi052.stdout: "addrvec": [ 2024-03-04T09:41:12.931 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-03-04T09:41:12.931 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "v2", 2024-03-04T09:41:12.931 INFO:teuthology.orchestra.run.smithi052.stdout: "addr": "172.21.15.52:6800", 2024-03-04T09:41:12.932 INFO:teuthology.orchestra.run.smithi052.stdout: "nonce": 478031757 2024-03-04T09:41:12.932 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:12.932 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-03-04T09:41:12.932 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "v1", 2024-03-04T09:41:12.932 INFO:teuthology.orchestra.run.smithi052.stdout: "addr": "172.21.15.52:6801", 2024-03-04T09:41:12.932 INFO:teuthology.orchestra.run.smithi052.stdout: "nonce": 478031757 2024-03-04T09:41:12.932 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-03-04T09:41:12.932 INFO:teuthology.orchestra.run.smithi052.stdout: ] 2024-03-04T09:41:12.933 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:12.933 INFO:teuthology.orchestra.run.smithi052.stdout: "active_addr": "172.21.15.52:6801/478031757", 2024-03-04T09:41:12.933 INFO:teuthology.orchestra.run.smithi052.stdout: "active_change": "2024-03-04T09:31:08.422305+0000", 2024-03-04T09:41:12.933 INFO:teuthology.orchestra.run.smithi052.stdout: "active_mgr_features": 4540701547738038271, 2024-03-04T09:41:12.933 INFO:teuthology.orchestra.run.smithi052.stdout: "available": true, 2024-03-04T09:41:12.933 INFO:teuthology.orchestra.run.smithi052.stdout: "standbys": [ 2024-03-04T09:41:12.933 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-03-04T09:41:12.933 INFO:teuthology.orchestra.run.smithi052.stdout: "gid": 15480, 2024-03-04T09:41:12.933 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "z", 2024-03-04T09:41:12.934 INFO:teuthology.orchestra.run.smithi052.stdout: "mgr_features": 4540701547738038271, 2024-03-04T09:41:12.934 INFO:teuthology.orchestra.run.smithi052.stdout: "available_modules": [ 2024-03-04T09:41:12.934 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-03-04T09:41:12.934 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "alerts", 2024-03-04T09:41:12.934 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-03-04T09:41:12.934 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-03-04T09:41:12.934 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-03-04T09:41:12.935 INFO:teuthology.orchestra.run.smithi052.stdout: "interval": { 2024-03-04T09:41:12.935 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "interval", 2024-03-04T09:41:12.935 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "secs", 2024-03-04T09:41:12.935 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:12.935 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:12.935 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "60", 2024-03-04T09:41:12.935 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:12.935 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:12.936 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:12.936 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "How frequently to reexamine health status", 2024-03-04T09:41:12.936 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:12.936 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:12.936 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:12.936 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:12.936 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-03-04T09:41:12.936 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-03-04T09:41:12.937 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:12.937 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:12.937 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:12.937 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-03-04T09:41:12.937 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:12.937 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:12.937 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-03-04T09:41:12.937 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-03-04T09:41:12.937 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-03-04T09:41:12.938 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-03-04T09:41:12.938 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-03-04T09:41:12.938 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-03-04T09:41:12.938 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-03-04T09:41:12.938 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-03-04T09:41:12.938 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:12.938 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:12.938 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:12.939 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:12.939 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:12.939 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-03-04T09:41:12.939 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-03-04T09:41:12.939 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:12.939 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:12.939 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:12.940 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-03-04T09:41:12.940 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:12.940 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:12.940 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:12.940 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:12.940 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:12.940 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:12.940 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:12.941 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:12.941 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-03-04T09:41:12.941 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-03-04T09:41:12.941 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:12.941 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:12.941 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:12.941 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-03-04T09:41:12.941 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:12.942 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:12.942 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-03-04T09:41:12.942 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-03-04T09:41:12.942 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-03-04T09:41:12.942 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-03-04T09:41:12.942 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-03-04T09:41:12.942 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-03-04T09:41:12.942 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-03-04T09:41:12.943 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-03-04T09:41:12.943 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:12.943 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:12.943 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:12.943 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:12.943 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:12.943 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-03-04T09:41:12.943 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-03-04T09:41:12.943 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:12.944 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:12.944 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:12.944 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-03-04T09:41:12.944 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:12.944 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:12.944 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:12.944 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:12.944 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:12.945 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:12.945 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:12.945 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:12.945 INFO:teuthology.orchestra.run.smithi052.stdout: "smtp_destination": { 2024-03-04T09:41:12.945 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "smtp_destination", 2024-03-04T09:41:12.945 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:12.945 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:12.946 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:12.946 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-03-04T09:41:12.946 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:12.946 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:12.946 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:12.946 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Email address to send alerts to", 2024-03-04T09:41:12.946 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:12.947 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:12.947 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:12.947 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:12.947 INFO:teuthology.orchestra.run.smithi052.stdout: "smtp_from_name": { 2024-03-04T09:41:12.947 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "smtp_from_name", 2024-03-04T09:41:12.947 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:12.947 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:12.947 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:12.947 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "Ceph", 2024-03-04T09:41:12.948 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:12.948 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:12.948 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:12.948 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Email From: name", 2024-03-04T09:41:12.948 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:12.948 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:12.948 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:12.948 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:12.949 INFO:teuthology.orchestra.run.smithi052.stdout: "smtp_host": { 2024-03-04T09:41:12.949 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "smtp_host", 2024-03-04T09:41:12.949 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:12.949 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:12.949 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:12.949 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-03-04T09:41:12.949 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:12.949 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:12.950 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:12.950 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "SMTP server", 2024-03-04T09:41:12.950 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:12.950 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:12.950 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:12.950 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:12.950 INFO:teuthology.orchestra.run.smithi052.stdout: "smtp_password": { 2024-03-04T09:41:12.950 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "smtp_password", 2024-03-04T09:41:12.951 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:12.951 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:12.951 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:12.951 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-03-04T09:41:12.951 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:12.951 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:12.951 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:12.951 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Password to authenticate with", 2024-03-04T09:41:12.952 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:12.952 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:12.952 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:12.952 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:12.952 INFO:teuthology.orchestra.run.smithi052.stdout: "smtp_port": { 2024-03-04T09:41:12.952 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "smtp_port", 2024-03-04T09:41:12.952 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-03-04T09:41:12.952 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:12.953 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:12.953 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "465", 2024-03-04T09:41:12.953 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:12.953 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:12.953 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:12.953 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "SMTP port", 2024-03-04T09:41:12.953 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:12.953 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:12.954 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:12.954 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:12.954 INFO:teuthology.orchestra.run.smithi052.stdout: "smtp_sender": { 2024-03-04T09:41:12.954 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "smtp_sender", 2024-03-04T09:41:12.954 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:12.954 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:12.954 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:12.954 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-03-04T09:41:12.955 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:12.955 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:12.955 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:12.955 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "SMTP envelope sender", 2024-03-04T09:41:12.955 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:12.955 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:12.955 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:12.956 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:12.956 INFO:teuthology.orchestra.run.smithi052.stdout: "smtp_ssl": { 2024-03-04T09:41:12.956 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "smtp_ssl", 2024-03-04T09:41:12.956 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:12.956 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:12.956 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:12.956 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-03-04T09:41:12.956 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:12.957 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:12.957 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:12.957 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Use SSL to connect to SMTP server", 2024-03-04T09:41:12.957 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:12.957 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:12.957 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:12.957 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:12.957 INFO:teuthology.orchestra.run.smithi052.stdout: "smtp_user": { 2024-03-04T09:41:12.958 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "smtp_user", 2024-03-04T09:41:12.958 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:12.958 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:12.958 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:12.958 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-03-04T09:41:12.958 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:12.958 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:12.958 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:12.959 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "User to authenticate as", 2024-03-04T09:41:12.959 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:12.959 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:12.959 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:12.959 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-03-04T09:41:12.959 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-03-04T09:41:12.959 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:12.959 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-03-04T09:41:12.959 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "balancer", 2024-03-04T09:41:12.960 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-03-04T09:41:12.960 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-03-04T09:41:12.960 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-03-04T09:41:12.960 INFO:teuthology.orchestra.run.smithi052.stdout: "active": { 2024-03-04T09:41:12.960 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "active", 2024-03-04T09:41:12.960 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:12.960 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:12.961 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:12.961 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-03-04T09:41:12.961 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:12.961 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:12.961 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:12.961 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "automatically balance PGs across cluster", 2024-03-04T09:41:12.961 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:12.961 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:12.962 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:12.962 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:12.962 INFO:teuthology.orchestra.run.smithi052.stdout: "begin_time": { 2024-03-04T09:41:12.962 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "begin_time", 2024-03-04T09:41:12.962 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:12.962 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:12.962 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:12.962 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "0000", 2024-03-04T09:41:12.963 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:12.963 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:12.963 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:12.963 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "beginning time of day to automatically balance", 2024-03-04T09:41:12.963 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-03-04T09:41:12.963 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:12.963 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:12.963 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:12.964 INFO:teuthology.orchestra.run.smithi052.stdout: "begin_weekday": { 2024-03-04T09:41:12.964 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "begin_weekday", 2024-03-04T09:41:12.964 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "uint", 2024-03-04T09:41:12.964 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:12.964 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:12.964 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "0", 2024-03-04T09:41:12.964 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "0", 2024-03-04T09:41:12.964 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "6", 2024-03-04T09:41:12.965 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:12.965 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-03-04T09:41:12.965 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-03-04T09:41:12.965 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:12.965 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:12.965 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:12.965 INFO:teuthology.orchestra.run.smithi052.stdout: "crush_compat_max_iterations": { 2024-03-04T09:41:12.965 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "crush_compat_max_iterations", 2024-03-04T09:41:12.965 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "uint", 2024-03-04T09:41:12.966 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:12.966 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:12.966 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "25", 2024-03-04T09:41:12.966 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "1", 2024-03-04T09:41:12.966 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "250", 2024-03-04T09:41:12.966 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:12.966 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-03-04T09:41:12.966 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:12.967 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:12.967 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:12.967 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:12.967 INFO:teuthology.orchestra.run.smithi052.stdout: "crush_compat_metrics": { 2024-03-04T09:41:12.967 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "crush_compat_metrics", 2024-03-04T09:41:12.967 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:12.967 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:12.967 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:12.968 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "pgs,objects,bytes", 2024-03-04T09:41:12.968 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:12.968 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:12.968 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:12.968 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-03-04T09:41:12.968 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-03-04T09:41:12.968 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:12.969 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:12.969 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:12.969 INFO:teuthology.orchestra.run.smithi052.stdout: "crush_compat_step": { 2024-03-04T09:41:12.969 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "crush_compat_step", 2024-03-04T09:41:12.969 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "float", 2024-03-04T09:41:12.969 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:12.969 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:12.969 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "0.5", 2024-03-04T09:41:12.970 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "0.001", 2024-03-04T09:41:12.970 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "0.999", 2024-03-04T09:41:12.970 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:12.970 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "aggressiveness of optimization", 2024-03-04T09:41:12.970 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-03-04T09:41:12.970 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:12.970 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:12.970 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:12.970 INFO:teuthology.orchestra.run.smithi052.stdout: "end_time": { 2024-03-04T09:41:12.971 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "end_time", 2024-03-04T09:41:12.971 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:12.971 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:12.971 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:12.971 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "2359", 2024-03-04T09:41:12.971 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:12.971 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:12.971 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:12.972 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "ending time of day to automatically balance", 2024-03-04T09:41:12.972 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-03-04T09:41:12.972 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:12.972 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:12.972 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:12.972 INFO:teuthology.orchestra.run.smithi052.stdout: "end_weekday": { 2024-03-04T09:41:12.972 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "end_weekday", 2024-03-04T09:41:12.972 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "uint", 2024-03-04T09:41:12.973 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:12.973 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:12.973 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "0", 2024-03-04T09:41:12.973 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "0", 2024-03-04T09:41:12.973 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "6", 2024-03-04T09:41:12.973 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:12.973 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-03-04T09:41:12.973 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-03-04T09:41:12.974 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:12.974 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:12.974 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:12.974 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-03-04T09:41:12.974 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-03-04T09:41:12.974 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:12.974 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:12.975 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:12.975 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-03-04T09:41:12.975 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:12.975 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:12.975 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-03-04T09:41:12.975 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-03-04T09:41:12.975 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-03-04T09:41:12.975 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-03-04T09:41:12.975 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-03-04T09:41:12.976 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-03-04T09:41:12.976 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-03-04T09:41:12.976 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-03-04T09:41:12.976 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:12.976 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:12.976 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:12.976 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:12.976 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:12.977 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-03-04T09:41:12.977 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-03-04T09:41:12.977 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:12.977 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:12.977 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:12.977 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-03-04T09:41:12.977 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:12.978 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:12.978 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:12.978 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:12.978 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:12.978 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:12.978 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:12.978 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:12.978 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-03-04T09:41:12.979 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-03-04T09:41:12.979 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:12.979 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:12.979 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:12.979 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-03-04T09:41:12.979 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:12.979 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:12.979 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-03-04T09:41:12.980 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-03-04T09:41:12.980 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-03-04T09:41:12.980 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-03-04T09:41:12.980 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-03-04T09:41:12.980 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-03-04T09:41:12.980 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-03-04T09:41:12.980 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-03-04T09:41:12.980 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:12.980 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:12.981 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:12.981 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:12.981 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:12.981 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-03-04T09:41:12.981 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-03-04T09:41:12.981 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:12.981 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:12.982 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:12.982 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-03-04T09:41:12.982 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:12.982 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:12.982 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:12.982 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:12.982 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:12.982 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:12.983 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:12.983 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:12.983 INFO:teuthology.orchestra.run.smithi052.stdout: "min_score": { 2024-03-04T09:41:12.983 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "min_score", 2024-03-04T09:41:12.983 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "float", 2024-03-04T09:41:12.983 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:12.983 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:12.983 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "0", 2024-03-04T09:41:12.983 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:12.984 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:12.984 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:12.984 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-03-04T09:41:12.984 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:12.984 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:12.984 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:12.984 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:12.984 INFO:teuthology.orchestra.run.smithi052.stdout: "mode": { 2024-03-04T09:41:12.985 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "mode", 2024-03-04T09:41:12.985 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:12.985 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:12.985 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:12.985 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "upmap", 2024-03-04T09:41:12.985 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:12.985 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:12.985 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-03-04T09:41:12.985 INFO:teuthology.orchestra.run.smithi052.stdout: "crush-compat", 2024-03-04T09:41:12.986 INFO:teuthology.orchestra.run.smithi052.stdout: "none", 2024-03-04T09:41:12.986 INFO:teuthology.orchestra.run.smithi052.stdout: "read", 2024-03-04T09:41:12.986 INFO:teuthology.orchestra.run.smithi052.stdout: "upmap", 2024-03-04T09:41:12.986 INFO:teuthology.orchestra.run.smithi052.stdout: "upmap-read" 2024-03-04T09:41:12.986 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-03-04T09:41:12.986 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Balancer mode", 2024-03-04T09:41:12.986 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:12.986 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:12.987 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:12.987 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:12.987 INFO:teuthology.orchestra.run.smithi052.stdout: "pool_ids": { 2024-03-04T09:41:12.987 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "pool_ids", 2024-03-04T09:41:12.987 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:12.987 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:12.987 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:12.987 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-03-04T09:41:12.988 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:12.988 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:12.988 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:12.988 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-03-04T09:41:12.988 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:12.988 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:12.988 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:12.988 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:12.988 INFO:teuthology.orchestra.run.smithi052.stdout: "sleep_interval": { 2024-03-04T09:41:12.989 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "sleep_interval", 2024-03-04T09:41:12.989 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "secs", 2024-03-04T09:41:12.989 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:12.989 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:12.989 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "60", 2024-03-04T09:41:12.989 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:12.989 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:12.989 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:12.990 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-03-04T09:41:12.990 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:12.990 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:12.990 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:12.990 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:12.990 INFO:teuthology.orchestra.run.smithi052.stdout: "upmap_max_deviation": { 2024-03-04T09:41:12.990 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "upmap_max_deviation", 2024-03-04T09:41:12.990 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-03-04T09:41:12.991 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:12.991 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:12.991 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "5", 2024-03-04T09:41:12.991 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "1", 2024-03-04T09:41:12.991 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:12.991 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:12.991 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "deviation below which no optimization is attempted", 2024-03-04T09:41:12.991 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-03-04T09:41:12.991 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:12.992 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:12.992 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:12.992 INFO:teuthology.orchestra.run.smithi052.stdout: "upmap_max_optimizations": { 2024-03-04T09:41:12.992 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "upmap_max_optimizations", 2024-03-04T09:41:12.992 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "uint", 2024-03-04T09:41:12.992 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:12.992 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:12.993 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "10", 2024-03-04T09:41:12.993 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:12.993 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:12.993 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:12.993 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-03-04T09:41:12.993 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:12.993 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:12.993 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:12.993 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-03-04T09:41:12.994 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-03-04T09:41:12.994 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:12.994 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-03-04T09:41:12.994 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "cephadm", 2024-03-04T09:41:12.994 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-03-04T09:41:12.994 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-03-04T09:41:12.994 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-03-04T09:41:12.994 INFO:teuthology.orchestra.run.smithi052.stdout: "agent_down_multiplier": { 2024-03-04T09:41:12.995 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "agent_down_multiplier", 2024-03-04T09:41:12.995 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "float", 2024-03-04T09:41:12.995 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:12.995 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:12.995 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "3.0", 2024-03-04T09:41:12.995 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:12.995 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:12.995 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:12.996 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-03-04T09:41:12.996 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:12.996 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:12.996 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:12.996 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:12.996 INFO:teuthology.orchestra.run.smithi052.stdout: "agent_refresh_rate": { 2024-03-04T09:41:12.996 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "agent_refresh_rate", 2024-03-04T09:41:12.997 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "secs", 2024-03-04T09:41:12.997 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:12.997 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:12.997 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "20", 2024-03-04T09:41:12.997 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:12.997 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:12.997 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:12.997 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-03-04T09:41:12.997 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:12.998 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:12.998 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:12.998 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:12.998 INFO:teuthology.orchestra.run.smithi052.stdout: "agent_starting_port": { 2024-03-04T09:41:12.998 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "agent_starting_port", 2024-03-04T09:41:12.998 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-03-04T09:41:12.998 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:12.998 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:12.999 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "4721", 2024-03-04T09:41:12.999 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:12.999 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:12.999 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:12.999 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-03-04T09:41:12.999 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.000 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.000 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.000 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.000 INFO:teuthology.orchestra.run.smithi052.stdout: "allow_ptrace": { 2024-03-04T09:41:13.000 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "allow_ptrace", 2024-03-04T09:41:13.000 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:13.000 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.000 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:13.001 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-03-04T09:41:13.001 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.001 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.001 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.001 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-03-04T09:41:13.001 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-03-04T09:41:13.001 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.002 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.002 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.002 INFO:teuthology.orchestra.run.smithi052.stdout: "autotune_interval": { 2024-03-04T09:41:13.002 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "autotune_interval", 2024-03-04T09:41:13.002 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "secs", 2024-03-04T09:41:13.002 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.002 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:13.002 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "600", 2024-03-04T09:41:13.003 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.003 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.003 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.003 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "how frequently to autotune daemon memory", 2024-03-04T09:41:13.003 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.003 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.003 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.003 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.004 INFO:teuthology.orchestra.run.smithi052.stdout: "autotune_memory_target_ratio": { 2024-03-04T09:41:13.004 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "autotune_memory_target_ratio", 2024-03-04T09:41:13.004 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "float", 2024-03-04T09:41:13.004 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.004 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:13.004 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "0.7", 2024-03-04T09:41:13.004 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.004 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.004 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.005 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-03-04T09:41:13.005 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.005 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.005 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.005 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.005 INFO:teuthology.orchestra.run.smithi052.stdout: "cephadm_log_destination": { 2024-03-04T09:41:13.005 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "cephadm_log_destination", 2024-03-04T09:41:13.006 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:13.006 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.006 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:13.006 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-03-04T09:41:13.006 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.006 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.006 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-03-04T09:41:13.006 INFO:teuthology.orchestra.run.smithi052.stdout: "file", 2024-03-04T09:41:13.007 INFO:teuthology.orchestra.run.smithi052.stdout: "file,syslog", 2024-03-04T09:41:13.007 INFO:teuthology.orchestra.run.smithi052.stdout: "syslog" 2024-03-04T09:41:13.007 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-03-04T09:41:13.007 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-03-04T09:41:13.007 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.007 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.007 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.008 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.008 INFO:teuthology.orchestra.run.smithi052.stdout: "cgroups_split": { 2024-03-04T09:41:13.008 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "cgroups_split", 2024-03-04T09:41:13.008 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:13.008 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.008 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:13.008 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-03-04T09:41:13.008 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.009 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.009 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.009 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-03-04T09:41:13.009 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.009 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.009 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.009 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.009 INFO:teuthology.orchestra.run.smithi052.stdout: "config_checks_enabled": { 2024-03-04T09:41:13.010 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "config_checks_enabled", 2024-03-04T09:41:13.010 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:13.010 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.010 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:13.010 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-03-04T09:41:13.010 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.010 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.010 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.011 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-03-04T09:41:13.011 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.011 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.011 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.011 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.011 INFO:teuthology.orchestra.run.smithi052.stdout: "config_dashboard": { 2024-03-04T09:41:13.011 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "config_dashboard", 2024-03-04T09:41:13.011 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:13.012 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.012 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:13.012 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-03-04T09:41:13.012 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.012 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.012 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.012 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-03-04T09:41:13.012 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.013 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.013 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.013 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.013 INFO:teuthology.orchestra.run.smithi052.stdout: "container_image_alertmanager": { 2024-03-04T09:41:13.013 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "container_image_alertmanager", 2024-03-04T09:41:13.013 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:13.013 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.013 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:13.014 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-03-04T09:41:13.014 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.014 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.014 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.014 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Prometheus container image", 2024-03-04T09:41:13.014 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.014 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.014 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.015 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.015 INFO:teuthology.orchestra.run.smithi052.stdout: "container_image_base": { 2024-03-04T09:41:13.015 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "container_image_base", 2024-03-04T09:41:13.015 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:13.015 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.015 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:13.015 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "quay.io/ceph/ceph", 2024-03-04T09:41:13.015 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.016 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.016 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.016 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Container image name, without the tag", 2024-03-04T09:41:13.016 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.016 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.016 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.016 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.016 INFO:teuthology.orchestra.run.smithi052.stdout: "container_image_elasticsearch": { 2024-03-04T09:41:13.017 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "container_image_elasticsearch", 2024-03-04T09:41:13.017 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:13.017 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.017 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:13.017 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-03-04T09:41:13.017 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.017 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.017 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.018 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "elasticsearch container image", 2024-03-04T09:41:13.018 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.018 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.018 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.018 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.018 INFO:teuthology.orchestra.run.smithi052.stdout: "container_image_grafana": { 2024-03-04T09:41:13.018 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "container_image_grafana", 2024-03-04T09:41:13.018 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:13.019 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.019 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:13.019 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "quay.io/ceph/ceph-grafana:9.4.12", 2024-03-04T09:41:13.019 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.019 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.019 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.019 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Prometheus container image", 2024-03-04T09:41:13.019 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.019 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.020 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.020 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.020 INFO:teuthology.orchestra.run.smithi052.stdout: "container_image_haproxy": { 2024-03-04T09:41:13.020 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "container_image_haproxy", 2024-03-04T09:41:13.020 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:13.020 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.020 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:13.020 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-03-04T09:41:13.021 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.021 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.021 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.021 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "HAproxy container image", 2024-03-04T09:41:13.021 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.021 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.021 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.022 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.022 INFO:teuthology.orchestra.run.smithi052.stdout: "container_image_jaeger_agent": { 2024-03-04T09:41:13.022 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "container_image_jaeger_agent", 2024-03-04T09:41:13.022 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:13.022 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.022 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:13.022 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-03-04T09:41:13.022 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.022 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.023 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.023 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Jaeger agent container image", 2024-03-04T09:41:13.023 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.023 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.023 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.023 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.023 INFO:teuthology.orchestra.run.smithi052.stdout: "container_image_jaeger_collector": { 2024-03-04T09:41:13.023 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "container_image_jaeger_collector", 2024-03-04T09:41:13.024 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:13.024 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.024 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:13.024 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-03-04T09:41:13.024 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.024 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.024 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.024 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Jaeger collector container image", 2024-03-04T09:41:13.024 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.025 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.025 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.025 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.025 INFO:teuthology.orchestra.run.smithi052.stdout: "container_image_jaeger_query": { 2024-03-04T09:41:13.025 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "container_image_jaeger_query", 2024-03-04T09:41:13.025 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:13.025 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.025 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:13.026 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-03-04T09:41:13.026 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.026 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.026 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.026 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Jaeger query container image", 2024-03-04T09:41:13.026 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.026 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.026 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.027 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.027 INFO:teuthology.orchestra.run.smithi052.stdout: "container_image_keepalived": { 2024-03-04T09:41:13.027 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "container_image_keepalived", 2024-03-04T09:41:13.027 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:13.027 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.027 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:13.027 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-03-04T09:41:13.027 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.028 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.028 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.028 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Keepalived container image", 2024-03-04T09:41:13.028 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.028 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.028 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.028 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.028 INFO:teuthology.orchestra.run.smithi052.stdout: "container_image_loki": { 2024-03-04T09:41:13.029 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "container_image_loki", 2024-03-04T09:41:13.029 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:13.029 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.029 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:13.029 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-03-04T09:41:13.029 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.029 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.029 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.030 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Loki container image", 2024-03-04T09:41:13.030 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.030 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.030 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.030 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.030 INFO:teuthology.orchestra.run.smithi052.stdout: "container_image_node_exporter": { 2024-03-04T09:41:13.030 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "container_image_node_exporter", 2024-03-04T09:41:13.030 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:13.030 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.031 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:13.031 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-03-04T09:41:13.031 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.031 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.031 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.031 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Prometheus container image", 2024-03-04T09:41:13.031 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.031 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.032 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.032 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.032 INFO:teuthology.orchestra.run.smithi052.stdout: "container_image_nvmeof": { 2024-03-04T09:41:13.032 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "container_image_nvmeof", 2024-03-04T09:41:13.032 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:13.032 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.032 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:13.032 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "quay.io/ceph/nvmeof:latest", 2024-03-04T09:41:13.033 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.033 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.033 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.033 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Nvme-of container image", 2024-03-04T09:41:13.033 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.033 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.033 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.033 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.034 INFO:teuthology.orchestra.run.smithi052.stdout: "container_image_prometheus": { 2024-03-04T09:41:13.034 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "container_image_prometheus", 2024-03-04T09:41:13.034 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:13.034 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.034 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:13.034 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-03-04T09:41:13.034 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.034 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.034 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.035 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Prometheus container image", 2024-03-04T09:41:13.035 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.035 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.035 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.035 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.035 INFO:teuthology.orchestra.run.smithi052.stdout: "container_image_promtail": { 2024-03-04T09:41:13.035 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "container_image_promtail", 2024-03-04T09:41:13.035 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:13.036 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.036 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:13.036 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-03-04T09:41:13.036 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.036 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.036 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.036 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Promtail container image", 2024-03-04T09:41:13.036 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.037 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.037 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.037 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.037 INFO:teuthology.orchestra.run.smithi052.stdout: "container_image_snmp_gateway": { 2024-03-04T09:41:13.037 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "container_image_snmp_gateway", 2024-03-04T09:41:13.037 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:13.037 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.037 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:13.037 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-03-04T09:41:13.038 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.038 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.038 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.038 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "SNMP Gateway container image", 2024-03-04T09:41:13.038 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.038 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.038 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.038 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.039 INFO:teuthology.orchestra.run.smithi052.stdout: "container_init": { 2024-03-04T09:41:13.039 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "container_init", 2024-03-04T09:41:13.039 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:13.039 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.039 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:13.039 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-03-04T09:41:13.039 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.039 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.040 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.040 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Run podman/docker with `--init`", 2024-03-04T09:41:13.040 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.040 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.040 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.040 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.041 INFO:teuthology.orchestra.run.smithi052.stdout: "daemon_cache_timeout": { 2024-03-04T09:41:13.041 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "daemon_cache_timeout", 2024-03-04T09:41:13.041 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "secs", 2024-03-04T09:41:13.041 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.041 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:13.041 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "600", 2024-03-04T09:41:13.041 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.041 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.042 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.042 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-03-04T09:41:13.042 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.042 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.042 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.042 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.042 INFO:teuthology.orchestra.run.smithi052.stdout: "default_cephadm_command_timeout": { 2024-03-04T09:41:13.042 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "default_cephadm_command_timeout", 2024-03-04T09:41:13.043 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-03-04T09:41:13.043 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.043 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:13.043 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "900", 2024-03-04T09:41:13.043 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.043 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.043 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.043 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-03-04T09:41:13.044 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.044 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.044 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.044 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.044 INFO:teuthology.orchestra.run.smithi052.stdout: "default_registry": { 2024-03-04T09:41:13.044 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "default_registry", 2024-03-04T09:41:13.044 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:13.045 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.045 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:13.045 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "docker.io", 2024-03-04T09:41:13.045 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.045 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.045 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.045 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-03-04T09:41:13.045 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.046 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.046 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.046 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.046 INFO:teuthology.orchestra.run.smithi052.stdout: "device_cache_timeout": { 2024-03-04T09:41:13.046 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "device_cache_timeout", 2024-03-04T09:41:13.046 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "secs", 2024-03-04T09:41:13.046 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.046 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:13.046 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "1800", 2024-03-04T09:41:13.047 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.047 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.047 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.047 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "seconds to cache device inventory", 2024-03-04T09:41:13.047 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.047 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.047 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.047 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.048 INFO:teuthology.orchestra.run.smithi052.stdout: "device_enhanced_scan": { 2024-03-04T09:41:13.048 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "device_enhanced_scan", 2024-03-04T09:41:13.048 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:13.048 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.048 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:13.048 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-03-04T09:41:13.048 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.048 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.049 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.049 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Use libstoragemgmt during device scans", 2024-03-04T09:41:13.049 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.049 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.049 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.049 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.049 INFO:teuthology.orchestra.run.smithi052.stdout: "facts_cache_timeout": { 2024-03-04T09:41:13.049 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "facts_cache_timeout", 2024-03-04T09:41:13.050 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "secs", 2024-03-04T09:41:13.050 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.050 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:13.050 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "60", 2024-03-04T09:41:13.050 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.050 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.050 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.050 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "seconds to cache host facts data", 2024-03-04T09:41:13.051 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.051 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.051 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.051 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.051 INFO:teuthology.orchestra.run.smithi052.stdout: "host_check_interval": { 2024-03-04T09:41:13.051 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "host_check_interval", 2024-03-04T09:41:13.051 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "secs", 2024-03-04T09:41:13.051 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.052 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:13.052 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "600", 2024-03-04T09:41:13.052 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.052 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.052 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.052 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "how frequently to perform a host check", 2024-03-04T09:41:13.052 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.052 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.053 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.053 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.053 INFO:teuthology.orchestra.run.smithi052.stdout: "hw_monitoring": { 2024-03-04T09:41:13.053 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "hw_monitoring", 2024-03-04T09:41:13.053 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:13.053 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.053 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:13.053 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-03-04T09:41:13.054 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.054 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.054 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.054 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-03-04T09:41:13.054 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.054 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.054 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.054 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.054 INFO:teuthology.orchestra.run.smithi052.stdout: "inventory_list_all": { 2024-03-04T09:41:13.055 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "inventory_list_all", 2024-03-04T09:41:13.055 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:13.055 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.055 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:13.055 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-03-04T09:41:13.055 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.055 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.055 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.056 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-03-04T09:41:13.056 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.056 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.056 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.056 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.056 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-03-04T09:41:13.056 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-03-04T09:41:13.057 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:13.057 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.057 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:13.057 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-03-04T09:41:13.057 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.057 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.057 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-03-04T09:41:13.057 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-03-04T09:41:13.058 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-03-04T09:41:13.058 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-03-04T09:41:13.058 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-03-04T09:41:13.058 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-03-04T09:41:13.058 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-03-04T09:41:13.058 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-03-04T09:41:13.058 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:13.058 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.058 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.059 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.059 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.059 INFO:teuthology.orchestra.run.smithi052.stdout: "log_refresh_metadata": { 2024-03-04T09:41:13.059 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_refresh_metadata", 2024-03-04T09:41:13.059 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:13.059 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.059 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:13.059 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-03-04T09:41:13.060 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.060 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.060 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.060 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-03-04T09:41:13.060 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.060 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.060 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.060 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.061 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-03-04T09:41:13.061 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-03-04T09:41:13.061 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:13.061 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.061 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:13.061 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-03-04T09:41:13.061 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.061 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.061 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.062 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-03-04T09:41:13.062 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.062 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.062 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.062 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.062 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-03-04T09:41:13.062 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-03-04T09:41:13.062 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:13.063 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.063 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:13.063 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-03-04T09:41:13.063 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.063 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.063 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-03-04T09:41:13.063 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-03-04T09:41:13.064 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-03-04T09:41:13.064 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-03-04T09:41:13.064 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-03-04T09:41:13.064 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-03-04T09:41:13.064 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-03-04T09:41:13.064 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-03-04T09:41:13.064 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:13.064 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.065 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.065 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.065 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.065 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-03-04T09:41:13.065 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-03-04T09:41:13.065 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:13.065 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.065 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:13.065 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-03-04T09:41:13.066 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.066 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.066 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.066 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:13.066 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.066 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.066 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.066 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.067 INFO:teuthology.orchestra.run.smithi052.stdout: "manage_etc_ceph_ceph_conf": { 2024-03-04T09:41:13.067 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-03-04T09:41:13.067 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:13.067 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.067 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:13.067 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-03-04T09:41:13.067 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.067 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.068 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.068 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-03-04T09:41:13.068 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.068 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.068 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.068 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.068 INFO:teuthology.orchestra.run.smithi052.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-03-04T09:41:13.068 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-03-04T09:41:13.068 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:13.069 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.069 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:13.069 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "*", 2024-03-04T09:41:13.069 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.069 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.069 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.069 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-03-04T09:41:13.069 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.070 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.070 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.070 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.070 INFO:teuthology.orchestra.run.smithi052.stdout: "max_count_per_host": { 2024-03-04T09:41:13.070 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "max_count_per_host", 2024-03-04T09:41:13.070 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-03-04T09:41:13.070 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.070 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:13.071 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "10", 2024-03-04T09:41:13.071 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.071 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.071 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.071 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "max number of daemons per service per host", 2024-03-04T09:41:13.071 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.071 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.071 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.071 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.072 INFO:teuthology.orchestra.run.smithi052.stdout: "max_osd_draining_count": { 2024-03-04T09:41:13.072 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "max_osd_draining_count", 2024-03-04T09:41:13.072 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-03-04T09:41:13.072 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.072 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:13.072 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "10", 2024-03-04T09:41:13.072 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.072 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.073 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.073 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-03-04T09:41:13.073 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.073 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.073 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.073 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.073 INFO:teuthology.orchestra.run.smithi052.stdout: "migration_current": { 2024-03-04T09:41:13.073 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "migration_current", 2024-03-04T09:41:13.074 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-03-04T09:41:13.074 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.074 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:13.074 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-03-04T09:41:13.074 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.074 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.074 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.074 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "internal - do not modify", 2024-03-04T09:41:13.074 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.075 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.075 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.075 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.075 INFO:teuthology.orchestra.run.smithi052.stdout: "mode": { 2024-03-04T09:41:13.075 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "mode", 2024-03-04T09:41:13.075 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:13.075 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.075 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:13.076 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "root", 2024-03-04T09:41:13.076 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.076 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.076 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-03-04T09:41:13.076 INFO:teuthology.orchestra.run.smithi052.stdout: "cephadm-package", 2024-03-04T09:41:13.076 INFO:teuthology.orchestra.run.smithi052.stdout: "root" 2024-03-04T09:41:13.076 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-03-04T09:41:13.076 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "mode for remote execution of cephadm", 2024-03-04T09:41:13.076 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.077 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.077 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.077 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.077 INFO:teuthology.orchestra.run.smithi052.stdout: "oob_default_addr": { 2024-03-04T09:41:13.077 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "oob_default_addr", 2024-03-04T09:41:13.077 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:13.077 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.077 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:13.078 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "169.254.1.1", 2024-03-04T09:41:13.078 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.078 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.078 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.078 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Default address for RedFish API (oob management).", 2024-03-04T09:41:13.078 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.078 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.078 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.079 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.079 INFO:teuthology.orchestra.run.smithi052.stdout: "prometheus_alerts_path": { 2024-03-04T09:41:13.079 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "prometheus_alerts_path", 2024-03-04T09:41:13.079 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:13.079 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.079 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:13.079 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-03-04T09:41:13.079 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.080 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.080 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.080 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-03-04T09:41:13.080 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.080 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.080 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.080 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.080 INFO:teuthology.orchestra.run.smithi052.stdout: "registry_insecure": { 2024-03-04T09:41:13.080 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "registry_insecure", 2024-03-04T09:41:13.081 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:13.081 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.081 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:13.081 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-03-04T09:41:13.081 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.081 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.081 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.081 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-03-04T09:41:13.082 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.082 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.082 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.082 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.082 INFO:teuthology.orchestra.run.smithi052.stdout: "registry_password": { 2024-03-04T09:41:13.082 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "registry_password", 2024-03-04T09:41:13.082 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:13.082 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.083 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:13.083 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-03-04T09:41:13.083 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.083 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.083 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.083 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-03-04T09:41:13.083 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.083 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.084 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.084 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.084 INFO:teuthology.orchestra.run.smithi052.stdout: "registry_url": { 2024-03-04T09:41:13.084 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "registry_url", 2024-03-04T09:41:13.084 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:13.084 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.084 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:13.085 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-03-04T09:41:13.085 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.085 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.085 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.085 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-03-04T09:41:13.085 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.085 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.085 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.086 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.086 INFO:teuthology.orchestra.run.smithi052.stdout: "registry_username": { 2024-03-04T09:41:13.086 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "registry_username", 2024-03-04T09:41:13.086 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:13.086 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.086 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:13.086 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-03-04T09:41:13.086 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.086 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.087 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.087 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-03-04T09:41:13.087 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.087 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.087 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.087 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.087 INFO:teuthology.orchestra.run.smithi052.stdout: "secure_monitoring_stack": { 2024-03-04T09:41:13.087 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "secure_monitoring_stack", 2024-03-04T09:41:13.088 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:13.088 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.088 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:13.088 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-03-04T09:41:13.088 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.088 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.088 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.088 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-03-04T09:41:13.089 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.089 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.089 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.089 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.089 INFO:teuthology.orchestra.run.smithi052.stdout: "service_discovery_port": { 2024-03-04T09:41:13.089 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "service_discovery_port", 2024-03-04T09:41:13.089 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-03-04T09:41:13.089 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.090 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:13.090 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "8765", 2024-03-04T09:41:13.090 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.090 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.090 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.090 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "cephadm service discovery port", 2024-03-04T09:41:13.090 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.090 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.090 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.091 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.091 INFO:teuthology.orchestra.run.smithi052.stdout: "ssh_config_file": { 2024-03-04T09:41:13.091 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "ssh_config_file", 2024-03-04T09:41:13.091 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:13.091 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.091 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:13.091 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-03-04T09:41:13.091 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.092 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.092 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.092 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-03-04T09:41:13.092 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.092 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.092 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.092 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.092 INFO:teuthology.orchestra.run.smithi052.stdout: "use_agent": { 2024-03-04T09:41:13.092 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "use_agent", 2024-03-04T09:41:13.093 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:13.093 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.093 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:13.093 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-03-04T09:41:13.093 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.093 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.093 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.093 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-03-04T09:41:13.094 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.094 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.094 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.094 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.094 INFO:teuthology.orchestra.run.smithi052.stdout: "use_repo_digest": { 2024-03-04T09:41:13.094 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "use_repo_digest", 2024-03-04T09:41:13.094 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:13.094 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.094 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:13.095 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-03-04T09:41:13.095 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.095 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.095 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.095 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-03-04T09:41:13.095 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.095 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.095 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.096 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.096 INFO:teuthology.orchestra.run.smithi052.stdout: "warn_on_failed_host_check": { 2024-03-04T09:41:13.096 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "warn_on_failed_host_check", 2024-03-04T09:41:13.096 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:13.096 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.096 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:13.096 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-03-04T09:41:13.096 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.097 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.097 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.097 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "raise a health warning if the host check fails", 2024-03-04T09:41:13.097 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.097 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.097 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.097 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.097 INFO:teuthology.orchestra.run.smithi052.stdout: "warn_on_stray_daemons": { 2024-03-04T09:41:13.097 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "warn_on_stray_daemons", 2024-03-04T09:41:13.098 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:13.098 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.098 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:13.098 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-03-04T09:41:13.098 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.098 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.098 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.098 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-03-04T09:41:13.099 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.099 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.099 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.099 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.099 INFO:teuthology.orchestra.run.smithi052.stdout: "warn_on_stray_hosts": { 2024-03-04T09:41:13.099 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "warn_on_stray_hosts", 2024-03-04T09:41:13.099 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:13.099 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.099 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:13.100 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-03-04T09:41:13.100 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.100 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.100 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.100 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-03-04T09:41:13.100 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.100 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.100 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.101 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-03-04T09:41:13.101 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-03-04T09:41:13.101 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.101 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-03-04T09:41:13.101 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "crash", 2024-03-04T09:41:13.101 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-03-04T09:41:13.101 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-03-04T09:41:13.101 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-03-04T09:41:13.101 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-03-04T09:41:13.102 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-03-04T09:41:13.102 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:13.102 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.102 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:13.102 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-03-04T09:41:13.102 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.102 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.102 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-03-04T09:41:13.103 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-03-04T09:41:13.103 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-03-04T09:41:13.103 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-03-04T09:41:13.103 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-03-04T09:41:13.103 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-03-04T09:41:13.103 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-03-04T09:41:13.103 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-03-04T09:41:13.103 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:13.103 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.104 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.104 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.104 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.104 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-03-04T09:41:13.104 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-03-04T09:41:13.104 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:13.104 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.104 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:13.105 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-03-04T09:41:13.105 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.105 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.105 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.105 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:13.105 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.105 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.106 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.106 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.106 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-03-04T09:41:13.106 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-03-04T09:41:13.106 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:13.106 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.106 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:13.106 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-03-04T09:41:13.106 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.107 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.107 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-03-04T09:41:13.107 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-03-04T09:41:13.107 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-03-04T09:41:13.107 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-03-04T09:41:13.107 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-03-04T09:41:13.107 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-03-04T09:41:13.107 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-03-04T09:41:13.108 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-03-04T09:41:13.108 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:13.108 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.108 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.108 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.108 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.108 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-03-04T09:41:13.108 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-03-04T09:41:13.108 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:13.109 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.109 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:13.109 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-03-04T09:41:13.109 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.109 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.109 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.109 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:13.109 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.110 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.110 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.110 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.110 INFO:teuthology.orchestra.run.smithi052.stdout: "retain_interval": { 2024-03-04T09:41:13.110 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "retain_interval", 2024-03-04T09:41:13.110 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "secs", 2024-03-04T09:41:13.110 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.110 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:13.110 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "31536000", 2024-03-04T09:41:13.111 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.111 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.111 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.111 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "how long to retain crashes before pruning them", 2024-03-04T09:41:13.111 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.111 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.111 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.111 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.112 INFO:teuthology.orchestra.run.smithi052.stdout: "warn_recent_interval": { 2024-03-04T09:41:13.112 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "warn_recent_interval", 2024-03-04T09:41:13.112 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "secs", 2024-03-04T09:41:13.112 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.112 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:13.112 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "1209600", 2024-03-04T09:41:13.112 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.112 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.113 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.113 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "time interval in which to warn about recent crashes", 2024-03-04T09:41:13.113 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.113 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.113 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.113 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-03-04T09:41:13.113 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-03-04T09:41:13.113 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.113 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-03-04T09:41:13.114 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "dashboard", 2024-03-04T09:41:13.114 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-03-04T09:41:13.114 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-03-04T09:41:13.114 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-03-04T09:41:13.114 INFO:teuthology.orchestra.run.smithi052.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-03-04T09:41:13.114 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-03-04T09:41:13.114 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-03-04T09:41:13.114 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.115 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:13.115 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "10", 2024-03-04T09:41:13.115 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.115 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.115 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.115 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:13.115 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.115 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.115 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.116 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.116 INFO:teuthology.orchestra.run.smithi052.stdout: "ALERTMANAGER_API_HOST": { 2024-03-04T09:41:13.116 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "ALERTMANAGER_API_HOST", 2024-03-04T09:41:13.116 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:13.116 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.116 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:13.116 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-03-04T09:41:13.117 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.117 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.117 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.117 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:13.117 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.117 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.117 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.117 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.118 INFO:teuthology.orchestra.run.smithi052.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-03-04T09:41:13.118 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-03-04T09:41:13.118 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:13.118 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.118 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:13.118 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-03-04T09:41:13.118 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.118 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.119 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.119 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:13.119 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.119 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.119 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.119 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.119 INFO:teuthology.orchestra.run.smithi052.stdout: "AUDIT_API_ENABLED": { 2024-03-04T09:41:13.119 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "AUDIT_API_ENABLED", 2024-03-04T09:41:13.119 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:13.120 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.120 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:13.120 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-03-04T09:41:13.120 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.120 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.120 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.120 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:13.121 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.121 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.121 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.121 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.121 INFO:teuthology.orchestra.run.smithi052.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-03-04T09:41:13.121 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-03-04T09:41:13.121 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:13.121 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.121 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:13.122 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-03-04T09:41:13.122 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.122 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.122 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.122 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:13.122 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.122 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.123 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.123 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.123 INFO:teuthology.orchestra.run.smithi052.stdout: "ENABLE_BROWSABLE_API": { 2024-03-04T09:41:13.123 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "ENABLE_BROWSABLE_API", 2024-03-04T09:41:13.123 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:13.123 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.123 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:13.123 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-03-04T09:41:13.123 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.124 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.124 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.124 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:13.124 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.124 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.124 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.124 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.124 INFO:teuthology.orchestra.run.smithi052.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-03-04T09:41:13.125 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-03-04T09:41:13.125 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:13.125 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.125 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:13.125 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-03-04T09:41:13.125 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.125 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.125 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.125 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:13.126 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.126 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.126 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.126 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.126 INFO:teuthology.orchestra.run.smithi052.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-03-04T09:41:13.126 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-03-04T09:41:13.126 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:13.126 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.127 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:13.127 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-03-04T09:41:13.127 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.127 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.127 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.127 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:13.127 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.127 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.127 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.128 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.128 INFO:teuthology.orchestra.run.smithi052.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-03-04T09:41:13.128 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-03-04T09:41:13.128 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:13.128 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.128 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:13.128 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-03-04T09:41:13.129 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.129 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.129 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.129 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:13.129 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.129 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.129 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.129 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.129 INFO:teuthology.orchestra.run.smithi052.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-03-04T09:41:13.130 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-03-04T09:41:13.130 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:13.130 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.130 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:13.130 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-03-04T09:41:13.130 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.130 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.130 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.131 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:13.131 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.131 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.131 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.131 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.131 INFO:teuthology.orchestra.run.smithi052.stdout: "FEATURE_TOGGLE_NFS": { 2024-03-04T09:41:13.131 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-03-04T09:41:13.131 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:13.132 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.132 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:13.132 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-03-04T09:41:13.132 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.132 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.132 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.132 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:13.132 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.133 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.133 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.133 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.133 INFO:teuthology.orchestra.run.smithi052.stdout: "FEATURE_TOGGLE_RBD": { 2024-03-04T09:41:13.133 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-03-04T09:41:13.133 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:13.133 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.133 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:13.133 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-03-04T09:41:13.134 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.134 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.134 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.134 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:13.134 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.134 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.134 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.134 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.135 INFO:teuthology.orchestra.run.smithi052.stdout: "FEATURE_TOGGLE_RGW": { 2024-03-04T09:41:13.135 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-03-04T09:41:13.135 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:13.135 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.135 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:13.135 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-03-04T09:41:13.135 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.135 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.135 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.136 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:13.136 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.136 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.136 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.136 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.136 INFO:teuthology.orchestra.run.smithi052.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-03-04T09:41:13.136 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-03-04T09:41:13.136 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:13.137 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.137 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:13.137 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-03-04T09:41:13.137 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.137 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.137 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.137 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:13.137 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.137 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.138 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.138 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.138 INFO:teuthology.orchestra.run.smithi052.stdout: "GRAFANA_API_PASSWORD": { 2024-03-04T09:41:13.138 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "GRAFANA_API_PASSWORD", 2024-03-04T09:41:13.138 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:13.138 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.138 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:13.138 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "admin", 2024-03-04T09:41:13.139 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.139 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.139 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.139 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:13.139 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.139 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.139 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.139 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.140 INFO:teuthology.orchestra.run.smithi052.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-03-04T09:41:13.140 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-03-04T09:41:13.140 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:13.140 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.140 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:13.140 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-03-04T09:41:13.140 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.140 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.141 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.141 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:13.141 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.141 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.141 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.141 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.141 INFO:teuthology.orchestra.run.smithi052.stdout: "GRAFANA_API_URL": { 2024-03-04T09:41:13.141 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "GRAFANA_API_URL", 2024-03-04T09:41:13.141 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:13.142 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.142 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:13.142 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-03-04T09:41:13.142 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.142 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.142 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.142 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:13.142 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.143 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.143 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.143 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.143 INFO:teuthology.orchestra.run.smithi052.stdout: "GRAFANA_API_USERNAME": { 2024-03-04T09:41:13.143 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "GRAFANA_API_USERNAME", 2024-03-04T09:41:13.143 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:13.143 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.143 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:13.144 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "admin", 2024-03-04T09:41:13.144 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.144 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.144 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.144 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:13.144 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.144 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.144 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.145 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.145 INFO:teuthology.orchestra.run.smithi052.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-03-04T09:41:13.145 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-03-04T09:41:13.145 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:13.145 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.145 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:13.145 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-03-04T09:41:13.146 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.146 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.146 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.146 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:13.146 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.146 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.146 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.146 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.147 INFO:teuthology.orchestra.run.smithi052.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-03-04T09:41:13.147 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-03-04T09:41:13.147 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:13.147 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.147 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:13.147 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-03-04T09:41:13.147 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.147 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.148 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.148 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:13.148 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.148 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.148 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.148 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.148 INFO:teuthology.orchestra.run.smithi052.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-03-04T09:41:13.148 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-03-04T09:41:13.149 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:13.149 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.149 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:13.149 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-03-04T09:41:13.149 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.149 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.149 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.150 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:13.150 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.150 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.150 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.150 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.150 INFO:teuthology.orchestra.run.smithi052.stdout: "ISSUE_TRACKER_API_KEY": { 2024-03-04T09:41:13.150 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-03-04T09:41:13.150 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:13.151 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.151 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:13.151 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-03-04T09:41:13.151 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.151 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.151 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.151 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:13.151 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.152 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.152 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.152 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.152 INFO:teuthology.orchestra.run.smithi052.stdout: "PROMETHEUS_API_HOST": { 2024-03-04T09:41:13.152 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "PROMETHEUS_API_HOST", 2024-03-04T09:41:13.152 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:13.152 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.152 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:13.153 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-03-04T09:41:13.153 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.153 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.153 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.153 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:13.153 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.153 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.153 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.154 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.154 INFO:teuthology.orchestra.run.smithi052.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-03-04T09:41:13.154 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-03-04T09:41:13.154 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:13.154 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.154 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:13.154 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-03-04T09:41:13.154 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.154 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.155 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.155 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:13.155 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.155 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.155 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.155 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.155 INFO:teuthology.orchestra.run.smithi052.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-03-04T09:41:13.155 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-03-04T09:41:13.156 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:13.156 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.156 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:13.156 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-03-04T09:41:13.156 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.156 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.156 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.156 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:13.156 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.157 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.157 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.157 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.157 INFO:teuthology.orchestra.run.smithi052.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-03-04T09:41:13.157 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-03-04T09:41:13.157 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:13.157 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.157 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:13.158 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-03-04T09:41:13.158 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.158 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.158 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.158 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:13.158 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.158 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.158 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.159 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.159 INFO:teuthology.orchestra.run.smithi052.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-03-04T09:41:13.159 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-03-04T09:41:13.159 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:13.159 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.159 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:13.159 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-03-04T09:41:13.159 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.160 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.160 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.160 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:13.160 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.160 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.160 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.160 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.160 INFO:teuthology.orchestra.run.smithi052.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-03-04T09:41:13.160 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-03-04T09:41:13.161 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:13.161 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.161 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:13.161 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-03-04T09:41:13.161 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.161 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.161 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.161 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:13.162 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.162 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.162 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.162 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.162 INFO:teuthology.orchestra.run.smithi052.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-03-04T09:41:13.162 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-03-04T09:41:13.162 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:13.162 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.162 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:13.163 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-03-04T09:41:13.163 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.163 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.163 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.163 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:13.163 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.163 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.163 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.164 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.164 INFO:teuthology.orchestra.run.smithi052.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-03-04T09:41:13.164 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-03-04T09:41:13.164 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:13.164 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.164 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:13.164 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-03-04T09:41:13.164 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.165 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.165 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.165 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:13.165 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.165 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.165 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.165 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.165 INFO:teuthology.orchestra.run.smithi052.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-03-04T09:41:13.165 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-03-04T09:41:13.166 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:13.166 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.166 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:13.166 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-03-04T09:41:13.166 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.166 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.166 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.167 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:13.167 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.167 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.167 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.167 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.167 INFO:teuthology.orchestra.run.smithi052.stdout: "PWD_POLICY_ENABLED": { 2024-03-04T09:41:13.167 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "PWD_POLICY_ENABLED", 2024-03-04T09:41:13.167 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:13.167 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.168 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:13.168 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-03-04T09:41:13.168 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.168 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.168 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.168 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:13.168 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.168 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.169 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.169 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.169 INFO:teuthology.orchestra.run.smithi052.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-03-04T09:41:13.169 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-03-04T09:41:13.169 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:13.169 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.169 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:13.169 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-03-04T09:41:13.170 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.170 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.170 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.170 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:13.170 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.170 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.170 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.170 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.171 INFO:teuthology.orchestra.run.smithi052.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-03-04T09:41:13.171 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-03-04T09:41:13.171 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-03-04T09:41:13.171 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.171 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:13.171 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "10", 2024-03-04T09:41:13.171 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.171 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.172 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.172 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:13.172 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.172 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.172 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.172 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.172 INFO:teuthology.orchestra.run.smithi052.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-03-04T09:41:13.172 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-03-04T09:41:13.173 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-03-04T09:41:13.173 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.173 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:13.173 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "8", 2024-03-04T09:41:13.173 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.173 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.173 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.173 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:13.174 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.174 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.174 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.174 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.174 INFO:teuthology.orchestra.run.smithi052.stdout: "REST_REQUESTS_TIMEOUT": { 2024-03-04T09:41:13.174 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-03-04T09:41:13.174 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-03-04T09:41:13.174 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.174 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:13.175 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "45", 2024-03-04T09:41:13.175 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.175 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.175 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.175 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:13.175 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.175 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.175 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.176 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.176 INFO:teuthology.orchestra.run.smithi052.stdout: "RGW_API_ACCESS_KEY": { 2024-03-04T09:41:13.176 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "RGW_API_ACCESS_KEY", 2024-03-04T09:41:13.176 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:13.176 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.176 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:13.176 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-03-04T09:41:13.176 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.177 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.177 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.177 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:13.177 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.177 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.177 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.177 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.177 INFO:teuthology.orchestra.run.smithi052.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-03-04T09:41:13.177 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-03-04T09:41:13.178 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:13.178 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.178 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:13.178 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "admin", 2024-03-04T09:41:13.178 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.178 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.178 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.178 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:13.179 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.179 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.179 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.179 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.179 INFO:teuthology.orchestra.run.smithi052.stdout: "RGW_API_SECRET_KEY": { 2024-03-04T09:41:13.179 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "RGW_API_SECRET_KEY", 2024-03-04T09:41:13.179 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:13.179 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.180 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:13.180 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-03-04T09:41:13.180 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.180 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.180 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.180 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:13.180 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.180 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.180 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.181 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.181 INFO:teuthology.orchestra.run.smithi052.stdout: "RGW_API_SSL_VERIFY": { 2024-03-04T09:41:13.181 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "RGW_API_SSL_VERIFY", 2024-03-04T09:41:13.181 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:13.181 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.181 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:13.181 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-03-04T09:41:13.181 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.182 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.182 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.182 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:13.182 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.182 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.182 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.182 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.182 INFO:teuthology.orchestra.run.smithi052.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-03-04T09:41:13.183 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-03-04T09:41:13.183 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-03-04T09:41:13.183 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.183 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:13.183 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "0", 2024-03-04T09:41:13.183 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.183 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.183 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.184 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:13.184 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.184 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.184 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.184 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.184 INFO:teuthology.orchestra.run.smithi052.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-03-04T09:41:13.184 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-03-04T09:41:13.184 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-03-04T09:41:13.185 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.185 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:13.185 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "10", 2024-03-04T09:41:13.185 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.185 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.185 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.185 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:13.185 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.185 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.186 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.186 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.186 INFO:teuthology.orchestra.run.smithi052.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-03-04T09:41:13.186 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-03-04T09:41:13.186 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-03-04T09:41:13.186 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.186 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:13.186 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "5", 2024-03-04T09:41:13.187 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.187 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.187 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.187 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:13.187 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.187 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.187 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.187 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.187 INFO:teuthology.orchestra.run.smithi052.stdout: "cross_origin_url": { 2024-03-04T09:41:13.188 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "cross_origin_url", 2024-03-04T09:41:13.188 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:13.188 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.188 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:13.188 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-03-04T09:41:13.188 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.188 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.189 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.189 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:13.189 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.189 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.189 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.189 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.189 INFO:teuthology.orchestra.run.smithi052.stdout: "crt_file": { 2024-03-04T09:41:13.189 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "crt_file", 2024-03-04T09:41:13.189 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:13.190 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.190 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:13.190 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-03-04T09:41:13.190 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.190 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.190 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.190 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:13.190 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.191 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.191 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.191 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.191 INFO:teuthology.orchestra.run.smithi052.stdout: "debug": { 2024-03-04T09:41:13.191 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "debug", 2024-03-04T09:41:13.191 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:13.191 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.191 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:13.191 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-03-04T09:41:13.192 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.192 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.192 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.192 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Enable/disable debug options", 2024-03-04T09:41:13.192 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.192 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.192 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.193 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.193 INFO:teuthology.orchestra.run.smithi052.stdout: "jwt_token_ttl": { 2024-03-04T09:41:13.193 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "jwt_token_ttl", 2024-03-04T09:41:13.193 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-03-04T09:41:13.193 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.193 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:13.193 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "28800", 2024-03-04T09:41:13.193 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.194 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.194 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.194 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:13.194 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.194 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.194 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.194 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.194 INFO:teuthology.orchestra.run.smithi052.stdout: "key_file": { 2024-03-04T09:41:13.194 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "key_file", 2024-03-04T09:41:13.195 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:13.195 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.195 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:13.195 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-03-04T09:41:13.195 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.195 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.195 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.196 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:13.196 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.196 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.196 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.196 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.196 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-03-04T09:41:13.196 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-03-04T09:41:13.196 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:13.197 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.197 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:13.197 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-03-04T09:41:13.197 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.197 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.197 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-03-04T09:41:13.197 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-03-04T09:41:13.197 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-03-04T09:41:13.198 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-03-04T09:41:13.198 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-03-04T09:41:13.198 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-03-04T09:41:13.198 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-03-04T09:41:13.198 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-03-04T09:41:13.198 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:13.198 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.198 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.199 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.199 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.199 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-03-04T09:41:13.199 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-03-04T09:41:13.199 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:13.199 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.199 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:13.199 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-03-04T09:41:13.200 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.200 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.200 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.200 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:13.200 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.200 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.200 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.200 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.201 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-03-04T09:41:13.201 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-03-04T09:41:13.201 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:13.201 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.201 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:13.201 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-03-04T09:41:13.201 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.201 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.202 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-03-04T09:41:13.202 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-03-04T09:41:13.202 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-03-04T09:41:13.202 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-03-04T09:41:13.202 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-03-04T09:41:13.202 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-03-04T09:41:13.202 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-03-04T09:41:13.202 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-03-04T09:41:13.203 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:13.203 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.203 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.203 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.203 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.203 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-03-04T09:41:13.203 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-03-04T09:41:13.203 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:13.204 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.204 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:13.204 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-03-04T09:41:13.204 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.204 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.204 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.204 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:13.204 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.205 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.205 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.205 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.205 INFO:teuthology.orchestra.run.smithi052.stdout: "motd": { 2024-03-04T09:41:13.205 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "motd", 2024-03-04T09:41:13.205 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:13.205 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.205 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:13.206 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-03-04T09:41:13.206 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.206 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.206 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.206 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "The message of the day", 2024-03-04T09:41:13.206 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.206 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.206 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.207 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.207 INFO:teuthology.orchestra.run.smithi052.stdout: "redirect_resolve_ip_addr": { 2024-03-04T09:41:13.207 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "redirect_resolve_ip_addr", 2024-03-04T09:41:13.207 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:13.207 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.207 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:13.207 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-03-04T09:41:13.207 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.208 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.208 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.208 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:13.208 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.208 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.208 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.208 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.208 INFO:teuthology.orchestra.run.smithi052.stdout: "server_addr": { 2024-03-04T09:41:13.209 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "server_addr", 2024-03-04T09:41:13.209 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:13.209 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.209 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:13.209 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "::", 2024-03-04T09:41:13.209 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.209 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.209 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.210 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:13.210 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.210 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.210 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.210 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.210 INFO:teuthology.orchestra.run.smithi052.stdout: "server_port": { 2024-03-04T09:41:13.210 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "server_port", 2024-03-04T09:41:13.210 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-03-04T09:41:13.211 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.211 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:13.211 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "8080", 2024-03-04T09:41:13.211 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.211 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.211 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.211 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:13.211 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.212 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.212 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.212 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.212 INFO:teuthology.orchestra.run.smithi052.stdout: "ssl": { 2024-03-04T09:41:13.212 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "ssl", 2024-03-04T09:41:13.212 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:13.212 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.212 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:13.213 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-03-04T09:41:13.213 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.213 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.213 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.213 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:13.213 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.213 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.213 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.214 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.214 INFO:teuthology.orchestra.run.smithi052.stdout: "ssl_server_port": { 2024-03-04T09:41:13.214 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "ssl_server_port", 2024-03-04T09:41:13.214 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-03-04T09:41:13.214 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.214 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:13.214 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "8443", 2024-03-04T09:41:13.214 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.214 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.215 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.215 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:13.215 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.215 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.215 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.215 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.215 INFO:teuthology.orchestra.run.smithi052.stdout: "standby_behaviour": { 2024-03-04T09:41:13.215 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "standby_behaviour", 2024-03-04T09:41:13.216 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:13.216 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.216 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:13.216 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "redirect", 2024-03-04T09:41:13.216 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.216 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.216 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-03-04T09:41:13.216 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-03-04T09:41:13.217 INFO:teuthology.orchestra.run.smithi052.stdout: "redirect" 2024-03-04T09:41:13.217 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-03-04T09:41:13.217 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:13.217 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.217 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.217 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.217 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.217 INFO:teuthology.orchestra.run.smithi052.stdout: "standby_error_status_code": { 2024-03-04T09:41:13.217 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "standby_error_status_code", 2024-03-04T09:41:13.218 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-03-04T09:41:13.218 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.218 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:13.218 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "500", 2024-03-04T09:41:13.218 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "400", 2024-03-04T09:41:13.218 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "599", 2024-03-04T09:41:13.218 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.218 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:13.218 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.219 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.219 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.219 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.219 INFO:teuthology.orchestra.run.smithi052.stdout: "url_prefix": { 2024-03-04T09:41:13.219 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "url_prefix", 2024-03-04T09:41:13.219 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:13.219 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.220 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:13.220 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-03-04T09:41:13.220 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.220 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.220 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.220 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:13.220 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.221 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.221 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.221 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-03-04T09:41:13.221 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-03-04T09:41:13.221 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.221 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-03-04T09:41:13.221 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "devicehealth", 2024-03-04T09:41:13.221 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-03-04T09:41:13.222 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-03-04T09:41:13.222 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-03-04T09:41:13.222 INFO:teuthology.orchestra.run.smithi052.stdout: "enable_monitoring": { 2024-03-04T09:41:13.222 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "enable_monitoring", 2024-03-04T09:41:13.222 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:13.222 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.222 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:13.222 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-03-04T09:41:13.223 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.223 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.223 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.223 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "monitor device health metrics", 2024-03-04T09:41:13.223 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.223 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.223 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.223 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.223 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-03-04T09:41:13.224 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-03-04T09:41:13.224 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:13.224 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.224 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:13.224 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-03-04T09:41:13.224 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.224 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.224 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-03-04T09:41:13.225 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-03-04T09:41:13.225 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-03-04T09:41:13.225 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-03-04T09:41:13.225 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-03-04T09:41:13.225 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-03-04T09:41:13.225 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-03-04T09:41:13.225 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-03-04T09:41:13.225 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:13.226 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.226 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.226 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.226 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.226 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-03-04T09:41:13.226 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-03-04T09:41:13.226 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:13.226 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.226 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:13.227 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-03-04T09:41:13.227 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.227 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.227 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.227 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:13.227 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.227 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.227 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.228 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.228 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-03-04T09:41:13.228 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-03-04T09:41:13.228 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:13.228 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.228 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:13.228 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-03-04T09:41:13.229 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.229 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.229 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-03-04T09:41:13.229 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-03-04T09:41:13.229 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-03-04T09:41:13.229 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-03-04T09:41:13.229 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-03-04T09:41:13.229 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-03-04T09:41:13.229 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-03-04T09:41:13.230 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-03-04T09:41:13.230 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:13.230 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.230 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.230 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.230 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.230 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-03-04T09:41:13.230 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-03-04T09:41:13.231 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:13.231 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.231 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:13.231 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-03-04T09:41:13.231 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.231 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.231 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.231 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:13.232 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.232 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.232 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.232 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.232 INFO:teuthology.orchestra.run.smithi052.stdout: "mark_out_threshold": { 2024-03-04T09:41:13.232 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "mark_out_threshold", 2024-03-04T09:41:13.232 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "secs", 2024-03-04T09:41:13.232 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.233 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:13.233 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "2419200", 2024-03-04T09:41:13.233 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.233 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.233 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.233 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-03-04T09:41:13.233 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.233 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.233 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.234 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.234 INFO:teuthology.orchestra.run.smithi052.stdout: "pool_name": { 2024-03-04T09:41:13.234 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "pool_name", 2024-03-04T09:41:13.234 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:13.234 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.234 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:13.234 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "device_health_metrics", 2024-03-04T09:41:13.234 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.235 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.235 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.235 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "name of pool in which to store device health metrics", 2024-03-04T09:41:13.235 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.235 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.235 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.235 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.235 INFO:teuthology.orchestra.run.smithi052.stdout: "retention_period": { 2024-03-04T09:41:13.235 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "retention_period", 2024-03-04T09:41:13.236 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "secs", 2024-03-04T09:41:13.236 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.236 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:13.236 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "15552000", 2024-03-04T09:41:13.236 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.236 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.236 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.236 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "how long to retain device health metrics", 2024-03-04T09:41:13.237 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.237 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.237 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.237 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.237 INFO:teuthology.orchestra.run.smithi052.stdout: "scrape_frequency": { 2024-03-04T09:41:13.237 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "scrape_frequency", 2024-03-04T09:41:13.237 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "secs", 2024-03-04T09:41:13.238 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.238 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:13.238 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "86400", 2024-03-04T09:41:13.238 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.238 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.238 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.238 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "how frequently to scrape device health metrics", 2024-03-04T09:41:13.238 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.238 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.239 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.239 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.239 INFO:teuthology.orchestra.run.smithi052.stdout: "self_heal": { 2024-03-04T09:41:13.239 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "self_heal", 2024-03-04T09:41:13.239 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:13.239 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.239 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:13.239 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-03-04T09:41:13.240 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.240 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.240 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.240 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-03-04T09:41:13.240 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.240 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.240 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.240 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.241 INFO:teuthology.orchestra.run.smithi052.stdout: "sleep_interval": { 2024-03-04T09:41:13.241 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "sleep_interval", 2024-03-04T09:41:13.241 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "secs", 2024-03-04T09:41:13.241 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.241 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:13.241 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "600", 2024-03-04T09:41:13.241 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.241 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.242 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.242 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "how frequently to wake up and check device health", 2024-03-04T09:41:13.242 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.242 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.242 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.242 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.242 INFO:teuthology.orchestra.run.smithi052.stdout: "warn_threshold": { 2024-03-04T09:41:13.242 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "warn_threshold", 2024-03-04T09:41:13.243 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "secs", 2024-03-04T09:41:13.243 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.243 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:13.243 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "7257600", 2024-03-04T09:41:13.243 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.243 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.243 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.243 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-03-04T09:41:13.244 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.244 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.244 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.244 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-03-04T09:41:13.244 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-03-04T09:41:13.244 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.245 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-03-04T09:41:13.245 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "diskprediction_local", 2024-03-04T09:41:13.245 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-03-04T09:41:13.245 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-03-04T09:41:13.245 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-03-04T09:41:13.245 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-03-04T09:41:13.245 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-03-04T09:41:13.245 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:13.246 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.246 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:13.246 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-03-04T09:41:13.246 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.246 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.246 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-03-04T09:41:13.246 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-03-04T09:41:13.246 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-03-04T09:41:13.246 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-03-04T09:41:13.247 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-03-04T09:41:13.247 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-03-04T09:41:13.247 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-03-04T09:41:13.247 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-03-04T09:41:13.247 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:13.247 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.247 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.247 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.248 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.248 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-03-04T09:41:13.248 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-03-04T09:41:13.248 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:13.248 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.248 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:13.248 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-03-04T09:41:13.249 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.249 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.249 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.249 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:13.249 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.249 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.249 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.249 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.250 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-03-04T09:41:13.250 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-03-04T09:41:13.250 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:13.250 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.250 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:13.250 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-03-04T09:41:13.250 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.251 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.251 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-03-04T09:41:13.251 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-03-04T09:41:13.251 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-03-04T09:41:13.251 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-03-04T09:41:13.251 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-03-04T09:41:13.251 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-03-04T09:41:13.251 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-03-04T09:41:13.252 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-03-04T09:41:13.252 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:13.252 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.252 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.252 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.252 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.252 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-03-04T09:41:13.253 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-03-04T09:41:13.253 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:13.253 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.253 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:13.253 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-03-04T09:41:13.253 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.253 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.253 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.254 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:13.254 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.254 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.254 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.254 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.254 INFO:teuthology.orchestra.run.smithi052.stdout: "predict_interval": { 2024-03-04T09:41:13.254 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "predict_interval", 2024-03-04T09:41:13.254 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:13.255 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.255 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:13.255 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "86400", 2024-03-04T09:41:13.255 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.255 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.255 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.255 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:13.256 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.256 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.256 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.256 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.256 INFO:teuthology.orchestra.run.smithi052.stdout: "predictor_model": { 2024-03-04T09:41:13.256 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "predictor_model", 2024-03-04T09:41:13.256 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:13.256 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.257 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:13.257 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "prophetstor", 2024-03-04T09:41:13.257 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.257 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.257 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.257 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:13.257 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.257 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.258 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.258 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.258 INFO:teuthology.orchestra.run.smithi052.stdout: "sleep_interval": { 2024-03-04T09:41:13.258 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "sleep_interval", 2024-03-04T09:41:13.258 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:13.258 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.258 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:13.258 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "600", 2024-03-04T09:41:13.259 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.259 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.259 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.259 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:13.259 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.259 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.259 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.259 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-03-04T09:41:13.260 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-03-04T09:41:13.260 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.260 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-03-04T09:41:13.260 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "influx", 2024-03-04T09:41:13.260 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": false, 2024-03-04T09:41:13.260 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "influxdb python module not found", 2024-03-04T09:41:13.260 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-03-04T09:41:13.260 INFO:teuthology.orchestra.run.smithi052.stdout: "batch_size": { 2024-03-04T09:41:13.261 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "batch_size", 2024-03-04T09:41:13.261 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-03-04T09:41:13.261 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.261 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:13.261 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "5000", 2024-03-04T09:41:13.261 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.261 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.261 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.261 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-03-04T09:41:13.262 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.262 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.262 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.262 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.262 INFO:teuthology.orchestra.run.smithi052.stdout: "database": { 2024-03-04T09:41:13.262 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "database", 2024-03-04T09:41:13.262 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:13.263 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.263 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:13.263 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "ceph", 2024-03-04T09:41:13.263 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.263 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.263 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.263 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-03-04T09:41:13.263 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.263 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.264 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.264 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.264 INFO:teuthology.orchestra.run.smithi052.stdout: "hostname": { 2024-03-04T09:41:13.264 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "hostname", 2024-03-04T09:41:13.264 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:13.264 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.264 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:13.265 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-03-04T09:41:13.265 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.265 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.265 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.265 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "InfluxDB server hostname", 2024-03-04T09:41:13.267 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.267 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.267 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.268 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.268 INFO:teuthology.orchestra.run.smithi052.stdout: "interval": { 2024-03-04T09:41:13.268 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "interval", 2024-03-04T09:41:13.268 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "secs", 2024-03-04T09:41:13.268 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.268 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:13.268 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "30", 2024-03-04T09:41:13.269 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "5", 2024-03-04T09:41:13.269 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.269 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.269 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-03-04T09:41:13.269 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.269 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.269 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.270 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.270 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-03-04T09:41:13.270 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-03-04T09:41:13.270 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:13.270 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.270 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:13.270 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-03-04T09:41:13.270 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.270 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.271 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-03-04T09:41:13.271 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-03-04T09:41:13.271 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-03-04T09:41:13.271 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-03-04T09:41:13.271 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-03-04T09:41:13.271 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-03-04T09:41:13.271 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-03-04T09:41:13.271 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-03-04T09:41:13.272 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:13.272 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.272 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.272 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.272 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.272 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-03-04T09:41:13.272 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-03-04T09:41:13.272 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:13.273 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.273 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:13.273 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-03-04T09:41:13.273 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.273 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.273 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.273 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:13.273 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.273 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.274 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.274 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.274 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-03-04T09:41:13.274 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-03-04T09:41:13.274 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:13.274 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.274 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:13.274 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-03-04T09:41:13.274 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.275 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.275 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-03-04T09:41:13.275 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-03-04T09:41:13.275 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-03-04T09:41:13.275 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-03-04T09:41:13.275 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-03-04T09:41:13.275 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-03-04T09:41:13.275 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-03-04T09:41:13.276 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-03-04T09:41:13.276 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:13.276 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.276 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.276 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.276 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.276 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-03-04T09:41:13.276 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-03-04T09:41:13.277 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:13.277 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.277 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:13.277 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-03-04T09:41:13.277 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.277 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.277 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.277 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:13.278 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.278 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.278 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.278 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.278 INFO:teuthology.orchestra.run.smithi052.stdout: "password": { 2024-03-04T09:41:13.278 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "password", 2024-03-04T09:41:13.278 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:13.278 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.279 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:13.279 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-03-04T09:41:13.279 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.279 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.279 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.279 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "password of InfluxDB server user", 2024-03-04T09:41:13.279 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.279 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.280 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.280 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.280 INFO:teuthology.orchestra.run.smithi052.stdout: "port": { 2024-03-04T09:41:13.280 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "port", 2024-03-04T09:41:13.280 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-03-04T09:41:13.280 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.280 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:13.280 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "8086", 2024-03-04T09:41:13.281 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.281 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.281 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.281 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "InfluxDB server port", 2024-03-04T09:41:13.281 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.281 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.281 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.281 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.282 INFO:teuthology.orchestra.run.smithi052.stdout: "ssl": { 2024-03-04T09:41:13.282 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "ssl", 2024-03-04T09:41:13.282 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:13.282 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.282 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:13.282 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "false", 2024-03-04T09:41:13.282 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.282 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.283 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.283 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-03-04T09:41:13.283 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.283 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.283 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.283 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.283 INFO:teuthology.orchestra.run.smithi052.stdout: "threads": { 2024-03-04T09:41:13.283 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "threads", 2024-03-04T09:41:13.284 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-03-04T09:41:13.284 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.284 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:13.284 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "5", 2024-03-04T09:41:13.284 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "1", 2024-03-04T09:41:13.284 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "32", 2024-03-04T09:41:13.284 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.284 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-03-04T09:41:13.285 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.285 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.285 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.285 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.285 INFO:teuthology.orchestra.run.smithi052.stdout: "username": { 2024-03-04T09:41:13.285 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "username", 2024-03-04T09:41:13.285 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:13.285 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.285 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:13.286 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-03-04T09:41:13.286 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.286 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.286 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.286 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "username of InfluxDB server user", 2024-03-04T09:41:13.286 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.286 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.286 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.287 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.287 INFO:teuthology.orchestra.run.smithi052.stdout: "verify_ssl": { 2024-03-04T09:41:13.287 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "verify_ssl", 2024-03-04T09:41:13.287 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:13.287 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.287 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:13.287 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "true", 2024-03-04T09:41:13.287 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.287 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.288 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.288 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-03-04T09:41:13.288 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.288 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.288 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.288 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-03-04T09:41:13.288 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-03-04T09:41:13.289 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.289 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-03-04T09:41:13.289 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "insights", 2024-03-04T09:41:13.289 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-03-04T09:41:13.289 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-03-04T09:41:13.289 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-03-04T09:41:13.289 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-03-04T09:41:13.289 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-03-04T09:41:13.289 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:13.290 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.290 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:13.290 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-03-04T09:41:13.290 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.290 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.290 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-03-04T09:41:13.290 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-03-04T09:41:13.290 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-03-04T09:41:13.290 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-03-04T09:41:13.290 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-03-04T09:41:13.291 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-03-04T09:41:13.291 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-03-04T09:41:13.291 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-03-04T09:41:13.291 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:13.291 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.291 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.291 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.291 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.292 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-03-04T09:41:13.292 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-03-04T09:41:13.292 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:13.292 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.292 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:13.292 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-03-04T09:41:13.292 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.292 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.292 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.293 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:13.293 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.293 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.293 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.293 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.293 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-03-04T09:41:13.293 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-03-04T09:41:13.293 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:13.293 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.293 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:13.294 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-03-04T09:41:13.294 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.294 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.294 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-03-04T09:41:13.294 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-03-04T09:41:13.294 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-03-04T09:41:13.294 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-03-04T09:41:13.294 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-03-04T09:41:13.295 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-03-04T09:41:13.295 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-03-04T09:41:13.295 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-03-04T09:41:13.295 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:13.295 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.295 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.295 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.295 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.295 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-03-04T09:41:13.296 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-03-04T09:41:13.296 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:13.296 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.296 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:13.296 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-03-04T09:41:13.296 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.296 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.297 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.297 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:13.297 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.297 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.297 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.297 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-03-04T09:41:13.297 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-03-04T09:41:13.297 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.298 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-03-04T09:41:13.298 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "iostat", 2024-03-04T09:41:13.298 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-03-04T09:41:13.298 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-03-04T09:41:13.298 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-03-04T09:41:13.298 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-03-04T09:41:13.298 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-03-04T09:41:13.299 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:13.299 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.299 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:13.299 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-03-04T09:41:13.299 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.299 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.299 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-03-04T09:41:13.299 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-03-04T09:41:13.299 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-03-04T09:41:13.300 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-03-04T09:41:13.300 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-03-04T09:41:13.300 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-03-04T09:41:13.300 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-03-04T09:41:13.300 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-03-04T09:41:13.300 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:13.300 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.301 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.301 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.301 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.301 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-03-04T09:41:13.301 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-03-04T09:41:13.301 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:13.301 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.301 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:13.302 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-03-04T09:41:13.302 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.302 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.302 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.302 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:13.302 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.302 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.302 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.303 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.303 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-03-04T09:41:13.303 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-03-04T09:41:13.303 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:13.303 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.303 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:13.303 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-03-04T09:41:13.304 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.304 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.304 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-03-04T09:41:13.304 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-03-04T09:41:13.304 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-03-04T09:41:13.304 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-03-04T09:41:13.304 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-03-04T09:41:13.304 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-03-04T09:41:13.305 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-03-04T09:41:13.305 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-03-04T09:41:13.305 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:13.305 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.305 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.305 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.305 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.305 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-03-04T09:41:13.305 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-03-04T09:41:13.306 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:13.306 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.306 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:13.306 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-03-04T09:41:13.306 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.306 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.306 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.306 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:13.307 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.307 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.307 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.307 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-03-04T09:41:13.307 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-03-04T09:41:13.307 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.307 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-03-04T09:41:13.307 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "k8sevents", 2024-03-04T09:41:13.308 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-03-04T09:41:13.308 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-03-04T09:41:13.308 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-03-04T09:41:13.308 INFO:teuthology.orchestra.run.smithi052.stdout: "ceph_event_retention_days": { 2024-03-04T09:41:13.308 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "ceph_event_retention_days", 2024-03-04T09:41:13.308 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-03-04T09:41:13.308 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.308 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:13.309 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "7", 2024-03-04T09:41:13.309 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.309 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.309 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.309 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Days to hold ceph event information within local cache", 2024-03-04T09:41:13.309 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.309 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.309 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.310 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.310 INFO:teuthology.orchestra.run.smithi052.stdout: "config_check_secs": { 2024-03-04T09:41:13.310 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "config_check_secs", 2024-03-04T09:41:13.310 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-03-04T09:41:13.310 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.310 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:13.310 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "10", 2024-03-04T09:41:13.310 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "10", 2024-03-04T09:41:13.310 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.311 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.311 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-03-04T09:41:13.311 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.311 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.311 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.311 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.311 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-03-04T09:41:13.311 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-03-04T09:41:13.312 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:13.312 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.312 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:13.312 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-03-04T09:41:13.312 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.312 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.312 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-03-04T09:41:13.312 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-03-04T09:41:13.313 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-03-04T09:41:13.313 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-03-04T09:41:13.313 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-03-04T09:41:13.313 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-03-04T09:41:13.313 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-03-04T09:41:13.313 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-03-04T09:41:13.313 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:13.313 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.313 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.314 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.314 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.314 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-03-04T09:41:13.314 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-03-04T09:41:13.314 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:13.314 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.314 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:13.314 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-03-04T09:41:13.315 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.315 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.315 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.315 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:13.315 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.315 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.315 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.315 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.316 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-03-04T09:41:13.316 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-03-04T09:41:13.316 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:13.316 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.316 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:13.316 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-03-04T09:41:13.316 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.316 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.317 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-03-04T09:41:13.317 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-03-04T09:41:13.317 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-03-04T09:41:13.317 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-03-04T09:41:13.317 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-03-04T09:41:13.317 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-03-04T09:41:13.317 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-03-04T09:41:13.318 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-03-04T09:41:13.318 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:13.318 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.318 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.318 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.318 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.318 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-03-04T09:41:13.318 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-03-04T09:41:13.319 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:13.319 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.319 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:13.319 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-03-04T09:41:13.319 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.319 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.319 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.319 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:13.320 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.320 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.320 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.320 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-03-04T09:41:13.320 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-03-04T09:41:13.320 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.320 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-03-04T09:41:13.320 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "localpool", 2024-03-04T09:41:13.321 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-03-04T09:41:13.321 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-03-04T09:41:13.321 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-03-04T09:41:13.321 INFO:teuthology.orchestra.run.smithi052.stdout: "failure_domain": { 2024-03-04T09:41:13.321 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "failure_domain", 2024-03-04T09:41:13.321 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:13.321 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.321 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:13.322 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "host", 2024-03-04T09:41:13.322 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.322 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.322 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.322 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "failure domain for any created local pool", 2024-03-04T09:41:13.322 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-03-04T09:41:13.322 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.322 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.323 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.323 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-03-04T09:41:13.323 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-03-04T09:41:13.323 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:13.323 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.323 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:13.323 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-03-04T09:41:13.323 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.323 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.324 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-03-04T09:41:13.324 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-03-04T09:41:13.324 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-03-04T09:41:13.324 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-03-04T09:41:13.324 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-03-04T09:41:13.324 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-03-04T09:41:13.324 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-03-04T09:41:13.325 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-03-04T09:41:13.325 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:13.325 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.325 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.325 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.325 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.325 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-03-04T09:41:13.325 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-03-04T09:41:13.326 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:13.326 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.326 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:13.326 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-03-04T09:41:13.326 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.326 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.326 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.326 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:13.327 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.327 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.327 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.327 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.327 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-03-04T09:41:13.327 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-03-04T09:41:13.327 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:13.327 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.327 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:13.328 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-03-04T09:41:13.328 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.328 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.328 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-03-04T09:41:13.328 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-03-04T09:41:13.328 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-03-04T09:41:13.328 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-03-04T09:41:13.328 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-03-04T09:41:13.329 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-03-04T09:41:13.329 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-03-04T09:41:13.329 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-03-04T09:41:13.329 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:13.329 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.329 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.329 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.329 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.330 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-03-04T09:41:13.330 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-03-04T09:41:13.330 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:13.330 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.330 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:13.330 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-03-04T09:41:13.330 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.330 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.331 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.331 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:13.331 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.331 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.331 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.331 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.331 INFO:teuthology.orchestra.run.smithi052.stdout: "min_size": { 2024-03-04T09:41:13.331 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "min_size", 2024-03-04T09:41:13.332 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-03-04T09:41:13.332 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.332 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:13.332 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-03-04T09:41:13.332 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.332 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.332 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.333 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "default min_size for any created local pool", 2024-03-04T09:41:13.333 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-03-04T09:41:13.333 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.333 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.333 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.333 INFO:teuthology.orchestra.run.smithi052.stdout: "num_rep": { 2024-03-04T09:41:13.333 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "num_rep", 2024-03-04T09:41:13.333 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-03-04T09:41:13.334 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.334 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:13.334 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "3", 2024-03-04T09:41:13.334 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.334 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.334 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.334 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "default replica count for any created local pool", 2024-03-04T09:41:13.334 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.335 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.335 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.335 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.335 INFO:teuthology.orchestra.run.smithi052.stdout: "pg_num": { 2024-03-04T09:41:13.335 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "pg_num", 2024-03-04T09:41:13.335 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-03-04T09:41:13.335 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.335 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:13.336 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "128", 2024-03-04T09:41:13.336 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.336 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.336 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.336 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "default pg_num for any created local pool", 2024-03-04T09:41:13.336 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.336 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.336 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.337 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.337 INFO:teuthology.orchestra.run.smithi052.stdout: "prefix": { 2024-03-04T09:41:13.337 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "prefix", 2024-03-04T09:41:13.337 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:13.337 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.337 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:13.337 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-03-04T09:41:13.337 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.338 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.338 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.338 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "name prefix for any created local pool", 2024-03-04T09:41:13.338 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.338 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.338 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.338 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.339 INFO:teuthology.orchestra.run.smithi052.stdout: "subtree": { 2024-03-04T09:41:13.339 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "subtree", 2024-03-04T09:41:13.339 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:13.339 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.339 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:13.339 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "rack", 2024-03-04T09:41:13.339 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.339 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.340 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.340 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "CRUSH level for which to create a local pool", 2024-03-04T09:41:13.340 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-03-04T09:41:13.340 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.340 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.340 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-03-04T09:41:13.340 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-03-04T09:41:13.341 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.341 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-03-04T09:41:13.341 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "mds_autoscaler", 2024-03-04T09:41:13.341 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-03-04T09:41:13.341 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-03-04T09:41:13.341 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-03-04T09:41:13.341 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-03-04T09:41:13.342 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-03-04T09:41:13.342 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:13.342 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.342 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:13.342 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-03-04T09:41:13.342 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.342 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.343 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-03-04T09:41:13.343 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-03-04T09:41:13.343 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-03-04T09:41:13.343 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-03-04T09:41:13.343 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-03-04T09:41:13.343 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-03-04T09:41:13.343 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-03-04T09:41:13.343 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-03-04T09:41:13.344 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:13.344 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.344 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.344 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.344 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.344 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-03-04T09:41:13.344 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-03-04T09:41:13.344 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:13.345 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.345 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:13.345 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-03-04T09:41:13.345 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.345 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.345 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.345 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:13.345 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.346 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.346 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.346 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.346 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-03-04T09:41:13.346 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-03-04T09:41:13.346 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:13.346 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.346 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:13.346 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-03-04T09:41:13.347 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.347 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.347 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-03-04T09:41:13.347 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-03-04T09:41:13.347 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-03-04T09:41:13.347 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-03-04T09:41:13.347 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-03-04T09:41:13.347 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-03-04T09:41:13.348 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-03-04T09:41:13.348 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-03-04T09:41:13.348 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:13.348 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.348 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.348 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.348 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.348 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-03-04T09:41:13.349 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-03-04T09:41:13.349 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:13.349 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.349 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:13.349 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-03-04T09:41:13.349 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.349 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.349 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.349 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:13.350 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.350 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.350 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.350 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-03-04T09:41:13.350 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-03-04T09:41:13.350 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.350 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-03-04T09:41:13.350 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "mirroring", 2024-03-04T09:41:13.351 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-03-04T09:41:13.351 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-03-04T09:41:13.351 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-03-04T09:41:13.351 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-03-04T09:41:13.351 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-03-04T09:41:13.351 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:13.351 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.351 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:13.352 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-03-04T09:41:13.352 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.352 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.352 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-03-04T09:41:13.352 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-03-04T09:41:13.352 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-03-04T09:41:13.352 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-03-04T09:41:13.352 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-03-04T09:41:13.353 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-03-04T09:41:13.353 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-03-04T09:41:13.353 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-03-04T09:41:13.353 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:13.353 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.353 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.353 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.353 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.353 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-03-04T09:41:13.354 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-03-04T09:41:13.354 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:13.354 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.354 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:13.354 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-03-04T09:41:13.354 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.354 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.354 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.355 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:13.355 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.355 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.355 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.355 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.355 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-03-04T09:41:13.355 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-03-04T09:41:13.355 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:13.356 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.356 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:13.356 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-03-04T09:41:13.356 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.356 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.356 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-03-04T09:41:13.356 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-03-04T09:41:13.356 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-03-04T09:41:13.357 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-03-04T09:41:13.357 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-03-04T09:41:13.357 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-03-04T09:41:13.357 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-03-04T09:41:13.357 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-03-04T09:41:13.357 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:13.357 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.357 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.358 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.358 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.358 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-03-04T09:41:13.358 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-03-04T09:41:13.358 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:13.358 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.358 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:13.358 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-03-04T09:41:13.358 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.359 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.359 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.359 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:13.359 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.359 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.359 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.359 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-03-04T09:41:13.359 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-03-04T09:41:13.360 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.360 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-03-04T09:41:13.360 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "nfs", 2024-03-04T09:41:13.360 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-03-04T09:41:13.360 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-03-04T09:41:13.360 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-03-04T09:41:13.360 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-03-04T09:41:13.360 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-03-04T09:41:13.361 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:13.361 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.361 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:13.361 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-03-04T09:41:13.361 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.361 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.361 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-03-04T09:41:13.361 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-03-04T09:41:13.362 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-03-04T09:41:13.362 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-03-04T09:41:13.362 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-03-04T09:41:13.362 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-03-04T09:41:13.362 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-03-04T09:41:13.362 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-03-04T09:41:13.362 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:13.362 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.363 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.363 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.363 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.363 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-03-04T09:41:13.363 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-03-04T09:41:13.363 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:13.363 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.363 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:13.364 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-03-04T09:41:13.364 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.364 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.364 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.364 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:13.364 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.364 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.364 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.365 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.365 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-03-04T09:41:13.365 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-03-04T09:41:13.365 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:13.365 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.365 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:13.365 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-03-04T09:41:13.365 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.366 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.366 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-03-04T09:41:13.366 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-03-04T09:41:13.366 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-03-04T09:41:13.366 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-03-04T09:41:13.366 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-03-04T09:41:13.366 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-03-04T09:41:13.366 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-03-04T09:41:13.367 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-03-04T09:41:13.367 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:13.367 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.367 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.367 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.367 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.367 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-03-04T09:41:13.367 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-03-04T09:41:13.367 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:13.368 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.368 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:13.368 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-03-04T09:41:13.368 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.368 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.368 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.368 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:13.368 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.369 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.369 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.369 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-03-04T09:41:13.369 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-03-04T09:41:13.369 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.369 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-03-04T09:41:13.369 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "orchestrator", 2024-03-04T09:41:13.369 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-03-04T09:41:13.370 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-03-04T09:41:13.370 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-03-04T09:41:13.370 INFO:teuthology.orchestra.run.smithi052.stdout: "fail_fs": { 2024-03-04T09:41:13.370 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "fail_fs", 2024-03-04T09:41:13.370 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:13.370 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.370 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:13.370 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-03-04T09:41:13.371 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.371 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.371 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.371 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-03-04T09:41:13.371 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.371 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.371 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.371 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.371 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-03-04T09:41:13.372 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-03-04T09:41:13.372 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:13.372 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.372 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:13.372 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-03-04T09:41:13.372 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.372 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.372 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-03-04T09:41:13.373 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-03-04T09:41:13.373 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-03-04T09:41:13.373 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-03-04T09:41:13.373 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-03-04T09:41:13.373 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-03-04T09:41:13.373 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-03-04T09:41:13.373 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-03-04T09:41:13.373 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:13.374 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.374 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.374 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.374 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.374 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-03-04T09:41:13.374 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-03-04T09:41:13.374 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:13.374 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.374 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:13.375 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-03-04T09:41:13.375 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.375 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.375 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.375 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:13.375 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.375 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.375 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.375 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.376 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-03-04T09:41:13.376 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-03-04T09:41:13.376 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:13.376 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.376 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:13.376 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-03-04T09:41:13.376 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.376 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.377 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-03-04T09:41:13.377 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-03-04T09:41:13.377 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-03-04T09:41:13.377 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-03-04T09:41:13.377 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-03-04T09:41:13.377 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-03-04T09:41:13.377 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-03-04T09:41:13.377 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-03-04T09:41:13.378 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:13.378 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.378 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.378 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.378 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.378 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-03-04T09:41:13.378 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-03-04T09:41:13.378 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:13.378 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.379 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:13.379 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-03-04T09:41:13.379 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.379 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.379 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.379 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:13.379 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.379 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.380 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.380 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.380 INFO:teuthology.orchestra.run.smithi052.stdout: "orchestrator": { 2024-03-04T09:41:13.380 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "orchestrator", 2024-03-04T09:41:13.380 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:13.380 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.380 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:13.380 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-03-04T09:41:13.380 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.381 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.381 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-03-04T09:41:13.381 INFO:teuthology.orchestra.run.smithi052.stdout: "cephadm", 2024-03-04T09:41:13.381 INFO:teuthology.orchestra.run.smithi052.stdout: "rook", 2024-03-04T09:41:13.381 INFO:teuthology.orchestra.run.smithi052.stdout: "test_orchestrator" 2024-03-04T09:41:13.381 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-03-04T09:41:13.381 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Orchestrator backend", 2024-03-04T09:41:13.382 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.382 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.382 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.382 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-03-04T09:41:13.382 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-03-04T09:41:13.382 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.382 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-03-04T09:41:13.382 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "osd_perf_query", 2024-03-04T09:41:13.383 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-03-04T09:41:13.383 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-03-04T09:41:13.383 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-03-04T09:41:13.383 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-03-04T09:41:13.383 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-03-04T09:41:13.383 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:13.383 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.383 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:13.384 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-03-04T09:41:13.384 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.384 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.384 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-03-04T09:41:13.384 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-03-04T09:41:13.384 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-03-04T09:41:13.384 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-03-04T09:41:13.384 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-03-04T09:41:13.385 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-03-04T09:41:13.385 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-03-04T09:41:13.385 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-03-04T09:41:13.385 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:13.385 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.385 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.385 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.385 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.386 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-03-04T09:41:13.386 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-03-04T09:41:13.386 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:13.386 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.386 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:13.386 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-03-04T09:41:13.386 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.386 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.387 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.387 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:13.387 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.387 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.387 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.387 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.387 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-03-04T09:41:13.387 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-03-04T09:41:13.388 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:13.388 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.388 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:13.388 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-03-04T09:41:13.388 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.388 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.388 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-03-04T09:41:13.388 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-03-04T09:41:13.389 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-03-04T09:41:13.389 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-03-04T09:41:13.389 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-03-04T09:41:13.389 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-03-04T09:41:13.389 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-03-04T09:41:13.389 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-03-04T09:41:13.389 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:13.389 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.390 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.390 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.390 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.390 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-03-04T09:41:13.390 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-03-04T09:41:13.390 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:13.390 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.390 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:13.391 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-03-04T09:41:13.391 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.391 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.391 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.391 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:13.391 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.391 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.391 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.392 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-03-04T09:41:13.392 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-03-04T09:41:13.392 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.392 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-03-04T09:41:13.392 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "osd_support", 2024-03-04T09:41:13.392 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-03-04T09:41:13.392 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-03-04T09:41:13.392 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-03-04T09:41:13.392 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-03-04T09:41:13.393 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-03-04T09:41:13.393 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:13.393 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.393 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:13.393 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-03-04T09:41:13.393 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.393 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.393 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-03-04T09:41:13.394 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-03-04T09:41:13.394 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-03-04T09:41:13.394 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-03-04T09:41:13.394 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-03-04T09:41:13.394 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-03-04T09:41:13.394 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-03-04T09:41:13.394 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-03-04T09:41:13.394 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:13.394 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.395 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.395 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.395 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.395 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-03-04T09:41:13.395 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-03-04T09:41:13.395 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:13.395 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.395 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:13.396 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-03-04T09:41:13.396 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.396 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.396 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.396 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:13.396 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.396 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.396 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.397 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.397 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-03-04T09:41:13.397 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-03-04T09:41:13.397 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:13.397 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.397 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:13.397 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-03-04T09:41:13.397 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.397 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.398 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-03-04T09:41:13.398 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-03-04T09:41:13.398 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-03-04T09:41:13.398 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-03-04T09:41:13.398 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-03-04T09:41:13.398 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-03-04T09:41:13.398 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-03-04T09:41:13.398 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-03-04T09:41:13.399 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:13.399 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.399 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.399 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.399 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.399 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-03-04T09:41:13.399 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-03-04T09:41:13.399 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:13.400 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.400 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:13.400 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-03-04T09:41:13.400 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.400 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.400 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.400 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:13.400 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.400 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.401 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.401 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-03-04T09:41:13.401 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-03-04T09:41:13.401 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.401 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-03-04T09:41:13.401 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "pg_autoscaler", 2024-03-04T09:41:13.401 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-03-04T09:41:13.401 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-03-04T09:41:13.402 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-03-04T09:41:13.402 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-03-04T09:41:13.402 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-03-04T09:41:13.402 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:13.402 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.402 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:13.402 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-03-04T09:41:13.402 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.403 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.403 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-03-04T09:41:13.403 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-03-04T09:41:13.403 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-03-04T09:41:13.403 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-03-04T09:41:13.403 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-03-04T09:41:13.403 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-03-04T09:41:13.403 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-03-04T09:41:13.404 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-03-04T09:41:13.404 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:13.404 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.404 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.404 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.404 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.404 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-03-04T09:41:13.404 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-03-04T09:41:13.405 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:13.405 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.405 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:13.405 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-03-04T09:41:13.405 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.405 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.405 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.405 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:13.406 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.406 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.406 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.406 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.406 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-03-04T09:41:13.406 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-03-04T09:41:13.406 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:13.406 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.407 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:13.407 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-03-04T09:41:13.407 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.407 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.407 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-03-04T09:41:13.407 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-03-04T09:41:13.407 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-03-04T09:41:13.407 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-03-04T09:41:13.408 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-03-04T09:41:13.408 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-03-04T09:41:13.408 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-03-04T09:41:13.408 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-03-04T09:41:13.408 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:13.408 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.408 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.408 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.409 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.409 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-03-04T09:41:13.409 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-03-04T09:41:13.409 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:13.409 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.409 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:13.409 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-03-04T09:41:13.409 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.410 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.410 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.410 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:13.410 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.410 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.410 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.410 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.410 INFO:teuthology.orchestra.run.smithi052.stdout: "sleep_interval": { 2024-03-04T09:41:13.411 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "sleep_interval", 2024-03-04T09:41:13.411 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "secs", 2024-03-04T09:41:13.411 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.411 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:13.411 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "60", 2024-03-04T09:41:13.411 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.411 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.412 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.412 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:13.412 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.412 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.412 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.412 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.412 INFO:teuthology.orchestra.run.smithi052.stdout: "threshold": { 2024-03-04T09:41:13.413 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "threshold", 2024-03-04T09:41:13.413 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "float", 2024-03-04T09:41:13.413 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.413 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:13.413 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "3.0", 2024-03-04T09:41:13.413 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "1.0", 2024-03-04T09:41:13.413 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.413 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.414 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "scaling threshold", 2024-03-04T09:41:13.414 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-03-04T09:41:13.414 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.414 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.414 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-03-04T09:41:13.414 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-03-04T09:41:13.414 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.414 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-03-04T09:41:13.415 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "progress", 2024-03-04T09:41:13.415 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-03-04T09:41:13.415 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-03-04T09:41:13.415 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-03-04T09:41:13.415 INFO:teuthology.orchestra.run.smithi052.stdout: "allow_pg_recovery_event": { 2024-03-04T09:41:13.415 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "allow_pg_recovery_event", 2024-03-04T09:41:13.415 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:13.415 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.416 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:13.416 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-03-04T09:41:13.416 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.416 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.416 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.416 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "allow the module to show pg recovery progress", 2024-03-04T09:41:13.416 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.416 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.416 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.417 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.417 INFO:teuthology.orchestra.run.smithi052.stdout: "enabled": { 2024-03-04T09:41:13.417 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "enabled", 2024-03-04T09:41:13.417 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:13.417 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.417 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:13.417 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-03-04T09:41:13.417 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.418 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.418 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.418 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:13.418 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.418 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.418 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.418 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.418 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-03-04T09:41:13.418 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-03-04T09:41:13.419 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:13.419 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.419 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:13.419 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-03-04T09:41:13.419 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.419 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.419 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-03-04T09:41:13.419 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-03-04T09:41:13.420 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-03-04T09:41:13.420 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-03-04T09:41:13.420 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-03-04T09:41:13.420 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-03-04T09:41:13.420 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-03-04T09:41:13.420 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-03-04T09:41:13.420 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:13.420 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.421 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.421 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.421 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.421 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-03-04T09:41:13.421 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-03-04T09:41:13.421 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:13.421 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.421 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:13.421 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-03-04T09:41:13.422 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.422 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.422 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.422 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:13.422 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.422 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.422 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.422 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.423 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-03-04T09:41:13.423 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-03-04T09:41:13.423 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:13.423 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.423 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:13.423 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-03-04T09:41:13.423 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.423 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.424 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-03-04T09:41:13.424 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-03-04T09:41:13.424 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-03-04T09:41:13.424 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-03-04T09:41:13.424 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-03-04T09:41:13.424 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-03-04T09:41:13.424 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-03-04T09:41:13.424 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-03-04T09:41:13.425 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:13.425 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.425 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.425 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.425 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.425 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-03-04T09:41:13.425 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-03-04T09:41:13.425 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:13.425 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.426 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:13.426 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-03-04T09:41:13.426 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.426 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.426 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.426 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:13.426 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.426 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.427 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.427 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.427 INFO:teuthology.orchestra.run.smithi052.stdout: "max_completed_events": { 2024-03-04T09:41:13.427 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "max_completed_events", 2024-03-04T09:41:13.427 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-03-04T09:41:13.427 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.427 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:13.427 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "50", 2024-03-04T09:41:13.427 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.428 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.428 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.428 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "number of past completed events to remember", 2024-03-04T09:41:13.428 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.428 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.428 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.428 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.429 INFO:teuthology.orchestra.run.smithi052.stdout: "sleep_interval": { 2024-03-04T09:41:13.429 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "sleep_interval", 2024-03-04T09:41:13.429 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "secs", 2024-03-04T09:41:13.429 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.429 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:13.429 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "5", 2024-03-04T09:41:13.429 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.429 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.429 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.430 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "how long the module is going to sleep", 2024-03-04T09:41:13.430 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.430 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.430 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.430 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-03-04T09:41:13.430 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-03-04T09:41:13.430 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.430 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-03-04T09:41:13.431 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "prometheus", 2024-03-04T09:41:13.431 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-03-04T09:41:13.431 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-03-04T09:41:13.431 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-03-04T09:41:13.431 INFO:teuthology.orchestra.run.smithi052.stdout: "cache": { 2024-03-04T09:41:13.431 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "cache", 2024-03-04T09:41:13.431 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:13.431 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.431 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:13.432 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-03-04T09:41:13.432 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.432 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.432 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.432 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:13.432 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.432 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.432 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.432 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.433 INFO:teuthology.orchestra.run.smithi052.stdout: "exclude_perf_counters": { 2024-03-04T09:41:13.433 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "exclude_perf_counters", 2024-03-04T09:41:13.433 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:13.433 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.433 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:13.433 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-03-04T09:41:13.433 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.433 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.434 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.434 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-03-04T09:41:13.434 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-03-04T09:41:13.434 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.434 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.434 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.434 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-03-04T09:41:13.434 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-03-04T09:41:13.435 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:13.435 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.435 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:13.435 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-03-04T09:41:13.435 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.435 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.435 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-03-04T09:41:13.435 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-03-04T09:41:13.435 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-03-04T09:41:13.436 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-03-04T09:41:13.436 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-03-04T09:41:13.436 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-03-04T09:41:13.436 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-03-04T09:41:13.436 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-03-04T09:41:13.436 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:13.436 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.436 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.437 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.437 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.437 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-03-04T09:41:13.437 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-03-04T09:41:13.437 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:13.437 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.437 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:13.437 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-03-04T09:41:13.437 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.438 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.438 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.438 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:13.438 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.438 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.438 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.438 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.438 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-03-04T09:41:13.438 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-03-04T09:41:13.439 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:13.439 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.439 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:13.439 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-03-04T09:41:13.439 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.439 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.439 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-03-04T09:41:13.439 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-03-04T09:41:13.439 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-03-04T09:41:13.440 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-03-04T09:41:13.440 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-03-04T09:41:13.440 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-03-04T09:41:13.440 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-03-04T09:41:13.440 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-03-04T09:41:13.440 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:13.440 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.440 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.441 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.441 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.441 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-03-04T09:41:13.441 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-03-04T09:41:13.441 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:13.441 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.441 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:13.441 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-03-04T09:41:13.442 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.442 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.442 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.442 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:13.442 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.442 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.442 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.442 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.443 INFO:teuthology.orchestra.run.smithi052.stdout: "rbd_stats_pools": { 2024-03-04T09:41:13.443 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "rbd_stats_pools", 2024-03-04T09:41:13.443 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:13.443 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.443 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:13.443 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-03-04T09:41:13.443 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.443 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.444 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.444 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:13.444 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.444 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.444 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.444 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.444 INFO:teuthology.orchestra.run.smithi052.stdout: "rbd_stats_pools_refresh_interval": { 2024-03-04T09:41:13.444 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-03-04T09:41:13.445 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-03-04T09:41:13.445 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.445 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:13.445 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "300", 2024-03-04T09:41:13.445 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.445 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.445 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.445 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:13.445 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.446 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.446 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.446 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.446 INFO:teuthology.orchestra.run.smithi052.stdout: "scrape_interval": { 2024-03-04T09:41:13.446 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "scrape_interval", 2024-03-04T09:41:13.446 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "float", 2024-03-04T09:41:13.446 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.446 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:13.447 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "15.0", 2024-03-04T09:41:13.447 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.447 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.447 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.447 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:13.447 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.447 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.447 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.448 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.448 INFO:teuthology.orchestra.run.smithi052.stdout: "server_addr": { 2024-03-04T09:41:13.448 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "server_addr", 2024-03-04T09:41:13.448 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:13.448 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.448 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:13.448 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "::", 2024-03-04T09:41:13.448 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.448 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.449 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.449 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-03-04T09:41:13.449 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.449 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.449 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.449 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.449 INFO:teuthology.orchestra.run.smithi052.stdout: "server_port": { 2024-03-04T09:41:13.449 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "server_port", 2024-03-04T09:41:13.450 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-03-04T09:41:13.450 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.450 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:13.450 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "9283", 2024-03-04T09:41:13.450 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.450 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.450 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.450 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-03-04T09:41:13.451 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.451 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.451 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.451 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.451 INFO:teuthology.orchestra.run.smithi052.stdout: "stale_cache_strategy": { 2024-03-04T09:41:13.451 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "stale_cache_strategy", 2024-03-04T09:41:13.451 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:13.451 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.451 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:13.452 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "log", 2024-03-04T09:41:13.452 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.452 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.452 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.452 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:13.452 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.452 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.452 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.453 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.453 INFO:teuthology.orchestra.run.smithi052.stdout: "standby_behaviour": { 2024-03-04T09:41:13.453 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "standby_behaviour", 2024-03-04T09:41:13.453 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:13.453 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.453 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:13.453 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "default", 2024-03-04T09:41:13.453 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.454 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.454 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-03-04T09:41:13.454 INFO:teuthology.orchestra.run.smithi052.stdout: "default", 2024-03-04T09:41:13.454 INFO:teuthology.orchestra.run.smithi052.stdout: "error" 2024-03-04T09:41:13.454 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-03-04T09:41:13.454 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:13.454 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.454 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.454 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.455 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.455 INFO:teuthology.orchestra.run.smithi052.stdout: "standby_error_status_code": { 2024-03-04T09:41:13.455 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "standby_error_status_code", 2024-03-04T09:41:13.455 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-03-04T09:41:13.455 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.455 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:13.455 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "500", 2024-03-04T09:41:13.455 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "400", 2024-03-04T09:41:13.456 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "599", 2024-03-04T09:41:13.456 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.456 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:13.456 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.456 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.456 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.456 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-03-04T09:41:13.456 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-03-04T09:41:13.457 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.457 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-03-04T09:41:13.457 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "rbd_support", 2024-03-04T09:41:13.457 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-03-04T09:41:13.457 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-03-04T09:41:13.457 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-03-04T09:41:13.457 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-03-04T09:41:13.457 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-03-04T09:41:13.458 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:13.458 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.458 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:13.458 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-03-04T09:41:13.458 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.458 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.458 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-03-04T09:41:13.458 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-03-04T09:41:13.458 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-03-04T09:41:13.459 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-03-04T09:41:13.459 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-03-04T09:41:13.459 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-03-04T09:41:13.459 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-03-04T09:41:13.459 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-03-04T09:41:13.459 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:13.459 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.459 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.460 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.460 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.460 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-03-04T09:41:13.460 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-03-04T09:41:13.460 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:13.460 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.460 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:13.460 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-03-04T09:41:13.461 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.461 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.461 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.461 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:13.461 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.461 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.461 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.461 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.462 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-03-04T09:41:13.462 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-03-04T09:41:13.462 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:13.462 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.462 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:13.462 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-03-04T09:41:13.462 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.462 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.462 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-03-04T09:41:13.463 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-03-04T09:41:13.463 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-03-04T09:41:13.463 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-03-04T09:41:13.463 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-03-04T09:41:13.463 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-03-04T09:41:13.463 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-03-04T09:41:13.463 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-03-04T09:41:13.463 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:13.464 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.464 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.464 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.464 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.464 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-03-04T09:41:13.464 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-03-04T09:41:13.464 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:13.464 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.465 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:13.465 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-03-04T09:41:13.465 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.465 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.465 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.465 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:13.465 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.466 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.466 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.466 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.466 INFO:teuthology.orchestra.run.smithi052.stdout: "max_concurrent_snap_create": { 2024-03-04T09:41:13.466 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "max_concurrent_snap_create", 2024-03-04T09:41:13.466 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-03-04T09:41:13.466 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.466 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:13.467 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "10", 2024-03-04T09:41:13.467 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.467 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.467 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.467 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:13.467 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.467 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.467 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.468 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.468 INFO:teuthology.orchestra.run.smithi052.stdout: "mirror_snapshot_schedule": { 2024-03-04T09:41:13.468 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "mirror_snapshot_schedule", 2024-03-04T09:41:13.468 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:13.468 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.468 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:13.468 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-03-04T09:41:13.469 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.469 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.469 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.471 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:13.471 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.471 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.472 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.472 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.472 INFO:teuthology.orchestra.run.smithi052.stdout: "trash_purge_schedule": { 2024-03-04T09:41:13.472 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "trash_purge_schedule", 2024-03-04T09:41:13.472 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:13.472 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.472 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:13.472 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-03-04T09:41:13.473 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.473 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.473 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.473 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:13.473 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.473 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.473 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.473 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-03-04T09:41:13.474 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-03-04T09:41:13.474 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.474 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-03-04T09:41:13.474 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "restful", 2024-03-04T09:41:13.474 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-03-04T09:41:13.474 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-03-04T09:41:13.474 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-03-04T09:41:13.474 INFO:teuthology.orchestra.run.smithi052.stdout: "enable_auth": { 2024-03-04T09:41:13.474 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "enable_auth", 2024-03-04T09:41:13.475 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:13.475 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.475 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:13.475 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-03-04T09:41:13.475 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.475 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.475 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.475 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:13.476 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.476 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.476 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.476 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.476 INFO:teuthology.orchestra.run.smithi052.stdout: "key_file": { 2024-03-04T09:41:13.476 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "key_file", 2024-03-04T09:41:13.476 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:13.476 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.477 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:13.477 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-03-04T09:41:13.477 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.477 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.477 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.477 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:13.477 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.477 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.478 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.478 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.478 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-03-04T09:41:13.478 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-03-04T09:41:13.478 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:13.478 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.478 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:13.478 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-03-04T09:41:13.478 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.479 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.479 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-03-04T09:41:13.479 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-03-04T09:41:13.479 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-03-04T09:41:13.479 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-03-04T09:41:13.479 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-03-04T09:41:13.479 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-03-04T09:41:13.479 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-03-04T09:41:13.479 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-03-04T09:41:13.480 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:13.480 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.480 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.480 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.480 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.480 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-03-04T09:41:13.480 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-03-04T09:41:13.480 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:13.481 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.481 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:13.481 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-03-04T09:41:13.481 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.481 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.481 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.481 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:13.482 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.482 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.482 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.482 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.482 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-03-04T09:41:13.482 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-03-04T09:41:13.482 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:13.483 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.483 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:13.483 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-03-04T09:41:13.483 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.483 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.483 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-03-04T09:41:13.483 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-03-04T09:41:13.483 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-03-04T09:41:13.484 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-03-04T09:41:13.484 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-03-04T09:41:13.484 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-03-04T09:41:13.484 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-03-04T09:41:13.484 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-03-04T09:41:13.484 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:13.484 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.484 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.485 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.485 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.485 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-03-04T09:41:13.485 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-03-04T09:41:13.485 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:13.485 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.485 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:13.485 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-03-04T09:41:13.486 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.486 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.486 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.486 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:13.486 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.486 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.486 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.486 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.487 INFO:teuthology.orchestra.run.smithi052.stdout: "server_addr": { 2024-03-04T09:41:13.487 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "server_addr", 2024-03-04T09:41:13.487 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:13.487 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.487 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:13.487 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-03-04T09:41:13.487 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.487 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.487 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.488 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:13.488 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.488 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.488 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.488 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.488 INFO:teuthology.orchestra.run.smithi052.stdout: "server_port": { 2024-03-04T09:41:13.488 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "server_port", 2024-03-04T09:41:13.488 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:13.489 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.489 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:13.489 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-03-04T09:41:13.489 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.489 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.489 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.489 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:13.489 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.490 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.490 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.490 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-03-04T09:41:13.490 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-03-04T09:41:13.490 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.490 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-03-04T09:41:13.490 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "rgw", 2024-03-04T09:41:13.490 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-03-04T09:41:13.491 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-03-04T09:41:13.491 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-03-04T09:41:13.491 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-03-04T09:41:13.491 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-03-04T09:41:13.491 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:13.491 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.491 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:13.491 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-03-04T09:41:13.491 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.492 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.492 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-03-04T09:41:13.492 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-03-04T09:41:13.492 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-03-04T09:41:13.492 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-03-04T09:41:13.492 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-03-04T09:41:13.492 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-03-04T09:41:13.492 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-03-04T09:41:13.492 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-03-04T09:41:13.493 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:13.493 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.493 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.493 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.493 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.493 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-03-04T09:41:13.493 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-03-04T09:41:13.493 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:13.493 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.494 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:13.494 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-03-04T09:41:13.494 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.494 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.494 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.494 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:13.494 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.494 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.494 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.495 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.495 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-03-04T09:41:13.495 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-03-04T09:41:13.495 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:13.495 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.495 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:13.495 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-03-04T09:41:13.495 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.495 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.496 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-03-04T09:41:13.496 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-03-04T09:41:13.496 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-03-04T09:41:13.496 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-03-04T09:41:13.496 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-03-04T09:41:13.496 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-03-04T09:41:13.496 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-03-04T09:41:13.496 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-03-04T09:41:13.497 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:13.497 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.497 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.497 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.497 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.497 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-03-04T09:41:13.497 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-03-04T09:41:13.497 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:13.497 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.498 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:13.498 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-03-04T09:41:13.498 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.498 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.498 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.498 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:13.498 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.498 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.499 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.499 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-03-04T09:41:13.499 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-03-04T09:41:13.499 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.499 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-03-04T09:41:13.499 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "rook", 2024-03-04T09:41:13.499 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-03-04T09:41:13.499 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-03-04T09:41:13.500 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-03-04T09:41:13.500 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-03-04T09:41:13.500 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-03-04T09:41:13.500 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:13.500 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.500 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:13.500 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-03-04T09:41:13.500 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.501 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.501 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-03-04T09:41:13.501 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-03-04T09:41:13.501 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-03-04T09:41:13.501 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-03-04T09:41:13.501 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-03-04T09:41:13.501 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-03-04T09:41:13.501 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-03-04T09:41:13.502 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-03-04T09:41:13.502 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:13.502 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.502 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.502 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.502 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.502 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-03-04T09:41:13.502 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-03-04T09:41:13.503 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:13.503 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.503 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:13.503 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-03-04T09:41:13.503 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.503 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.503 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.503 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:13.503 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.504 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.504 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.504 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.504 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-03-04T09:41:13.504 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-03-04T09:41:13.504 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:13.504 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.504 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:13.505 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-03-04T09:41:13.505 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.505 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.505 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-03-04T09:41:13.505 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-03-04T09:41:13.505 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-03-04T09:41:13.505 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-03-04T09:41:13.506 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-03-04T09:41:13.506 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-03-04T09:41:13.506 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-03-04T09:41:13.506 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-03-04T09:41:13.506 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:13.506 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.506 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.506 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.506 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.507 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-03-04T09:41:13.507 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-03-04T09:41:13.507 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:13.507 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.507 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:13.507 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-03-04T09:41:13.507 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.507 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.508 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.508 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:13.508 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.508 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.508 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.508 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.508 INFO:teuthology.orchestra.run.smithi052.stdout: "storage_class": { 2024-03-04T09:41:13.508 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "storage_class", 2024-03-04T09:41:13.509 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:13.509 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.509 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:13.509 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "local", 2024-03-04T09:41:13.509 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.509 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.509 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.509 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-03-04T09:41:13.510 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.510 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.510 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.510 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-03-04T09:41:13.510 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-03-04T09:41:13.510 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.510 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-03-04T09:41:13.510 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "selftest", 2024-03-04T09:41:13.511 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-03-04T09:41:13.511 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-03-04T09:41:13.511 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-03-04T09:41:13.511 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-03-04T09:41:13.511 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-03-04T09:41:13.511 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:13.511 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.511 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:13.511 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-03-04T09:41:13.512 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.512 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.512 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-03-04T09:41:13.512 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-03-04T09:41:13.512 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-03-04T09:41:13.512 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-03-04T09:41:13.512 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-03-04T09:41:13.513 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-03-04T09:41:13.513 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-03-04T09:41:13.513 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-03-04T09:41:13.513 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:13.513 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.513 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.513 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.513 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.513 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-03-04T09:41:13.514 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-03-04T09:41:13.514 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:13.514 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.514 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:13.514 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-03-04T09:41:13.514 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.514 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.514 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.514 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:13.515 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.515 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.515 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.515 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.515 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-03-04T09:41:13.515 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-03-04T09:41:13.515 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:13.515 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.516 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:13.516 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-03-04T09:41:13.516 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.516 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.516 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-03-04T09:41:13.516 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-03-04T09:41:13.516 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-03-04T09:41:13.517 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-03-04T09:41:13.517 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-03-04T09:41:13.517 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-03-04T09:41:13.517 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-03-04T09:41:13.517 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-03-04T09:41:13.517 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:13.517 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.517 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.518 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.518 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.518 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-03-04T09:41:13.518 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-03-04T09:41:13.518 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:13.518 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.518 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:13.518 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-03-04T09:41:13.519 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.519 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.519 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.519 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:13.519 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.519 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.519 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.519 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.520 INFO:teuthology.orchestra.run.smithi052.stdout: "roption1": { 2024-03-04T09:41:13.520 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "roption1", 2024-03-04T09:41:13.520 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:13.520 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.520 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:13.520 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-03-04T09:41:13.520 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.520 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.521 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.521 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:13.521 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.521 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.521 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.521 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.521 INFO:teuthology.orchestra.run.smithi052.stdout: "roption2": { 2024-03-04T09:41:13.522 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "roption2", 2024-03-04T09:41:13.522 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:13.522 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.522 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:13.522 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "xyz", 2024-03-04T09:41:13.522 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.522 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.522 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.523 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:13.523 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.523 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.523 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.523 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.523 INFO:teuthology.orchestra.run.smithi052.stdout: "rwoption1": { 2024-03-04T09:41:13.523 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "rwoption1", 2024-03-04T09:41:13.524 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:13.524 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.524 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:13.524 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-03-04T09:41:13.524 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.524 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.524 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.525 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:13.525 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.525 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.525 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.525 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.525 INFO:teuthology.orchestra.run.smithi052.stdout: "rwoption2": { 2024-03-04T09:41:13.525 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "rwoption2", 2024-03-04T09:41:13.525 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-03-04T09:41:13.526 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.526 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:13.526 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-03-04T09:41:13.526 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.526 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.526 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.526 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:13.527 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.527 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.527 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.527 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.527 INFO:teuthology.orchestra.run.smithi052.stdout: "rwoption3": { 2024-03-04T09:41:13.527 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "rwoption3", 2024-03-04T09:41:13.527 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "float", 2024-03-04T09:41:13.527 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.527 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:13.528 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-03-04T09:41:13.528 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.528 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.528 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.528 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:13.528 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.528 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.529 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.529 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.529 INFO:teuthology.orchestra.run.smithi052.stdout: "rwoption4": { 2024-03-04T09:41:13.529 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "rwoption4", 2024-03-04T09:41:13.529 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:13.529 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.529 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:13.529 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-03-04T09:41:13.529 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.530 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.530 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.530 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:13.530 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.530 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.530 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.530 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.531 INFO:teuthology.orchestra.run.smithi052.stdout: "rwoption5": { 2024-03-04T09:41:13.531 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "rwoption5", 2024-03-04T09:41:13.531 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:13.531 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.531 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:13.531 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-03-04T09:41:13.531 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.531 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.532 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.532 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:13.532 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.532 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.532 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.532 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.532 INFO:teuthology.orchestra.run.smithi052.stdout: "rwoption6": { 2024-03-04T09:41:13.532 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "rwoption6", 2024-03-04T09:41:13.533 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:13.533 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.533 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:13.533 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-03-04T09:41:13.533 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.533 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.533 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.533 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:13.534 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.534 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.534 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.534 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.534 INFO:teuthology.orchestra.run.smithi052.stdout: "rwoption7": { 2024-03-04T09:41:13.534 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "rwoption7", 2024-03-04T09:41:13.534 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-03-04T09:41:13.534 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.535 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:13.535 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-03-04T09:41:13.535 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "1", 2024-03-04T09:41:13.535 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "42", 2024-03-04T09:41:13.535 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.535 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:13.535 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.535 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.536 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.536 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.536 INFO:teuthology.orchestra.run.smithi052.stdout: "testkey": { 2024-03-04T09:41:13.536 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "testkey", 2024-03-04T09:41:13.536 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:13.536 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.536 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:13.536 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-03-04T09:41:13.537 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.537 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.537 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.537 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:13.537 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.537 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.537 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.537 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.537 INFO:teuthology.orchestra.run.smithi052.stdout: "testlkey": { 2024-03-04T09:41:13.538 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "testlkey", 2024-03-04T09:41:13.538 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:13.538 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.538 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:13.538 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-03-04T09:41:13.538 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.538 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.538 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.539 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:13.539 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.539 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.539 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.539 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.539 INFO:teuthology.orchestra.run.smithi052.stdout: "testnewline": { 2024-03-04T09:41:13.539 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "testnewline", 2024-03-04T09:41:13.539 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:13.540 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.540 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:13.540 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-03-04T09:41:13.540 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.540 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.540 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.540 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:13.540 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.541 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.541 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.541 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-03-04T09:41:13.541 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-03-04T09:41:13.541 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.541 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-03-04T09:41:13.541 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "snap_schedule", 2024-03-04T09:41:13.541 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-03-04T09:41:13.542 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-03-04T09:41:13.542 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-03-04T09:41:13.542 INFO:teuthology.orchestra.run.smithi052.stdout: "allow_m_granularity": { 2024-03-04T09:41:13.542 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "allow_m_granularity", 2024-03-04T09:41:13.542 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:13.542 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.542 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:13.542 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-03-04T09:41:13.542 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.543 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.543 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.543 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "allow minute scheduled snapshots", 2024-03-04T09:41:13.543 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.543 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.543 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.543 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.543 INFO:teuthology.orchestra.run.smithi052.stdout: "dump_on_update": { 2024-03-04T09:41:13.544 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "dump_on_update", 2024-03-04T09:41:13.544 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:13.544 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.544 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:13.544 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-03-04T09:41:13.544 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.544 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.544 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.545 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "dump database to debug log on update", 2024-03-04T09:41:13.545 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.545 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.545 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.545 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.545 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-03-04T09:41:13.545 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-03-04T09:41:13.545 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:13.546 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.546 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:13.546 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-03-04T09:41:13.546 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.546 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.546 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-03-04T09:41:13.546 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-03-04T09:41:13.546 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-03-04T09:41:13.547 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-03-04T09:41:13.547 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-03-04T09:41:13.547 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-03-04T09:41:13.547 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-03-04T09:41:13.547 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-03-04T09:41:13.547 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:13.547 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.547 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.547 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.548 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.548 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-03-04T09:41:13.548 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-03-04T09:41:13.548 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:13.548 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.548 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:13.549 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-03-04T09:41:13.549 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.549 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.549 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.549 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:13.549 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.549 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.549 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.550 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.550 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-03-04T09:41:13.550 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-03-04T09:41:13.550 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:13.550 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.550 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:13.550 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-03-04T09:41:13.550 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.551 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.551 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-03-04T09:41:13.551 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-03-04T09:41:13.551 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-03-04T09:41:13.551 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-03-04T09:41:13.551 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-03-04T09:41:13.551 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-03-04T09:41:13.552 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-03-04T09:41:13.552 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-03-04T09:41:13.552 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:13.552 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.552 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.552 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.553 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.553 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-03-04T09:41:13.553 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-03-04T09:41:13.553 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:13.553 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.553 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:13.553 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-03-04T09:41:13.554 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.554 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.554 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.554 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:13.554 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.554 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.555 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.555 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-03-04T09:41:13.555 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-03-04T09:41:13.555 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.555 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-03-04T09:41:13.555 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "stats", 2024-03-04T09:41:13.555 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-03-04T09:41:13.555 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-03-04T09:41:13.556 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-03-04T09:41:13.556 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-03-04T09:41:13.556 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-03-04T09:41:13.556 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:13.556 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.556 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:13.556 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-03-04T09:41:13.557 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.557 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.557 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-03-04T09:41:13.557 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-03-04T09:41:13.557 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-03-04T09:41:13.557 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-03-04T09:41:13.557 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-03-04T09:41:13.557 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-03-04T09:41:13.557 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-03-04T09:41:13.558 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-03-04T09:41:13.558 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:13.558 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.558 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.558 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.558 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.558 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-03-04T09:41:13.558 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-03-04T09:41:13.559 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:13.559 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.559 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:13.559 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-03-04T09:41:13.559 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.559 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.559 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.559 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:13.560 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.560 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.560 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.560 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.560 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-03-04T09:41:13.560 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-03-04T09:41:13.560 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:13.560 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.561 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:13.561 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-03-04T09:41:13.561 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.561 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.561 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-03-04T09:41:13.561 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-03-04T09:41:13.561 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-03-04T09:41:13.561 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-03-04T09:41:13.561 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-03-04T09:41:13.562 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-03-04T09:41:13.562 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-03-04T09:41:13.562 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-03-04T09:41:13.562 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:13.562 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.562 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.562 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.563 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.563 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-03-04T09:41:13.563 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-03-04T09:41:13.563 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:13.563 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.563 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:13.563 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-03-04T09:41:13.563 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.563 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.564 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.564 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:13.564 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.564 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.564 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.564 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-03-04T09:41:13.564 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-03-04T09:41:13.564 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.565 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-03-04T09:41:13.565 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "status", 2024-03-04T09:41:13.565 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-03-04T09:41:13.565 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-03-04T09:41:13.565 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-03-04T09:41:13.565 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-03-04T09:41:13.565 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-03-04T09:41:13.565 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:13.566 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.566 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:13.566 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-03-04T09:41:13.566 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.566 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.566 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-03-04T09:41:13.566 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-03-04T09:41:13.566 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-03-04T09:41:13.566 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-03-04T09:41:13.567 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-03-04T09:41:13.567 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-03-04T09:41:13.567 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-03-04T09:41:13.567 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-03-04T09:41:13.567 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:13.567 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.567 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.567 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.568 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.568 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-03-04T09:41:13.568 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-03-04T09:41:13.568 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:13.568 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.568 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:13.568 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-03-04T09:41:13.568 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.569 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.569 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.569 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:13.569 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.569 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.569 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.569 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.569 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-03-04T09:41:13.570 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-03-04T09:41:13.570 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:13.570 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.570 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:13.570 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-03-04T09:41:13.570 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.570 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.570 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-03-04T09:41:13.570 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-03-04T09:41:13.571 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-03-04T09:41:13.571 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-03-04T09:41:13.571 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-03-04T09:41:13.571 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-03-04T09:41:13.571 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-03-04T09:41:13.571 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-03-04T09:41:13.571 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:13.572 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.572 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.572 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.572 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.572 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-03-04T09:41:13.572 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-03-04T09:41:13.572 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:13.572 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.573 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:13.573 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-03-04T09:41:13.573 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.573 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.573 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.573 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:13.573 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.573 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.574 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.574 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-03-04T09:41:13.574 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-03-04T09:41:13.574 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.574 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-03-04T09:41:13.574 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "telegraf", 2024-03-04T09:41:13.574 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-03-04T09:41:13.574 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-03-04T09:41:13.574 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-03-04T09:41:13.575 INFO:teuthology.orchestra.run.smithi052.stdout: "address": { 2024-03-04T09:41:13.575 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "address", 2024-03-04T09:41:13.575 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:13.575 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.575 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:13.575 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-03-04T09:41:13.575 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.575 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.576 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.576 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:13.576 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.576 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.576 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.576 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.576 INFO:teuthology.orchestra.run.smithi052.stdout: "interval": { 2024-03-04T09:41:13.576 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "interval", 2024-03-04T09:41:13.576 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "secs", 2024-03-04T09:41:13.577 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.577 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:13.577 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "15", 2024-03-04T09:41:13.577 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.577 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.577 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.577 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:13.577 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.578 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.578 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.578 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.578 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-03-04T09:41:13.578 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-03-04T09:41:13.578 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:13.578 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.578 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:13.578 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-03-04T09:41:13.579 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.579 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.579 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-03-04T09:41:13.579 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-03-04T09:41:13.579 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-03-04T09:41:13.579 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-03-04T09:41:13.579 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-03-04T09:41:13.580 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-03-04T09:41:13.580 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-03-04T09:41:13.580 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-03-04T09:41:13.580 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:13.580 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.580 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.580 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.580 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.580 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-03-04T09:41:13.581 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-03-04T09:41:13.581 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:13.581 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.581 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:13.581 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-03-04T09:41:13.581 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.581 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.581 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.582 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:13.582 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.582 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.582 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.582 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.582 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-03-04T09:41:13.582 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-03-04T09:41:13.582 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:13.582 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.583 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:13.583 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-03-04T09:41:13.583 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.583 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.583 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-03-04T09:41:13.583 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-03-04T09:41:13.583 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-03-04T09:41:13.583 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-03-04T09:41:13.583 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-03-04T09:41:13.584 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-03-04T09:41:13.584 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-03-04T09:41:13.584 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-03-04T09:41:13.584 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:13.584 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.584 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.584 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.584 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.585 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-03-04T09:41:13.585 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-03-04T09:41:13.585 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:13.585 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.585 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:13.585 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-03-04T09:41:13.585 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.585 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.586 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.586 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:13.586 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.586 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.586 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.586 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-03-04T09:41:13.586 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-03-04T09:41:13.586 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.587 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-03-04T09:41:13.587 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "telemetry", 2024-03-04T09:41:13.587 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-03-04T09:41:13.587 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-03-04T09:41:13.587 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-03-04T09:41:13.587 INFO:teuthology.orchestra.run.smithi052.stdout: "channel_basic": { 2024-03-04T09:41:13.587 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "channel_basic", 2024-03-04T09:41:13.587 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:13.587 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.588 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:13.588 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-03-04T09:41:13.588 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.588 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.588 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.588 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Share basic cluster information (size, version)", 2024-03-04T09:41:13.588 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.589 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.589 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.589 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.589 INFO:teuthology.orchestra.run.smithi052.stdout: "channel_crash": { 2024-03-04T09:41:13.589 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "channel_crash", 2024-03-04T09:41:13.589 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:13.589 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.589 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:13.590 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-03-04T09:41:13.590 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.590 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.590 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.590 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-03-04T09:41:13.590 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.590 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.590 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.590 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.591 INFO:teuthology.orchestra.run.smithi052.stdout: "channel_device": { 2024-03-04T09:41:13.591 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "channel_device", 2024-03-04T09:41:13.591 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:13.591 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.591 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:13.591 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-03-04T09:41:13.591 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.591 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.591 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.592 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-03-04T09:41:13.592 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.592 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.592 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.592 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.592 INFO:teuthology.orchestra.run.smithi052.stdout: "channel_ident": { 2024-03-04T09:41:13.592 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "channel_ident", 2024-03-04T09:41:13.592 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:13.592 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.593 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:13.593 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-03-04T09:41:13.593 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.593 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.593 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.593 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-03-04T09:41:13.593 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.593 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.594 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.594 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.594 INFO:teuthology.orchestra.run.smithi052.stdout: "channel_perf": { 2024-03-04T09:41:13.594 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "channel_perf", 2024-03-04T09:41:13.594 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:13.594 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.594 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:13.594 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-03-04T09:41:13.595 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.595 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.595 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.595 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Share various performance metrics of a cluster", 2024-03-04T09:41:13.595 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.595 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.595 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.595 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.595 INFO:teuthology.orchestra.run.smithi052.stdout: "contact": { 2024-03-04T09:41:13.596 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "contact", 2024-03-04T09:41:13.596 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:13.596 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.596 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:13.596 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-03-04T09:41:13.596 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.596 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.597 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.597 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:13.597 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.597 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.597 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.597 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.597 INFO:teuthology.orchestra.run.smithi052.stdout: "description": { 2024-03-04T09:41:13.597 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "description", 2024-03-04T09:41:13.597 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:13.598 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.598 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:13.598 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-03-04T09:41:13.598 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.598 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.598 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.598 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:13.598 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.599 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.599 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.599 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.599 INFO:teuthology.orchestra.run.smithi052.stdout: "device_url": { 2024-03-04T09:41:13.599 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "device_url", 2024-03-04T09:41:13.599 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:13.599 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.599 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:13.600 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-03-04T09:41:13.600 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.600 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.600 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.600 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:13.600 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.600 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.600 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.600 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.601 INFO:teuthology.orchestra.run.smithi052.stdout: "enabled": { 2024-03-04T09:41:13.601 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "enabled", 2024-03-04T09:41:13.601 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:13.601 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.601 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:13.601 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-03-04T09:41:13.601 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.601 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.602 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.602 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:13.602 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.602 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.602 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.602 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.602 INFO:teuthology.orchestra.run.smithi052.stdout: "interval": { 2024-03-04T09:41:13.602 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "interval", 2024-03-04T09:41:13.603 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-03-04T09:41:13.603 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.603 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:13.603 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "24", 2024-03-04T09:41:13.603 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "8", 2024-03-04T09:41:13.603 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.603 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.603 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:13.603 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.604 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.604 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.604 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.604 INFO:teuthology.orchestra.run.smithi052.stdout: "last_opt_revision": { 2024-03-04T09:41:13.604 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "last_opt_revision", 2024-03-04T09:41:13.604 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-03-04T09:41:13.604 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.604 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:13.605 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "1", 2024-03-04T09:41:13.605 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.605 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.605 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.605 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:13.605 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.605 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.605 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.606 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.606 INFO:teuthology.orchestra.run.smithi052.stdout: "leaderboard": { 2024-03-04T09:41:13.606 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "leaderboard", 2024-03-04T09:41:13.606 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:13.606 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.606 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:13.606 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-03-04T09:41:13.606 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.607 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.607 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.607 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:13.607 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.607 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.607 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.607 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.607 INFO:teuthology.orchestra.run.smithi052.stdout: "leaderboard_description": { 2024-03-04T09:41:13.607 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "leaderboard_description", 2024-03-04T09:41:13.608 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:13.608 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.608 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:13.608 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-03-04T09:41:13.608 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.608 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.608 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.609 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:13.609 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.609 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.609 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.609 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.609 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-03-04T09:41:13.609 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-03-04T09:41:13.609 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:13.609 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.610 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:13.610 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-03-04T09:41:13.610 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.610 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.610 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-03-04T09:41:13.610 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-03-04T09:41:13.610 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-03-04T09:41:13.610 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-03-04T09:41:13.611 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-03-04T09:41:13.611 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-03-04T09:41:13.611 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-03-04T09:41:13.611 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-03-04T09:41:13.611 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:13.611 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.611 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.611 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.611 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.612 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-03-04T09:41:13.612 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-03-04T09:41:13.612 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:13.612 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.612 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:13.612 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-03-04T09:41:13.612 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.612 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.613 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.613 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:13.613 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.613 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.613 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.613 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.613 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-03-04T09:41:13.613 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-03-04T09:41:13.613 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:13.614 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.614 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:13.614 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-03-04T09:41:13.614 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.614 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.614 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-03-04T09:41:13.614 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-03-04T09:41:13.614 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-03-04T09:41:13.614 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-03-04T09:41:13.615 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-03-04T09:41:13.615 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-03-04T09:41:13.615 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-03-04T09:41:13.615 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-03-04T09:41:13.615 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:13.615 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.615 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.615 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.616 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.616 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-03-04T09:41:13.616 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-03-04T09:41:13.616 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:13.616 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.616 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:13.616 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-03-04T09:41:13.616 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.617 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.617 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.617 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:13.617 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.617 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.617 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.617 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.617 INFO:teuthology.orchestra.run.smithi052.stdout: "organization": { 2024-03-04T09:41:13.618 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "organization", 2024-03-04T09:41:13.618 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:13.618 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.618 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:13.618 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-03-04T09:41:13.618 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.618 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.618 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.619 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:13.619 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.619 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.619 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.619 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.619 INFO:teuthology.orchestra.run.smithi052.stdout: "proxy": { 2024-03-04T09:41:13.619 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "proxy", 2024-03-04T09:41:13.619 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:13.620 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.620 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:13.620 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-03-04T09:41:13.620 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.620 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.620 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.620 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:13.620 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.621 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.621 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.621 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.621 INFO:teuthology.orchestra.run.smithi052.stdout: "url": { 2024-03-04T09:41:13.621 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "url", 2024-03-04T09:41:13.621 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:13.621 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.621 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:13.621 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-03-04T09:41:13.622 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.622 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.622 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.622 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:13.622 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.622 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.622 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.622 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-03-04T09:41:13.623 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-03-04T09:41:13.623 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.623 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-03-04T09:41:13.623 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "test_orchestrator", 2024-03-04T09:41:13.623 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-03-04T09:41:13.623 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-03-04T09:41:13.623 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-03-04T09:41:13.623 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-03-04T09:41:13.623 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-03-04T09:41:13.624 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:13.624 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.624 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:13.624 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-03-04T09:41:13.624 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.624 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.624 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-03-04T09:41:13.624 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-03-04T09:41:13.625 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-03-04T09:41:13.625 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-03-04T09:41:13.625 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-03-04T09:41:13.625 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-03-04T09:41:13.625 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-03-04T09:41:13.625 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-03-04T09:41:13.625 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:13.626 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.626 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.626 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.626 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.626 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-03-04T09:41:13.626 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-03-04T09:41:13.626 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:13.626 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.626 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:13.627 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-03-04T09:41:13.627 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.627 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.627 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.627 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:13.627 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.627 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.627 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.628 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.628 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-03-04T09:41:13.628 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-03-04T09:41:13.628 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:13.628 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.628 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:13.628 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-03-04T09:41:13.628 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.629 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.629 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-03-04T09:41:13.629 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-03-04T09:41:13.629 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-03-04T09:41:13.629 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-03-04T09:41:13.629 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-03-04T09:41:13.629 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-03-04T09:41:13.629 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-03-04T09:41:13.630 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-03-04T09:41:13.630 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:13.630 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.630 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.630 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.630 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.630 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-03-04T09:41:13.630 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-03-04T09:41:13.630 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:13.631 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.631 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:13.631 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-03-04T09:41:13.631 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.631 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.631 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.631 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:13.631 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.631 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.632 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.632 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-03-04T09:41:13.632 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-03-04T09:41:13.632 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.632 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-03-04T09:41:13.632 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "volumes", 2024-03-04T09:41:13.632 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-03-04T09:41:13.632 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-03-04T09:41:13.633 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-03-04T09:41:13.633 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-03-04T09:41:13.633 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-03-04T09:41:13.633 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:13.633 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.633 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:13.633 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-03-04T09:41:13.633 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.634 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.634 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-03-04T09:41:13.634 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-03-04T09:41:13.634 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-03-04T09:41:13.634 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-03-04T09:41:13.634 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-03-04T09:41:13.634 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-03-04T09:41:13.634 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-03-04T09:41:13.634 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-03-04T09:41:13.635 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:13.635 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.635 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.635 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.635 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.635 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-03-04T09:41:13.635 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-03-04T09:41:13.635 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:13.636 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.636 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:13.636 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-03-04T09:41:13.636 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.636 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.636 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.636 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:13.636 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.637 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.637 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.637 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.637 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-03-04T09:41:13.637 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-03-04T09:41:13.637 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:13.637 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.637 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:13.637 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-03-04T09:41:13.638 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.638 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.638 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-03-04T09:41:13.638 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-03-04T09:41:13.638 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-03-04T09:41:13.638 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-03-04T09:41:13.638 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-03-04T09:41:13.638 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-03-04T09:41:13.639 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-03-04T09:41:13.639 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-03-04T09:41:13.639 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:13.639 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.639 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.639 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.639 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.639 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-03-04T09:41:13.640 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-03-04T09:41:13.640 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:13.640 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.640 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:13.640 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-03-04T09:41:13.640 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.640 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.640 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.641 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:13.641 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.641 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.641 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.641 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.641 INFO:teuthology.orchestra.run.smithi052.stdout: "max_concurrent_clones": { 2024-03-04T09:41:13.641 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "max_concurrent_clones", 2024-03-04T09:41:13.641 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-03-04T09:41:13.641 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.642 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:13.642 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "4", 2024-03-04T09:41:13.642 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.642 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.642 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.642 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Number of asynchronous cloner threads", 2024-03-04T09:41:13.642 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.642 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.643 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.643 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.643 INFO:teuthology.orchestra.run.smithi052.stdout: "periodic_async_work": { 2024-03-04T09:41:13.643 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "periodic_async_work", 2024-03-04T09:41:13.643 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:13.643 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.643 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:13.643 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-03-04T09:41:13.643 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.644 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.644 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.644 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Periodically check for async work", 2024-03-04T09:41:13.644 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.644 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.644 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.644 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.644 INFO:teuthology.orchestra.run.smithi052.stdout: "snapshot_clone_delay": { 2024-03-04T09:41:13.645 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "snapshot_clone_delay", 2024-03-04T09:41:13.645 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-03-04T09:41:13.645 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.645 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:13.645 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "0", 2024-03-04T09:41:13.645 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.645 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.645 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.646 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-03-04T09:41:13.646 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.646 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.646 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.646 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.646 INFO:teuthology.orchestra.run.smithi052.stdout: "snapshot_clone_no_wait": { 2024-03-04T09:41:13.646 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "snapshot_clone_no_wait", 2024-03-04T09:41:13.646 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:13.647 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.647 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:13.647 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-03-04T09:41:13.647 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.647 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.647 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.647 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-03-04T09:41:13.647 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.647 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.648 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.648 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-03-04T09:41:13.648 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-03-04T09:41:13.648 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.648 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-03-04T09:41:13.648 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "zabbix", 2024-03-04T09:41:13.648 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-03-04T09:41:13.648 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-03-04T09:41:13.649 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-03-04T09:41:13.649 INFO:teuthology.orchestra.run.smithi052.stdout: "discovery_interval": { 2024-03-04T09:41:13.649 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "discovery_interval", 2024-03-04T09:41:13.649 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "uint", 2024-03-04T09:41:13.649 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.649 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:13.649 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "100", 2024-03-04T09:41:13.650 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.650 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.650 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.650 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:13.650 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.650 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.650 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.650 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.650 INFO:teuthology.orchestra.run.smithi052.stdout: "identifier": { 2024-03-04T09:41:13.651 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "identifier", 2024-03-04T09:41:13.651 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:13.651 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.651 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:13.651 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-03-04T09:41:13.651 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.651 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.651 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.652 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:13.652 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.652 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.652 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.652 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.652 INFO:teuthology.orchestra.run.smithi052.stdout: "interval": { 2024-03-04T09:41:13.652 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "interval", 2024-03-04T09:41:13.652 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "secs", 2024-03-04T09:41:13.653 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.653 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:13.653 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "60", 2024-03-04T09:41:13.653 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.653 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.653 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.653 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:13.653 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.653 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.654 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.654 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.654 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-03-04T09:41:13.654 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-03-04T09:41:13.654 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:13.654 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.654 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:13.654 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-03-04T09:41:13.655 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.655 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.655 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-03-04T09:41:13.655 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-03-04T09:41:13.655 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-03-04T09:41:13.655 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-03-04T09:41:13.655 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-03-04T09:41:13.655 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-03-04T09:41:13.656 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-03-04T09:41:13.656 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-03-04T09:41:13.656 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:13.656 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.656 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.656 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.656 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.657 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-03-04T09:41:13.657 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-03-04T09:41:13.657 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:13.657 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.657 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:13.657 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-03-04T09:41:13.657 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.657 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.658 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.658 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:13.658 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.658 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.658 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.658 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.658 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-03-04T09:41:13.658 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-03-04T09:41:13.659 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:13.659 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.659 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:13.659 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-03-04T09:41:13.659 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.659 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.659 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-03-04T09:41:13.659 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-03-04T09:41:13.660 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-03-04T09:41:13.660 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-03-04T09:41:13.660 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-03-04T09:41:13.660 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-03-04T09:41:13.660 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-03-04T09:41:13.660 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-03-04T09:41:13.660 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:13.660 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.661 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.661 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.661 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.661 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-03-04T09:41:13.661 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-03-04T09:41:13.661 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:13.661 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.661 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:13.661 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-03-04T09:41:13.662 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.662 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.662 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.662 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:13.662 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.662 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.662 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.662 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.662 INFO:teuthology.orchestra.run.smithi052.stdout: "zabbix_host": { 2024-03-04T09:41:13.663 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "zabbix_host", 2024-03-04T09:41:13.663 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:13.663 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.663 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:13.663 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-03-04T09:41:13.663 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.663 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.663 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.664 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:13.664 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.664 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.664 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.664 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.664 INFO:teuthology.orchestra.run.smithi052.stdout: "zabbix_port": { 2024-03-04T09:41:13.664 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "zabbix_port", 2024-03-04T09:41:13.664 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-03-04T09:41:13.665 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.665 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:13.665 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "10051", 2024-03-04T09:41:13.665 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.665 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.665 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.665 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:13.665 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.665 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.666 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.666 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.666 INFO:teuthology.orchestra.run.smithi052.stdout: "zabbix_sender": { 2024-03-04T09:41:13.666 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "zabbix_sender", 2024-03-04T09:41:13.666 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:13.666 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.666 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:13.667 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-03-04T09:41:13.667 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.667 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.667 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.667 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:13.667 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.667 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.667 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.668 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-03-04T09:41:13.668 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-03-04T09:41:13.668 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-03-04T09:41:13.668 INFO:teuthology.orchestra.run.smithi052.stdout: ] 2024-03-04T09:41:13.668 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.668 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-03-04T09:41:13.668 INFO:teuthology.orchestra.run.smithi052.stdout: "gid": 25331, 2024-03-04T09:41:13.668 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "y", 2024-03-04T09:41:13.669 INFO:teuthology.orchestra.run.smithi052.stdout: "mgr_features": 4540701547738038271, 2024-03-04T09:41:13.669 INFO:teuthology.orchestra.run.smithi052.stdout: "available_modules": [ 2024-03-04T09:41:13.669 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-03-04T09:41:13.669 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "alerts", 2024-03-04T09:41:13.669 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-03-04T09:41:13.669 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-03-04T09:41:13.669 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-03-04T09:41:13.670 INFO:teuthology.orchestra.run.smithi052.stdout: "interval": { 2024-03-04T09:41:13.670 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "interval", 2024-03-04T09:41:13.670 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "secs", 2024-03-04T09:41:13.670 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.670 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:13.670 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "60", 2024-03-04T09:41:13.670 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.670 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.671 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.671 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "How frequently to reexamine health status", 2024-03-04T09:41:13.671 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.671 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.671 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.671 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.671 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-03-04T09:41:13.671 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-03-04T09:41:13.672 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:13.672 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.672 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:13.672 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-03-04T09:41:13.672 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.672 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.672 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-03-04T09:41:13.673 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-03-04T09:41:13.673 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-03-04T09:41:13.673 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-03-04T09:41:13.673 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-03-04T09:41:13.673 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-03-04T09:41:13.673 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-03-04T09:41:13.673 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-03-04T09:41:13.673 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:13.674 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.674 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.674 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.674 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.674 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-03-04T09:41:13.674 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-03-04T09:41:13.674 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:13.674 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.675 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:13.675 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-03-04T09:41:13.675 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.675 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.675 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.675 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:13.675 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.675 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.675 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.676 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.676 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-03-04T09:41:13.676 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-03-04T09:41:13.676 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:13.676 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.676 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:13.676 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-03-04T09:41:13.677 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.677 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.677 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-03-04T09:41:13.677 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-03-04T09:41:13.677 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-03-04T09:41:13.677 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-03-04T09:41:13.677 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-03-04T09:41:13.677 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-03-04T09:41:13.678 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-03-04T09:41:13.678 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-03-04T09:41:13.678 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:13.678 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.678 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.678 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.678 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.678 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-03-04T09:41:13.678 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-03-04T09:41:13.679 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:13.679 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.679 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:13.679 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-03-04T09:41:13.679 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.679 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.679 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.679 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:13.680 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.680 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.680 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.680 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.680 INFO:teuthology.orchestra.run.smithi052.stdout: "smtp_destination": { 2024-03-04T09:41:13.680 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "smtp_destination", 2024-03-04T09:41:13.680 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:13.680 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.681 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:13.681 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-03-04T09:41:13.681 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.681 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.681 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.681 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Email address to send alerts to", 2024-03-04T09:41:13.681 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.681 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.682 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.682 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.682 INFO:teuthology.orchestra.run.smithi052.stdout: "smtp_from_name": { 2024-03-04T09:41:13.682 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "smtp_from_name", 2024-03-04T09:41:13.682 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:13.682 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.682 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:13.682 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "Ceph", 2024-03-04T09:41:13.683 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.683 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.683 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.683 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Email From: name", 2024-03-04T09:41:13.683 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.683 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.683 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.683 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.684 INFO:teuthology.orchestra.run.smithi052.stdout: "smtp_host": { 2024-03-04T09:41:13.684 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "smtp_host", 2024-03-04T09:41:13.684 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:13.684 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.684 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:13.684 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-03-04T09:41:13.684 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.684 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.684 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.685 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "SMTP server", 2024-03-04T09:41:13.685 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.685 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.685 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.685 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.685 INFO:teuthology.orchestra.run.smithi052.stdout: "smtp_password": { 2024-03-04T09:41:13.685 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "smtp_password", 2024-03-04T09:41:13.685 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:13.686 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.686 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:13.686 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-03-04T09:41:13.686 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.686 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.686 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.686 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Password to authenticate with", 2024-03-04T09:41:13.686 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.686 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.687 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.687 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.687 INFO:teuthology.orchestra.run.smithi052.stdout: "smtp_port": { 2024-03-04T09:41:13.687 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "smtp_port", 2024-03-04T09:41:13.687 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-03-04T09:41:13.687 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.687 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:13.687 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "465", 2024-03-04T09:41:13.688 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.688 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.688 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.688 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "SMTP port", 2024-03-04T09:41:13.688 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.688 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.688 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.688 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.689 INFO:teuthology.orchestra.run.smithi052.stdout: "smtp_sender": { 2024-03-04T09:41:13.689 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "smtp_sender", 2024-03-04T09:41:13.689 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:13.689 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.689 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:13.689 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-03-04T09:41:13.689 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.689 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.690 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.690 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "SMTP envelope sender", 2024-03-04T09:41:13.690 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.690 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.690 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.690 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.690 INFO:teuthology.orchestra.run.smithi052.stdout: "smtp_ssl": { 2024-03-04T09:41:13.690 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "smtp_ssl", 2024-03-04T09:41:13.691 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:13.691 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.691 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:13.691 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-03-04T09:41:13.691 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.691 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.691 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.691 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Use SSL to connect to SMTP server", 2024-03-04T09:41:13.692 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.692 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.692 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.692 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.692 INFO:teuthology.orchestra.run.smithi052.stdout: "smtp_user": { 2024-03-04T09:41:13.692 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "smtp_user", 2024-03-04T09:41:13.692 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:13.692 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.693 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:13.693 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-03-04T09:41:13.693 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.693 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.693 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.693 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "User to authenticate as", 2024-03-04T09:41:13.693 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.693 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.694 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.694 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-03-04T09:41:13.694 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-03-04T09:41:13.694 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.694 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-03-04T09:41:13.694 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "balancer", 2024-03-04T09:41:13.694 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-03-04T09:41:13.694 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-03-04T09:41:13.695 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-03-04T09:41:13.695 INFO:teuthology.orchestra.run.smithi052.stdout: "active": { 2024-03-04T09:41:13.695 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "active", 2024-03-04T09:41:13.695 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:13.695 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.695 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:13.695 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-03-04T09:41:13.695 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.695 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.696 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.696 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "automatically balance PGs across cluster", 2024-03-04T09:41:13.696 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.696 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.696 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.696 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.696 INFO:teuthology.orchestra.run.smithi052.stdout: "begin_time": { 2024-03-04T09:41:13.696 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "begin_time", 2024-03-04T09:41:13.697 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:13.697 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.697 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:13.697 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "0000", 2024-03-04T09:41:13.697 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.697 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.697 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.697 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "beginning time of day to automatically balance", 2024-03-04T09:41:13.698 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-03-04T09:41:13.698 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.698 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.698 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.698 INFO:teuthology.orchestra.run.smithi052.stdout: "begin_weekday": { 2024-03-04T09:41:13.698 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "begin_weekday", 2024-03-04T09:41:13.698 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "uint", 2024-03-04T09:41:13.698 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.698 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:13.699 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "0", 2024-03-04T09:41:13.699 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "0", 2024-03-04T09:41:13.699 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "6", 2024-03-04T09:41:13.699 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.699 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-03-04T09:41:13.699 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-03-04T09:41:13.699 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.699 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.699 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.700 INFO:teuthology.orchestra.run.smithi052.stdout: "crush_compat_max_iterations": { 2024-03-04T09:41:13.700 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "crush_compat_max_iterations", 2024-03-04T09:41:13.700 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "uint", 2024-03-04T09:41:13.700 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.700 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:13.700 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "25", 2024-03-04T09:41:13.700 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "1", 2024-03-04T09:41:13.700 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "250", 2024-03-04T09:41:13.701 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.701 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-03-04T09:41:13.701 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.701 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.701 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.701 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.701 INFO:teuthology.orchestra.run.smithi052.stdout: "crush_compat_metrics": { 2024-03-04T09:41:13.701 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "crush_compat_metrics", 2024-03-04T09:41:13.702 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:13.702 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.702 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:13.702 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "pgs,objects,bytes", 2024-03-04T09:41:13.702 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.702 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.702 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.702 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-03-04T09:41:13.702 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-03-04T09:41:13.703 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.703 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.703 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.703 INFO:teuthology.orchestra.run.smithi052.stdout: "crush_compat_step": { 2024-03-04T09:41:13.703 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "crush_compat_step", 2024-03-04T09:41:13.703 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "float", 2024-03-04T09:41:13.703 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.703 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:13.704 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "0.5", 2024-03-04T09:41:13.704 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "0.001", 2024-03-04T09:41:13.704 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "0.999", 2024-03-04T09:41:13.704 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.704 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "aggressiveness of optimization", 2024-03-04T09:41:13.704 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-03-04T09:41:13.704 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.704 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.705 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.705 INFO:teuthology.orchestra.run.smithi052.stdout: "end_time": { 2024-03-04T09:41:13.705 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "end_time", 2024-03-04T09:41:13.705 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:13.705 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.705 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:13.705 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "2359", 2024-03-04T09:41:13.705 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.705 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.706 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.706 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "ending time of day to automatically balance", 2024-03-04T09:41:13.706 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-03-04T09:41:13.706 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.706 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.706 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.706 INFO:teuthology.orchestra.run.smithi052.stdout: "end_weekday": { 2024-03-04T09:41:13.706 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "end_weekday", 2024-03-04T09:41:13.707 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "uint", 2024-03-04T09:41:13.707 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.707 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:13.707 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "0", 2024-03-04T09:41:13.707 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "0", 2024-03-04T09:41:13.707 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "6", 2024-03-04T09:41:13.707 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.707 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-03-04T09:41:13.708 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-03-04T09:41:13.708 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.708 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.708 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.708 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-03-04T09:41:13.708 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-03-04T09:41:13.708 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:13.708 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.709 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:13.709 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-03-04T09:41:13.709 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.709 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.709 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-03-04T09:41:13.709 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-03-04T09:41:13.709 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-03-04T09:41:13.709 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-03-04T09:41:13.709 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-03-04T09:41:13.710 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-03-04T09:41:13.710 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-03-04T09:41:13.710 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-03-04T09:41:13.710 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:13.710 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.710 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.710 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.710 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.711 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-03-04T09:41:13.711 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-03-04T09:41:13.711 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:13.711 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.711 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:13.711 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-03-04T09:41:13.711 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.711 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.711 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.712 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:13.712 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.712 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.712 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.712 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.712 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-03-04T09:41:13.712 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-03-04T09:41:13.712 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:13.713 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.713 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:13.713 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-03-04T09:41:13.713 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.713 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.713 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-03-04T09:41:13.713 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-03-04T09:41:13.713 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-03-04T09:41:13.713 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-03-04T09:41:13.714 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-03-04T09:41:13.714 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-03-04T09:41:13.714 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-03-04T09:41:13.714 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-03-04T09:41:13.714 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:13.714 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.714 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.715 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.715 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.715 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-03-04T09:41:13.715 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-03-04T09:41:13.715 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:13.715 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.715 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:13.715 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-03-04T09:41:13.715 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.716 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.716 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.716 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:13.716 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.716 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.716 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.716 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.716 INFO:teuthology.orchestra.run.smithi052.stdout: "min_score": { 2024-03-04T09:41:13.717 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "min_score", 2024-03-04T09:41:13.717 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "float", 2024-03-04T09:41:13.717 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.717 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:13.717 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "0", 2024-03-04T09:41:13.717 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.717 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.718 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.718 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-03-04T09:41:13.718 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.718 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.718 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.718 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.718 INFO:teuthology.orchestra.run.smithi052.stdout: "mode": { 2024-03-04T09:41:13.718 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "mode", 2024-03-04T09:41:13.719 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:13.719 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.719 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:13.719 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "upmap", 2024-03-04T09:41:13.719 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.719 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.719 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-03-04T09:41:13.719 INFO:teuthology.orchestra.run.smithi052.stdout: "crush-compat", 2024-03-04T09:41:13.720 INFO:teuthology.orchestra.run.smithi052.stdout: "none", 2024-03-04T09:41:13.720 INFO:teuthology.orchestra.run.smithi052.stdout: "read", 2024-03-04T09:41:13.720 INFO:teuthology.orchestra.run.smithi052.stdout: "upmap", 2024-03-04T09:41:13.720 INFO:teuthology.orchestra.run.smithi052.stdout: "upmap-read" 2024-03-04T09:41:13.720 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-03-04T09:41:13.720 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Balancer mode", 2024-03-04T09:41:13.720 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.720 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.720 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.721 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.721 INFO:teuthology.orchestra.run.smithi052.stdout: "pool_ids": { 2024-03-04T09:41:13.721 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "pool_ids", 2024-03-04T09:41:13.721 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:13.721 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.721 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:13.721 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-03-04T09:41:13.722 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.722 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.722 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.722 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-03-04T09:41:13.722 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.722 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.722 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.722 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.723 INFO:teuthology.orchestra.run.smithi052.stdout: "sleep_interval": { 2024-03-04T09:41:13.723 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "sleep_interval", 2024-03-04T09:41:13.723 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "secs", 2024-03-04T09:41:13.723 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.723 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:13.723 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "60", 2024-03-04T09:41:13.723 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.723 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.724 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.724 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-03-04T09:41:13.724 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.724 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.724 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.724 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.724 INFO:teuthology.orchestra.run.smithi052.stdout: "upmap_max_deviation": { 2024-03-04T09:41:13.724 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "upmap_max_deviation", 2024-03-04T09:41:13.725 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-03-04T09:41:13.725 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.725 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:13.725 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "5", 2024-03-04T09:41:13.725 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "1", 2024-03-04T09:41:13.725 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.725 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.725 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "deviation below which no optimization is attempted", 2024-03-04T09:41:13.726 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-03-04T09:41:13.726 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.726 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.726 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.726 INFO:teuthology.orchestra.run.smithi052.stdout: "upmap_max_optimizations": { 2024-03-04T09:41:13.726 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "upmap_max_optimizations", 2024-03-04T09:41:13.726 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "uint", 2024-03-04T09:41:13.726 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.727 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:13.727 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "10", 2024-03-04T09:41:13.727 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.727 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.727 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.727 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-03-04T09:41:13.727 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.727 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.727 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.728 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-03-04T09:41:13.728 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-03-04T09:41:13.728 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.728 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-03-04T09:41:13.728 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "cephadm", 2024-03-04T09:41:13.728 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-03-04T09:41:13.728 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-03-04T09:41:13.728 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-03-04T09:41:13.729 INFO:teuthology.orchestra.run.smithi052.stdout: "agent_down_multiplier": { 2024-03-04T09:41:13.729 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "agent_down_multiplier", 2024-03-04T09:41:13.729 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "float", 2024-03-04T09:41:13.729 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.729 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:13.729 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "3.0", 2024-03-04T09:41:13.729 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.729 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.730 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.730 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-03-04T09:41:13.730 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.730 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.730 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.730 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.730 INFO:teuthology.orchestra.run.smithi052.stdout: "agent_refresh_rate": { 2024-03-04T09:41:13.730 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "agent_refresh_rate", 2024-03-04T09:41:13.731 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "secs", 2024-03-04T09:41:13.731 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.731 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:13.731 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "20", 2024-03-04T09:41:13.731 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.731 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.731 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.731 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-03-04T09:41:13.731 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.732 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.732 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.732 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.732 INFO:teuthology.orchestra.run.smithi052.stdout: "agent_starting_port": { 2024-03-04T09:41:13.732 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "agent_starting_port", 2024-03-04T09:41:13.732 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-03-04T09:41:13.732 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.732 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:13.732 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "4721", 2024-03-04T09:41:13.733 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.733 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.733 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.733 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-03-04T09:41:13.733 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.733 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.733 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.733 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.734 INFO:teuthology.orchestra.run.smithi052.stdout: "allow_ptrace": { 2024-03-04T09:41:13.734 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "allow_ptrace", 2024-03-04T09:41:13.734 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:13.734 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.734 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:13.734 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-03-04T09:41:13.734 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.734 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.735 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.735 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-03-04T09:41:13.735 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-03-04T09:41:13.735 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.735 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.735 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.735 INFO:teuthology.orchestra.run.smithi052.stdout: "autotune_interval": { 2024-03-04T09:41:13.735 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "autotune_interval", 2024-03-04T09:41:13.735 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "secs", 2024-03-04T09:41:13.736 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.736 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:13.736 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "600", 2024-03-04T09:41:13.736 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.736 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.736 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.736 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "how frequently to autotune daemon memory", 2024-03-04T09:41:13.736 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.737 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.737 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.737 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.737 INFO:teuthology.orchestra.run.smithi052.stdout: "autotune_memory_target_ratio": { 2024-03-04T09:41:13.737 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "autotune_memory_target_ratio", 2024-03-04T09:41:13.737 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "float", 2024-03-04T09:41:13.737 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.738 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:13.738 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "0.7", 2024-03-04T09:41:13.738 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.738 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.738 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.738 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-03-04T09:41:13.738 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.738 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.739 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.739 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.739 INFO:teuthology.orchestra.run.smithi052.stdout: "cephadm_log_destination": { 2024-03-04T09:41:13.739 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "cephadm_log_destination", 2024-03-04T09:41:13.739 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:13.739 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.739 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:13.739 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-03-04T09:41:13.740 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.740 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.740 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-03-04T09:41:13.740 INFO:teuthology.orchestra.run.smithi052.stdout: "file", 2024-03-04T09:41:13.740 INFO:teuthology.orchestra.run.smithi052.stdout: "file,syslog", 2024-03-04T09:41:13.740 INFO:teuthology.orchestra.run.smithi052.stdout: "syslog" 2024-03-04T09:41:13.740 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-03-04T09:41:13.740 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-03-04T09:41:13.741 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.741 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.741 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.741 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.741 INFO:teuthology.orchestra.run.smithi052.stdout: "cgroups_split": { 2024-03-04T09:41:13.741 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "cgroups_split", 2024-03-04T09:41:13.741 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:13.741 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.742 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:13.742 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-03-04T09:41:13.742 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.742 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.742 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.742 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-03-04T09:41:13.742 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.742 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.742 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.743 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.743 INFO:teuthology.orchestra.run.smithi052.stdout: "config_checks_enabled": { 2024-03-04T09:41:13.743 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "config_checks_enabled", 2024-03-04T09:41:13.743 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:13.743 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.743 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:13.743 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-03-04T09:41:13.743 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.744 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.744 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.744 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-03-04T09:41:13.744 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.744 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.744 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.744 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.744 INFO:teuthology.orchestra.run.smithi052.stdout: "config_dashboard": { 2024-03-04T09:41:13.745 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "config_dashboard", 2024-03-04T09:41:13.745 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:13.745 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.745 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:13.745 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-03-04T09:41:13.745 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.745 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.745 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.746 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-03-04T09:41:13.746 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.746 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.746 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.746 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.746 INFO:teuthology.orchestra.run.smithi052.stdout: "container_image_alertmanager": { 2024-03-04T09:41:13.746 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "container_image_alertmanager", 2024-03-04T09:41:13.746 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:13.746 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.747 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:13.747 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-03-04T09:41:13.747 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.747 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.747 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.747 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Prometheus container image", 2024-03-04T09:41:13.747 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.747 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.748 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.748 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.748 INFO:teuthology.orchestra.run.smithi052.stdout: "container_image_base": { 2024-03-04T09:41:13.748 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "container_image_base", 2024-03-04T09:41:13.748 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:13.748 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.748 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:13.748 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "quay.io/ceph/ceph", 2024-03-04T09:41:13.749 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.749 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.749 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.749 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Container image name, without the tag", 2024-03-04T09:41:13.749 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.749 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.749 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.749 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.750 INFO:teuthology.orchestra.run.smithi052.stdout: "container_image_elasticsearch": { 2024-03-04T09:41:13.750 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "container_image_elasticsearch", 2024-03-04T09:41:13.750 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:13.750 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.750 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:13.750 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-03-04T09:41:13.750 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.750 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.751 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.751 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "elasticsearch container image", 2024-03-04T09:41:13.751 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.751 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.751 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.751 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.751 INFO:teuthology.orchestra.run.smithi052.stdout: "container_image_grafana": { 2024-03-04T09:41:13.751 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "container_image_grafana", 2024-03-04T09:41:13.752 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:13.752 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.752 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:13.752 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "quay.io/ceph/ceph-grafana:9.4.12", 2024-03-04T09:41:13.752 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.752 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.753 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.753 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Prometheus container image", 2024-03-04T09:41:13.753 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.753 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.753 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.753 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.753 INFO:teuthology.orchestra.run.smithi052.stdout: "container_image_haproxy": { 2024-03-04T09:41:13.753 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "container_image_haproxy", 2024-03-04T09:41:13.754 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:13.754 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.754 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:13.754 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-03-04T09:41:13.754 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.754 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.754 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.755 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "HAproxy container image", 2024-03-04T09:41:13.755 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.755 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.755 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.755 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.755 INFO:teuthology.orchestra.run.smithi052.stdout: "container_image_jaeger_agent": { 2024-03-04T09:41:13.755 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "container_image_jaeger_agent", 2024-03-04T09:41:13.755 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:13.756 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.756 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:13.756 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-03-04T09:41:13.756 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.756 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.756 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.756 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Jaeger agent container image", 2024-03-04T09:41:13.756 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.757 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.757 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.757 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.757 INFO:teuthology.orchestra.run.smithi052.stdout: "container_image_jaeger_collector": { 2024-03-04T09:41:13.757 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "container_image_jaeger_collector", 2024-03-04T09:41:13.757 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:13.757 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.757 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:13.757 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-03-04T09:41:13.758 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.758 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.758 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.758 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Jaeger collector container image", 2024-03-04T09:41:13.758 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.758 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.758 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.758 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.759 INFO:teuthology.orchestra.run.smithi052.stdout: "container_image_jaeger_query": { 2024-03-04T09:41:13.759 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "container_image_jaeger_query", 2024-03-04T09:41:13.759 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:13.759 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.759 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:13.759 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-03-04T09:41:13.759 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.759 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.760 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.760 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Jaeger query container image", 2024-03-04T09:41:13.760 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.760 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.760 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.760 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.760 INFO:teuthology.orchestra.run.smithi052.stdout: "container_image_keepalived": { 2024-03-04T09:41:13.760 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "container_image_keepalived", 2024-03-04T09:41:13.761 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:13.761 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.761 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:13.761 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-03-04T09:41:13.761 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.761 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.761 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.761 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Keepalived container image", 2024-03-04T09:41:13.762 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.762 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.762 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.762 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.762 INFO:teuthology.orchestra.run.smithi052.stdout: "container_image_loki": { 2024-03-04T09:41:13.762 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "container_image_loki", 2024-03-04T09:41:13.762 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:13.762 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.762 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:13.763 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-03-04T09:41:13.763 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.763 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.763 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.763 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Loki container image", 2024-03-04T09:41:13.763 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.763 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.763 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.764 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.764 INFO:teuthology.orchestra.run.smithi052.stdout: "container_image_node_exporter": { 2024-03-04T09:41:13.764 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "container_image_node_exporter", 2024-03-04T09:41:13.764 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:13.764 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.764 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:13.764 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-03-04T09:41:13.764 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.765 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.765 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.765 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Prometheus container image", 2024-03-04T09:41:13.765 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.765 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.765 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.765 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.765 INFO:teuthology.orchestra.run.smithi052.stdout: "container_image_nvmeof": { 2024-03-04T09:41:13.765 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "container_image_nvmeof", 2024-03-04T09:41:13.766 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:13.766 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.766 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:13.766 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "quay.io/ceph/nvmeof:latest", 2024-03-04T09:41:13.766 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.766 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.766 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.766 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Nvme-of container image", 2024-03-04T09:41:13.766 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.767 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.767 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.767 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.767 INFO:teuthology.orchestra.run.smithi052.stdout: "container_image_prometheus": { 2024-03-04T09:41:13.767 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "container_image_prometheus", 2024-03-04T09:41:13.767 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:13.767 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.767 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:13.768 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-03-04T09:41:13.768 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.768 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.768 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.768 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Prometheus container image", 2024-03-04T09:41:13.768 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.768 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.768 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.769 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.769 INFO:teuthology.orchestra.run.smithi052.stdout: "container_image_promtail": { 2024-03-04T09:41:13.769 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "container_image_promtail", 2024-03-04T09:41:13.769 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:13.769 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.769 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:13.769 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-03-04T09:41:13.769 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.769 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.770 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.770 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Promtail container image", 2024-03-04T09:41:13.770 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.770 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.770 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.770 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.770 INFO:teuthology.orchestra.run.smithi052.stdout: "container_image_snmp_gateway": { 2024-03-04T09:41:13.770 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "container_image_snmp_gateway", 2024-03-04T09:41:13.771 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:13.771 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.771 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:13.771 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-03-04T09:41:13.771 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.771 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.771 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.771 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "SNMP Gateway container image", 2024-03-04T09:41:13.771 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.772 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.772 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.772 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.772 INFO:teuthology.orchestra.run.smithi052.stdout: "container_init": { 2024-03-04T09:41:13.772 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "container_init", 2024-03-04T09:41:13.772 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:13.772 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.773 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:13.773 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-03-04T09:41:13.773 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.773 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.773 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.773 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Run podman/docker with `--init`", 2024-03-04T09:41:13.774 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.774 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.774 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.774 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.774 INFO:teuthology.orchestra.run.smithi052.stdout: "daemon_cache_timeout": { 2024-03-04T09:41:13.774 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "daemon_cache_timeout", 2024-03-04T09:41:13.774 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "secs", 2024-03-04T09:41:13.774 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.775 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:13.775 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "600", 2024-03-04T09:41:13.775 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.775 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.775 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.775 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-03-04T09:41:13.775 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.775 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.775 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.776 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.776 INFO:teuthology.orchestra.run.smithi052.stdout: "default_cephadm_command_timeout": { 2024-03-04T09:41:13.776 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "default_cephadm_command_timeout", 2024-03-04T09:41:13.776 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-03-04T09:41:13.776 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.776 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:13.776 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "900", 2024-03-04T09:41:13.776 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.777 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.777 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.777 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-03-04T09:41:13.777 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.777 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.777 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.777 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.777 INFO:teuthology.orchestra.run.smithi052.stdout: "default_registry": { 2024-03-04T09:41:13.778 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "default_registry", 2024-03-04T09:41:13.778 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:13.778 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.778 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:13.778 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "docker.io", 2024-03-04T09:41:13.778 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.778 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.778 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.779 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-03-04T09:41:13.779 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.779 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.779 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.779 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.779 INFO:teuthology.orchestra.run.smithi052.stdout: "device_cache_timeout": { 2024-03-04T09:41:13.779 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "device_cache_timeout", 2024-03-04T09:41:13.779 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "secs", 2024-03-04T09:41:13.779 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.780 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:13.780 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "1800", 2024-03-04T09:41:13.780 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.780 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.780 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.780 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "seconds to cache device inventory", 2024-03-04T09:41:13.780 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.781 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.781 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.781 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.781 INFO:teuthology.orchestra.run.smithi052.stdout: "device_enhanced_scan": { 2024-03-04T09:41:13.781 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "device_enhanced_scan", 2024-03-04T09:41:13.781 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:13.781 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.781 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:13.781 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-03-04T09:41:13.782 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.782 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.782 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.782 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Use libstoragemgmt during device scans", 2024-03-04T09:41:13.782 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.782 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.782 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.782 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.783 INFO:teuthology.orchestra.run.smithi052.stdout: "facts_cache_timeout": { 2024-03-04T09:41:13.783 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "facts_cache_timeout", 2024-03-04T09:41:13.783 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "secs", 2024-03-04T09:41:13.783 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.783 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:13.783 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "60", 2024-03-04T09:41:13.783 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.783 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.783 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.784 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "seconds to cache host facts data", 2024-03-04T09:41:13.784 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.784 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.784 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.784 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.784 INFO:teuthology.orchestra.run.smithi052.stdout: "host_check_interval": { 2024-03-04T09:41:13.784 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "host_check_interval", 2024-03-04T09:41:13.784 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "secs", 2024-03-04T09:41:13.785 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.785 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:13.785 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "600", 2024-03-04T09:41:13.785 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.785 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.785 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.785 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "how frequently to perform a host check", 2024-03-04T09:41:13.785 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.785 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.786 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.786 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.786 INFO:teuthology.orchestra.run.smithi052.stdout: "hw_monitoring": { 2024-03-04T09:41:13.786 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "hw_monitoring", 2024-03-04T09:41:13.786 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:13.786 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.786 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:13.786 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-03-04T09:41:13.787 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.787 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.787 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.787 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-03-04T09:41:13.787 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.787 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.787 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.787 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.787 INFO:teuthology.orchestra.run.smithi052.stdout: "inventory_list_all": { 2024-03-04T09:41:13.788 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "inventory_list_all", 2024-03-04T09:41:13.788 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:13.788 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.788 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:13.788 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-03-04T09:41:13.788 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.788 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.788 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.789 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-03-04T09:41:13.789 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.789 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.789 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.789 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.789 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-03-04T09:41:13.789 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-03-04T09:41:13.789 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:13.790 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.790 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:13.790 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-03-04T09:41:13.790 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.790 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.790 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-03-04T09:41:13.790 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-03-04T09:41:13.790 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-03-04T09:41:13.791 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-03-04T09:41:13.791 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-03-04T09:41:13.791 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-03-04T09:41:13.791 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-03-04T09:41:13.791 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-03-04T09:41:13.791 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:13.791 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.791 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.791 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.792 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.792 INFO:teuthology.orchestra.run.smithi052.stdout: "log_refresh_metadata": { 2024-03-04T09:41:13.792 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_refresh_metadata", 2024-03-04T09:41:13.792 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:13.792 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.792 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:13.792 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-03-04T09:41:13.792 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.793 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.793 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.793 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-03-04T09:41:13.793 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.793 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.793 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.793 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.793 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-03-04T09:41:13.793 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-03-04T09:41:13.794 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:13.794 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.794 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:13.794 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-03-04T09:41:13.794 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.794 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.794 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.794 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-03-04T09:41:13.794 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.795 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.795 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.795 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.795 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-03-04T09:41:13.795 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-03-04T09:41:13.795 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:13.795 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.796 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:13.796 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-03-04T09:41:13.796 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.796 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.796 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-03-04T09:41:13.796 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-03-04T09:41:13.796 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-03-04T09:41:13.796 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-03-04T09:41:13.797 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-03-04T09:41:13.797 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-03-04T09:41:13.797 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-03-04T09:41:13.797 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-03-04T09:41:13.797 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:13.797 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.797 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.797 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.797 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.798 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-03-04T09:41:13.798 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-03-04T09:41:13.798 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:13.798 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.798 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:13.798 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-03-04T09:41:13.798 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.798 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.799 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.799 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:13.799 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.799 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.799 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.799 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.799 INFO:teuthology.orchestra.run.smithi052.stdout: "manage_etc_ceph_ceph_conf": { 2024-03-04T09:41:13.799 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-03-04T09:41:13.799 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:13.800 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.800 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:13.800 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-03-04T09:41:13.800 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.800 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.800 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.800 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-03-04T09:41:13.801 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.801 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.801 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.801 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.801 INFO:teuthology.orchestra.run.smithi052.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-03-04T09:41:13.801 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-03-04T09:41:13.801 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:13.801 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.802 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:13.802 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "*", 2024-03-04T09:41:13.802 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.802 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.802 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.802 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-03-04T09:41:13.802 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.802 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.803 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.803 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.803 INFO:teuthology.orchestra.run.smithi052.stdout: "max_count_per_host": { 2024-03-04T09:41:13.803 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "max_count_per_host", 2024-03-04T09:41:13.803 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-03-04T09:41:13.803 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.803 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:13.803 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "10", 2024-03-04T09:41:13.804 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.804 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.804 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.804 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "max number of daemons per service per host", 2024-03-04T09:41:13.804 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.804 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.804 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.805 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.805 INFO:teuthology.orchestra.run.smithi052.stdout: "max_osd_draining_count": { 2024-03-04T09:41:13.805 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "max_osd_draining_count", 2024-03-04T09:41:13.805 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-03-04T09:41:13.805 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.805 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:13.805 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "10", 2024-03-04T09:41:13.805 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.805 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.806 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.806 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-03-04T09:41:13.806 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.806 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.806 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.806 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.806 INFO:teuthology.orchestra.run.smithi052.stdout: "migration_current": { 2024-03-04T09:41:13.806 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "migration_current", 2024-03-04T09:41:13.807 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-03-04T09:41:13.807 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.807 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:13.807 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-03-04T09:41:13.807 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.807 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.807 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.807 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "internal - do not modify", 2024-03-04T09:41:13.807 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.808 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.808 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.808 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.808 INFO:teuthology.orchestra.run.smithi052.stdout: "mode": { 2024-03-04T09:41:13.808 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "mode", 2024-03-04T09:41:13.808 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:13.808 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.808 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:13.809 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "root", 2024-03-04T09:41:13.809 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.809 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.809 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-03-04T09:41:13.809 INFO:teuthology.orchestra.run.smithi052.stdout: "cephadm-package", 2024-03-04T09:41:13.809 INFO:teuthology.orchestra.run.smithi052.stdout: "root" 2024-03-04T09:41:13.809 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-03-04T09:41:13.809 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "mode for remote execution of cephadm", 2024-03-04T09:41:13.810 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.810 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.810 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.810 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.810 INFO:teuthology.orchestra.run.smithi052.stdout: "oob_default_addr": { 2024-03-04T09:41:13.810 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "oob_default_addr", 2024-03-04T09:41:13.810 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:13.810 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.811 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:13.811 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "169.254.1.1", 2024-03-04T09:41:13.811 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.811 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.811 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.811 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Default address for RedFish API (oob management).", 2024-03-04T09:41:13.811 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.811 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.811 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.812 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.812 INFO:teuthology.orchestra.run.smithi052.stdout: "prometheus_alerts_path": { 2024-03-04T09:41:13.812 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "prometheus_alerts_path", 2024-03-04T09:41:13.812 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:13.812 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.812 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:13.812 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-03-04T09:41:13.812 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.813 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.813 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.813 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-03-04T09:41:13.813 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.813 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.813 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.813 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.813 INFO:teuthology.orchestra.run.smithi052.stdout: "registry_insecure": { 2024-03-04T09:41:13.813 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "registry_insecure", 2024-03-04T09:41:13.814 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:13.814 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.814 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:13.814 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-03-04T09:41:13.814 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.814 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.814 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.814 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-03-04T09:41:13.815 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.815 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.815 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.815 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.815 INFO:teuthology.orchestra.run.smithi052.stdout: "registry_password": { 2024-03-04T09:41:13.815 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "registry_password", 2024-03-04T09:41:13.815 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:13.815 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.816 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:13.816 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-03-04T09:41:13.816 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.816 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.816 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.816 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-03-04T09:41:13.816 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.816 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.817 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.817 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.817 INFO:teuthology.orchestra.run.smithi052.stdout: "registry_url": { 2024-03-04T09:41:13.817 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "registry_url", 2024-03-04T09:41:13.817 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:13.817 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.817 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:13.817 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-03-04T09:41:13.818 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.818 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.818 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.818 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-03-04T09:41:13.818 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.818 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.818 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.818 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.818 INFO:teuthology.orchestra.run.smithi052.stdout: "registry_username": { 2024-03-04T09:41:13.819 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "registry_username", 2024-03-04T09:41:13.819 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:13.819 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.819 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:13.819 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-03-04T09:41:13.819 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.819 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.819 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.820 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-03-04T09:41:13.820 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.820 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.820 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.820 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.820 INFO:teuthology.orchestra.run.smithi052.stdout: "secure_monitoring_stack": { 2024-03-04T09:41:13.820 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "secure_monitoring_stack", 2024-03-04T09:41:13.820 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:13.821 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.821 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:13.821 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-03-04T09:41:13.821 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.821 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.821 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.821 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-03-04T09:41:13.821 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.822 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.822 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.822 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.822 INFO:teuthology.orchestra.run.smithi052.stdout: "service_discovery_port": { 2024-03-04T09:41:13.822 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "service_discovery_port", 2024-03-04T09:41:13.822 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-03-04T09:41:13.822 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.822 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:13.823 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "8765", 2024-03-04T09:41:13.823 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.823 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.823 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.823 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "cephadm service discovery port", 2024-03-04T09:41:13.823 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.823 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.823 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.824 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.824 INFO:teuthology.orchestra.run.smithi052.stdout: "ssh_config_file": { 2024-03-04T09:41:13.824 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "ssh_config_file", 2024-03-04T09:41:13.824 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:13.824 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.824 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:13.824 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-03-04T09:41:13.824 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.824 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.825 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.825 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-03-04T09:41:13.825 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.825 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.825 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.825 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.825 INFO:teuthology.orchestra.run.smithi052.stdout: "use_agent": { 2024-03-04T09:41:13.825 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "use_agent", 2024-03-04T09:41:13.826 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:13.826 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.826 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:13.826 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-03-04T09:41:13.826 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.826 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.826 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.826 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-03-04T09:41:13.827 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.827 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.827 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.827 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.827 INFO:teuthology.orchestra.run.smithi052.stdout: "use_repo_digest": { 2024-03-04T09:41:13.827 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "use_repo_digest", 2024-03-04T09:41:13.827 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:13.827 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.827 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:13.828 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-03-04T09:41:13.828 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.828 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.828 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.828 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-03-04T09:41:13.828 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.828 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.828 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.829 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.829 INFO:teuthology.orchestra.run.smithi052.stdout: "warn_on_failed_host_check": { 2024-03-04T09:41:13.829 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "warn_on_failed_host_check", 2024-03-04T09:41:13.829 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:13.829 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.829 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:13.829 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-03-04T09:41:13.829 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.830 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.830 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.830 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "raise a health warning if the host check fails", 2024-03-04T09:41:13.830 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.830 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.830 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.830 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.830 INFO:teuthology.orchestra.run.smithi052.stdout: "warn_on_stray_daemons": { 2024-03-04T09:41:13.831 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "warn_on_stray_daemons", 2024-03-04T09:41:13.831 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:13.831 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.831 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:13.831 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-03-04T09:41:13.831 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.831 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.831 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.832 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-03-04T09:41:13.832 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.832 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.832 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.832 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.832 INFO:teuthology.orchestra.run.smithi052.stdout: "warn_on_stray_hosts": { 2024-03-04T09:41:13.832 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "warn_on_stray_hosts", 2024-03-04T09:41:13.833 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:13.833 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.833 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:13.833 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-03-04T09:41:13.833 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.833 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.833 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.833 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-03-04T09:41:13.834 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.834 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.834 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.834 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-03-04T09:41:13.834 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-03-04T09:41:13.834 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.834 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-03-04T09:41:13.835 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "crash", 2024-03-04T09:41:13.835 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-03-04T09:41:13.835 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-03-04T09:41:13.835 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-03-04T09:41:13.835 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-03-04T09:41:13.835 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-03-04T09:41:13.835 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:13.835 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.836 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:13.836 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-03-04T09:41:13.836 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.836 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.836 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-03-04T09:41:13.836 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-03-04T09:41:13.836 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-03-04T09:41:13.836 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-03-04T09:41:13.837 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-03-04T09:41:13.837 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-03-04T09:41:13.837 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-03-04T09:41:13.837 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-03-04T09:41:13.837 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:13.837 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.837 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.837 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.838 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.838 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-03-04T09:41:13.838 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-03-04T09:41:13.838 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:13.838 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.838 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:13.838 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-03-04T09:41:13.838 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.839 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.839 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.839 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:13.839 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.839 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.839 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.839 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.839 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-03-04T09:41:13.840 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-03-04T09:41:13.840 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:13.840 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.840 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:13.840 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-03-04T09:41:13.840 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.840 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.840 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-03-04T09:41:13.841 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-03-04T09:41:13.841 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-03-04T09:41:13.841 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-03-04T09:41:13.841 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-03-04T09:41:13.841 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-03-04T09:41:13.841 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-03-04T09:41:13.841 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-03-04T09:41:13.841 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:13.842 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.842 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.842 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.842 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.842 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-03-04T09:41:13.842 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-03-04T09:41:13.842 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:13.842 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.843 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:13.843 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-03-04T09:41:13.843 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.843 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.843 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.843 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:13.843 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.843 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.843 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.844 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.844 INFO:teuthology.orchestra.run.smithi052.stdout: "retain_interval": { 2024-03-04T09:41:13.844 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "retain_interval", 2024-03-04T09:41:13.844 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "secs", 2024-03-04T09:41:13.844 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.844 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:13.844 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "31536000", 2024-03-04T09:41:13.845 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.845 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.845 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.845 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "how long to retain crashes before pruning them", 2024-03-04T09:41:13.845 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.845 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.845 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.845 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.845 INFO:teuthology.orchestra.run.smithi052.stdout: "warn_recent_interval": { 2024-03-04T09:41:13.846 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "warn_recent_interval", 2024-03-04T09:41:13.846 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "secs", 2024-03-04T09:41:13.846 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.846 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:13.846 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "1209600", 2024-03-04T09:41:13.846 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.846 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.846 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.846 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "time interval in which to warn about recent crashes", 2024-03-04T09:41:13.847 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.847 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.847 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.847 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-03-04T09:41:13.847 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-03-04T09:41:13.847 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.847 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-03-04T09:41:13.847 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "dashboard", 2024-03-04T09:41:13.847 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-03-04T09:41:13.848 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-03-04T09:41:13.848 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-03-04T09:41:13.848 INFO:teuthology.orchestra.run.smithi052.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-03-04T09:41:13.848 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-03-04T09:41:13.848 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-03-04T09:41:13.848 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.848 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:13.848 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "10", 2024-03-04T09:41:13.848 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.849 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.849 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.849 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:13.849 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.849 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.849 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.849 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.849 INFO:teuthology.orchestra.run.smithi052.stdout: "ALERTMANAGER_API_HOST": { 2024-03-04T09:41:13.849 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "ALERTMANAGER_API_HOST", 2024-03-04T09:41:13.850 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:13.850 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.850 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:13.850 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-03-04T09:41:13.850 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.850 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.850 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.850 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:13.851 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.851 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.851 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.851 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.851 INFO:teuthology.orchestra.run.smithi052.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-03-04T09:41:13.851 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-03-04T09:41:13.851 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:13.851 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.852 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:13.852 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-03-04T09:41:13.852 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.852 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.852 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.852 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:13.852 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.853 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.853 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.853 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.853 INFO:teuthology.orchestra.run.smithi052.stdout: "AUDIT_API_ENABLED": { 2024-03-04T09:41:13.853 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "AUDIT_API_ENABLED", 2024-03-04T09:41:13.853 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:13.853 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.853 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:13.854 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-03-04T09:41:13.854 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.854 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.854 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.854 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:13.854 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.854 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.854 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.855 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.855 INFO:teuthology.orchestra.run.smithi052.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-03-04T09:41:13.855 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-03-04T09:41:13.855 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:13.855 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.855 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:13.855 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-03-04T09:41:13.855 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.856 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.856 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.856 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:13.856 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.856 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.856 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.856 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.856 INFO:teuthology.orchestra.run.smithi052.stdout: "ENABLE_BROWSABLE_API": { 2024-03-04T09:41:13.857 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "ENABLE_BROWSABLE_API", 2024-03-04T09:41:13.857 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:13.857 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.857 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:13.857 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-03-04T09:41:13.857 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.857 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.857 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.858 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:13.858 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.858 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.858 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.858 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.858 INFO:teuthology.orchestra.run.smithi052.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-03-04T09:41:13.858 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-03-04T09:41:13.858 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:13.859 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.859 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:13.859 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-03-04T09:41:13.859 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.859 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.859 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.859 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:13.859 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.859 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.860 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.860 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.860 INFO:teuthology.orchestra.run.smithi052.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-03-04T09:41:13.860 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-03-04T09:41:13.860 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:13.860 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.860 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:13.861 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-03-04T09:41:13.861 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.861 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.861 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.861 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:13.861 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.861 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.861 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.862 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.862 INFO:teuthology.orchestra.run.smithi052.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-03-04T09:41:13.862 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-03-04T09:41:13.862 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:13.862 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.862 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:13.862 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-03-04T09:41:13.862 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.863 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.863 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.863 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:13.863 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.863 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.863 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.863 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.863 INFO:teuthology.orchestra.run.smithi052.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-03-04T09:41:13.864 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-03-04T09:41:13.864 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:13.864 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.864 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:13.864 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-03-04T09:41:13.864 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.864 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.864 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.865 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:13.865 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.865 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.865 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.865 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.865 INFO:teuthology.orchestra.run.smithi052.stdout: "FEATURE_TOGGLE_NFS": { 2024-03-04T09:41:13.865 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-03-04T09:41:13.865 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:13.866 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.866 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:13.866 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-03-04T09:41:13.866 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.866 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.866 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.866 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:13.866 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.867 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.867 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.867 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.867 INFO:teuthology.orchestra.run.smithi052.stdout: "FEATURE_TOGGLE_RBD": { 2024-03-04T09:41:13.867 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-03-04T09:41:13.867 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:13.867 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.867 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:13.868 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-03-04T09:41:13.868 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.868 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.868 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.868 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:13.868 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.868 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.868 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.869 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.869 INFO:teuthology.orchestra.run.smithi052.stdout: "FEATURE_TOGGLE_RGW": { 2024-03-04T09:41:13.869 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-03-04T09:41:13.869 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:13.869 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.869 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:13.869 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-03-04T09:41:13.869 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.870 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.870 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.870 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:13.870 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.870 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.870 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.870 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.871 INFO:teuthology.orchestra.run.smithi052.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-03-04T09:41:13.871 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-03-04T09:41:13.871 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:13.871 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.871 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:13.871 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-03-04T09:41:13.872 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.872 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.872 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.872 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:13.872 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.872 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.872 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.873 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.873 INFO:teuthology.orchestra.run.smithi052.stdout: "GRAFANA_API_PASSWORD": { 2024-03-04T09:41:13.873 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "GRAFANA_API_PASSWORD", 2024-03-04T09:41:13.873 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:13.873 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.873 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:13.873 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "admin", 2024-03-04T09:41:13.873 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.873 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.874 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.874 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:13.874 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.874 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.874 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.874 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.874 INFO:teuthology.orchestra.run.smithi052.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-03-04T09:41:13.874 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-03-04T09:41:13.874 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:13.875 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.875 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:13.875 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-03-04T09:41:13.875 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.875 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.875 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.875 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:13.875 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.875 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.876 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.876 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.876 INFO:teuthology.orchestra.run.smithi052.stdout: "GRAFANA_API_URL": { 2024-03-04T09:41:13.876 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "GRAFANA_API_URL", 2024-03-04T09:41:13.876 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:13.876 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.876 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:13.876 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-03-04T09:41:13.877 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.877 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.877 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.877 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:13.879 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.879 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.879 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.880 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.880 INFO:teuthology.orchestra.run.smithi052.stdout: "GRAFANA_API_USERNAME": { 2024-03-04T09:41:13.880 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "GRAFANA_API_USERNAME", 2024-03-04T09:41:13.880 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:13.880 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.880 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:13.880 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "admin", 2024-03-04T09:41:13.880 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.881 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.881 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.881 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:13.881 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.881 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.881 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.881 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.881 INFO:teuthology.orchestra.run.smithi052.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-03-04T09:41:13.881 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-03-04T09:41:13.882 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:13.882 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.882 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:13.882 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-03-04T09:41:13.882 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.882 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.882 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.882 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:13.882 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.883 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.883 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.883 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.883 INFO:teuthology.orchestra.run.smithi052.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-03-04T09:41:13.883 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-03-04T09:41:13.883 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:13.883 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.883 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:13.883 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-03-04T09:41:13.884 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.884 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.884 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.884 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:13.884 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.884 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.884 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.884 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.885 INFO:teuthology.orchestra.run.smithi052.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-03-04T09:41:13.885 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-03-04T09:41:13.885 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:13.885 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.885 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:13.885 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-03-04T09:41:13.885 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.886 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.886 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.886 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:13.886 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.886 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.886 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.886 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.887 INFO:teuthology.orchestra.run.smithi052.stdout: "ISSUE_TRACKER_API_KEY": { 2024-03-04T09:41:13.887 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-03-04T09:41:13.887 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:13.887 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.887 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:13.887 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-03-04T09:41:13.887 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.887 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.888 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.888 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:13.888 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.888 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.888 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.888 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.889 INFO:teuthology.orchestra.run.smithi052.stdout: "PROMETHEUS_API_HOST": { 2024-03-04T09:41:13.889 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "PROMETHEUS_API_HOST", 2024-03-04T09:41:13.889 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:13.889 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.889 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:13.889 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-03-04T09:41:13.889 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.890 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.890 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.890 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:13.890 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.890 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.890 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.890 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.891 INFO:teuthology.orchestra.run.smithi052.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-03-04T09:41:13.891 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-03-04T09:41:13.891 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:13.891 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.891 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:13.891 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-03-04T09:41:13.891 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.892 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.892 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.892 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:13.892 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.892 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.892 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.893 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.893 INFO:teuthology.orchestra.run.smithi052.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-03-04T09:41:13.893 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-03-04T09:41:13.893 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:13.893 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.893 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:13.893 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-03-04T09:41:13.894 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.894 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.894 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.894 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:13.894 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.894 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.894 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.894 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.895 INFO:teuthology.orchestra.run.smithi052.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-03-04T09:41:13.895 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-03-04T09:41:13.895 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:13.895 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.895 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:13.895 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-03-04T09:41:13.895 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.896 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.896 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.896 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:13.896 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.896 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.896 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.896 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.897 INFO:teuthology.orchestra.run.smithi052.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-03-04T09:41:13.897 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-03-04T09:41:13.897 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:13.897 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.897 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:13.897 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-03-04T09:41:13.897 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.897 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.898 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.898 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:13.898 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.898 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.898 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.898 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.898 INFO:teuthology.orchestra.run.smithi052.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-03-04T09:41:13.898 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-03-04T09:41:13.899 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:13.899 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.899 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:13.899 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-03-04T09:41:13.899 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.899 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.899 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.899 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:13.900 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.900 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.900 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.900 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.900 INFO:teuthology.orchestra.run.smithi052.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-03-04T09:41:13.900 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-03-04T09:41:13.900 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:13.901 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.901 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:13.901 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-03-04T09:41:13.901 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.901 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.901 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.901 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:13.901 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.902 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.902 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.902 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.902 INFO:teuthology.orchestra.run.smithi052.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-03-04T09:41:13.902 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-03-04T09:41:13.902 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:13.902 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.902 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:13.902 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-03-04T09:41:13.903 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.903 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.903 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.903 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:13.903 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.903 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.903 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.903 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.904 INFO:teuthology.orchestra.run.smithi052.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-03-04T09:41:13.904 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-03-04T09:41:13.904 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:13.904 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.904 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:13.904 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-03-04T09:41:13.904 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.904 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.905 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.905 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:13.905 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.905 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.905 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.905 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.905 INFO:teuthology.orchestra.run.smithi052.stdout: "PWD_POLICY_ENABLED": { 2024-03-04T09:41:13.905 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "PWD_POLICY_ENABLED", 2024-03-04T09:41:13.905 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:13.906 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.906 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:13.906 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-03-04T09:41:13.906 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.906 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.906 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.906 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:13.906 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.907 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.907 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.907 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.907 INFO:teuthology.orchestra.run.smithi052.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-03-04T09:41:13.907 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-03-04T09:41:13.907 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:13.907 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.907 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:13.907 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-03-04T09:41:13.908 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.908 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.908 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.908 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:13.908 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.908 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.908 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.909 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.909 INFO:teuthology.orchestra.run.smithi052.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-03-04T09:41:13.909 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-03-04T09:41:13.909 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-03-04T09:41:13.909 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.909 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:13.909 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "10", 2024-03-04T09:41:13.909 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.910 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.910 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.910 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:13.910 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.910 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.910 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.910 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.910 INFO:teuthology.orchestra.run.smithi052.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-03-04T09:41:13.910 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-03-04T09:41:13.911 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-03-04T09:41:13.911 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.911 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:13.911 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "8", 2024-03-04T09:41:13.911 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.911 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.911 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.911 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:13.911 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.912 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.912 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.912 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.912 INFO:teuthology.orchestra.run.smithi052.stdout: "REST_REQUESTS_TIMEOUT": { 2024-03-04T09:41:13.912 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-03-04T09:41:13.912 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-03-04T09:41:13.912 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.912 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:13.913 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "45", 2024-03-04T09:41:13.913 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.913 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.913 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.913 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:13.913 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.913 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.913 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.914 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.914 INFO:teuthology.orchestra.run.smithi052.stdout: "RGW_API_ACCESS_KEY": { 2024-03-04T09:41:13.914 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "RGW_API_ACCESS_KEY", 2024-03-04T09:41:13.914 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:13.914 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.914 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:13.914 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-03-04T09:41:13.914 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.915 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.915 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.915 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:13.915 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.915 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.915 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.915 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.915 INFO:teuthology.orchestra.run.smithi052.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-03-04T09:41:13.915 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-03-04T09:41:13.916 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:13.916 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.916 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:13.916 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "admin", 2024-03-04T09:41:13.916 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.916 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.916 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.916 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:13.917 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.917 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.917 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.917 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.917 INFO:teuthology.orchestra.run.smithi052.stdout: "RGW_API_SECRET_KEY": { 2024-03-04T09:41:13.917 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "RGW_API_SECRET_KEY", 2024-03-04T09:41:13.917 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:13.917 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.918 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:13.918 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-03-04T09:41:13.918 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.918 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.918 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.918 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:13.918 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.918 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.919 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.919 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.919 INFO:teuthology.orchestra.run.smithi052.stdout: "RGW_API_SSL_VERIFY": { 2024-03-04T09:41:13.919 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "RGW_API_SSL_VERIFY", 2024-03-04T09:41:13.919 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:13.919 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.919 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:13.919 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-03-04T09:41:13.919 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.920 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.920 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.920 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:13.920 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.920 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.920 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.920 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.920 INFO:teuthology.orchestra.run.smithi052.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-03-04T09:41:13.921 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-03-04T09:41:13.921 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-03-04T09:41:13.921 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.921 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:13.921 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "0", 2024-03-04T09:41:13.921 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.921 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.921 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.922 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:13.922 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.922 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.922 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.922 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.922 INFO:teuthology.orchestra.run.smithi052.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-03-04T09:41:13.922 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-03-04T09:41:13.922 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-03-04T09:41:13.922 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.923 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:13.923 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "10", 2024-03-04T09:41:13.923 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.923 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.923 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.923 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:13.923 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.923 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.924 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.924 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.924 INFO:teuthology.orchestra.run.smithi052.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-03-04T09:41:13.924 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-03-04T09:41:13.924 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-03-04T09:41:13.924 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.924 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:13.924 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "5", 2024-03-04T09:41:13.925 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.925 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.925 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.925 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:13.925 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.925 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.925 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.925 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.926 INFO:teuthology.orchestra.run.smithi052.stdout: "cross_origin_url": { 2024-03-04T09:41:13.926 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "cross_origin_url", 2024-03-04T09:41:13.926 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:13.926 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.926 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:13.926 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-03-04T09:41:13.926 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.926 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.927 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.927 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:13.927 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.927 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.927 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.927 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.927 INFO:teuthology.orchestra.run.smithi052.stdout: "crt_file": { 2024-03-04T09:41:13.927 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "crt_file", 2024-03-04T09:41:13.928 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:13.928 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.928 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:13.928 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-03-04T09:41:13.928 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.928 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.928 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.928 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:13.929 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.929 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.929 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.929 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.929 INFO:teuthology.orchestra.run.smithi052.stdout: "debug": { 2024-03-04T09:41:13.929 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "debug", 2024-03-04T09:41:13.929 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:13.929 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.930 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:13.930 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-03-04T09:41:13.930 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.930 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.930 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.930 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Enable/disable debug options", 2024-03-04T09:41:13.930 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.930 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.931 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.931 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.931 INFO:teuthology.orchestra.run.smithi052.stdout: "jwt_token_ttl": { 2024-03-04T09:41:13.931 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "jwt_token_ttl", 2024-03-04T09:41:13.931 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-03-04T09:41:13.931 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.931 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:13.931 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "28800", 2024-03-04T09:41:13.931 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.932 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.932 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.932 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:13.932 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.932 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.932 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.932 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.932 INFO:teuthology.orchestra.run.smithi052.stdout: "key_file": { 2024-03-04T09:41:13.933 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "key_file", 2024-03-04T09:41:13.933 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:13.933 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.933 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:13.933 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-03-04T09:41:13.933 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.933 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.933 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.934 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:13.934 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.934 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.934 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.934 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.934 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-03-04T09:41:13.934 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-03-04T09:41:13.934 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:13.935 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.935 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:13.935 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-03-04T09:41:13.935 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.935 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.935 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-03-04T09:41:13.935 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-03-04T09:41:13.935 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-03-04T09:41:13.935 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-03-04T09:41:13.936 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-03-04T09:41:13.936 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-03-04T09:41:13.936 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-03-04T09:41:13.936 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-03-04T09:41:13.936 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:13.936 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.936 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.936 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.937 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.937 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-03-04T09:41:13.937 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-03-04T09:41:13.937 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:13.937 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.937 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:13.937 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-03-04T09:41:13.937 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.937 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.938 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.938 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:13.938 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.938 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.938 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.938 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.938 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-03-04T09:41:13.938 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-03-04T09:41:13.939 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:13.939 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.939 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:13.939 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-03-04T09:41:13.939 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.939 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.939 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-03-04T09:41:13.939 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-03-04T09:41:13.940 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-03-04T09:41:13.940 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-03-04T09:41:13.940 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-03-04T09:41:13.940 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-03-04T09:41:13.940 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-03-04T09:41:13.940 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-03-04T09:41:13.940 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:13.940 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.941 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.941 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.941 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.941 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-03-04T09:41:13.941 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-03-04T09:41:13.941 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:13.941 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.941 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:13.941 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-03-04T09:41:13.941 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.942 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.942 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.942 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:13.942 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.942 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.942 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.942 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.942 INFO:teuthology.orchestra.run.smithi052.stdout: "motd": { 2024-03-04T09:41:13.943 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "motd", 2024-03-04T09:41:13.943 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:13.943 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.943 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:13.943 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-03-04T09:41:13.943 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.943 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.943 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.944 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "The message of the day", 2024-03-04T09:41:13.944 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.944 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.944 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.944 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.944 INFO:teuthology.orchestra.run.smithi052.stdout: "redirect_resolve_ip_addr": { 2024-03-04T09:41:13.944 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "redirect_resolve_ip_addr", 2024-03-04T09:41:13.944 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:13.945 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.945 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:13.945 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-03-04T09:41:13.945 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.945 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.945 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.945 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:13.945 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.946 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.946 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.946 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.946 INFO:teuthology.orchestra.run.smithi052.stdout: "server_addr": { 2024-03-04T09:41:13.946 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "server_addr", 2024-03-04T09:41:13.946 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:13.946 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.946 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:13.947 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "::", 2024-03-04T09:41:13.947 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.947 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.947 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.947 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:13.947 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.947 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.947 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.947 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.948 INFO:teuthology.orchestra.run.smithi052.stdout: "server_port": { 2024-03-04T09:41:13.948 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "server_port", 2024-03-04T09:41:13.948 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-03-04T09:41:13.948 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.948 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:13.948 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "8080", 2024-03-04T09:41:13.948 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.949 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.949 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.949 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:13.949 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.949 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.949 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.949 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.949 INFO:teuthology.orchestra.run.smithi052.stdout: "ssl": { 2024-03-04T09:41:13.950 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "ssl", 2024-03-04T09:41:13.950 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:13.950 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.950 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:13.950 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-03-04T09:41:13.950 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.950 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.950 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.950 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:13.951 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.951 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.951 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.951 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.951 INFO:teuthology.orchestra.run.smithi052.stdout: "ssl_server_port": { 2024-03-04T09:41:13.951 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "ssl_server_port", 2024-03-04T09:41:13.952 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-03-04T09:41:13.952 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.952 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:13.952 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "8443", 2024-03-04T09:41:13.952 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.953 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.953 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.953 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:13.953 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.953 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.953 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.954 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.954 INFO:teuthology.orchestra.run.smithi052.stdout: "standby_behaviour": { 2024-03-04T09:41:13.954 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "standby_behaviour", 2024-03-04T09:41:13.954 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:13.954 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.954 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:13.954 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "redirect", 2024-03-04T09:41:13.954 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.955 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.955 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-03-04T09:41:13.955 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-03-04T09:41:13.955 INFO:teuthology.orchestra.run.smithi052.stdout: "redirect" 2024-03-04T09:41:13.955 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-03-04T09:41:13.955 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:13.955 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.955 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.955 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.956 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.956 INFO:teuthology.orchestra.run.smithi052.stdout: "standby_error_status_code": { 2024-03-04T09:41:13.956 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "standby_error_status_code", 2024-03-04T09:41:13.956 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-03-04T09:41:13.956 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.956 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:13.956 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "500", 2024-03-04T09:41:13.956 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "400", 2024-03-04T09:41:13.957 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "599", 2024-03-04T09:41:13.957 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.957 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:13.957 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.957 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.957 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.957 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.957 INFO:teuthology.orchestra.run.smithi052.stdout: "url_prefix": { 2024-03-04T09:41:13.958 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "url_prefix", 2024-03-04T09:41:13.958 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:13.958 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.958 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:13.958 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-03-04T09:41:13.958 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.958 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.958 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.959 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:13.959 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.959 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.959 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.959 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-03-04T09:41:13.959 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-03-04T09:41:13.959 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.959 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-03-04T09:41:13.960 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "devicehealth", 2024-03-04T09:41:13.960 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-03-04T09:41:13.960 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-03-04T09:41:13.960 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-03-04T09:41:13.960 INFO:teuthology.orchestra.run.smithi052.stdout: "enable_monitoring": { 2024-03-04T09:41:13.960 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "enable_monitoring", 2024-03-04T09:41:13.960 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:13.960 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.960 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:13.961 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-03-04T09:41:13.961 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.961 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.961 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.961 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "monitor device health metrics", 2024-03-04T09:41:13.961 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.961 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.962 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.962 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.962 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-03-04T09:41:13.962 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-03-04T09:41:13.962 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:13.962 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.962 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:13.962 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-03-04T09:41:13.962 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.963 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.963 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-03-04T09:41:13.963 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-03-04T09:41:13.963 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-03-04T09:41:13.963 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-03-04T09:41:13.963 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-03-04T09:41:13.963 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-03-04T09:41:13.964 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-03-04T09:41:13.964 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-03-04T09:41:13.964 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:13.964 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.964 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.964 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.964 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.965 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-03-04T09:41:13.965 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-03-04T09:41:13.965 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:13.965 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.965 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:13.965 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-03-04T09:41:13.965 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.965 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.966 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.966 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:13.966 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.966 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.966 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.966 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.966 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-03-04T09:41:13.966 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-03-04T09:41:13.967 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:13.967 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.967 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:13.967 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-03-04T09:41:13.967 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.967 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.967 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-03-04T09:41:13.967 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-03-04T09:41:13.967 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-03-04T09:41:13.968 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-03-04T09:41:13.968 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-03-04T09:41:13.968 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-03-04T09:41:13.968 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-03-04T09:41:13.968 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-03-04T09:41:13.968 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:13.968 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.968 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.969 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.969 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.969 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-03-04T09:41:13.969 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-03-04T09:41:13.969 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:13.969 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.969 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:13.969 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-03-04T09:41:13.970 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.970 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.970 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.970 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:13.970 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.970 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.970 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.970 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.971 INFO:teuthology.orchestra.run.smithi052.stdout: "mark_out_threshold": { 2024-03-04T09:41:13.971 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "mark_out_threshold", 2024-03-04T09:41:13.971 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "secs", 2024-03-04T09:41:13.971 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.971 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:13.971 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "2419200", 2024-03-04T09:41:13.971 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.971 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.972 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.972 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-03-04T09:41:13.972 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.972 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.972 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.972 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.972 INFO:teuthology.orchestra.run.smithi052.stdout: "pool_name": { 2024-03-04T09:41:13.972 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "pool_name", 2024-03-04T09:41:13.973 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:13.973 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.973 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:13.973 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "device_health_metrics", 2024-03-04T09:41:13.973 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.973 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.973 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.973 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "name of pool in which to store device health metrics", 2024-03-04T09:41:13.974 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.974 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.974 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.974 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.974 INFO:teuthology.orchestra.run.smithi052.stdout: "retention_period": { 2024-03-04T09:41:13.974 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "retention_period", 2024-03-04T09:41:13.974 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "secs", 2024-03-04T09:41:13.974 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.975 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:13.975 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "15552000", 2024-03-04T09:41:13.975 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.975 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.975 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.975 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "how long to retain device health metrics", 2024-03-04T09:41:13.975 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.975 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.975 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.976 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.976 INFO:teuthology.orchestra.run.smithi052.stdout: "scrape_frequency": { 2024-03-04T09:41:13.976 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "scrape_frequency", 2024-03-04T09:41:13.976 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "secs", 2024-03-04T09:41:13.976 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.976 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:13.976 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "86400", 2024-03-04T09:41:13.976 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.977 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.977 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.977 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "how frequently to scrape device health metrics", 2024-03-04T09:41:13.977 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.977 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.977 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.977 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.978 INFO:teuthology.orchestra.run.smithi052.stdout: "self_heal": { 2024-03-04T09:41:13.978 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "self_heal", 2024-03-04T09:41:13.978 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:13.978 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.978 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:13.978 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-03-04T09:41:13.978 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.978 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.978 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.979 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-03-04T09:41:13.979 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.979 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.979 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.979 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.979 INFO:teuthology.orchestra.run.smithi052.stdout: "sleep_interval": { 2024-03-04T09:41:13.979 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "sleep_interval", 2024-03-04T09:41:13.980 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "secs", 2024-03-04T09:41:13.980 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.980 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:13.980 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "600", 2024-03-04T09:41:13.980 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.980 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.980 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.980 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "how frequently to wake up and check device health", 2024-03-04T09:41:13.981 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.981 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.981 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.981 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.981 INFO:teuthology.orchestra.run.smithi052.stdout: "warn_threshold": { 2024-03-04T09:41:13.981 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "warn_threshold", 2024-03-04T09:41:13.981 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "secs", 2024-03-04T09:41:13.981 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.982 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:13.982 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "7257600", 2024-03-04T09:41:13.982 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.982 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.982 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.982 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-03-04T09:41:13.982 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.982 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.982 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.983 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-03-04T09:41:13.983 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-03-04T09:41:13.983 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.983 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-03-04T09:41:13.983 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "diskprediction_local", 2024-03-04T09:41:13.983 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-03-04T09:41:13.983 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-03-04T09:41:13.983 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-03-04T09:41:13.984 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-03-04T09:41:13.984 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-03-04T09:41:13.984 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:13.984 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.984 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:13.984 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-03-04T09:41:13.984 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.984 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.985 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-03-04T09:41:13.985 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-03-04T09:41:13.985 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-03-04T09:41:13.985 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-03-04T09:41:13.985 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-03-04T09:41:13.985 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-03-04T09:41:13.985 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-03-04T09:41:13.985 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-03-04T09:41:13.985 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:13.986 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.986 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.986 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.986 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.986 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-03-04T09:41:13.986 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-03-04T09:41:13.986 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:13.986 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.987 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:13.987 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-03-04T09:41:13.987 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.987 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.987 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.987 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:13.987 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.987 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.988 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.988 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.988 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-03-04T09:41:13.988 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-03-04T09:41:13.988 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:13.988 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.988 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:13.988 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-03-04T09:41:13.989 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.989 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.989 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-03-04T09:41:13.989 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-03-04T09:41:13.989 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-03-04T09:41:13.989 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-03-04T09:41:13.989 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-03-04T09:41:13.990 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-03-04T09:41:13.990 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-03-04T09:41:13.990 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-03-04T09:41:13.990 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:13.990 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.990 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.990 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.990 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.991 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-03-04T09:41:13.991 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-03-04T09:41:13.991 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:13.991 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.991 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:13.991 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-03-04T09:41:13.991 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.991 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.991 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.992 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:13.992 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.992 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.992 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.992 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.992 INFO:teuthology.orchestra.run.smithi052.stdout: "predict_interval": { 2024-03-04T09:41:13.992 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "predict_interval", 2024-03-04T09:41:13.992 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:13.993 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.993 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:13.993 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "86400", 2024-03-04T09:41:13.993 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.993 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.993 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.993 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:13.993 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.993 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.994 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.994 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.994 INFO:teuthology.orchestra.run.smithi052.stdout: "predictor_model": { 2024-03-04T09:41:13.994 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "predictor_model", 2024-03-04T09:41:13.994 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:13.994 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.994 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:13.994 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "prophetstor", 2024-03-04T09:41:13.995 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.995 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.995 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.995 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:13.995 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.995 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.995 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.995 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.995 INFO:teuthology.orchestra.run.smithi052.stdout: "sleep_interval": { 2024-03-04T09:41:13.996 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "sleep_interval", 2024-03-04T09:41:13.996 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:13.996 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.996 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:13.996 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "600", 2024-03-04T09:41:13.996 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.996 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.996 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.997 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:13.997 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:13.997 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:13.997 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:13.997 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-03-04T09:41:13.997 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-03-04T09:41:13.997 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:13.998 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-03-04T09:41:13.998 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "influx", 2024-03-04T09:41:13.998 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": false, 2024-03-04T09:41:13.998 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "influxdb python module not found", 2024-03-04T09:41:13.998 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-03-04T09:41:13.998 INFO:teuthology.orchestra.run.smithi052.stdout: "batch_size": { 2024-03-04T09:41:13.998 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "batch_size", 2024-03-04T09:41:13.999 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-03-04T09:41:13.999 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:13.999 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:13.999 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "5000", 2024-03-04T09:41:13.999 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:13.999 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:13.999 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:13.999 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-03-04T09:41:14.000 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.000 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.000 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.000 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.000 INFO:teuthology.orchestra.run.smithi052.stdout: "database": { 2024-03-04T09:41:14.000 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "database", 2024-03-04T09:41:14.000 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:14.001 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.001 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:14.001 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "ceph", 2024-03-04T09:41:14.001 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.001 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.001 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:14.001 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-03-04T09:41:14.002 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.002 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.002 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.002 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.002 INFO:teuthology.orchestra.run.smithi052.stdout: "hostname": { 2024-03-04T09:41:14.002 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "hostname", 2024-03-04T09:41:14.002 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:14.002 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.003 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:14.003 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-03-04T09:41:14.003 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.003 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.003 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:14.003 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "InfluxDB server hostname", 2024-03-04T09:41:14.003 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.004 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.004 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.004 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.004 INFO:teuthology.orchestra.run.smithi052.stdout: "interval": { 2024-03-04T09:41:14.004 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "interval", 2024-03-04T09:41:14.004 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "secs", 2024-03-04T09:41:14.004 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.005 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:14.005 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "30", 2024-03-04T09:41:14.005 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "5", 2024-03-04T09:41:14.005 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.005 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:14.005 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-03-04T09:41:14.005 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.005 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.006 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.006 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.006 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-03-04T09:41:14.006 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-03-04T09:41:14.006 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:14.006 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.006 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:14.006 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-03-04T09:41:14.006 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.007 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.007 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-03-04T09:41:14.007 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-03-04T09:41:14.007 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-03-04T09:41:14.007 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-03-04T09:41:14.007 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-03-04T09:41:14.007 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-03-04T09:41:14.008 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-03-04T09:41:14.008 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-03-04T09:41:14.008 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:14.008 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.008 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.008 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.008 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.008 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-03-04T09:41:14.009 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-03-04T09:41:14.009 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:14.009 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.009 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:14.009 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-03-04T09:41:14.009 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.009 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.009 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:14.010 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:14.010 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.010 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.010 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.010 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.010 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-03-04T09:41:14.010 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-03-04T09:41:14.011 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:14.011 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.011 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:14.011 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-03-04T09:41:14.011 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.011 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.011 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-03-04T09:41:14.011 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-03-04T09:41:14.011 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-03-04T09:41:14.012 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-03-04T09:41:14.012 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-03-04T09:41:14.012 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-03-04T09:41:14.012 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-03-04T09:41:14.012 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-03-04T09:41:14.012 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:14.012 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.012 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.013 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.013 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.013 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-03-04T09:41:14.013 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-03-04T09:41:14.013 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:14.013 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.013 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:14.013 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-03-04T09:41:14.013 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.014 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.014 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:14.014 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:14.014 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.014 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.014 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.014 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.014 INFO:teuthology.orchestra.run.smithi052.stdout: "password": { 2024-03-04T09:41:14.015 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "password", 2024-03-04T09:41:14.015 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:14.015 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.015 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:14.015 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-03-04T09:41:14.015 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.015 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.015 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:14.016 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "password of InfluxDB server user", 2024-03-04T09:41:14.016 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.016 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.016 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.016 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.016 INFO:teuthology.orchestra.run.smithi052.stdout: "port": { 2024-03-04T09:41:14.016 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "port", 2024-03-04T09:41:14.016 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-03-04T09:41:14.017 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.017 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:14.017 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "8086", 2024-03-04T09:41:14.017 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.017 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.017 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:14.017 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "InfluxDB server port", 2024-03-04T09:41:14.017 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.018 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.018 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.018 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.018 INFO:teuthology.orchestra.run.smithi052.stdout: "ssl": { 2024-03-04T09:41:14.018 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "ssl", 2024-03-04T09:41:14.018 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:14.018 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.018 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:14.019 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "false", 2024-03-04T09:41:14.019 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.019 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.019 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:14.019 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-03-04T09:41:14.019 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.019 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.019 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.020 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.020 INFO:teuthology.orchestra.run.smithi052.stdout: "threads": { 2024-03-04T09:41:14.020 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "threads", 2024-03-04T09:41:14.020 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-03-04T09:41:14.020 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.020 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:14.020 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "5", 2024-03-04T09:41:14.020 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "1", 2024-03-04T09:41:14.021 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "32", 2024-03-04T09:41:14.021 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:14.021 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-03-04T09:41:14.021 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.021 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.021 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.021 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.021 INFO:teuthology.orchestra.run.smithi052.stdout: "username": { 2024-03-04T09:41:14.021 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "username", 2024-03-04T09:41:14.022 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:14.022 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.022 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:14.022 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-03-04T09:41:14.022 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.022 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.022 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:14.022 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "username of InfluxDB server user", 2024-03-04T09:41:14.023 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.023 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.023 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.023 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.023 INFO:teuthology.orchestra.run.smithi052.stdout: "verify_ssl": { 2024-03-04T09:41:14.023 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "verify_ssl", 2024-03-04T09:41:14.023 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:14.023 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.023 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:14.024 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "true", 2024-03-04T09:41:14.024 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.024 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.024 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:14.024 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-03-04T09:41:14.024 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.024 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.025 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.025 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-03-04T09:41:14.025 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-03-04T09:41:14.025 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.025 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-03-04T09:41:14.025 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "insights", 2024-03-04T09:41:14.025 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-03-04T09:41:14.025 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-03-04T09:41:14.025 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-03-04T09:41:14.026 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-03-04T09:41:14.026 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-03-04T09:41:14.026 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:14.026 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.026 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:14.026 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-03-04T09:41:14.026 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.026 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.027 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-03-04T09:41:14.027 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-03-04T09:41:14.027 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-03-04T09:41:14.027 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-03-04T09:41:14.027 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-03-04T09:41:14.027 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-03-04T09:41:14.027 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-03-04T09:41:14.027 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-03-04T09:41:14.028 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:14.028 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.028 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.028 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.028 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.028 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-03-04T09:41:14.028 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-03-04T09:41:14.028 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:14.029 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.029 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:14.029 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-03-04T09:41:14.029 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.029 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.029 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:14.029 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:14.029 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.030 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.030 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.030 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.030 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-03-04T09:41:14.030 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-03-04T09:41:14.030 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:14.030 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.030 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:14.030 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-03-04T09:41:14.031 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.031 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.031 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-03-04T09:41:14.031 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-03-04T09:41:14.031 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-03-04T09:41:14.031 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-03-04T09:41:14.031 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-03-04T09:41:14.031 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-03-04T09:41:14.032 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-03-04T09:41:14.032 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-03-04T09:41:14.032 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:14.032 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.032 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.032 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.032 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.032 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-03-04T09:41:14.033 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-03-04T09:41:14.033 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:14.033 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.033 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:14.033 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-03-04T09:41:14.033 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.033 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.033 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:14.034 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:14.034 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.034 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.034 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.034 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-03-04T09:41:14.034 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-03-04T09:41:14.034 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.034 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-03-04T09:41:14.035 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "iostat", 2024-03-04T09:41:14.035 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-03-04T09:41:14.035 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-03-04T09:41:14.035 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-03-04T09:41:14.035 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-03-04T09:41:14.035 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-03-04T09:41:14.035 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:14.035 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.035 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:14.036 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-03-04T09:41:14.036 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.036 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.036 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-03-04T09:41:14.036 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-03-04T09:41:14.036 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-03-04T09:41:14.036 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-03-04T09:41:14.037 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-03-04T09:41:14.037 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-03-04T09:41:14.037 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-03-04T09:41:14.037 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-03-04T09:41:14.037 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:14.037 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.037 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.037 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.037 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.038 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-03-04T09:41:14.038 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-03-04T09:41:14.038 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:14.038 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.038 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:14.038 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-03-04T09:41:14.038 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.038 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.039 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:14.039 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:14.039 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.039 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.039 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.039 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.039 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-03-04T09:41:14.039 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-03-04T09:41:14.039 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:14.040 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.040 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:14.040 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-03-04T09:41:14.040 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.040 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.040 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-03-04T09:41:14.040 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-03-04T09:41:14.041 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-03-04T09:41:14.041 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-03-04T09:41:14.041 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-03-04T09:41:14.041 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-03-04T09:41:14.041 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-03-04T09:41:14.041 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-03-04T09:41:14.041 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:14.041 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.042 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.042 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.042 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.042 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-03-04T09:41:14.042 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-03-04T09:41:14.042 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:14.042 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.042 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:14.043 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-03-04T09:41:14.043 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.043 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.043 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:14.043 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:14.043 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.043 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.044 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.044 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-03-04T09:41:14.044 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-03-04T09:41:14.044 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.044 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-03-04T09:41:14.044 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "k8sevents", 2024-03-04T09:41:14.044 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-03-04T09:41:14.045 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-03-04T09:41:14.045 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-03-04T09:41:14.045 INFO:teuthology.orchestra.run.smithi052.stdout: "ceph_event_retention_days": { 2024-03-04T09:41:14.045 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "ceph_event_retention_days", 2024-03-04T09:41:14.045 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-03-04T09:41:14.045 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.045 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:14.046 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "7", 2024-03-04T09:41:14.046 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.046 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.046 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:14.046 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Days to hold ceph event information within local cache", 2024-03-04T09:41:14.046 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.046 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.046 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.046 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.047 INFO:teuthology.orchestra.run.smithi052.stdout: "config_check_secs": { 2024-03-04T09:41:14.047 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "config_check_secs", 2024-03-04T09:41:14.047 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-03-04T09:41:14.047 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.047 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:14.047 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "10", 2024-03-04T09:41:14.047 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "10", 2024-03-04T09:41:14.047 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.048 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:14.048 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-03-04T09:41:14.048 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.048 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.048 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.048 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.048 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-03-04T09:41:14.049 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-03-04T09:41:14.049 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:14.049 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.049 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:14.049 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-03-04T09:41:14.049 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.049 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.049 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-03-04T09:41:14.050 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-03-04T09:41:14.050 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-03-04T09:41:14.050 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-03-04T09:41:14.050 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-03-04T09:41:14.050 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-03-04T09:41:14.050 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-03-04T09:41:14.050 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-03-04T09:41:14.050 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:14.051 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.051 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.051 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.051 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.051 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-03-04T09:41:14.051 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-03-04T09:41:14.051 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:14.051 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.052 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:14.052 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-03-04T09:41:14.052 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.052 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.052 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:14.052 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:14.052 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.053 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.053 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.053 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.053 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-03-04T09:41:14.053 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-03-04T09:41:14.053 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:14.053 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.053 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:14.053 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-03-04T09:41:14.054 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.054 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.054 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-03-04T09:41:14.054 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-03-04T09:41:14.054 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-03-04T09:41:14.054 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-03-04T09:41:14.054 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-03-04T09:41:14.054 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-03-04T09:41:14.055 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-03-04T09:41:14.055 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-03-04T09:41:14.055 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:14.055 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.055 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.055 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.055 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.056 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-03-04T09:41:14.056 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-03-04T09:41:14.056 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:14.056 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.056 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:14.056 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-03-04T09:41:14.056 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.056 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.057 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:14.057 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:14.057 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.057 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.057 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.057 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-03-04T09:41:14.057 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-03-04T09:41:14.057 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.058 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-03-04T09:41:14.058 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "localpool", 2024-03-04T09:41:14.058 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-03-04T09:41:14.058 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-03-04T09:41:14.058 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-03-04T09:41:14.058 INFO:teuthology.orchestra.run.smithi052.stdout: "failure_domain": { 2024-03-04T09:41:14.058 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "failure_domain", 2024-03-04T09:41:14.059 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:14.059 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.059 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:14.059 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "host", 2024-03-04T09:41:14.059 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.059 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.059 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:14.059 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "failure domain for any created local pool", 2024-03-04T09:41:14.059 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-03-04T09:41:14.060 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.060 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.060 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.060 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-03-04T09:41:14.060 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-03-04T09:41:14.060 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:14.060 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.060 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:14.061 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-03-04T09:41:14.061 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.061 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.061 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-03-04T09:41:14.061 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-03-04T09:41:14.061 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-03-04T09:41:14.061 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-03-04T09:41:14.061 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-03-04T09:41:14.062 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-03-04T09:41:14.062 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-03-04T09:41:14.062 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-03-04T09:41:14.062 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:14.062 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.062 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.062 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.062 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.062 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-03-04T09:41:14.063 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-03-04T09:41:14.063 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:14.063 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.063 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:14.063 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-03-04T09:41:14.063 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.063 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.064 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:14.064 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:14.064 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.064 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.064 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.064 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.064 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-03-04T09:41:14.064 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-03-04T09:41:14.065 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:14.065 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.065 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:14.065 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-03-04T09:41:14.065 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.065 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.065 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-03-04T09:41:14.065 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-03-04T09:41:14.066 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-03-04T09:41:14.066 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-03-04T09:41:14.066 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-03-04T09:41:14.066 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-03-04T09:41:14.066 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-03-04T09:41:14.066 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-03-04T09:41:14.066 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:14.066 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.066 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.067 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.067 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.067 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-03-04T09:41:14.067 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-03-04T09:41:14.067 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:14.067 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.067 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:14.067 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-03-04T09:41:14.067 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.068 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.068 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:14.068 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:14.068 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.068 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.068 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.068 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.068 INFO:teuthology.orchestra.run.smithi052.stdout: "min_size": { 2024-03-04T09:41:14.069 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "min_size", 2024-03-04T09:41:14.069 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-03-04T09:41:14.069 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.069 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:14.069 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-03-04T09:41:14.069 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.069 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.069 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:14.069 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "default min_size for any created local pool", 2024-03-04T09:41:14.070 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-03-04T09:41:14.070 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.070 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.070 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.070 INFO:teuthology.orchestra.run.smithi052.stdout: "num_rep": { 2024-03-04T09:41:14.070 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "num_rep", 2024-03-04T09:41:14.070 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-03-04T09:41:14.070 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.071 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:14.071 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "3", 2024-03-04T09:41:14.071 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.071 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.071 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:14.071 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "default replica count for any created local pool", 2024-03-04T09:41:14.071 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.071 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.071 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.072 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.072 INFO:teuthology.orchestra.run.smithi052.stdout: "pg_num": { 2024-03-04T09:41:14.072 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "pg_num", 2024-03-04T09:41:14.072 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-03-04T09:41:14.072 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.072 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:14.072 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "128", 2024-03-04T09:41:14.072 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.073 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.073 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:14.073 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "default pg_num for any created local pool", 2024-03-04T09:41:14.073 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.073 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.073 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.073 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.073 INFO:teuthology.orchestra.run.smithi052.stdout: "prefix": { 2024-03-04T09:41:14.074 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "prefix", 2024-03-04T09:41:14.074 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:14.074 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.074 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:14.074 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-03-04T09:41:14.074 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.074 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.074 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:14.075 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "name prefix for any created local pool", 2024-03-04T09:41:14.075 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.075 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.075 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.075 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.075 INFO:teuthology.orchestra.run.smithi052.stdout: "subtree": { 2024-03-04T09:41:14.075 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "subtree", 2024-03-04T09:41:14.075 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:14.076 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.076 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:14.076 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "rack", 2024-03-04T09:41:14.076 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.076 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.076 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:14.076 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "CRUSH level for which to create a local pool", 2024-03-04T09:41:14.076 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-03-04T09:41:14.077 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.077 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.077 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-03-04T09:41:14.077 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-03-04T09:41:14.077 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.077 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-03-04T09:41:14.077 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "mds_autoscaler", 2024-03-04T09:41:14.077 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-03-04T09:41:14.078 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-03-04T09:41:14.078 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-03-04T09:41:14.078 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-03-04T09:41:14.078 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-03-04T09:41:14.078 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:14.078 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.078 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:14.078 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-03-04T09:41:14.079 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.079 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.079 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-03-04T09:41:14.079 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-03-04T09:41:14.079 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-03-04T09:41:14.079 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-03-04T09:41:14.079 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-03-04T09:41:14.079 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-03-04T09:41:14.079 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-03-04T09:41:14.080 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-03-04T09:41:14.080 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:14.080 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.083 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.083 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.084 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.084 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-03-04T09:41:14.084 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-03-04T09:41:14.084 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:14.084 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.085 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:14.085 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-03-04T09:41:14.085 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.085 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.085 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:14.085 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:14.085 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.085 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.086 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.086 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.086 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-03-04T09:41:14.086 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-03-04T09:41:14.086 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:14.086 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.086 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:14.086 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-03-04T09:41:14.087 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.087 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.087 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-03-04T09:41:14.087 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-03-04T09:41:14.087 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-03-04T09:41:14.087 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-03-04T09:41:14.087 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-03-04T09:41:14.087 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-03-04T09:41:14.088 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-03-04T09:41:14.088 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-03-04T09:41:14.088 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:14.088 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.088 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.088 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.088 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.088 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-03-04T09:41:14.089 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-03-04T09:41:14.089 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:14.089 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.089 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:14.089 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-03-04T09:41:14.089 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.089 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.089 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:14.090 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:14.090 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.090 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.090 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.090 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-03-04T09:41:14.090 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-03-04T09:41:14.090 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.090 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-03-04T09:41:14.091 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "mirroring", 2024-03-04T09:41:14.091 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-03-04T09:41:14.091 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-03-04T09:41:14.091 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-03-04T09:41:14.091 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-03-04T09:41:14.091 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-03-04T09:41:14.091 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:14.091 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.091 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:14.092 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-03-04T09:41:14.092 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.092 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.092 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-03-04T09:41:14.092 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-03-04T09:41:14.092 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-03-04T09:41:14.092 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-03-04T09:41:14.092 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-03-04T09:41:14.093 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-03-04T09:41:14.093 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-03-04T09:41:14.093 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-03-04T09:41:14.093 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:14.093 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.093 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.093 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.093 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.093 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-03-04T09:41:14.094 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-03-04T09:41:14.094 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:14.094 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.094 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:14.094 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-03-04T09:41:14.094 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.094 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.094 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:14.095 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:14.095 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.095 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.095 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.095 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.095 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-03-04T09:41:14.095 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-03-04T09:41:14.095 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:14.095 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.096 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:14.096 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-03-04T09:41:14.096 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.096 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.096 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-03-04T09:41:14.096 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-03-04T09:41:14.096 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-03-04T09:41:14.096 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-03-04T09:41:14.097 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-03-04T09:41:14.097 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-03-04T09:41:14.097 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-03-04T09:41:14.097 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-03-04T09:41:14.097 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:14.097 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.097 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.097 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.098 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.098 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-03-04T09:41:14.098 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-03-04T09:41:14.098 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:14.098 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.098 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:14.098 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-03-04T09:41:14.098 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.099 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.099 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:14.099 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:14.099 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.099 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.099 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.099 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-03-04T09:41:14.099 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-03-04T09:41:14.099 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.100 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-03-04T09:41:14.100 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "nfs", 2024-03-04T09:41:14.100 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-03-04T09:41:14.100 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-03-04T09:41:14.100 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-03-04T09:41:14.100 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-03-04T09:41:14.100 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-03-04T09:41:14.100 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:14.101 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.101 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:14.101 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-03-04T09:41:14.101 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.101 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.101 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-03-04T09:41:14.101 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-03-04T09:41:14.101 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-03-04T09:41:14.101 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-03-04T09:41:14.102 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-03-04T09:41:14.102 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-03-04T09:41:14.102 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-03-04T09:41:14.102 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-03-04T09:41:14.102 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:14.102 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.102 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.102 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.103 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.103 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-03-04T09:41:14.103 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-03-04T09:41:14.103 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:14.103 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.103 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:14.103 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-03-04T09:41:14.103 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.103 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.104 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:14.104 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:14.104 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.104 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.104 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.104 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.104 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-03-04T09:41:14.104 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-03-04T09:41:14.105 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:14.105 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.105 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:14.105 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-03-04T09:41:14.105 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.105 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.105 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-03-04T09:41:14.105 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-03-04T09:41:14.106 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-03-04T09:41:14.106 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-03-04T09:41:14.106 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-03-04T09:41:14.106 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-03-04T09:41:14.106 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-03-04T09:41:14.106 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-03-04T09:41:14.106 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:14.106 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.106 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.107 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.107 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.107 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-03-04T09:41:14.107 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-03-04T09:41:14.107 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:14.107 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.107 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:14.108 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-03-04T09:41:14.108 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.108 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.108 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:14.108 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:14.108 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.108 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.109 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.109 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-03-04T09:41:14.109 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-03-04T09:41:14.109 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.109 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-03-04T09:41:14.109 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "orchestrator", 2024-03-04T09:41:14.109 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-03-04T09:41:14.109 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-03-04T09:41:14.110 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-03-04T09:41:14.110 INFO:teuthology.orchestra.run.smithi052.stdout: "fail_fs": { 2024-03-04T09:41:14.110 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "fail_fs", 2024-03-04T09:41:14.110 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:14.110 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.110 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:14.110 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-03-04T09:41:14.110 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.110 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.111 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:14.111 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-03-04T09:41:14.111 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.111 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.111 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.111 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.111 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-03-04T09:41:14.111 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-03-04T09:41:14.112 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:14.112 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.112 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:14.112 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-03-04T09:41:14.112 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.112 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.112 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-03-04T09:41:14.112 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-03-04T09:41:14.113 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-03-04T09:41:14.113 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-03-04T09:41:14.113 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-03-04T09:41:14.113 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-03-04T09:41:14.113 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-03-04T09:41:14.113 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-03-04T09:41:14.113 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:14.113 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.114 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.114 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.114 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.114 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-03-04T09:41:14.114 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-03-04T09:41:14.114 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:14.114 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.114 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:14.114 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-03-04T09:41:14.115 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.115 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.115 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:14.115 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:14.115 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.115 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.115 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.115 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.116 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-03-04T09:41:14.116 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-03-04T09:41:14.116 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:14.116 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.116 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:14.116 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-03-04T09:41:14.116 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.116 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.117 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-03-04T09:41:14.117 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-03-04T09:41:14.117 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-03-04T09:41:14.117 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-03-04T09:41:14.117 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-03-04T09:41:14.117 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-03-04T09:41:14.117 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-03-04T09:41:14.117 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-03-04T09:41:14.118 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:14.118 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.118 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.118 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.118 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.118 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-03-04T09:41:14.118 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-03-04T09:41:14.118 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:14.118 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.119 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:14.119 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-03-04T09:41:14.119 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.119 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.119 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:14.119 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:14.119 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.119 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.120 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.120 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.120 INFO:teuthology.orchestra.run.smithi052.stdout: "orchestrator": { 2024-03-04T09:41:14.120 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "orchestrator", 2024-03-04T09:41:14.120 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:14.120 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.120 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:14.121 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-03-04T09:41:14.121 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.121 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.121 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-03-04T09:41:14.121 INFO:teuthology.orchestra.run.smithi052.stdout: "cephadm", 2024-03-04T09:41:14.121 INFO:teuthology.orchestra.run.smithi052.stdout: "rook", 2024-03-04T09:41:14.121 INFO:teuthology.orchestra.run.smithi052.stdout: "test_orchestrator" 2024-03-04T09:41:14.121 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-03-04T09:41:14.121 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Orchestrator backend", 2024-03-04T09:41:14.122 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.122 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.122 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.122 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-03-04T09:41:14.122 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-03-04T09:41:14.122 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.122 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-03-04T09:41:14.122 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "osd_perf_query", 2024-03-04T09:41:14.123 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-03-04T09:41:14.123 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-03-04T09:41:14.123 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-03-04T09:41:14.123 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-03-04T09:41:14.123 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-03-04T09:41:14.123 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:14.123 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.123 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:14.123 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-03-04T09:41:14.123 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.124 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.124 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-03-04T09:41:14.124 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-03-04T09:41:14.124 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-03-04T09:41:14.124 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-03-04T09:41:14.124 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-03-04T09:41:14.124 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-03-04T09:41:14.124 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-03-04T09:41:14.125 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-03-04T09:41:14.125 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:14.125 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.125 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.125 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.125 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.125 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-03-04T09:41:14.125 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-03-04T09:41:14.125 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:14.125 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.126 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:14.126 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-03-04T09:41:14.126 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.126 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.126 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:14.126 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:14.126 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.126 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.127 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.127 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.127 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-03-04T09:41:14.127 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-03-04T09:41:14.127 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:14.127 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.127 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:14.127 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-03-04T09:41:14.128 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.128 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.128 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-03-04T09:41:14.128 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-03-04T09:41:14.128 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-03-04T09:41:14.128 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-03-04T09:41:14.128 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-03-04T09:41:14.128 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-03-04T09:41:14.129 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-03-04T09:41:14.129 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-03-04T09:41:14.129 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:14.129 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.129 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.129 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.129 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.129 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-03-04T09:41:14.130 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-03-04T09:41:14.130 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:14.130 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.130 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:14.130 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-03-04T09:41:14.130 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.130 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.130 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:14.131 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:14.131 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.131 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.131 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.131 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-03-04T09:41:14.131 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-03-04T09:41:14.131 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.131 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-03-04T09:41:14.131 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "osd_support", 2024-03-04T09:41:14.132 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-03-04T09:41:14.132 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-03-04T09:41:14.132 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-03-04T09:41:14.132 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-03-04T09:41:14.132 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-03-04T09:41:14.132 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:14.132 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.132 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:14.133 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-03-04T09:41:14.133 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.133 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.133 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-03-04T09:41:14.133 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-03-04T09:41:14.133 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-03-04T09:41:14.133 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-03-04T09:41:14.133 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-03-04T09:41:14.134 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-03-04T09:41:14.134 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-03-04T09:41:14.134 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-03-04T09:41:14.134 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:14.134 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.134 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.134 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.134 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.135 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-03-04T09:41:14.135 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-03-04T09:41:14.135 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:14.135 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.135 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:14.135 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-03-04T09:41:14.135 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.135 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.135 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:14.136 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:14.136 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.136 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.136 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.136 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.136 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-03-04T09:41:14.136 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-03-04T09:41:14.136 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:14.137 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.137 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:14.137 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-03-04T09:41:14.137 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.137 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.137 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-03-04T09:41:14.137 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-03-04T09:41:14.137 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-03-04T09:41:14.137 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-03-04T09:41:14.138 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-03-04T09:41:14.138 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-03-04T09:41:14.138 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-03-04T09:41:14.138 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-03-04T09:41:14.138 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:14.138 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.138 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.138 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.139 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.139 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-03-04T09:41:14.139 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-03-04T09:41:14.139 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:14.139 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.139 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:14.139 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-03-04T09:41:14.139 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.139 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.140 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:14.140 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:14.140 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.140 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.140 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.140 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-03-04T09:41:14.140 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-03-04T09:41:14.140 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.141 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-03-04T09:41:14.141 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "pg_autoscaler", 2024-03-04T09:41:14.141 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-03-04T09:41:14.141 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-03-04T09:41:14.141 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-03-04T09:41:14.141 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-03-04T09:41:14.141 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-03-04T09:41:14.141 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:14.142 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.142 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:14.142 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-03-04T09:41:14.142 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.142 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.142 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-03-04T09:41:14.142 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-03-04T09:41:14.142 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-03-04T09:41:14.143 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-03-04T09:41:14.143 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-03-04T09:41:14.143 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-03-04T09:41:14.143 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-03-04T09:41:14.143 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-03-04T09:41:14.143 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:14.143 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.143 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.144 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.144 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.144 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-03-04T09:41:14.144 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-03-04T09:41:14.144 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:14.144 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.144 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:14.145 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-03-04T09:41:14.145 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.145 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.145 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:14.145 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:14.145 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.145 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.145 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.145 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.146 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-03-04T09:41:14.146 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-03-04T09:41:14.146 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:14.146 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.146 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:14.146 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-03-04T09:41:14.146 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.146 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.147 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-03-04T09:41:14.147 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-03-04T09:41:14.147 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-03-04T09:41:14.147 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-03-04T09:41:14.147 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-03-04T09:41:14.147 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-03-04T09:41:14.147 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-03-04T09:41:14.147 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-03-04T09:41:14.148 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:14.148 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.148 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.148 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.148 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.148 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-03-04T09:41:14.148 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-03-04T09:41:14.148 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:14.149 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.149 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:14.149 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-03-04T09:41:14.149 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.149 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.149 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:14.149 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:14.149 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.150 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.150 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.150 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.150 INFO:teuthology.orchestra.run.smithi052.stdout: "sleep_interval": { 2024-03-04T09:41:14.150 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "sleep_interval", 2024-03-04T09:41:14.150 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "secs", 2024-03-04T09:41:14.150 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.150 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:14.150 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "60", 2024-03-04T09:41:14.151 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.151 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.151 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:14.151 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:14.151 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.151 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.151 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.151 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.152 INFO:teuthology.orchestra.run.smithi052.stdout: "threshold": { 2024-03-04T09:41:14.152 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "threshold", 2024-03-04T09:41:14.152 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "float", 2024-03-04T09:41:14.152 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.152 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:14.152 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "3.0", 2024-03-04T09:41:14.152 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "1.0", 2024-03-04T09:41:14.152 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.153 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:14.153 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "scaling threshold", 2024-03-04T09:41:14.153 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-03-04T09:41:14.153 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.153 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.153 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-03-04T09:41:14.153 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-03-04T09:41:14.153 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.154 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-03-04T09:41:14.154 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "progress", 2024-03-04T09:41:14.154 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-03-04T09:41:14.154 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-03-04T09:41:14.154 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-03-04T09:41:14.154 INFO:teuthology.orchestra.run.smithi052.stdout: "allow_pg_recovery_event": { 2024-03-04T09:41:14.154 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "allow_pg_recovery_event", 2024-03-04T09:41:14.154 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:14.154 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.155 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:14.155 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-03-04T09:41:14.155 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.155 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.155 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:14.155 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "allow the module to show pg recovery progress", 2024-03-04T09:41:14.155 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.156 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.156 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.156 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.156 INFO:teuthology.orchestra.run.smithi052.stdout: "enabled": { 2024-03-04T09:41:14.156 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "enabled", 2024-03-04T09:41:14.156 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:14.156 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.156 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:14.157 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-03-04T09:41:14.157 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.157 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.157 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:14.157 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:14.157 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.157 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.157 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.158 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.158 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-03-04T09:41:14.158 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-03-04T09:41:14.158 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:14.158 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.158 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:14.158 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-03-04T09:41:14.158 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.159 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.159 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-03-04T09:41:14.159 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-03-04T09:41:14.159 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-03-04T09:41:14.159 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-03-04T09:41:14.159 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-03-04T09:41:14.159 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-03-04T09:41:14.159 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-03-04T09:41:14.159 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-03-04T09:41:14.160 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:14.160 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.160 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.160 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.160 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.160 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-03-04T09:41:14.160 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-03-04T09:41:14.160 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:14.161 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.161 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:14.161 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-03-04T09:41:14.161 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.161 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.161 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:14.161 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:14.161 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.162 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.162 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.162 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.162 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-03-04T09:41:14.162 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-03-04T09:41:14.162 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:14.162 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.162 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:14.162 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-03-04T09:41:14.163 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.163 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.163 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-03-04T09:41:14.163 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-03-04T09:41:14.163 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-03-04T09:41:14.163 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-03-04T09:41:14.163 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-03-04T09:41:14.163 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-03-04T09:41:14.164 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-03-04T09:41:14.164 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-03-04T09:41:14.164 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:14.164 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.164 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.164 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.164 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.165 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-03-04T09:41:14.165 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-03-04T09:41:14.165 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:14.165 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.165 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:14.165 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-03-04T09:41:14.165 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.165 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.165 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:14.166 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:14.166 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.166 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.166 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.166 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.166 INFO:teuthology.orchestra.run.smithi052.stdout: "max_completed_events": { 2024-03-04T09:41:14.166 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "max_completed_events", 2024-03-04T09:41:14.166 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-03-04T09:41:14.167 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.167 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:14.167 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "50", 2024-03-04T09:41:14.167 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.167 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.167 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:14.167 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "number of past completed events to remember", 2024-03-04T09:41:14.167 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.168 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.168 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.168 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.168 INFO:teuthology.orchestra.run.smithi052.stdout: "sleep_interval": { 2024-03-04T09:41:14.168 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "sleep_interval", 2024-03-04T09:41:14.168 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "secs", 2024-03-04T09:41:14.168 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.168 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:14.169 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "5", 2024-03-04T09:41:14.169 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.169 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.169 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:14.169 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "how long the module is going to sleep", 2024-03-04T09:41:14.169 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.169 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.169 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.170 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-03-04T09:41:14.170 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-03-04T09:41:14.170 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.170 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-03-04T09:41:14.170 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "prometheus", 2024-03-04T09:41:14.170 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-03-04T09:41:14.170 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-03-04T09:41:14.171 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-03-04T09:41:14.171 INFO:teuthology.orchestra.run.smithi052.stdout: "cache": { 2024-03-04T09:41:14.171 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "cache", 2024-03-04T09:41:14.171 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:14.171 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.171 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:14.171 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-03-04T09:41:14.171 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.172 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.172 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:14.172 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:14.172 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.172 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.172 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.172 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.172 INFO:teuthology.orchestra.run.smithi052.stdout: "exclude_perf_counters": { 2024-03-04T09:41:14.173 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "exclude_perf_counters", 2024-03-04T09:41:14.173 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:14.173 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.173 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:14.173 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-03-04T09:41:14.173 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.173 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.173 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:14.173 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-03-04T09:41:14.174 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-03-04T09:41:14.174 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.174 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.174 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.174 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-03-04T09:41:14.174 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-03-04T09:41:14.174 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:14.174 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.175 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:14.175 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-03-04T09:41:14.175 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.175 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.175 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-03-04T09:41:14.175 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-03-04T09:41:14.175 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-03-04T09:41:14.175 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-03-04T09:41:14.175 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-03-04T09:41:14.176 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-03-04T09:41:14.176 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-03-04T09:41:14.176 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-03-04T09:41:14.176 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:14.176 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.176 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.176 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.176 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.177 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-03-04T09:41:14.177 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-03-04T09:41:14.177 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:14.177 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.177 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:14.177 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-03-04T09:41:14.177 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.177 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.178 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:14.178 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:14.178 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.178 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.178 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.178 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.178 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-03-04T09:41:14.178 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-03-04T09:41:14.179 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:14.179 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.179 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:14.179 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-03-04T09:41:14.179 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.179 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.179 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-03-04T09:41:14.179 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-03-04T09:41:14.180 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-03-04T09:41:14.180 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-03-04T09:41:14.180 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-03-04T09:41:14.180 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-03-04T09:41:14.180 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-03-04T09:41:14.180 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-03-04T09:41:14.180 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:14.180 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.181 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.181 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.181 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.181 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-03-04T09:41:14.181 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-03-04T09:41:14.181 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:14.181 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.181 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:14.182 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-03-04T09:41:14.182 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.182 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.182 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:14.182 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:14.182 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.182 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.182 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.182 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.183 INFO:teuthology.orchestra.run.smithi052.stdout: "rbd_stats_pools": { 2024-03-04T09:41:14.183 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "rbd_stats_pools", 2024-03-04T09:41:14.183 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:14.183 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.183 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:14.183 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-03-04T09:41:14.183 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.184 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.184 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:14.184 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:14.184 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.184 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.184 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.184 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.184 INFO:teuthology.orchestra.run.smithi052.stdout: "rbd_stats_pools_refresh_interval": { 2024-03-04T09:41:14.184 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-03-04T09:41:14.185 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-03-04T09:41:14.185 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.185 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:14.185 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "300", 2024-03-04T09:41:14.185 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.185 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.185 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:14.186 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:14.186 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.186 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.186 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.186 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.186 INFO:teuthology.orchestra.run.smithi052.stdout: "scrape_interval": { 2024-03-04T09:41:14.186 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "scrape_interval", 2024-03-04T09:41:14.186 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "float", 2024-03-04T09:41:14.187 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.187 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:14.187 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "15.0", 2024-03-04T09:41:14.187 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.187 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.187 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:14.187 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:14.187 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.188 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.188 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.188 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.188 INFO:teuthology.orchestra.run.smithi052.stdout: "server_addr": { 2024-03-04T09:41:14.188 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "server_addr", 2024-03-04T09:41:14.188 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:14.188 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.188 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:14.189 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "::", 2024-03-04T09:41:14.189 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.189 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.189 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:14.189 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-03-04T09:41:14.189 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.189 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.189 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.190 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.190 INFO:teuthology.orchestra.run.smithi052.stdout: "server_port": { 2024-03-04T09:41:14.190 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "server_port", 2024-03-04T09:41:14.190 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-03-04T09:41:14.190 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.190 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:14.190 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "9283", 2024-03-04T09:41:14.190 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.191 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.191 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:14.191 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-03-04T09:41:14.191 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.191 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.191 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.191 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.192 INFO:teuthology.orchestra.run.smithi052.stdout: "stale_cache_strategy": { 2024-03-04T09:41:14.192 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "stale_cache_strategy", 2024-03-04T09:41:14.192 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:14.192 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.192 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:14.192 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "log", 2024-03-04T09:41:14.192 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.193 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.193 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:14.193 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:14.193 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.193 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.193 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.193 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.193 INFO:teuthology.orchestra.run.smithi052.stdout: "standby_behaviour": { 2024-03-04T09:41:14.193 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "standby_behaviour", 2024-03-04T09:41:14.194 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:14.194 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.194 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:14.194 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "default", 2024-03-04T09:41:14.194 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.194 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.194 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-03-04T09:41:14.194 INFO:teuthology.orchestra.run.smithi052.stdout: "default", 2024-03-04T09:41:14.195 INFO:teuthology.orchestra.run.smithi052.stdout: "error" 2024-03-04T09:41:14.195 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-03-04T09:41:14.195 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:14.195 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.195 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.195 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.195 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.195 INFO:teuthology.orchestra.run.smithi052.stdout: "standby_error_status_code": { 2024-03-04T09:41:14.196 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "standby_error_status_code", 2024-03-04T09:41:14.196 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-03-04T09:41:14.196 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.196 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:14.196 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "500", 2024-03-04T09:41:14.196 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "400", 2024-03-04T09:41:14.196 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "599", 2024-03-04T09:41:14.196 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:14.196 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:14.197 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.197 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.197 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.197 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-03-04T09:41:14.197 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-03-04T09:41:14.197 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.197 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-03-04T09:41:14.197 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "rbd_support", 2024-03-04T09:41:14.198 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-03-04T09:41:14.198 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-03-04T09:41:14.198 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-03-04T09:41:14.198 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-03-04T09:41:14.198 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-03-04T09:41:14.198 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:14.198 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.198 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:14.198 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-03-04T09:41:14.199 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.199 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.199 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-03-04T09:41:14.199 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-03-04T09:41:14.199 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-03-04T09:41:14.199 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-03-04T09:41:14.199 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-03-04T09:41:14.199 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-03-04T09:41:14.200 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-03-04T09:41:14.200 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-03-04T09:41:14.200 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:14.200 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.200 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.200 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.200 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.201 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-03-04T09:41:14.201 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-03-04T09:41:14.201 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:14.201 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.201 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:14.201 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-03-04T09:41:14.201 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.201 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.202 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:14.202 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:14.202 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.202 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.202 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.202 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.202 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-03-04T09:41:14.203 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-03-04T09:41:14.203 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:14.203 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.203 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:14.203 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-03-04T09:41:14.203 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.203 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.204 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-03-04T09:41:14.204 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-03-04T09:41:14.204 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-03-04T09:41:14.204 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-03-04T09:41:14.204 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-03-04T09:41:14.204 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-03-04T09:41:14.204 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-03-04T09:41:14.204 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-03-04T09:41:14.205 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:14.205 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.205 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.205 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.205 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.205 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-03-04T09:41:14.205 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-03-04T09:41:14.205 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:14.206 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.206 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:14.206 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-03-04T09:41:14.206 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.206 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.206 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:14.206 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:14.206 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.207 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.207 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.207 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.207 INFO:teuthology.orchestra.run.smithi052.stdout: "max_concurrent_snap_create": { 2024-03-04T09:41:14.207 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "max_concurrent_snap_create", 2024-03-04T09:41:14.207 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-03-04T09:41:14.207 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.207 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:14.208 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "10", 2024-03-04T09:41:14.208 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.208 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.208 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:14.208 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:14.208 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.208 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.209 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.209 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.209 INFO:teuthology.orchestra.run.smithi052.stdout: "mirror_snapshot_schedule": { 2024-03-04T09:41:14.209 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "mirror_snapshot_schedule", 2024-03-04T09:41:14.209 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:14.209 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.209 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:14.209 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-03-04T09:41:14.210 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.210 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.210 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:14.210 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:14.210 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.210 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.210 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.210 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.211 INFO:teuthology.orchestra.run.smithi052.stdout: "trash_purge_schedule": { 2024-03-04T09:41:14.211 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "trash_purge_schedule", 2024-03-04T09:41:14.211 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:14.211 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.211 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:14.211 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-03-04T09:41:14.211 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.211 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.212 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:14.212 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:14.212 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.212 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.212 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.212 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-03-04T09:41:14.212 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-03-04T09:41:14.212 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.213 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-03-04T09:41:14.213 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "restful", 2024-03-04T09:41:14.213 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-03-04T09:41:14.213 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-03-04T09:41:14.213 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-03-04T09:41:14.213 INFO:teuthology.orchestra.run.smithi052.stdout: "enable_auth": { 2024-03-04T09:41:14.213 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "enable_auth", 2024-03-04T09:41:14.213 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:14.214 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.214 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:14.214 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-03-04T09:41:14.214 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.214 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.214 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:14.214 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:14.214 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.215 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.215 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.215 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.215 INFO:teuthology.orchestra.run.smithi052.stdout: "key_file": { 2024-03-04T09:41:14.215 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "key_file", 2024-03-04T09:41:14.215 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:14.215 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.215 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:14.215 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-03-04T09:41:14.216 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.216 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.216 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:14.216 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:14.216 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.216 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.216 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.217 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.217 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-03-04T09:41:14.217 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-03-04T09:41:14.217 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:14.217 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.217 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:14.217 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-03-04T09:41:14.217 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.217 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.218 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-03-04T09:41:14.218 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-03-04T09:41:14.218 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-03-04T09:41:14.218 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-03-04T09:41:14.218 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-03-04T09:41:14.218 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-03-04T09:41:14.218 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-03-04T09:41:14.218 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-03-04T09:41:14.219 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:14.219 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.219 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.219 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.219 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.219 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-03-04T09:41:14.219 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-03-04T09:41:14.219 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:14.219 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.220 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:14.220 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-03-04T09:41:14.220 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.220 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.220 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:14.220 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:14.220 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.220 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.221 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.221 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.221 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-03-04T09:41:14.221 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-03-04T09:41:14.221 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:14.221 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.221 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:14.221 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-03-04T09:41:14.222 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.222 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.222 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-03-04T09:41:14.222 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-03-04T09:41:14.222 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-03-04T09:41:14.222 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-03-04T09:41:14.222 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-03-04T09:41:14.222 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-03-04T09:41:14.222 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-03-04T09:41:14.223 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-03-04T09:41:14.223 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:14.223 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.223 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.223 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.223 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.223 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-03-04T09:41:14.223 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-03-04T09:41:14.224 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:14.224 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.224 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:14.224 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-03-04T09:41:14.224 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.224 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.224 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:14.224 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:14.224 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.225 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.225 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.225 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.225 INFO:teuthology.orchestra.run.smithi052.stdout: "server_addr": { 2024-03-04T09:41:14.225 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "server_addr", 2024-03-04T09:41:14.225 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:14.225 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.225 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:14.226 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-03-04T09:41:14.226 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.226 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.226 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:14.226 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:14.226 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.226 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.227 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.227 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.227 INFO:teuthology.orchestra.run.smithi052.stdout: "server_port": { 2024-03-04T09:41:14.227 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "server_port", 2024-03-04T09:41:14.227 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:14.227 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.227 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:14.227 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-03-04T09:41:14.228 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.228 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.228 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:14.228 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:14.228 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.228 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.228 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.228 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-03-04T09:41:14.228 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-03-04T09:41:14.229 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.229 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-03-04T09:41:14.229 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "rgw", 2024-03-04T09:41:14.229 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-03-04T09:41:14.229 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-03-04T09:41:14.229 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-03-04T09:41:14.229 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-03-04T09:41:14.229 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-03-04T09:41:14.230 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:14.230 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.230 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:14.230 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-03-04T09:41:14.230 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.230 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.230 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-03-04T09:41:14.230 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-03-04T09:41:14.231 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-03-04T09:41:14.231 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-03-04T09:41:14.231 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-03-04T09:41:14.231 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-03-04T09:41:14.231 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-03-04T09:41:14.231 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-03-04T09:41:14.231 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:14.231 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.231 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.232 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.232 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.232 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-03-04T09:41:14.232 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-03-04T09:41:14.232 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:14.232 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.232 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:14.232 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-03-04T09:41:14.233 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.233 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.233 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:14.233 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:14.233 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.233 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.233 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.233 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.233 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-03-04T09:41:14.234 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-03-04T09:41:14.234 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:14.234 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.234 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:14.234 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-03-04T09:41:14.234 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.234 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.234 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-03-04T09:41:14.235 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-03-04T09:41:14.235 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-03-04T09:41:14.235 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-03-04T09:41:14.235 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-03-04T09:41:14.235 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-03-04T09:41:14.235 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-03-04T09:41:14.235 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-03-04T09:41:14.235 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:14.235 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.236 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.236 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.236 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.236 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-03-04T09:41:14.236 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-03-04T09:41:14.236 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:14.236 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.236 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:14.237 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-03-04T09:41:14.237 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.237 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.237 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:14.237 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:14.237 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.237 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.237 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.238 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-03-04T09:41:14.238 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-03-04T09:41:14.238 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.238 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-03-04T09:41:14.238 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "rook", 2024-03-04T09:41:14.238 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-03-04T09:41:14.238 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-03-04T09:41:14.238 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-03-04T09:41:14.238 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-03-04T09:41:14.239 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-03-04T09:41:14.239 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:14.239 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.239 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:14.239 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-03-04T09:41:14.239 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.239 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.239 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-03-04T09:41:14.240 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-03-04T09:41:14.240 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-03-04T09:41:14.240 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-03-04T09:41:14.240 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-03-04T09:41:14.240 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-03-04T09:41:14.240 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-03-04T09:41:14.240 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-03-04T09:41:14.240 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:14.241 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.241 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.241 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.241 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.241 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-03-04T09:41:14.241 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-03-04T09:41:14.241 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:14.241 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.242 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:14.242 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-03-04T09:41:14.242 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.242 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.242 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:14.242 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:14.242 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.242 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.243 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.243 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.243 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-03-04T09:41:14.243 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-03-04T09:41:14.243 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:14.243 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.243 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:14.243 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-03-04T09:41:14.244 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.244 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.244 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-03-04T09:41:14.244 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-03-04T09:41:14.244 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-03-04T09:41:14.244 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-03-04T09:41:14.244 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-03-04T09:41:14.244 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-03-04T09:41:14.245 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-03-04T09:41:14.245 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-03-04T09:41:14.245 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:14.245 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.245 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.245 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.245 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.245 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-03-04T09:41:14.246 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-03-04T09:41:14.246 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:14.246 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.246 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:14.246 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-03-04T09:41:14.246 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.246 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.246 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:14.247 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:14.247 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.247 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.247 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.247 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.247 INFO:teuthology.orchestra.run.smithi052.stdout: "storage_class": { 2024-03-04T09:41:14.247 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "storage_class", 2024-03-04T09:41:14.247 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:14.248 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.248 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:14.248 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "local", 2024-03-04T09:41:14.248 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.248 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.248 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:14.248 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-03-04T09:41:14.248 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.249 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.249 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.249 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-03-04T09:41:14.249 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-03-04T09:41:14.249 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.249 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-03-04T09:41:14.249 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "selftest", 2024-03-04T09:41:14.249 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-03-04T09:41:14.249 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-03-04T09:41:14.250 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-03-04T09:41:14.250 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-03-04T09:41:14.250 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-03-04T09:41:14.250 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:14.250 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.250 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:14.250 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-03-04T09:41:14.251 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.251 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.251 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-03-04T09:41:14.251 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-03-04T09:41:14.251 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-03-04T09:41:14.251 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-03-04T09:41:14.251 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-03-04T09:41:14.251 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-03-04T09:41:14.251 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-03-04T09:41:14.252 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-03-04T09:41:14.252 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:14.252 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.252 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.252 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.252 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.252 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-03-04T09:41:14.252 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-03-04T09:41:14.253 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:14.253 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.253 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:14.253 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-03-04T09:41:14.253 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.253 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.253 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:14.253 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:14.254 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.254 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.254 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.254 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.254 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-03-04T09:41:14.254 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-03-04T09:41:14.254 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:14.254 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.254 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:14.255 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-03-04T09:41:14.255 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.255 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.255 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-03-04T09:41:14.255 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-03-04T09:41:14.255 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-03-04T09:41:14.255 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-03-04T09:41:14.255 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-03-04T09:41:14.256 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-03-04T09:41:14.256 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-03-04T09:41:14.256 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-03-04T09:41:14.256 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:14.256 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.256 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.256 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.256 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.257 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-03-04T09:41:14.257 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-03-04T09:41:14.257 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:14.257 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.257 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:14.257 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-03-04T09:41:14.257 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.257 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.258 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:14.258 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:14.258 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.258 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.258 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.258 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.258 INFO:teuthology.orchestra.run.smithi052.stdout: "roption1": { 2024-03-04T09:41:14.258 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "roption1", 2024-03-04T09:41:14.259 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:14.259 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.259 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:14.259 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-03-04T09:41:14.259 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.259 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.259 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:14.259 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:14.260 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.260 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.260 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.260 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.260 INFO:teuthology.orchestra.run.smithi052.stdout: "roption2": { 2024-03-04T09:41:14.260 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "roption2", 2024-03-04T09:41:14.260 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:14.260 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.261 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:14.261 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "xyz", 2024-03-04T09:41:14.261 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.261 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.261 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:14.261 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:14.261 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.261 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.262 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.262 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.262 INFO:teuthology.orchestra.run.smithi052.stdout: "rwoption1": { 2024-03-04T09:41:14.262 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "rwoption1", 2024-03-04T09:41:14.262 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:14.262 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.262 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:14.262 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-03-04T09:41:14.263 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.263 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.263 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:14.263 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:14.263 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.263 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.263 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.263 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.264 INFO:teuthology.orchestra.run.smithi052.stdout: "rwoption2": { 2024-03-04T09:41:14.264 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "rwoption2", 2024-03-04T09:41:14.264 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-03-04T09:41:14.264 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.264 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:14.264 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-03-04T09:41:14.264 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.264 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.265 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:14.265 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:14.265 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.265 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.265 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.265 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.265 INFO:teuthology.orchestra.run.smithi052.stdout: "rwoption3": { 2024-03-04T09:41:14.265 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "rwoption3", 2024-03-04T09:41:14.266 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "float", 2024-03-04T09:41:14.266 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.266 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:14.266 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-03-04T09:41:14.266 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.266 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.266 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:14.266 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:14.267 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.267 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.267 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.267 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.267 INFO:teuthology.orchestra.run.smithi052.stdout: "rwoption4": { 2024-03-04T09:41:14.267 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "rwoption4", 2024-03-04T09:41:14.267 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:14.267 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.268 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:14.268 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-03-04T09:41:14.268 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.268 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.268 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:14.268 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:14.268 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.268 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.269 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.269 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.269 INFO:teuthology.orchestra.run.smithi052.stdout: "rwoption5": { 2024-03-04T09:41:14.269 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "rwoption5", 2024-03-04T09:41:14.269 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:14.269 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.269 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:14.269 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-03-04T09:41:14.269 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.270 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.270 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:14.270 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:14.270 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.270 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.270 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.270 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.270 INFO:teuthology.orchestra.run.smithi052.stdout: "rwoption6": { 2024-03-04T09:41:14.271 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "rwoption6", 2024-03-04T09:41:14.271 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:14.271 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.271 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:14.271 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-03-04T09:41:14.271 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.271 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.271 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:14.272 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:14.272 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.272 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.272 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.272 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.272 INFO:teuthology.orchestra.run.smithi052.stdout: "rwoption7": { 2024-03-04T09:41:14.272 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "rwoption7", 2024-03-04T09:41:14.273 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-03-04T09:41:14.273 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.273 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:14.273 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-03-04T09:41:14.273 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "1", 2024-03-04T09:41:14.273 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "42", 2024-03-04T09:41:14.273 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:14.273 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:14.274 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.274 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.274 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.274 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.274 INFO:teuthology.orchestra.run.smithi052.stdout: "testkey": { 2024-03-04T09:41:14.274 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "testkey", 2024-03-04T09:41:14.274 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:14.274 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.275 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:14.275 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-03-04T09:41:14.275 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.275 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.275 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:14.275 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:14.275 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.275 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.276 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.276 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.276 INFO:teuthology.orchestra.run.smithi052.stdout: "testlkey": { 2024-03-04T09:41:14.276 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "testlkey", 2024-03-04T09:41:14.276 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:14.276 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.276 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:14.276 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-03-04T09:41:14.277 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.277 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.277 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:14.277 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:14.277 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.277 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.277 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.277 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.277 INFO:teuthology.orchestra.run.smithi052.stdout: "testnewline": { 2024-03-04T09:41:14.278 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "testnewline", 2024-03-04T09:41:14.278 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:14.278 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.278 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:14.278 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-03-04T09:41:14.278 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.278 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.278 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:14.279 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:14.279 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.279 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.279 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.279 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-03-04T09:41:14.279 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-03-04T09:41:14.279 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.279 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-03-04T09:41:14.279 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "snap_schedule", 2024-03-04T09:41:14.280 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-03-04T09:41:14.280 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-03-04T09:41:14.280 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-03-04T09:41:14.280 INFO:teuthology.orchestra.run.smithi052.stdout: "allow_m_granularity": { 2024-03-04T09:41:14.280 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "allow_m_granularity", 2024-03-04T09:41:14.280 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:14.280 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.280 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:14.281 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-03-04T09:41:14.281 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.281 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.281 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:14.281 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "allow minute scheduled snapshots", 2024-03-04T09:41:14.281 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.281 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.281 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.282 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.282 INFO:teuthology.orchestra.run.smithi052.stdout: "dump_on_update": { 2024-03-04T09:41:14.282 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "dump_on_update", 2024-03-04T09:41:14.282 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:14.282 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.282 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:14.282 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-03-04T09:41:14.282 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.283 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.283 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:14.283 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "dump database to debug log on update", 2024-03-04T09:41:14.283 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.283 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.283 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.283 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.283 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-03-04T09:41:14.284 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-03-04T09:41:14.284 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:14.284 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.284 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:14.284 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-03-04T09:41:14.284 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.284 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.284 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-03-04T09:41:14.285 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-03-04T09:41:14.285 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-03-04T09:41:14.285 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-03-04T09:41:14.285 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-03-04T09:41:14.285 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-03-04T09:41:14.285 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-03-04T09:41:14.285 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-03-04T09:41:14.285 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:14.285 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.286 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.286 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.286 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.286 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-03-04T09:41:14.286 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-03-04T09:41:14.286 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:14.286 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.286 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:14.287 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-03-04T09:41:14.287 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.287 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.287 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:14.287 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:14.288 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.288 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.288 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.288 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.288 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-03-04T09:41:14.288 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-03-04T09:41:14.288 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:14.288 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.289 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:14.289 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-03-04T09:41:14.289 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.289 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.289 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-03-04T09:41:14.289 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-03-04T09:41:14.289 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-03-04T09:41:14.289 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-03-04T09:41:14.290 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-03-04T09:41:14.290 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-03-04T09:41:14.290 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-03-04T09:41:14.290 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-03-04T09:41:14.290 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:14.290 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.290 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.290 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.290 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.291 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-03-04T09:41:14.291 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-03-04T09:41:14.291 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:14.291 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.291 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:14.291 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-03-04T09:41:14.291 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.291 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.292 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:14.292 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:14.292 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.292 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.292 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.292 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-03-04T09:41:14.292 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-03-04T09:41:14.293 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.293 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-03-04T09:41:14.293 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "stats", 2024-03-04T09:41:14.293 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-03-04T09:41:14.293 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-03-04T09:41:14.293 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-03-04T09:41:14.293 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-03-04T09:41:14.293 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-03-04T09:41:14.294 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:14.294 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.294 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:14.294 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-03-04T09:41:14.294 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.294 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.294 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-03-04T09:41:14.294 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-03-04T09:41:14.294 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-03-04T09:41:14.295 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-03-04T09:41:14.295 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-03-04T09:41:14.295 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-03-04T09:41:14.295 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-03-04T09:41:14.295 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-03-04T09:41:14.295 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:14.295 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.295 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.295 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.296 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.296 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-03-04T09:41:14.296 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-03-04T09:41:14.296 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:14.296 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.296 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:14.296 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-03-04T09:41:14.296 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.296 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.297 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:14.297 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:14.297 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.297 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.297 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.297 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.297 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-03-04T09:41:14.297 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-03-04T09:41:14.297 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:14.298 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.298 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:14.298 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-03-04T09:41:14.298 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.298 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.298 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-03-04T09:41:14.298 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-03-04T09:41:14.299 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-03-04T09:41:14.299 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-03-04T09:41:14.299 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-03-04T09:41:14.299 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-03-04T09:41:14.299 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-03-04T09:41:14.299 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-03-04T09:41:14.299 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:14.299 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.300 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.300 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.300 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.300 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-03-04T09:41:14.300 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-03-04T09:41:14.300 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:14.300 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.300 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:14.300 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-03-04T09:41:14.301 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.301 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.301 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:14.301 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:14.301 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.301 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.301 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.302 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-03-04T09:41:14.302 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-03-04T09:41:14.302 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.302 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-03-04T09:41:14.302 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "status", 2024-03-04T09:41:14.302 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-03-04T09:41:14.302 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-03-04T09:41:14.303 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-03-04T09:41:14.303 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-03-04T09:41:14.303 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-03-04T09:41:14.303 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:14.303 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.303 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:14.303 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-03-04T09:41:14.303 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.304 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.304 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-03-04T09:41:14.304 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-03-04T09:41:14.304 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-03-04T09:41:14.304 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-03-04T09:41:14.304 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-03-04T09:41:14.304 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-03-04T09:41:14.305 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-03-04T09:41:14.305 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-03-04T09:41:14.305 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:14.305 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.305 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.305 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.305 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.305 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-03-04T09:41:14.306 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-03-04T09:41:14.306 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:14.306 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.306 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:14.306 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-03-04T09:41:14.306 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.306 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.306 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:14.307 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:14.307 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.307 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.307 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.307 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.307 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-03-04T09:41:14.307 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-03-04T09:41:14.307 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:14.308 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.308 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:14.308 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-03-04T09:41:14.308 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.308 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.308 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-03-04T09:41:14.308 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-03-04T09:41:14.308 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-03-04T09:41:14.309 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-03-04T09:41:14.309 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-03-04T09:41:14.309 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-03-04T09:41:14.309 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-03-04T09:41:14.309 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-03-04T09:41:14.309 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:14.309 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.309 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.310 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.310 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.310 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-03-04T09:41:14.310 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-03-04T09:41:14.310 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:14.310 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.310 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:14.310 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-03-04T09:41:14.311 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.311 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.311 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:14.311 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:14.311 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.311 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.311 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.311 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-03-04T09:41:14.311 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-03-04T09:41:14.312 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.312 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-03-04T09:41:14.312 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "telegraf", 2024-03-04T09:41:14.312 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-03-04T09:41:14.312 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-03-04T09:41:14.312 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-03-04T09:41:14.312 INFO:teuthology.orchestra.run.smithi052.stdout: "address": { 2024-03-04T09:41:14.313 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "address", 2024-03-04T09:41:14.313 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:14.313 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.313 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:14.313 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-03-04T09:41:14.313 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.313 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.313 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:14.314 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:14.314 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.314 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.314 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.314 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.314 INFO:teuthology.orchestra.run.smithi052.stdout: "interval": { 2024-03-04T09:41:14.314 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "interval", 2024-03-04T09:41:14.314 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "secs", 2024-03-04T09:41:14.315 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.315 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:14.315 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "15", 2024-03-04T09:41:14.315 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.315 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.315 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:14.315 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:14.315 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.315 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.316 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.316 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.316 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-03-04T09:41:14.316 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-03-04T09:41:14.316 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:14.316 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.316 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:14.317 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-03-04T09:41:14.317 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.317 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.317 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-03-04T09:41:14.317 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-03-04T09:41:14.317 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-03-04T09:41:14.317 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-03-04T09:41:14.317 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-03-04T09:41:14.317 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-03-04T09:41:14.318 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-03-04T09:41:14.318 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-03-04T09:41:14.318 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:14.318 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.318 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.318 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.318 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.318 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-03-04T09:41:14.318 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-03-04T09:41:14.318 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:14.319 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.319 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:14.319 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-03-04T09:41:14.319 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.319 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.319 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:14.319 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:14.319 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.320 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.320 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.320 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.320 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-03-04T09:41:14.320 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-03-04T09:41:14.320 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:14.320 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.321 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:14.321 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-03-04T09:41:14.321 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.321 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.321 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-03-04T09:41:14.321 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-03-04T09:41:14.321 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-03-04T09:41:14.321 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-03-04T09:41:14.322 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-03-04T09:41:14.322 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-03-04T09:41:14.322 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-03-04T09:41:14.322 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-03-04T09:41:14.322 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:14.322 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.322 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.322 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.323 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.323 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-03-04T09:41:14.323 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-03-04T09:41:14.323 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:14.323 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.323 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:14.323 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-03-04T09:41:14.323 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.324 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.324 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:14.324 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:14.324 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.324 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.324 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.324 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-03-04T09:41:14.324 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-03-04T09:41:14.325 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.325 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-03-04T09:41:14.325 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "telemetry", 2024-03-04T09:41:14.325 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-03-04T09:41:14.325 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-03-04T09:41:14.325 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-03-04T09:41:14.325 INFO:teuthology.orchestra.run.smithi052.stdout: "channel_basic": { 2024-03-04T09:41:14.325 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "channel_basic", 2024-03-04T09:41:14.326 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:14.326 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.326 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:14.326 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-03-04T09:41:14.326 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.326 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.326 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:14.326 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Share basic cluster information (size, version)", 2024-03-04T09:41:14.326 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.327 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.327 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.327 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.327 INFO:teuthology.orchestra.run.smithi052.stdout: "channel_crash": { 2024-03-04T09:41:14.327 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "channel_crash", 2024-03-04T09:41:14.327 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:14.327 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.328 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:14.328 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-03-04T09:41:14.328 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.328 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.328 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:14.328 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-03-04T09:41:14.328 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.328 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.329 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.329 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.329 INFO:teuthology.orchestra.run.smithi052.stdout: "channel_device": { 2024-03-04T09:41:14.329 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "channel_device", 2024-03-04T09:41:14.329 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:14.329 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.329 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:14.329 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-03-04T09:41:14.330 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.330 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.330 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:14.330 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-03-04T09:41:14.330 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.330 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.330 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.330 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.330 INFO:teuthology.orchestra.run.smithi052.stdout: "channel_ident": { 2024-03-04T09:41:14.331 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "channel_ident", 2024-03-04T09:41:14.331 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:14.331 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.331 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:14.331 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-03-04T09:41:14.331 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.331 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.332 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:14.332 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-03-04T09:41:14.332 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.332 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.332 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.332 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.332 INFO:teuthology.orchestra.run.smithi052.stdout: "channel_perf": { 2024-03-04T09:41:14.333 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "channel_perf", 2024-03-04T09:41:14.333 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:14.333 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.333 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:14.333 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-03-04T09:41:14.333 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.333 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.333 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:14.334 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Share various performance metrics of a cluster", 2024-03-04T09:41:14.334 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.334 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.334 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.334 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.334 INFO:teuthology.orchestra.run.smithi052.stdout: "contact": { 2024-03-04T09:41:14.334 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "contact", 2024-03-04T09:41:14.334 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:14.335 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.335 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:14.335 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-03-04T09:41:14.335 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.335 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.335 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:14.335 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:14.335 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.335 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.336 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.336 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.336 INFO:teuthology.orchestra.run.smithi052.stdout: "description": { 2024-03-04T09:41:14.336 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "description", 2024-03-04T09:41:14.336 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:14.336 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.336 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:14.337 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-03-04T09:41:14.337 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.337 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.337 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:14.337 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:14.337 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.337 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.337 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.338 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.338 INFO:teuthology.orchestra.run.smithi052.stdout: "device_url": { 2024-03-04T09:41:14.338 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "device_url", 2024-03-04T09:41:14.338 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:14.338 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.338 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:14.338 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-03-04T09:41:14.339 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.339 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.339 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:14.339 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:14.339 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.339 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.339 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.339 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.339 INFO:teuthology.orchestra.run.smithi052.stdout: "enabled": { 2024-03-04T09:41:14.340 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "enabled", 2024-03-04T09:41:14.340 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:14.340 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.340 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:14.340 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-03-04T09:41:14.340 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.340 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.341 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:14.341 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:14.341 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.341 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.341 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.341 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.341 INFO:teuthology.orchestra.run.smithi052.stdout: "interval": { 2024-03-04T09:41:14.341 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "interval", 2024-03-04T09:41:14.342 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-03-04T09:41:14.342 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.342 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:14.342 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "24", 2024-03-04T09:41:14.342 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "8", 2024-03-04T09:41:14.342 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.342 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:14.342 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:14.342 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.343 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.343 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.343 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.343 INFO:teuthology.orchestra.run.smithi052.stdout: "last_opt_revision": { 2024-03-04T09:41:14.343 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "last_opt_revision", 2024-03-04T09:41:14.343 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-03-04T09:41:14.343 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.343 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:14.344 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "1", 2024-03-04T09:41:14.344 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.344 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.344 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:14.344 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:14.344 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.344 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.344 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.344 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.345 INFO:teuthology.orchestra.run.smithi052.stdout: "leaderboard": { 2024-03-04T09:41:14.345 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "leaderboard", 2024-03-04T09:41:14.345 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:14.345 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.345 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:14.345 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-03-04T09:41:14.345 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.345 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.346 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:14.346 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:14.346 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.346 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.346 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.346 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.346 INFO:teuthology.orchestra.run.smithi052.stdout: "leaderboard_description": { 2024-03-04T09:41:14.346 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "leaderboard_description", 2024-03-04T09:41:14.347 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:14.347 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.347 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:14.347 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-03-04T09:41:14.347 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.347 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.347 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:14.347 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:14.348 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.348 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.348 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.348 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.348 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-03-04T09:41:14.348 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-03-04T09:41:14.348 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:14.348 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.349 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:14.349 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-03-04T09:41:14.349 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.349 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.349 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-03-04T09:41:14.349 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-03-04T09:41:14.349 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-03-04T09:41:14.349 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-03-04T09:41:14.349 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-03-04T09:41:14.350 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-03-04T09:41:14.350 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-03-04T09:41:14.350 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-03-04T09:41:14.350 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:14.350 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.350 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.350 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.350 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.351 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-03-04T09:41:14.351 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-03-04T09:41:14.351 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:14.351 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.351 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:14.351 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-03-04T09:41:14.351 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.351 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.352 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:14.352 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:14.352 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.352 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.352 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.352 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.352 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-03-04T09:41:14.352 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-03-04T09:41:14.353 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:14.353 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.353 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:14.353 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-03-04T09:41:14.353 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.353 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.353 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-03-04T09:41:14.353 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-03-04T09:41:14.353 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-03-04T09:41:14.354 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-03-04T09:41:14.354 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-03-04T09:41:14.354 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-03-04T09:41:14.354 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-03-04T09:41:14.354 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-03-04T09:41:14.354 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:14.354 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.354 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.355 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.355 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.355 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-03-04T09:41:14.355 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-03-04T09:41:14.355 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:14.355 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.355 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:14.355 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-03-04T09:41:14.356 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.356 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.356 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:14.356 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:14.356 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.356 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.356 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.356 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.356 INFO:teuthology.orchestra.run.smithi052.stdout: "organization": { 2024-03-04T09:41:14.357 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "organization", 2024-03-04T09:41:14.357 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:14.357 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.357 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:14.357 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-03-04T09:41:14.357 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.357 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.357 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:14.358 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:14.358 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.358 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.358 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.358 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.358 INFO:teuthology.orchestra.run.smithi052.stdout: "proxy": { 2024-03-04T09:41:14.358 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "proxy", 2024-03-04T09:41:14.358 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:14.359 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.359 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:14.359 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-03-04T09:41:14.359 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.359 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.359 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:14.359 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:14.360 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.360 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.360 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.360 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.360 INFO:teuthology.orchestra.run.smithi052.stdout: "url": { 2024-03-04T09:41:14.360 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "url", 2024-03-04T09:41:14.361 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:14.361 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.361 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:14.361 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-03-04T09:41:14.361 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.361 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.361 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:14.362 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:14.362 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.362 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.362 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.362 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-03-04T09:41:14.362 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-03-04T09:41:14.362 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.363 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-03-04T09:41:14.363 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "test_orchestrator", 2024-03-04T09:41:14.363 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-03-04T09:41:14.363 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-03-04T09:41:14.363 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-03-04T09:41:14.363 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-03-04T09:41:14.363 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-03-04T09:41:14.363 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:14.364 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.364 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:14.364 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-03-04T09:41:14.364 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.364 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.364 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-03-04T09:41:14.364 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-03-04T09:41:14.365 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-03-04T09:41:14.365 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-03-04T09:41:14.365 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-03-04T09:41:14.365 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-03-04T09:41:14.365 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-03-04T09:41:14.365 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-03-04T09:41:14.365 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:14.365 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.366 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.366 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.366 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.366 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-03-04T09:41:14.366 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-03-04T09:41:14.366 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:14.366 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.366 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:14.367 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-03-04T09:41:14.367 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.367 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.367 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:14.367 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:14.367 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.367 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.367 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.367 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.368 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-03-04T09:41:14.368 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-03-04T09:41:14.368 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:14.368 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.368 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:14.368 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-03-04T09:41:14.368 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.368 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.369 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-03-04T09:41:14.369 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-03-04T09:41:14.369 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-03-04T09:41:14.369 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-03-04T09:41:14.369 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-03-04T09:41:14.369 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-03-04T09:41:14.369 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-03-04T09:41:14.369 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-03-04T09:41:14.369 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:14.370 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.370 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.370 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.370 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.370 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-03-04T09:41:14.370 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-03-04T09:41:14.370 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:14.371 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.371 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:14.371 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-03-04T09:41:14.371 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.371 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.371 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:14.371 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:14.371 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.372 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.372 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.372 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-03-04T09:41:14.372 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-03-04T09:41:14.372 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.372 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-03-04T09:41:14.372 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "volumes", 2024-03-04T09:41:14.372 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-03-04T09:41:14.372 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-03-04T09:41:14.373 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-03-04T09:41:14.373 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-03-04T09:41:14.373 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-03-04T09:41:14.373 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:14.373 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.373 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:14.373 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-03-04T09:41:14.374 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.374 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.374 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-03-04T09:41:14.374 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-03-04T09:41:14.374 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-03-04T09:41:14.374 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-03-04T09:41:14.374 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-03-04T09:41:14.374 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-03-04T09:41:14.374 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-03-04T09:41:14.375 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-03-04T09:41:14.375 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:14.375 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.375 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.375 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.375 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.375 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-03-04T09:41:14.375 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-03-04T09:41:14.376 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:14.376 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.376 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:14.376 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-03-04T09:41:14.376 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.376 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.376 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:14.376 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:14.377 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.377 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.377 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.377 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.377 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-03-04T09:41:14.377 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-03-04T09:41:14.377 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:14.377 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.377 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:14.378 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-03-04T09:41:14.378 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.378 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.378 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-03-04T09:41:14.378 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-03-04T09:41:14.378 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-03-04T09:41:14.378 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-03-04T09:41:14.378 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-03-04T09:41:14.379 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-03-04T09:41:14.379 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-03-04T09:41:14.379 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-03-04T09:41:14.379 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:14.379 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.379 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.379 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.379 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.379 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-03-04T09:41:14.380 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-03-04T09:41:14.380 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:14.380 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.380 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:14.380 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-03-04T09:41:14.380 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.380 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.380 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:14.381 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:14.381 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.381 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.381 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.381 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.381 INFO:teuthology.orchestra.run.smithi052.stdout: "max_concurrent_clones": { 2024-03-04T09:41:14.381 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "max_concurrent_clones", 2024-03-04T09:41:14.381 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-03-04T09:41:14.382 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.382 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:14.382 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "4", 2024-03-04T09:41:14.382 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.382 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.382 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:14.382 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Number of asynchronous cloner threads", 2024-03-04T09:41:14.382 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.382 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.383 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.383 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.383 INFO:teuthology.orchestra.run.smithi052.stdout: "periodic_async_work": { 2024-03-04T09:41:14.383 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "periodic_async_work", 2024-03-04T09:41:14.383 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:14.383 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.383 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:14.383 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-03-04T09:41:14.384 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.384 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.384 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:14.384 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Periodically check for async work", 2024-03-04T09:41:14.384 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.384 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.384 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.384 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.385 INFO:teuthology.orchestra.run.smithi052.stdout: "snapshot_clone_delay": { 2024-03-04T09:41:14.385 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "snapshot_clone_delay", 2024-03-04T09:41:14.385 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-03-04T09:41:14.385 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.385 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:14.385 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "0", 2024-03-04T09:41:14.385 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.386 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.386 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:14.386 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-03-04T09:41:14.386 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.386 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.386 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.386 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.387 INFO:teuthology.orchestra.run.smithi052.stdout: "snapshot_clone_no_wait": { 2024-03-04T09:41:14.387 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "snapshot_clone_no_wait", 2024-03-04T09:41:14.387 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:14.387 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.387 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:14.387 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-03-04T09:41:14.387 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.388 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.388 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:14.388 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-03-04T09:41:14.388 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.388 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.388 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.388 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-03-04T09:41:14.388 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-03-04T09:41:14.389 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.389 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-03-04T09:41:14.389 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "zabbix", 2024-03-04T09:41:14.389 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-03-04T09:41:14.389 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-03-04T09:41:14.389 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-03-04T09:41:14.389 INFO:teuthology.orchestra.run.smithi052.stdout: "discovery_interval": { 2024-03-04T09:41:14.389 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "discovery_interval", 2024-03-04T09:41:14.389 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "uint", 2024-03-04T09:41:14.390 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.390 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:14.390 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "100", 2024-03-04T09:41:14.390 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.390 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.390 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:14.390 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:14.390 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.391 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.391 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.391 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.391 INFO:teuthology.orchestra.run.smithi052.stdout: "identifier": { 2024-03-04T09:41:14.391 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "identifier", 2024-03-04T09:41:14.391 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:14.391 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.391 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:14.391 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-03-04T09:41:14.392 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.392 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.392 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:14.392 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:14.392 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.392 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.392 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.392 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.393 INFO:teuthology.orchestra.run.smithi052.stdout: "interval": { 2024-03-04T09:41:14.393 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "interval", 2024-03-04T09:41:14.393 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "secs", 2024-03-04T09:41:14.393 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.393 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:14.393 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "60", 2024-03-04T09:41:14.393 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.394 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.394 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:14.394 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:14.394 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.394 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.394 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.394 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.394 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-03-04T09:41:14.395 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-03-04T09:41:14.395 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:14.395 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.395 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:14.395 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-03-04T09:41:14.395 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.395 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.395 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-03-04T09:41:14.396 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-03-04T09:41:14.396 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-03-04T09:41:14.396 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-03-04T09:41:14.396 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-03-04T09:41:14.396 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-03-04T09:41:14.396 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-03-04T09:41:14.396 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-03-04T09:41:14.397 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:14.397 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.397 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.397 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.397 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.397 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-03-04T09:41:14.397 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-03-04T09:41:14.397 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:14.398 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.398 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:14.398 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-03-04T09:41:14.398 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.398 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.398 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:14.399 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:14.399 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.399 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.399 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.399 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.399 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-03-04T09:41:14.399 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-03-04T09:41:14.400 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:14.400 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.400 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:14.400 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-03-04T09:41:14.400 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.400 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.401 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-03-04T09:41:14.401 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-03-04T09:41:14.401 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-03-04T09:41:14.401 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-03-04T09:41:14.401 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-03-04T09:41:14.401 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-03-04T09:41:14.401 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-03-04T09:41:14.402 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-03-04T09:41:14.402 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:14.402 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.402 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.402 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.402 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.403 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-03-04T09:41:14.403 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-03-04T09:41:14.403 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:14.403 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.403 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:14.403 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-03-04T09:41:14.404 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.404 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.404 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:14.404 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:14.404 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.405 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.405 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.405 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.405 INFO:teuthology.orchestra.run.smithi052.stdout: "zabbix_host": { 2024-03-04T09:41:14.405 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "zabbix_host", 2024-03-04T09:41:14.405 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:14.405 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.406 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:14.406 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-03-04T09:41:14.406 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.406 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.406 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:14.406 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:14.407 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.407 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.407 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.407 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.407 INFO:teuthology.orchestra.run.smithi052.stdout: "zabbix_port": { 2024-03-04T09:41:14.407 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "zabbix_port", 2024-03-04T09:41:14.408 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-03-04T09:41:14.408 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.408 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:14.408 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "10051", 2024-03-04T09:41:14.408 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.408 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.409 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:14.409 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:14.409 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.409 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.409 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.409 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.410 INFO:teuthology.orchestra.run.smithi052.stdout: "zabbix_sender": { 2024-03-04T09:41:14.410 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "zabbix_sender", 2024-03-04T09:41:14.410 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:14.410 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.410 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:14.410 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-03-04T09:41:14.411 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.411 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.411 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:14.411 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:14.411 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.411 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.411 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.412 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-03-04T09:41:14.412 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-03-04T09:41:14.412 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-03-04T09:41:14.412 INFO:teuthology.orchestra.run.smithi052.stdout: ] 2024-03-04T09:41:14.412 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-03-04T09:41:14.412 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-03-04T09:41:14.412 INFO:teuthology.orchestra.run.smithi052.stdout: "modules": [ 2024-03-04T09:41:14.412 INFO:teuthology.orchestra.run.smithi052.stdout: "cephadm", 2024-03-04T09:41:14.413 INFO:teuthology.orchestra.run.smithi052.stdout: "dashboard", 2024-03-04T09:41:14.413 INFO:teuthology.orchestra.run.smithi052.stdout: "iostat", 2024-03-04T09:41:14.413 INFO:teuthology.orchestra.run.smithi052.stdout: "nfs", 2024-03-04T09:41:14.413 INFO:teuthology.orchestra.run.smithi052.stdout: "restful", 2024-03-04T09:41:14.413 INFO:teuthology.orchestra.run.smithi052.stdout: "snap_schedule" 2024-03-04T09:41:14.413 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-03-04T09:41:14.413 INFO:teuthology.orchestra.run.smithi052.stdout: "available_modules": [ 2024-03-04T09:41:14.413 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-03-04T09:41:14.414 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "alerts", 2024-03-04T09:41:14.414 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-03-04T09:41:14.414 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-03-04T09:41:14.414 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-03-04T09:41:14.414 INFO:teuthology.orchestra.run.smithi052.stdout: "interval": { 2024-03-04T09:41:14.414 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "interval", 2024-03-04T09:41:14.414 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "secs", 2024-03-04T09:41:14.414 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.415 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:14.415 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "60", 2024-03-04T09:41:14.415 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.415 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.415 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:14.415 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "How frequently to reexamine health status", 2024-03-04T09:41:14.415 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.415 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.416 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.416 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.416 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-03-04T09:41:14.416 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-03-04T09:41:14.416 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:14.416 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.416 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:14.416 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-03-04T09:41:14.417 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.417 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.417 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-03-04T09:41:14.417 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-03-04T09:41:14.417 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-03-04T09:41:14.417 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-03-04T09:41:14.417 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-03-04T09:41:14.418 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-03-04T09:41:14.418 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-03-04T09:41:14.418 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-03-04T09:41:14.418 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:14.418 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.418 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.418 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.418 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.418 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-03-04T09:41:14.419 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-03-04T09:41:14.419 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:14.419 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.419 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:14.419 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-03-04T09:41:14.419 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.419 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.419 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:14.420 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:14.420 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.420 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.420 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.420 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.420 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-03-04T09:41:14.421 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-03-04T09:41:14.421 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:14.421 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.421 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:14.421 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-03-04T09:41:14.421 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.422 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.422 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-03-04T09:41:14.422 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-03-04T09:41:14.422 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-03-04T09:41:14.422 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-03-04T09:41:14.422 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-03-04T09:41:14.422 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-03-04T09:41:14.422 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-03-04T09:41:14.422 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-03-04T09:41:14.423 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:14.423 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.423 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.423 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.423 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.423 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-03-04T09:41:14.423 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-03-04T09:41:14.423 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:14.424 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.424 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:14.424 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-03-04T09:41:14.424 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.424 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.424 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:14.424 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:14.424 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.425 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.425 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.425 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.425 INFO:teuthology.orchestra.run.smithi052.stdout: "smtp_destination": { 2024-03-04T09:41:14.425 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "smtp_destination", 2024-03-04T09:41:14.425 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:14.425 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.426 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:14.426 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-03-04T09:41:14.426 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.426 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.426 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:14.426 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Email address to send alerts to", 2024-03-04T09:41:14.426 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.426 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.426 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.427 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.427 INFO:teuthology.orchestra.run.smithi052.stdout: "smtp_from_name": { 2024-03-04T09:41:14.427 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "smtp_from_name", 2024-03-04T09:41:14.427 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:14.427 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.427 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:14.427 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "Ceph", 2024-03-04T09:41:14.427 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.428 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.428 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:14.428 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Email From: name", 2024-03-04T09:41:14.428 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.431 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.431 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.431 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.431 INFO:teuthology.orchestra.run.smithi052.stdout: "smtp_host": { 2024-03-04T09:41:14.432 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "smtp_host", 2024-03-04T09:41:14.432 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:14.432 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.432 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:14.432 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-03-04T09:41:14.432 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.432 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.432 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:14.433 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "SMTP server", 2024-03-04T09:41:14.433 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.433 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.433 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.433 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.433 INFO:teuthology.orchestra.run.smithi052.stdout: "smtp_password": { 2024-03-04T09:41:14.433 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "smtp_password", 2024-03-04T09:41:14.434 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:14.434 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.434 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:14.434 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-03-04T09:41:14.434 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.434 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.434 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:14.434 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Password to authenticate with", 2024-03-04T09:41:14.435 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.435 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.435 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.435 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.435 INFO:teuthology.orchestra.run.smithi052.stdout: "smtp_port": { 2024-03-04T09:41:14.435 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "smtp_port", 2024-03-04T09:41:14.435 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-03-04T09:41:14.436 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.436 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:14.436 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "465", 2024-03-04T09:41:14.436 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.436 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.436 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:14.436 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "SMTP port", 2024-03-04T09:41:14.436 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.437 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.437 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.437 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.437 INFO:teuthology.orchestra.run.smithi052.stdout: "smtp_sender": { 2024-03-04T09:41:14.437 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "smtp_sender", 2024-03-04T09:41:14.437 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:14.437 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.437 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:14.438 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-03-04T09:41:14.438 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.438 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.438 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:14.438 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "SMTP envelope sender", 2024-03-04T09:41:14.438 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.438 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.438 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.439 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.439 INFO:teuthology.orchestra.run.smithi052.stdout: "smtp_ssl": { 2024-03-04T09:41:14.439 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "smtp_ssl", 2024-03-04T09:41:14.439 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:14.439 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.439 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:14.439 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-03-04T09:41:14.440 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.440 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.440 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:14.440 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Use SSL to connect to SMTP server", 2024-03-04T09:41:14.440 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.440 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.440 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.440 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.441 INFO:teuthology.orchestra.run.smithi052.stdout: "smtp_user": { 2024-03-04T09:41:14.441 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "smtp_user", 2024-03-04T09:41:14.441 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:14.441 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.441 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:14.441 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-03-04T09:41:14.441 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.441 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.442 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:14.442 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "User to authenticate as", 2024-03-04T09:41:14.442 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.442 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.442 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.442 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-03-04T09:41:14.442 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-03-04T09:41:14.442 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.443 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-03-04T09:41:14.443 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "balancer", 2024-03-04T09:41:14.443 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-03-04T09:41:14.443 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-03-04T09:41:14.443 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-03-04T09:41:14.443 INFO:teuthology.orchestra.run.smithi052.stdout: "active": { 2024-03-04T09:41:14.443 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "active", 2024-03-04T09:41:14.443 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:14.443 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.444 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:14.444 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-03-04T09:41:14.444 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.444 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.444 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:14.444 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "automatically balance PGs across cluster", 2024-03-04T09:41:14.444 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.444 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.445 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.445 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.445 INFO:teuthology.orchestra.run.smithi052.stdout: "begin_time": { 2024-03-04T09:41:14.445 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "begin_time", 2024-03-04T09:41:14.445 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:14.445 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.445 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:14.445 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "0000", 2024-03-04T09:41:14.446 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.446 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.446 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:14.446 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "beginning time of day to automatically balance", 2024-03-04T09:41:14.446 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-03-04T09:41:14.446 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.446 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.446 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.447 INFO:teuthology.orchestra.run.smithi052.stdout: "begin_weekday": { 2024-03-04T09:41:14.447 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "begin_weekday", 2024-03-04T09:41:14.447 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "uint", 2024-03-04T09:41:14.447 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.447 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:14.447 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "0", 2024-03-04T09:41:14.447 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "0", 2024-03-04T09:41:14.447 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "6", 2024-03-04T09:41:14.448 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:14.448 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-03-04T09:41:14.448 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-03-04T09:41:14.448 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.448 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.448 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.448 INFO:teuthology.orchestra.run.smithi052.stdout: "crush_compat_max_iterations": { 2024-03-04T09:41:14.448 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "crush_compat_max_iterations", 2024-03-04T09:41:14.449 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "uint", 2024-03-04T09:41:14.449 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.449 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:14.449 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "25", 2024-03-04T09:41:14.449 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "1", 2024-03-04T09:41:14.449 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "250", 2024-03-04T09:41:14.449 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:14.450 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-03-04T09:41:14.450 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.450 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.450 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.450 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.450 INFO:teuthology.orchestra.run.smithi052.stdout: "crush_compat_metrics": { 2024-03-04T09:41:14.450 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "crush_compat_metrics", 2024-03-04T09:41:14.450 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:14.451 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.451 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:14.451 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "pgs,objects,bytes", 2024-03-04T09:41:14.451 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.451 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.451 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:14.451 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-03-04T09:41:14.452 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-03-04T09:41:14.452 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.452 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.452 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.452 INFO:teuthology.orchestra.run.smithi052.stdout: "crush_compat_step": { 2024-03-04T09:41:14.452 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "crush_compat_step", 2024-03-04T09:41:14.452 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "float", 2024-03-04T09:41:14.452 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.453 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:14.453 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "0.5", 2024-03-04T09:41:14.453 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "0.001", 2024-03-04T09:41:14.453 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "0.999", 2024-03-04T09:41:14.453 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:14.453 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "aggressiveness of optimization", 2024-03-04T09:41:14.453 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-03-04T09:41:14.453 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.454 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.454 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.454 INFO:teuthology.orchestra.run.smithi052.stdout: "end_time": { 2024-03-04T09:41:14.454 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "end_time", 2024-03-04T09:41:14.454 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:14.454 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.454 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:14.454 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "2359", 2024-03-04T09:41:14.454 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.455 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.455 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:14.455 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "ending time of day to automatically balance", 2024-03-04T09:41:14.455 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-03-04T09:41:14.455 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.455 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.455 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.455 INFO:teuthology.orchestra.run.smithi052.stdout: "end_weekday": { 2024-03-04T09:41:14.456 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "end_weekday", 2024-03-04T09:41:14.456 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "uint", 2024-03-04T09:41:14.456 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.456 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:14.456 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "0", 2024-03-04T09:41:14.456 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "0", 2024-03-04T09:41:14.456 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "6", 2024-03-04T09:41:14.457 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:14.457 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-03-04T09:41:14.457 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-03-04T09:41:14.457 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.457 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.457 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.457 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-03-04T09:41:14.457 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-03-04T09:41:14.457 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:14.458 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.458 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:14.458 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-03-04T09:41:14.458 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.458 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.458 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-03-04T09:41:14.458 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-03-04T09:41:14.458 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-03-04T09:41:14.459 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-03-04T09:41:14.459 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-03-04T09:41:14.459 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-03-04T09:41:14.459 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-03-04T09:41:14.459 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-03-04T09:41:14.459 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:14.459 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.459 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.459 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.460 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.460 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-03-04T09:41:14.460 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-03-04T09:41:14.460 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:14.460 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.460 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:14.460 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-03-04T09:41:14.460 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.461 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.461 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:14.461 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:14.461 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.461 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.461 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.461 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.461 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-03-04T09:41:14.462 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-03-04T09:41:14.462 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:14.462 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.462 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:14.462 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-03-04T09:41:14.462 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.462 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.462 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-03-04T09:41:14.462 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-03-04T09:41:14.463 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-03-04T09:41:14.463 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-03-04T09:41:14.463 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-03-04T09:41:14.463 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-03-04T09:41:14.463 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-03-04T09:41:14.463 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-03-04T09:41:14.463 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:14.463 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.463 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.464 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.464 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.464 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-03-04T09:41:14.464 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-03-04T09:41:14.464 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:14.464 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.464 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:14.464 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-03-04T09:41:14.465 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.465 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.465 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:14.465 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:14.465 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.465 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.465 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.465 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.465 INFO:teuthology.orchestra.run.smithi052.stdout: "min_score": { 2024-03-04T09:41:14.466 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "min_score", 2024-03-04T09:41:14.466 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "float", 2024-03-04T09:41:14.466 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.466 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:14.466 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "0", 2024-03-04T09:41:14.466 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.466 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.466 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:14.467 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-03-04T09:41:14.467 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.467 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.467 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.467 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.467 INFO:teuthology.orchestra.run.smithi052.stdout: "mode": { 2024-03-04T09:41:14.467 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "mode", 2024-03-04T09:41:14.467 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:14.467 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.468 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:14.468 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "upmap", 2024-03-04T09:41:14.468 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.468 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.468 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-03-04T09:41:14.468 INFO:teuthology.orchestra.run.smithi052.stdout: "crush-compat", 2024-03-04T09:41:14.468 INFO:teuthology.orchestra.run.smithi052.stdout: "none", 2024-03-04T09:41:14.468 INFO:teuthology.orchestra.run.smithi052.stdout: "read", 2024-03-04T09:41:14.469 INFO:teuthology.orchestra.run.smithi052.stdout: "upmap", 2024-03-04T09:41:14.469 INFO:teuthology.orchestra.run.smithi052.stdout: "upmap-read" 2024-03-04T09:41:14.469 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-03-04T09:41:14.469 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Balancer mode", 2024-03-04T09:41:14.469 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.469 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.469 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.469 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.469 INFO:teuthology.orchestra.run.smithi052.stdout: "pool_ids": { 2024-03-04T09:41:14.470 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "pool_ids", 2024-03-04T09:41:14.470 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:14.470 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.470 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:14.470 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-03-04T09:41:14.470 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.470 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.470 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:14.471 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-03-04T09:41:14.471 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.471 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.471 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.471 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.471 INFO:teuthology.orchestra.run.smithi052.stdout: "sleep_interval": { 2024-03-04T09:41:14.471 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "sleep_interval", 2024-03-04T09:41:14.471 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "secs", 2024-03-04T09:41:14.471 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.472 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:14.472 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "60", 2024-03-04T09:41:14.472 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.472 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.472 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:14.472 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-03-04T09:41:14.472 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.472 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.473 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.473 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.473 INFO:teuthology.orchestra.run.smithi052.stdout: "upmap_max_deviation": { 2024-03-04T09:41:14.473 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "upmap_max_deviation", 2024-03-04T09:41:14.473 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-03-04T09:41:14.473 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.473 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:14.473 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "5", 2024-03-04T09:41:14.473 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "1", 2024-03-04T09:41:14.474 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.474 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:14.474 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "deviation below which no optimization is attempted", 2024-03-04T09:41:14.474 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-03-04T09:41:14.474 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.474 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.474 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.474 INFO:teuthology.orchestra.run.smithi052.stdout: "upmap_max_optimizations": { 2024-03-04T09:41:14.475 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "upmap_max_optimizations", 2024-03-04T09:41:14.475 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "uint", 2024-03-04T09:41:14.475 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.475 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:14.475 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "10", 2024-03-04T09:41:14.475 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.475 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.475 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:14.475 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-03-04T09:41:14.476 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.476 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.476 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.476 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-03-04T09:41:14.476 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-03-04T09:41:14.476 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.476 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-03-04T09:41:14.476 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "cephadm", 2024-03-04T09:41:14.477 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-03-04T09:41:14.477 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-03-04T09:41:14.477 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-03-04T09:41:14.477 INFO:teuthology.orchestra.run.smithi052.stdout: "agent_down_multiplier": { 2024-03-04T09:41:14.477 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "agent_down_multiplier", 2024-03-04T09:41:14.477 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "float", 2024-03-04T09:41:14.477 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.477 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:14.478 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "3.0", 2024-03-04T09:41:14.478 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.478 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.478 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:14.478 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-03-04T09:41:14.478 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.478 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.478 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.479 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.479 INFO:teuthology.orchestra.run.smithi052.stdout: "agent_refresh_rate": { 2024-03-04T09:41:14.479 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "agent_refresh_rate", 2024-03-04T09:41:14.479 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "secs", 2024-03-04T09:41:14.479 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.479 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:14.479 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "20", 2024-03-04T09:41:14.479 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.479 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.480 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:14.480 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-03-04T09:41:14.480 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.480 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.480 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.480 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.480 INFO:teuthology.orchestra.run.smithi052.stdout: "agent_starting_port": { 2024-03-04T09:41:14.481 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "agent_starting_port", 2024-03-04T09:41:14.481 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-03-04T09:41:14.481 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.481 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:14.481 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "4721", 2024-03-04T09:41:14.481 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.481 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.481 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:14.481 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-03-04T09:41:14.482 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.482 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.482 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.482 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.482 INFO:teuthology.orchestra.run.smithi052.stdout: "allow_ptrace": { 2024-03-04T09:41:14.482 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "allow_ptrace", 2024-03-04T09:41:14.482 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:14.482 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.483 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:14.483 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-03-04T09:41:14.483 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.483 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.483 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:14.483 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-03-04T09:41:14.483 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-03-04T09:41:14.483 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.483 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.484 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.484 INFO:teuthology.orchestra.run.smithi052.stdout: "autotune_interval": { 2024-03-04T09:41:14.484 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "autotune_interval", 2024-03-04T09:41:14.484 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "secs", 2024-03-04T09:41:14.484 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.484 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:14.484 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "600", 2024-03-04T09:41:14.485 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.485 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.485 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:14.485 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "how frequently to autotune daemon memory", 2024-03-04T09:41:14.485 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.485 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.485 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.485 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.486 INFO:teuthology.orchestra.run.smithi052.stdout: "autotune_memory_target_ratio": { 2024-03-04T09:41:14.486 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "autotune_memory_target_ratio", 2024-03-04T09:41:14.486 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "float", 2024-03-04T09:41:14.486 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.486 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:14.486 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "0.7", 2024-03-04T09:41:14.486 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.486 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.487 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:14.487 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-03-04T09:41:14.487 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.487 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.487 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.487 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.487 INFO:teuthology.orchestra.run.smithi052.stdout: "cephadm_log_destination": { 2024-03-04T09:41:14.487 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "cephadm_log_destination", 2024-03-04T09:41:14.488 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:14.488 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.488 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:14.488 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-03-04T09:41:14.488 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.488 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.488 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-03-04T09:41:14.488 INFO:teuthology.orchestra.run.smithi052.stdout: "file", 2024-03-04T09:41:14.489 INFO:teuthology.orchestra.run.smithi052.stdout: "file,syslog", 2024-03-04T09:41:14.489 INFO:teuthology.orchestra.run.smithi052.stdout: "syslog" 2024-03-04T09:41:14.489 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-03-04T09:41:14.489 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-03-04T09:41:14.489 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.489 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.489 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.489 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.490 INFO:teuthology.orchestra.run.smithi052.stdout: "cgroups_split": { 2024-03-04T09:41:14.490 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "cgroups_split", 2024-03-04T09:41:14.490 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:14.490 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.490 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:14.490 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-03-04T09:41:14.490 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.490 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.491 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:14.491 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-03-04T09:41:14.491 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.491 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.491 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.491 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.491 INFO:teuthology.orchestra.run.smithi052.stdout: "config_checks_enabled": { 2024-03-04T09:41:14.491 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "config_checks_enabled", 2024-03-04T09:41:14.491 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:14.492 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.492 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:14.492 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-03-04T09:41:14.492 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.492 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.492 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:14.492 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-03-04T09:41:14.492 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.493 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.493 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.493 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.493 INFO:teuthology.orchestra.run.smithi052.stdout: "config_dashboard": { 2024-03-04T09:41:14.493 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "config_dashboard", 2024-03-04T09:41:14.493 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:14.493 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.493 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:14.494 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-03-04T09:41:14.494 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.494 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.494 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:14.494 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-03-04T09:41:14.494 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.494 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.494 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.495 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.495 INFO:teuthology.orchestra.run.smithi052.stdout: "container_image_alertmanager": { 2024-03-04T09:41:14.495 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "container_image_alertmanager", 2024-03-04T09:41:14.495 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:14.495 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.495 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:14.495 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-03-04T09:41:14.495 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.496 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.496 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:14.496 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Prometheus container image", 2024-03-04T09:41:14.496 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.496 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.496 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.496 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.497 INFO:teuthology.orchestra.run.smithi052.stdout: "container_image_base": { 2024-03-04T09:41:14.497 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "container_image_base", 2024-03-04T09:41:14.497 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:14.497 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.497 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:14.497 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "quay.io/ceph/ceph", 2024-03-04T09:41:14.497 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.497 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.497 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:14.498 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Container image name, without the tag", 2024-03-04T09:41:14.498 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.498 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.498 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.498 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.498 INFO:teuthology.orchestra.run.smithi052.stdout: "container_image_elasticsearch": { 2024-03-04T09:41:14.498 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "container_image_elasticsearch", 2024-03-04T09:41:14.498 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:14.499 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.499 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:14.499 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-03-04T09:41:14.499 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.499 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.499 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:14.499 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "elasticsearch container image", 2024-03-04T09:41:14.499 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.500 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.500 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.500 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.500 INFO:teuthology.orchestra.run.smithi052.stdout: "container_image_grafana": { 2024-03-04T09:41:14.500 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "container_image_grafana", 2024-03-04T09:41:14.500 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:14.500 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.500 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:14.501 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "quay.io/ceph/ceph-grafana:9.4.12", 2024-03-04T09:41:14.501 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.501 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.501 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:14.501 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Prometheus container image", 2024-03-04T09:41:14.501 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.501 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.502 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.502 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.502 INFO:teuthology.orchestra.run.smithi052.stdout: "container_image_haproxy": { 2024-03-04T09:41:14.502 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "container_image_haproxy", 2024-03-04T09:41:14.502 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:14.502 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.502 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:14.502 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-03-04T09:41:14.503 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.503 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.503 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:14.503 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "HAproxy container image", 2024-03-04T09:41:14.503 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.503 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.503 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.503 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.504 INFO:teuthology.orchestra.run.smithi052.stdout: "container_image_jaeger_agent": { 2024-03-04T09:41:14.504 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "container_image_jaeger_agent", 2024-03-04T09:41:14.504 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:14.504 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.504 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:14.504 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-03-04T09:41:14.504 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.504 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.504 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:14.505 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Jaeger agent container image", 2024-03-04T09:41:14.505 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.505 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.505 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.505 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.505 INFO:teuthology.orchestra.run.smithi052.stdout: "container_image_jaeger_collector": { 2024-03-04T09:41:14.505 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "container_image_jaeger_collector", 2024-03-04T09:41:14.505 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:14.506 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.506 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:14.506 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-03-04T09:41:14.506 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.506 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.506 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:14.506 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Jaeger collector container image", 2024-03-04T09:41:14.506 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.507 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.507 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.507 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.507 INFO:teuthology.orchestra.run.smithi052.stdout: "container_image_jaeger_query": { 2024-03-04T09:41:14.507 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "container_image_jaeger_query", 2024-03-04T09:41:14.507 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:14.507 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.507 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:14.508 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-03-04T09:41:14.508 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.508 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.508 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:14.508 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Jaeger query container image", 2024-03-04T09:41:14.508 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.508 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.508 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.509 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.509 INFO:teuthology.orchestra.run.smithi052.stdout: "container_image_keepalived": { 2024-03-04T09:41:14.509 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "container_image_keepalived", 2024-03-04T09:41:14.509 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:14.509 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.509 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:14.509 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-03-04T09:41:14.509 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.509 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.510 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:14.510 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Keepalived container image", 2024-03-04T09:41:14.510 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.510 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.510 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.510 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.510 INFO:teuthology.orchestra.run.smithi052.stdout: "container_image_loki": { 2024-03-04T09:41:14.511 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "container_image_loki", 2024-03-04T09:41:14.511 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:14.511 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.511 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:14.511 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-03-04T09:41:14.511 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.511 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.511 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:14.511 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Loki container image", 2024-03-04T09:41:14.512 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.512 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.512 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.512 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.512 INFO:teuthology.orchestra.run.smithi052.stdout: "container_image_node_exporter": { 2024-03-04T09:41:14.512 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "container_image_node_exporter", 2024-03-04T09:41:14.512 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:14.512 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.513 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:14.513 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-03-04T09:41:14.513 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.513 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.513 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:14.513 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Prometheus container image", 2024-03-04T09:41:14.513 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.513 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.513 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.514 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.514 INFO:teuthology.orchestra.run.smithi052.stdout: "container_image_nvmeof": { 2024-03-04T09:41:14.514 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "container_image_nvmeof", 2024-03-04T09:41:14.514 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:14.514 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.514 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:14.514 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "quay.io/ceph/nvmeof:latest", 2024-03-04T09:41:14.514 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.515 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.515 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:14.515 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Nvme-of container image", 2024-03-04T09:41:14.515 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.515 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.515 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.515 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.515 INFO:teuthology.orchestra.run.smithi052.stdout: "container_image_prometheus": { 2024-03-04T09:41:14.516 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "container_image_prometheus", 2024-03-04T09:41:14.516 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:14.516 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.516 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:14.516 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-03-04T09:41:14.516 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.516 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.516 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:14.517 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Prometheus container image", 2024-03-04T09:41:14.517 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.517 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.517 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.517 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.517 INFO:teuthology.orchestra.run.smithi052.stdout: "container_image_promtail": { 2024-03-04T09:41:14.517 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "container_image_promtail", 2024-03-04T09:41:14.517 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:14.518 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.518 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:14.518 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-03-04T09:41:14.518 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.518 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.518 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:14.518 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Promtail container image", 2024-03-04T09:41:14.518 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.519 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.519 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.519 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.519 INFO:teuthology.orchestra.run.smithi052.stdout: "container_image_snmp_gateway": { 2024-03-04T09:41:14.519 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "container_image_snmp_gateway", 2024-03-04T09:41:14.519 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:14.519 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.519 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:14.520 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-03-04T09:41:14.520 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.520 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.520 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:14.520 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "SNMP Gateway container image", 2024-03-04T09:41:14.520 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.520 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.521 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.521 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.521 INFO:teuthology.orchestra.run.smithi052.stdout: "container_init": { 2024-03-04T09:41:14.521 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "container_init", 2024-03-04T09:41:14.521 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:14.521 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.521 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:14.521 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-03-04T09:41:14.521 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.522 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.522 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:14.522 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Run podman/docker with `--init`", 2024-03-04T09:41:14.522 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.522 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.522 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.522 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.522 INFO:teuthology.orchestra.run.smithi052.stdout: "daemon_cache_timeout": { 2024-03-04T09:41:14.523 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "daemon_cache_timeout", 2024-03-04T09:41:14.523 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "secs", 2024-03-04T09:41:14.523 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.523 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:14.523 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "600", 2024-03-04T09:41:14.523 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.523 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.523 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:14.524 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-03-04T09:41:14.524 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.524 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.524 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.524 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.524 INFO:teuthology.orchestra.run.smithi052.stdout: "default_cephadm_command_timeout": { 2024-03-04T09:41:14.524 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "default_cephadm_command_timeout", 2024-03-04T09:41:14.524 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-03-04T09:41:14.525 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.525 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:14.525 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "900", 2024-03-04T09:41:14.525 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.525 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.525 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:14.525 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-03-04T09:41:14.525 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.526 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.526 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.526 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.526 INFO:teuthology.orchestra.run.smithi052.stdout: "default_registry": { 2024-03-04T09:41:14.526 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "default_registry", 2024-03-04T09:41:14.526 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:14.526 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.526 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:14.527 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "docker.io", 2024-03-04T09:41:14.527 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.527 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.527 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:14.527 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-03-04T09:41:14.527 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.527 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.527 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.527 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.528 INFO:teuthology.orchestra.run.smithi052.stdout: "device_cache_timeout": { 2024-03-04T09:41:14.528 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "device_cache_timeout", 2024-03-04T09:41:14.528 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "secs", 2024-03-04T09:41:14.528 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.528 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:14.528 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "1800", 2024-03-04T09:41:14.528 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.528 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.529 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:14.529 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "seconds to cache device inventory", 2024-03-04T09:41:14.529 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.529 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.529 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.529 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.529 INFO:teuthology.orchestra.run.smithi052.stdout: "device_enhanced_scan": { 2024-03-04T09:41:14.529 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "device_enhanced_scan", 2024-03-04T09:41:14.530 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:14.530 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.530 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:14.530 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-03-04T09:41:14.530 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.530 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.530 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:14.530 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Use libstoragemgmt during device scans", 2024-03-04T09:41:14.530 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.531 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.531 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.531 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.531 INFO:teuthology.orchestra.run.smithi052.stdout: "facts_cache_timeout": { 2024-03-04T09:41:14.531 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "facts_cache_timeout", 2024-03-04T09:41:14.531 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "secs", 2024-03-04T09:41:14.531 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.531 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:14.532 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "60", 2024-03-04T09:41:14.532 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.532 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.532 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:14.532 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "seconds to cache host facts data", 2024-03-04T09:41:14.532 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.532 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.532 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.533 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.533 INFO:teuthology.orchestra.run.smithi052.stdout: "host_check_interval": { 2024-03-04T09:41:14.533 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "host_check_interval", 2024-03-04T09:41:14.533 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "secs", 2024-03-04T09:41:14.533 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.533 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:14.533 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "600", 2024-03-04T09:41:14.533 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.534 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.534 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:14.534 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "how frequently to perform a host check", 2024-03-04T09:41:14.534 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.534 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.534 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.534 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.534 INFO:teuthology.orchestra.run.smithi052.stdout: "hw_monitoring": { 2024-03-04T09:41:14.535 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "hw_monitoring", 2024-03-04T09:41:14.535 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:14.535 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.535 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:14.535 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-03-04T09:41:14.535 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.535 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.535 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:14.536 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-03-04T09:41:14.536 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.536 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.536 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.536 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.536 INFO:teuthology.orchestra.run.smithi052.stdout: "inventory_list_all": { 2024-03-04T09:41:14.536 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "inventory_list_all", 2024-03-04T09:41:14.536 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:14.537 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.537 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:14.537 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-03-04T09:41:14.537 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.537 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.537 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:14.537 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-03-04T09:41:14.538 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.538 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.538 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.538 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.538 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-03-04T09:41:14.538 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-03-04T09:41:14.538 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:14.538 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.539 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:14.539 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-03-04T09:41:14.539 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.539 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.539 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-03-04T09:41:14.539 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-03-04T09:41:14.539 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-03-04T09:41:14.539 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-03-04T09:41:14.540 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-03-04T09:41:14.540 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-03-04T09:41:14.540 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-03-04T09:41:14.540 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-03-04T09:41:14.540 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:14.540 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.540 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.540 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.541 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.541 INFO:teuthology.orchestra.run.smithi052.stdout: "log_refresh_metadata": { 2024-03-04T09:41:14.541 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_refresh_metadata", 2024-03-04T09:41:14.541 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:14.541 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.541 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:14.541 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-03-04T09:41:14.541 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.542 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.542 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:14.542 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-03-04T09:41:14.542 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.542 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.542 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.542 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.542 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-03-04T09:41:14.543 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-03-04T09:41:14.543 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:14.543 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.543 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:14.543 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-03-04T09:41:14.543 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.543 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.543 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:14.543 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-03-04T09:41:14.544 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.544 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.544 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.544 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.544 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-03-04T09:41:14.544 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-03-04T09:41:14.544 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:14.544 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.545 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:14.545 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-03-04T09:41:14.545 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.545 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.545 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-03-04T09:41:14.545 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-03-04T09:41:14.545 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-03-04T09:41:14.545 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-03-04T09:41:14.546 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-03-04T09:41:14.546 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-03-04T09:41:14.546 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-03-04T09:41:14.546 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-03-04T09:41:14.546 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:14.546 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.546 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.546 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.547 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.547 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-03-04T09:41:14.547 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-03-04T09:41:14.547 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:14.547 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.547 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:14.547 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-03-04T09:41:14.547 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.547 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.548 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:14.548 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:14.548 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.548 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.548 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.548 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.548 INFO:teuthology.orchestra.run.smithi052.stdout: "manage_etc_ceph_ceph_conf": { 2024-03-04T09:41:14.548 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-03-04T09:41:14.549 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:14.549 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.549 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:14.549 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-03-04T09:41:14.549 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.549 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.549 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:14.549 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-03-04T09:41:14.550 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.550 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.550 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.550 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.550 INFO:teuthology.orchestra.run.smithi052.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-03-04T09:41:14.550 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-03-04T09:41:14.550 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:14.550 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.550 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:14.551 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "*", 2024-03-04T09:41:14.551 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.551 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.551 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:14.551 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-03-04T09:41:14.551 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.551 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.551 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.552 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.552 INFO:teuthology.orchestra.run.smithi052.stdout: "max_count_per_host": { 2024-03-04T09:41:14.552 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "max_count_per_host", 2024-03-04T09:41:14.552 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-03-04T09:41:14.552 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.552 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:14.553 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "10", 2024-03-04T09:41:14.553 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.553 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.553 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:14.553 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "max number of daemons per service per host", 2024-03-04T09:41:14.553 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.553 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.553 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.553 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.554 INFO:teuthology.orchestra.run.smithi052.stdout: "max_osd_draining_count": { 2024-03-04T09:41:14.554 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "max_osd_draining_count", 2024-03-04T09:41:14.554 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-03-04T09:41:14.554 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.554 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:14.554 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "10", 2024-03-04T09:41:14.554 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.554 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.555 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:14.555 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-03-04T09:41:14.555 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.555 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.555 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.555 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.555 INFO:teuthology.orchestra.run.smithi052.stdout: "migration_current": { 2024-03-04T09:41:14.555 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "migration_current", 2024-03-04T09:41:14.556 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-03-04T09:41:14.556 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.556 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:14.556 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-03-04T09:41:14.556 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.556 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.556 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:14.557 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "internal - do not modify", 2024-03-04T09:41:14.557 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.557 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.557 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.557 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.557 INFO:teuthology.orchestra.run.smithi052.stdout: "mode": { 2024-03-04T09:41:14.557 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "mode", 2024-03-04T09:41:14.557 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:14.558 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.558 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:14.558 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "root", 2024-03-04T09:41:14.558 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.558 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.558 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-03-04T09:41:14.558 INFO:teuthology.orchestra.run.smithi052.stdout: "cephadm-package", 2024-03-04T09:41:14.559 INFO:teuthology.orchestra.run.smithi052.stdout: "root" 2024-03-04T09:41:14.559 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-03-04T09:41:14.559 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "mode for remote execution of cephadm", 2024-03-04T09:41:14.559 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.559 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.559 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.559 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.559 INFO:teuthology.orchestra.run.smithi052.stdout: "oob_default_addr": { 2024-03-04T09:41:14.560 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "oob_default_addr", 2024-03-04T09:41:14.560 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:14.560 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.560 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:14.560 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "169.254.1.1", 2024-03-04T09:41:14.560 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.560 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.560 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:14.561 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Default address for RedFish API (oob management).", 2024-03-04T09:41:14.561 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.561 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.561 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.561 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.561 INFO:teuthology.orchestra.run.smithi052.stdout: "prometheus_alerts_path": { 2024-03-04T09:41:14.561 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "prometheus_alerts_path", 2024-03-04T09:41:14.562 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:14.562 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.562 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:14.562 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-03-04T09:41:14.562 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.562 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.562 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:14.563 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-03-04T09:41:14.563 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.563 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.563 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.563 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.563 INFO:teuthology.orchestra.run.smithi052.stdout: "registry_insecure": { 2024-03-04T09:41:14.563 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "registry_insecure", 2024-03-04T09:41:14.563 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:14.564 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.564 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:14.564 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-03-04T09:41:14.564 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.564 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.564 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:14.564 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-03-04T09:41:14.565 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.565 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.565 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.565 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.565 INFO:teuthology.orchestra.run.smithi052.stdout: "registry_password": { 2024-03-04T09:41:14.565 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "registry_password", 2024-03-04T09:41:14.565 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:14.565 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.565 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:14.566 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-03-04T09:41:14.566 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.566 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.566 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:14.566 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-03-04T09:41:14.566 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.566 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.566 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.567 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.567 INFO:teuthology.orchestra.run.smithi052.stdout: "registry_url": { 2024-03-04T09:41:14.567 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "registry_url", 2024-03-04T09:41:14.567 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:14.567 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.567 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:14.567 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-03-04T09:41:14.567 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.567 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.568 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:14.568 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-03-04T09:41:14.568 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.568 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.568 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.568 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.568 INFO:teuthology.orchestra.run.smithi052.stdout: "registry_username": { 2024-03-04T09:41:14.569 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "registry_username", 2024-03-04T09:41:14.569 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:14.569 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.569 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:14.569 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-03-04T09:41:14.569 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.569 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.569 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:14.569 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-03-04T09:41:14.570 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.570 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.570 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.570 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.570 INFO:teuthology.orchestra.run.smithi052.stdout: "secure_monitoring_stack": { 2024-03-04T09:41:14.570 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "secure_monitoring_stack", 2024-03-04T09:41:14.570 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:14.571 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.571 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:14.571 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-03-04T09:41:14.571 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.571 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.571 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:14.571 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-03-04T09:41:14.571 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.572 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.572 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.572 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.572 INFO:teuthology.orchestra.run.smithi052.stdout: "service_discovery_port": { 2024-03-04T09:41:14.572 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "service_discovery_port", 2024-03-04T09:41:14.572 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-03-04T09:41:14.572 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.572 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:14.573 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "8765", 2024-03-04T09:41:14.573 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.573 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.573 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:14.573 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "cephadm service discovery port", 2024-03-04T09:41:14.573 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.573 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.573 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.574 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.574 INFO:teuthology.orchestra.run.smithi052.stdout: "ssh_config_file": { 2024-03-04T09:41:14.574 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "ssh_config_file", 2024-03-04T09:41:14.574 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:14.574 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.574 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:14.574 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-03-04T09:41:14.574 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.574 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.575 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:14.575 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-03-04T09:41:14.575 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.575 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.575 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.575 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.575 INFO:teuthology.orchestra.run.smithi052.stdout: "use_agent": { 2024-03-04T09:41:14.575 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "use_agent", 2024-03-04T09:41:14.576 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:14.576 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.576 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:14.576 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-03-04T09:41:14.576 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.576 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.576 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:14.576 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-03-04T09:41:14.577 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.577 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.577 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.577 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.577 INFO:teuthology.orchestra.run.smithi052.stdout: "use_repo_digest": { 2024-03-04T09:41:14.577 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "use_repo_digest", 2024-03-04T09:41:14.577 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:14.577 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.577 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:14.578 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-03-04T09:41:14.578 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.578 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.578 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:14.578 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-03-04T09:41:14.578 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.578 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.578 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.579 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.579 INFO:teuthology.orchestra.run.smithi052.stdout: "warn_on_failed_host_check": { 2024-03-04T09:41:14.579 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "warn_on_failed_host_check", 2024-03-04T09:41:14.579 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:14.579 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.579 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:14.579 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-03-04T09:41:14.579 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.579 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.580 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:14.580 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "raise a health warning if the host check fails", 2024-03-04T09:41:14.580 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.580 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.580 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.580 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.580 INFO:teuthology.orchestra.run.smithi052.stdout: "warn_on_stray_daemons": { 2024-03-04T09:41:14.580 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "warn_on_stray_daemons", 2024-03-04T09:41:14.581 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:14.581 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.581 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:14.581 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-03-04T09:41:14.581 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.581 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.581 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:14.581 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-03-04T09:41:14.581 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.582 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.582 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.582 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.582 INFO:teuthology.orchestra.run.smithi052.stdout: "warn_on_stray_hosts": { 2024-03-04T09:41:14.582 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "warn_on_stray_hosts", 2024-03-04T09:41:14.582 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:14.582 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.582 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:14.583 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-03-04T09:41:14.583 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.583 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.583 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:14.583 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-03-04T09:41:14.583 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.583 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.584 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.584 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-03-04T09:41:14.584 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-03-04T09:41:14.584 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.584 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-03-04T09:41:14.584 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "crash", 2024-03-04T09:41:14.584 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-03-04T09:41:14.584 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-03-04T09:41:14.584 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-03-04T09:41:14.585 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-03-04T09:41:14.585 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-03-04T09:41:14.585 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:14.585 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.585 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:14.585 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-03-04T09:41:14.585 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.585 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.586 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-03-04T09:41:14.586 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-03-04T09:41:14.586 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-03-04T09:41:14.586 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-03-04T09:41:14.586 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-03-04T09:41:14.586 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-03-04T09:41:14.586 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-03-04T09:41:14.586 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-03-04T09:41:14.586 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:14.587 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.587 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.587 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.587 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.587 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-03-04T09:41:14.587 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-03-04T09:41:14.587 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:14.587 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.588 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:14.588 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-03-04T09:41:14.588 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.588 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.588 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:14.588 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:14.588 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.588 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.589 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.589 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.589 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-03-04T09:41:14.589 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-03-04T09:41:14.589 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:14.589 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.589 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:14.589 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-03-04T09:41:14.590 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.590 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.590 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-03-04T09:41:14.590 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-03-04T09:41:14.590 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-03-04T09:41:14.590 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-03-04T09:41:14.590 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-03-04T09:41:14.590 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-03-04T09:41:14.591 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-03-04T09:41:14.591 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-03-04T09:41:14.591 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:14.591 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.591 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.591 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.591 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.591 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-03-04T09:41:14.591 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-03-04T09:41:14.592 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:14.592 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.592 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:14.592 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-03-04T09:41:14.592 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.592 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.592 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:14.593 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:14.593 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.593 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.593 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.593 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.593 INFO:teuthology.orchestra.run.smithi052.stdout: "retain_interval": { 2024-03-04T09:41:14.593 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "retain_interval", 2024-03-04T09:41:14.593 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "secs", 2024-03-04T09:41:14.594 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.594 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:14.594 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "31536000", 2024-03-04T09:41:14.594 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.594 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.594 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:14.594 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "how long to retain crashes before pruning them", 2024-03-04T09:41:14.594 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.594 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.595 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.595 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.595 INFO:teuthology.orchestra.run.smithi052.stdout: "warn_recent_interval": { 2024-03-04T09:41:14.595 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "warn_recent_interval", 2024-03-04T09:41:14.595 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "secs", 2024-03-04T09:41:14.595 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.595 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:14.595 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "1209600", 2024-03-04T09:41:14.596 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.596 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.596 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:14.596 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "time interval in which to warn about recent crashes", 2024-03-04T09:41:14.596 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.596 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.596 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.596 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-03-04T09:41:14.597 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-03-04T09:41:14.597 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.597 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-03-04T09:41:14.597 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "dashboard", 2024-03-04T09:41:14.597 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-03-04T09:41:14.597 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-03-04T09:41:14.597 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-03-04T09:41:14.597 INFO:teuthology.orchestra.run.smithi052.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-03-04T09:41:14.597 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-03-04T09:41:14.598 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-03-04T09:41:14.598 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.598 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:14.598 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "10", 2024-03-04T09:41:14.598 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.598 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.598 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:14.598 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:14.599 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.599 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.599 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.599 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.599 INFO:teuthology.orchestra.run.smithi052.stdout: "ALERTMANAGER_API_HOST": { 2024-03-04T09:41:14.599 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "ALERTMANAGER_API_HOST", 2024-03-04T09:41:14.599 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:14.599 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.600 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:14.600 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-03-04T09:41:14.600 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.600 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.600 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:14.600 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:14.600 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.600 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.601 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.601 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.601 INFO:teuthology.orchestra.run.smithi052.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-03-04T09:41:14.601 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-03-04T09:41:14.601 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:14.601 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.601 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:14.601 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-03-04T09:41:14.602 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.602 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.602 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:14.602 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:14.602 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.602 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.602 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.603 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.603 INFO:teuthology.orchestra.run.smithi052.stdout: "AUDIT_API_ENABLED": { 2024-03-04T09:41:14.603 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "AUDIT_API_ENABLED", 2024-03-04T09:41:14.603 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:14.603 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.603 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:14.603 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-03-04T09:41:14.603 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.603 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.604 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:14.604 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:14.604 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.604 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.604 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.604 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.604 INFO:teuthology.orchestra.run.smithi052.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-03-04T09:41:14.604 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-03-04T09:41:14.605 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:14.605 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.605 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:14.605 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-03-04T09:41:14.605 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.605 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.605 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:14.605 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:14.606 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.606 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.606 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.606 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.606 INFO:teuthology.orchestra.run.smithi052.stdout: "ENABLE_BROWSABLE_API": { 2024-03-04T09:41:14.606 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "ENABLE_BROWSABLE_API", 2024-03-04T09:41:14.606 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:14.606 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.606 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:14.607 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-03-04T09:41:14.607 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.607 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.607 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:14.607 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:14.607 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.607 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.607 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.608 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.608 INFO:teuthology.orchestra.run.smithi052.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-03-04T09:41:14.608 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-03-04T09:41:14.608 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:14.608 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.608 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:14.608 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-03-04T09:41:14.608 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.609 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.609 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:14.609 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:14.609 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.609 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.609 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.609 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.609 INFO:teuthology.orchestra.run.smithi052.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-03-04T09:41:14.609 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-03-04T09:41:14.610 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:14.610 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.610 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:14.610 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-03-04T09:41:14.610 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.610 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.610 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:14.610 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:14.610 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.611 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.611 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.611 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.611 INFO:teuthology.orchestra.run.smithi052.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-03-04T09:41:14.611 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-03-04T09:41:14.611 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:14.611 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.611 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:14.612 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-03-04T09:41:14.612 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.612 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.612 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:14.612 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:14.612 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.612 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.612 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.613 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.613 INFO:teuthology.orchestra.run.smithi052.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-03-04T09:41:14.613 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-03-04T09:41:14.613 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:14.613 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.613 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:14.613 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-03-04T09:41:14.613 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.613 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.614 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:14.614 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:14.614 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.614 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.614 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.614 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.614 INFO:teuthology.orchestra.run.smithi052.stdout: "FEATURE_TOGGLE_NFS": { 2024-03-04T09:41:14.614 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-03-04T09:41:14.615 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:14.615 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.615 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:14.615 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-03-04T09:41:14.615 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.615 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.615 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:14.615 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:14.616 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.616 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.616 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.616 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.616 INFO:teuthology.orchestra.run.smithi052.stdout: "FEATURE_TOGGLE_RBD": { 2024-03-04T09:41:14.616 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-03-04T09:41:14.616 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:14.616 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.617 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:14.617 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-03-04T09:41:14.617 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.617 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.617 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:14.617 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:14.617 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.617 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.617 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.618 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.618 INFO:teuthology.orchestra.run.smithi052.stdout: "FEATURE_TOGGLE_RGW": { 2024-03-04T09:41:14.618 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-03-04T09:41:14.618 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:14.618 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.618 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:14.618 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-03-04T09:41:14.618 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.619 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.619 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:14.619 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:14.619 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.619 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.619 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.619 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.619 INFO:teuthology.orchestra.run.smithi052.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-03-04T09:41:14.620 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-03-04T09:41:14.620 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:14.620 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.620 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:14.620 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-03-04T09:41:14.620 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.620 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.620 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:14.620 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:14.621 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.621 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.621 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.621 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.621 INFO:teuthology.orchestra.run.smithi052.stdout: "GRAFANA_API_PASSWORD": { 2024-03-04T09:41:14.621 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "GRAFANA_API_PASSWORD", 2024-03-04T09:41:14.621 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:14.621 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.622 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:14.622 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "admin", 2024-03-04T09:41:14.622 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.622 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.622 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:14.622 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:14.622 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.622 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.622 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.623 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.623 INFO:teuthology.orchestra.run.smithi052.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-03-04T09:41:14.623 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-03-04T09:41:14.623 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:14.623 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.623 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:14.623 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-03-04T09:41:14.623 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.623 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.624 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:14.624 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:14.624 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.624 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.624 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.624 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.624 INFO:teuthology.orchestra.run.smithi052.stdout: "GRAFANA_API_URL": { 2024-03-04T09:41:14.624 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "GRAFANA_API_URL", 2024-03-04T09:41:14.625 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:14.625 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.625 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:14.625 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-03-04T09:41:14.625 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.625 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.625 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:14.625 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:14.625 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.626 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.626 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.626 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.626 INFO:teuthology.orchestra.run.smithi052.stdout: "GRAFANA_API_USERNAME": { 2024-03-04T09:41:14.626 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "GRAFANA_API_USERNAME", 2024-03-04T09:41:14.626 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:14.626 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.626 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:14.627 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "admin", 2024-03-04T09:41:14.627 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.627 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.627 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:14.627 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:14.627 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.627 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.627 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.627 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.628 INFO:teuthology.orchestra.run.smithi052.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-03-04T09:41:14.628 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-03-04T09:41:14.628 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:14.628 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.628 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:14.628 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-03-04T09:41:14.628 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.629 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.629 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:14.629 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:14.629 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.629 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.629 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.629 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.629 INFO:teuthology.orchestra.run.smithi052.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-03-04T09:41:14.630 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-03-04T09:41:14.630 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:14.630 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.630 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:14.630 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-03-04T09:41:14.630 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.630 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.630 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:14.631 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:14.631 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.631 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.631 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.631 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.631 INFO:teuthology.orchestra.run.smithi052.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-03-04T09:41:14.631 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-03-04T09:41:14.631 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:14.631 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.631 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:14.632 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-03-04T09:41:14.632 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.632 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.632 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:14.632 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:14.632 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.632 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.633 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.633 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.633 INFO:teuthology.orchestra.run.smithi052.stdout: "ISSUE_TRACKER_API_KEY": { 2024-03-04T09:41:14.633 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-03-04T09:41:14.633 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:14.633 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.633 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:14.633 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-03-04T09:41:14.634 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.634 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.634 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:14.634 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:14.634 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.634 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.634 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.634 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.635 INFO:teuthology.orchestra.run.smithi052.stdout: "PROMETHEUS_API_HOST": { 2024-03-04T09:41:14.635 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "PROMETHEUS_API_HOST", 2024-03-04T09:41:14.635 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:14.635 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.635 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:14.635 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-03-04T09:41:14.635 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.635 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.636 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:14.636 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:14.636 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.636 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.636 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.636 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.636 INFO:teuthology.orchestra.run.smithi052.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-03-04T09:41:14.637 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-03-04T09:41:14.637 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:14.637 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.637 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:14.637 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-03-04T09:41:14.637 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.637 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.637 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:14.637 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:14.638 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.638 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.638 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.638 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.638 INFO:teuthology.orchestra.run.smithi052.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-03-04T09:41:14.638 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-03-04T09:41:14.638 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:14.638 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.639 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:14.639 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-03-04T09:41:14.639 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.639 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.639 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:14.639 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:14.639 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.639 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.640 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.640 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.640 INFO:teuthology.orchestra.run.smithi052.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-03-04T09:41:14.640 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-03-04T09:41:14.640 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:14.640 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.640 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:14.641 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-03-04T09:41:14.641 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.641 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.641 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:14.641 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:14.641 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.641 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.641 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.642 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.642 INFO:teuthology.orchestra.run.smithi052.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-03-04T09:41:14.642 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-03-04T09:41:14.642 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:14.642 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.642 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:14.642 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-03-04T09:41:14.642 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.642 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.643 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:14.643 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:14.643 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.643 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.643 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.643 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.643 INFO:teuthology.orchestra.run.smithi052.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-03-04T09:41:14.643 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-03-04T09:41:14.644 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:14.644 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.644 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:14.644 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-03-04T09:41:14.644 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.644 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.644 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:14.645 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:14.645 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.645 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.645 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.645 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.645 INFO:teuthology.orchestra.run.smithi052.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-03-04T09:41:14.645 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-03-04T09:41:14.645 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:14.646 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.646 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:14.646 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-03-04T09:41:14.646 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.646 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.646 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:14.646 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:14.646 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.646 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.647 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.647 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.647 INFO:teuthology.orchestra.run.smithi052.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-03-04T09:41:14.647 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-03-04T09:41:14.647 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:14.647 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.647 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:14.647 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-03-04T09:41:14.648 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.648 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.648 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:14.648 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:14.648 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.648 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.648 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.648 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.649 INFO:teuthology.orchestra.run.smithi052.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-03-04T09:41:14.649 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-03-04T09:41:14.649 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:14.649 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.649 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:14.649 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-03-04T09:41:14.649 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.649 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.650 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:14.650 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:14.650 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.650 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.650 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.650 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.650 INFO:teuthology.orchestra.run.smithi052.stdout: "PWD_POLICY_ENABLED": { 2024-03-04T09:41:14.650 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "PWD_POLICY_ENABLED", 2024-03-04T09:41:14.651 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:14.651 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.651 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:14.651 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-03-04T09:41:14.651 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.651 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.651 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:14.651 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:14.652 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.652 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.652 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.652 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.652 INFO:teuthology.orchestra.run.smithi052.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-03-04T09:41:14.652 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-03-04T09:41:14.652 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:14.652 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.652 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:14.653 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-03-04T09:41:14.653 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.653 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.653 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:14.653 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:14.653 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.653 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.653 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.654 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.654 INFO:teuthology.orchestra.run.smithi052.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-03-04T09:41:14.654 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-03-04T09:41:14.654 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-03-04T09:41:14.654 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.654 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:14.654 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "10", 2024-03-04T09:41:14.654 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.655 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.655 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:14.655 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:14.655 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.655 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.655 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.655 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.655 INFO:teuthology.orchestra.run.smithi052.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-03-04T09:41:14.656 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-03-04T09:41:14.656 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-03-04T09:41:14.656 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.656 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:14.656 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "8", 2024-03-04T09:41:14.656 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.656 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.657 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:14.657 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:14.657 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.657 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.657 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.657 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.657 INFO:teuthology.orchestra.run.smithi052.stdout: "REST_REQUESTS_TIMEOUT": { 2024-03-04T09:41:14.657 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-03-04T09:41:14.658 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-03-04T09:41:14.658 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.658 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:14.658 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "45", 2024-03-04T09:41:14.658 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.658 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.658 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:14.658 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:14.659 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.659 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.659 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.659 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.659 INFO:teuthology.orchestra.run.smithi052.stdout: "RGW_API_ACCESS_KEY": { 2024-03-04T09:41:14.659 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "RGW_API_ACCESS_KEY", 2024-03-04T09:41:14.659 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:14.660 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.660 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:14.660 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-03-04T09:41:14.660 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.660 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.660 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:14.660 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:14.660 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.661 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.661 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.661 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.661 INFO:teuthology.orchestra.run.smithi052.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-03-04T09:41:14.661 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-03-04T09:41:14.661 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:14.661 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.662 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:14.662 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "admin", 2024-03-04T09:41:14.662 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.662 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.662 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:14.662 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:14.662 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.662 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.663 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.663 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.663 INFO:teuthology.orchestra.run.smithi052.stdout: "RGW_API_SECRET_KEY": { 2024-03-04T09:41:14.663 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "RGW_API_SECRET_KEY", 2024-03-04T09:41:14.663 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:14.663 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.663 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:14.663 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-03-04T09:41:14.664 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.664 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.664 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:14.664 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:14.664 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.664 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.664 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.664 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.665 INFO:teuthology.orchestra.run.smithi052.stdout: "RGW_API_SSL_VERIFY": { 2024-03-04T09:41:14.665 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "RGW_API_SSL_VERIFY", 2024-03-04T09:41:14.665 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:14.665 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.665 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:14.665 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-03-04T09:41:14.665 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.665 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.666 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:14.666 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:14.666 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.666 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.666 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.666 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.666 INFO:teuthology.orchestra.run.smithi052.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-03-04T09:41:14.666 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-03-04T09:41:14.667 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-03-04T09:41:14.667 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.667 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:14.667 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "0", 2024-03-04T09:41:14.667 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.667 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.667 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:14.668 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:14.668 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.668 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.668 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.668 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.668 INFO:teuthology.orchestra.run.smithi052.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-03-04T09:41:14.669 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-03-04T09:41:14.669 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-03-04T09:41:14.669 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.669 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:14.669 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "10", 2024-03-04T09:41:14.669 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.669 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.670 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:14.670 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:14.670 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.670 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.670 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.670 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.670 INFO:teuthology.orchestra.run.smithi052.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-03-04T09:41:14.671 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-03-04T09:41:14.671 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-03-04T09:41:14.671 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.671 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:14.671 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "5", 2024-03-04T09:41:14.671 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.671 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.672 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:14.672 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:14.672 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.672 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.672 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.672 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.673 INFO:teuthology.orchestra.run.smithi052.stdout: "cross_origin_url": { 2024-03-04T09:41:14.673 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "cross_origin_url", 2024-03-04T09:41:14.673 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:14.673 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.673 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:14.673 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-03-04T09:41:14.673 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.674 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.674 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:14.674 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:14.674 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.674 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.674 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.675 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.675 INFO:teuthology.orchestra.run.smithi052.stdout: "crt_file": { 2024-03-04T09:41:14.675 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "crt_file", 2024-03-04T09:41:14.675 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:14.675 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.675 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:14.675 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-03-04T09:41:14.676 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.676 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.676 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:14.676 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:14.676 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.676 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.677 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.677 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.677 INFO:teuthology.orchestra.run.smithi052.stdout: "debug": { 2024-03-04T09:41:14.677 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "debug", 2024-03-04T09:41:14.677 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:14.677 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.677 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:14.677 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-03-04T09:41:14.678 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.678 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.678 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:14.678 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Enable/disable debug options", 2024-03-04T09:41:14.678 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.678 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.678 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.678 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.679 INFO:teuthology.orchestra.run.smithi052.stdout: "jwt_token_ttl": { 2024-03-04T09:41:14.679 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "jwt_token_ttl", 2024-03-04T09:41:14.679 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-03-04T09:41:14.679 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.679 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:14.679 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "28800", 2024-03-04T09:41:14.679 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.679 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.680 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:14.680 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:14.680 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.680 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.680 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.680 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.680 INFO:teuthology.orchestra.run.smithi052.stdout: "key_file": { 2024-03-04T09:41:14.680 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "key_file", 2024-03-04T09:41:14.680 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:14.681 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.681 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:14.681 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-03-04T09:41:14.681 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.681 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.681 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:14.681 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:14.681 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.682 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.682 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.682 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.682 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-03-04T09:41:14.682 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-03-04T09:41:14.682 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:14.682 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.682 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:14.682 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-03-04T09:41:14.683 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.683 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.683 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-03-04T09:41:14.683 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-03-04T09:41:14.683 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-03-04T09:41:14.683 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-03-04T09:41:14.683 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-03-04T09:41:14.683 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-03-04T09:41:14.683 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-03-04T09:41:14.684 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-03-04T09:41:14.684 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:14.684 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.684 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.684 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.684 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.684 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-03-04T09:41:14.684 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-03-04T09:41:14.685 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:14.685 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.685 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:14.685 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-03-04T09:41:14.685 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.685 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.685 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:14.685 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:14.686 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.686 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.686 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.686 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.686 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-03-04T09:41:14.686 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-03-04T09:41:14.686 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:14.686 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.686 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:14.687 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-03-04T09:41:14.687 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.687 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.687 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-03-04T09:41:14.687 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-03-04T09:41:14.687 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-03-04T09:41:14.687 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-03-04T09:41:14.687 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-03-04T09:41:14.687 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-03-04T09:41:14.688 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-03-04T09:41:14.688 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-03-04T09:41:14.688 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:14.688 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.688 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.688 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.688 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.688 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-03-04T09:41:14.689 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-03-04T09:41:14.689 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:14.689 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.689 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:14.689 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-03-04T09:41:14.689 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.689 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.689 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:14.690 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:14.690 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.690 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.690 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.690 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.690 INFO:teuthology.orchestra.run.smithi052.stdout: "motd": { 2024-03-04T09:41:14.690 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "motd", 2024-03-04T09:41:14.690 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:14.691 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.691 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:14.691 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-03-04T09:41:14.691 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.691 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.691 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:14.691 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "The message of the day", 2024-03-04T09:41:14.691 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.692 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.692 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.692 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.692 INFO:teuthology.orchestra.run.smithi052.stdout: "redirect_resolve_ip_addr": { 2024-03-04T09:41:14.692 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "redirect_resolve_ip_addr", 2024-03-04T09:41:14.692 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:14.692 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.692 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:14.693 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-03-04T09:41:14.693 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.693 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.693 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:14.693 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:14.693 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.693 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.693 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.695 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.695 INFO:teuthology.orchestra.run.smithi052.stdout: "server_addr": { 2024-03-04T09:41:14.695 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "server_addr", 2024-03-04T09:41:14.695 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:14.696 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.696 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:14.696 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "::", 2024-03-04T09:41:14.696 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.696 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.696 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:14.696 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:14.696 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.697 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.697 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.697 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.697 INFO:teuthology.orchestra.run.smithi052.stdout: "server_port": { 2024-03-04T09:41:14.697 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "server_port", 2024-03-04T09:41:14.697 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-03-04T09:41:14.697 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.698 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:14.698 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "8080", 2024-03-04T09:41:14.698 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.698 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.698 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:14.698 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:14.698 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.698 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.699 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.699 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.699 INFO:teuthology.orchestra.run.smithi052.stdout: "ssl": { 2024-03-04T09:41:14.699 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "ssl", 2024-03-04T09:41:14.699 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:14.699 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.699 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:14.699 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-03-04T09:41:14.700 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.700 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.700 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:14.700 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:14.700 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.700 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.700 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.700 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.701 INFO:teuthology.orchestra.run.smithi052.stdout: "ssl_server_port": { 2024-03-04T09:41:14.701 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "ssl_server_port", 2024-03-04T09:41:14.701 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-03-04T09:41:14.701 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.701 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:14.701 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "8443", 2024-03-04T09:41:14.701 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.701 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.702 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:14.702 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:14.702 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.702 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.702 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.702 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.702 INFO:teuthology.orchestra.run.smithi052.stdout: "standby_behaviour": { 2024-03-04T09:41:14.702 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "standby_behaviour", 2024-03-04T09:41:14.702 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:14.703 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.703 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:14.703 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "redirect", 2024-03-04T09:41:14.703 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.703 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.703 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-03-04T09:41:14.703 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-03-04T09:41:14.703 INFO:teuthology.orchestra.run.smithi052.stdout: "redirect" 2024-03-04T09:41:14.704 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-03-04T09:41:14.704 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:14.704 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.704 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.704 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.704 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.704 INFO:teuthology.orchestra.run.smithi052.stdout: "standby_error_status_code": { 2024-03-04T09:41:14.704 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "standby_error_status_code", 2024-03-04T09:41:14.705 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-03-04T09:41:14.705 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.705 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:14.705 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "500", 2024-03-04T09:41:14.705 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "400", 2024-03-04T09:41:14.705 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "599", 2024-03-04T09:41:14.705 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:14.705 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:14.706 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.706 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.706 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.706 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.706 INFO:teuthology.orchestra.run.smithi052.stdout: "url_prefix": { 2024-03-04T09:41:14.706 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "url_prefix", 2024-03-04T09:41:14.706 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:14.706 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.706 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:14.707 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-03-04T09:41:14.707 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.707 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.707 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:14.707 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:14.707 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.707 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.707 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.708 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-03-04T09:41:14.708 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-03-04T09:41:14.708 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.708 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-03-04T09:41:14.708 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "devicehealth", 2024-03-04T09:41:14.708 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-03-04T09:41:14.708 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-03-04T09:41:14.709 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-03-04T09:41:14.709 INFO:teuthology.orchestra.run.smithi052.stdout: "enable_monitoring": { 2024-03-04T09:41:14.709 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "enable_monitoring", 2024-03-04T09:41:14.709 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:14.709 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.709 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:14.710 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-03-04T09:41:14.710 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.710 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.710 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:14.710 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "monitor device health metrics", 2024-03-04T09:41:14.710 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.710 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.711 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.711 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.711 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-03-04T09:41:14.711 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-03-04T09:41:14.711 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:14.711 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.712 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:14.712 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-03-04T09:41:14.712 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.712 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.712 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-03-04T09:41:14.712 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-03-04T09:41:14.712 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-03-04T09:41:14.713 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-03-04T09:41:14.713 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-03-04T09:41:14.713 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-03-04T09:41:14.713 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-03-04T09:41:14.713 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-03-04T09:41:14.713 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:14.714 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.714 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.714 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.714 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.714 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-03-04T09:41:14.714 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-03-04T09:41:14.714 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:14.714 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.715 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:14.715 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-03-04T09:41:14.715 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.715 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.715 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:14.715 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:14.715 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.716 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.716 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.716 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.716 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-03-04T09:41:14.716 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-03-04T09:41:14.716 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:14.716 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.717 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:14.717 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-03-04T09:41:14.717 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.717 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.717 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-03-04T09:41:14.717 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-03-04T09:41:14.717 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-03-04T09:41:14.717 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-03-04T09:41:14.718 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-03-04T09:41:14.718 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-03-04T09:41:14.718 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-03-04T09:41:14.718 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-03-04T09:41:14.718 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:14.718 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.718 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.718 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.718 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.719 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-03-04T09:41:14.719 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-03-04T09:41:14.719 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:14.719 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.719 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:14.719 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-03-04T09:41:14.719 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.720 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.720 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:14.720 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:14.720 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.720 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.720 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.720 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.720 INFO:teuthology.orchestra.run.smithi052.stdout: "mark_out_threshold": { 2024-03-04T09:41:14.720 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "mark_out_threshold", 2024-03-04T09:41:14.721 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "secs", 2024-03-04T09:41:14.721 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.721 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:14.721 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "2419200", 2024-03-04T09:41:14.721 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.721 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.721 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:14.721 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-03-04T09:41:14.721 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.722 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.722 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.722 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.722 INFO:teuthology.orchestra.run.smithi052.stdout: "pool_name": { 2024-03-04T09:41:14.722 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "pool_name", 2024-03-04T09:41:14.722 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:14.722 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.722 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:14.723 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "device_health_metrics", 2024-03-04T09:41:14.723 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.723 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.723 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:14.723 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "name of pool in which to store device health metrics", 2024-03-04T09:41:14.723 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.723 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.723 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.724 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.724 INFO:teuthology.orchestra.run.smithi052.stdout: "retention_period": { 2024-03-04T09:41:14.724 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "retention_period", 2024-03-04T09:41:14.724 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "secs", 2024-03-04T09:41:14.724 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.724 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:14.725 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "15552000", 2024-03-04T09:41:14.725 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.725 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.725 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:14.725 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "how long to retain device health metrics", 2024-03-04T09:41:14.725 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.725 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.726 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.726 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.726 INFO:teuthology.orchestra.run.smithi052.stdout: "scrape_frequency": { 2024-03-04T09:41:14.726 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "scrape_frequency", 2024-03-04T09:41:14.726 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "secs", 2024-03-04T09:41:14.726 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.726 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:14.727 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "86400", 2024-03-04T09:41:14.727 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.727 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.727 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:14.727 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "how frequently to scrape device health metrics", 2024-03-04T09:41:14.727 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.727 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.727 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.728 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.728 INFO:teuthology.orchestra.run.smithi052.stdout: "self_heal": { 2024-03-04T09:41:14.728 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "self_heal", 2024-03-04T09:41:14.728 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:14.728 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.728 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:14.728 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-03-04T09:41:14.729 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.729 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.729 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:14.729 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-03-04T09:41:14.729 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.729 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.729 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.729 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.730 INFO:teuthology.orchestra.run.smithi052.stdout: "sleep_interval": { 2024-03-04T09:41:14.730 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "sleep_interval", 2024-03-04T09:41:14.730 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "secs", 2024-03-04T09:41:14.730 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.730 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:14.730 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "600", 2024-03-04T09:41:14.730 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.730 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.731 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:14.731 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "how frequently to wake up and check device health", 2024-03-04T09:41:14.731 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.731 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.731 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.731 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.731 INFO:teuthology.orchestra.run.smithi052.stdout: "warn_threshold": { 2024-03-04T09:41:14.731 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "warn_threshold", 2024-03-04T09:41:14.731 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "secs", 2024-03-04T09:41:14.732 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.732 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:14.732 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "7257600", 2024-03-04T09:41:14.732 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.732 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.732 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:14.732 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-03-04T09:41:14.732 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.733 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.733 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.733 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-03-04T09:41:14.733 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-03-04T09:41:14.733 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.733 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-03-04T09:41:14.733 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "diskprediction_local", 2024-03-04T09:41:14.733 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-03-04T09:41:14.733 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-03-04T09:41:14.734 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-03-04T09:41:14.734 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-03-04T09:41:14.734 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-03-04T09:41:14.734 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:14.734 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.734 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:14.734 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-03-04T09:41:14.734 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.735 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.735 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-03-04T09:41:14.735 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-03-04T09:41:14.735 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-03-04T09:41:14.735 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-03-04T09:41:14.735 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-03-04T09:41:14.735 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-03-04T09:41:14.736 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-03-04T09:41:14.736 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-03-04T09:41:14.736 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:14.736 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.736 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.736 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.736 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.736 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-03-04T09:41:14.736 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-03-04T09:41:14.737 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:14.737 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.737 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:14.737 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-03-04T09:41:14.737 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.737 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.737 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:14.737 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:14.738 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.738 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.738 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.738 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.738 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-03-04T09:41:14.738 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-03-04T09:41:14.738 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:14.738 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.739 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:14.739 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-03-04T09:41:14.739 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.739 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.739 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-03-04T09:41:14.739 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-03-04T09:41:14.739 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-03-04T09:41:14.739 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-03-04T09:41:14.740 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-03-04T09:41:14.740 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-03-04T09:41:14.740 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-03-04T09:41:14.740 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-03-04T09:41:14.740 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:14.740 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.740 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.740 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.741 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.741 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-03-04T09:41:14.741 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-03-04T09:41:14.741 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:14.741 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.741 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:14.741 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-03-04T09:41:14.741 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.742 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.742 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:14.742 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:14.742 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.742 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.742 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.742 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.742 INFO:teuthology.orchestra.run.smithi052.stdout: "predict_interval": { 2024-03-04T09:41:14.743 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "predict_interval", 2024-03-04T09:41:14.743 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:14.743 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.743 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:14.743 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "86400", 2024-03-04T09:41:14.743 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.743 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.743 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:14.744 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:14.744 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.744 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.744 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.744 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.744 INFO:teuthology.orchestra.run.smithi052.stdout: "predictor_model": { 2024-03-04T09:41:14.744 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "predictor_model", 2024-03-04T09:41:14.744 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:14.745 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.745 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:14.745 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "prophetstor", 2024-03-04T09:41:14.745 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.745 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.745 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:14.745 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:14.745 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.746 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.746 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.746 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.746 INFO:teuthology.orchestra.run.smithi052.stdout: "sleep_interval": { 2024-03-04T09:41:14.746 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "sleep_interval", 2024-03-04T09:41:14.746 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:14.746 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.746 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:14.747 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "600", 2024-03-04T09:41:14.747 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.747 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.747 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:14.747 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:14.747 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.747 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.747 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.747 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-03-04T09:41:14.748 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-03-04T09:41:14.748 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.748 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-03-04T09:41:14.748 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "influx", 2024-03-04T09:41:14.748 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": false, 2024-03-04T09:41:14.748 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "influxdb python module not found", 2024-03-04T09:41:14.748 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-03-04T09:41:14.748 INFO:teuthology.orchestra.run.smithi052.stdout: "batch_size": { 2024-03-04T09:41:14.749 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "batch_size", 2024-03-04T09:41:14.749 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-03-04T09:41:14.749 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.749 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:14.749 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "5000", 2024-03-04T09:41:14.749 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.749 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.749 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:14.750 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-03-04T09:41:14.750 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.750 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.750 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.750 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.750 INFO:teuthology.orchestra.run.smithi052.stdout: "database": { 2024-03-04T09:41:14.750 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "database", 2024-03-04T09:41:14.750 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:14.751 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.751 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:14.751 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "ceph", 2024-03-04T09:41:14.751 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.751 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.751 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:14.751 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-03-04T09:41:14.751 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.751 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.752 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.752 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.752 INFO:teuthology.orchestra.run.smithi052.stdout: "hostname": { 2024-03-04T09:41:14.752 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "hostname", 2024-03-04T09:41:14.752 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:14.752 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.752 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:14.752 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-03-04T09:41:14.753 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.753 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.753 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:14.753 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "InfluxDB server hostname", 2024-03-04T09:41:14.753 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.753 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.753 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.753 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.754 INFO:teuthology.orchestra.run.smithi052.stdout: "interval": { 2024-03-04T09:41:14.754 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "interval", 2024-03-04T09:41:14.754 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "secs", 2024-03-04T09:41:14.754 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.754 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:14.754 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "30", 2024-03-04T09:41:14.754 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "5", 2024-03-04T09:41:14.754 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.754 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:14.755 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-03-04T09:41:14.755 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.755 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.755 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.755 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.755 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-03-04T09:41:14.755 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-03-04T09:41:14.755 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:14.756 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.756 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:14.756 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-03-04T09:41:14.756 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.756 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.756 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-03-04T09:41:14.757 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-03-04T09:41:14.757 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-03-04T09:41:14.757 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-03-04T09:41:14.757 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-03-04T09:41:14.757 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-03-04T09:41:14.757 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-03-04T09:41:14.757 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-03-04T09:41:14.758 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:14.758 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.758 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.758 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.758 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.758 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-03-04T09:41:14.758 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-03-04T09:41:14.758 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:14.759 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.759 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:14.759 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-03-04T09:41:14.759 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.759 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.759 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:14.759 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:14.759 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.760 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.760 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.760 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.760 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-03-04T09:41:14.760 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-03-04T09:41:14.760 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:14.760 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.760 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:14.760 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-03-04T09:41:14.761 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.761 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.761 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-03-04T09:41:14.761 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-03-04T09:41:14.761 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-03-04T09:41:14.761 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-03-04T09:41:14.761 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-03-04T09:41:14.761 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-03-04T09:41:14.762 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-03-04T09:41:14.762 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-03-04T09:41:14.762 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:14.762 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.762 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.762 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.762 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.762 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-03-04T09:41:14.763 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-03-04T09:41:14.763 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:14.763 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.763 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:14.763 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-03-04T09:41:14.763 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.763 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.763 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:14.763 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:14.764 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.764 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.764 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.764 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.764 INFO:teuthology.orchestra.run.smithi052.stdout: "password": { 2024-03-04T09:41:14.764 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "password", 2024-03-04T09:41:14.764 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:14.764 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.765 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:14.765 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-03-04T09:41:14.765 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.765 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.765 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:14.765 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "password of InfluxDB server user", 2024-03-04T09:41:14.765 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.765 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.765 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.766 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.766 INFO:teuthology.orchestra.run.smithi052.stdout: "port": { 2024-03-04T09:41:14.766 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "port", 2024-03-04T09:41:14.766 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-03-04T09:41:14.766 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.766 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:14.766 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "8086", 2024-03-04T09:41:14.767 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.767 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.767 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:14.767 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "InfluxDB server port", 2024-03-04T09:41:14.767 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.767 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.767 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.767 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.767 INFO:teuthology.orchestra.run.smithi052.stdout: "ssl": { 2024-03-04T09:41:14.768 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "ssl", 2024-03-04T09:41:14.768 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:14.768 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.768 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:14.768 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "false", 2024-03-04T09:41:14.768 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.768 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.768 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:14.769 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-03-04T09:41:14.769 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.769 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.769 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.769 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.769 INFO:teuthology.orchestra.run.smithi052.stdout: "threads": { 2024-03-04T09:41:14.769 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "threads", 2024-03-04T09:41:14.769 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-03-04T09:41:14.770 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.770 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:14.770 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "5", 2024-03-04T09:41:14.770 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "1", 2024-03-04T09:41:14.770 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "32", 2024-03-04T09:41:14.770 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:14.770 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-03-04T09:41:14.770 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.770 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.771 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.771 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.771 INFO:teuthology.orchestra.run.smithi052.stdout: "username": { 2024-03-04T09:41:14.771 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "username", 2024-03-04T09:41:14.771 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:14.771 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.771 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:14.771 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-03-04T09:41:14.772 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.772 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.772 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:14.772 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "username of InfluxDB server user", 2024-03-04T09:41:14.772 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.772 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.772 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.772 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.773 INFO:teuthology.orchestra.run.smithi052.stdout: "verify_ssl": { 2024-03-04T09:41:14.773 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "verify_ssl", 2024-03-04T09:41:14.773 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:14.773 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.773 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:14.773 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "true", 2024-03-04T09:41:14.773 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.773 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.773 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:14.774 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-03-04T09:41:14.774 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.774 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.774 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.774 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-03-04T09:41:14.774 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-03-04T09:41:14.775 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.775 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-03-04T09:41:14.775 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "insights", 2024-03-04T09:41:14.775 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-03-04T09:41:14.775 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-03-04T09:41:14.775 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-03-04T09:41:14.775 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-03-04T09:41:14.775 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-03-04T09:41:14.776 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:14.776 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.776 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:14.776 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-03-04T09:41:14.776 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.776 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.776 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-03-04T09:41:14.776 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-03-04T09:41:14.777 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-03-04T09:41:14.777 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-03-04T09:41:14.777 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-03-04T09:41:14.777 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-03-04T09:41:14.777 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-03-04T09:41:14.777 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-03-04T09:41:14.777 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:14.777 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.777 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.778 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.778 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.778 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-03-04T09:41:14.778 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-03-04T09:41:14.778 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:14.778 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.778 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:14.778 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-03-04T09:41:14.779 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.779 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.779 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:14.779 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:14.779 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.779 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.779 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.779 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.780 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-03-04T09:41:14.780 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-03-04T09:41:14.780 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:14.780 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.780 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:14.780 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-03-04T09:41:14.780 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.781 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.781 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-03-04T09:41:14.781 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-03-04T09:41:14.781 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-03-04T09:41:14.781 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-03-04T09:41:14.781 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-03-04T09:41:14.781 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-03-04T09:41:14.781 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-03-04T09:41:14.782 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-03-04T09:41:14.782 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:14.782 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.782 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.782 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.782 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.782 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-03-04T09:41:14.782 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-03-04T09:41:14.782 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:14.783 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.783 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:14.783 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-03-04T09:41:14.783 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.783 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.783 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:14.783 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:14.783 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.784 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.784 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.784 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-03-04T09:41:14.784 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-03-04T09:41:14.784 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.784 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-03-04T09:41:14.784 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "iostat", 2024-03-04T09:41:14.784 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-03-04T09:41:14.785 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-03-04T09:41:14.785 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-03-04T09:41:14.785 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-03-04T09:41:14.785 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-03-04T09:41:14.785 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:14.785 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.785 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:14.786 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-03-04T09:41:14.786 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.786 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.786 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-03-04T09:41:14.786 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-03-04T09:41:14.786 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-03-04T09:41:14.786 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-03-04T09:41:14.786 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-03-04T09:41:14.787 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-03-04T09:41:14.787 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-03-04T09:41:14.787 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-03-04T09:41:14.787 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:14.787 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.787 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.787 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.787 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.787 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-03-04T09:41:14.788 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-03-04T09:41:14.788 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:14.788 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.788 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:14.788 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-03-04T09:41:14.788 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.788 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.789 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:14.789 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:14.789 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.789 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.789 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.789 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.789 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-03-04T09:41:14.789 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-03-04T09:41:14.789 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:14.790 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.790 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:14.790 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-03-04T09:41:14.790 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.790 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.790 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-03-04T09:41:14.790 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-03-04T09:41:14.790 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-03-04T09:41:14.791 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-03-04T09:41:14.791 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-03-04T09:41:14.791 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-03-04T09:41:14.791 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-03-04T09:41:14.791 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-03-04T09:41:14.791 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:14.791 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.791 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.792 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.792 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.792 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-03-04T09:41:14.792 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-03-04T09:41:14.792 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:14.792 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.792 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:14.792 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-03-04T09:41:14.793 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.793 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.793 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:14.793 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:14.793 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.793 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.793 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.793 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-03-04T09:41:14.794 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-03-04T09:41:14.794 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.794 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-03-04T09:41:14.794 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "k8sevents", 2024-03-04T09:41:14.794 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-03-04T09:41:14.794 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-03-04T09:41:14.794 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-03-04T09:41:14.794 INFO:teuthology.orchestra.run.smithi052.stdout: "ceph_event_retention_days": { 2024-03-04T09:41:14.794 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "ceph_event_retention_days", 2024-03-04T09:41:14.795 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-03-04T09:41:14.795 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.795 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:14.795 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "7", 2024-03-04T09:41:14.795 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.795 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.795 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:14.795 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Days to hold ceph event information within local cache", 2024-03-04T09:41:14.796 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.796 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.796 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.796 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.796 INFO:teuthology.orchestra.run.smithi052.stdout: "config_check_secs": { 2024-03-04T09:41:14.796 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "config_check_secs", 2024-03-04T09:41:14.796 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-03-04T09:41:14.796 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.797 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:14.797 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "10", 2024-03-04T09:41:14.797 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "10", 2024-03-04T09:41:14.797 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.797 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:14.797 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-03-04T09:41:14.797 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.797 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.798 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.798 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.798 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-03-04T09:41:14.798 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-03-04T09:41:14.798 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:14.798 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.798 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:14.798 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-03-04T09:41:14.798 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.799 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.799 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-03-04T09:41:14.799 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-03-04T09:41:14.799 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-03-04T09:41:14.799 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-03-04T09:41:14.799 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-03-04T09:41:14.799 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-03-04T09:41:14.799 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-03-04T09:41:14.800 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-03-04T09:41:14.800 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:14.800 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.800 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.800 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.800 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.800 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-03-04T09:41:14.800 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-03-04T09:41:14.801 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:14.801 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.801 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:14.801 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-03-04T09:41:14.801 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.801 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.801 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:14.801 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:14.801 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.802 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.802 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.802 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.802 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-03-04T09:41:14.802 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-03-04T09:41:14.802 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:14.802 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.802 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:14.803 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-03-04T09:41:14.803 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.803 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.803 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-03-04T09:41:14.803 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-03-04T09:41:14.803 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-03-04T09:41:14.803 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-03-04T09:41:14.803 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-03-04T09:41:14.803 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-03-04T09:41:14.804 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-03-04T09:41:14.804 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-03-04T09:41:14.804 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:14.804 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.804 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.804 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.804 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.804 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-03-04T09:41:14.805 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-03-04T09:41:14.805 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:14.805 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.805 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:14.805 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-03-04T09:41:14.805 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.805 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.805 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:14.806 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:14.806 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.806 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.806 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.806 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-03-04T09:41:14.806 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-03-04T09:41:14.806 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.806 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-03-04T09:41:14.807 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "localpool", 2024-03-04T09:41:14.807 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-03-04T09:41:14.807 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-03-04T09:41:14.807 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-03-04T09:41:14.807 INFO:teuthology.orchestra.run.smithi052.stdout: "failure_domain": { 2024-03-04T09:41:14.807 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "failure_domain", 2024-03-04T09:41:14.807 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:14.807 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.807 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:14.808 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "host", 2024-03-04T09:41:14.808 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.808 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.808 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:14.808 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "failure domain for any created local pool", 2024-03-04T09:41:14.808 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-03-04T09:41:14.808 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.808 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.809 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.809 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-03-04T09:41:14.809 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-03-04T09:41:14.809 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:14.809 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.809 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:14.809 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-03-04T09:41:14.809 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.809 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.810 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-03-04T09:41:14.810 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-03-04T09:41:14.810 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-03-04T09:41:14.810 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-03-04T09:41:14.810 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-03-04T09:41:14.810 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-03-04T09:41:14.810 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-03-04T09:41:14.810 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-03-04T09:41:14.811 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:14.811 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.811 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.811 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.811 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.811 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-03-04T09:41:14.811 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-03-04T09:41:14.811 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:14.812 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.812 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:14.812 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-03-04T09:41:14.812 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.812 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.812 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:14.812 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:14.812 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.813 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.813 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.813 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.813 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-03-04T09:41:14.813 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-03-04T09:41:14.813 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:14.813 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.813 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:14.814 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-03-04T09:41:14.814 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.814 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.814 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-03-04T09:41:14.814 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-03-04T09:41:14.814 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-03-04T09:41:14.814 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-03-04T09:41:14.814 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-03-04T09:41:14.815 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-03-04T09:41:14.815 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-03-04T09:41:14.815 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-03-04T09:41:14.815 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:14.815 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.815 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.815 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.816 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.816 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-03-04T09:41:14.816 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-03-04T09:41:14.816 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:14.816 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.816 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:14.816 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-03-04T09:41:14.816 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.816 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.817 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:14.817 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:14.817 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.817 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.817 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.817 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.817 INFO:teuthology.orchestra.run.smithi052.stdout: "min_size": { 2024-03-04T09:41:14.817 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "min_size", 2024-03-04T09:41:14.818 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-03-04T09:41:14.818 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.818 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:14.818 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-03-04T09:41:14.818 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.818 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.818 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:14.818 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "default min_size for any created local pool", 2024-03-04T09:41:14.818 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-03-04T09:41:14.819 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.819 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.819 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.819 INFO:teuthology.orchestra.run.smithi052.stdout: "num_rep": { 2024-03-04T09:41:14.819 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "num_rep", 2024-03-04T09:41:14.819 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-03-04T09:41:14.819 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.819 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:14.820 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "3", 2024-03-04T09:41:14.820 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.820 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.820 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:14.820 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "default replica count for any created local pool", 2024-03-04T09:41:14.820 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.820 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.820 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.820 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.821 INFO:teuthology.orchestra.run.smithi052.stdout: "pg_num": { 2024-03-04T09:41:14.821 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "pg_num", 2024-03-04T09:41:14.821 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-03-04T09:41:14.821 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.821 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:14.821 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "128", 2024-03-04T09:41:14.821 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.821 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.822 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:14.822 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "default pg_num for any created local pool", 2024-03-04T09:41:14.822 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.822 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.822 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.822 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.822 INFO:teuthology.orchestra.run.smithi052.stdout: "prefix": { 2024-03-04T09:41:14.822 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "prefix", 2024-03-04T09:41:14.823 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:14.823 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.823 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:14.823 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-03-04T09:41:14.823 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.823 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.823 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:14.823 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "name prefix for any created local pool", 2024-03-04T09:41:14.823 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.824 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.824 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.824 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.824 INFO:teuthology.orchestra.run.smithi052.stdout: "subtree": { 2024-03-04T09:41:14.824 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "subtree", 2024-03-04T09:41:14.824 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:14.824 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.824 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:14.825 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "rack", 2024-03-04T09:41:14.825 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.825 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.825 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:14.825 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "CRUSH level for which to create a local pool", 2024-03-04T09:41:14.825 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-03-04T09:41:14.825 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.825 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.826 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-03-04T09:41:14.826 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-03-04T09:41:14.826 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.826 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-03-04T09:41:14.826 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "mds_autoscaler", 2024-03-04T09:41:14.826 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-03-04T09:41:14.826 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-03-04T09:41:14.827 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-03-04T09:41:14.827 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-03-04T09:41:14.827 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-03-04T09:41:14.827 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:14.827 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.827 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:14.827 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-03-04T09:41:14.827 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.827 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.828 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-03-04T09:41:14.828 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-03-04T09:41:14.828 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-03-04T09:41:14.828 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-03-04T09:41:14.828 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-03-04T09:41:14.828 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-03-04T09:41:14.828 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-03-04T09:41:14.828 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-03-04T09:41:14.829 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:14.829 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.829 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.829 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.829 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.829 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-03-04T09:41:14.829 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-03-04T09:41:14.829 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:14.829 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.830 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:14.830 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-03-04T09:41:14.830 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.830 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.830 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:14.830 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:14.830 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.830 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.831 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.831 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.831 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-03-04T09:41:14.831 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-03-04T09:41:14.831 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:14.831 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.831 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:14.831 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-03-04T09:41:14.831 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.832 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.832 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-03-04T09:41:14.832 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-03-04T09:41:14.832 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-03-04T09:41:14.832 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-03-04T09:41:14.832 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-03-04T09:41:14.832 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-03-04T09:41:14.832 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-03-04T09:41:14.833 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-03-04T09:41:14.833 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:14.833 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.833 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.833 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.833 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.833 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-03-04T09:41:14.833 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-03-04T09:41:14.833 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:14.834 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.834 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:14.834 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-03-04T09:41:14.834 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.834 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.834 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:14.834 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:14.834 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.835 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.835 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.835 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-03-04T09:41:14.835 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-03-04T09:41:14.835 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.835 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-03-04T09:41:14.835 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "mirroring", 2024-03-04T09:41:14.835 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-03-04T09:41:14.836 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-03-04T09:41:14.836 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-03-04T09:41:14.836 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-03-04T09:41:14.836 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-03-04T09:41:14.836 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:14.836 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.836 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:14.836 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-03-04T09:41:14.837 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.837 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.837 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-03-04T09:41:14.837 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-03-04T09:41:14.837 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-03-04T09:41:14.837 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-03-04T09:41:14.837 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-03-04T09:41:14.837 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-03-04T09:41:14.838 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-03-04T09:41:14.838 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-03-04T09:41:14.838 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:14.838 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.838 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.838 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.838 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.838 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-03-04T09:41:14.839 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-03-04T09:41:14.839 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:14.839 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.839 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:14.839 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-03-04T09:41:14.839 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.839 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.839 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:14.840 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:14.840 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.840 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.840 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.840 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.840 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-03-04T09:41:14.840 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-03-04T09:41:14.840 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:14.840 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.841 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:14.841 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-03-04T09:41:14.841 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.841 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.841 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-03-04T09:41:14.841 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-03-04T09:41:14.841 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-03-04T09:41:14.841 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-03-04T09:41:14.842 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-03-04T09:41:14.842 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-03-04T09:41:14.842 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-03-04T09:41:14.842 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-03-04T09:41:14.842 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:14.842 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.842 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.843 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.843 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.843 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-03-04T09:41:14.843 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-03-04T09:41:14.843 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:14.843 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.843 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:14.843 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-03-04T09:41:14.844 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.844 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.844 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:14.844 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:14.844 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.844 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.844 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.844 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-03-04T09:41:14.845 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-03-04T09:41:14.845 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.845 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-03-04T09:41:14.845 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "nfs", 2024-03-04T09:41:14.845 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-03-04T09:41:14.845 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-03-04T09:41:14.845 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-03-04T09:41:14.845 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-03-04T09:41:14.846 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-03-04T09:41:14.846 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:14.846 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.846 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:14.846 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-03-04T09:41:14.846 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.846 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.846 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-03-04T09:41:14.846 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-03-04T09:41:14.847 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-03-04T09:41:14.847 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-03-04T09:41:14.847 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-03-04T09:41:14.847 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-03-04T09:41:14.847 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-03-04T09:41:14.847 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-03-04T09:41:14.847 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:14.847 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.848 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.848 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.848 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.848 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-03-04T09:41:14.848 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-03-04T09:41:14.848 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:14.848 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.848 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:14.849 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-03-04T09:41:14.849 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.849 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.849 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:14.849 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:14.849 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.849 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.849 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.849 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.850 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-03-04T09:41:14.850 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-03-04T09:41:14.850 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:14.850 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.850 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:14.850 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-03-04T09:41:14.850 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.850 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.851 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-03-04T09:41:14.851 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-03-04T09:41:14.851 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-03-04T09:41:14.851 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-03-04T09:41:14.851 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-03-04T09:41:14.851 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-03-04T09:41:14.851 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-03-04T09:41:14.851 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-03-04T09:41:14.851 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:14.852 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.852 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.852 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.852 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.852 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-03-04T09:41:14.852 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-03-04T09:41:14.852 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:14.852 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.853 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:14.853 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-03-04T09:41:14.853 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.853 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.853 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:14.853 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:14.853 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.853 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.854 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.854 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-03-04T09:41:14.854 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-03-04T09:41:14.854 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.854 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-03-04T09:41:14.854 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "orchestrator", 2024-03-04T09:41:14.854 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-03-04T09:41:14.854 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-03-04T09:41:14.854 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-03-04T09:41:14.855 INFO:teuthology.orchestra.run.smithi052.stdout: "fail_fs": { 2024-03-04T09:41:14.855 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "fail_fs", 2024-03-04T09:41:14.855 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:14.855 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.855 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:14.855 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-03-04T09:41:14.855 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.855 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.856 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:14.856 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-03-04T09:41:14.856 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.856 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.856 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.856 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.856 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-03-04T09:41:14.856 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-03-04T09:41:14.857 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:14.857 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.857 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:14.857 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-03-04T09:41:14.857 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.857 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.857 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-03-04T09:41:14.857 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-03-04T09:41:14.858 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-03-04T09:41:14.858 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-03-04T09:41:14.858 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-03-04T09:41:14.858 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-03-04T09:41:14.858 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-03-04T09:41:14.858 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-03-04T09:41:14.858 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:14.858 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.859 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.859 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.859 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.859 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-03-04T09:41:14.859 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-03-04T09:41:14.859 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:14.859 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.859 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:14.860 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-03-04T09:41:14.860 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.860 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.860 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:14.860 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:14.860 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.860 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.860 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.861 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.861 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-03-04T09:41:14.861 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-03-04T09:41:14.861 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:14.861 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.861 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:14.861 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-03-04T09:41:14.862 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.862 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.862 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-03-04T09:41:14.862 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-03-04T09:41:14.862 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-03-04T09:41:14.862 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-03-04T09:41:14.862 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-03-04T09:41:14.862 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-03-04T09:41:14.863 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-03-04T09:41:14.863 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-03-04T09:41:14.863 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:14.863 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.863 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.863 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.863 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.863 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-03-04T09:41:14.864 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-03-04T09:41:14.864 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:14.864 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.864 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:14.864 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-03-04T09:41:14.864 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.864 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.864 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:14.864 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:14.865 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.865 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.865 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.865 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.865 INFO:teuthology.orchestra.run.smithi052.stdout: "orchestrator": { 2024-03-04T09:41:14.865 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "orchestrator", 2024-03-04T09:41:14.865 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:14.865 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.866 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:14.866 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-03-04T09:41:14.866 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.866 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.866 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-03-04T09:41:14.866 INFO:teuthology.orchestra.run.smithi052.stdout: "cephadm", 2024-03-04T09:41:14.866 INFO:teuthology.orchestra.run.smithi052.stdout: "rook", 2024-03-04T09:41:14.866 INFO:teuthology.orchestra.run.smithi052.stdout: "test_orchestrator" 2024-03-04T09:41:14.866 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-03-04T09:41:14.867 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Orchestrator backend", 2024-03-04T09:41:14.867 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.867 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.867 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.867 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-03-04T09:41:14.867 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-03-04T09:41:14.867 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.867 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-03-04T09:41:14.868 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "osd_perf_query", 2024-03-04T09:41:14.868 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-03-04T09:41:14.868 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-03-04T09:41:14.868 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-03-04T09:41:14.868 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-03-04T09:41:14.868 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-03-04T09:41:14.868 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:14.868 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.869 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:14.869 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-03-04T09:41:14.869 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.869 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.869 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-03-04T09:41:14.869 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-03-04T09:41:14.869 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-03-04T09:41:14.869 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-03-04T09:41:14.870 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-03-04T09:41:14.870 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-03-04T09:41:14.870 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-03-04T09:41:14.870 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-03-04T09:41:14.870 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:14.870 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.870 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.870 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.871 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.871 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-03-04T09:41:14.871 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-03-04T09:41:14.871 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:14.871 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.871 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:14.871 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-03-04T09:41:14.871 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.871 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.872 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:14.872 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:14.872 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.872 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.872 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.872 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.872 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-03-04T09:41:14.872 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-03-04T09:41:14.873 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:14.873 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.873 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:14.873 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-03-04T09:41:14.873 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.873 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.873 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-03-04T09:41:14.873 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-03-04T09:41:14.874 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-03-04T09:41:14.874 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-03-04T09:41:14.874 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-03-04T09:41:14.874 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-03-04T09:41:14.874 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-03-04T09:41:14.874 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-03-04T09:41:14.874 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:14.874 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.874 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.875 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.875 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.875 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-03-04T09:41:14.875 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-03-04T09:41:14.875 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:14.875 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.875 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:14.875 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-03-04T09:41:14.876 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.876 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.876 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:14.876 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:14.876 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.876 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.876 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.876 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-03-04T09:41:14.877 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-03-04T09:41:14.877 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.877 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-03-04T09:41:14.877 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "osd_support", 2024-03-04T09:41:14.877 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-03-04T09:41:14.877 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-03-04T09:41:14.877 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-03-04T09:41:14.877 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-03-04T09:41:14.878 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-03-04T09:41:14.878 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:14.878 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.878 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:14.878 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-03-04T09:41:14.878 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.878 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.878 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-03-04T09:41:14.878 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-03-04T09:41:14.879 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-03-04T09:41:14.879 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-03-04T09:41:14.879 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-03-04T09:41:14.879 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-03-04T09:41:14.879 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-03-04T09:41:14.879 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-03-04T09:41:14.879 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:14.879 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.880 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.880 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.880 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.880 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-03-04T09:41:14.880 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-03-04T09:41:14.880 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:14.880 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.880 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:14.881 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-03-04T09:41:14.881 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.881 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.881 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:14.881 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:14.881 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.881 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.881 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.882 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.882 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-03-04T09:41:14.882 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-03-04T09:41:14.882 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:14.882 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.882 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:14.882 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-03-04T09:41:14.882 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.883 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.883 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-03-04T09:41:14.883 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-03-04T09:41:14.883 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-03-04T09:41:14.883 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-03-04T09:41:14.883 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-03-04T09:41:14.883 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-03-04T09:41:14.883 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-03-04T09:41:14.884 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-03-04T09:41:14.884 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:14.884 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.884 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.884 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.884 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.884 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-03-04T09:41:14.884 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-03-04T09:41:14.885 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:14.885 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.885 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:14.885 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-03-04T09:41:14.885 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.885 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.885 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:14.885 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:14.885 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.886 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.886 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.886 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-03-04T09:41:14.886 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-03-04T09:41:14.886 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.886 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-03-04T09:41:14.886 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "pg_autoscaler", 2024-03-04T09:41:14.886 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-03-04T09:41:14.886 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-03-04T09:41:14.887 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-03-04T09:41:14.887 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-03-04T09:41:14.887 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-03-04T09:41:14.887 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:14.887 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.887 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:14.887 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-03-04T09:41:14.887 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.888 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.888 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-03-04T09:41:14.888 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-03-04T09:41:14.888 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-03-04T09:41:14.888 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-03-04T09:41:14.888 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-03-04T09:41:14.888 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-03-04T09:41:14.888 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-03-04T09:41:14.889 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-03-04T09:41:14.889 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:14.889 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.889 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.889 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.889 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.889 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-03-04T09:41:14.889 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-03-04T09:41:14.889 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:14.890 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.890 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:14.890 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-03-04T09:41:14.890 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.890 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.890 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:14.890 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:14.890 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.891 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.891 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.891 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.891 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-03-04T09:41:14.891 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-03-04T09:41:14.891 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:14.891 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.891 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:14.891 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-03-04T09:41:14.892 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.892 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.892 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-03-04T09:41:14.892 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-03-04T09:41:14.892 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-03-04T09:41:14.892 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-03-04T09:41:14.892 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-03-04T09:41:14.892 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-03-04T09:41:14.893 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-03-04T09:41:14.893 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-03-04T09:41:14.893 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:14.893 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.893 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.893 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.893 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.893 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-03-04T09:41:14.893 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-03-04T09:41:14.894 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:14.894 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.894 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:14.894 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-03-04T09:41:14.894 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.894 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.894 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:14.894 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:14.895 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.895 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.895 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.895 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.895 INFO:teuthology.orchestra.run.smithi052.stdout: "sleep_interval": { 2024-03-04T09:41:14.895 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "sleep_interval", 2024-03-04T09:41:14.895 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "secs", 2024-03-04T09:41:14.895 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.896 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:14.896 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "60", 2024-03-04T09:41:14.896 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.896 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.896 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:14.896 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:14.896 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.896 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.896 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.897 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.897 INFO:teuthology.orchestra.run.smithi052.stdout: "threshold": { 2024-03-04T09:41:14.897 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "threshold", 2024-03-04T09:41:14.897 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "float", 2024-03-04T09:41:14.897 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.897 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:14.897 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "3.0", 2024-03-04T09:41:14.897 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "1.0", 2024-03-04T09:41:14.898 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.898 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:14.898 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "scaling threshold", 2024-03-04T09:41:14.898 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-03-04T09:41:14.898 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.898 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.898 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-03-04T09:41:14.898 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-03-04T09:41:14.898 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.899 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-03-04T09:41:14.899 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "progress", 2024-03-04T09:41:14.899 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-03-04T09:41:14.899 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-03-04T09:41:14.899 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-03-04T09:41:14.899 INFO:teuthology.orchestra.run.smithi052.stdout: "allow_pg_recovery_event": { 2024-03-04T09:41:14.899 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "allow_pg_recovery_event", 2024-03-04T09:41:14.899 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:14.900 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.900 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:14.900 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-03-04T09:41:14.900 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.900 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.900 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:14.900 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "allow the module to show pg recovery progress", 2024-03-04T09:41:14.900 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.900 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.901 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.901 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.901 INFO:teuthology.orchestra.run.smithi052.stdout: "enabled": { 2024-03-04T09:41:14.901 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "enabled", 2024-03-04T09:41:14.901 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:14.901 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.901 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:14.901 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-03-04T09:41:14.902 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.902 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.902 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:14.902 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:14.902 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.902 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.902 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.902 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.903 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-03-04T09:41:14.903 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-03-04T09:41:14.903 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:14.903 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.903 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:14.903 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-03-04T09:41:14.903 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.903 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.904 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-03-04T09:41:14.904 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-03-04T09:41:14.904 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-03-04T09:41:14.904 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-03-04T09:41:14.904 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-03-04T09:41:14.904 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-03-04T09:41:14.904 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-03-04T09:41:14.904 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-03-04T09:41:14.905 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:14.905 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.905 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.905 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.905 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.905 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-03-04T09:41:14.905 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-03-04T09:41:14.905 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:14.905 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.906 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:14.906 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-03-04T09:41:14.906 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.906 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.906 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:14.906 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:14.906 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.906 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.907 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.907 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.907 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-03-04T09:41:14.907 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-03-04T09:41:14.907 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:14.907 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.907 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:14.907 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-03-04T09:41:14.908 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.908 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.908 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-03-04T09:41:14.908 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-03-04T09:41:14.908 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-03-04T09:41:14.908 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-03-04T09:41:14.908 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-03-04T09:41:14.908 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-03-04T09:41:14.908 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-03-04T09:41:14.909 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-03-04T09:41:14.909 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:14.909 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.909 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.909 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.909 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.909 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-03-04T09:41:14.909 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-03-04T09:41:14.910 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:14.910 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.910 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:14.910 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-03-04T09:41:14.910 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.910 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.910 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:14.910 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:14.910 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.911 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.911 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.911 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.911 INFO:teuthology.orchestra.run.smithi052.stdout: "max_completed_events": { 2024-03-04T09:41:14.911 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "max_completed_events", 2024-03-04T09:41:14.911 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-03-04T09:41:14.911 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.912 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:14.912 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "50", 2024-03-04T09:41:14.912 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.912 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.912 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:14.912 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "number of past completed events to remember", 2024-03-04T09:41:14.913 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.913 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.913 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.913 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.913 INFO:teuthology.orchestra.run.smithi052.stdout: "sleep_interval": { 2024-03-04T09:41:14.913 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "sleep_interval", 2024-03-04T09:41:14.913 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "secs", 2024-03-04T09:41:14.913 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.913 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:14.914 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "5", 2024-03-04T09:41:14.914 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.914 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.914 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:14.914 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "how long the module is going to sleep", 2024-03-04T09:41:14.914 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.914 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.914 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.915 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-03-04T09:41:14.915 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-03-04T09:41:14.915 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.915 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-03-04T09:41:14.915 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "prometheus", 2024-03-04T09:41:14.915 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-03-04T09:41:14.915 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-03-04T09:41:14.915 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-03-04T09:41:14.916 INFO:teuthology.orchestra.run.smithi052.stdout: "cache": { 2024-03-04T09:41:14.916 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "cache", 2024-03-04T09:41:14.916 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:14.916 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.916 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:14.916 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-03-04T09:41:14.916 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.916 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.917 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:14.917 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:14.917 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.917 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.917 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.917 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.917 INFO:teuthology.orchestra.run.smithi052.stdout: "exclude_perf_counters": { 2024-03-04T09:41:14.917 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "exclude_perf_counters", 2024-03-04T09:41:14.917 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:14.918 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.918 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:14.918 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-03-04T09:41:14.918 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.918 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.918 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:14.918 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-03-04T09:41:14.918 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-03-04T09:41:14.919 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.919 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.919 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.919 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-03-04T09:41:14.919 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-03-04T09:41:14.919 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:14.919 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.919 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:14.920 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-03-04T09:41:14.920 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.920 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.920 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-03-04T09:41:14.920 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-03-04T09:41:14.920 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-03-04T09:41:14.920 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-03-04T09:41:14.920 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-03-04T09:41:14.921 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-03-04T09:41:14.921 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-03-04T09:41:14.921 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-03-04T09:41:14.921 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:14.921 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.921 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.921 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.921 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.922 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-03-04T09:41:14.922 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-03-04T09:41:14.922 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:14.922 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.922 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:14.922 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-03-04T09:41:14.922 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.922 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.923 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:14.923 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:14.923 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.923 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.923 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.923 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.923 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-03-04T09:41:14.923 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-03-04T09:41:14.923 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:14.924 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.924 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:14.924 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-03-04T09:41:14.924 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.924 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.924 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-03-04T09:41:14.924 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-03-04T09:41:14.924 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-03-04T09:41:14.925 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-03-04T09:41:14.925 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-03-04T09:41:14.925 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-03-04T09:41:14.925 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-03-04T09:41:14.925 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-03-04T09:41:14.925 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:14.925 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.925 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.926 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.926 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.926 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-03-04T09:41:14.926 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-03-04T09:41:14.926 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:14.926 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.926 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:14.926 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-03-04T09:41:14.927 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.927 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.927 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:14.927 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:14.927 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.927 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.927 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.927 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.928 INFO:teuthology.orchestra.run.smithi052.stdout: "rbd_stats_pools": { 2024-03-04T09:41:14.928 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "rbd_stats_pools", 2024-03-04T09:41:14.928 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:14.928 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.928 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:14.928 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-03-04T09:41:14.928 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.928 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.929 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:14.929 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:14.929 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.929 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.929 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.929 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.929 INFO:teuthology.orchestra.run.smithi052.stdout: "rbd_stats_pools_refresh_interval": { 2024-03-04T09:41:14.929 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-03-04T09:41:14.930 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-03-04T09:41:14.930 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.930 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:14.930 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "300", 2024-03-04T09:41:14.930 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.930 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.930 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:14.930 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:14.931 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.931 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.931 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.931 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.931 INFO:teuthology.orchestra.run.smithi052.stdout: "scrape_interval": { 2024-03-04T09:41:14.931 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "scrape_interval", 2024-03-04T09:41:14.931 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "float", 2024-03-04T09:41:14.931 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.931 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:14.932 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "15.0", 2024-03-04T09:41:14.932 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.932 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.932 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:14.932 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:14.932 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.932 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.932 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.933 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.933 INFO:teuthology.orchestra.run.smithi052.stdout: "server_addr": { 2024-03-04T09:41:14.933 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "server_addr", 2024-03-04T09:41:14.933 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:14.933 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.933 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:14.933 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "::", 2024-03-04T09:41:14.934 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.934 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.934 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:14.934 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-03-04T09:41:14.934 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.934 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.934 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.934 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.934 INFO:teuthology.orchestra.run.smithi052.stdout: "server_port": { 2024-03-04T09:41:14.935 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "server_port", 2024-03-04T09:41:14.935 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-03-04T09:41:14.935 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.935 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:14.935 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "9283", 2024-03-04T09:41:14.935 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.935 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.935 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:14.936 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-03-04T09:41:14.936 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.936 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.936 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.936 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.936 INFO:teuthology.orchestra.run.smithi052.stdout: "stale_cache_strategy": { 2024-03-04T09:41:14.936 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "stale_cache_strategy", 2024-03-04T09:41:14.937 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:14.937 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.937 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:14.937 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "log", 2024-03-04T09:41:14.937 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.937 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.937 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:14.937 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:14.937 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.938 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.938 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.938 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.938 INFO:teuthology.orchestra.run.smithi052.stdout: "standby_behaviour": { 2024-03-04T09:41:14.938 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "standby_behaviour", 2024-03-04T09:41:14.938 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:14.938 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.938 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:14.939 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "default", 2024-03-04T09:41:14.939 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.939 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.939 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-03-04T09:41:14.939 INFO:teuthology.orchestra.run.smithi052.stdout: "default", 2024-03-04T09:41:14.939 INFO:teuthology.orchestra.run.smithi052.stdout: "error" 2024-03-04T09:41:14.939 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-03-04T09:41:14.939 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:14.940 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.940 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.940 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.940 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.940 INFO:teuthology.orchestra.run.smithi052.stdout: "standby_error_status_code": { 2024-03-04T09:41:14.940 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "standby_error_status_code", 2024-03-04T09:41:14.940 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-03-04T09:41:14.940 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.941 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:14.941 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "500", 2024-03-04T09:41:14.941 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "400", 2024-03-04T09:41:14.941 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "599", 2024-03-04T09:41:14.941 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:14.941 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:14.941 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.941 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.941 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.942 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-03-04T09:41:14.942 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-03-04T09:41:14.942 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.942 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-03-04T09:41:14.942 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "rbd_support", 2024-03-04T09:41:14.942 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-03-04T09:41:14.942 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-03-04T09:41:14.942 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-03-04T09:41:14.943 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-03-04T09:41:14.943 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-03-04T09:41:14.943 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:14.943 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.943 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:14.943 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-03-04T09:41:14.943 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.943 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.944 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-03-04T09:41:14.944 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-03-04T09:41:14.944 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-03-04T09:41:14.944 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-03-04T09:41:14.944 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-03-04T09:41:14.944 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-03-04T09:41:14.944 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-03-04T09:41:14.944 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-03-04T09:41:14.945 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:14.945 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.945 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.945 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.945 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.945 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-03-04T09:41:14.945 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-03-04T09:41:14.945 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:14.945 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.946 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:14.946 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-03-04T09:41:14.946 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.946 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.946 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:14.946 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:14.946 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.946 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.947 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.947 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.947 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-03-04T09:41:14.947 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-03-04T09:41:14.947 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:14.947 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.947 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:14.947 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-03-04T09:41:14.947 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.948 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.948 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-03-04T09:41:14.948 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-03-04T09:41:14.948 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-03-04T09:41:14.948 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-03-04T09:41:14.948 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-03-04T09:41:14.948 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-03-04T09:41:14.949 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-03-04T09:41:14.949 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-03-04T09:41:14.949 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:14.949 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.949 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.949 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.949 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.949 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-03-04T09:41:14.950 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-03-04T09:41:14.950 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:14.950 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.950 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:14.950 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-03-04T09:41:14.950 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.950 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.950 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:14.950 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:14.951 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.951 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.951 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.951 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.951 INFO:teuthology.orchestra.run.smithi052.stdout: "max_concurrent_snap_create": { 2024-03-04T09:41:14.951 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "max_concurrent_snap_create", 2024-03-04T09:41:14.951 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-03-04T09:41:14.951 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.952 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:14.952 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "10", 2024-03-04T09:41:14.952 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.952 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.952 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:14.952 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:14.952 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.952 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.953 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.953 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.953 INFO:teuthology.orchestra.run.smithi052.stdout: "mirror_snapshot_schedule": { 2024-03-04T09:41:14.953 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "mirror_snapshot_schedule", 2024-03-04T09:41:14.953 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:14.953 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.953 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:14.953 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-03-04T09:41:14.953 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.954 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.954 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:14.954 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:14.954 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.954 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.954 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.954 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.954 INFO:teuthology.orchestra.run.smithi052.stdout: "trash_purge_schedule": { 2024-03-04T09:41:14.954 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "trash_purge_schedule", 2024-03-04T09:41:14.955 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:14.955 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.955 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:14.955 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-03-04T09:41:14.955 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.955 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.955 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:14.955 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:14.956 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.956 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.956 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.956 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-03-04T09:41:14.956 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-03-04T09:41:14.956 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.956 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-03-04T09:41:14.956 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "restful", 2024-03-04T09:41:14.957 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-03-04T09:41:14.957 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-03-04T09:41:14.957 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-03-04T09:41:14.957 INFO:teuthology.orchestra.run.smithi052.stdout: "enable_auth": { 2024-03-04T09:41:14.957 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "enable_auth", 2024-03-04T09:41:14.957 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:14.957 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.957 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:14.957 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-03-04T09:41:14.958 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.958 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.958 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:14.958 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:14.958 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.958 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.958 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.958 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.959 INFO:teuthology.orchestra.run.smithi052.stdout: "key_file": { 2024-03-04T09:41:14.959 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "key_file", 2024-03-04T09:41:14.959 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:14.959 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.959 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:14.959 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-03-04T09:41:14.959 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.959 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.959 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:14.960 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:14.960 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.960 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.960 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.960 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.960 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-03-04T09:41:14.960 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-03-04T09:41:14.960 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:14.961 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.961 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:14.961 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-03-04T09:41:14.961 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.961 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.961 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-03-04T09:41:14.961 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-03-04T09:41:14.961 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-03-04T09:41:14.962 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-03-04T09:41:14.962 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-03-04T09:41:14.962 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-03-04T09:41:14.962 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-03-04T09:41:14.962 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-03-04T09:41:14.962 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:14.962 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.962 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.962 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.963 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.963 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-03-04T09:41:14.963 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-03-04T09:41:14.963 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:14.963 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.963 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:14.963 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-03-04T09:41:14.963 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.964 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.964 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:14.964 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:14.964 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.964 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.964 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.964 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.964 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-03-04T09:41:14.965 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-03-04T09:41:14.965 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:14.965 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.965 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:14.965 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-03-04T09:41:14.965 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.965 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.965 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-03-04T09:41:14.965 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-03-04T09:41:14.966 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-03-04T09:41:14.966 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-03-04T09:41:14.966 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-03-04T09:41:14.966 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-03-04T09:41:14.966 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-03-04T09:41:14.966 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-03-04T09:41:14.966 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:14.966 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.967 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.967 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.967 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.967 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-03-04T09:41:14.967 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-03-04T09:41:14.967 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:14.967 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.967 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:14.967 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-03-04T09:41:14.968 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.968 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.968 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:14.968 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:14.968 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.968 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.968 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.968 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.969 INFO:teuthology.orchestra.run.smithi052.stdout: "server_addr": { 2024-03-04T09:41:14.969 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "server_addr", 2024-03-04T09:41:14.969 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:14.969 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.969 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:14.969 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-03-04T09:41:14.969 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.969 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.969 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:14.970 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:14.970 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.970 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.970 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.970 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.970 INFO:teuthology.orchestra.run.smithi052.stdout: "server_port": { 2024-03-04T09:41:14.970 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "server_port", 2024-03-04T09:41:14.970 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:14.971 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.971 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:14.971 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-03-04T09:41:14.971 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.971 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.971 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:14.971 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:14.971 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.971 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.972 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.972 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-03-04T09:41:14.972 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-03-04T09:41:14.972 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.972 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-03-04T09:41:14.972 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "rgw", 2024-03-04T09:41:14.972 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-03-04T09:41:14.972 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-03-04T09:41:14.973 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-03-04T09:41:14.973 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-03-04T09:41:14.973 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-03-04T09:41:14.973 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:14.973 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.973 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:14.973 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-03-04T09:41:14.973 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.974 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.974 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-03-04T09:41:14.974 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-03-04T09:41:14.974 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-03-04T09:41:14.974 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-03-04T09:41:14.974 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-03-04T09:41:14.974 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-03-04T09:41:14.974 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-03-04T09:41:14.975 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-03-04T09:41:14.975 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:14.975 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.975 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.975 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.975 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.975 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-03-04T09:41:14.975 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-03-04T09:41:14.975 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:14.976 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.976 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:14.976 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-03-04T09:41:14.976 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.976 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.976 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:14.976 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:14.976 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.977 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.977 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.977 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.977 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-03-04T09:41:14.977 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-03-04T09:41:14.977 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:14.977 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.977 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:14.977 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-03-04T09:41:14.978 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.978 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.978 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-03-04T09:41:14.978 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-03-04T09:41:14.978 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-03-04T09:41:14.978 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-03-04T09:41:14.978 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-03-04T09:41:14.978 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-03-04T09:41:14.978 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-03-04T09:41:14.979 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-03-04T09:41:14.979 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:14.979 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.979 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.979 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.979 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.979 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-03-04T09:41:14.979 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-03-04T09:41:14.980 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:14.980 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.980 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:14.980 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-03-04T09:41:14.980 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.980 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.980 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:14.980 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:14.980 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.981 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.981 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.981 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-03-04T09:41:14.981 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-03-04T09:41:14.981 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.981 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-03-04T09:41:14.981 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "rook", 2024-03-04T09:41:14.981 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-03-04T09:41:14.982 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-03-04T09:41:14.982 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-03-04T09:41:14.982 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-03-04T09:41:14.982 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-03-04T09:41:14.982 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:14.982 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.982 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:14.982 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-03-04T09:41:14.982 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.983 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.983 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-03-04T09:41:14.983 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-03-04T09:41:14.983 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-03-04T09:41:14.983 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-03-04T09:41:14.983 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-03-04T09:41:14.983 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-03-04T09:41:14.983 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-03-04T09:41:14.983 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-03-04T09:41:14.984 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:14.984 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.984 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.984 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.984 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.984 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-03-04T09:41:14.984 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-03-04T09:41:14.984 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:14.985 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.985 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:14.985 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-03-04T09:41:14.985 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.985 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.985 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:14.985 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:14.985 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.986 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.986 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.986 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.986 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-03-04T09:41:14.986 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-03-04T09:41:14.986 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:14.986 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.986 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:14.987 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-03-04T09:41:14.987 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.987 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.987 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-03-04T09:41:14.987 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-03-04T09:41:14.987 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-03-04T09:41:14.987 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-03-04T09:41:14.987 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-03-04T09:41:14.988 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-03-04T09:41:14.988 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-03-04T09:41:14.988 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-03-04T09:41:14.988 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:14.988 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.988 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.988 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.988 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.989 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-03-04T09:41:14.989 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-03-04T09:41:14.989 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:14.989 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.989 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:14.989 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-03-04T09:41:14.989 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.989 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.989 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:14.990 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:14.990 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.990 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.990 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.990 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.990 INFO:teuthology.orchestra.run.smithi052.stdout: "storage_class": { 2024-03-04T09:41:14.990 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "storage_class", 2024-03-04T09:41:14.990 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:14.991 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.991 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:14.991 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "local", 2024-03-04T09:41:14.991 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.991 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.991 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:14.991 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-03-04T09:41:14.991 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.992 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.992 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.992 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-03-04T09:41:14.992 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-03-04T09:41:14.992 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.992 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-03-04T09:41:14.992 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "selftest", 2024-03-04T09:41:14.993 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-03-04T09:41:14.993 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-03-04T09:41:14.993 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-03-04T09:41:14.993 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-03-04T09:41:14.993 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-03-04T09:41:14.993 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:14.993 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.993 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:14.994 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-03-04T09:41:14.994 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.994 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.994 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-03-04T09:41:14.994 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-03-04T09:41:14.994 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-03-04T09:41:14.994 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-03-04T09:41:14.994 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-03-04T09:41:14.994 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-03-04T09:41:14.995 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-03-04T09:41:14.995 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-03-04T09:41:14.995 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:14.995 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.995 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.995 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.995 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.995 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-03-04T09:41:14.996 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-03-04T09:41:14.996 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:14.996 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.996 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:14.996 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-03-04T09:41:14.996 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.996 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.996 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:14.997 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:14.997 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.997 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:14.997 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:14.997 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:14.997 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-03-04T09:41:14.997 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-03-04T09:41:14.997 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:14.998 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:14.998 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:14.998 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-03-04T09:41:14.998 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:14.998 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:14.998 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-03-04T09:41:14.998 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-03-04T09:41:14.998 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-03-04T09:41:14.999 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-03-04T09:41:14.999 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-03-04T09:41:14.999 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-03-04T09:41:14.999 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-03-04T09:41:14.999 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-03-04T09:41:14.999 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:14.999 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:14.999 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:15.000 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:15.000 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:15.000 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-03-04T09:41:15.000 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-03-04T09:41:15.000 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:15.000 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:15.000 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:15.001 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-03-04T09:41:15.001 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:15.001 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:15.001 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:15.001 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:15.001 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:15.001 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:15.001 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:15.002 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:15.002 INFO:teuthology.orchestra.run.smithi052.stdout: "roption1": { 2024-03-04T09:41:15.002 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "roption1", 2024-03-04T09:41:15.002 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:15.002 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:15.002 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:15.002 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-03-04T09:41:15.002 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:15.003 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:15.003 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:15.003 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:15.003 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:15.003 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:15.003 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:15.003 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:15.003 INFO:teuthology.orchestra.run.smithi052.stdout: "roption2": { 2024-03-04T09:41:15.004 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "roption2", 2024-03-04T09:41:15.004 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:15.004 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:15.004 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:15.004 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "xyz", 2024-03-04T09:41:15.004 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:15.004 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:15.005 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:15.005 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:15.005 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:15.005 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:15.005 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:15.006 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:15.006 INFO:teuthology.orchestra.run.smithi052.stdout: "rwoption1": { 2024-03-04T09:41:15.006 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "rwoption1", 2024-03-04T09:41:15.006 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:15.006 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:15.006 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:15.006 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-03-04T09:41:15.006 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:15.007 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:15.007 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:15.007 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:15.007 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:15.007 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:15.007 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:15.007 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:15.007 INFO:teuthology.orchestra.run.smithi052.stdout: "rwoption2": { 2024-03-04T09:41:15.008 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "rwoption2", 2024-03-04T09:41:15.008 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-03-04T09:41:15.008 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:15.008 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:15.008 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-03-04T09:41:15.008 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:15.008 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:15.008 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:15.008 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:15.009 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:15.009 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:15.009 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:15.009 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:15.009 INFO:teuthology.orchestra.run.smithi052.stdout: "rwoption3": { 2024-03-04T09:41:15.009 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "rwoption3", 2024-03-04T09:41:15.009 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "float", 2024-03-04T09:41:15.010 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:15.010 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:15.010 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-03-04T09:41:15.010 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:15.010 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:15.010 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:15.010 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:15.010 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:15.010 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:15.011 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:15.011 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:15.011 INFO:teuthology.orchestra.run.smithi052.stdout: "rwoption4": { 2024-03-04T09:41:15.011 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "rwoption4", 2024-03-04T09:41:15.011 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:15.011 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:15.011 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:15.011 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-03-04T09:41:15.012 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:15.012 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:15.012 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:15.012 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:15.012 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:15.012 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:15.012 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:15.012 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:15.013 INFO:teuthology.orchestra.run.smithi052.stdout: "rwoption5": { 2024-03-04T09:41:15.013 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "rwoption5", 2024-03-04T09:41:15.013 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:15.013 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:15.013 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:15.013 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-03-04T09:41:15.013 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:15.013 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:15.014 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:15.014 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:15.014 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:15.014 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:15.014 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:15.014 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:15.014 INFO:teuthology.orchestra.run.smithi052.stdout: "rwoption6": { 2024-03-04T09:41:15.014 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "rwoption6", 2024-03-04T09:41:15.015 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:15.015 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:15.015 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:15.015 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-03-04T09:41:15.015 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:15.015 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:15.015 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:15.015 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:15.016 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:15.016 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:15.016 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:15.016 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:15.016 INFO:teuthology.orchestra.run.smithi052.stdout: "rwoption7": { 2024-03-04T09:41:15.016 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "rwoption7", 2024-03-04T09:41:15.016 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-03-04T09:41:15.016 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:15.017 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:15.017 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-03-04T09:41:15.017 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "1", 2024-03-04T09:41:15.017 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "42", 2024-03-04T09:41:15.017 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:15.017 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:15.017 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:15.017 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:15.018 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:15.018 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:15.018 INFO:teuthology.orchestra.run.smithi052.stdout: "testkey": { 2024-03-04T09:41:15.018 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "testkey", 2024-03-04T09:41:15.018 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:15.018 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:15.018 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:15.018 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-03-04T09:41:15.018 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:15.019 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:15.019 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:15.019 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:15.019 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:15.019 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:15.019 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:15.019 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:15.019 INFO:teuthology.orchestra.run.smithi052.stdout: "testlkey": { 2024-03-04T09:41:15.020 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "testlkey", 2024-03-04T09:41:15.020 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:15.020 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:15.020 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:15.020 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-03-04T09:41:15.020 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:15.020 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:15.020 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:15.021 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:15.021 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:15.021 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:15.021 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:15.021 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:15.021 INFO:teuthology.orchestra.run.smithi052.stdout: "testnewline": { 2024-03-04T09:41:15.021 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "testnewline", 2024-03-04T09:41:15.021 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:15.021 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:15.022 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:15.022 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-03-04T09:41:15.022 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:15.022 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:15.022 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:15.022 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:15.022 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:15.022 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:15.023 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:15.023 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-03-04T09:41:15.023 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-03-04T09:41:15.023 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:15.023 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-03-04T09:41:15.023 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "snap_schedule", 2024-03-04T09:41:15.023 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-03-04T09:41:15.023 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-03-04T09:41:15.024 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-03-04T09:41:15.024 INFO:teuthology.orchestra.run.smithi052.stdout: "allow_m_granularity": { 2024-03-04T09:41:15.024 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "allow_m_granularity", 2024-03-04T09:41:15.024 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:15.024 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:15.024 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:15.024 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-03-04T09:41:15.024 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:15.024 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:15.025 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:15.025 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "allow minute scheduled snapshots", 2024-03-04T09:41:15.025 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:15.025 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:15.025 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:15.025 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:15.025 INFO:teuthology.orchestra.run.smithi052.stdout: "dump_on_update": { 2024-03-04T09:41:15.025 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "dump_on_update", 2024-03-04T09:41:15.026 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:15.026 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:15.026 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:15.026 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-03-04T09:41:15.026 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:15.026 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:15.026 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:15.026 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "dump database to debug log on update", 2024-03-04T09:41:15.027 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:15.027 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:15.027 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:15.027 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:15.027 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-03-04T09:41:15.027 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-03-04T09:41:15.027 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:15.027 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:15.028 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:15.028 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-03-04T09:41:15.028 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:15.028 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:15.028 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-03-04T09:41:15.028 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-03-04T09:41:15.028 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-03-04T09:41:15.028 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-03-04T09:41:15.029 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-03-04T09:41:15.029 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-03-04T09:41:15.029 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-03-04T09:41:15.029 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-03-04T09:41:15.029 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:15.029 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:15.029 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:15.029 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:15.030 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:15.030 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-03-04T09:41:15.030 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-03-04T09:41:15.030 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:15.030 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:15.030 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:15.030 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-03-04T09:41:15.030 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:15.030 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:15.031 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:15.031 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:15.031 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:15.031 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:15.031 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:15.031 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:15.031 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-03-04T09:41:15.031 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-03-04T09:41:15.032 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:15.032 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:15.032 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:15.032 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-03-04T09:41:15.032 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:15.032 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:15.032 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-03-04T09:41:15.032 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-03-04T09:41:15.033 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-03-04T09:41:15.033 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-03-04T09:41:15.033 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-03-04T09:41:15.033 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-03-04T09:41:15.033 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-03-04T09:41:15.033 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-03-04T09:41:15.033 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:15.033 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:15.034 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:15.034 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:15.034 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:15.034 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-03-04T09:41:15.034 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-03-04T09:41:15.034 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:15.034 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:15.034 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:15.034 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-03-04T09:41:15.035 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:15.035 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:15.035 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:15.035 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:15.035 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:15.035 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:15.035 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:15.035 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-03-04T09:41:15.035 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-03-04T09:41:15.036 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:15.036 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-03-04T09:41:15.036 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "stats", 2024-03-04T09:41:15.036 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-03-04T09:41:15.036 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-03-04T09:41:15.036 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-03-04T09:41:15.036 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-03-04T09:41:15.037 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-03-04T09:41:15.037 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:15.037 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:15.037 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:15.037 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-03-04T09:41:15.037 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:15.037 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:15.037 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-03-04T09:41:15.037 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-03-04T09:41:15.038 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-03-04T09:41:15.038 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-03-04T09:41:15.038 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-03-04T09:41:15.038 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-03-04T09:41:15.038 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-03-04T09:41:15.038 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-03-04T09:41:15.038 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:15.038 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:15.039 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:15.039 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:15.039 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:15.039 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-03-04T09:41:15.039 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-03-04T09:41:15.039 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:15.039 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:15.040 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:15.040 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-03-04T09:41:15.040 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:15.040 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:15.040 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:15.040 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:15.040 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:15.040 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:15.041 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:15.041 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:15.041 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-03-04T09:41:15.041 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-03-04T09:41:15.041 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:15.041 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:15.041 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:15.041 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-03-04T09:41:15.041 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:15.042 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:15.042 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-03-04T09:41:15.042 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-03-04T09:41:15.042 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-03-04T09:41:15.042 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-03-04T09:41:15.042 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-03-04T09:41:15.042 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-03-04T09:41:15.042 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-03-04T09:41:15.043 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-03-04T09:41:15.043 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:15.043 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:15.043 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:15.043 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:15.043 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:15.043 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-03-04T09:41:15.043 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-03-04T09:41:15.044 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:15.044 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:15.044 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:15.044 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-03-04T09:41:15.044 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:15.044 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:15.044 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:15.044 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:15.045 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:15.045 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:15.045 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:15.045 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-03-04T09:41:15.045 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-03-04T09:41:15.045 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:15.045 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-03-04T09:41:15.045 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "status", 2024-03-04T09:41:15.046 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-03-04T09:41:15.046 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-03-04T09:41:15.046 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-03-04T09:41:15.046 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-03-04T09:41:15.046 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-03-04T09:41:15.046 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:15.046 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:15.046 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:15.047 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-03-04T09:41:15.047 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:15.047 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:15.047 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-03-04T09:41:15.047 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-03-04T09:41:15.047 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-03-04T09:41:15.047 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-03-04T09:41:15.047 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-03-04T09:41:15.048 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-03-04T09:41:15.048 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-03-04T09:41:15.048 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-03-04T09:41:15.048 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:15.048 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:15.048 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:15.048 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:15.048 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:15.049 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-03-04T09:41:15.049 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-03-04T09:41:15.049 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:15.049 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:15.049 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:15.049 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-03-04T09:41:15.049 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:15.049 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:15.049 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:15.050 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:15.050 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:15.050 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:15.050 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:15.050 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:15.050 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-03-04T09:41:15.050 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-03-04T09:41:15.050 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:15.051 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:15.051 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:15.051 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-03-04T09:41:15.051 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:15.051 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:15.051 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-03-04T09:41:15.051 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-03-04T09:41:15.051 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-03-04T09:41:15.051 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-03-04T09:41:15.052 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-03-04T09:41:15.052 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-03-04T09:41:15.052 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-03-04T09:41:15.052 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-03-04T09:41:15.052 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:15.052 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:15.052 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:15.053 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:15.053 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:15.053 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-03-04T09:41:15.053 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-03-04T09:41:15.053 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:15.053 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:15.053 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:15.053 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-03-04T09:41:15.054 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:15.054 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:15.054 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:15.054 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:15.054 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:15.054 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:15.054 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:15.054 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-03-04T09:41:15.055 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-03-04T09:41:15.055 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:15.055 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-03-04T09:41:15.055 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "telegraf", 2024-03-04T09:41:15.055 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-03-04T09:41:15.055 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-03-04T09:41:15.055 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-03-04T09:41:15.055 INFO:teuthology.orchestra.run.smithi052.stdout: "address": { 2024-03-04T09:41:15.055 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "address", 2024-03-04T09:41:15.056 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:15.056 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:15.056 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:15.056 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-03-04T09:41:15.056 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:15.056 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:15.056 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:15.056 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:15.057 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:15.057 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:15.057 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:15.057 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:15.057 INFO:teuthology.orchestra.run.smithi052.stdout: "interval": { 2024-03-04T09:41:15.057 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "interval", 2024-03-04T09:41:15.057 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "secs", 2024-03-04T09:41:15.058 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:15.058 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:15.058 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "15", 2024-03-04T09:41:15.058 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:15.058 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:15.058 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:15.058 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:15.058 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:15.059 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:15.059 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:15.059 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:15.059 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-03-04T09:41:15.059 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-03-04T09:41:15.059 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:15.059 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:15.059 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:15.060 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-03-04T09:41:15.060 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:15.060 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:15.060 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-03-04T09:41:15.060 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-03-04T09:41:15.060 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-03-04T09:41:15.060 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-03-04T09:41:15.060 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-03-04T09:41:15.061 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-03-04T09:41:15.061 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-03-04T09:41:15.061 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-03-04T09:41:15.061 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:15.061 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:15.061 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:15.061 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:15.061 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:15.062 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-03-04T09:41:15.062 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-03-04T09:41:15.062 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:15.062 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:15.062 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:15.062 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-03-04T09:41:15.062 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:15.062 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:15.063 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:15.063 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:15.063 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:15.063 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:15.063 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:15.063 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:15.063 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-03-04T09:41:15.063 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-03-04T09:41:15.063 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:15.064 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:15.064 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:15.064 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-03-04T09:41:15.064 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:15.064 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:15.064 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-03-04T09:41:15.064 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-03-04T09:41:15.064 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-03-04T09:41:15.065 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-03-04T09:41:15.065 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-03-04T09:41:15.065 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-03-04T09:41:15.065 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-03-04T09:41:15.065 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-03-04T09:41:15.065 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:15.065 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:15.065 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:15.066 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:15.066 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:15.066 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-03-04T09:41:15.066 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-03-04T09:41:15.066 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:15.066 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:15.066 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:15.066 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-03-04T09:41:15.067 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:15.067 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:15.067 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:15.067 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:15.067 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:15.067 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:15.067 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:15.067 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-03-04T09:41:15.068 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-03-04T09:41:15.068 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:15.068 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-03-04T09:41:15.068 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "telemetry", 2024-03-04T09:41:15.068 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-03-04T09:41:15.068 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-03-04T09:41:15.068 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-03-04T09:41:15.068 INFO:teuthology.orchestra.run.smithi052.stdout: "channel_basic": { 2024-03-04T09:41:15.069 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "channel_basic", 2024-03-04T09:41:15.069 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:15.069 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:15.069 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:15.069 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-03-04T09:41:15.069 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:15.069 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:15.069 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:15.069 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Share basic cluster information (size, version)", 2024-03-04T09:41:15.070 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:15.070 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:15.070 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:15.070 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:15.070 INFO:teuthology.orchestra.run.smithi052.stdout: "channel_crash": { 2024-03-04T09:41:15.070 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "channel_crash", 2024-03-04T09:41:15.070 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:15.070 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:15.071 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:15.071 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-03-04T09:41:15.071 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:15.071 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:15.071 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:15.071 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-03-04T09:41:15.071 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:15.071 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:15.072 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:15.072 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:15.072 INFO:teuthology.orchestra.run.smithi052.stdout: "channel_device": { 2024-03-04T09:41:15.072 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "channel_device", 2024-03-04T09:41:15.072 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:15.072 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:15.072 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:15.072 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-03-04T09:41:15.073 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:15.073 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:15.073 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:15.073 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-03-04T09:41:15.073 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:15.073 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:15.073 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:15.073 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:15.073 INFO:teuthology.orchestra.run.smithi052.stdout: "channel_ident": { 2024-03-04T09:41:15.074 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "channel_ident", 2024-03-04T09:41:15.074 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:15.074 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:15.074 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:15.074 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-03-04T09:41:15.074 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:15.074 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:15.074 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:15.075 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-03-04T09:41:15.075 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:15.075 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:15.075 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:15.075 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:15.075 INFO:teuthology.orchestra.run.smithi052.stdout: "channel_perf": { 2024-03-04T09:41:15.075 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "channel_perf", 2024-03-04T09:41:15.075 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:15.076 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:15.076 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:15.076 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-03-04T09:41:15.076 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:15.076 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:15.076 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:15.076 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Share various performance metrics of a cluster", 2024-03-04T09:41:15.076 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:15.076 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:15.077 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:15.077 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:15.077 INFO:teuthology.orchestra.run.smithi052.stdout: "contact": { 2024-03-04T09:41:15.077 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "contact", 2024-03-04T09:41:15.077 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:15.077 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:15.077 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:15.077 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-03-04T09:41:15.078 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:15.078 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:15.078 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:15.078 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:15.078 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:15.078 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:15.078 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:15.078 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:15.078 INFO:teuthology.orchestra.run.smithi052.stdout: "description": { 2024-03-04T09:41:15.079 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "description", 2024-03-04T09:41:15.079 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:15.079 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:15.079 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:15.079 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-03-04T09:41:15.079 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:15.079 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:15.079 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:15.080 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:15.080 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:15.080 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:15.080 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:15.080 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:15.080 INFO:teuthology.orchestra.run.smithi052.stdout: "device_url": { 2024-03-04T09:41:15.080 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "device_url", 2024-03-04T09:41:15.080 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:15.080 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:15.081 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:15.081 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-03-04T09:41:15.081 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:15.081 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:15.081 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:15.081 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:15.081 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:15.081 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:15.082 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:15.082 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:15.082 INFO:teuthology.orchestra.run.smithi052.stdout: "enabled": { 2024-03-04T09:41:15.082 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "enabled", 2024-03-04T09:41:15.082 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:15.082 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:15.082 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:15.082 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-03-04T09:41:15.082 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:15.083 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:15.083 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:15.083 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:15.083 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:15.083 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:15.083 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:15.083 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:15.083 INFO:teuthology.orchestra.run.smithi052.stdout: "interval": { 2024-03-04T09:41:15.084 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "interval", 2024-03-04T09:41:15.084 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-03-04T09:41:15.084 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:15.084 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:15.084 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "24", 2024-03-04T09:41:15.084 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "8", 2024-03-04T09:41:15.084 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:15.084 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:15.085 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:15.085 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:15.085 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:15.085 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:15.085 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:15.085 INFO:teuthology.orchestra.run.smithi052.stdout: "last_opt_revision": { 2024-03-04T09:41:15.085 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "last_opt_revision", 2024-03-04T09:41:15.085 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-03-04T09:41:15.085 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:15.086 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:15.086 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "1", 2024-03-04T09:41:15.086 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:15.086 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:15.086 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:15.086 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:15.086 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:15.086 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:15.087 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:15.087 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:15.087 INFO:teuthology.orchestra.run.smithi052.stdout: "leaderboard": { 2024-03-04T09:41:15.087 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "leaderboard", 2024-03-04T09:41:15.087 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:15.087 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:15.087 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:15.087 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-03-04T09:41:15.088 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:15.088 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:15.088 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:15.088 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:15.088 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:15.088 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:15.088 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:15.088 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:15.089 INFO:teuthology.orchestra.run.smithi052.stdout: "leaderboard_description": { 2024-03-04T09:41:15.089 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "leaderboard_description", 2024-03-04T09:41:15.089 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:15.089 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:15.089 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:15.089 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-03-04T09:41:15.089 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:15.089 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:15.090 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:15.090 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:15.090 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:15.090 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:15.090 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:15.090 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:15.090 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-03-04T09:41:15.090 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-03-04T09:41:15.090 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:15.091 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:15.091 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:15.091 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-03-04T09:41:15.091 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:15.091 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:15.091 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-03-04T09:41:15.091 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-03-04T09:41:15.091 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-03-04T09:41:15.092 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-03-04T09:41:15.092 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-03-04T09:41:15.092 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-03-04T09:41:15.092 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-03-04T09:41:15.092 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-03-04T09:41:15.092 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:15.092 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:15.092 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:15.093 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:15.093 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:15.093 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-03-04T09:41:15.093 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-03-04T09:41:15.093 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:15.093 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:15.093 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:15.093 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-03-04T09:41:15.094 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:15.094 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:15.094 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:15.094 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:15.094 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:15.094 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:15.094 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:15.094 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:15.095 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-03-04T09:41:15.095 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-03-04T09:41:15.095 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:15.095 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:15.095 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:15.095 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-03-04T09:41:15.095 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:15.095 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:15.096 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-03-04T09:41:15.096 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-03-04T09:41:15.096 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-03-04T09:41:15.096 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-03-04T09:41:15.096 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-03-04T09:41:15.096 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-03-04T09:41:15.096 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-03-04T09:41:15.096 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-03-04T09:41:15.097 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:15.097 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:15.097 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:15.097 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:15.097 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:15.097 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-03-04T09:41:15.097 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-03-04T09:41:15.097 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:15.097 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:15.098 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:15.098 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-03-04T09:41:15.098 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:15.098 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:15.098 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:15.098 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:15.098 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:15.098 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:15.099 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:15.099 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:15.099 INFO:teuthology.orchestra.run.smithi052.stdout: "organization": { 2024-03-04T09:41:15.099 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "organization", 2024-03-04T09:41:15.099 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:15.099 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:15.099 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:15.099 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-03-04T09:41:15.100 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:15.100 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:15.100 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:15.100 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:15.100 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:15.100 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:15.100 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:15.101 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:15.101 INFO:teuthology.orchestra.run.smithi052.stdout: "proxy": { 2024-03-04T09:41:15.101 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "proxy", 2024-03-04T09:41:15.101 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:15.101 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:15.101 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:15.101 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-03-04T09:41:15.101 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:15.101 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:15.102 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:15.102 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:15.102 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:15.102 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:15.102 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:15.102 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:15.102 INFO:teuthology.orchestra.run.smithi052.stdout: "url": { 2024-03-04T09:41:15.102 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "url", 2024-03-04T09:41:15.103 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:15.103 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:15.103 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:15.103 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-03-04T09:41:15.103 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:15.103 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:15.103 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:15.104 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:15.104 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:15.104 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:15.104 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:15.104 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-03-04T09:41:15.104 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-03-04T09:41:15.104 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:15.104 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-03-04T09:41:15.105 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "test_orchestrator", 2024-03-04T09:41:15.105 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-03-04T09:41:15.105 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-03-04T09:41:15.105 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-03-04T09:41:15.105 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-03-04T09:41:15.105 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-03-04T09:41:15.105 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:15.105 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:15.106 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:15.106 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-03-04T09:41:15.106 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:15.106 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:15.106 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-03-04T09:41:15.106 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-03-04T09:41:15.106 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-03-04T09:41:15.106 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-03-04T09:41:15.107 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-03-04T09:41:15.107 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-03-04T09:41:15.107 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-03-04T09:41:15.107 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-03-04T09:41:15.107 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:15.107 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:15.107 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:15.107 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:15.108 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:15.108 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-03-04T09:41:15.108 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-03-04T09:41:15.108 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:15.108 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:15.108 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:15.108 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-03-04T09:41:15.108 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:15.109 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:15.109 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:15.109 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:15.109 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:15.109 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:15.109 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:15.109 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:15.109 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-03-04T09:41:15.109 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-03-04T09:41:15.110 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:15.110 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:15.110 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:15.110 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-03-04T09:41:15.110 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:15.110 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:15.110 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-03-04T09:41:15.110 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-03-04T09:41:15.110 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-03-04T09:41:15.111 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-03-04T09:41:15.111 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-03-04T09:41:15.111 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-03-04T09:41:15.111 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-03-04T09:41:15.111 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-03-04T09:41:15.111 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:15.111 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:15.111 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:15.112 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:15.112 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:15.112 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-03-04T09:41:15.112 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-03-04T09:41:15.112 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:15.112 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:15.112 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:15.112 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-03-04T09:41:15.113 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:15.113 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:15.113 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:15.113 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:15.113 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:15.113 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:15.113 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:15.113 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-03-04T09:41:15.113 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-03-04T09:41:15.114 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:15.114 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-03-04T09:41:15.114 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "volumes", 2024-03-04T09:41:15.114 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-03-04T09:41:15.114 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-03-04T09:41:15.114 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-03-04T09:41:15.114 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-03-04T09:41:15.114 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-03-04T09:41:15.115 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:15.115 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:15.115 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:15.115 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-03-04T09:41:15.115 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:15.115 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:15.115 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-03-04T09:41:15.115 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-03-04T09:41:15.115 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-03-04T09:41:15.116 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-03-04T09:41:15.116 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-03-04T09:41:15.116 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-03-04T09:41:15.116 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-03-04T09:41:15.116 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-03-04T09:41:15.116 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:15.116 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:15.116 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:15.117 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:15.117 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:15.117 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-03-04T09:41:15.117 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-03-04T09:41:15.117 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:15.117 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:15.117 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:15.117 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-03-04T09:41:15.118 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:15.118 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:15.118 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:15.118 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:15.118 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:15.118 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:15.118 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:15.118 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:15.119 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-03-04T09:41:15.119 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-03-04T09:41:15.119 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:15.119 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:15.119 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:15.119 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-03-04T09:41:15.119 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:15.119 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:15.120 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-03-04T09:41:15.120 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-03-04T09:41:15.120 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-03-04T09:41:15.120 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-03-04T09:41:15.120 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-03-04T09:41:15.120 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-03-04T09:41:15.120 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-03-04T09:41:15.120 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-03-04T09:41:15.120 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:15.121 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:15.121 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:15.121 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:15.121 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:15.121 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-03-04T09:41:15.121 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-03-04T09:41:15.121 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:15.121 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:15.122 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:15.122 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-03-04T09:41:15.122 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:15.122 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:15.122 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:15.122 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:15.122 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:15.122 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:15.122 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:15.123 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:15.123 INFO:teuthology.orchestra.run.smithi052.stdout: "max_concurrent_clones": { 2024-03-04T09:41:15.123 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "max_concurrent_clones", 2024-03-04T09:41:15.123 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-03-04T09:41:15.123 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:15.123 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:15.123 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "4", 2024-03-04T09:41:15.123 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:15.124 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:15.124 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:15.124 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Number of asynchronous cloner threads", 2024-03-04T09:41:15.124 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:15.124 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:15.124 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:15.124 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:15.124 INFO:teuthology.orchestra.run.smithi052.stdout: "periodic_async_work": { 2024-03-04T09:41:15.125 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "periodic_async_work", 2024-03-04T09:41:15.125 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:15.125 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:15.125 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:15.125 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-03-04T09:41:15.125 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:15.125 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:15.125 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:15.125 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Periodically check for async work", 2024-03-04T09:41:15.126 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:15.126 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:15.126 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:15.126 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:15.126 INFO:teuthology.orchestra.run.smithi052.stdout: "snapshot_clone_delay": { 2024-03-04T09:41:15.126 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "snapshot_clone_delay", 2024-03-04T09:41:15.126 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-03-04T09:41:15.126 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:15.126 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:15.127 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "0", 2024-03-04T09:41:15.127 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:15.127 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:15.127 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:15.127 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-03-04T09:41:15.127 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:15.127 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:15.127 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:15.128 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:15.128 INFO:teuthology.orchestra.run.smithi052.stdout: "snapshot_clone_no_wait": { 2024-03-04T09:41:15.128 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "snapshot_clone_no_wait", 2024-03-04T09:41:15.128 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:15.128 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:15.128 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:15.128 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-03-04T09:41:15.128 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:15.129 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:15.129 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:15.129 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-03-04T09:41:15.129 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:15.129 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:15.129 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:15.129 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-03-04T09:41:15.129 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-03-04T09:41:15.129 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:15.130 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-03-04T09:41:15.130 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "zabbix", 2024-03-04T09:41:15.130 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-03-04T09:41:15.130 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-03-04T09:41:15.130 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-03-04T09:41:15.130 INFO:teuthology.orchestra.run.smithi052.stdout: "discovery_interval": { 2024-03-04T09:41:15.130 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "discovery_interval", 2024-03-04T09:41:15.130 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "uint", 2024-03-04T09:41:15.131 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:15.131 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:15.131 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "100", 2024-03-04T09:41:15.131 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:15.131 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:15.131 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:15.131 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:15.131 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:15.132 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:15.132 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:15.132 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:15.132 INFO:teuthology.orchestra.run.smithi052.stdout: "identifier": { 2024-03-04T09:41:15.132 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "identifier", 2024-03-04T09:41:15.132 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:15.132 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:15.133 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:15.133 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-03-04T09:41:15.133 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:15.133 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:15.133 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:15.133 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:15.133 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:15.133 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:15.133 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:15.134 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:15.134 INFO:teuthology.orchestra.run.smithi052.stdout: "interval": { 2024-03-04T09:41:15.134 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "interval", 2024-03-04T09:41:15.134 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "secs", 2024-03-04T09:41:15.134 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:15.134 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:15.134 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "60", 2024-03-04T09:41:15.134 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:15.135 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:15.135 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:15.135 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:15.135 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:15.135 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:15.135 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:15.135 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:15.135 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-03-04T09:41:15.135 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-03-04T09:41:15.136 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:15.136 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:15.136 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:15.136 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-03-04T09:41:15.136 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:15.136 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:15.136 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-03-04T09:41:15.136 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-03-04T09:41:15.137 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-03-04T09:41:15.137 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-03-04T09:41:15.137 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-03-04T09:41:15.137 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-03-04T09:41:15.137 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-03-04T09:41:15.137 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-03-04T09:41:15.137 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:15.137 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:15.137 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:15.138 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:15.138 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:15.138 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-03-04T09:41:15.138 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-03-04T09:41:15.138 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:15.138 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:15.138 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:15.138 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-03-04T09:41:15.139 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:15.139 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:15.139 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:15.139 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:15.139 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:15.139 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:15.139 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:15.139 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:15.139 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-03-04T09:41:15.140 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-03-04T09:41:15.140 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:15.140 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:15.140 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:15.140 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-03-04T09:41:15.140 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:15.140 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:15.140 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-03-04T09:41:15.141 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-03-04T09:41:15.141 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-03-04T09:41:15.141 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-03-04T09:41:15.141 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-03-04T09:41:15.141 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-03-04T09:41:15.141 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-03-04T09:41:15.141 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-03-04T09:41:15.141 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:15.141 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:15.142 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:15.142 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:15.142 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:15.142 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-03-04T09:41:15.142 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-03-04T09:41:15.142 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-03-04T09:41:15.142 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:15.142 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-03-04T09:41:15.143 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-03-04T09:41:15.143 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:15.143 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:15.143 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:15.143 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:15.143 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:15.143 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:15.143 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:15.143 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:15.144 INFO:teuthology.orchestra.run.smithi052.stdout: "zabbix_host": { 2024-03-04T09:41:15.144 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "zabbix_host", 2024-03-04T09:41:15.144 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:15.144 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:15.144 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:15.144 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-03-04T09:41:15.144 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:15.144 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:15.145 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:15.145 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:15.145 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:15.145 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:15.145 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:15.145 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:15.145 INFO:teuthology.orchestra.run.smithi052.stdout: "zabbix_port": { 2024-03-04T09:41:15.145 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "zabbix_port", 2024-03-04T09:41:15.146 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-03-04T09:41:15.146 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:15.146 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:15.146 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "10051", 2024-03-04T09:41:15.146 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:15.146 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:15.146 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:15.146 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:15.147 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:15.147 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:15.147 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:15.147 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:15.147 INFO:teuthology.orchestra.run.smithi052.stdout: "zabbix_sender": { 2024-03-04T09:41:15.147 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "zabbix_sender", 2024-03-04T09:41:15.147 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-03-04T09:41:15.147 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-03-04T09:41:15.147 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-03-04T09:41:15.148 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-03-04T09:41:15.148 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-03-04T09:41:15.148 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-03-04T09:41:15.148 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-03-04T09:41:15.148 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-03-04T09:41:15.148 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-03-04T09:41:15.148 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-03-04T09:41:15.148 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-03-04T09:41:15.149 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-03-04T09:41:15.149 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-03-04T09:41:15.149 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-03-04T09:41:15.149 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-03-04T09:41:15.149 INFO:teuthology.orchestra.run.smithi052.stdout: "services": { 2024-03-04T09:41:15.149 INFO:teuthology.orchestra.run.smithi052.stdout: "dashboard": "https://172.21.15.52:8443/" 2024-03-04T09:41:15.149 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:15.149 INFO:teuthology.orchestra.run.smithi052.stdout: "always_on_modules": { 2024-03-04T09:41:15.149 INFO:teuthology.orchestra.run.smithi052.stdout: "octopus": [ 2024-03-04T09:41:15.150 INFO:teuthology.orchestra.run.smithi052.stdout: "balancer", 2024-03-04T09:41:15.150 INFO:teuthology.orchestra.run.smithi052.stdout: "crash", 2024-03-04T09:41:15.150 INFO:teuthology.orchestra.run.smithi052.stdout: "devicehealth", 2024-03-04T09:41:15.150 INFO:teuthology.orchestra.run.smithi052.stdout: "orchestrator", 2024-03-04T09:41:15.150 INFO:teuthology.orchestra.run.smithi052.stdout: "pg_autoscaler", 2024-03-04T09:41:15.150 INFO:teuthology.orchestra.run.smithi052.stdout: "progress", 2024-03-04T09:41:15.150 INFO:teuthology.orchestra.run.smithi052.stdout: "rbd_support", 2024-03-04T09:41:15.150 INFO:teuthology.orchestra.run.smithi052.stdout: "status", 2024-03-04T09:41:15.151 INFO:teuthology.orchestra.run.smithi052.stdout: "telemetry", 2024-03-04T09:41:15.151 INFO:teuthology.orchestra.run.smithi052.stdout: "volumes" 2024-03-04T09:41:15.151 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-03-04T09:41:15.151 INFO:teuthology.orchestra.run.smithi052.stdout: "pacific": [ 2024-03-04T09:41:15.151 INFO:teuthology.orchestra.run.smithi052.stdout: "balancer", 2024-03-04T09:41:15.151 INFO:teuthology.orchestra.run.smithi052.stdout: "crash", 2024-03-04T09:41:15.151 INFO:teuthology.orchestra.run.smithi052.stdout: "devicehealth", 2024-03-04T09:41:15.151 INFO:teuthology.orchestra.run.smithi052.stdout: "orchestrator", 2024-03-04T09:41:15.152 INFO:teuthology.orchestra.run.smithi052.stdout: "pg_autoscaler", 2024-03-04T09:41:15.152 INFO:teuthology.orchestra.run.smithi052.stdout: "progress", 2024-03-04T09:41:15.152 INFO:teuthology.orchestra.run.smithi052.stdout: "rbd_support", 2024-03-04T09:41:15.152 INFO:teuthology.orchestra.run.smithi052.stdout: "status", 2024-03-04T09:41:15.152 INFO:teuthology.orchestra.run.smithi052.stdout: "telemetry", 2024-03-04T09:41:15.152 INFO:teuthology.orchestra.run.smithi052.stdout: "volumes" 2024-03-04T09:41:15.152 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-03-04T09:41:15.152 INFO:teuthology.orchestra.run.smithi052.stdout: "quincy": [ 2024-03-04T09:41:15.152 INFO:teuthology.orchestra.run.smithi052.stdout: "balancer", 2024-03-04T09:41:15.153 INFO:teuthology.orchestra.run.smithi052.stdout: "crash", 2024-03-04T09:41:15.153 INFO:teuthology.orchestra.run.smithi052.stdout: "devicehealth", 2024-03-04T09:41:15.153 INFO:teuthology.orchestra.run.smithi052.stdout: "orchestrator", 2024-03-04T09:41:15.153 INFO:teuthology.orchestra.run.smithi052.stdout: "pg_autoscaler", 2024-03-04T09:41:15.153 INFO:teuthology.orchestra.run.smithi052.stdout: "progress", 2024-03-04T09:41:15.153 INFO:teuthology.orchestra.run.smithi052.stdout: "rbd_support", 2024-03-04T09:41:15.153 INFO:teuthology.orchestra.run.smithi052.stdout: "status", 2024-03-04T09:41:15.153 INFO:teuthology.orchestra.run.smithi052.stdout: "telemetry", 2024-03-04T09:41:15.154 INFO:teuthology.orchestra.run.smithi052.stdout: "volumes" 2024-03-04T09:41:15.154 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-03-04T09:41:15.154 INFO:teuthology.orchestra.run.smithi052.stdout: "reef": [ 2024-03-04T09:41:15.154 INFO:teuthology.orchestra.run.smithi052.stdout: "balancer", 2024-03-04T09:41:15.154 INFO:teuthology.orchestra.run.smithi052.stdout: "crash", 2024-03-04T09:41:15.154 INFO:teuthology.orchestra.run.smithi052.stdout: "devicehealth", 2024-03-04T09:41:15.154 INFO:teuthology.orchestra.run.smithi052.stdout: "orchestrator", 2024-03-04T09:41:15.154 INFO:teuthology.orchestra.run.smithi052.stdout: "pg_autoscaler", 2024-03-04T09:41:15.155 INFO:teuthology.orchestra.run.smithi052.stdout: "progress", 2024-03-04T09:41:15.155 INFO:teuthology.orchestra.run.smithi052.stdout: "rbd_support", 2024-03-04T09:41:15.155 INFO:teuthology.orchestra.run.smithi052.stdout: "status", 2024-03-04T09:41:15.155 INFO:teuthology.orchestra.run.smithi052.stdout: "telemetry", 2024-03-04T09:41:15.155 INFO:teuthology.orchestra.run.smithi052.stdout: "volumes" 2024-03-04T09:41:15.155 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-03-04T09:41:15.155 INFO:teuthology.orchestra.run.smithi052.stdout: "squid": [ 2024-03-04T09:41:15.155 INFO:teuthology.orchestra.run.smithi052.stdout: "balancer", 2024-03-04T09:41:15.155 INFO:teuthology.orchestra.run.smithi052.stdout: "crash", 2024-03-04T09:41:15.156 INFO:teuthology.orchestra.run.smithi052.stdout: "devicehealth", 2024-03-04T09:41:15.156 INFO:teuthology.orchestra.run.smithi052.stdout: "orchestrator", 2024-03-04T09:41:15.156 INFO:teuthology.orchestra.run.smithi052.stdout: "pg_autoscaler", 2024-03-04T09:41:15.156 INFO:teuthology.orchestra.run.smithi052.stdout: "progress", 2024-03-04T09:41:15.156 INFO:teuthology.orchestra.run.smithi052.stdout: "rbd_support", 2024-03-04T09:41:15.156 INFO:teuthology.orchestra.run.smithi052.stdout: "status", 2024-03-04T09:41:15.156 INFO:teuthology.orchestra.run.smithi052.stdout: "telemetry", 2024-03-04T09:41:15.156 INFO:teuthology.orchestra.run.smithi052.stdout: "volumes" 2024-03-04T09:41:15.157 INFO:teuthology.orchestra.run.smithi052.stdout: ] 2024-03-04T09:41:15.157 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:15.157 INFO:teuthology.orchestra.run.smithi052.stdout: "last_failure_osd_epoch": 94, 2024-03-04T09:41:15.157 INFO:teuthology.orchestra.run.smithi052.stdout: "active_clients": [ 2024-03-04T09:41:15.157 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-03-04T09:41:15.157 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "devicehealth", 2024-03-04T09:41:15.157 INFO:teuthology.orchestra.run.smithi052.stdout: "addrvec": [ 2024-03-04T09:41:15.157 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-03-04T09:41:15.157 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "v2", 2024-03-04T09:41:15.158 INFO:teuthology.orchestra.run.smithi052.stdout: "addr": "172.21.15.52:0", 2024-03-04T09:41:15.158 INFO:teuthology.orchestra.run.smithi052.stdout: "nonce": 3895001843 2024-03-04T09:41:15.158 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-03-04T09:41:15.158 INFO:teuthology.orchestra.run.smithi052.stdout: ] 2024-03-04T09:41:15.158 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:15.158 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-03-04T09:41:15.158 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "libcephsqlite", 2024-03-04T09:41:15.158 INFO:teuthology.orchestra.run.smithi052.stdout: "addrvec": [ 2024-03-04T09:41:15.159 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-03-04T09:41:15.159 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "v2", 2024-03-04T09:41:15.159 INFO:teuthology.orchestra.run.smithi052.stdout: "addr": "172.21.15.52:0", 2024-03-04T09:41:15.159 INFO:teuthology.orchestra.run.smithi052.stdout: "nonce": 2391843749 2024-03-04T09:41:15.159 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-03-04T09:41:15.159 INFO:teuthology.orchestra.run.smithi052.stdout: ] 2024-03-04T09:41:15.159 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:15.159 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-03-04T09:41:15.159 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "rbd_support", 2024-03-04T09:41:15.160 INFO:teuthology.orchestra.run.smithi052.stdout: "addrvec": [ 2024-03-04T09:41:15.160 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-03-04T09:41:15.160 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "v2", 2024-03-04T09:41:15.160 INFO:teuthology.orchestra.run.smithi052.stdout: "addr": "172.21.15.52:0", 2024-03-04T09:41:15.160 INFO:teuthology.orchestra.run.smithi052.stdout: "nonce": 1965757792 2024-03-04T09:41:15.160 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-03-04T09:41:15.160 INFO:teuthology.orchestra.run.smithi052.stdout: ] 2024-03-04T09:41:15.160 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:15.161 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-03-04T09:41:15.161 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "snap_schedule", 2024-03-04T09:41:15.161 INFO:teuthology.orchestra.run.smithi052.stdout: "addrvec": [ 2024-03-04T09:41:15.161 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-03-04T09:41:15.161 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "v2", 2024-03-04T09:41:15.161 INFO:teuthology.orchestra.run.smithi052.stdout: "addr": "172.21.15.52:0", 2024-03-04T09:41:15.161 INFO:teuthology.orchestra.run.smithi052.stdout: "nonce": 234343398 2024-03-04T09:41:15.161 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-03-04T09:41:15.162 INFO:teuthology.orchestra.run.smithi052.stdout: ] 2024-03-04T09:41:15.162 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:15.162 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-03-04T09:41:15.162 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "snap_schedule", 2024-03-04T09:41:15.162 INFO:teuthology.orchestra.run.smithi052.stdout: "addrvec": [ 2024-03-04T09:41:15.162 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-03-04T09:41:15.162 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "v2", 2024-03-04T09:41:15.162 INFO:teuthology.orchestra.run.smithi052.stdout: "addr": "172.21.15.52:0", 2024-03-04T09:41:15.162 INFO:teuthology.orchestra.run.smithi052.stdout: "nonce": 3977212089 2024-03-04T09:41:15.162 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-03-04T09:41:15.163 INFO:teuthology.orchestra.run.smithi052.stdout: ] 2024-03-04T09:41:15.163 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:15.163 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-03-04T09:41:15.163 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "volumes", 2024-03-04T09:41:15.163 INFO:teuthology.orchestra.run.smithi052.stdout: "addrvec": [ 2024-03-04T09:41:15.163 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-03-04T09:41:15.163 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "v2", 2024-03-04T09:41:15.164 INFO:teuthology.orchestra.run.smithi052.stdout: "addr": "172.21.15.52:0", 2024-03-04T09:41:15.164 INFO:teuthology.orchestra.run.smithi052.stdout: "nonce": 664692289 2024-03-04T09:41:15.164 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-03-04T09:41:15.164 INFO:teuthology.orchestra.run.smithi052.stdout: ] 2024-03-04T09:41:15.164 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-03-04T09:41:15.164 INFO:teuthology.orchestra.run.smithi052.stdout: ] 2024-03-04T09:41:15.164 INFO:teuthology.orchestra.run.smithi052.stdout:} 2024-03-04T09:41:15.166 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:41:13 smithi106 ceph-mon[36484]: from='client.? 172.21.15.52:0/1656086961' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-04T09:41:15.167 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:41:13 smithi106 ceph-mon[36484]: from='client.? 172.21.15.52:0/2700761962' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-03-04T09:41:15.167 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:41:14 smithi106 ceph-mon[36484]: pgmap v312: 129 pgs: 2 active+clean+snaptrim, 7 active+clean+snaptrim_wait, 120 active+clean; 78 MiB data, 687 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-03-04T09:41:15.167 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:41:13 smithi191 ceph-mon[36816]: from='client.? 172.21.15.52:0/1656086961' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-04T09:41:15.167 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:41:13 smithi191 ceph-mon[36816]: from='client.? 172.21.15.52:0/2700761962' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-03-04T09:41:15.167 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:41:14 smithi191 ceph-mon[36816]: pgmap v312: 129 pgs: 2 active+clean+snaptrim, 7 active+clean+snaptrim_wait, 120 active+clean; 78 MiB data, 687 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-03-04T09:41:15.168 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:12.927+0000 7f945ab7a640 1 -- 172.21.15.52:0/2700761962 >> [v2:172.21.15.52:6800/478031757,v1:172.21.15.52:6801/478031757] conn(0x7f94440a1ef0 msgr2=0x7f94440a43b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:41:15.168 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:12.927+0000 7f945ab7a640 1 --2- 172.21.15.52:0/2700761962 >> [v2:172.21.15.52:6800/478031757,v1:172.21.15.52:6801/478031757] conn(0x7f94440a1ef0 0x7f94440a43b0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f9448009b50 tx=0x7f9448005f90 comp rx=0 tx=0).stop 2024-03-04T09:41:15.168 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:12.927+0000 7f945ab7a640 1 -- 172.21.15.52:0/2700761962 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9454101f40 msgr2=0x7f9454073e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:41:15.168 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:12.927+0000 7f945ab7a640 1 --2- 172.21.15.52:0/2700761962 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9454101f40 0x7f9454073e10 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f944000e9f0 tx=0x7f944000edf0 comp rx=0 tx=0).stop 2024-03-04T09:41:15.168 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:12.927+0000 7f945ab7a640 1 -- 172.21.15.52:0/2700761962 shutdown_connections 2024-03-04T09:41:15.168 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:12.927+0000 7f945ab7a640 1 --2- 172.21.15.52:0/2700761962 >> [v2:172.21.15.52:6800/478031757,v1:172.21.15.52:6801/478031757] conn(0x7f94440a1ef0 0x7f94440a43b0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:41:15.169 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:12.927+0000 7f945ab7a640 1 --2- 172.21.15.52:0/2700761962 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9454107640 0x7f9454074350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:41:15.169 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:12.927+0000 7f945ab7a640 1 --2- 172.21.15.52:0/2700761962 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9454101f40 0x7f9454073e10 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:41:15.169 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:12.927+0000 7f945ab7a640 1 --2- 172.21.15.52:0/2700761962 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9454101580 0x7f94540738d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:41:15.169 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:12.927+0000 7f945ab7a640 1 -- 172.21.15.52:0/2700761962 >> 172.21.15.52:0/2700761962 conn(0x7f94540fd070 msgr2=0x7f94540fe830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:41:15.169 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:12.927+0000 7f945ab7a640 1 -- 172.21.15.52:0/2700761962 shutdown_connections 2024-03-04T09:41:15.169 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:12.927+0000 7f945ab7a640 1 -- 172.21.15.52:0/2700761962 wait complete. 2024-03-04T09:41:15.175 DEBUG:tasks.check_counter:Getting stats from a 2024-03-04T09:41:15.175 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd shell --fsid 4e059f08-da08-11ee-95c6-87774f69a715 -- ceph daemon mds.a perf dump 2024-03-04T09:41:15.176 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:41:13 smithi052 ceph-mon[32473]: from='client.? 172.21.15.52:0/1656086961' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-04T09:41:15.176 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:41:13 smithi052 ceph-mon[32473]: from='client.? 172.21.15.52:0/2700761962' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-03-04T09:41:15.177 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:41:14 smithi052 ceph-mon[32473]: pgmap v312: 129 pgs: 2 active+clean+snaptrim, 7 active+clean+snaptrim_wait, 120 active+clean; 78 MiB data, 687 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-03-04T09:41:15.430 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/4e059f08-da08-11ee-95c6-87774f69a715/mon.a/config 2024-03-04T09:41:16.836 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:41:16 smithi052 ceph-mon[32473]: pgmap v313: 129 pgs: 129 active+clean; 78 MiB data, 691 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 9 op/s 2024-03-04T09:41:16.836 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:41:16 smithi052 ceph-mon[32473]: osdmap e102: 12 total, 12 up, 12 in 2024-03-04T09:41:16.837 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:41:16 smithi191 ceph-mon[36816]: pgmap v313: 129 pgs: 129 active+clean; 78 MiB data, 691 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 9 op/s 2024-03-04T09:41:16.837 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:41:16 smithi191 ceph-mon[36816]: osdmap e102: 12 total, 12 up, 12 in 2024-03-04T09:41:16.882 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:41:16 smithi106 ceph-mon[36484]: pgmap v313: 129 pgs: 129 active+clean; 78 MiB data, 691 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 9 op/s 2024-03-04T09:41:16.882 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:41:16 smithi106 ceph-mon[36484]: osdmap e102: 12 total, 12 up, 12 in 2024-03-04T09:41:17.353 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2024-03-04T09:41:17.353 INFO:teuthology.orchestra.run.smithi052.stdout: "AsyncMessenger::Worker-0": { 2024-03-04T09:41:17.353 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_recv_messages": 220, 2024-03-04T09:41:17.354 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_send_messages": 208, 2024-03-04T09:41:17.354 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_recv_bytes": 1433728, 2024-03-04T09:41:17.354 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_send_bytes": 123984, 2024-03-04T09:41:17.354 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_created_connections": 4, 2024-03-04T09:41:17.354 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_active_connections": 0, 2024-03-04T09:41:17.354 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_running_total_time": 0.043086248, 2024-03-04T09:41:17.354 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_running_send_time": 0.014048887, 2024-03-04T09:41:17.355 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_running_recv_time": 0.033343686, 2024-03-04T09:41:17.355 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_running_fast_dispatch_time": 0.004116952, 2024-03-04T09:41:17.355 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_send_messages_queue_lat": { 2024-03-04T09:41:17.355 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 208, 2024-03-04T09:41:17.355 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.011839281, 2024-03-04T09:41:17.355 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000056919 2024-03-04T09:41:17.355 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:17.355 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_handle_ack_lat": { 2024-03-04T09:41:17.356 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 0, 2024-03-04T09:41:17.356 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000000000, 2024-03-04T09:41:17.356 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000000 2024-03-04T09:41:17.356 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:17.356 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_recv_encrypted_bytes": 1433728, 2024-03-04T09:41:17.356 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_send_encrypted_bytes": 123984 2024-03-04T09:41:17.356 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:17.357 INFO:teuthology.orchestra.run.smithi052.stdout: "AsyncMessenger::Worker-1": { 2024-03-04T09:41:17.357 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_recv_messages": 2, 2024-03-04T09:41:17.357 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_send_messages": 158, 2024-03-04T09:41:17.357 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_recv_bytes": 320, 2024-03-04T09:41:17.357 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_send_bytes": 182832, 2024-03-04T09:41:17.357 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_created_connections": 5, 2024-03-04T09:41:17.357 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_active_connections": 3, 2024-03-04T09:41:17.357 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_running_total_time": 0.014452760, 2024-03-04T09:41:17.358 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_running_send_time": 0.010182184, 2024-03-04T09:41:17.359 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_running_recv_time": 0.001928606, 2024-03-04T09:41:17.360 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-03-04T09:41:17.360 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_send_messages_queue_lat": { 2024-03-04T09:41:17.360 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 155, 2024-03-04T09:41:17.360 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.010910681, 2024-03-04T09:41:17.360 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000070391 2024-03-04T09:41:17.360 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:17.360 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_handle_ack_lat": { 2024-03-04T09:41:17.361 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 0, 2024-03-04T09:41:17.361 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000000000, 2024-03-04T09:41:17.361 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000000 2024-03-04T09:41:17.361 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:17.361 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_recv_encrypted_bytes": 320, 2024-03-04T09:41:17.361 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_send_encrypted_bytes": 182832 2024-03-04T09:41:17.361 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:17.361 INFO:teuthology.orchestra.run.smithi052.stdout: "AsyncMessenger::Worker-2": { 2024-03-04T09:41:17.362 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_recv_messages": 0, 2024-03-04T09:41:17.362 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_send_messages": 2, 2024-03-04T09:41:17.362 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_recv_bytes": 0, 2024-03-04T09:41:17.362 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_send_bytes": 0, 2024-03-04T09:41:17.362 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_created_connections": 2, 2024-03-04T09:41:17.362 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_active_connections": 1, 2024-03-04T09:41:17.362 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_running_total_time": 0.000395003, 2024-03-04T09:41:17.363 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_running_send_time": 0.000000000, 2024-03-04T09:41:17.363 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_running_recv_time": 0.000324184, 2024-03-04T09:41:17.363 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-03-04T09:41:17.363 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_send_messages_queue_lat": { 2024-03-04T09:41:17.363 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 0, 2024-03-04T09:41:17.363 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000000000, 2024-03-04T09:41:17.363 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000000 2024-03-04T09:41:17.363 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:17.363 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_handle_ack_lat": { 2024-03-04T09:41:17.364 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 0, 2024-03-04T09:41:17.364 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000000000, 2024-03-04T09:41:17.364 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000000 2024-03-04T09:41:17.364 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:17.364 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_recv_encrypted_bytes": 0, 2024-03-04T09:41:17.364 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_send_encrypted_bytes": 0 2024-03-04T09:41:17.364 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:17.365 INFO:teuthology.orchestra.run.smithi052.stdout: "cct": { 2024-03-04T09:41:17.365 INFO:teuthology.orchestra.run.smithi052.stdout: "total_workers": 0, 2024-03-04T09:41:17.365 INFO:teuthology.orchestra.run.smithi052.stdout: "unhealthy_workers": 0 2024-03-04T09:41:17.365 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:17.365 INFO:teuthology.orchestra.run.smithi052.stdout: "mempool": { 2024-03-04T09:41:17.365 INFO:teuthology.orchestra.run.smithi052.stdout: "bloom_filter_bytes": 0, 2024-03-04T09:41:17.365 INFO:teuthology.orchestra.run.smithi052.stdout: "bloom_filter_items": 0, 2024-03-04T09:41:17.366 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_alloc_bytes": 0, 2024-03-04T09:41:17.366 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_alloc_items": 0, 2024-03-04T09:41:17.366 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_cache_data_bytes": 0, 2024-03-04T09:41:17.366 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_cache_data_items": 0, 2024-03-04T09:41:17.366 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-04T09:41:17.366 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_cache_onode_items": 0, 2024-03-04T09:41:17.366 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-04T09:41:17.367 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_cache_meta_items": 0, 2024-03-04T09:41:17.367 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_cache_other_bytes": 0, 2024-03-04T09:41:17.367 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_cache_other_items": 0, 2024-03-04T09:41:17.367 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-04T09:41:17.367 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_cache_buffer_items": 0, 2024-03-04T09:41:17.367 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_extent_bytes": 0, 2024-03-04T09:41:17.367 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_extent_items": 0, 2024-03-04T09:41:17.368 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_blob_bytes": 0, 2024-03-04T09:41:17.368 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_blob_items": 0, 2024-03-04T09:41:17.368 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-04T09:41:17.368 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_shared_blob_items": 0, 2024-03-04T09:41:17.368 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-04T09:41:17.368 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_inline_bl_items": 0, 2024-03-04T09:41:17.368 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_fsck_bytes": 0, 2024-03-04T09:41:17.369 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_fsck_items": 0, 2024-03-04T09:41:17.369 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_txc_bytes": 0, 2024-03-04T09:41:17.369 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_txc_items": 0, 2024-03-04T09:41:17.369 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-04T09:41:17.369 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_writing_deferred_items": 0, 2024-03-04T09:41:17.369 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_writing_bytes": 0, 2024-03-04T09:41:17.369 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_writing_items": 0, 2024-03-04T09:41:17.369 INFO:teuthology.orchestra.run.smithi052.stdout: "bluefs_bytes": 0, 2024-03-04T09:41:17.370 INFO:teuthology.orchestra.run.smithi052.stdout: "bluefs_items": 0, 2024-03-04T09:41:17.370 INFO:teuthology.orchestra.run.smithi052.stdout: "bluefs_file_reader_bytes": 0, 2024-03-04T09:41:17.370 INFO:teuthology.orchestra.run.smithi052.stdout: "bluefs_file_reader_items": 0, 2024-03-04T09:41:17.370 INFO:teuthology.orchestra.run.smithi052.stdout: "bluefs_file_writer_bytes": 0, 2024-03-04T09:41:17.370 INFO:teuthology.orchestra.run.smithi052.stdout: "bluefs_file_writer_items": 0, 2024-03-04T09:41:17.370 INFO:teuthology.orchestra.run.smithi052.stdout: "buffer_anon_bytes": 149017, 2024-03-04T09:41:17.370 INFO:teuthology.orchestra.run.smithi052.stdout: "buffer_anon_items": 31, 2024-03-04T09:41:17.371 INFO:teuthology.orchestra.run.smithi052.stdout: "buffer_meta_bytes": 0, 2024-03-04T09:41:17.371 INFO:teuthology.orchestra.run.smithi052.stdout: "buffer_meta_items": 0, 2024-03-04T09:41:17.371 INFO:teuthology.orchestra.run.smithi052.stdout: "osd_bytes": 0, 2024-03-04T09:41:17.371 INFO:teuthology.orchestra.run.smithi052.stdout: "osd_items": 0, 2024-03-04T09:41:17.371 INFO:teuthology.orchestra.run.smithi052.stdout: "osd_mapbl_bytes": 0, 2024-03-04T09:41:17.371 INFO:teuthology.orchestra.run.smithi052.stdout: "osd_mapbl_items": 0, 2024-03-04T09:41:17.372 INFO:teuthology.orchestra.run.smithi052.stdout: "osd_pglog_bytes": 0, 2024-03-04T09:41:17.372 INFO:teuthology.orchestra.run.smithi052.stdout: "osd_pglog_items": 0, 2024-03-04T09:41:17.372 INFO:teuthology.orchestra.run.smithi052.stdout: "osdmap_bytes": 0, 2024-03-04T09:41:17.372 INFO:teuthology.orchestra.run.smithi052.stdout: "osdmap_items": 0, 2024-03-04T09:41:17.372 INFO:teuthology.orchestra.run.smithi052.stdout: "osdmap_mapping_bytes": 0, 2024-03-04T09:41:17.372 INFO:teuthology.orchestra.run.smithi052.stdout: "osdmap_mapping_items": 0, 2024-03-04T09:41:17.372 INFO:teuthology.orchestra.run.smithi052.stdout: "pgmap_bytes": 0, 2024-03-04T09:41:17.373 INFO:teuthology.orchestra.run.smithi052.stdout: "pgmap_items": 0, 2024-03-04T09:41:17.373 INFO:teuthology.orchestra.run.smithi052.stdout: "mds_co_bytes": 928, 2024-03-04T09:41:17.373 INFO:teuthology.orchestra.run.smithi052.stdout: "mds_co_items": 2, 2024-03-04T09:41:17.373 INFO:teuthology.orchestra.run.smithi052.stdout: "unittest_1_bytes": 0, 2024-03-04T09:41:17.373 INFO:teuthology.orchestra.run.smithi052.stdout: "unittest_1_items": 0, 2024-03-04T09:41:17.373 INFO:teuthology.orchestra.run.smithi052.stdout: "unittest_2_bytes": 0, 2024-03-04T09:41:17.373 INFO:teuthology.orchestra.run.smithi052.stdout: "unittest_2_items": 0 2024-03-04T09:41:17.373 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:17.374 INFO:teuthology.orchestra.run.smithi052.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-04T09:41:17.374 INFO:teuthology.orchestra.run.smithi052.stdout: "val": 0, 2024-03-04T09:41:17.374 INFO:teuthology.orchestra.run.smithi052.stdout: "max": 104857600, 2024-03-04T09:41:17.374 INFO:teuthology.orchestra.run.smithi052.stdout: "get_started": 0, 2024-03-04T09:41:17.374 INFO:teuthology.orchestra.run.smithi052.stdout: "get": 218, 2024-03-04T09:41:17.374 INFO:teuthology.orchestra.run.smithi052.stdout: "get_sum": 1400254, 2024-03-04T09:41:17.374 INFO:teuthology.orchestra.run.smithi052.stdout: "get_or_fail_fail": 0, 2024-03-04T09:41:17.375 INFO:teuthology.orchestra.run.smithi052.stdout: "get_or_fail_success": 218, 2024-03-04T09:41:17.375 INFO:teuthology.orchestra.run.smithi052.stdout: "take": 0, 2024-03-04T09:41:17.375 INFO:teuthology.orchestra.run.smithi052.stdout: "take_sum": 0, 2024-03-04T09:41:17.375 INFO:teuthology.orchestra.run.smithi052.stdout: "put": 218, 2024-03-04T09:41:17.375 INFO:teuthology.orchestra.run.smithi052.stdout: "put_sum": 1400254, 2024-03-04T09:41:17.375 INFO:teuthology.orchestra.run.smithi052.stdout: "wait": { 2024-03-04T09:41:17.375 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 0, 2024-03-04T09:41:17.375 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000000000, 2024-03-04T09:41:17.376 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000000 2024-03-04T09:41:17.376 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-03-04T09:41:17.376 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-03-04T09:41:17.376 INFO:teuthology.orchestra.run.smithi052.stdout:} 2024-03-04T09:41:17.952 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.a 2024-03-04T09:41:17.952 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.a 2024-03-04T09:41:17.952 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.a 2024-03-04T09:41:17.952 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.a 2024-03-04T09:41:17.953 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.a 2024-03-04T09:41:17.953 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.a 2024-03-04T09:41:17.953 DEBUG:tasks.check_counter:Getting stats from d 2024-03-04T09:41:17.953 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd shell --fsid 4e059f08-da08-11ee-95c6-87774f69a715 -- ceph daemon mds.d perf dump 2024-03-04T09:41:18.207 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/4e059f08-da08-11ee-95c6-87774f69a715/mon.a/config 2024-03-04T09:41:18.586 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:41:18 smithi052 ceph-mon[32473]: pgmap v315: 129 pgs: 129 active+clean; 78 MiB data, 691 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 255 B/s wr, 11 op/s 2024-03-04T09:41:18.837 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:41:18 smithi191 ceph-mon[36816]: pgmap v315: 129 pgs: 129 active+clean; 78 MiB data, 691 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 255 B/s wr, 11 op/s 2024-03-04T09:41:18.882 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:41:18 smithi106 ceph-mon[36484]: pgmap v315: 129 pgs: 129 active+clean; 78 MiB data, 691 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 255 B/s wr, 11 op/s 2024-03-04T09:41:20.089 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2024-03-04T09:41:20.089 INFO:teuthology.orchestra.run.smithi052.stdout: "AsyncMessenger::Worker-0": { 2024-03-04T09:41:20.089 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_recv_messages": 0, 2024-03-04T09:41:20.089 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_send_messages": 2, 2024-03-04T09:41:20.089 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_recv_bytes": 0, 2024-03-04T09:41:20.089 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_send_bytes": 0, 2024-03-04T09:41:20.090 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_created_connections": 4, 2024-03-04T09:41:20.090 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_active_connections": 0, 2024-03-04T09:41:20.090 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_running_total_time": 0.001230756, 2024-03-04T09:41:20.090 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_running_send_time": 0.000000000, 2024-03-04T09:41:20.090 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_running_recv_time": 0.000307351, 2024-03-04T09:41:20.090 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-03-04T09:41:20.090 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_send_messages_queue_lat": { 2024-03-04T09:41:20.091 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 0, 2024-03-04T09:41:20.091 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000000000, 2024-03-04T09:41:20.091 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000000 2024-03-04T09:41:20.091 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:20.091 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_handle_ack_lat": { 2024-03-04T09:41:20.091 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 0, 2024-03-04T09:41:20.091 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000000000, 2024-03-04T09:41:20.091 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000000 2024-03-04T09:41:20.091 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:20.092 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_recv_encrypted_bytes": 0, 2024-03-04T09:41:20.092 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_send_encrypted_bytes": 0 2024-03-04T09:41:20.092 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:20.092 INFO:teuthology.orchestra.run.smithi052.stdout: "AsyncMessenger::Worker-1": { 2024-03-04T09:41:20.092 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_recv_messages": 6, 2024-03-04T09:41:20.092 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_send_messages": 165, 2024-03-04T09:41:20.092 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_recv_bytes": 3344, 2024-03-04T09:41:20.092 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_send_bytes": 184160, 2024-03-04T09:41:20.094 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_created_connections": 5, 2024-03-04T09:41:20.094 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_active_connections": 3, 2024-03-04T09:41:20.094 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_running_total_time": 0.013910832, 2024-03-04T09:41:20.094 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_running_send_time": 0.009433192, 2024-03-04T09:41:20.094 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_running_recv_time": 0.002379867, 2024-03-04T09:41:20.094 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-03-04T09:41:20.094 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_send_messages_queue_lat": { 2024-03-04T09:41:20.095 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 163, 2024-03-04T09:41:20.095 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.012593329, 2024-03-04T09:41:20.095 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000077259 2024-03-04T09:41:20.095 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:20.095 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_handle_ack_lat": { 2024-03-04T09:41:20.095 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 0, 2024-03-04T09:41:20.095 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000000000, 2024-03-04T09:41:20.095 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000000 2024-03-04T09:41:20.095 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:20.096 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_recv_encrypted_bytes": 3344, 2024-03-04T09:41:20.096 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_send_encrypted_bytes": 184160 2024-03-04T09:41:20.096 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:20.096 INFO:teuthology.orchestra.run.smithi052.stdout: "AsyncMessenger::Worker-2": { 2024-03-04T09:41:20.096 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_recv_messages": 222, 2024-03-04T09:41:20.096 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_send_messages": 212, 2024-03-04T09:41:20.097 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_recv_bytes": 1432336, 2024-03-04T09:41:20.097 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_send_bytes": 127280, 2024-03-04T09:41:20.097 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_created_connections": 2, 2024-03-04T09:41:20.097 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_active_connections": 1, 2024-03-04T09:41:20.097 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_running_total_time": 0.044028380, 2024-03-04T09:41:20.097 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_running_send_time": 0.015194541, 2024-03-04T09:41:20.098 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_running_recv_time": 0.032156663, 2024-03-04T09:41:20.098 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_running_fast_dispatch_time": 0.004406288, 2024-03-04T09:41:20.098 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_send_messages_queue_lat": { 2024-03-04T09:41:20.098 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 211, 2024-03-04T09:41:20.098 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.011083367, 2024-03-04T09:41:20.098 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000052527 2024-03-04T09:41:20.098 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:20.098 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_handle_ack_lat": { 2024-03-04T09:41:20.099 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 0, 2024-03-04T09:41:20.099 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000000000, 2024-03-04T09:41:20.099 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000000 2024-03-04T09:41:20.099 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:20.099 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_recv_encrypted_bytes": 1432336, 2024-03-04T09:41:20.099 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_send_encrypted_bytes": 127280 2024-03-04T09:41:20.099 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:20.099 INFO:teuthology.orchestra.run.smithi052.stdout: "cct": { 2024-03-04T09:41:20.099 INFO:teuthology.orchestra.run.smithi052.stdout: "total_workers": 0, 2024-03-04T09:41:20.100 INFO:teuthology.orchestra.run.smithi052.stdout: "unhealthy_workers": 0 2024-03-04T09:41:20.100 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:20.100 INFO:teuthology.orchestra.run.smithi052.stdout: "mempool": { 2024-03-04T09:41:20.100 INFO:teuthology.orchestra.run.smithi052.stdout: "bloom_filter_bytes": 0, 2024-03-04T09:41:20.100 INFO:teuthology.orchestra.run.smithi052.stdout: "bloom_filter_items": 0, 2024-03-04T09:41:20.100 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_alloc_bytes": 0, 2024-03-04T09:41:20.100 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_alloc_items": 0, 2024-03-04T09:41:20.100 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_cache_data_bytes": 0, 2024-03-04T09:41:20.101 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_cache_data_items": 0, 2024-03-04T09:41:20.101 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-04T09:41:20.101 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_cache_onode_items": 0, 2024-03-04T09:41:20.101 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-04T09:41:20.101 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_cache_meta_items": 0, 2024-03-04T09:41:20.101 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_cache_other_bytes": 0, 2024-03-04T09:41:20.102 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_cache_other_items": 0, 2024-03-04T09:41:20.102 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-04T09:41:20.102 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_cache_buffer_items": 0, 2024-03-04T09:41:20.102 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_extent_bytes": 0, 2024-03-04T09:41:20.102 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_extent_items": 0, 2024-03-04T09:41:20.102 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_blob_bytes": 0, 2024-03-04T09:41:20.102 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_blob_items": 0, 2024-03-04T09:41:20.102 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-04T09:41:20.103 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_shared_blob_items": 0, 2024-03-04T09:41:20.103 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-04T09:41:20.103 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_inline_bl_items": 0, 2024-03-04T09:41:20.103 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_fsck_bytes": 0, 2024-03-04T09:41:20.103 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_fsck_items": 0, 2024-03-04T09:41:20.103 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_txc_bytes": 0, 2024-03-04T09:41:20.103 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_txc_items": 0, 2024-03-04T09:41:20.103 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-04T09:41:20.104 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_writing_deferred_items": 0, 2024-03-04T09:41:20.104 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_writing_bytes": 0, 2024-03-04T09:41:20.104 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_writing_items": 0, 2024-03-04T09:41:20.104 INFO:teuthology.orchestra.run.smithi052.stdout: "bluefs_bytes": 0, 2024-03-04T09:41:20.104 INFO:teuthology.orchestra.run.smithi052.stdout: "bluefs_items": 0, 2024-03-04T09:41:20.104 INFO:teuthology.orchestra.run.smithi052.stdout: "bluefs_file_reader_bytes": 0, 2024-03-04T09:41:20.104 INFO:teuthology.orchestra.run.smithi052.stdout: "bluefs_file_reader_items": 0, 2024-03-04T09:41:20.104 INFO:teuthology.orchestra.run.smithi052.stdout: "bluefs_file_writer_bytes": 0, 2024-03-04T09:41:20.105 INFO:teuthology.orchestra.run.smithi052.stdout: "bluefs_file_writer_items": 0, 2024-03-04T09:41:20.105 INFO:teuthology.orchestra.run.smithi052.stdout: "buffer_anon_bytes": 149017, 2024-03-04T09:41:20.105 INFO:teuthology.orchestra.run.smithi052.stdout: "buffer_anon_items": 31, 2024-03-04T09:41:20.105 INFO:teuthology.orchestra.run.smithi052.stdout: "buffer_meta_bytes": 0, 2024-03-04T09:41:20.105 INFO:teuthology.orchestra.run.smithi052.stdout: "buffer_meta_items": 0, 2024-03-04T09:41:20.105 INFO:teuthology.orchestra.run.smithi052.stdout: "osd_bytes": 0, 2024-03-04T09:41:20.105 INFO:teuthology.orchestra.run.smithi052.stdout: "osd_items": 0, 2024-03-04T09:41:20.105 INFO:teuthology.orchestra.run.smithi052.stdout: "osd_mapbl_bytes": 0, 2024-03-04T09:41:20.105 INFO:teuthology.orchestra.run.smithi052.stdout: "osd_mapbl_items": 0, 2024-03-04T09:41:20.106 INFO:teuthology.orchestra.run.smithi052.stdout: "osd_pglog_bytes": 0, 2024-03-04T09:41:20.106 INFO:teuthology.orchestra.run.smithi052.stdout: "osd_pglog_items": 0, 2024-03-04T09:41:20.106 INFO:teuthology.orchestra.run.smithi052.stdout: "osdmap_bytes": 0, 2024-03-04T09:41:20.106 INFO:teuthology.orchestra.run.smithi052.stdout: "osdmap_items": 0, 2024-03-04T09:41:20.106 INFO:teuthology.orchestra.run.smithi052.stdout: "osdmap_mapping_bytes": 0, 2024-03-04T09:41:20.106 INFO:teuthology.orchestra.run.smithi052.stdout: "osdmap_mapping_items": 0, 2024-03-04T09:41:20.106 INFO:teuthology.orchestra.run.smithi052.stdout: "pgmap_bytes": 0, 2024-03-04T09:41:20.106 INFO:teuthology.orchestra.run.smithi052.stdout: "pgmap_items": 0, 2024-03-04T09:41:20.107 INFO:teuthology.orchestra.run.smithi052.stdout: "mds_co_bytes": 928, 2024-03-04T09:41:20.107 INFO:teuthology.orchestra.run.smithi052.stdout: "mds_co_items": 2, 2024-03-04T09:41:20.107 INFO:teuthology.orchestra.run.smithi052.stdout: "unittest_1_bytes": 0, 2024-03-04T09:41:20.107 INFO:teuthology.orchestra.run.smithi052.stdout: "unittest_1_items": 0, 2024-03-04T09:41:20.107 INFO:teuthology.orchestra.run.smithi052.stdout: "unittest_2_bytes": 0, 2024-03-04T09:41:20.107 INFO:teuthology.orchestra.run.smithi052.stdout: "unittest_2_items": 0 2024-03-04T09:41:20.107 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:20.107 INFO:teuthology.orchestra.run.smithi052.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-04T09:41:20.107 INFO:teuthology.orchestra.run.smithi052.stdout: "val": 0, 2024-03-04T09:41:20.108 INFO:teuthology.orchestra.run.smithi052.stdout: "max": 104857600, 2024-03-04T09:41:20.108 INFO:teuthology.orchestra.run.smithi052.stdout: "get_started": 0, 2024-03-04T09:41:20.108 INFO:teuthology.orchestra.run.smithi052.stdout: "get": 224, 2024-03-04T09:41:20.108 INFO:teuthology.orchestra.run.smithi052.stdout: "get_sum": 1401034, 2024-03-04T09:41:20.108 INFO:teuthology.orchestra.run.smithi052.stdout: "get_or_fail_fail": 0, 2024-03-04T09:41:20.108 INFO:teuthology.orchestra.run.smithi052.stdout: "get_or_fail_success": 224, 2024-03-04T09:41:20.108 INFO:teuthology.orchestra.run.smithi052.stdout: "take": 0, 2024-03-04T09:41:20.109 INFO:teuthology.orchestra.run.smithi052.stdout: "take_sum": 0, 2024-03-04T09:41:20.109 INFO:teuthology.orchestra.run.smithi052.stdout: "put": 224, 2024-03-04T09:41:20.109 INFO:teuthology.orchestra.run.smithi052.stdout: "put_sum": 1401034, 2024-03-04T09:41:20.109 INFO:teuthology.orchestra.run.smithi052.stdout: "wait": { 2024-03-04T09:41:20.109 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 0, 2024-03-04T09:41:20.109 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000000000, 2024-03-04T09:41:20.109 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000000 2024-03-04T09:41:20.109 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-03-04T09:41:20.109 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-03-04T09:41:20.110 INFO:teuthology.orchestra.run.smithi052.stdout:} 2024-03-04T09:41:20.835 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:41:20 smithi052 ceph-mon[32473]: pgmap v316: 129 pgs: 129 active+clean; 78 MiB data, 691 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 223 B/s wr, 10 op/s 2024-03-04T09:41:20.837 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:41:20 smithi191 ceph-mon[36816]: pgmap v316: 129 pgs: 129 active+clean; 78 MiB data, 691 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 223 B/s wr, 10 op/s 2024-03-04T09:41:20.882 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:41:20 smithi106 ceph-mon[36484]: pgmap v316: 129 pgs: 129 active+clean; 78 MiB data, 691 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 223 B/s wr, 10 op/s 2024-03-04T09:41:20.996 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.d 2024-03-04T09:41:20.996 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.d 2024-03-04T09:41:20.996 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.d 2024-03-04T09:41:20.996 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.d 2024-03-04T09:41:20.996 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.d 2024-03-04T09:41:20.996 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.d 2024-03-04T09:41:20.997 DEBUG:tasks.check_counter:Getting stats from g 2024-03-04T09:41:20.997 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd shell --fsid 4e059f08-da08-11ee-95c6-87774f69a715 -- ceph daemon mds.g perf dump 2024-03-04T09:41:21.252 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/4e059f08-da08-11ee-95c6-87774f69a715/mon.a/config 2024-03-04T09:41:21.639 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:41:21 smithi052 ceph-mon[32473]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T09:41:21.837 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:41:21 smithi191 ceph-mon[36816]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T09:41:21.882 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:41:21 smithi106 ceph-mon[36484]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T09:41:22.806 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:41:22 smithi052 ceph-mon[32473]: pgmap v317: 129 pgs: 129 active+clean; 78 MiB data, 691 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 9 op/s 2024-03-04T09:41:22.837 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:41:22 smithi191 ceph-mon[36816]: pgmap v317: 129 pgs: 129 active+clean; 78 MiB data, 691 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 9 op/s 2024-03-04T09:41:22.882 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:41:22 smithi106 ceph-mon[36484]: pgmap v317: 129 pgs: 129 active+clean; 78 MiB data, 691 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 9 op/s 2024-03-04T09:41:23.080 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2024-03-04T09:41:23.081 INFO:teuthology.orchestra.run.smithi052.stdout: "AsyncMessenger::Worker-0": { 2024-03-04T09:41:23.081 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_recv_messages": 0, 2024-03-04T09:41:23.081 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_send_messages": 2, 2024-03-04T09:41:23.081 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_recv_bytes": 0, 2024-03-04T09:41:23.081 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_send_bytes": 0, 2024-03-04T09:41:23.081 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_created_connections": 4, 2024-03-04T09:41:23.081 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_active_connections": 0, 2024-03-04T09:41:23.081 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_running_total_time": 0.001191845, 2024-03-04T09:41:23.082 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_running_send_time": 0.000000000, 2024-03-04T09:41:23.082 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_running_recv_time": 0.000276050, 2024-03-04T09:41:23.082 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-03-04T09:41:23.082 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_send_messages_queue_lat": { 2024-03-04T09:41:23.082 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 0, 2024-03-04T09:41:23.082 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000000000, 2024-03-04T09:41:23.082 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000000 2024-03-04T09:41:23.083 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:23.083 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_handle_ack_lat": { 2024-03-04T09:41:23.083 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 0, 2024-03-04T09:41:23.083 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000000000, 2024-03-04T09:41:23.083 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000000 2024-03-04T09:41:23.083 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:23.083 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_recv_encrypted_bytes": 0, 2024-03-04T09:41:23.084 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_send_encrypted_bytes": 0 2024-03-04T09:41:23.084 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:23.084 INFO:teuthology.orchestra.run.smithi052.stdout: "AsyncMessenger::Worker-1": { 2024-03-04T09:41:23.084 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_recv_messages": 223, 2024-03-04T09:41:23.084 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_send_messages": 211, 2024-03-04T09:41:23.084 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_recv_bytes": 1434608, 2024-03-04T09:41:23.084 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_send_bytes": 125392, 2024-03-04T09:41:23.084 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_created_connections": 2, 2024-03-04T09:41:23.085 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_active_connections": 1, 2024-03-04T09:41:23.085 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_running_total_time": 0.042139729, 2024-03-04T09:41:23.085 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_running_send_time": 0.014167684, 2024-03-04T09:41:23.085 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_running_recv_time": 0.031932970, 2024-03-04T09:41:23.085 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_running_fast_dispatch_time": 0.004143441, 2024-03-04T09:41:23.085 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_send_messages_queue_lat": { 2024-03-04T09:41:23.085 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 211, 2024-03-04T09:41:23.086 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.011231307, 2024-03-04T09:41:23.086 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000053228 2024-03-04T09:41:23.086 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:23.086 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_handle_ack_lat": { 2024-03-04T09:41:23.086 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 0, 2024-03-04T09:41:23.086 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000000000, 2024-03-04T09:41:23.086 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000000 2024-03-04T09:41:23.086 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:23.087 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_recv_encrypted_bytes": 1434608, 2024-03-04T09:41:23.087 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_send_encrypted_bytes": 125392 2024-03-04T09:41:23.087 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:23.087 INFO:teuthology.orchestra.run.smithi052.stdout: "AsyncMessenger::Worker-2": { 2024-03-04T09:41:23.087 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_recv_messages": 2, 2024-03-04T09:41:23.087 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_send_messages": 159, 2024-03-04T09:41:23.087 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_recv_bytes": 320, 2024-03-04T09:41:23.087 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_send_bytes": 183008, 2024-03-04T09:41:23.088 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_created_connections": 5, 2024-03-04T09:41:23.088 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_active_connections": 3, 2024-03-04T09:41:23.088 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_running_total_time": 0.013502482, 2024-03-04T09:41:23.088 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_running_send_time": 0.009408178, 2024-03-04T09:41:23.088 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_running_recv_time": 0.001765874, 2024-03-04T09:41:23.088 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-03-04T09:41:23.088 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_send_messages_queue_lat": { 2024-03-04T09:41:23.091 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 156, 2024-03-04T09:41:23.092 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.013827010, 2024-03-04T09:41:23.092 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000088634 2024-03-04T09:41:23.092 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:23.092 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_handle_ack_lat": { 2024-03-04T09:41:23.092 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 0, 2024-03-04T09:41:23.092 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000000000, 2024-03-04T09:41:23.093 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000000 2024-03-04T09:41:23.093 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:23.093 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_recv_encrypted_bytes": 320, 2024-03-04T09:41:23.093 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_send_encrypted_bytes": 183008 2024-03-04T09:41:23.093 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:23.093 INFO:teuthology.orchestra.run.smithi052.stdout: "cct": { 2024-03-04T09:41:23.093 INFO:teuthology.orchestra.run.smithi052.stdout: "total_workers": 0, 2024-03-04T09:41:23.094 INFO:teuthology.orchestra.run.smithi052.stdout: "unhealthy_workers": 0 2024-03-04T09:41:23.094 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:23.094 INFO:teuthology.orchestra.run.smithi052.stdout: "mempool": { 2024-03-04T09:41:23.094 INFO:teuthology.orchestra.run.smithi052.stdout: "bloom_filter_bytes": 0, 2024-03-04T09:41:23.094 INFO:teuthology.orchestra.run.smithi052.stdout: "bloom_filter_items": 0, 2024-03-04T09:41:23.094 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_alloc_bytes": 0, 2024-03-04T09:41:23.094 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_alloc_items": 0, 2024-03-04T09:41:23.094 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_cache_data_bytes": 0, 2024-03-04T09:41:23.095 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_cache_data_items": 0, 2024-03-04T09:41:23.095 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-04T09:41:23.095 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_cache_onode_items": 0, 2024-03-04T09:41:23.095 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-04T09:41:23.095 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_cache_meta_items": 0, 2024-03-04T09:41:23.095 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_cache_other_bytes": 0, 2024-03-04T09:41:23.095 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_cache_other_items": 0, 2024-03-04T09:41:23.095 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-04T09:41:23.096 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_cache_buffer_items": 0, 2024-03-04T09:41:23.096 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_extent_bytes": 0, 2024-03-04T09:41:23.096 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_extent_items": 0, 2024-03-04T09:41:23.096 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_blob_bytes": 0, 2024-03-04T09:41:23.096 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_blob_items": 0, 2024-03-04T09:41:23.096 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-04T09:41:23.096 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_shared_blob_items": 0, 2024-03-04T09:41:23.097 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-04T09:41:23.097 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_inline_bl_items": 0, 2024-03-04T09:41:23.097 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_fsck_bytes": 0, 2024-03-04T09:41:23.097 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_fsck_items": 0, 2024-03-04T09:41:23.097 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_txc_bytes": 0, 2024-03-04T09:41:23.097 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_txc_items": 0, 2024-03-04T09:41:23.097 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-04T09:41:23.098 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_writing_deferred_items": 0, 2024-03-04T09:41:23.098 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_writing_bytes": 0, 2024-03-04T09:41:23.098 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_writing_items": 0, 2024-03-04T09:41:23.098 INFO:teuthology.orchestra.run.smithi052.stdout: "bluefs_bytes": 0, 2024-03-04T09:41:23.098 INFO:teuthology.orchestra.run.smithi052.stdout: "bluefs_items": 0, 2024-03-04T09:41:23.098 INFO:teuthology.orchestra.run.smithi052.stdout: "bluefs_file_reader_bytes": 0, 2024-03-04T09:41:23.098 INFO:teuthology.orchestra.run.smithi052.stdout: "bluefs_file_reader_items": 0, 2024-03-04T09:41:23.099 INFO:teuthology.orchestra.run.smithi052.stdout: "bluefs_file_writer_bytes": 0, 2024-03-04T09:41:23.099 INFO:teuthology.orchestra.run.smithi052.stdout: "bluefs_file_writer_items": 0, 2024-03-04T09:41:23.099 INFO:teuthology.orchestra.run.smithi052.stdout: "buffer_anon_bytes": 148933, 2024-03-04T09:41:23.099 INFO:teuthology.orchestra.run.smithi052.stdout: "buffer_anon_items": 30, 2024-03-04T09:41:23.099 INFO:teuthology.orchestra.run.smithi052.stdout: "buffer_meta_bytes": 0, 2024-03-04T09:41:23.099 INFO:teuthology.orchestra.run.smithi052.stdout: "buffer_meta_items": 0, 2024-03-04T09:41:23.099 INFO:teuthology.orchestra.run.smithi052.stdout: "osd_bytes": 0, 2024-03-04T09:41:23.099 INFO:teuthology.orchestra.run.smithi052.stdout: "osd_items": 0, 2024-03-04T09:41:23.100 INFO:teuthology.orchestra.run.smithi052.stdout: "osd_mapbl_bytes": 0, 2024-03-04T09:41:23.100 INFO:teuthology.orchestra.run.smithi052.stdout: "osd_mapbl_items": 0, 2024-03-04T09:41:23.100 INFO:teuthology.orchestra.run.smithi052.stdout: "osd_pglog_bytes": 0, 2024-03-04T09:41:23.100 INFO:teuthology.orchestra.run.smithi052.stdout: "osd_pglog_items": 0, 2024-03-04T09:41:23.100 INFO:teuthology.orchestra.run.smithi052.stdout: "osdmap_bytes": 0, 2024-03-04T09:41:23.100 INFO:teuthology.orchestra.run.smithi052.stdout: "osdmap_items": 0, 2024-03-04T09:41:23.100 INFO:teuthology.orchestra.run.smithi052.stdout: "osdmap_mapping_bytes": 0, 2024-03-04T09:41:23.101 INFO:teuthology.orchestra.run.smithi052.stdout: "osdmap_mapping_items": 0, 2024-03-04T09:41:23.101 INFO:teuthology.orchestra.run.smithi052.stdout: "pgmap_bytes": 0, 2024-03-04T09:41:23.101 INFO:teuthology.orchestra.run.smithi052.stdout: "pgmap_items": 0, 2024-03-04T09:41:23.101 INFO:teuthology.orchestra.run.smithi052.stdout: "mds_co_bytes": 928, 2024-03-04T09:41:23.101 INFO:teuthology.orchestra.run.smithi052.stdout: "mds_co_items": 2, 2024-03-04T09:41:23.101 INFO:teuthology.orchestra.run.smithi052.stdout: "unittest_1_bytes": 0, 2024-03-04T09:41:23.101 INFO:teuthology.orchestra.run.smithi052.stdout: "unittest_1_items": 0, 2024-03-04T09:41:23.102 INFO:teuthology.orchestra.run.smithi052.stdout: "unittest_2_bytes": 0, 2024-03-04T09:41:23.102 INFO:teuthology.orchestra.run.smithi052.stdout: "unittest_2_items": 0 2024-03-04T09:41:23.102 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:23.102 INFO:teuthology.orchestra.run.smithi052.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-04T09:41:23.102 INFO:teuthology.orchestra.run.smithi052.stdout: "val": 0, 2024-03-04T09:41:23.102 INFO:teuthology.orchestra.run.smithi052.stdout: "max": 104857600, 2024-03-04T09:41:23.102 INFO:teuthology.orchestra.run.smithi052.stdout: "get_started": 0, 2024-03-04T09:41:23.103 INFO:teuthology.orchestra.run.smithi052.stdout: "get": 221, 2024-03-04T09:41:23.103 INFO:teuthology.orchestra.run.smithi052.stdout: "get_sum": 1400708, 2024-03-04T09:41:23.103 INFO:teuthology.orchestra.run.smithi052.stdout: "get_or_fail_fail": 0, 2024-03-04T09:41:23.103 INFO:teuthology.orchestra.run.smithi052.stdout: "get_or_fail_success": 221, 2024-03-04T09:41:23.103 INFO:teuthology.orchestra.run.smithi052.stdout: "take": 0, 2024-03-04T09:41:23.103 INFO:teuthology.orchestra.run.smithi052.stdout: "take_sum": 0, 2024-03-04T09:41:23.103 INFO:teuthology.orchestra.run.smithi052.stdout: "put": 221, 2024-03-04T09:41:23.104 INFO:teuthology.orchestra.run.smithi052.stdout: "put_sum": 1400708, 2024-03-04T09:41:23.104 INFO:teuthology.orchestra.run.smithi052.stdout: "wait": { 2024-03-04T09:41:23.104 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 0, 2024-03-04T09:41:23.104 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000000000, 2024-03-04T09:41:23.104 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000000 2024-03-04T09:41:23.104 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-03-04T09:41:23.104 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-03-04T09:41:23.105 INFO:teuthology.orchestra.run.smithi052.stdout:} 2024-03-04T09:41:23.752 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.g 2024-03-04T09:41:23.752 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.g 2024-03-04T09:41:23.752 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.g 2024-03-04T09:41:23.752 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.g 2024-03-04T09:41:23.752 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.g 2024-03-04T09:41:23.752 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.g 2024-03-04T09:41:23.753 DEBUG:tasks.check_counter:Getting stats from j 2024-03-04T09:41:23.753 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd shell --fsid 4e059f08-da08-11ee-95c6-87774f69a715 -- ceph daemon mds.j perf dump 2024-03-04T09:41:24.057 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:41:23 smithi052 ceph-mon[32473]: pgmap v318: 129 pgs: 129 active+clean; 78 MiB data, 691 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 9 op/s 2024-03-04T09:41:24.057 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:41:23 smithi052 ceph-mon[32473]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' 2024-03-04T09:41:24.057 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:41:23 smithi052 ceph-mon[32473]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' 2024-03-04T09:41:24.058 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:41:23 smithi052 ceph-mon[32473]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' 2024-03-04T09:41:24.058 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:41:23 smithi052 ceph-mon[32473]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' 2024-03-04T09:41:24.058 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:41:23 smithi052 ceph-mon[32473]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-04T09:41:24.058 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:41:23 smithi052 ceph-mon[32473]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-04T09:41:24.058 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:41:23 smithi052 ceph-mon[32473]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-04T09:41:24.058 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:41:23 smithi052 ceph-mon[32473]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-04T09:41:24.058 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:41:23 smithi052 ceph-mon[32473]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-04T09:41:24.058 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:41:23 smithi052 ceph-mon[32473]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-04T09:41:24.059 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:41:23 smithi052 ceph-mon[32473]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-04T09:41:24.059 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:41:23 smithi052 ceph-mon[32473]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-04T09:41:24.337 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:41:23 smithi191 ceph-mon[36816]: pgmap v318: 129 pgs: 129 active+clean; 78 MiB data, 691 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 9 op/s 2024-03-04T09:41:24.337 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:41:23 smithi191 ceph-mon[36816]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' 2024-03-04T09:41:24.338 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:41:23 smithi191 ceph-mon[36816]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' 2024-03-04T09:41:24.338 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:41:23 smithi191 ceph-mon[36816]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' 2024-03-04T09:41:24.338 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:41:23 smithi191 ceph-mon[36816]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' 2024-03-04T09:41:24.338 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:41:23 smithi191 ceph-mon[36816]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-04T09:41:24.338 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:41:23 smithi191 ceph-mon[36816]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-04T09:41:24.338 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:41:23 smithi191 ceph-mon[36816]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-04T09:41:24.338 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:41:23 smithi191 ceph-mon[36816]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-04T09:41:24.338 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:41:23 smithi191 ceph-mon[36816]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-04T09:41:24.339 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:41:23 smithi191 ceph-mon[36816]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-04T09:41:24.339 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:41:23 smithi191 ceph-mon[36816]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-04T09:41:24.339 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:41:23 smithi191 ceph-mon[36816]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-04T09:41:24.382 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:41:23 smithi106 ceph-mon[36484]: pgmap v318: 129 pgs: 129 active+clean; 78 MiB data, 691 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 9 op/s 2024-03-04T09:41:24.383 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:41:23 smithi106 ceph-mon[36484]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' 2024-03-04T09:41:24.383 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:41:23 smithi106 ceph-mon[36484]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' 2024-03-04T09:41:24.383 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:41:23 smithi106 ceph-mon[36484]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' 2024-03-04T09:41:24.383 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:41:23 smithi106 ceph-mon[36484]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' 2024-03-04T09:41:24.383 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:41:23 smithi106 ceph-mon[36484]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-04T09:41:24.383 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:41:23 smithi106 ceph-mon[36484]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-04T09:41:24.383 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:41:23 smithi106 ceph-mon[36484]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-04T09:41:24.384 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:41:23 smithi106 ceph-mon[36484]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-04T09:41:24.384 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:41:23 smithi106 ceph-mon[36484]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-04T09:41:24.384 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:41:23 smithi106 ceph-mon[36484]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-04T09:41:24.384 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:41:23 smithi106 ceph-mon[36484]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-04T09:41:24.384 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:41:23 smithi106 ceph-mon[36484]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-04T09:41:24.410 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/4e059f08-da08-11ee-95c6-87774f69a715/mon.a/config 2024-03-04T09:41:25.086 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:41:24 smithi052 ceph-mon[32473]: Adjusting osd_memory_target on smithi106 to 174.1M 2024-03-04T09:41:25.086 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:41:24 smithi052 ceph-mon[32473]: Unable to set osd_memory_target on smithi106 to 182633472: error parsing value: Value '182633472' is below minimum 939524096 2024-03-04T09:41:25.086 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:41:24 smithi052 ceph-mon[32473]: Adjusting osd_memory_target on smithi191 to 174.1M 2024-03-04T09:41:25.087 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:41:24 smithi052 ceph-mon[32473]: Unable to set osd_memory_target on smithi191 to 182640640: error parsing value: Value '182640640' is below minimum 939524096 2024-03-04T09:41:25.337 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:41:24 smithi191 ceph-mon[36816]: Adjusting osd_memory_target on smithi106 to 174.1M 2024-03-04T09:41:25.337 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:41:24 smithi191 ceph-mon[36816]: Unable to set osd_memory_target on smithi106 to 182633472: error parsing value: Value '182633472' is below minimum 939524096 2024-03-04T09:41:25.337 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:41:24 smithi191 ceph-mon[36816]: Adjusting osd_memory_target on smithi191 to 174.1M 2024-03-04T09:41:25.338 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:41:24 smithi191 ceph-mon[36816]: Unable to set osd_memory_target on smithi191 to 182640640: error parsing value: Value '182640640' is below minimum 939524096 2024-03-04T09:41:25.382 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:41:24 smithi106 ceph-mon[36484]: Adjusting osd_memory_target on smithi106 to 174.1M 2024-03-04T09:41:25.382 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:41:24 smithi106 ceph-mon[36484]: Unable to set osd_memory_target on smithi106 to 182633472: error parsing value: Value '182633472' is below minimum 939524096 2024-03-04T09:41:25.382 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:41:24 smithi106 ceph-mon[36484]: Adjusting osd_memory_target on smithi191 to 174.1M 2024-03-04T09:41:25.382 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:41:24 smithi106 ceph-mon[36484]: Unable to set osd_memory_target on smithi191 to 182640640: error parsing value: Value '182640640' is below minimum 939524096 2024-03-04T09:41:26.086 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:41:25 smithi052 ceph-mon[32473]: pgmap v319: 129 pgs: 129 active+clean; 78 MiB data, 691 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-04T09:41:26.238 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:41:25 smithi191 ceph-mon[36816]: pgmap v319: 129 pgs: 129 active+clean; 78 MiB data, 691 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-04T09:41:26.268 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2024-03-04T09:41:26.268 INFO:teuthology.orchestra.run.smithi052.stdout: "AsyncMessenger::Worker-0": { 2024-03-04T09:41:26.268 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_recv_messages": 0, 2024-03-04T09:41:26.268 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_send_messages": 2, 2024-03-04T09:41:26.269 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_recv_bytes": 0, 2024-03-04T09:41:26.269 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_send_bytes": 0, 2024-03-04T09:41:26.269 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_created_connections": 4, 2024-03-04T09:41:26.269 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_active_connections": 0, 2024-03-04T09:41:26.269 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_running_total_time": 0.001127776, 2024-03-04T09:41:26.269 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_running_send_time": 0.000000000, 2024-03-04T09:41:26.269 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_running_recv_time": 0.000255106, 2024-03-04T09:41:26.269 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-03-04T09:41:26.269 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_send_messages_queue_lat": { 2024-03-04T09:41:26.270 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 0, 2024-03-04T09:41:26.270 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000000000, 2024-03-04T09:41:26.270 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000000 2024-03-04T09:41:26.270 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:26.270 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_handle_ack_lat": { 2024-03-04T09:41:26.270 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 0, 2024-03-04T09:41:26.270 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000000000, 2024-03-04T09:41:26.270 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000000 2024-03-04T09:41:26.270 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:26.271 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_recv_encrypted_bytes": 0, 2024-03-04T09:41:26.271 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_send_encrypted_bytes": 0 2024-03-04T09:41:26.271 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:26.271 INFO:teuthology.orchestra.run.smithi052.stdout: "AsyncMessenger::Worker-1": { 2024-03-04T09:41:26.271 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_recv_messages": 225, 2024-03-04T09:41:26.271 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_send_messages": 215, 2024-03-04T09:41:26.271 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_recv_bytes": 1433216, 2024-03-04T09:41:26.272 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_send_bytes": 128688, 2024-03-04T09:41:26.272 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_created_connections": 2, 2024-03-04T09:41:26.272 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_active_connections": 1, 2024-03-04T09:41:26.272 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_running_total_time": 0.045320130, 2024-03-04T09:41:26.272 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_running_send_time": 0.015281099, 2024-03-04T09:41:26.272 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_running_recv_time": 0.033326366, 2024-03-04T09:41:26.272 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_running_fast_dispatch_time": 0.004485966, 2024-03-04T09:41:26.272 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_send_messages_queue_lat": { 2024-03-04T09:41:26.272 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 214, 2024-03-04T09:41:26.273 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.011079712, 2024-03-04T09:41:26.273 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000051774 2024-03-04T09:41:26.273 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:26.273 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_handle_ack_lat": { 2024-03-04T09:41:26.273 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 0, 2024-03-04T09:41:26.273 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000000000, 2024-03-04T09:41:26.273 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000000 2024-03-04T09:41:26.273 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:26.274 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_recv_encrypted_bytes": 1433216, 2024-03-04T09:41:26.274 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_send_encrypted_bytes": 128688 2024-03-04T09:41:26.274 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:26.274 INFO:teuthology.orchestra.run.smithi052.stdout: "AsyncMessenger::Worker-2": { 2024-03-04T09:41:26.274 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_recv_messages": 6, 2024-03-04T09:41:26.274 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_send_messages": 166, 2024-03-04T09:41:26.274 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_recv_bytes": 3344, 2024-03-04T09:41:26.274 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_send_bytes": 184336, 2024-03-04T09:41:26.274 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_created_connections": 5, 2024-03-04T09:41:26.275 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_active_connections": 3, 2024-03-04T09:41:26.275 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_running_total_time": 0.015597112, 2024-03-04T09:41:26.275 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_running_send_time": 0.010899849, 2024-03-04T09:41:26.275 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_running_recv_time": 0.002275940, 2024-03-04T09:41:26.275 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-03-04T09:41:26.275 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_send_messages_queue_lat": { 2024-03-04T09:41:26.275 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 164, 2024-03-04T09:41:26.275 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.014476561, 2024-03-04T09:41:26.276 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000088271 2024-03-04T09:41:26.276 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:26.276 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_handle_ack_lat": { 2024-03-04T09:41:26.276 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 0, 2024-03-04T09:41:26.276 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000000000, 2024-03-04T09:41:26.276 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000000 2024-03-04T09:41:26.276 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:26.276 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_recv_encrypted_bytes": 3344, 2024-03-04T09:41:26.276 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_send_encrypted_bytes": 184336 2024-03-04T09:41:26.277 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:26.277 INFO:teuthology.orchestra.run.smithi052.stdout: "cct": { 2024-03-04T09:41:26.277 INFO:teuthology.orchestra.run.smithi052.stdout: "total_workers": 0, 2024-03-04T09:41:26.277 INFO:teuthology.orchestra.run.smithi052.stdout: "unhealthy_workers": 0 2024-03-04T09:41:26.277 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:26.277 INFO:teuthology.orchestra.run.smithi052.stdout: "mempool": { 2024-03-04T09:41:26.277 INFO:teuthology.orchestra.run.smithi052.stdout: "bloom_filter_bytes": 0, 2024-03-04T09:41:26.277 INFO:teuthology.orchestra.run.smithi052.stdout: "bloom_filter_items": 0, 2024-03-04T09:41:26.278 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_alloc_bytes": 0, 2024-03-04T09:41:26.278 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_alloc_items": 0, 2024-03-04T09:41:26.278 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_cache_data_bytes": 0, 2024-03-04T09:41:26.278 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_cache_data_items": 0, 2024-03-04T09:41:26.278 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-04T09:41:26.278 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_cache_onode_items": 0, 2024-03-04T09:41:26.278 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-04T09:41:26.278 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_cache_meta_items": 0, 2024-03-04T09:41:26.279 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_cache_other_bytes": 0, 2024-03-04T09:41:26.279 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_cache_other_items": 0, 2024-03-04T09:41:26.279 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-04T09:41:26.279 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_cache_buffer_items": 0, 2024-03-04T09:41:26.279 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_extent_bytes": 0, 2024-03-04T09:41:26.279 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_extent_items": 0, 2024-03-04T09:41:26.279 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_blob_bytes": 0, 2024-03-04T09:41:26.279 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_blob_items": 0, 2024-03-04T09:41:26.280 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-04T09:41:26.280 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_shared_blob_items": 0, 2024-03-04T09:41:26.280 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-04T09:41:26.280 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_inline_bl_items": 0, 2024-03-04T09:41:26.280 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_fsck_bytes": 0, 2024-03-04T09:41:26.280 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_fsck_items": 0, 2024-03-04T09:41:26.280 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_txc_bytes": 0, 2024-03-04T09:41:26.280 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_txc_items": 0, 2024-03-04T09:41:26.280 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-04T09:41:26.281 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_writing_deferred_items": 0, 2024-03-04T09:41:26.281 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_writing_bytes": 0, 2024-03-04T09:41:26.281 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_writing_items": 0, 2024-03-04T09:41:26.281 INFO:teuthology.orchestra.run.smithi052.stdout: "bluefs_bytes": 0, 2024-03-04T09:41:26.281 INFO:teuthology.orchestra.run.smithi052.stdout: "bluefs_items": 0, 2024-03-04T09:41:26.281 INFO:teuthology.orchestra.run.smithi052.stdout: "bluefs_file_reader_bytes": 0, 2024-03-04T09:41:26.281 INFO:teuthology.orchestra.run.smithi052.stdout: "bluefs_file_reader_items": 0, 2024-03-04T09:41:26.281 INFO:teuthology.orchestra.run.smithi052.stdout: "bluefs_file_writer_bytes": 0, 2024-03-04T09:41:26.282 INFO:teuthology.orchestra.run.smithi052.stdout: "bluefs_file_writer_items": 0, 2024-03-04T09:41:26.282 INFO:teuthology.orchestra.run.smithi052.stdout: "buffer_anon_bytes": 148771, 2024-03-04T09:41:26.282 INFO:teuthology.orchestra.run.smithi052.stdout: "buffer_anon_items": 32, 2024-03-04T09:41:26.282 INFO:teuthology.orchestra.run.smithi052.stdout: "buffer_meta_bytes": 0, 2024-03-04T09:41:26.282 INFO:teuthology.orchestra.run.smithi052.stdout: "buffer_meta_items": 0, 2024-03-04T09:41:26.282 INFO:teuthology.orchestra.run.smithi052.stdout: "osd_bytes": 0, 2024-03-04T09:41:26.282 INFO:teuthology.orchestra.run.smithi052.stdout: "osd_items": 0, 2024-03-04T09:41:26.282 INFO:teuthology.orchestra.run.smithi052.stdout: "osd_mapbl_bytes": 0, 2024-03-04T09:41:26.282 INFO:teuthology.orchestra.run.smithi052.stdout: "osd_mapbl_items": 0, 2024-03-04T09:41:26.283 INFO:teuthology.orchestra.run.smithi052.stdout: "osd_pglog_bytes": 0, 2024-03-04T09:41:26.283 INFO:teuthology.orchestra.run.smithi052.stdout: "osd_pglog_items": 0, 2024-03-04T09:41:26.283 INFO:teuthology.orchestra.run.smithi052.stdout: "osdmap_bytes": 0, 2024-03-04T09:41:26.283 INFO:teuthology.orchestra.run.smithi052.stdout: "osdmap_items": 0, 2024-03-04T09:41:26.283 INFO:teuthology.orchestra.run.smithi052.stdout: "osdmap_mapping_bytes": 0, 2024-03-04T09:41:26.283 INFO:teuthology.orchestra.run.smithi052.stdout: "osdmap_mapping_items": 0, 2024-03-04T09:41:26.283 INFO:teuthology.orchestra.run.smithi052.stdout: "pgmap_bytes": 0, 2024-03-04T09:41:26.283 INFO:teuthology.orchestra.run.smithi052.stdout: "pgmap_items": 0, 2024-03-04T09:41:26.284 INFO:teuthology.orchestra.run.smithi052.stdout: "mds_co_bytes": 928, 2024-03-04T09:41:26.284 INFO:teuthology.orchestra.run.smithi052.stdout: "mds_co_items": 2, 2024-03-04T09:41:26.284 INFO:teuthology.orchestra.run.smithi052.stdout: "unittest_1_bytes": 0, 2024-03-04T09:41:26.284 INFO:teuthology.orchestra.run.smithi052.stdout: "unittest_1_items": 0, 2024-03-04T09:41:26.284 INFO:teuthology.orchestra.run.smithi052.stdout: "unittest_2_bytes": 0, 2024-03-04T09:41:26.284 INFO:teuthology.orchestra.run.smithi052.stdout: "unittest_2_items": 0 2024-03-04T09:41:26.284 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-03-04T09:41:26.284 INFO:teuthology.orchestra.run.smithi052.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-04T09:41:26.284 INFO:teuthology.orchestra.run.smithi052.stdout: "val": 0, 2024-03-04T09:41:26.285 INFO:teuthology.orchestra.run.smithi052.stdout: "max": 104857600, 2024-03-04T09:41:26.285 INFO:teuthology.orchestra.run.smithi052.stdout: "get_started": 0, 2024-03-04T09:41:26.285 INFO:teuthology.orchestra.run.smithi052.stdout: "get": 227, 2024-03-04T09:41:26.285 INFO:teuthology.orchestra.run.smithi052.stdout: "get_sum": 1401488, 2024-03-04T09:41:26.285 INFO:teuthology.orchestra.run.smithi052.stdout: "get_or_fail_fail": 0, 2024-03-04T09:41:26.285 INFO:teuthology.orchestra.run.smithi052.stdout: "get_or_fail_success": 227, 2024-03-04T09:41:26.285 INFO:teuthology.orchestra.run.smithi052.stdout: "take": 0, 2024-03-04T09:41:26.285 INFO:teuthology.orchestra.run.smithi052.stdout: "take_sum": 0, 2024-03-04T09:41:26.286 INFO:teuthology.orchestra.run.smithi052.stdout: "put": 227, 2024-03-04T09:41:26.286 INFO:teuthology.orchestra.run.smithi052.stdout: "put_sum": 1401488, 2024-03-04T09:41:26.286 INFO:teuthology.orchestra.run.smithi052.stdout: "wait": { 2024-03-04T09:41:26.286 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 0, 2024-03-04T09:41:26.286 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000000000, 2024-03-04T09:41:26.286 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000000 2024-03-04T09:41:26.286 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-03-04T09:41:26.286 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-03-04T09:41:26.286 INFO:teuthology.orchestra.run.smithi052.stdout:} 2024-03-04T09:41:26.382 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:41:25 smithi106 ceph-mon[36484]: pgmap v319: 129 pgs: 129 active+clean; 78 MiB data, 691 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-04T09:41:26.998 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.j 2024-03-04T09:41:26.999 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.j 2024-03-04T09:41:26.999 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.j 2024-03-04T09:41:26.999 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.j 2024-03-04T09:41:26.999 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.j 2024-03-04T09:41:26.999 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.j 2024-03-04T09:41:26.999 DEBUG:tasks.check_counter:Getting stats from b 2024-03-04T09:41:27.000 DEBUG:teuthology.orchestra.run.smithi106:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd shell --fsid 4e059f08-da08-11ee-95c6-87774f69a715 -- ceph daemon mds.b perf dump 2024-03-04T09:41:27.262 INFO:teuthology.orchestra.run.smithi106.stderr:Inferring config /var/lib/ceph/4e059f08-da08-11ee-95c6-87774f69a715/mon.b/config 2024-03-04T09:41:28.336 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:41:28 smithi052 ceph-mon[32473]: pgmap v320: 129 pgs: 129 active+clean; 78 MiB data, 691 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-03-04T09:41:28.336 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:41:28 smithi052 ceph-mon[32473]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' 2024-03-04T09:41:28.337 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:41:28 smithi052 ceph-mon[32473]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' 2024-03-04T09:41:28.337 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:41:28 smithi052 ceph-mon[32473]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-04T09:41:28.337 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:41:28 smithi052 ceph-mon[32473]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-04T09:41:28.337 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:41:28 smithi052 ceph-mon[32473]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-04T09:41:28.337 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:41:28 smithi052 ceph-mon[32473]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-04T09:41:28.337 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:41:28 smithi052 ceph-mon[32473]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T09:41:28.337 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:41:28 smithi052 ceph-mon[32473]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T09:41:28.337 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:41:28 smithi052 ceph-mon[32473]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' 2024-03-04T09:41:28.382 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:41:28 smithi106 ceph-mon[36484]: pgmap v320: 129 pgs: 129 active+clean; 78 MiB data, 691 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-03-04T09:41:28.382 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:41:28 smithi106 ceph-mon[36484]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' 2024-03-04T09:41:28.382 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:41:28 smithi106 ceph-mon[36484]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' 2024-03-04T09:41:28.382 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:41:28 smithi106 ceph-mon[36484]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-04T09:41:28.382 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:41:28 smithi106 ceph-mon[36484]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-04T09:41:28.383 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:41:28 smithi106 ceph-mon[36484]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-04T09:41:28.383 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:41:28 smithi106 ceph-mon[36484]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-04T09:41:28.383 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:41:28 smithi106 ceph-mon[36484]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T09:41:28.383 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:41:28 smithi106 ceph-mon[36484]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T09:41:28.383 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:41:28 smithi106 ceph-mon[36484]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' 2024-03-04T09:41:28.587 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:41:28 smithi191 ceph-mon[36816]: pgmap v320: 129 pgs: 129 active+clean; 78 MiB data, 691 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-03-04T09:41:28.588 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:41:28 smithi191 ceph-mon[36816]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' 2024-03-04T09:41:28.588 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:41:28 smithi191 ceph-mon[36816]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' 2024-03-04T09:41:28.588 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:41:28 smithi191 ceph-mon[36816]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-04T09:41:28.588 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:41:28 smithi191 ceph-mon[36816]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-04T09:41:28.589 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:41:28 smithi191 ceph-mon[36816]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-04T09:41:28.589 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:41:28 smithi191 ceph-mon[36816]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-04T09:41:28.589 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:41:28 smithi191 ceph-mon[36816]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T09:41:28.589 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:41:28 smithi191 ceph-mon[36816]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T09:41:28.589 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:41:28 smithi191 ceph-mon[36816]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' 2024-03-04T09:41:29.143 INFO:teuthology.orchestra.run.smithi106.stdout:{ 2024-03-04T09:41:29.144 INFO:teuthology.orchestra.run.smithi106.stdout: "AsyncMessenger::Worker-0": { 2024-03-04T09:41:29.144 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_recv_messages": 3158, 2024-03-04T09:41:29.144 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_send_messages": 3002, 2024-03-04T09:41:29.144 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_recv_bytes": 2326480, 2024-03-04T09:41:29.144 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_send_bytes": 23837221, 2024-03-04T09:41:29.144 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_created_connections": 11, 2024-03-04T09:41:29.145 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_active_connections": 4, 2024-03-04T09:41:29.145 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_running_total_time": 0.361861790, 2024-03-04T09:41:29.145 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_running_send_time": 0.148590180, 2024-03-04T09:41:29.145 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_running_recv_time": 0.107393671, 2024-03-04T09:41:29.145 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_running_fast_dispatch_time": 0.099851521, 2024-03-04T09:41:29.145 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_send_messages_queue_lat": { 2024-03-04T09:41:29.145 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 3000, 2024-03-04T09:41:29.145 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.677833264, 2024-03-04T09:41:29.145 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000225944 2024-03-04T09:41:29.146 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-03-04T09:41:29.146 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_handle_ack_lat": { 2024-03-04T09:41:29.146 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 1167, 2024-03-04T09:41:29.146 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.000197986, 2024-03-04T09:41:29.146 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000000169 2024-03-04T09:41:29.146 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-03-04T09:41:29.146 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_recv_encrypted_bytes": 1540560, 2024-03-04T09:41:29.146 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_send_encrypted_bytes": 138176 2024-03-04T09:41:29.147 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-03-04T09:41:29.147 INFO:teuthology.orchestra.run.smithi106.stdout: "AsyncMessenger::Worker-1": { 2024-03-04T09:41:29.147 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_recv_messages": 21849, 2024-03-04T09:41:29.147 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_send_messages": 40481, 2024-03-04T09:41:29.147 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_recv_bytes": 7050866, 2024-03-04T09:41:29.147 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_send_bytes": 23849360, 2024-03-04T09:41:29.147 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_created_connections": 17, 2024-03-04T09:41:29.147 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_active_connections": 5, 2024-03-04T09:41:29.148 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_running_total_time": 1.174245583, 2024-03-04T09:41:29.148 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_running_send_time": 0.508081032, 2024-03-04T09:41:29.148 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_running_recv_time": 1.249897261, 2024-03-04T09:41:29.148 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_running_fast_dispatch_time": 0.135958654, 2024-03-04T09:41:29.148 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_send_messages_queue_lat": { 2024-03-04T09:41:29.148 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 40479, 2024-03-04T09:41:29.148 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 2.862224797, 2024-03-04T09:41:29.148 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000070708 2024-03-04T09:41:29.148 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-03-04T09:41:29.149 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_handle_ack_lat": { 2024-03-04T09:41:29.149 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 33458, 2024-03-04T09:41:29.149 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.006740922, 2024-03-04T09:41:29.149 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000000201 2024-03-04T09:41:29.149 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-03-04T09:41:29.149 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_recv_encrypted_bytes": 93392, 2024-03-04T09:41:29.149 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_send_encrypted_bytes": 482112 2024-03-04T09:41:29.149 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-03-04T09:41:29.150 INFO:teuthology.orchestra.run.smithi106.stdout: "AsyncMessenger::Worker-2": { 2024-03-04T09:41:29.150 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_recv_messages": 4290, 2024-03-04T09:41:29.150 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_send_messages": 3657, 2024-03-04T09:41:29.150 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_recv_bytes": 1369707, 2024-03-04T09:41:29.150 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_send_bytes": 13447715, 2024-03-04T09:41:29.150 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_created_connections": 12, 2024-03-04T09:41:29.150 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_active_connections": 7, 2024-03-04T09:41:29.150 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_running_total_time": 0.511790508, 2024-03-04T09:41:29.150 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_running_send_time": 0.229389935, 2024-03-04T09:41:29.151 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_running_recv_time": 0.111617788, 2024-03-04T09:41:29.151 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_running_fast_dispatch_time": 0.142338770, 2024-03-04T09:41:29.151 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_send_messages_queue_lat": { 2024-03-04T09:41:29.151 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 3655, 2024-03-04T09:41:29.151 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 1.697309996, 2024-03-04T09:41:29.151 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000464380 2024-03-04T09:41:29.151 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-03-04T09:41:29.151 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_handle_ack_lat": { 2024-03-04T09:41:29.152 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 1969, 2024-03-04T09:41:29.152 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.000361962, 2024-03-04T09:41:29.152 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000000183 2024-03-04T09:41:29.152 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-03-04T09:41:29.152 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_recv_encrypted_bytes": 299808, 2024-03-04T09:41:29.152 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_send_encrypted_bytes": 87536 2024-03-04T09:41:29.152 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-03-04T09:41:29.152 INFO:teuthology.orchestra.run.smithi106.stdout: "cct": { 2024-03-04T09:41:29.153 INFO:teuthology.orchestra.run.smithi106.stdout: "total_workers": 1, 2024-03-04T09:41:29.153 INFO:teuthology.orchestra.run.smithi106.stdout: "unhealthy_workers": 0 2024-03-04T09:41:29.153 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-03-04T09:41:29.153 INFO:teuthology.orchestra.run.smithi106.stdout: "finisher-MDSRank": { 2024-03-04T09:41:29.153 INFO:teuthology.orchestra.run.smithi106.stdout: "queue_len": 0, 2024-03-04T09:41:29.153 INFO:teuthology.orchestra.run.smithi106.stdout: "complete_latency": { 2024-03-04T09:41:29.153 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 875, 2024-03-04T09:41:29.153 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 11.837934309, 2024-03-04T09:41:29.153 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.013529067 2024-03-04T09:41:29.154 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-03-04T09:41:29.154 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-03-04T09:41:29.154 INFO:teuthology.orchestra.run.smithi106.stdout: "finisher-PurgeQueue": { 2024-03-04T09:41:29.154 INFO:teuthology.orchestra.run.smithi106.stdout: "queue_len": 0, 2024-03-04T09:41:29.154 INFO:teuthology.orchestra.run.smithi106.stdout: "complete_latency": { 2024-03-04T09:41:29.154 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 5035, 2024-03-04T09:41:29.154 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 1.953727153, 2024-03-04T09:41:29.154 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000388029 2024-03-04T09:41:29.154 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-03-04T09:41:29.155 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-03-04T09:41:29.155 INFO:teuthology.orchestra.run.smithi106.stdout: "mds": { 2024-03-04T09:41:29.155 INFO:teuthology.orchestra.run.smithi106.stdout: "request": 10739, 2024-03-04T09:41:29.155 INFO:teuthology.orchestra.run.smithi106.stdout: "reply": 10739, 2024-03-04T09:41:29.155 INFO:teuthology.orchestra.run.smithi106.stdout: "reply_latency": { 2024-03-04T09:41:29.155 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 10739, 2024-03-04T09:41:29.155 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 520.079099765, 2024-03-04T09:41:29.155 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.048429006 2024-03-04T09:41:29.156 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-03-04T09:41:29.156 INFO:teuthology.orchestra.run.smithi106.stdout: "slow_reply": 0, 2024-03-04T09:41:29.156 INFO:teuthology.orchestra.run.smithi106.stdout: "forward": 0, 2024-03-04T09:41:29.156 INFO:teuthology.orchestra.run.smithi106.stdout: "dir_fetch_complete": 0, 2024-03-04T09:41:29.156 INFO:teuthology.orchestra.run.smithi106.stdout: "dir_fetch_keys": 0, 2024-03-04T09:41:29.156 INFO:teuthology.orchestra.run.smithi106.stdout: "dir_commit": 194, 2024-03-04T09:41:29.156 INFO:teuthology.orchestra.run.smithi106.stdout: "dir_split": 22, 2024-03-04T09:41:29.156 INFO:teuthology.orchestra.run.smithi106.stdout: "dir_merge": 0, 2024-03-04T09:41:29.156 INFO:teuthology.orchestra.run.smithi106.stdout: "inodes": 6769, 2024-03-04T09:41:29.157 INFO:teuthology.orchestra.run.smithi106.stdout: "inodes_top": 230, 2024-03-04T09:41:29.157 INFO:teuthology.orchestra.run.smithi106.stdout: "inodes_bottom": 6486, 2024-03-04T09:41:29.157 INFO:teuthology.orchestra.run.smithi106.stdout: "inodes_pin_tail": 53, 2024-03-04T09:41:29.157 INFO:teuthology.orchestra.run.smithi106.stdout: "inodes_pinned": 6441, 2024-03-04T09:41:29.157 INFO:teuthology.orchestra.run.smithi106.stdout: "inodes_expired": 171, 2024-03-04T09:41:29.157 INFO:teuthology.orchestra.run.smithi106.stdout: "inodes_with_caps": 3366, 2024-03-04T09:41:29.157 INFO:teuthology.orchestra.run.smithi106.stdout: "caps": 3366, 2024-03-04T09:41:29.157 INFO:teuthology.orchestra.run.smithi106.stdout: "subtrees": 6, 2024-03-04T09:41:29.157 INFO:teuthology.orchestra.run.smithi106.stdout: "traverse": 10983, 2024-03-04T09:41:29.158 INFO:teuthology.orchestra.run.smithi106.stdout: "traverse_hit": 10444, 2024-03-04T09:41:29.158 INFO:teuthology.orchestra.run.smithi106.stdout: "traverse_forward": 0, 2024-03-04T09:41:29.158 INFO:teuthology.orchestra.run.smithi106.stdout: "traverse_discover": 0, 2024-03-04T09:41:29.158 INFO:teuthology.orchestra.run.smithi106.stdout: "traverse_dir_fetch": 0, 2024-03-04T09:41:29.158 INFO:teuthology.orchestra.run.smithi106.stdout: "traverse_remote_ino": 0, 2024-03-04T09:41:29.158 INFO:teuthology.orchestra.run.smithi106.stdout: "traverse_lock": 11, 2024-03-04T09:41:29.158 INFO:teuthology.orchestra.run.smithi106.stdout: "load_cent": 0, 2024-03-04T09:41:29.158 INFO:teuthology.orchestra.run.smithi106.stdout: "q": 0, 2024-03-04T09:41:29.159 INFO:teuthology.orchestra.run.smithi106.stdout: "exported": 11, 2024-03-04T09:41:29.159 INFO:teuthology.orchestra.run.smithi106.stdout: "exported_inodes": 156, 2024-03-04T09:41:29.159 INFO:teuthology.orchestra.run.smithi106.stdout: "imported": 0, 2024-03-04T09:41:29.159 INFO:teuthology.orchestra.run.smithi106.stdout: "imported_inodes": 0, 2024-03-04T09:41:29.159 INFO:teuthology.orchestra.run.smithi106.stdout: "openino_dir_fetch": 0, 2024-03-04T09:41:29.159 INFO:teuthology.orchestra.run.smithi106.stdout: "openino_backtrace_fetch": 0, 2024-03-04T09:41:29.159 INFO:teuthology.orchestra.run.smithi106.stdout: "openino_peer_discover": 0, 2024-03-04T09:41:29.159 INFO:teuthology.orchestra.run.smithi106.stdout: "root_rfiles": 3451, 2024-03-04T09:41:29.159 INFO:teuthology.orchestra.run.smithi106.stdout: "root_rbytes": 26478731, 2024-03-04T09:41:29.160 INFO:teuthology.orchestra.run.smithi106.stdout: "root_rsnaps": 6, 2024-03-04T09:41:29.160 INFO:teuthology.orchestra.run.smithi106.stdout: "scrub_backtrace_fetch": 0, 2024-03-04T09:41:29.160 INFO:teuthology.orchestra.run.smithi106.stdout: "scrub_set_tag": 0, 2024-03-04T09:41:29.160 INFO:teuthology.orchestra.run.smithi106.stdout: "scrub_backtrace_repaired": 0, 2024-03-04T09:41:29.160 INFO:teuthology.orchestra.run.smithi106.stdout: "scrub_inotable_repaired": 0, 2024-03-04T09:41:29.160 INFO:teuthology.orchestra.run.smithi106.stdout: "scrub_dir_inodes": 0, 2024-03-04T09:41:29.160 INFO:teuthology.orchestra.run.smithi106.stdout: "scrub_dir_base_inodes": 0, 2024-03-04T09:41:29.160 INFO:teuthology.orchestra.run.smithi106.stdout: "scrub_dirfrag_rstats": 0, 2024-03-04T09:41:29.161 INFO:teuthology.orchestra.run.smithi106.stdout: "scrub_file_inodes": 0, 2024-03-04T09:41:29.161 INFO:teuthology.orchestra.run.smithi106.stdout: "handle_inode_file_caps": 1, 2024-03-04T09:41:29.161 INFO:teuthology.orchestra.run.smithi106.stdout: "ceph_cap_op_revoke": 297, 2024-03-04T09:41:29.161 INFO:teuthology.orchestra.run.smithi106.stdout: "ceph_cap_op_grant": 10138, 2024-03-04T09:41:29.161 INFO:teuthology.orchestra.run.smithi106.stdout: "ceph_cap_op_trunc": 8, 2024-03-04T09:41:29.161 INFO:teuthology.orchestra.run.smithi106.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-03-04T09:41:29.161 INFO:teuthology.orchestra.run.smithi106.stdout: "ceph_cap_op_flush_ack": 0, 2024-03-04T09:41:29.161 INFO:teuthology.orchestra.run.smithi106.stdout: "handle_client_caps": 7627, 2024-03-04T09:41:29.161 INFO:teuthology.orchestra.run.smithi106.stdout: "handle_client_caps_dirty": 6433, 2024-03-04T09:41:29.162 INFO:teuthology.orchestra.run.smithi106.stdout: "handle_client_cap_release": 155, 2024-03-04T09:41:29.162 INFO:teuthology.orchestra.run.smithi106.stdout: "process_request_cap_release": 3515 2024-03-04T09:41:29.162 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-03-04T09:41:29.162 INFO:teuthology.orchestra.run.smithi106.stdout: "mds_cache": { 2024-03-04T09:41:29.162 INFO:teuthology.orchestra.run.smithi106.stdout: "dir_update": 0, 2024-03-04T09:41:29.162 INFO:teuthology.orchestra.run.smithi106.stdout: "dir_update_receipt": 0, 2024-03-04T09:41:29.162 INFO:teuthology.orchestra.run.smithi106.stdout: "dir_try_discover": 0, 2024-03-04T09:41:29.162 INFO:teuthology.orchestra.run.smithi106.stdout: "dir_send_discover": 0, 2024-03-04T09:41:29.162 INFO:teuthology.orchestra.run.smithi106.stdout: "dir_handle_discover": 16, 2024-03-04T09:41:29.163 INFO:teuthology.orchestra.run.smithi106.stdout: "num_strays": 0, 2024-03-04T09:41:29.163 INFO:teuthology.orchestra.run.smithi106.stdout: "num_strays_delayed": 0, 2024-03-04T09:41:29.163 INFO:teuthology.orchestra.run.smithi106.stdout: "num_strays_enqueuing": 0, 2024-03-04T09:41:29.163 INFO:teuthology.orchestra.run.smithi106.stdout: "strays_created": 3217, 2024-03-04T09:41:29.163 INFO:teuthology.orchestra.run.smithi106.stdout: "strays_enqueued": 3215, 2024-03-04T09:41:29.163 INFO:teuthology.orchestra.run.smithi106.stdout: "strays_reintegrated": 2, 2024-03-04T09:41:29.163 INFO:teuthology.orchestra.run.smithi106.stdout: "strays_migrated": 0, 2024-03-04T09:41:29.163 INFO:teuthology.orchestra.run.smithi106.stdout: "num_recovering_processing": 0, 2024-03-04T09:41:29.163 INFO:teuthology.orchestra.run.smithi106.stdout: "num_recovering_enqueued": 0, 2024-03-04T09:41:29.164 INFO:teuthology.orchestra.run.smithi106.stdout: "num_recovering_prioritized": 0, 2024-03-04T09:41:29.164 INFO:teuthology.orchestra.run.smithi106.stdout: "recovery_started": 0, 2024-03-04T09:41:29.164 INFO:teuthology.orchestra.run.smithi106.stdout: "recovery_completed": 0, 2024-03-04T09:41:29.164 INFO:teuthology.orchestra.run.smithi106.stdout: "ireq_enqueue_scrub": 0, 2024-03-04T09:41:29.164 INFO:teuthology.orchestra.run.smithi106.stdout: "ireq_exportdir": 11, 2024-03-04T09:41:29.164 INFO:teuthology.orchestra.run.smithi106.stdout: "ireq_flush": 0, 2024-03-04T09:41:29.164 INFO:teuthology.orchestra.run.smithi106.stdout: "ireq_fragmentdir": 22, 2024-03-04T09:41:29.164 INFO:teuthology.orchestra.run.smithi106.stdout: "ireq_fragstats": 0, 2024-03-04T09:41:29.165 INFO:teuthology.orchestra.run.smithi106.stdout: "ireq_inodestats": 0 2024-03-04T09:41:29.165 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-03-04T09:41:29.165 INFO:teuthology.orchestra.run.smithi106.stdout: "mds_log": { 2024-03-04T09:41:29.165 INFO:teuthology.orchestra.run.smithi106.stdout: "evlrg": 0, 2024-03-04T09:41:29.165 INFO:teuthology.orchestra.run.smithi106.stdout: "evadd": 22598, 2024-03-04T09:41:29.165 INFO:teuthology.orchestra.run.smithi106.stdout: "evex": 0, 2024-03-04T09:41:29.165 INFO:teuthology.orchestra.run.smithi106.stdout: "evtrm": 0, 2024-03-04T09:41:29.165 INFO:teuthology.orchestra.run.smithi106.stdout: "ev": 22598, 2024-03-04T09:41:29.165 INFO:teuthology.orchestra.run.smithi106.stdout: "evexg": 0, 2024-03-04T09:41:29.166 INFO:teuthology.orchestra.run.smithi106.stdout: "evexd": 0, 2024-03-04T09:41:29.166 INFO:teuthology.orchestra.run.smithi106.stdout: "segadd": 29, 2024-03-04T09:41:29.166 INFO:teuthology.orchestra.run.smithi106.stdout: "segex": 0, 2024-03-04T09:41:29.166 INFO:teuthology.orchestra.run.smithi106.stdout: "segtrm": 0, 2024-03-04T09:41:29.166 INFO:teuthology.orchestra.run.smithi106.stdout: "seg": 29, 2024-03-04T09:41:29.166 INFO:teuthology.orchestra.run.smithi106.stdout: "segmjr": 3, 2024-03-04T09:41:29.166 INFO:teuthology.orchestra.run.smithi106.stdout: "segexg": 0, 2024-03-04T09:41:29.166 INFO:teuthology.orchestra.run.smithi106.stdout: "segexd": 0, 2024-03-04T09:41:29.166 INFO:teuthology.orchestra.run.smithi106.stdout: "expos": 4194304, 2024-03-04T09:41:29.167 INFO:teuthology.orchestra.run.smithi106.stdout: "wrpos": 56812162, 2024-03-04T09:41:29.167 INFO:teuthology.orchestra.run.smithi106.stdout: "rdpos": 0, 2024-03-04T09:41:29.167 INFO:teuthology.orchestra.run.smithi106.stdout: "jlat": { 2024-03-04T09:41:29.167 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 448, 2024-03-04T09:41:29.167 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 35.853583106, 2024-03-04T09:41:29.167 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.080030319 2024-03-04T09:41:29.167 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-03-04T09:41:29.167 INFO:teuthology.orchestra.run.smithi106.stdout: "replayed": 0 2024-03-04T09:41:29.168 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-03-04T09:41:29.168 INFO:teuthology.orchestra.run.smithi106.stdout: "mds_mem": { 2024-03-04T09:41:29.168 INFO:teuthology.orchestra.run.smithi106.stdout: "ino": 5324, 2024-03-04T09:41:29.168 INFO:teuthology.orchestra.run.smithi106.stdout: "ino+": 8673, 2024-03-04T09:41:29.168 INFO:teuthology.orchestra.run.smithi106.stdout: "ino-": 3349, 2024-03-04T09:41:29.168 INFO:teuthology.orchestra.run.smithi106.stdout: "dir": 233, 2024-03-04T09:41:29.168 INFO:teuthology.orchestra.run.smithi106.stdout: "dir+": 273, 2024-03-04T09:41:29.168 INFO:teuthology.orchestra.run.smithi106.stdout: "dir-": 40, 2024-03-04T09:41:29.168 INFO:teuthology.orchestra.run.smithi106.stdout: "dn": 6769, 2024-03-04T09:41:29.169 INFO:teuthology.orchestra.run.smithi106.stdout: "dn+": 10295, 2024-03-04T09:41:29.169 INFO:teuthology.orchestra.run.smithi106.stdout: "dn-": 3526, 2024-03-04T09:41:29.169 INFO:teuthology.orchestra.run.smithi106.stdout: "cap": 3366, 2024-03-04T09:41:29.169 INFO:teuthology.orchestra.run.smithi106.stdout: "cap+": 6749, 2024-03-04T09:41:29.169 INFO:teuthology.orchestra.run.smithi106.stdout: "cap-": 3383, 2024-03-04T09:41:29.169 INFO:teuthology.orchestra.run.smithi106.stdout: "rss": 102800, 2024-03-04T09:41:29.169 INFO:teuthology.orchestra.run.smithi106.stdout: "heap": 231804 2024-03-04T09:41:29.169 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-03-04T09:41:29.169 INFO:teuthology.orchestra.run.smithi106.stdout: "mds_server": { 2024-03-04T09:41:29.170 INFO:teuthology.orchestra.run.smithi106.stdout: "dispatch_client_request": 11031, 2024-03-04T09:41:29.170 INFO:teuthology.orchestra.run.smithi106.stdout: "dispatch_server_request": 1, 2024-03-04T09:41:29.170 INFO:teuthology.orchestra.run.smithi106.stdout: "handle_client_request": 10739, 2024-03-04T09:41:29.170 INFO:teuthology.orchestra.run.smithi106.stdout: "handle_client_session": 49, 2024-03-04T09:41:29.170 INFO:teuthology.orchestra.run.smithi106.stdout: "handle_peer_request": 2, 2024-03-04T09:41:29.170 INFO:teuthology.orchestra.run.smithi106.stdout: "req_create_latency": { 2024-03-04T09:41:29.170 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 6649, 2024-03-04T09:41:29.170 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 502.770809191, 2024-03-04T09:41:29.171 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.075616003 2024-03-04T09:41:29.171 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-03-04T09:41:29.171 INFO:teuthology.orchestra.run.smithi106.stdout: "req_getattr_latency": { 2024-03-04T09:41:29.171 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 116, 2024-03-04T09:41:29.171 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.038789167, 2024-03-04T09:41:29.171 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000334389 2024-03-04T09:41:29.171 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-03-04T09:41:29.171 INFO:teuthology.orchestra.run.smithi106.stdout: "req_getfilelock_latency": { 2024-03-04T09:41:29.171 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 0, 2024-03-04T09:41:29.172 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.000000000, 2024-03-04T09:41:29.172 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000000000 2024-03-04T09:41:29.172 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-03-04T09:41:29.172 INFO:teuthology.orchestra.run.smithi106.stdout: "req_link_latency": { 2024-03-04T09:41:29.172 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 2, 2024-03-04T09:41:29.172 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.004142907, 2024-03-04T09:41:29.172 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.002071453 2024-03-04T09:41:29.172 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-03-04T09:41:29.173 INFO:teuthology.orchestra.run.smithi106.stdout: "req_lookup_latency": { 2024-03-04T09:41:29.173 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 19, 2024-03-04T09:41:29.173 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.011476640, 2024-03-04T09:41:29.173 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000604033 2024-03-04T09:41:29.173 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-03-04T09:41:29.173 INFO:teuthology.orchestra.run.smithi106.stdout: "req_lookuphash_latency": { 2024-03-04T09:41:29.173 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 0, 2024-03-04T09:41:29.173 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.000000000, 2024-03-04T09:41:29.173 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000000000 2024-03-04T09:41:29.174 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-03-04T09:41:29.174 INFO:teuthology.orchestra.run.smithi106.stdout: "req_lookupino_latency": { 2024-03-04T09:41:29.174 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 1, 2024-03-04T09:41:29.174 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.000214211, 2024-03-04T09:41:29.174 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000214211 2024-03-04T09:41:29.174 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-03-04T09:41:29.174 INFO:teuthology.orchestra.run.smithi106.stdout: "req_lookupname_latency": { 2024-03-04T09:41:29.174 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 0, 2024-03-04T09:41:29.174 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.000000000, 2024-03-04T09:41:29.175 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000000000 2024-03-04T09:41:29.175 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-03-04T09:41:29.175 INFO:teuthology.orchestra.run.smithi106.stdout: "req_lookupparent_latency": { 2024-03-04T09:41:29.175 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 0, 2024-03-04T09:41:29.175 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.000000000, 2024-03-04T09:41:29.175 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000000000 2024-03-04T09:41:29.175 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-03-04T09:41:29.175 INFO:teuthology.orchestra.run.smithi106.stdout: "req_lookupsnap_latency": { 2024-03-04T09:41:29.175 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 0, 2024-03-04T09:41:29.176 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.000000000, 2024-03-04T09:41:29.176 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000000000 2024-03-04T09:41:29.176 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-03-04T09:41:29.176 INFO:teuthology.orchestra.run.smithi106.stdout: "req_lssnap_latency": { 2024-03-04T09:41:29.176 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 10, 2024-03-04T09:41:29.176 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.002632093, 2024-03-04T09:41:29.177 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000263209 2024-03-04T09:41:29.177 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-03-04T09:41:29.177 INFO:teuthology.orchestra.run.smithi106.stdout: "req_mkdir_latency": { 2024-03-04T09:41:29.177 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 85, 2024-03-04T09:41:29.177 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 3.123861754, 2024-03-04T09:41:29.178 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.036751314 2024-03-04T09:41:29.178 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-03-04T09:41:29.178 INFO:teuthology.orchestra.run.smithi106.stdout: "req_mknod_latency": { 2024-03-04T09:41:29.178 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 0, 2024-03-04T09:41:29.178 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.000000000, 2024-03-04T09:41:29.178 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000000000 2024-03-04T09:41:29.178 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-03-04T09:41:29.178 INFO:teuthology.orchestra.run.smithi106.stdout: "req_mksnap_latency": { 2024-03-04T09:41:29.178 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 10, 2024-03-04T09:41:29.179 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 1.221539849, 2024-03-04T09:41:29.179 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.122153984 2024-03-04T09:41:29.179 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-03-04T09:41:29.179 INFO:teuthology.orchestra.run.smithi106.stdout: "req_open_latency": { 2024-03-04T09:41:29.179 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 441, 2024-03-04T09:41:29.179 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 2.586811178, 2024-03-04T09:41:29.179 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.005865784 2024-03-04T09:41:29.179 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-03-04T09:41:29.180 INFO:teuthology.orchestra.run.smithi106.stdout: "req_readdir_latency": { 2024-03-04T09:41:29.180 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 107, 2024-03-04T09:41:29.180 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.719284487, 2024-03-04T09:41:29.180 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.006722284 2024-03-04T09:41:29.180 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-03-04T09:41:29.180 INFO:teuthology.orchestra.run.smithi106.stdout: "req_rename_latency": { 2024-03-04T09:41:29.180 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 81, 2024-03-04T09:41:29.180 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.492318767, 2024-03-04T09:41:29.181 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.006078009 2024-03-04T09:41:29.181 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-03-04T09:41:29.181 INFO:teuthology.orchestra.run.smithi106.stdout: "req_renamesnap_latency": { 2024-03-04T09:41:29.181 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 0, 2024-03-04T09:41:29.181 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.000000000, 2024-03-04T09:41:29.181 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000000000 2024-03-04T09:41:29.181 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-03-04T09:41:29.181 INFO:teuthology.orchestra.run.smithi106.stdout: "req_snapdiff_latency": { 2024-03-04T09:41:29.182 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 0, 2024-03-04T09:41:29.182 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.000000000, 2024-03-04T09:41:29.182 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000000000 2024-03-04T09:41:29.182 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-03-04T09:41:29.182 INFO:teuthology.orchestra.run.smithi106.stdout: "req_rmdir_latency": { 2024-03-04T09:41:29.182 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 22, 2024-03-04T09:41:29.182 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 1.228952899, 2024-03-04T09:41:29.182 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.055861495 2024-03-04T09:41:29.182 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-03-04T09:41:29.183 INFO:teuthology.orchestra.run.smithi106.stdout: "req_rmsnap_latency": { 2024-03-04T09:41:29.183 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 4, 2024-03-04T09:41:29.183 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.014911990, 2024-03-04T09:41:29.183 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.003727997 2024-03-04T09:41:29.183 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-03-04T09:41:29.183 INFO:teuthology.orchestra.run.smithi106.stdout: "req_rmxattr_latency": { 2024-03-04T09:41:29.183 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 0, 2024-03-04T09:41:29.183 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.000000000, 2024-03-04T09:41:29.183 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000000000 2024-03-04T09:41:29.184 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-03-04T09:41:29.184 INFO:teuthology.orchestra.run.smithi106.stdout: "req_setattr_latency": { 2024-03-04T09:41:29.184 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 10, 2024-03-04T09:41:29.184 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.048450046, 2024-03-04T09:41:29.184 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.004845004 2024-03-04T09:41:29.184 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-03-04T09:41:29.184 INFO:teuthology.orchestra.run.smithi106.stdout: "req_setdirlayout_latency": { 2024-03-04T09:41:29.184 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 0, 2024-03-04T09:41:29.185 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.000000000, 2024-03-04T09:41:29.185 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000000000 2024-03-04T09:41:29.185 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-03-04T09:41:29.185 INFO:teuthology.orchestra.run.smithi106.stdout: "req_setfilelock_latency": { 2024-03-04T09:41:29.185 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 8, 2024-03-04T09:41:29.185 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.002365823, 2024-03-04T09:41:29.185 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000295727 2024-03-04T09:41:29.185 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-03-04T09:41:29.185 INFO:teuthology.orchestra.run.smithi106.stdout: "req_setlayout_latency": { 2024-03-04T09:41:29.186 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 0, 2024-03-04T09:41:29.186 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.000000000, 2024-03-04T09:41:29.186 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000000000 2024-03-04T09:41:29.186 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-03-04T09:41:29.186 INFO:teuthology.orchestra.run.smithi106.stdout: "req_setxattr_latency": { 2024-03-04T09:41:29.186 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 9, 2024-03-04T09:41:29.186 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.018681787, 2024-03-04T09:41:29.186 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.002075754 2024-03-04T09:41:29.186 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-03-04T09:41:29.187 INFO:teuthology.orchestra.run.smithi106.stdout: "req_symlink_latency": { 2024-03-04T09:41:29.187 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 7, 2024-03-04T09:41:29.187 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.005072241, 2024-03-04T09:41:29.187 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000724605 2024-03-04T09:41:29.187 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-03-04T09:41:29.187 INFO:teuthology.orchestra.run.smithi106.stdout: "req_unlink_latency": { 2024-03-04T09:41:29.187 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 3158, 2024-03-04T09:41:29.187 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 7.788784735, 2024-03-04T09:41:29.188 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.002466366 2024-03-04T09:41:29.188 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-03-04T09:41:29.188 INFO:teuthology.orchestra.run.smithi106.stdout: "cap_revoke_eviction": 0, 2024-03-04T09:41:29.188 INFO:teuthology.orchestra.run.smithi106.stdout: "cap_acquisition_throttle": 0, 2024-03-04T09:41:29.188 INFO:teuthology.orchestra.run.smithi106.stdout: "req_getvxattr_latency": { 2024-03-04T09:41:29.188 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 0, 2024-03-04T09:41:29.188 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.000000000, 2024-03-04T09:41:29.188 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000000000 2024-03-04T09:41:29.188 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-03-04T09:41:29.189 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-03-04T09:41:29.189 INFO:teuthology.orchestra.run.smithi106.stdout: "mds_sessions": { 2024-03-04T09:41:29.189 INFO:teuthology.orchestra.run.smithi106.stdout: "session_count": 2, 2024-03-04T09:41:29.189 INFO:teuthology.orchestra.run.smithi106.stdout: "session_add": 7, 2024-03-04T09:41:29.189 INFO:teuthology.orchestra.run.smithi106.stdout: "session_remove": 5, 2024-03-04T09:41:29.189 INFO:teuthology.orchestra.run.smithi106.stdout: "sessions_open": 2, 2024-03-04T09:41:29.189 INFO:teuthology.orchestra.run.smithi106.stdout: "sessions_stale": 0, 2024-03-04T09:41:29.189 INFO:teuthology.orchestra.run.smithi106.stdout: "total_load": 67, 2024-03-04T09:41:29.190 INFO:teuthology.orchestra.run.smithi106.stdout: "average_load": 33, 2024-03-04T09:41:29.190 INFO:teuthology.orchestra.run.smithi106.stdout: "avg_session_uptime": 598, 2024-03-04T09:41:29.190 INFO:teuthology.orchestra.run.smithi106.stdout: "mdthresh_evicted": 0 2024-03-04T09:41:29.190 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-03-04T09:41:29.190 INFO:teuthology.orchestra.run.smithi106.stdout: "mempool": { 2024-03-04T09:41:29.190 INFO:teuthology.orchestra.run.smithi106.stdout: "bloom_filter_bytes": 0, 2024-03-04T09:41:29.190 INFO:teuthology.orchestra.run.smithi106.stdout: "bloom_filter_items": 0, 2024-03-04T09:41:29.190 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_alloc_bytes": 0, 2024-03-04T09:41:29.190 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_alloc_items": 0, 2024-03-04T09:41:29.191 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_cache_data_bytes": 0, 2024-03-04T09:41:29.191 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_cache_data_items": 0, 2024-03-04T09:41:29.191 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-04T09:41:29.191 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_cache_onode_items": 0, 2024-03-04T09:41:29.191 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-04T09:41:29.191 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_cache_meta_items": 0, 2024-03-04T09:41:29.191 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_cache_other_bytes": 0, 2024-03-04T09:41:29.191 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_cache_other_items": 0, 2024-03-04T09:41:29.191 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-04T09:41:29.192 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_cache_buffer_items": 0, 2024-03-04T09:41:29.192 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_extent_bytes": 0, 2024-03-04T09:41:29.192 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_extent_items": 0, 2024-03-04T09:41:29.192 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_blob_bytes": 0, 2024-03-04T09:41:29.192 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_blob_items": 0, 2024-03-04T09:41:29.195 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-04T09:41:29.195 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_shared_blob_items": 0, 2024-03-04T09:41:29.195 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-04T09:41:29.195 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_inline_bl_items": 0, 2024-03-04T09:41:29.196 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_fsck_bytes": 0, 2024-03-04T09:41:29.196 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_fsck_items": 0, 2024-03-04T09:41:29.196 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_txc_bytes": 0, 2024-03-04T09:41:29.196 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_txc_items": 0, 2024-03-04T09:41:29.196 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-04T09:41:29.196 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_writing_deferred_items": 0, 2024-03-04T09:41:29.196 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_writing_bytes": 0, 2024-03-04T09:41:29.196 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_writing_items": 0, 2024-03-04T09:41:29.197 INFO:teuthology.orchestra.run.smithi106.stdout: "bluefs_bytes": 0, 2024-03-04T09:41:29.197 INFO:teuthology.orchestra.run.smithi106.stdout: "bluefs_items": 0, 2024-03-04T09:41:29.197 INFO:teuthology.orchestra.run.smithi106.stdout: "bluefs_file_reader_bytes": 0, 2024-03-04T09:41:29.197 INFO:teuthology.orchestra.run.smithi106.stdout: "bluefs_file_reader_items": 0, 2024-03-04T09:41:29.197 INFO:teuthology.orchestra.run.smithi106.stdout: "bluefs_file_writer_bytes": 0, 2024-03-04T09:41:29.197 INFO:teuthology.orchestra.run.smithi106.stdout: "bluefs_file_writer_items": 0, 2024-03-04T09:41:29.197 INFO:teuthology.orchestra.run.smithi106.stdout: "buffer_anon_bytes": 517469, 2024-03-04T09:41:29.197 INFO:teuthology.orchestra.run.smithi106.stdout: "buffer_anon_items": 5425, 2024-03-04T09:41:29.198 INFO:teuthology.orchestra.run.smithi106.stdout: "buffer_meta_bytes": 968, 2024-03-04T09:41:29.198 INFO:teuthology.orchestra.run.smithi106.stdout: "buffer_meta_items": 11, 2024-03-04T09:41:29.198 INFO:teuthology.orchestra.run.smithi106.stdout: "osd_bytes": 0, 2024-03-04T09:41:29.198 INFO:teuthology.orchestra.run.smithi106.stdout: "osd_items": 0, 2024-03-04T09:41:29.198 INFO:teuthology.orchestra.run.smithi106.stdout: "osd_mapbl_bytes": 0, 2024-03-04T09:41:29.198 INFO:teuthology.orchestra.run.smithi106.stdout: "osd_mapbl_items": 0, 2024-03-04T09:41:29.198 INFO:teuthology.orchestra.run.smithi106.stdout: "osd_pglog_bytes": 0, 2024-03-04T09:41:29.198 INFO:teuthology.orchestra.run.smithi106.stdout: "osd_pglog_items": 0, 2024-03-04T09:41:29.198 INFO:teuthology.orchestra.run.smithi106.stdout: "osdmap_bytes": 9744, 2024-03-04T09:41:29.199 INFO:teuthology.orchestra.run.smithi106.stdout: "osdmap_items": 204, 2024-03-04T09:41:29.199 INFO:teuthology.orchestra.run.smithi106.stdout: "osdmap_mapping_bytes": 0, 2024-03-04T09:41:29.199 INFO:teuthology.orchestra.run.smithi106.stdout: "osdmap_mapping_items": 0, 2024-03-04T09:41:29.199 INFO:teuthology.orchestra.run.smithi106.stdout: "pgmap_bytes": 0, 2024-03-04T09:41:29.199 INFO:teuthology.orchestra.run.smithi106.stdout: "pgmap_items": 0, 2024-03-04T09:41:29.199 INFO:teuthology.orchestra.run.smithi106.stdout: "mds_co_bytes": 18784164, 2024-03-04T09:41:29.199 INFO:teuthology.orchestra.run.smithi106.stdout: "mds_co_items": 208706, 2024-03-04T09:41:29.199 INFO:teuthology.orchestra.run.smithi106.stdout: "unittest_1_bytes": 0, 2024-03-04T09:41:29.199 INFO:teuthology.orchestra.run.smithi106.stdout: "unittest_1_items": 0, 2024-03-04T09:41:29.200 INFO:teuthology.orchestra.run.smithi106.stdout: "unittest_2_bytes": 0, 2024-03-04T09:41:29.200 INFO:teuthology.orchestra.run.smithi106.stdout: "unittest_2_items": 0 2024-03-04T09:41:29.200 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-03-04T09:41:29.200 INFO:teuthology.orchestra.run.smithi106.stdout: "objecter": { 2024-03-04T09:41:29.200 INFO:teuthology.orchestra.run.smithi106.stdout: "op_active": 0, 2024-03-04T09:41:29.200 INFO:teuthology.orchestra.run.smithi106.stdout: "op_laggy": 0, 2024-03-04T09:41:29.200 INFO:teuthology.orchestra.run.smithi106.stdout: "op_send": 7811, 2024-03-04T09:41:29.200 INFO:teuthology.orchestra.run.smithi106.stdout: "op_send_bytes": 56687996, 2024-03-04T09:41:29.201 INFO:teuthology.orchestra.run.smithi106.stdout: "op_resend": 31, 2024-03-04T09:41:29.201 INFO:teuthology.orchestra.run.smithi106.stdout: "op_reply": 7811, 2024-03-04T09:41:29.201 INFO:teuthology.orchestra.run.smithi106.stdout: "op_latency": { 2024-03-04T09:41:29.201 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 7811, 2024-03-04T09:41:29.201 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 57.720644386, 2024-03-04T09:41:29.201 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.007389661 2024-03-04T09:41:29.201 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-03-04T09:41:29.201 INFO:teuthology.orchestra.run.smithi106.stdout: "op_inflight": 0, 2024-03-04T09:41:29.201 INFO:teuthology.orchestra.run.smithi106.stdout: "oplen_avg": { 2024-03-04T09:41:29.202 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 7811, 2024-03-04T09:41:29.202 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 8095 2024-03-04T09:41:29.202 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-03-04T09:41:29.202 INFO:teuthology.orchestra.run.smithi106.stdout: "op": 7811, 2024-03-04T09:41:29.202 INFO:teuthology.orchestra.run.smithi106.stdout: "op_r": 43, 2024-03-04T09:41:29.202 INFO:teuthology.orchestra.run.smithi106.stdout: "op_w": 7768, 2024-03-04T09:41:29.202 INFO:teuthology.orchestra.run.smithi106.stdout: "op_rmw": 0, 2024-03-04T09:41:29.202 INFO:teuthology.orchestra.run.smithi106.stdout: "op_pg": 0, 2024-03-04T09:41:29.202 INFO:teuthology.orchestra.run.smithi106.stdout: "osdop_stat": 0, 2024-03-04T09:41:29.203 INFO:teuthology.orchestra.run.smithi106.stdout: "osdop_create": 11, 2024-03-04T09:41:29.203 INFO:teuthology.orchestra.run.smithi106.stdout: "osdop_read": 43, 2024-03-04T09:41:29.203 INFO:teuthology.orchestra.run.smithi106.stdout: "osdop_write": 491, 2024-03-04T09:41:29.203 INFO:teuthology.orchestra.run.smithi106.stdout: "osdop_writefull": 46, 2024-03-04T09:41:29.203 INFO:teuthology.orchestra.run.smithi106.stdout: "osdop_writesame": 0, 2024-03-04T09:41:29.203 INFO:teuthology.orchestra.run.smithi106.stdout: "osdop_append": 0, 2024-03-04T09:41:29.203 INFO:teuthology.orchestra.run.smithi106.stdout: "osdop_zero": 0, 2024-03-04T09:41:29.203 INFO:teuthology.orchestra.run.smithi106.stdout: "osdop_truncate": 6, 2024-03-04T09:41:29.204 INFO:teuthology.orchestra.run.smithi106.stdout: "osdop_delete": 6955, 2024-03-04T09:41:29.204 INFO:teuthology.orchestra.run.smithi106.stdout: "osdop_mapext": 0, 2024-03-04T09:41:29.204 INFO:teuthology.orchestra.run.smithi106.stdout: "osdop_sparse_read": 0, 2024-03-04T09:41:29.204 INFO:teuthology.orchestra.run.smithi106.stdout: "osdop_clonerange": 0, 2024-03-04T09:41:29.204 INFO:teuthology.orchestra.run.smithi106.stdout: "osdop_getxattr": 0, 2024-03-04T09:41:29.204 INFO:teuthology.orchestra.run.smithi106.stdout: "osdop_setxattr": 22, 2024-03-04T09:41:29.204 INFO:teuthology.orchestra.run.smithi106.stdout: "osdop_cmpxattr": 0, 2024-03-04T09:41:29.204 INFO:teuthology.orchestra.run.smithi106.stdout: "osdop_rmxattr": 0, 2024-03-04T09:41:29.205 INFO:teuthology.orchestra.run.smithi106.stdout: "osdop_resetxattrs": 0, 2024-03-04T09:41:29.205 INFO:teuthology.orchestra.run.smithi106.stdout: "osdop_call": 0, 2024-03-04T09:41:29.205 INFO:teuthology.orchestra.run.smithi106.stdout: "osdop_watch": 0, 2024-03-04T09:41:29.205 INFO:teuthology.orchestra.run.smithi106.stdout: "osdop_notify": 0, 2024-03-04T09:41:29.205 INFO:teuthology.orchestra.run.smithi106.stdout: "osdop_src_cmpxattr": 0, 2024-03-04T09:41:29.205 INFO:teuthology.orchestra.run.smithi106.stdout: "osdop_pgls": 0, 2024-03-04T09:41:29.205 INFO:teuthology.orchestra.run.smithi106.stdout: "osdop_pgls_filter": 0, 2024-03-04T09:41:29.205 INFO:teuthology.orchestra.run.smithi106.stdout: "osdop_other": 8, 2024-03-04T09:41:29.205 INFO:teuthology.orchestra.run.smithi106.stdout: "linger_active": 0, 2024-03-04T09:41:29.206 INFO:teuthology.orchestra.run.smithi106.stdout: "linger_send": 0, 2024-03-04T09:41:29.206 INFO:teuthology.orchestra.run.smithi106.stdout: "linger_resend": 0, 2024-03-04T09:41:29.206 INFO:teuthology.orchestra.run.smithi106.stdout: "linger_ping": 0, 2024-03-04T09:41:29.206 INFO:teuthology.orchestra.run.smithi106.stdout: "poolop_active": 0, 2024-03-04T09:41:29.206 INFO:teuthology.orchestra.run.smithi106.stdout: "poolop_send": 0, 2024-03-04T09:41:29.206 INFO:teuthology.orchestra.run.smithi106.stdout: "poolop_resend": 0, 2024-03-04T09:41:29.206 INFO:teuthology.orchestra.run.smithi106.stdout: "poolstat_active": 0, 2024-03-04T09:41:29.206 INFO:teuthology.orchestra.run.smithi106.stdout: "poolstat_send": 0, 2024-03-04T09:41:29.207 INFO:teuthology.orchestra.run.smithi106.stdout: "poolstat_resend": 0, 2024-03-04T09:41:29.207 INFO:teuthology.orchestra.run.smithi106.stdout: "statfs_active": 0, 2024-03-04T09:41:29.207 INFO:teuthology.orchestra.run.smithi106.stdout: "statfs_send": 0, 2024-03-04T09:41:29.207 INFO:teuthology.orchestra.run.smithi106.stdout: "statfs_resend": 0, 2024-03-04T09:41:29.207 INFO:teuthology.orchestra.run.smithi106.stdout: "command_active": 0, 2024-03-04T09:41:29.207 INFO:teuthology.orchestra.run.smithi106.stdout: "command_send": 0, 2024-03-04T09:41:29.207 INFO:teuthology.orchestra.run.smithi106.stdout: "command_resend": 0, 2024-03-04T09:41:29.207 INFO:teuthology.orchestra.run.smithi106.stdout: "map_epoch": 102, 2024-03-04T09:41:29.207 INFO:teuthology.orchestra.run.smithi106.stdout: "map_full": 0, 2024-03-04T09:41:29.208 INFO:teuthology.orchestra.run.smithi106.stdout: "map_inc": 18, 2024-03-04T09:41:29.208 INFO:teuthology.orchestra.run.smithi106.stdout: "osd_sessions": 12, 2024-03-04T09:41:29.208 INFO:teuthology.orchestra.run.smithi106.stdout: "osd_session_open": 12, 2024-03-04T09:41:29.208 INFO:teuthology.orchestra.run.smithi106.stdout: "osd_session_close": 0, 2024-03-04T09:41:29.208 INFO:teuthology.orchestra.run.smithi106.stdout: "osd_laggy": 0, 2024-03-04T09:41:29.208 INFO:teuthology.orchestra.run.smithi106.stdout: "omap_wr": 480, 2024-03-04T09:41:29.208 INFO:teuthology.orchestra.run.smithi106.stdout: "omap_rd": 0, 2024-03-04T09:41:29.208 INFO:teuthology.orchestra.run.smithi106.stdout: "omap_del": 33 2024-03-04T09:41:29.209 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-03-04T09:41:29.209 INFO:teuthology.orchestra.run.smithi106.stdout: "oft": { 2024-03-04T09:41:29.209 INFO:teuthology.orchestra.run.smithi106.stdout: "omap_total_objs": 1, 2024-03-04T09:41:29.209 INFO:teuthology.orchestra.run.smithi106.stdout: "omap_total_kv_pairs": 0, 2024-03-04T09:41:29.209 INFO:teuthology.orchestra.run.smithi106.stdout: "omap_total_updates": 0, 2024-03-04T09:41:29.209 INFO:teuthology.orchestra.run.smithi106.stdout: "omap_total_removes": 0 2024-03-04T09:41:29.209 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-03-04T09:41:29.209 INFO:teuthology.orchestra.run.smithi106.stdout: "osd-slow-ops": { 2024-03-04T09:41:29.209 INFO:teuthology.orchestra.run.smithi106.stdout: "slow_ops_count": 0 2024-03-04T09:41:29.210 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-03-04T09:41:29.210 INFO:teuthology.orchestra.run.smithi106.stdout: "purge_queue": { 2024-03-04T09:41:29.210 INFO:teuthology.orchestra.run.smithi106.stdout: "pq_executing_ops": 0, 2024-03-04T09:41:29.210 INFO:teuthology.orchestra.run.smithi106.stdout: "pq_executing_ops_high_water": 17, 2024-03-04T09:41:29.210 INFO:teuthology.orchestra.run.smithi106.stdout: "pq_executing": 0, 2024-03-04T09:41:29.210 INFO:teuthology.orchestra.run.smithi106.stdout: "pq_executing_high_water": 10, 2024-03-04T09:41:29.210 INFO:teuthology.orchestra.run.smithi106.stdout: "pq_executed_ops": 6415, 2024-03-04T09:41:29.210 INFO:teuthology.orchestra.run.smithi106.stdout: "pq_executed": 3215, 2024-03-04T09:41:29.211 INFO:teuthology.orchestra.run.smithi106.stdout: "pq_item_in_journal": 0 2024-03-04T09:41:29.211 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-03-04T09:41:29.211 INFO:teuthology.orchestra.run.smithi106.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-04T09:41:29.211 INFO:teuthology.orchestra.run.smithi106.stdout: "val": 0, 2024-03-04T09:41:29.211 INFO:teuthology.orchestra.run.smithi106.stdout: "max": 104857600, 2024-03-04T09:41:29.211 INFO:teuthology.orchestra.run.smithi106.stdout: "get_started": 0, 2024-03-04T09:41:29.211 INFO:teuthology.orchestra.run.smithi106.stdout: "get": 29293, 2024-03-04T09:41:29.211 INFO:teuthology.orchestra.run.smithi106.stdout: "get_sum": 8351704, 2024-03-04T09:41:29.212 INFO:teuthology.orchestra.run.smithi106.stdout: "get_or_fail_fail": 0, 2024-03-04T09:41:29.212 INFO:teuthology.orchestra.run.smithi106.stdout: "get_or_fail_success": 29293, 2024-03-04T09:41:29.212 INFO:teuthology.orchestra.run.smithi106.stdout: "take": 0, 2024-03-04T09:41:29.212 INFO:teuthology.orchestra.run.smithi106.stdout: "take_sum": 0, 2024-03-04T09:41:29.212 INFO:teuthology.orchestra.run.smithi106.stdout: "put": 29293, 2024-03-04T09:41:29.212 INFO:teuthology.orchestra.run.smithi106.stdout: "put_sum": 8351704, 2024-03-04T09:41:29.212 INFO:teuthology.orchestra.run.smithi106.stdout: "wait": { 2024-03-04T09:41:29.212 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 0, 2024-03-04T09:41:29.213 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.000000000, 2024-03-04T09:41:29.213 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000000000 2024-03-04T09:41:29.213 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-03-04T09:41:29.213 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-03-04T09:41:29.213 INFO:teuthology.orchestra.run.smithi106.stdout: "throttle-objecter_bytes": { 2024-03-04T09:41:29.213 INFO:teuthology.orchestra.run.smithi106.stdout: "val": 0, 2024-03-04T09:41:29.213 INFO:teuthology.orchestra.run.smithi106.stdout: "max": 104857600, 2024-03-04T09:41:29.213 INFO:teuthology.orchestra.run.smithi106.stdout: "get_started": 0, 2024-03-04T09:41:29.213 INFO:teuthology.orchestra.run.smithi106.stdout: "get": 0, 2024-03-04T09:41:29.214 INFO:teuthology.orchestra.run.smithi106.stdout: "get_sum": 0, 2024-03-04T09:41:29.214 INFO:teuthology.orchestra.run.smithi106.stdout: "get_or_fail_fail": 0, 2024-03-04T09:41:29.214 INFO:teuthology.orchestra.run.smithi106.stdout: "get_or_fail_success": 0, 2024-03-04T09:41:29.214 INFO:teuthology.orchestra.run.smithi106.stdout: "take": 7811, 2024-03-04T09:41:29.214 INFO:teuthology.orchestra.run.smithi106.stdout: "take_sum": 57053623, 2024-03-04T09:41:29.214 INFO:teuthology.orchestra.run.smithi106.stdout: "put": 7811, 2024-03-04T09:41:29.214 INFO:teuthology.orchestra.run.smithi106.stdout: "put_sum": 57053623, 2024-03-04T09:41:29.214 INFO:teuthology.orchestra.run.smithi106.stdout: "wait": { 2024-03-04T09:41:29.215 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 0, 2024-03-04T09:41:29.215 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.000000000, 2024-03-04T09:41:29.215 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000000000 2024-03-04T09:41:29.215 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-03-04T09:41:29.215 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-03-04T09:41:29.215 INFO:teuthology.orchestra.run.smithi106.stdout: "throttle-objecter_ops": { 2024-03-04T09:41:29.215 INFO:teuthology.orchestra.run.smithi106.stdout: "val": 0, 2024-03-04T09:41:29.215 INFO:teuthology.orchestra.run.smithi106.stdout: "max": 1024, 2024-03-04T09:41:29.216 INFO:teuthology.orchestra.run.smithi106.stdout: "get_started": 0, 2024-03-04T09:41:29.216 INFO:teuthology.orchestra.run.smithi106.stdout: "get": 0, 2024-03-04T09:41:29.216 INFO:teuthology.orchestra.run.smithi106.stdout: "get_sum": 0, 2024-03-04T09:41:29.216 INFO:teuthology.orchestra.run.smithi106.stdout: "get_or_fail_fail": 0, 2024-03-04T09:41:29.216 INFO:teuthology.orchestra.run.smithi106.stdout: "get_or_fail_success": 0, 2024-03-04T09:41:29.216 INFO:teuthology.orchestra.run.smithi106.stdout: "take": 7811, 2024-03-04T09:41:29.216 INFO:teuthology.orchestra.run.smithi106.stdout: "take_sum": 7811, 2024-03-04T09:41:29.216 INFO:teuthology.orchestra.run.smithi106.stdout: "put": 7811, 2024-03-04T09:41:29.217 INFO:teuthology.orchestra.run.smithi106.stdout: "put_sum": 7811, 2024-03-04T09:41:29.217 INFO:teuthology.orchestra.run.smithi106.stdout: "wait": { 2024-03-04T09:41:29.217 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 0, 2024-03-04T09:41:29.217 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.000000000, 2024-03-04T09:41:29.217 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000000000 2024-03-04T09:41:29.217 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-03-04T09:41:29.217 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-03-04T09:41:29.217 INFO:teuthology.orchestra.run.smithi106.stdout: "throttle-write_buf_throttle": { 2024-03-04T09:41:29.218 INFO:teuthology.orchestra.run.smithi106.stdout: "val": 0, 2024-03-04T09:41:29.218 INFO:teuthology.orchestra.run.smithi106.stdout: "max": 3758096384, 2024-03-04T09:41:29.218 INFO:teuthology.orchestra.run.smithi106.stdout: "get_started": 0, 2024-03-04T09:41:29.218 INFO:teuthology.orchestra.run.smithi106.stdout: "get": 3215, 2024-03-04T09:41:29.218 INFO:teuthology.orchestra.run.smithi106.stdout: "get_sum": 365627, 2024-03-04T09:41:29.218 INFO:teuthology.orchestra.run.smithi106.stdout: "get_or_fail_fail": 0, 2024-03-04T09:41:29.218 INFO:teuthology.orchestra.run.smithi106.stdout: "get_or_fail_success": 3215, 2024-03-04T09:41:29.218 INFO:teuthology.orchestra.run.smithi106.stdout: "take": 0, 2024-03-04T09:41:29.218 INFO:teuthology.orchestra.run.smithi106.stdout: "take_sum": 0, 2024-03-04T09:41:29.219 INFO:teuthology.orchestra.run.smithi106.stdout: "put": 43, 2024-03-04T09:41:29.219 INFO:teuthology.orchestra.run.smithi106.stdout: "put_sum": 365627, 2024-03-04T09:41:29.219 INFO:teuthology.orchestra.run.smithi106.stdout: "wait": { 2024-03-04T09:41:29.219 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 0, 2024-03-04T09:41:29.219 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.000000000, 2024-03-04T09:41:29.219 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000000000 2024-03-04T09:41:29.219 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-03-04T09:41:29.219 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-03-04T09:41:29.219 INFO:teuthology.orchestra.run.smithi106.stdout: "throttle-write_buf_throttle-0x557e6d63e6e0": { 2024-03-04T09:41:29.220 INFO:teuthology.orchestra.run.smithi106.stdout: "val": 0, 2024-03-04T09:41:29.220 INFO:teuthology.orchestra.run.smithi106.stdout: "max": 3758096384, 2024-03-04T09:41:29.220 INFO:teuthology.orchestra.run.smithi106.stdout: "get_started": 0, 2024-03-04T09:41:29.220 INFO:teuthology.orchestra.run.smithi106.stdout: "get": 22598, 2024-03-04T09:41:29.220 INFO:teuthology.orchestra.run.smithi106.stdout: "get_sum": 52617858, 2024-03-04T09:41:29.220 INFO:teuthology.orchestra.run.smithi106.stdout: "get_or_fail_fail": 0, 2024-03-04T09:41:29.220 INFO:teuthology.orchestra.run.smithi106.stdout: "get_or_fail_success": 22598, 2024-03-04T09:41:29.220 INFO:teuthology.orchestra.run.smithi106.stdout: "take": 0, 2024-03-04T09:41:29.221 INFO:teuthology.orchestra.run.smithi106.stdout: "take_sum": 0, 2024-03-04T09:41:29.221 INFO:teuthology.orchestra.run.smithi106.stdout: "put": 448, 2024-03-04T09:41:29.221 INFO:teuthology.orchestra.run.smithi106.stdout: "put_sum": 52617858, 2024-03-04T09:41:29.221 INFO:teuthology.orchestra.run.smithi106.stdout: "wait": { 2024-03-04T09:41:29.221 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 0, 2024-03-04T09:41:29.221 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.000000000, 2024-03-04T09:41:29.221 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000000000 2024-03-04T09:41:29.221 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-03-04T09:41:29.221 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-03-04T09:41:29.222 INFO:teuthology.orchestra.run.smithi106.stdout:} 2024-03-04T09:41:29.382 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:41:29 smithi106 ceph-mon[36484]: Adjusting osd_memory_target on smithi052 to 177.2M 2024-03-04T09:41:29.382 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:41:29 smithi106 ceph-mon[36484]: Unable to set osd_memory_target on smithi052 to 185897062: error parsing value: Value '185897062' is below minimum 939524096 2024-03-04T09:41:29.585 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:41:29 smithi052 ceph-mon[32473]: Adjusting osd_memory_target on smithi052 to 177.2M 2024-03-04T09:41:29.586 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:41:29 smithi052 ceph-mon[32473]: Unable to set osd_memory_target on smithi052 to 185897062: error parsing value: Value '185897062' is below minimum 939524096 2024-03-04T09:41:29.587 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:41:29 smithi191 ceph-mon[36816]: Adjusting osd_memory_target on smithi052 to 177.2M 2024-03-04T09:41:29.587 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:41:29 smithi191 ceph-mon[36816]: Unable to set osd_memory_target on smithi052 to 185897062: error parsing value: Value '185897062' is below minimum 939524096 2024-03-04T09:41:29.871 INFO:tasks.check_counter:Daemon mds.b mds.exported=11 2024-03-04T09:41:29.871 INFO:tasks.check_counter:Daemon mds.b mds.imported=0 2024-03-04T09:41:29.871 INFO:tasks.check_counter:Daemon mds.b mds.root_rsnaps=6 2024-03-04T09:41:29.872 INFO:tasks.check_counter:Daemon mds.b mds_server.req_mksnap_latency.avgcount=10 2024-03-04T09:41:29.872 INFO:tasks.check_counter:Daemon mds.b mds_server.req_rmsnap_latency.avgcount=4 2024-03-04T09:41:29.872 INFO:tasks.check_counter:Daemon mds.b mds.dir_split=22 2024-03-04T09:41:29.872 DEBUG:tasks.check_counter:Getting stats from e 2024-03-04T09:41:29.872 DEBUG:teuthology.orchestra.run.smithi106:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd shell --fsid 4e059f08-da08-11ee-95c6-87774f69a715 -- ceph daemon mds.e perf dump 2024-03-04T09:41:30.147 INFO:teuthology.orchestra.run.smithi106.stderr:Inferring config /var/lib/ceph/4e059f08-da08-11ee-95c6-87774f69a715/mon.b/config 2024-03-04T09:41:30.434 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:41:30 smithi106 ceph-mon[36484]: pgmap v321: 129 pgs: 129 active+clean; 78 MiB data, 691 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-04T09:41:30.586 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:41:30 smithi052 ceph-mon[32473]: pgmap v321: 129 pgs: 129 active+clean; 78 MiB data, 691 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-04T09:41:30.587 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:41:30 smithi191 ceph-mon[36816]: pgmap v321: 129 pgs: 129 active+clean; 78 MiB data, 691 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-04T09:41:31.943 INFO:teuthology.orchestra.run.smithi106.stdout:{ 2024-03-04T09:41:31.943 INFO:teuthology.orchestra.run.smithi106.stdout: "AsyncMessenger::Worker-0": { 2024-03-04T09:41:31.944 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_recv_messages": 763, 2024-03-04T09:41:31.944 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_send_messages": 765, 2024-03-04T09:41:31.944 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_recv_bytes": 340139, 2024-03-04T09:41:31.944 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_send_bytes": 242698, 2024-03-04T09:41:31.944 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_created_connections": 6, 2024-03-04T09:41:31.944 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_active_connections": 2, 2024-03-04T09:41:31.944 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_running_total_time": 0.048195466, 2024-03-04T09:41:31.945 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_running_send_time": 0.017833310, 2024-03-04T09:41:31.945 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_running_recv_time": 0.011827114, 2024-03-04T09:41:31.945 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_running_fast_dispatch_time": 0.013624795, 2024-03-04T09:41:31.945 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_send_messages_queue_lat": { 2024-03-04T09:41:31.945 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 763, 2024-03-04T09:41:31.945 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.016813961, 2024-03-04T09:41:31.945 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000022036 2024-03-04T09:41:31.945 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-03-04T09:41:31.946 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_handle_ack_lat": { 2024-03-04T09:41:31.946 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 0, 2024-03-04T09:41:31.946 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.000000000, 2024-03-04T09:41:31.946 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000000000 2024-03-04T09:41:31.946 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-03-04T09:41:31.946 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_recv_encrypted_bytes": 0, 2024-03-04T09:41:31.946 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_send_encrypted_bytes": 0 2024-03-04T09:41:31.946 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-03-04T09:41:31.947 INFO:teuthology.orchestra.run.smithi106.stdout: "AsyncMessenger::Worker-1": { 2024-03-04T09:41:31.947 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_recv_messages": 689, 2024-03-04T09:41:31.947 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_send_messages": 849, 2024-03-04T09:41:31.947 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_recv_bytes": 171384, 2024-03-04T09:41:31.947 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_send_bytes": 658143, 2024-03-04T09:41:31.947 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_created_connections": 7, 2024-03-04T09:41:31.947 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_active_connections": 5, 2024-03-04T09:41:31.948 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_running_total_time": 0.053085886, 2024-03-04T09:41:31.948 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_running_send_time": 0.016658511, 2024-03-04T09:41:31.948 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_running_recv_time": 0.013831669, 2024-03-04T09:41:31.948 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_running_fast_dispatch_time": 0.017717212, 2024-03-04T09:41:31.948 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_send_messages_queue_lat": { 2024-03-04T09:41:31.948 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 847, 2024-03-04T09:41:31.949 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.026932330, 2024-03-04T09:41:31.949 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000031797 2024-03-04T09:41:31.949 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-03-04T09:41:31.949 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_handle_ack_lat": { 2024-03-04T09:41:31.949 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 0, 2024-03-04T09:41:31.949 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.000000000, 2024-03-04T09:41:31.949 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000000000 2024-03-04T09:41:31.949 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-03-04T09:41:31.950 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_recv_encrypted_bytes": 3344, 2024-03-04T09:41:31.950 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_send_encrypted_bytes": 440944 2024-03-04T09:41:31.950 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-03-04T09:41:31.950 INFO:teuthology.orchestra.run.smithi106.stdout: "AsyncMessenger::Worker-2": { 2024-03-04T09:41:31.950 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_recv_messages": 994, 2024-03-04T09:41:31.950 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_send_messages": 980, 2024-03-04T09:41:31.950 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_recv_bytes": 1724496, 2024-03-04T09:41:31.951 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_send_bytes": 369354, 2024-03-04T09:41:31.951 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_created_connections": 3, 2024-03-04T09:41:31.951 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_active_connections": 2, 2024-03-04T09:41:31.951 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_running_total_time": 0.086906410, 2024-03-04T09:41:31.951 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_running_send_time": 0.024158910, 2024-03-04T09:41:31.952 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_running_recv_time": 0.045590030, 2024-03-04T09:41:31.952 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_running_fast_dispatch_time": 0.022871632, 2024-03-04T09:41:31.952 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_send_messages_queue_lat": { 2024-03-04T09:41:31.952 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 979, 2024-03-04T09:41:31.952 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.029393664, 2024-03-04T09:41:31.952 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000030024 2024-03-04T09:41:31.952 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-03-04T09:41:31.953 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_handle_ack_lat": { 2024-03-04T09:41:31.953 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 0, 2024-03-04T09:41:31.953 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.000000000, 2024-03-04T09:41:31.953 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000000000 2024-03-04T09:41:31.953 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-03-04T09:41:31.953 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_recv_encrypted_bytes": 1473504, 2024-03-04T09:41:31.953 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_send_encrypted_bytes": 131808 2024-03-04T09:41:31.953 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-03-04T09:41:31.954 INFO:teuthology.orchestra.run.smithi106.stdout: "cct": { 2024-03-04T09:41:31.954 INFO:teuthology.orchestra.run.smithi106.stdout: "total_workers": 1, 2024-03-04T09:41:31.954 INFO:teuthology.orchestra.run.smithi106.stdout: "unhealthy_workers": 0 2024-03-04T09:41:31.954 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-03-04T09:41:31.954 INFO:teuthology.orchestra.run.smithi106.stdout: "finisher-MDSRank": { 2024-03-04T09:41:31.954 INFO:teuthology.orchestra.run.smithi106.stdout: "queue_len": 0, 2024-03-04T09:41:31.954 INFO:teuthology.orchestra.run.smithi106.stdout: "complete_latency": { 2024-03-04T09:41:31.954 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 3664, 2024-03-04T09:41:31.955 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.074630372, 2024-03-04T09:41:31.955 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000020368 2024-03-04T09:41:31.955 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-03-04T09:41:31.955 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-03-04T09:41:31.955 INFO:teuthology.orchestra.run.smithi106.stdout: "finisher-PurgeQueue": { 2024-03-04T09:41:31.955 INFO:teuthology.orchestra.run.smithi106.stdout: "queue_len": 0, 2024-03-04T09:41:31.955 INFO:teuthology.orchestra.run.smithi106.stdout: "complete_latency": { 2024-03-04T09:41:31.955 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 0, 2024-03-04T09:41:31.955 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.000000000, 2024-03-04T09:41:31.956 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000000000 2024-03-04T09:41:31.956 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-03-04T09:41:31.956 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-03-04T09:41:31.956 INFO:teuthology.orchestra.run.smithi106.stdout: "mds": { 2024-03-04T09:41:31.956 INFO:teuthology.orchestra.run.smithi106.stdout: "request": 0, 2024-03-04T09:41:31.956 INFO:teuthology.orchestra.run.smithi106.stdout: "reply": 0, 2024-03-04T09:41:31.956 INFO:teuthology.orchestra.run.smithi106.stdout: "reply_latency": { 2024-03-04T09:41:31.957 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 0, 2024-03-04T09:41:31.957 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.000000000, 2024-03-04T09:41:31.957 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000000000 2024-03-04T09:41:31.957 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-03-04T09:41:31.957 INFO:teuthology.orchestra.run.smithi106.stdout: "slow_reply": 0, 2024-03-04T09:41:31.957 INFO:teuthology.orchestra.run.smithi106.stdout: "forward": 0, 2024-03-04T09:41:31.957 INFO:teuthology.orchestra.run.smithi106.stdout: "dir_fetch_complete": 0, 2024-03-04T09:41:31.957 INFO:teuthology.orchestra.run.smithi106.stdout: "dir_fetch_keys": 0, 2024-03-04T09:41:31.957 INFO:teuthology.orchestra.run.smithi106.stdout: "dir_commit": 0, 2024-03-04T09:41:31.958 INFO:teuthology.orchestra.run.smithi106.stdout: "dir_split": 0, 2024-03-04T09:41:31.958 INFO:teuthology.orchestra.run.smithi106.stdout: "dir_merge": 0, 2024-03-04T09:41:31.958 INFO:teuthology.orchestra.run.smithi106.stdout: "inodes": 45, 2024-03-04T09:41:31.958 INFO:teuthology.orchestra.run.smithi106.stdout: "inodes_top": 3, 2024-03-04T09:41:31.958 INFO:teuthology.orchestra.run.smithi106.stdout: "inodes_bottom": 42, 2024-03-04T09:41:31.958 INFO:teuthology.orchestra.run.smithi106.stdout: "inodes_pin_tail": 0, 2024-03-04T09:41:31.958 INFO:teuthology.orchestra.run.smithi106.stdout: "inodes_pinned": 40, 2024-03-04T09:41:31.958 INFO:teuthology.orchestra.run.smithi106.stdout: "inodes_expired": 0, 2024-03-04T09:41:31.959 INFO:teuthology.orchestra.run.smithi106.stdout: "inodes_with_caps": 0, 2024-03-04T09:41:31.959 INFO:teuthology.orchestra.run.smithi106.stdout: "caps": 0, 2024-03-04T09:41:31.959 INFO:teuthology.orchestra.run.smithi106.stdout: "subtrees": 4, 2024-03-04T09:41:31.959 INFO:teuthology.orchestra.run.smithi106.stdout: "traverse": 0, 2024-03-04T09:41:31.959 INFO:teuthology.orchestra.run.smithi106.stdout: "traverse_hit": 0, 2024-03-04T09:41:31.959 INFO:teuthology.orchestra.run.smithi106.stdout: "traverse_forward": 0, 2024-03-04T09:41:31.959 INFO:teuthology.orchestra.run.smithi106.stdout: "traverse_discover": 0, 2024-03-04T09:41:31.960 INFO:teuthology.orchestra.run.smithi106.stdout: "traverse_dir_fetch": 0, 2024-03-04T09:41:31.960 INFO:teuthology.orchestra.run.smithi106.stdout: "traverse_remote_ino": 0, 2024-03-04T09:41:31.960 INFO:teuthology.orchestra.run.smithi106.stdout: "traverse_lock": 0, 2024-03-04T09:41:31.960 INFO:teuthology.orchestra.run.smithi106.stdout: "load_cent": 0, 2024-03-04T09:41:31.960 INFO:teuthology.orchestra.run.smithi106.stdout: "q": 0, 2024-03-04T09:41:31.960 INFO:teuthology.orchestra.run.smithi106.stdout: "exported": 0, 2024-03-04T09:41:31.960 INFO:teuthology.orchestra.run.smithi106.stdout: "exported_inodes": 0, 2024-03-04T09:41:31.961 INFO:teuthology.orchestra.run.smithi106.stdout: "imported": 0, 2024-03-04T09:41:31.961 INFO:teuthology.orchestra.run.smithi106.stdout: "imported_inodes": 0, 2024-03-04T09:41:31.961 INFO:teuthology.orchestra.run.smithi106.stdout: "openino_dir_fetch": 0, 2024-03-04T09:41:31.961 INFO:teuthology.orchestra.run.smithi106.stdout: "openino_backtrace_fetch": 0, 2024-03-04T09:41:31.961 INFO:teuthology.orchestra.run.smithi106.stdout: "openino_peer_discover": 0, 2024-03-04T09:41:31.961 INFO:teuthology.orchestra.run.smithi106.stdout: "root_rfiles": 0, 2024-03-04T09:41:31.961 INFO:teuthology.orchestra.run.smithi106.stdout: "root_rbytes": 0, 2024-03-04T09:41:31.961 INFO:teuthology.orchestra.run.smithi106.stdout: "root_rsnaps": 0, 2024-03-04T09:41:31.962 INFO:teuthology.orchestra.run.smithi106.stdout: "scrub_backtrace_fetch": 0, 2024-03-04T09:41:31.962 INFO:teuthology.orchestra.run.smithi106.stdout: "scrub_set_tag": 0, 2024-03-04T09:41:31.962 INFO:teuthology.orchestra.run.smithi106.stdout: "scrub_backtrace_repaired": 0, 2024-03-04T09:41:31.962 INFO:teuthology.orchestra.run.smithi106.stdout: "scrub_inotable_repaired": 0, 2024-03-04T09:41:31.962 INFO:teuthology.orchestra.run.smithi106.stdout: "scrub_dir_inodes": 0, 2024-03-04T09:41:31.962 INFO:teuthology.orchestra.run.smithi106.stdout: "scrub_dir_base_inodes": 0, 2024-03-04T09:41:31.963 INFO:teuthology.orchestra.run.smithi106.stdout: "scrub_dirfrag_rstats": 0, 2024-03-04T09:41:31.963 INFO:teuthology.orchestra.run.smithi106.stdout: "scrub_file_inodes": 0, 2024-03-04T09:41:31.963 INFO:teuthology.orchestra.run.smithi106.stdout: "handle_inode_file_caps": 0, 2024-03-04T09:41:31.963 INFO:teuthology.orchestra.run.smithi106.stdout: "ceph_cap_op_revoke": 0, 2024-03-04T09:41:31.963 INFO:teuthology.orchestra.run.smithi106.stdout: "ceph_cap_op_grant": 0, 2024-03-04T09:41:31.963 INFO:teuthology.orchestra.run.smithi106.stdout: "ceph_cap_op_trunc": 0, 2024-03-04T09:41:31.963 INFO:teuthology.orchestra.run.smithi106.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-03-04T09:41:31.963 INFO:teuthology.orchestra.run.smithi106.stdout: "ceph_cap_op_flush_ack": 0, 2024-03-04T09:41:31.964 INFO:teuthology.orchestra.run.smithi106.stdout: "handle_client_caps": 0, 2024-03-04T09:41:31.964 INFO:teuthology.orchestra.run.smithi106.stdout: "handle_client_caps_dirty": 0, 2024-03-04T09:41:31.964 INFO:teuthology.orchestra.run.smithi106.stdout: "handle_client_cap_release": 0, 2024-03-04T09:41:31.964 INFO:teuthology.orchestra.run.smithi106.stdout: "process_request_cap_release": 0 2024-03-04T09:41:31.964 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-03-04T09:41:31.964 INFO:teuthology.orchestra.run.smithi106.stdout: "mds_cache": { 2024-03-04T09:41:31.964 INFO:teuthology.orchestra.run.smithi106.stdout: "dir_update": 0, 2024-03-04T09:41:31.965 INFO:teuthology.orchestra.run.smithi106.stdout: "dir_update_receipt": 0, 2024-03-04T09:41:31.965 INFO:teuthology.orchestra.run.smithi106.stdout: "dir_try_discover": 0, 2024-03-04T09:41:31.965 INFO:teuthology.orchestra.run.smithi106.stdout: "dir_send_discover": 0, 2024-03-04T09:41:31.965 INFO:teuthology.orchestra.run.smithi106.stdout: "dir_handle_discover": 0, 2024-03-04T09:41:31.965 INFO:teuthology.orchestra.run.smithi106.stdout: "num_strays": 0, 2024-03-04T09:41:31.965 INFO:teuthology.orchestra.run.smithi106.stdout: "num_strays_delayed": 0, 2024-03-04T09:41:31.965 INFO:teuthology.orchestra.run.smithi106.stdout: "num_strays_enqueuing": 0, 2024-03-04T09:41:31.966 INFO:teuthology.orchestra.run.smithi106.stdout: "strays_created": 0, 2024-03-04T09:41:31.966 INFO:teuthology.orchestra.run.smithi106.stdout: "strays_enqueued": 0, 2024-03-04T09:41:31.966 INFO:teuthology.orchestra.run.smithi106.stdout: "strays_reintegrated": 0, 2024-03-04T09:41:31.966 INFO:teuthology.orchestra.run.smithi106.stdout: "strays_migrated": 0, 2024-03-04T09:41:31.966 INFO:teuthology.orchestra.run.smithi106.stdout: "num_recovering_processing": 0, 2024-03-04T09:41:31.966 INFO:teuthology.orchestra.run.smithi106.stdout: "num_recovering_enqueued": 0, 2024-03-04T09:41:31.967 INFO:teuthology.orchestra.run.smithi106.stdout: "num_recovering_prioritized": 0, 2024-03-04T09:41:31.967 INFO:teuthology.orchestra.run.smithi106.stdout: "recovery_started": 0, 2024-03-04T09:41:31.967 INFO:teuthology.orchestra.run.smithi106.stdout: "recovery_completed": 0, 2024-03-04T09:41:31.967 INFO:teuthology.orchestra.run.smithi106.stdout: "ireq_enqueue_scrub": 0, 2024-03-04T09:41:31.967 INFO:teuthology.orchestra.run.smithi106.stdout: "ireq_exportdir": 0, 2024-03-04T09:41:31.967 INFO:teuthology.orchestra.run.smithi106.stdout: "ireq_flush": 0, 2024-03-04T09:41:31.967 INFO:teuthology.orchestra.run.smithi106.stdout: "ireq_fragmentdir": 0, 2024-03-04T09:41:31.968 INFO:teuthology.orchestra.run.smithi106.stdout: "ireq_fragstats": 0, 2024-03-04T09:41:31.968 INFO:teuthology.orchestra.run.smithi106.stdout: "ireq_inodestats": 0 2024-03-04T09:41:31.968 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-03-04T09:41:31.968 INFO:teuthology.orchestra.run.smithi106.stdout: "mds_log": { 2024-03-04T09:41:31.968 INFO:teuthology.orchestra.run.smithi106.stdout: "evlrg": 0, 2024-03-04T09:41:31.968 INFO:teuthology.orchestra.run.smithi106.stdout: "evadd": 0, 2024-03-04T09:41:31.968 INFO:teuthology.orchestra.run.smithi106.stdout: "evex": 0, 2024-03-04T09:41:31.968 INFO:teuthology.orchestra.run.smithi106.stdout: "evtrm": 0, 2024-03-04T09:41:31.969 INFO:teuthology.orchestra.run.smithi106.stdout: "ev": 47, 2024-03-04T09:41:31.969 INFO:teuthology.orchestra.run.smithi106.stdout: "evexg": 0, 2024-03-04T09:41:31.969 INFO:teuthology.orchestra.run.smithi106.stdout: "evexd": 0, 2024-03-04T09:41:31.969 INFO:teuthology.orchestra.run.smithi106.stdout: "segadd": 0, 2024-03-04T09:41:31.969 INFO:teuthology.orchestra.run.smithi106.stdout: "segex": 0, 2024-03-04T09:41:31.969 INFO:teuthology.orchestra.run.smithi106.stdout: "segtrm": 0, 2024-03-04T09:41:31.969 INFO:teuthology.orchestra.run.smithi106.stdout: "seg": 2, 2024-03-04T09:41:31.969 INFO:teuthology.orchestra.run.smithi106.stdout: "segmjr": 2, 2024-03-04T09:41:31.970 INFO:teuthology.orchestra.run.smithi106.stdout: "segexg": 0, 2024-03-04T09:41:31.970 INFO:teuthology.orchestra.run.smithi106.stdout: "segexd": 0, 2024-03-04T09:41:31.970 INFO:teuthology.orchestra.run.smithi106.stdout: "expos": 4194304, 2024-03-04T09:41:31.970 INFO:teuthology.orchestra.run.smithi106.stdout: "wrpos": 4334106, 2024-03-04T09:41:31.970 INFO:teuthology.orchestra.run.smithi106.stdout: "rdpos": 4333532, 2024-03-04T09:41:31.970 INFO:teuthology.orchestra.run.smithi106.stdout: "jlat": { 2024-03-04T09:41:31.970 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 0, 2024-03-04T09:41:31.970 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.000000000, 2024-03-04T09:41:31.971 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000000000 2024-03-04T09:41:31.971 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-03-04T09:41:31.971 INFO:teuthology.orchestra.run.smithi106.stdout: "replayed": 47 2024-03-04T09:41:31.971 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-03-04T09:41:31.971 INFO:teuthology.orchestra.run.smithi106.stdout: "mds_mem": { 2024-03-04T09:41:31.971 INFO:teuthology.orchestra.run.smithi106.stdout: "ino": 43, 2024-03-04T09:41:31.971 INFO:teuthology.orchestra.run.smithi106.stdout: "ino+": 43, 2024-03-04T09:41:31.971 INFO:teuthology.orchestra.run.smithi106.stdout: "ino-": 0, 2024-03-04T09:41:31.971 INFO:teuthology.orchestra.run.smithi106.stdout: "dir": 17, 2024-03-04T09:41:31.972 INFO:teuthology.orchestra.run.smithi106.stdout: "dir+": 17, 2024-03-04T09:41:31.972 INFO:teuthology.orchestra.run.smithi106.stdout: "dir-": 0, 2024-03-04T09:41:31.972 INFO:teuthology.orchestra.run.smithi106.stdout: "dn": 45, 2024-03-04T09:41:31.972 INFO:teuthology.orchestra.run.smithi106.stdout: "dn+": 45, 2024-03-04T09:41:31.973 INFO:teuthology.orchestra.run.smithi106.stdout: "dn-": 0, 2024-03-04T09:41:31.973 INFO:teuthology.orchestra.run.smithi106.stdout: "cap": 0, 2024-03-04T09:41:31.973 INFO:teuthology.orchestra.run.smithi106.stdout: "cap+": 0, 2024-03-04T09:41:31.973 INFO:teuthology.orchestra.run.smithi106.stdout: "cap-": 0, 2024-03-04T09:41:31.973 INFO:teuthology.orchestra.run.smithi106.stdout: "rss": 48144, 2024-03-04T09:41:31.973 INFO:teuthology.orchestra.run.smithi106.stdout: "heap": 215420 2024-03-04T09:41:31.973 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-03-04T09:41:31.973 INFO:teuthology.orchestra.run.smithi106.stdout: "mds_server": { 2024-03-04T09:41:31.973 INFO:teuthology.orchestra.run.smithi106.stdout: "dispatch_client_request": 0, 2024-03-04T09:41:31.974 INFO:teuthology.orchestra.run.smithi106.stdout: "dispatch_server_request": 0, 2024-03-04T09:41:31.974 INFO:teuthology.orchestra.run.smithi106.stdout: "handle_client_request": 0, 2024-03-04T09:41:31.974 INFO:teuthology.orchestra.run.smithi106.stdout: "handle_client_session": 0, 2024-03-04T09:41:31.974 INFO:teuthology.orchestra.run.smithi106.stdout: "handle_peer_request": 0, 2024-03-04T09:41:31.974 INFO:teuthology.orchestra.run.smithi106.stdout: "req_create_latency": { 2024-03-04T09:41:31.974 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 0, 2024-03-04T09:41:31.974 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.000000000, 2024-03-04T09:41:31.974 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000000000 2024-03-04T09:41:31.975 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-03-04T09:41:31.975 INFO:teuthology.orchestra.run.smithi106.stdout: "req_getattr_latency": { 2024-03-04T09:41:31.975 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 0, 2024-03-04T09:41:31.975 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.000000000, 2024-03-04T09:41:31.975 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000000000 2024-03-04T09:41:31.975 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-03-04T09:41:31.975 INFO:teuthology.orchestra.run.smithi106.stdout: "req_getfilelock_latency": { 2024-03-04T09:41:31.975 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 0, 2024-03-04T09:41:31.975 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.000000000, 2024-03-04T09:41:31.976 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000000000 2024-03-04T09:41:31.976 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-03-04T09:41:31.976 INFO:teuthology.orchestra.run.smithi106.stdout: "req_link_latency": { 2024-03-04T09:41:31.976 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 0, 2024-03-04T09:41:31.976 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.000000000, 2024-03-04T09:41:31.976 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000000000 2024-03-04T09:41:31.976 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-03-04T09:41:31.977 INFO:teuthology.orchestra.run.smithi106.stdout: "req_lookup_latency": { 2024-03-04T09:41:31.977 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 0, 2024-03-04T09:41:31.977 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.000000000, 2024-03-04T09:41:31.977 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000000000 2024-03-04T09:41:31.977 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-03-04T09:41:31.977 INFO:teuthology.orchestra.run.smithi106.stdout: "req_lookuphash_latency": { 2024-03-04T09:41:31.977 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 0, 2024-03-04T09:41:31.977 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.000000000, 2024-03-04T09:41:31.978 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000000000 2024-03-04T09:41:31.978 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-03-04T09:41:31.978 INFO:teuthology.orchestra.run.smithi106.stdout: "req_lookupino_latency": { 2024-03-04T09:41:31.978 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 0, 2024-03-04T09:41:31.978 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.000000000, 2024-03-04T09:41:31.978 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000000000 2024-03-04T09:41:31.978 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-03-04T09:41:31.978 INFO:teuthology.orchestra.run.smithi106.stdout: "req_lookupname_latency": { 2024-03-04T09:41:31.978 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 0, 2024-03-04T09:41:31.979 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.000000000, 2024-03-04T09:41:31.979 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000000000 2024-03-04T09:41:31.979 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-03-04T09:41:31.979 INFO:teuthology.orchestra.run.smithi106.stdout: "req_lookupparent_latency": { 2024-03-04T09:41:31.979 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 0, 2024-03-04T09:41:31.979 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.000000000, 2024-03-04T09:41:31.979 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000000000 2024-03-04T09:41:31.979 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-03-04T09:41:31.980 INFO:teuthology.orchestra.run.smithi106.stdout: "req_lookupsnap_latency": { 2024-03-04T09:41:31.980 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 0, 2024-03-04T09:41:31.980 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.000000000, 2024-03-04T09:41:31.980 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000000000 2024-03-04T09:41:31.980 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-03-04T09:41:31.980 INFO:teuthology.orchestra.run.smithi106.stdout: "req_lssnap_latency": { 2024-03-04T09:41:31.980 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 0, 2024-03-04T09:41:31.980 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.000000000, 2024-03-04T09:41:31.980 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000000000 2024-03-04T09:41:31.981 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-03-04T09:41:31.981 INFO:teuthology.orchestra.run.smithi106.stdout: "req_mkdir_latency": { 2024-03-04T09:41:31.981 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 0, 2024-03-04T09:41:31.981 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.000000000, 2024-03-04T09:41:31.981 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000000000 2024-03-04T09:41:31.982 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-03-04T09:41:31.982 INFO:teuthology.orchestra.run.smithi106.stdout: "req_mknod_latency": { 2024-03-04T09:41:31.982 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 0, 2024-03-04T09:41:31.982 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.000000000, 2024-03-04T09:41:31.982 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000000000 2024-03-04T09:41:31.982 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-03-04T09:41:31.983 INFO:teuthology.orchestra.run.smithi106.stdout: "req_mksnap_latency": { 2024-03-04T09:41:31.983 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 0, 2024-03-04T09:41:31.983 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.000000000, 2024-03-04T09:41:31.983 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000000000 2024-03-04T09:41:31.983 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-03-04T09:41:31.984 INFO:teuthology.orchestra.run.smithi106.stdout: "req_open_latency": { 2024-03-04T09:41:31.984 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 0, 2024-03-04T09:41:31.984 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.000000000, 2024-03-04T09:41:31.984 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000000000 2024-03-04T09:41:31.984 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-03-04T09:41:31.984 INFO:teuthology.orchestra.run.smithi106.stdout: "req_readdir_latency": { 2024-03-04T09:41:31.984 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 0, 2024-03-04T09:41:31.984 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.000000000, 2024-03-04T09:41:31.984 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000000000 2024-03-04T09:41:31.985 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-03-04T09:41:31.985 INFO:teuthology.orchestra.run.smithi106.stdout: "req_rename_latency": { 2024-03-04T09:41:31.985 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 0, 2024-03-04T09:41:31.985 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.000000000, 2024-03-04T09:41:31.985 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000000000 2024-03-04T09:41:31.985 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-03-04T09:41:31.985 INFO:teuthology.orchestra.run.smithi106.stdout: "req_renamesnap_latency": { 2024-03-04T09:41:31.985 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 0, 2024-03-04T09:41:31.986 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.000000000, 2024-03-04T09:41:31.986 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000000000 2024-03-04T09:41:31.986 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-03-04T09:41:31.986 INFO:teuthology.orchestra.run.smithi106.stdout: "req_snapdiff_latency": { 2024-03-04T09:41:31.986 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 0, 2024-03-04T09:41:31.986 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.000000000, 2024-03-04T09:41:31.986 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000000000 2024-03-04T09:41:31.986 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-03-04T09:41:31.987 INFO:teuthology.orchestra.run.smithi106.stdout: "req_rmdir_latency": { 2024-03-04T09:41:31.987 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 0, 2024-03-04T09:41:31.987 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.000000000, 2024-03-04T09:41:31.987 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000000000 2024-03-04T09:41:31.987 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-03-04T09:41:31.987 INFO:teuthology.orchestra.run.smithi106.stdout: "req_rmsnap_latency": { 2024-03-04T09:41:31.987 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 0, 2024-03-04T09:41:31.987 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.000000000, 2024-03-04T09:41:31.988 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000000000 2024-03-04T09:41:31.988 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-03-04T09:41:31.988 INFO:teuthology.orchestra.run.smithi106.stdout: "req_rmxattr_latency": { 2024-03-04T09:41:31.988 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 0, 2024-03-04T09:41:31.988 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.000000000, 2024-03-04T09:41:31.988 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000000000 2024-03-04T09:41:31.988 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-03-04T09:41:31.988 INFO:teuthology.orchestra.run.smithi106.stdout: "req_setattr_latency": { 2024-03-04T09:41:31.989 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 0, 2024-03-04T09:41:31.989 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.000000000, 2024-03-04T09:41:31.989 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000000000 2024-03-04T09:41:31.989 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-03-04T09:41:31.989 INFO:teuthology.orchestra.run.smithi106.stdout: "req_setdirlayout_latency": { 2024-03-04T09:41:31.989 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 0, 2024-03-04T09:41:31.989 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.000000000, 2024-03-04T09:41:31.989 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000000000 2024-03-04T09:41:31.989 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-03-04T09:41:31.990 INFO:teuthology.orchestra.run.smithi106.stdout: "req_setfilelock_latency": { 2024-03-04T09:41:31.990 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 0, 2024-03-04T09:41:31.990 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.000000000, 2024-03-04T09:41:31.990 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000000000 2024-03-04T09:41:31.990 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-03-04T09:41:31.990 INFO:teuthology.orchestra.run.smithi106.stdout: "req_setlayout_latency": { 2024-03-04T09:41:31.990 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 0, 2024-03-04T09:41:31.990 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.000000000, 2024-03-04T09:41:31.991 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000000000 2024-03-04T09:41:31.991 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-03-04T09:41:31.991 INFO:teuthology.orchestra.run.smithi106.stdout: "req_setxattr_latency": { 2024-03-04T09:41:31.991 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 0, 2024-03-04T09:41:31.991 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.000000000, 2024-03-04T09:41:31.991 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000000000 2024-03-04T09:41:31.991 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-03-04T09:41:31.991 INFO:teuthology.orchestra.run.smithi106.stdout: "req_symlink_latency": { 2024-03-04T09:41:31.991 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 0, 2024-03-04T09:41:31.992 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.000000000, 2024-03-04T09:41:31.992 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000000000 2024-03-04T09:41:31.992 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-03-04T09:41:31.992 INFO:teuthology.orchestra.run.smithi106.stdout: "req_unlink_latency": { 2024-03-04T09:41:31.992 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 0, 2024-03-04T09:41:31.993 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.000000000, 2024-03-04T09:41:31.993 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000000000 2024-03-04T09:41:31.993 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-03-04T09:41:31.993 INFO:teuthology.orchestra.run.smithi106.stdout: "cap_revoke_eviction": 0, 2024-03-04T09:41:31.994 INFO:teuthology.orchestra.run.smithi106.stdout: "cap_acquisition_throttle": 0, 2024-03-04T09:41:31.994 INFO:teuthology.orchestra.run.smithi106.stdout: "req_getvxattr_latency": { 2024-03-04T09:41:31.994 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 0, 2024-03-04T09:41:31.994 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.000000000, 2024-03-04T09:41:31.994 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000000000 2024-03-04T09:41:31.995 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-03-04T09:41:31.995 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-03-04T09:41:31.995 INFO:teuthology.orchestra.run.smithi106.stdout: "mds_sessions": { 2024-03-04T09:41:31.995 INFO:teuthology.orchestra.run.smithi106.stdout: "session_count": 1, 2024-03-04T09:41:31.995 INFO:teuthology.orchestra.run.smithi106.stdout: "session_add": 4, 2024-03-04T09:41:31.995 INFO:teuthology.orchestra.run.smithi106.stdout: "session_remove": 3, 2024-03-04T09:41:31.995 INFO:teuthology.orchestra.run.smithi106.stdout: "sessions_open": 2, 2024-03-04T09:41:31.995 INFO:teuthology.orchestra.run.smithi106.stdout: "sessions_stale": 0, 2024-03-04T09:41:31.996 INFO:teuthology.orchestra.run.smithi106.stdout: "total_load": 0, 2024-03-04T09:41:31.996 INFO:teuthology.orchestra.run.smithi106.stdout: "average_load": 0, 2024-03-04T09:41:31.996 INFO:teuthology.orchestra.run.smithi106.stdout: "avg_session_uptime": 6141, 2024-03-04T09:41:31.996 INFO:teuthology.orchestra.run.smithi106.stdout: "mdthresh_evicted": 0 2024-03-04T09:41:31.996 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-03-04T09:41:31.996 INFO:teuthology.orchestra.run.smithi106.stdout: "mempool": { 2024-03-04T09:41:31.996 INFO:teuthology.orchestra.run.smithi106.stdout: "bloom_filter_bytes": 0, 2024-03-04T09:41:31.997 INFO:teuthology.orchestra.run.smithi106.stdout: "bloom_filter_items": 0, 2024-03-04T09:41:31.997 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_alloc_bytes": 0, 2024-03-04T09:41:31.997 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_alloc_items": 0, 2024-03-04T09:41:31.997 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_cache_data_bytes": 0, 2024-03-04T09:41:31.997 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_cache_data_items": 0, 2024-03-04T09:41:31.997 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-04T09:41:31.997 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_cache_onode_items": 0, 2024-03-04T09:41:31.997 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-04T09:41:31.998 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_cache_meta_items": 0, 2024-03-04T09:41:31.998 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_cache_other_bytes": 0, 2024-03-04T09:41:31.998 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_cache_other_items": 0, 2024-03-04T09:41:31.998 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-04T09:41:31.998 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_cache_buffer_items": 0, 2024-03-04T09:41:31.998 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_extent_bytes": 0, 2024-03-04T09:41:31.998 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_extent_items": 0, 2024-03-04T09:41:31.998 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_blob_bytes": 0, 2024-03-04T09:41:31.999 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_blob_items": 0, 2024-03-04T09:41:31.999 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-04T09:41:31.999 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_shared_blob_items": 0, 2024-03-04T09:41:31.999 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-04T09:41:31.999 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_inline_bl_items": 0, 2024-03-04T09:41:31.999 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_fsck_bytes": 0, 2024-03-04T09:41:31.999 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_fsck_items": 0, 2024-03-04T09:41:31.999 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_txc_bytes": 0, 2024-03-04T09:41:31.999 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_txc_items": 0, 2024-03-04T09:41:32.000 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-04T09:41:32.000 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_writing_deferred_items": 0, 2024-03-04T09:41:32.000 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_writing_bytes": 0, 2024-03-04T09:41:32.000 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_writing_items": 0, 2024-03-04T09:41:32.000 INFO:teuthology.orchestra.run.smithi106.stdout: "bluefs_bytes": 0, 2024-03-04T09:41:32.000 INFO:teuthology.orchestra.run.smithi106.stdout: "bluefs_items": 0, 2024-03-04T09:41:32.000 INFO:teuthology.orchestra.run.smithi106.stdout: "bluefs_file_reader_bytes": 0, 2024-03-04T09:41:32.000 INFO:teuthology.orchestra.run.smithi106.stdout: "bluefs_file_reader_items": 0, 2024-03-04T09:41:32.001 INFO:teuthology.orchestra.run.smithi106.stdout: "bluefs_file_writer_bytes": 0, 2024-03-04T09:41:32.001 INFO:teuthology.orchestra.run.smithi106.stdout: "bluefs_file_writer_items": 0, 2024-03-04T09:41:32.001 INFO:teuthology.orchestra.run.smithi106.stdout: "buffer_anon_bytes": 190930, 2024-03-04T09:41:32.001 INFO:teuthology.orchestra.run.smithi106.stdout: "buffer_anon_items": 93, 2024-03-04T09:41:32.001 INFO:teuthology.orchestra.run.smithi106.stdout: "buffer_meta_bytes": 0, 2024-03-04T09:41:32.001 INFO:teuthology.orchestra.run.smithi106.stdout: "buffer_meta_items": 0, 2024-03-04T09:41:32.001 INFO:teuthology.orchestra.run.smithi106.stdout: "osd_bytes": 0, 2024-03-04T09:41:32.001 INFO:teuthology.orchestra.run.smithi106.stdout: "osd_items": 0, 2024-03-04T09:41:32.002 INFO:teuthology.orchestra.run.smithi106.stdout: "osd_mapbl_bytes": 0, 2024-03-04T09:41:32.002 INFO:teuthology.orchestra.run.smithi106.stdout: "osd_mapbl_items": 0, 2024-03-04T09:41:32.002 INFO:teuthology.orchestra.run.smithi106.stdout: "osd_pglog_bytes": 0, 2024-03-04T09:41:32.002 INFO:teuthology.orchestra.run.smithi106.stdout: "osd_pglog_items": 0, 2024-03-04T09:41:32.002 INFO:teuthology.orchestra.run.smithi106.stdout: "osdmap_bytes": 9744, 2024-03-04T09:41:32.002 INFO:teuthology.orchestra.run.smithi106.stdout: "osdmap_items": 204, 2024-03-04T09:41:32.002 INFO:teuthology.orchestra.run.smithi106.stdout: "osdmap_mapping_bytes": 0, 2024-03-04T09:41:32.002 INFO:teuthology.orchestra.run.smithi106.stdout: "osdmap_mapping_items": 0, 2024-03-04T09:41:32.002 INFO:teuthology.orchestra.run.smithi106.stdout: "pgmap_bytes": 0, 2024-03-04T09:41:32.003 INFO:teuthology.orchestra.run.smithi106.stdout: "pgmap_items": 0, 2024-03-04T09:41:32.003 INFO:teuthology.orchestra.run.smithi106.stdout: "mds_co_bytes": 155103, 2024-03-04T09:41:32.003 INFO:teuthology.orchestra.run.smithi106.stdout: "mds_co_items": 1304, 2024-03-04T09:41:32.003 INFO:teuthology.orchestra.run.smithi106.stdout: "unittest_1_bytes": 0, 2024-03-04T09:41:32.003 INFO:teuthology.orchestra.run.smithi106.stdout: "unittest_1_items": 0, 2024-03-04T09:41:32.003 INFO:teuthology.orchestra.run.smithi106.stdout: "unittest_2_bytes": 0, 2024-03-04T09:41:32.003 INFO:teuthology.orchestra.run.smithi106.stdout: "unittest_2_items": 0 2024-03-04T09:41:32.004 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-03-04T09:41:32.004 INFO:teuthology.orchestra.run.smithi106.stdout: "objecter": { 2024-03-04T09:41:32.004 INFO:teuthology.orchestra.run.smithi106.stdout: "op_active": 0, 2024-03-04T09:41:32.004 INFO:teuthology.orchestra.run.smithi106.stdout: "op_laggy": 0, 2024-03-04T09:41:32.004 INFO:teuthology.orchestra.run.smithi106.stdout: "op_send": 2193, 2024-03-04T09:41:32.005 INFO:teuthology.orchestra.run.smithi106.stdout: "op_send_bytes": 21, 2024-03-04T09:41:32.005 INFO:teuthology.orchestra.run.smithi106.stdout: "op_resend": 3, 2024-03-04T09:41:32.005 INFO:teuthology.orchestra.run.smithi106.stdout: "op_reply": 2193, 2024-03-04T09:41:32.005 INFO:teuthology.orchestra.run.smithi106.stdout: "op_latency": { 2024-03-04T09:41:32.005 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 2193, 2024-03-04T09:41:32.005 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 1.095974683, 2024-03-04T09:41:32.005 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000499760 2024-03-04T09:41:32.005 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-03-04T09:41:32.006 INFO:teuthology.orchestra.run.smithi106.stdout: "op_inflight": 0, 2024-03-04T09:41:32.006 INFO:teuthology.orchestra.run.smithi106.stdout: "oplen_avg": { 2024-03-04T09:41:32.006 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 2193, 2024-03-04T09:41:32.006 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 2194 2024-03-04T09:41:32.006 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-03-04T09:41:32.006 INFO:teuthology.orchestra.run.smithi106.stdout: "op": 2193, 2024-03-04T09:41:32.006 INFO:teuthology.orchestra.run.smithi106.stdout: "op_r": 2193, 2024-03-04T09:41:32.006 INFO:teuthology.orchestra.run.smithi106.stdout: "op_w": 0, 2024-03-04T09:41:32.007 INFO:teuthology.orchestra.run.smithi106.stdout: "op_rmw": 0, 2024-03-04T09:41:32.007 INFO:teuthology.orchestra.run.smithi106.stdout: "op_pg": 0, 2024-03-04T09:41:32.007 INFO:teuthology.orchestra.run.smithi106.stdout: "osdop_stat": 1428, 2024-03-04T09:41:32.007 INFO:teuthology.orchestra.run.smithi106.stdout: "osdop_create": 0, 2024-03-04T09:41:32.007 INFO:teuthology.orchestra.run.smithi106.stdout: "osdop_read": 763, 2024-03-04T09:41:32.007 INFO:teuthology.orchestra.run.smithi106.stdout: "osdop_write": 0, 2024-03-04T09:41:32.007 INFO:teuthology.orchestra.run.smithi106.stdout: "osdop_writefull": 0, 2024-03-04T09:41:32.007 INFO:teuthology.orchestra.run.smithi106.stdout: "osdop_writesame": 0, 2024-03-04T09:41:32.007 INFO:teuthology.orchestra.run.smithi106.stdout: "osdop_append": 0, 2024-03-04T09:41:32.008 INFO:teuthology.orchestra.run.smithi106.stdout: "osdop_zero": 0, 2024-03-04T09:41:32.008 INFO:teuthology.orchestra.run.smithi106.stdout: "osdop_truncate": 0, 2024-03-04T09:41:32.008 INFO:teuthology.orchestra.run.smithi106.stdout: "osdop_delete": 0, 2024-03-04T09:41:32.008 INFO:teuthology.orchestra.run.smithi106.stdout: "osdop_mapext": 0, 2024-03-04T09:41:32.008 INFO:teuthology.orchestra.run.smithi106.stdout: "osdop_sparse_read": 0, 2024-03-04T09:41:32.008 INFO:teuthology.orchestra.run.smithi106.stdout: "osdop_clonerange": 0, 2024-03-04T09:41:32.008 INFO:teuthology.orchestra.run.smithi106.stdout: "osdop_getxattr": 1, 2024-03-04T09:41:32.008 INFO:teuthology.orchestra.run.smithi106.stdout: "osdop_setxattr": 0, 2024-03-04T09:41:32.009 INFO:teuthology.orchestra.run.smithi106.stdout: "osdop_cmpxattr": 0, 2024-03-04T09:41:32.009 INFO:teuthology.orchestra.run.smithi106.stdout: "osdop_rmxattr": 0, 2024-03-04T09:41:32.009 INFO:teuthology.orchestra.run.smithi106.stdout: "osdop_resetxattrs": 0, 2024-03-04T09:41:32.009 INFO:teuthology.orchestra.run.smithi106.stdout: "osdop_call": 0, 2024-03-04T09:41:32.009 INFO:teuthology.orchestra.run.smithi106.stdout: "osdop_watch": 0, 2024-03-04T09:41:32.009 INFO:teuthology.orchestra.run.smithi106.stdout: "osdop_notify": 0, 2024-03-04T09:41:32.009 INFO:teuthology.orchestra.run.smithi106.stdout: "osdop_src_cmpxattr": 0, 2024-03-04T09:41:32.009 INFO:teuthology.orchestra.run.smithi106.stdout: "osdop_pgls": 0, 2024-03-04T09:41:32.009 INFO:teuthology.orchestra.run.smithi106.stdout: "osdop_pgls_filter": 0, 2024-03-04T09:41:32.010 INFO:teuthology.orchestra.run.smithi106.stdout: "osdop_other": 0, 2024-03-04T09:41:32.010 INFO:teuthology.orchestra.run.smithi106.stdout: "linger_active": 0, 2024-03-04T09:41:32.010 INFO:teuthology.orchestra.run.smithi106.stdout: "linger_send": 0, 2024-03-04T09:41:32.010 INFO:teuthology.orchestra.run.smithi106.stdout: "linger_resend": 0, 2024-03-04T09:41:32.010 INFO:teuthology.orchestra.run.smithi106.stdout: "linger_ping": 0, 2024-03-04T09:41:32.010 INFO:teuthology.orchestra.run.smithi106.stdout: "poolop_active": 0, 2024-03-04T09:41:32.010 INFO:teuthology.orchestra.run.smithi106.stdout: "poolop_send": 0, 2024-03-04T09:41:32.010 INFO:teuthology.orchestra.run.smithi106.stdout: "poolop_resend": 0, 2024-03-04T09:41:32.011 INFO:teuthology.orchestra.run.smithi106.stdout: "poolstat_active": 0, 2024-03-04T09:41:32.011 INFO:teuthology.orchestra.run.smithi106.stdout: "poolstat_send": 0, 2024-03-04T09:41:32.011 INFO:teuthology.orchestra.run.smithi106.stdout: "poolstat_resend": 0, 2024-03-04T09:41:32.011 INFO:teuthology.orchestra.run.smithi106.stdout: "statfs_active": 0, 2024-03-04T09:41:32.011 INFO:teuthology.orchestra.run.smithi106.stdout: "statfs_send": 0, 2024-03-04T09:41:32.011 INFO:teuthology.orchestra.run.smithi106.stdout: "statfs_resend": 0, 2024-03-04T09:41:32.011 INFO:teuthology.orchestra.run.smithi106.stdout: "command_active": 0, 2024-03-04T09:41:32.011 INFO:teuthology.orchestra.run.smithi106.stdout: "command_send": 0, 2024-03-04T09:41:32.011 INFO:teuthology.orchestra.run.smithi106.stdout: "command_resend": 0, 2024-03-04T09:41:32.012 INFO:teuthology.orchestra.run.smithi106.stdout: "map_epoch": 102, 2024-03-04T09:41:32.012 INFO:teuthology.orchestra.run.smithi106.stdout: "map_full": 0, 2024-03-04T09:41:32.012 INFO:teuthology.orchestra.run.smithi106.stdout: "map_inc": 13, 2024-03-04T09:41:32.012 INFO:teuthology.orchestra.run.smithi106.stdout: "osd_sessions": 5, 2024-03-04T09:41:32.012 INFO:teuthology.orchestra.run.smithi106.stdout: "osd_session_open": 5, 2024-03-04T09:41:32.012 INFO:teuthology.orchestra.run.smithi106.stdout: "osd_session_close": 0, 2024-03-04T09:41:32.012 INFO:teuthology.orchestra.run.smithi106.stdout: "osd_laggy": 0, 2024-03-04T09:41:32.013 INFO:teuthology.orchestra.run.smithi106.stdout: "omap_wr": 0, 2024-03-04T09:41:32.013 INFO:teuthology.orchestra.run.smithi106.stdout: "omap_rd": 2, 2024-03-04T09:41:32.013 INFO:teuthology.orchestra.run.smithi106.stdout: "omap_del": 0 2024-03-04T09:41:32.013 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-03-04T09:41:32.013 INFO:teuthology.orchestra.run.smithi106.stdout: "oft": { 2024-03-04T09:41:32.013 INFO:teuthology.orchestra.run.smithi106.stdout: "omap_total_objs": 0, 2024-03-04T09:41:32.013 INFO:teuthology.orchestra.run.smithi106.stdout: "omap_total_kv_pairs": 0, 2024-03-04T09:41:32.013 INFO:teuthology.orchestra.run.smithi106.stdout: "omap_total_updates": 0, 2024-03-04T09:41:32.013 INFO:teuthology.orchestra.run.smithi106.stdout: "omap_total_removes": 0 2024-03-04T09:41:32.014 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-03-04T09:41:32.014 INFO:teuthology.orchestra.run.smithi106.stdout: "osd-slow-ops": { 2024-03-04T09:41:32.014 INFO:teuthology.orchestra.run.smithi106.stdout: "slow_ops_count": 0 2024-03-04T09:41:32.014 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-03-04T09:41:32.014 INFO:teuthology.orchestra.run.smithi106.stdout: "purge_queue": { 2024-03-04T09:41:32.014 INFO:teuthology.orchestra.run.smithi106.stdout: "pq_executing_ops": 0, 2024-03-04T09:41:32.014 INFO:teuthology.orchestra.run.smithi106.stdout: "pq_executing_ops_high_water": 0, 2024-03-04T09:41:32.014 INFO:teuthology.orchestra.run.smithi106.stdout: "pq_executing": 0, 2024-03-04T09:41:32.015 INFO:teuthology.orchestra.run.smithi106.stdout: "pq_executing_high_water": 0, 2024-03-04T09:41:32.015 INFO:teuthology.orchestra.run.smithi106.stdout: "pq_executed_ops": 0, 2024-03-04T09:41:32.015 INFO:teuthology.orchestra.run.smithi106.stdout: "pq_executed": 0, 2024-03-04T09:41:32.015 INFO:teuthology.orchestra.run.smithi106.stdout: "pq_item_in_journal": 0 2024-03-04T09:41:32.015 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-03-04T09:41:32.016 INFO:teuthology.orchestra.run.smithi106.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-04T09:41:32.016 INFO:teuthology.orchestra.run.smithi106.stdout: "val": 0, 2024-03-04T09:41:32.016 INFO:teuthology.orchestra.run.smithi106.stdout: "max": 104857600, 2024-03-04T09:41:32.016 INFO:teuthology.orchestra.run.smithi106.stdout: "get_started": 0, 2024-03-04T09:41:32.016 INFO:teuthology.orchestra.run.smithi106.stdout: "get": 2442, 2024-03-04T09:41:32.016 INFO:teuthology.orchestra.run.smithi106.stdout: "get_sum": 2000571, 2024-03-04T09:41:32.016 INFO:teuthology.orchestra.run.smithi106.stdout: "get_or_fail_fail": 0, 2024-03-04T09:41:32.016 INFO:teuthology.orchestra.run.smithi106.stdout: "get_or_fail_success": 2442, 2024-03-04T09:41:32.017 INFO:teuthology.orchestra.run.smithi106.stdout: "take": 0, 2024-03-04T09:41:32.017 INFO:teuthology.orchestra.run.smithi106.stdout: "take_sum": 0, 2024-03-04T09:41:32.017 INFO:teuthology.orchestra.run.smithi106.stdout: "put": 2442, 2024-03-04T09:41:32.017 INFO:teuthology.orchestra.run.smithi106.stdout: "put_sum": 2000571, 2024-03-04T09:41:32.017 INFO:teuthology.orchestra.run.smithi106.stdout: "wait": { 2024-03-04T09:41:32.017 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 0, 2024-03-04T09:41:32.017 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.000000000, 2024-03-04T09:41:32.017 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000000000 2024-03-04T09:41:32.018 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-03-04T09:41:32.018 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-03-04T09:41:32.018 INFO:teuthology.orchestra.run.smithi106.stdout: "throttle-objecter_bytes": { 2024-03-04T09:41:32.018 INFO:teuthology.orchestra.run.smithi106.stdout: "val": 0, 2024-03-04T09:41:32.018 INFO:teuthology.orchestra.run.smithi106.stdout: "max": 104857600, 2024-03-04T09:41:32.018 INFO:teuthology.orchestra.run.smithi106.stdout: "get_started": 0, 2024-03-04T09:41:32.018 INFO:teuthology.orchestra.run.smithi106.stdout: "get": 0, 2024-03-04T09:41:32.018 INFO:teuthology.orchestra.run.smithi106.stdout: "get_sum": 0, 2024-03-04T09:41:32.018 INFO:teuthology.orchestra.run.smithi106.stdout: "get_or_fail_fail": 0, 2024-03-04T09:41:32.019 INFO:teuthology.orchestra.run.smithi106.stdout: "get_or_fail_success": 0, 2024-03-04T09:41:32.019 INFO:teuthology.orchestra.run.smithi106.stdout: "take": 2193, 2024-03-04T09:41:32.019 INFO:teuthology.orchestra.run.smithi106.stdout: "take_sum": 139807, 2024-03-04T09:41:32.019 INFO:teuthology.orchestra.run.smithi106.stdout: "put": 2193, 2024-03-04T09:41:32.019 INFO:teuthology.orchestra.run.smithi106.stdout: "put_sum": 139807, 2024-03-04T09:41:32.019 INFO:teuthology.orchestra.run.smithi106.stdout: "wait": { 2024-03-04T09:41:32.019 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 0, 2024-03-04T09:41:32.019 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.000000000, 2024-03-04T09:41:32.019 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000000000 2024-03-04T09:41:32.020 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-03-04T09:41:32.020 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-03-04T09:41:32.020 INFO:teuthology.orchestra.run.smithi106.stdout: "throttle-objecter_ops": { 2024-03-04T09:41:32.020 INFO:teuthology.orchestra.run.smithi106.stdout: "val": 0, 2024-03-04T09:41:32.020 INFO:teuthology.orchestra.run.smithi106.stdout: "max": 1024, 2024-03-04T09:41:32.020 INFO:teuthology.orchestra.run.smithi106.stdout: "get_started": 0, 2024-03-04T09:41:32.020 INFO:teuthology.orchestra.run.smithi106.stdout: "get": 0, 2024-03-04T09:41:32.020 INFO:teuthology.orchestra.run.smithi106.stdout: "get_sum": 0, 2024-03-04T09:41:32.021 INFO:teuthology.orchestra.run.smithi106.stdout: "get_or_fail_fail": 0, 2024-03-04T09:41:32.021 INFO:teuthology.orchestra.run.smithi106.stdout: "get_or_fail_success": 0, 2024-03-04T09:41:32.021 INFO:teuthology.orchestra.run.smithi106.stdout: "take": 2193, 2024-03-04T09:41:32.021 INFO:teuthology.orchestra.run.smithi106.stdout: "take_sum": 2193, 2024-03-04T09:41:32.021 INFO:teuthology.orchestra.run.smithi106.stdout: "put": 2193, 2024-03-04T09:41:32.021 INFO:teuthology.orchestra.run.smithi106.stdout: "put_sum": 2193, 2024-03-04T09:41:32.021 INFO:teuthology.orchestra.run.smithi106.stdout: "wait": { 2024-03-04T09:41:32.021 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 0, 2024-03-04T09:41:32.022 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.000000000, 2024-03-04T09:41:32.022 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000000000 2024-03-04T09:41:32.022 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-03-04T09:41:32.022 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-03-04T09:41:32.022 INFO:teuthology.orchestra.run.smithi106.stdout: "throttle-write_buf_throttle": { 2024-03-04T09:41:32.022 INFO:teuthology.orchestra.run.smithi106.stdout: "val": 0, 2024-03-04T09:41:32.022 INFO:teuthology.orchestra.run.smithi106.stdout: "max": 3758096384, 2024-03-04T09:41:32.022 INFO:teuthology.orchestra.run.smithi106.stdout: "get_started": 0, 2024-03-04T09:41:32.023 INFO:teuthology.orchestra.run.smithi106.stdout: "get": 0, 2024-03-04T09:41:32.023 INFO:teuthology.orchestra.run.smithi106.stdout: "get_sum": 0, 2024-03-04T09:41:32.023 INFO:teuthology.orchestra.run.smithi106.stdout: "get_or_fail_fail": 0, 2024-03-04T09:41:32.023 INFO:teuthology.orchestra.run.smithi106.stdout: "get_or_fail_success": 0, 2024-03-04T09:41:32.023 INFO:teuthology.orchestra.run.smithi106.stdout: "take": 0, 2024-03-04T09:41:32.024 INFO:teuthology.orchestra.run.smithi106.stdout: "take_sum": 0, 2024-03-04T09:41:32.024 INFO:teuthology.orchestra.run.smithi106.stdout: "put": 0, 2024-03-04T09:41:32.024 INFO:teuthology.orchestra.run.smithi106.stdout: "put_sum": 0, 2024-03-04T09:41:32.024 INFO:teuthology.orchestra.run.smithi106.stdout: "wait": { 2024-03-04T09:41:32.024 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 0, 2024-03-04T09:41:32.025 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.000000000, 2024-03-04T09:41:32.025 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000000000 2024-03-04T09:41:32.025 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-03-04T09:41:32.025 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-03-04T09:41:32.025 INFO:teuthology.orchestra.run.smithi106.stdout: "throttle-write_buf_throttle-0x55bfc74d8960": { 2024-03-04T09:41:32.026 INFO:teuthology.orchestra.run.smithi106.stdout: "val": 0, 2024-03-04T09:41:32.026 INFO:teuthology.orchestra.run.smithi106.stdout: "max": 3758096384, 2024-03-04T09:41:32.026 INFO:teuthology.orchestra.run.smithi106.stdout: "get_started": 0, 2024-03-04T09:41:32.026 INFO:teuthology.orchestra.run.smithi106.stdout: "get": 0, 2024-03-04T09:41:32.026 INFO:teuthology.orchestra.run.smithi106.stdout: "get_sum": 0, 2024-03-04T09:41:32.027 INFO:teuthology.orchestra.run.smithi106.stdout: "get_or_fail_fail": 0, 2024-03-04T09:41:32.027 INFO:teuthology.orchestra.run.smithi106.stdout: "get_or_fail_success": 0, 2024-03-04T09:41:32.027 INFO:teuthology.orchestra.run.smithi106.stdout: "take": 0, 2024-03-04T09:41:32.027 INFO:teuthology.orchestra.run.smithi106.stdout: "take_sum": 0, 2024-03-04T09:41:32.027 INFO:teuthology.orchestra.run.smithi106.stdout: "put": 0, 2024-03-04T09:41:32.027 INFO:teuthology.orchestra.run.smithi106.stdout: "put_sum": 0, 2024-03-04T09:41:32.027 INFO:teuthology.orchestra.run.smithi106.stdout: "wait": { 2024-03-04T09:41:32.027 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 0, 2024-03-04T09:41:32.028 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.000000000, 2024-03-04T09:41:32.028 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000000000 2024-03-04T09:41:32.028 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-03-04T09:41:32.028 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-03-04T09:41:32.028 INFO:teuthology.orchestra.run.smithi106.stdout:} 2024-03-04T09:41:32.382 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:41:32 smithi106 ceph-mon[36484]: pgmap v322: 129 pgs: 129 active+clean; 78 MiB data, 691 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-04T09:41:32.585 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:41:32 smithi052 ceph-mon[32473]: pgmap v322: 129 pgs: 129 active+clean; 78 MiB data, 691 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-04T09:41:32.587 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:41:32 smithi191 ceph-mon[36816]: pgmap v322: 129 pgs: 129 active+clean; 78 MiB data, 691 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-04T09:41:32.742 INFO:tasks.check_counter:Daemon mds.e mds.exported=0 2024-03-04T09:41:32.742 INFO:tasks.check_counter:Daemon mds.e mds.imported=0 2024-03-04T09:41:32.742 INFO:tasks.check_counter:Daemon mds.e mds.root_rsnaps=0 2024-03-04T09:41:32.742 INFO:tasks.check_counter:Daemon mds.e mds_server.req_mksnap_latency.avgcount=0 2024-03-04T09:41:32.743 INFO:tasks.check_counter:Daemon mds.e mds_server.req_rmsnap_latency.avgcount=0 2024-03-04T09:41:32.743 INFO:tasks.check_counter:Daemon mds.e mds.dir_split=0 2024-03-04T09:41:32.743 DEBUG:tasks.check_counter:Getting stats from h 2024-03-04T09:41:32.743 DEBUG:teuthology.orchestra.run.smithi106:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd shell --fsid 4e059f08-da08-11ee-95c6-87774f69a715 -- ceph daemon mds.h perf dump 2024-03-04T09:41:33.045 INFO:teuthology.orchestra.run.smithi106.stderr:Inferring config /var/lib/ceph/4e059f08-da08-11ee-95c6-87774f69a715/mon.b/config 2024-03-04T09:41:34.201 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:41:34 smithi106 ceph-mon[36484]: pgmap v323: 129 pgs: 129 active+clean; 78 MiB data, 691 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-04T09:41:34.586 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:41:34 smithi052 ceph-mon[32473]: pgmap v323: 129 pgs: 129 active+clean; 78 MiB data, 691 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-04T09:41:34.587 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:41:34 smithi191 ceph-mon[36816]: pgmap v323: 129 pgs: 129 active+clean; 78 MiB data, 691 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-04T09:41:34.805 INFO:teuthology.orchestra.run.smithi106.stdout:{ 2024-03-04T09:41:34.805 INFO:teuthology.orchestra.run.smithi106.stdout: "AsyncMessenger::Worker-0": { 2024-03-04T09:41:34.805 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_recv_messages": 551, 2024-03-04T09:41:34.805 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_send_messages": 604, 2024-03-04T09:41:34.805 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_recv_bytes": 202234, 2024-03-04T09:41:34.805 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_send_bytes": 380791, 2024-03-04T09:41:34.806 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_created_connections": 8, 2024-03-04T09:41:34.806 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_active_connections": 2, 2024-03-04T09:41:34.806 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_running_total_time": 0.046824511, 2024-03-04T09:41:34.806 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_running_send_time": 0.020529349, 2024-03-04T09:41:34.806 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_running_recv_time": 0.022592423, 2024-03-04T09:41:34.806 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_running_fast_dispatch_time": 0.003719146, 2024-03-04T09:41:34.807 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_send_messages_queue_lat": { 2024-03-04T09:41:34.807 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 606, 2024-03-04T09:41:34.807 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.028453158, 2024-03-04T09:41:34.807 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000046952 2024-03-04T09:41:34.807 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-03-04T09:41:34.807 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_handle_ack_lat": { 2024-03-04T09:41:34.807 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 1065, 2024-03-04T09:41:34.808 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.000205292, 2024-03-04T09:41:34.808 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000000192 2024-03-04T09:41:34.808 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-03-04T09:41:34.808 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_recv_encrypted_bytes": 0, 2024-03-04T09:41:34.808 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_send_encrypted_bytes": 0 2024-03-04T09:41:34.808 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-03-04T09:41:34.808 INFO:teuthology.orchestra.run.smithi106.stdout: "AsyncMessenger::Worker-1": { 2024-03-04T09:41:34.808 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_recv_messages": 304, 2024-03-04T09:41:34.809 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_send_messages": 412, 2024-03-04T09:41:34.809 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_recv_bytes": 1492578, 2024-03-04T09:41:34.809 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_send_bytes": 466206, 2024-03-04T09:41:34.809 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_created_connections": 7, 2024-03-04T09:41:34.809 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_active_connections": 5, 2024-03-04T09:41:34.809 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_running_total_time": 0.041047094, 2024-03-04T09:41:34.809 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_running_send_time": 0.012315093, 2024-03-04T09:41:34.810 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_running_recv_time": 0.035426668, 2024-03-04T09:41:34.810 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_running_fast_dispatch_time": 0.004153571, 2024-03-04T09:41:34.810 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_send_messages_queue_lat": { 2024-03-04T09:41:34.810 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 412, 2024-03-04T09:41:34.810 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.031450346, 2024-03-04T09:41:34.810 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000076335 2024-03-04T09:41:34.810 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-03-04T09:41:34.811 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_handle_ack_lat": { 2024-03-04T09:41:34.811 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 4, 2024-03-04T09:41:34.811 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.000000403, 2024-03-04T09:41:34.811 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000000100 2024-03-04T09:41:34.811 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-03-04T09:41:34.811 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_recv_encrypted_bytes": 1486160, 2024-03-04T09:41:34.811 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_send_encrypted_bytes": 447040 2024-03-04T09:41:34.812 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-03-04T09:41:34.812 INFO:teuthology.orchestra.run.smithi106.stdout: "AsyncMessenger::Worker-2": { 2024-03-04T09:41:34.812 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_recv_messages": 86, 2024-03-04T09:41:34.812 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_send_messages": 178, 2024-03-04T09:41:34.812 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_recv_bytes": 26421, 2024-03-04T09:41:34.812 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_send_bytes": 221114, 2024-03-04T09:41:34.812 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_created_connections": 15, 2024-03-04T09:41:34.813 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_active_connections": 6, 2024-03-04T09:41:34.813 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_running_total_time": 0.014687300, 2024-03-04T09:41:34.813 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_running_send_time": 0.004448592, 2024-03-04T09:41:34.813 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_running_recv_time": 0.009324234, 2024-03-04T09:41:34.814 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_running_fast_dispatch_time": 0.000111273, 2024-03-04T09:41:34.814 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_send_messages_queue_lat": { 2024-03-04T09:41:34.814 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 171, 2024-03-04T09:41:34.814 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.014631808, 2024-03-04T09:41:34.814 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000085566 2024-03-04T09:41:34.814 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-03-04T09:41:34.814 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_handle_ack_lat": { 2024-03-04T09:41:34.815 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 146, 2024-03-04T09:41:34.815 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.000035325, 2024-03-04T09:41:34.815 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000000241 2024-03-04T09:41:34.815 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-03-04T09:41:34.815 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_recv_encrypted_bytes": 3264, 2024-03-04T09:41:34.815 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_send_encrypted_bytes": 133248 2024-03-04T09:41:34.815 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-03-04T09:41:34.816 INFO:teuthology.orchestra.run.smithi106.stdout: "cct": { 2024-03-04T09:41:34.816 INFO:teuthology.orchestra.run.smithi106.stdout: "total_workers": 1, 2024-03-04T09:41:34.816 INFO:teuthology.orchestra.run.smithi106.stdout: "unhealthy_workers": 0 2024-03-04T09:41:34.816 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-03-04T09:41:34.816 INFO:teuthology.orchestra.run.smithi106.stdout: "finisher-MDSRank": { 2024-03-04T09:41:34.816 INFO:teuthology.orchestra.run.smithi106.stdout: "queue_len": 0, 2024-03-04T09:41:34.816 INFO:teuthology.orchestra.run.smithi106.stdout: "complete_latency": { 2024-03-04T09:41:34.817 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 83, 2024-03-04T09:41:34.817 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.018236299, 2024-03-04T09:41:34.817 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000219714 2024-03-04T09:41:34.817 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-03-04T09:41:34.817 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-03-04T09:41:34.817 INFO:teuthology.orchestra.run.smithi106.stdout: "finisher-PurgeQueue": { 2024-03-04T09:41:34.817 INFO:teuthology.orchestra.run.smithi106.stdout: "queue_len": 0, 2024-03-04T09:41:34.818 INFO:teuthology.orchestra.run.smithi106.stdout: "complete_latency": { 2024-03-04T09:41:34.818 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 2, 2024-03-04T09:41:34.818 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.000007249, 2024-03-04T09:41:34.818 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000003624 2024-03-04T09:41:34.818 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-03-04T09:41:34.818 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-03-04T09:41:34.818 INFO:teuthology.orchestra.run.smithi106.stdout: "mds": { 2024-03-04T09:41:34.819 INFO:teuthology.orchestra.run.smithi106.stdout: "request": 8, 2024-03-04T09:41:34.819 INFO:teuthology.orchestra.run.smithi106.stdout: "reply": 5, 2024-03-04T09:41:34.819 INFO:teuthology.orchestra.run.smithi106.stdout: "reply_latency": { 2024-03-04T09:41:34.819 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 5, 2024-03-04T09:41:34.819 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.001778267, 2024-03-04T09:41:34.819 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000355653 2024-03-04T09:41:34.819 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-03-04T09:41:34.820 INFO:teuthology.orchestra.run.smithi106.stdout: "slow_reply": 0, 2024-03-04T09:41:34.820 INFO:teuthology.orchestra.run.smithi106.stdout: "forward": 3, 2024-03-04T09:41:34.820 INFO:teuthology.orchestra.run.smithi106.stdout: "dir_fetch_complete": 0, 2024-03-04T09:41:34.820 INFO:teuthology.orchestra.run.smithi106.stdout: "dir_fetch_keys": 0, 2024-03-04T09:41:34.820 INFO:teuthology.orchestra.run.smithi106.stdout: "dir_commit": 11, 2024-03-04T09:41:34.820 INFO:teuthology.orchestra.run.smithi106.stdout: "dir_split": 0, 2024-03-04T09:41:34.820 INFO:teuthology.orchestra.run.smithi106.stdout: "dir_merge": 0, 2024-03-04T09:41:34.821 INFO:teuthology.orchestra.run.smithi106.stdout: "inodes": 55, 2024-03-04T09:41:34.821 INFO:teuthology.orchestra.run.smithi106.stdout: "inodes_top": 3, 2024-03-04T09:41:34.821 INFO:teuthology.orchestra.run.smithi106.stdout: "inodes_bottom": 52, 2024-03-04T09:41:34.821 INFO:teuthology.orchestra.run.smithi106.stdout: "inodes_pin_tail": 0, 2024-03-04T09:41:34.821 INFO:teuthology.orchestra.run.smithi106.stdout: "inodes_pinned": 50, 2024-03-04T09:41:34.821 INFO:teuthology.orchestra.run.smithi106.stdout: "inodes_expired": 1, 2024-03-04T09:41:34.821 INFO:teuthology.orchestra.run.smithi106.stdout: "inodes_with_caps": 7, 2024-03-04T09:41:34.822 INFO:teuthology.orchestra.run.smithi106.stdout: "caps": 7, 2024-03-04T09:41:34.822 INFO:teuthology.orchestra.run.smithi106.stdout: "subtrees": 5, 2024-03-04T09:41:34.822 INFO:teuthology.orchestra.run.smithi106.stdout: "traverse": 18, 2024-03-04T09:41:34.822 INFO:teuthology.orchestra.run.smithi106.stdout: "traverse_hit": 5, 2024-03-04T09:41:34.822 INFO:teuthology.orchestra.run.smithi106.stdout: "traverse_forward": 2, 2024-03-04T09:41:34.822 INFO:teuthology.orchestra.run.smithi106.stdout: "traverse_discover": 8, 2024-03-04T09:41:34.822 INFO:teuthology.orchestra.run.smithi106.stdout: "traverse_dir_fetch": 0, 2024-03-04T09:41:34.822 INFO:teuthology.orchestra.run.smithi106.stdout: "traverse_remote_ino": 0, 2024-03-04T09:41:34.823 INFO:teuthology.orchestra.run.smithi106.stdout: "traverse_lock": 0, 2024-03-04T09:41:34.823 INFO:teuthology.orchestra.run.smithi106.stdout: "load_cent": 0, 2024-03-04T09:41:34.823 INFO:teuthology.orchestra.run.smithi106.stdout: "q": 0, 2024-03-04T09:41:34.823 INFO:teuthology.orchestra.run.smithi106.stdout: "exported": 0, 2024-03-04T09:41:34.823 INFO:teuthology.orchestra.run.smithi106.stdout: "exported_inodes": 0, 2024-03-04T09:41:34.823 INFO:teuthology.orchestra.run.smithi106.stdout: "imported": 5, 2024-03-04T09:41:34.823 INFO:teuthology.orchestra.run.smithi106.stdout: "imported_inodes": 35, 2024-03-04T09:41:34.823 INFO:teuthology.orchestra.run.smithi106.stdout: "openino_dir_fetch": 0, 2024-03-04T09:41:34.823 INFO:teuthology.orchestra.run.smithi106.stdout: "openino_backtrace_fetch": 0, 2024-03-04T09:41:34.824 INFO:teuthology.orchestra.run.smithi106.stdout: "openino_peer_discover": 0, 2024-03-04T09:41:34.824 INFO:teuthology.orchestra.run.smithi106.stdout: "root_rfiles": 4, 2024-03-04T09:41:34.824 INFO:teuthology.orchestra.run.smithi106.stdout: "root_rbytes": 234, 2024-03-04T09:41:34.824 INFO:teuthology.orchestra.run.smithi106.stdout: "root_rsnaps": 0, 2024-03-04T09:41:34.824 INFO:teuthology.orchestra.run.smithi106.stdout: "scrub_backtrace_fetch": 0, 2024-03-04T09:41:34.825 INFO:teuthology.orchestra.run.smithi106.stdout: "scrub_set_tag": 0, 2024-03-04T09:41:34.825 INFO:teuthology.orchestra.run.smithi106.stdout: "scrub_backtrace_repaired": 0, 2024-03-04T09:41:34.825 INFO:teuthology.orchestra.run.smithi106.stdout: "scrub_inotable_repaired": 0, 2024-03-04T09:41:34.825 INFO:teuthology.orchestra.run.smithi106.stdout: "scrub_dir_inodes": 0, 2024-03-04T09:41:34.825 INFO:teuthology.orchestra.run.smithi106.stdout: "scrub_dir_base_inodes": 0, 2024-03-04T09:41:34.825 INFO:teuthology.orchestra.run.smithi106.stdout: "scrub_dirfrag_rstats": 0, 2024-03-04T09:41:34.825 INFO:teuthology.orchestra.run.smithi106.stdout: "scrub_file_inodes": 0, 2024-03-04T09:41:34.825 INFO:teuthology.orchestra.run.smithi106.stdout: "handle_inode_file_caps": 0, 2024-03-04T09:41:34.826 INFO:teuthology.orchestra.run.smithi106.stdout: "ceph_cap_op_revoke": 1, 2024-03-04T09:41:34.826 INFO:teuthology.orchestra.run.smithi106.stdout: "ceph_cap_op_grant": 6, 2024-03-04T09:41:34.826 INFO:teuthology.orchestra.run.smithi106.stdout: "ceph_cap_op_trunc": 0, 2024-03-04T09:41:34.826 INFO:teuthology.orchestra.run.smithi106.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-03-04T09:41:34.826 INFO:teuthology.orchestra.run.smithi106.stdout: "ceph_cap_op_flush_ack": 0, 2024-03-04T09:41:34.826 INFO:teuthology.orchestra.run.smithi106.stdout: "handle_client_caps": 30, 2024-03-04T09:41:34.826 INFO:teuthology.orchestra.run.smithi106.stdout: "handle_client_caps_dirty": 25, 2024-03-04T09:41:34.826 INFO:teuthology.orchestra.run.smithi106.stdout: "handle_client_cap_release": 1, 2024-03-04T09:41:34.826 INFO:teuthology.orchestra.run.smithi106.stdout: "process_request_cap_release": 1 2024-03-04T09:41:34.827 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-03-04T09:41:34.827 INFO:teuthology.orchestra.run.smithi106.stdout: "mds_cache": { 2024-03-04T09:41:34.827 INFO:teuthology.orchestra.run.smithi106.stdout: "dir_update": 0, 2024-03-04T09:41:34.827 INFO:teuthology.orchestra.run.smithi106.stdout: "dir_update_receipt": 0, 2024-03-04T09:41:34.827 INFO:teuthology.orchestra.run.smithi106.stdout: "dir_try_discover": 0, 2024-03-04T09:41:34.827 INFO:teuthology.orchestra.run.smithi106.stdout: "dir_send_discover": 9, 2024-03-04T09:41:34.827 INFO:teuthology.orchestra.run.smithi106.stdout: "dir_handle_discover": 0, 2024-03-04T09:41:34.827 INFO:teuthology.orchestra.run.smithi106.stdout: "num_strays": 0, 2024-03-04T09:41:34.827 INFO:teuthology.orchestra.run.smithi106.stdout: "num_strays_delayed": 0, 2024-03-04T09:41:34.828 INFO:teuthology.orchestra.run.smithi106.stdout: "num_strays_enqueuing": 0, 2024-03-04T09:41:34.828 INFO:teuthology.orchestra.run.smithi106.stdout: "strays_created": 0, 2024-03-04T09:41:34.828 INFO:teuthology.orchestra.run.smithi106.stdout: "strays_enqueued": 0, 2024-03-04T09:41:34.828 INFO:teuthology.orchestra.run.smithi106.stdout: "strays_reintegrated": 0, 2024-03-04T09:41:34.828 INFO:teuthology.orchestra.run.smithi106.stdout: "strays_migrated": 0, 2024-03-04T09:41:34.828 INFO:teuthology.orchestra.run.smithi106.stdout: "num_recovering_processing": 0, 2024-03-04T09:41:34.828 INFO:teuthology.orchestra.run.smithi106.stdout: "num_recovering_enqueued": 0, 2024-03-04T09:41:34.828 INFO:teuthology.orchestra.run.smithi106.stdout: "num_recovering_prioritized": 0, 2024-03-04T09:41:34.829 INFO:teuthology.orchestra.run.smithi106.stdout: "recovery_started": 0, 2024-03-04T09:41:34.829 INFO:teuthology.orchestra.run.smithi106.stdout: "recovery_completed": 0, 2024-03-04T09:41:34.829 INFO:teuthology.orchestra.run.smithi106.stdout: "ireq_enqueue_scrub": 0, 2024-03-04T09:41:34.829 INFO:teuthology.orchestra.run.smithi106.stdout: "ireq_exportdir": 0, 2024-03-04T09:41:34.829 INFO:teuthology.orchestra.run.smithi106.stdout: "ireq_flush": 0, 2024-03-04T09:41:34.829 INFO:teuthology.orchestra.run.smithi106.stdout: "ireq_fragmentdir": 0, 2024-03-04T09:41:34.829 INFO:teuthology.orchestra.run.smithi106.stdout: "ireq_fragstats": 0, 2024-03-04T09:41:34.829 INFO:teuthology.orchestra.run.smithi106.stdout: "ireq_inodestats": 0 2024-03-04T09:41:34.829 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-03-04T09:41:34.830 INFO:teuthology.orchestra.run.smithi106.stdout: "mds_log": { 2024-03-04T09:41:34.830 INFO:teuthology.orchestra.run.smithi106.stdout: "evlrg": 0, 2024-03-04T09:41:34.830 INFO:teuthology.orchestra.run.smithi106.stdout: "evadd": 47, 2024-03-04T09:41:34.830 INFO:teuthology.orchestra.run.smithi106.stdout: "evex": 0, 2024-03-04T09:41:34.830 INFO:teuthology.orchestra.run.smithi106.stdout: "evtrm": 0, 2024-03-04T09:41:34.830 INFO:teuthology.orchestra.run.smithi106.stdout: "ev": 47, 2024-03-04T09:41:34.830 INFO:teuthology.orchestra.run.smithi106.stdout: "evexg": 0, 2024-03-04T09:41:34.830 INFO:teuthology.orchestra.run.smithi106.stdout: "evexd": 0, 2024-03-04T09:41:34.831 INFO:teuthology.orchestra.run.smithi106.stdout: "segadd": 2, 2024-03-04T09:41:34.831 INFO:teuthology.orchestra.run.smithi106.stdout: "segex": 0, 2024-03-04T09:41:34.831 INFO:teuthology.orchestra.run.smithi106.stdout: "segtrm": 0, 2024-03-04T09:41:34.831 INFO:teuthology.orchestra.run.smithi106.stdout: "seg": 2, 2024-03-04T09:41:34.831 INFO:teuthology.orchestra.run.smithi106.stdout: "segmjr": 2, 2024-03-04T09:41:34.831 INFO:teuthology.orchestra.run.smithi106.stdout: "segexg": 0, 2024-03-04T09:41:34.831 INFO:teuthology.orchestra.run.smithi106.stdout: "segexd": 0, 2024-03-04T09:41:34.831 INFO:teuthology.orchestra.run.smithi106.stdout: "expos": 4194304, 2024-03-04T09:41:34.832 INFO:teuthology.orchestra.run.smithi106.stdout: "wrpos": 4334106, 2024-03-04T09:41:34.832 INFO:teuthology.orchestra.run.smithi106.stdout: "rdpos": 0, 2024-03-04T09:41:34.832 INFO:teuthology.orchestra.run.smithi106.stdout: "jlat": { 2024-03-04T09:41:34.832 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 17, 2024-03-04T09:41:34.832 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.132253442, 2024-03-04T09:41:34.832 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.007779614 2024-03-04T09:41:34.832 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-03-04T09:41:34.833 INFO:teuthology.orchestra.run.smithi106.stdout: "replayed": 0 2024-03-04T09:41:34.833 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-03-04T09:41:34.833 INFO:teuthology.orchestra.run.smithi106.stdout: "mds_mem": { 2024-03-04T09:41:34.833 INFO:teuthology.orchestra.run.smithi106.stdout: "ino": 53, 2024-03-04T09:41:34.833 INFO:teuthology.orchestra.run.smithi106.stdout: "ino+": 53, 2024-03-04T09:41:34.833 INFO:teuthology.orchestra.run.smithi106.stdout: "ino-": 0, 2024-03-04T09:41:34.833 INFO:teuthology.orchestra.run.smithi106.stdout: "dir": 27, 2024-03-04T09:41:34.834 INFO:teuthology.orchestra.run.smithi106.stdout: "dir+": 28, 2024-03-04T09:41:34.834 INFO:teuthology.orchestra.run.smithi106.stdout: "dir-": 1, 2024-03-04T09:41:34.834 INFO:teuthology.orchestra.run.smithi106.stdout: "dn": 55, 2024-03-04T09:41:34.834 INFO:teuthology.orchestra.run.smithi106.stdout: "dn+": 56, 2024-03-04T09:41:34.834 INFO:teuthology.orchestra.run.smithi106.stdout: "dn-": 1, 2024-03-04T09:41:34.834 INFO:teuthology.orchestra.run.smithi106.stdout: "cap": 7, 2024-03-04T09:41:34.834 INFO:teuthology.orchestra.run.smithi106.stdout: "cap+": 34, 2024-03-04T09:41:34.834 INFO:teuthology.orchestra.run.smithi106.stdout: "cap-": 27, 2024-03-04T09:41:34.835 INFO:teuthology.orchestra.run.smithi106.stdout: "rss": 50984, 2024-03-04T09:41:34.835 INFO:teuthology.orchestra.run.smithi106.stdout: "heap": 223612 2024-03-04T09:41:34.835 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-03-04T09:41:34.835 INFO:teuthology.orchestra.run.smithi106.stdout: "mds_server": { 2024-03-04T09:41:34.835 INFO:teuthology.orchestra.run.smithi106.stdout: "dispatch_client_request": 10, 2024-03-04T09:41:34.835 INFO:teuthology.orchestra.run.smithi106.stdout: "dispatch_server_request": 0, 2024-03-04T09:41:34.835 INFO:teuthology.orchestra.run.smithi106.stdout: "handle_client_request": 8, 2024-03-04T09:41:34.836 INFO:teuthology.orchestra.run.smithi106.stdout: "handle_client_session": 19, 2024-03-04T09:41:34.836 INFO:teuthology.orchestra.run.smithi106.stdout: "handle_peer_request": 0, 2024-03-04T09:41:34.836 INFO:teuthology.orchestra.run.smithi106.stdout: "req_create_latency": { 2024-03-04T09:41:34.836 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 0, 2024-03-04T09:41:34.836 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.000000000, 2024-03-04T09:41:34.836 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000000000 2024-03-04T09:41:34.837 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-03-04T09:41:34.837 INFO:teuthology.orchestra.run.smithi106.stdout: "req_getattr_latency": { 2024-03-04T09:41:34.837 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 2, 2024-03-04T09:41:34.837 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.001129338, 2024-03-04T09:41:34.837 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000564669 2024-03-04T09:41:34.837 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-03-04T09:41:34.837 INFO:teuthology.orchestra.run.smithi106.stdout: "req_getfilelock_latency": { 2024-03-04T09:41:34.837 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 0, 2024-03-04T09:41:34.838 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.000000000, 2024-03-04T09:41:34.838 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000000000 2024-03-04T09:41:34.838 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-03-04T09:41:34.838 INFO:teuthology.orchestra.run.smithi106.stdout: "req_link_latency": { 2024-03-04T09:41:34.838 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 0, 2024-03-04T09:41:34.838 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.000000000, 2024-03-04T09:41:34.839 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000000000 2024-03-04T09:41:34.839 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-03-04T09:41:34.839 INFO:teuthology.orchestra.run.smithi106.stdout: "req_lookup_latency": { 2024-03-04T09:41:34.839 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 2, 2024-03-04T09:41:34.839 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.000539080, 2024-03-04T09:41:34.839 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000269540 2024-03-04T09:41:34.839 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-03-04T09:41:34.839 INFO:teuthology.orchestra.run.smithi106.stdout: "req_lookuphash_latency": { 2024-03-04T09:41:34.840 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 0, 2024-03-04T09:41:34.840 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.000000000, 2024-03-04T09:41:34.840 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000000000 2024-03-04T09:41:34.840 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-03-04T09:41:34.840 INFO:teuthology.orchestra.run.smithi106.stdout: "req_lookupino_latency": { 2024-03-04T09:41:34.840 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 1, 2024-03-04T09:41:34.840 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.000109849, 2024-03-04T09:41:34.841 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000109849 2024-03-04T09:41:34.841 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-03-04T09:41:34.841 INFO:teuthology.orchestra.run.smithi106.stdout: "req_lookupname_latency": { 2024-03-04T09:41:34.841 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 0, 2024-03-04T09:41:34.841 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.000000000, 2024-03-04T09:41:34.841 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000000000 2024-03-04T09:41:34.841 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-03-04T09:41:34.842 INFO:teuthology.orchestra.run.smithi106.stdout: "req_lookupparent_latency": { 2024-03-04T09:41:34.842 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 0, 2024-03-04T09:41:34.842 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.000000000, 2024-03-04T09:41:34.842 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000000000 2024-03-04T09:41:34.842 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-03-04T09:41:34.842 INFO:teuthology.orchestra.run.smithi106.stdout: "req_lookupsnap_latency": { 2024-03-04T09:41:34.842 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 0, 2024-03-04T09:41:34.842 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.000000000, 2024-03-04T09:41:34.843 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000000000 2024-03-04T09:41:34.843 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-03-04T09:41:34.843 INFO:teuthology.orchestra.run.smithi106.stdout: "req_lssnap_latency": { 2024-03-04T09:41:34.843 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 0, 2024-03-04T09:41:34.843 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.000000000, 2024-03-04T09:41:34.843 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000000000 2024-03-04T09:41:34.843 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-03-04T09:41:34.843 INFO:teuthology.orchestra.run.smithi106.stdout: "req_mkdir_latency": { 2024-03-04T09:41:34.843 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 0, 2024-03-04T09:41:34.844 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.000000000, 2024-03-04T09:41:34.844 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000000000 2024-03-04T09:41:34.845 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-03-04T09:41:34.845 INFO:teuthology.orchestra.run.smithi106.stdout: "req_mknod_latency": { 2024-03-04T09:41:34.845 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 0, 2024-03-04T09:41:34.845 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.000000000, 2024-03-04T09:41:34.845 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000000000 2024-03-04T09:41:34.845 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-03-04T09:41:34.845 INFO:teuthology.orchestra.run.smithi106.stdout: "req_mksnap_latency": { 2024-03-04T09:41:34.846 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 0, 2024-03-04T09:41:34.846 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.000000000, 2024-03-04T09:41:34.846 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000000000 2024-03-04T09:41:34.846 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-03-04T09:41:34.846 INFO:teuthology.orchestra.run.smithi106.stdout: "req_open_latency": { 2024-03-04T09:41:34.846 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 0, 2024-03-04T09:41:34.846 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.000000000, 2024-03-04T09:41:34.847 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000000000 2024-03-04T09:41:34.847 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-03-04T09:41:34.847 INFO:teuthology.orchestra.run.smithi106.stdout: "req_readdir_latency": { 2024-03-04T09:41:34.847 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 0, 2024-03-04T09:41:34.847 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.000000000, 2024-03-04T09:41:34.847 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000000000 2024-03-04T09:41:34.847 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-03-04T09:41:34.848 INFO:teuthology.orchestra.run.smithi106.stdout: "req_rename_latency": { 2024-03-04T09:41:34.848 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 0, 2024-03-04T09:41:34.848 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.000000000, 2024-03-04T09:41:34.848 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000000000 2024-03-04T09:41:34.848 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-03-04T09:41:34.848 INFO:teuthology.orchestra.run.smithi106.stdout: "req_renamesnap_latency": { 2024-03-04T09:41:34.848 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 0, 2024-03-04T09:41:34.849 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.000000000, 2024-03-04T09:41:34.849 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000000000 2024-03-04T09:41:34.849 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-03-04T09:41:34.849 INFO:teuthology.orchestra.run.smithi106.stdout: "req_snapdiff_latency": { 2024-03-04T09:41:34.849 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 0, 2024-03-04T09:41:34.849 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.000000000, 2024-03-04T09:41:34.849 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000000000 2024-03-04T09:41:34.850 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-03-04T09:41:34.850 INFO:teuthology.orchestra.run.smithi106.stdout: "req_rmdir_latency": { 2024-03-04T09:41:34.850 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 0, 2024-03-04T09:41:34.850 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.000000000, 2024-03-04T09:41:34.850 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000000000 2024-03-04T09:41:34.850 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-03-04T09:41:34.850 INFO:teuthology.orchestra.run.smithi106.stdout: "req_rmsnap_latency": { 2024-03-04T09:41:34.850 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 0, 2024-03-04T09:41:34.851 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.000000000, 2024-03-04T09:41:34.851 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000000000 2024-03-04T09:41:34.851 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-03-04T09:41:34.851 INFO:teuthology.orchestra.run.smithi106.stdout: "req_rmxattr_latency": { 2024-03-04T09:41:34.851 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 0, 2024-03-04T09:41:34.851 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.000000000, 2024-03-04T09:41:34.851 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000000000 2024-03-04T09:41:34.852 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-03-04T09:41:34.852 INFO:teuthology.orchestra.run.smithi106.stdout: "req_setattr_latency": { 2024-03-04T09:41:34.852 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 0, 2024-03-04T09:41:34.852 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.000000000, 2024-03-04T09:41:34.852 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000000000 2024-03-04T09:41:34.852 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-03-04T09:41:34.852 INFO:teuthology.orchestra.run.smithi106.stdout: "req_setdirlayout_latency": { 2024-03-04T09:41:34.852 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 0, 2024-03-04T09:41:34.853 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.000000000, 2024-03-04T09:41:34.853 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000000000 2024-03-04T09:41:34.853 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-03-04T09:41:34.853 INFO:teuthology.orchestra.run.smithi106.stdout: "req_setfilelock_latency": { 2024-03-04T09:41:34.853 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 0, 2024-03-04T09:41:34.853 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.000000000, 2024-03-04T09:41:34.853 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000000000 2024-03-04T09:41:34.853 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-03-04T09:41:34.854 INFO:teuthology.orchestra.run.smithi106.stdout: "req_setlayout_latency": { 2024-03-04T09:41:34.854 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 0, 2024-03-04T09:41:34.854 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.000000000, 2024-03-04T09:41:34.854 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000000000 2024-03-04T09:41:34.854 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-03-04T09:41:34.854 INFO:teuthology.orchestra.run.smithi106.stdout: "req_setxattr_latency": { 2024-03-04T09:41:34.854 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 0, 2024-03-04T09:41:34.854 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.000000000, 2024-03-04T09:41:34.854 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000000000 2024-03-04T09:41:34.855 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-03-04T09:41:34.855 INFO:teuthology.orchestra.run.smithi106.stdout: "req_symlink_latency": { 2024-03-04T09:41:34.855 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 0, 2024-03-04T09:41:34.855 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.000000000, 2024-03-04T09:41:34.855 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000000000 2024-03-04T09:41:34.855 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-03-04T09:41:34.855 INFO:teuthology.orchestra.run.smithi106.stdout: "req_unlink_latency": { 2024-03-04T09:41:34.855 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 0, 2024-03-04T09:41:34.855 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.000000000, 2024-03-04T09:41:34.856 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000000000 2024-03-04T09:41:34.856 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-03-04T09:41:34.856 INFO:teuthology.orchestra.run.smithi106.stdout: "cap_revoke_eviction": 0, 2024-03-04T09:41:34.856 INFO:teuthology.orchestra.run.smithi106.stdout: "cap_acquisition_throttle": 0, 2024-03-04T09:41:34.856 INFO:teuthology.orchestra.run.smithi106.stdout: "req_getvxattr_latency": { 2024-03-04T09:41:34.856 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 0, 2024-03-04T09:41:34.856 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.000000000, 2024-03-04T09:41:34.856 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000000000 2024-03-04T09:41:34.857 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-03-04T09:41:34.857 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-03-04T09:41:34.857 INFO:teuthology.orchestra.run.smithi106.stdout: "mds_sessions": { 2024-03-04T09:41:34.857 INFO:teuthology.orchestra.run.smithi106.stdout: "session_count": 1, 2024-03-04T09:41:34.857 INFO:teuthology.orchestra.run.smithi106.stdout: "session_add": 4, 2024-03-04T09:41:34.857 INFO:teuthology.orchestra.run.smithi106.stdout: "session_remove": 3, 2024-03-04T09:41:34.857 INFO:teuthology.orchestra.run.smithi106.stdout: "sessions_open": 1, 2024-03-04T09:41:34.857 INFO:teuthology.orchestra.run.smithi106.stdout: "sessions_stale": 0, 2024-03-04T09:41:34.857 INFO:teuthology.orchestra.run.smithi106.stdout: "total_load": 4, 2024-03-04T09:41:34.858 INFO:teuthology.orchestra.run.smithi106.stdout: "average_load": 2, 2024-03-04T09:41:34.858 INFO:teuthology.orchestra.run.smithi106.stdout: "avg_session_uptime": 633, 2024-03-04T09:41:34.858 INFO:teuthology.orchestra.run.smithi106.stdout: "mdthresh_evicted": 0 2024-03-04T09:41:34.858 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-03-04T09:41:34.858 INFO:teuthology.orchestra.run.smithi106.stdout: "mempool": { 2024-03-04T09:41:34.858 INFO:teuthology.orchestra.run.smithi106.stdout: "bloom_filter_bytes": 0, 2024-03-04T09:41:34.858 INFO:teuthology.orchestra.run.smithi106.stdout: "bloom_filter_items": 0, 2024-03-04T09:41:34.858 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_alloc_bytes": 0, 2024-03-04T09:41:34.858 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_alloc_items": 0, 2024-03-04T09:41:34.859 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_cache_data_bytes": 0, 2024-03-04T09:41:34.859 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_cache_data_items": 0, 2024-03-04T09:41:34.859 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-04T09:41:34.859 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_cache_onode_items": 0, 2024-03-04T09:41:34.859 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-04T09:41:34.859 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_cache_meta_items": 0, 2024-03-04T09:41:34.859 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_cache_other_bytes": 0, 2024-03-04T09:41:34.859 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_cache_other_items": 0, 2024-03-04T09:41:34.859 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-04T09:41:34.860 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_cache_buffer_items": 0, 2024-03-04T09:41:34.860 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_extent_bytes": 0, 2024-03-04T09:41:34.860 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_extent_items": 0, 2024-03-04T09:41:34.860 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_blob_bytes": 0, 2024-03-04T09:41:34.860 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_blob_items": 0, 2024-03-04T09:41:34.860 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-04T09:41:34.860 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_shared_blob_items": 0, 2024-03-04T09:41:34.860 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-04T09:41:34.861 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_inline_bl_items": 0, 2024-03-04T09:41:34.861 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_fsck_bytes": 0, 2024-03-04T09:41:34.861 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_fsck_items": 0, 2024-03-04T09:41:34.861 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_txc_bytes": 0, 2024-03-04T09:41:34.861 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_txc_items": 0, 2024-03-04T09:41:34.861 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-04T09:41:34.861 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_writing_deferred_items": 0, 2024-03-04T09:41:34.861 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_writing_bytes": 0, 2024-03-04T09:41:34.861 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_writing_items": 0, 2024-03-04T09:41:34.862 INFO:teuthology.orchestra.run.smithi106.stdout: "bluefs_bytes": 0, 2024-03-04T09:41:34.862 INFO:teuthology.orchestra.run.smithi106.stdout: "bluefs_items": 0, 2024-03-04T09:41:34.862 INFO:teuthology.orchestra.run.smithi106.stdout: "bluefs_file_reader_bytes": 0, 2024-03-04T09:41:34.862 INFO:teuthology.orchestra.run.smithi106.stdout: "bluefs_file_reader_items": 0, 2024-03-04T09:41:34.862 INFO:teuthology.orchestra.run.smithi106.stdout: "bluefs_file_writer_bytes": 0, 2024-03-04T09:41:34.862 INFO:teuthology.orchestra.run.smithi106.stdout: "bluefs_file_writer_items": 0, 2024-03-04T09:41:34.862 INFO:teuthology.orchestra.run.smithi106.stdout: "buffer_anon_bytes": 265967, 2024-03-04T09:41:34.862 INFO:teuthology.orchestra.run.smithi106.stdout: "buffer_anon_items": 130, 2024-03-04T09:41:34.862 INFO:teuthology.orchestra.run.smithi106.stdout: "buffer_meta_bytes": 0, 2024-03-04T09:41:34.863 INFO:teuthology.orchestra.run.smithi106.stdout: "buffer_meta_items": 0, 2024-03-04T09:41:34.863 INFO:teuthology.orchestra.run.smithi106.stdout: "osd_bytes": 0, 2024-03-04T09:41:34.863 INFO:teuthology.orchestra.run.smithi106.stdout: "osd_items": 0, 2024-03-04T09:41:34.863 INFO:teuthology.orchestra.run.smithi106.stdout: "osd_mapbl_bytes": 0, 2024-03-04T09:41:34.863 INFO:teuthology.orchestra.run.smithi106.stdout: "osd_mapbl_items": 0, 2024-03-04T09:41:34.863 INFO:teuthology.orchestra.run.smithi106.stdout: "osd_pglog_bytes": 0, 2024-03-04T09:41:34.863 INFO:teuthology.orchestra.run.smithi106.stdout: "osd_pglog_items": 0, 2024-03-04T09:41:34.864 INFO:teuthology.orchestra.run.smithi106.stdout: "osdmap_bytes": 9744, 2024-03-04T09:41:34.864 INFO:teuthology.orchestra.run.smithi106.stdout: "osdmap_items": 204, 2024-03-04T09:41:34.864 INFO:teuthology.orchestra.run.smithi106.stdout: "osdmap_mapping_bytes": 0, 2024-03-04T09:41:34.864 INFO:teuthology.orchestra.run.smithi106.stdout: "osdmap_mapping_items": 0, 2024-03-04T09:41:34.864 INFO:teuthology.orchestra.run.smithi106.stdout: "pgmap_bytes": 0, 2024-03-04T09:41:34.864 INFO:teuthology.orchestra.run.smithi106.stdout: "pgmap_items": 0, 2024-03-04T09:41:34.864 INFO:teuthology.orchestra.run.smithi106.stdout: "mds_co_bytes": 202831, 2024-03-04T09:41:34.865 INFO:teuthology.orchestra.run.smithi106.stdout: "mds_co_items": 1853, 2024-03-04T09:41:34.865 INFO:teuthology.orchestra.run.smithi106.stdout: "unittest_1_bytes": 0, 2024-03-04T09:41:34.865 INFO:teuthology.orchestra.run.smithi106.stdout: "unittest_1_items": 0, 2024-03-04T09:41:34.865 INFO:teuthology.orchestra.run.smithi106.stdout: "unittest_2_bytes": 0, 2024-03-04T09:41:34.865 INFO:teuthology.orchestra.run.smithi106.stdout: "unittest_2_items": 0 2024-03-04T09:41:34.865 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-03-04T09:41:34.865 INFO:teuthology.orchestra.run.smithi106.stdout: "objecter": { 2024-03-04T09:41:34.865 INFO:teuthology.orchestra.run.smithi106.stdout: "op_active": 0, 2024-03-04T09:41:34.866 INFO:teuthology.orchestra.run.smithi106.stdout: "op_laggy": 0, 2024-03-04T09:41:34.866 INFO:teuthology.orchestra.run.smithi106.stdout: "op_send": 65, 2024-03-04T09:41:34.866 INFO:teuthology.orchestra.run.smithi106.stdout: "op_send_bytes": 154196, 2024-03-04T09:41:34.866 INFO:teuthology.orchestra.run.smithi106.stdout: "op_resend": 29, 2024-03-04T09:41:34.866 INFO:teuthology.orchestra.run.smithi106.stdout: "op_reply": 65, 2024-03-04T09:41:34.866 INFO:teuthology.orchestra.run.smithi106.stdout: "op_latency": { 2024-03-04T09:41:34.866 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 65, 2024-03-04T09:41:34.867 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.366991410, 2024-03-04T09:41:34.867 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.005646021 2024-03-04T09:41:34.867 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-03-04T09:41:34.867 INFO:teuthology.orchestra.run.smithi106.stdout: "op_inflight": 0, 2024-03-04T09:41:34.867 INFO:teuthology.orchestra.run.smithi106.stdout: "oplen_avg": { 2024-03-04T09:41:34.867 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 65, 2024-03-04T09:41:34.867 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 96 2024-03-04T09:41:34.868 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-03-04T09:41:34.868 INFO:teuthology.orchestra.run.smithi106.stdout: "op": 65, 2024-03-04T09:41:34.868 INFO:teuthology.orchestra.run.smithi106.stdout: "op_r": 0, 2024-03-04T09:41:34.868 INFO:teuthology.orchestra.run.smithi106.stdout: "op_w": 65, 2024-03-04T09:41:34.868 INFO:teuthology.orchestra.run.smithi106.stdout: "op_rmw": 0, 2024-03-04T09:41:34.868 INFO:teuthology.orchestra.run.smithi106.stdout: "op_pg": 0, 2024-03-04T09:41:34.868 INFO:teuthology.orchestra.run.smithi106.stdout: "osdop_stat": 0, 2024-03-04T09:41:34.869 INFO:teuthology.orchestra.run.smithi106.stdout: "osdop_create": 10, 2024-03-04T09:41:34.869 INFO:teuthology.orchestra.run.smithi106.stdout: "osdop_read": 0, 2024-03-04T09:41:34.869 INFO:teuthology.orchestra.run.smithi106.stdout: "osdop_write": 17, 2024-03-04T09:41:34.869 INFO:teuthology.orchestra.run.smithi106.stdout: "osdop_writefull": 10, 2024-03-04T09:41:34.869 INFO:teuthology.orchestra.run.smithi106.stdout: "osdop_writesame": 0, 2024-03-04T09:41:34.869 INFO:teuthology.orchestra.run.smithi106.stdout: "osdop_append": 0, 2024-03-04T09:41:34.870 INFO:teuthology.orchestra.run.smithi106.stdout: "osdop_zero": 0, 2024-03-04T09:41:34.870 INFO:teuthology.orchestra.run.smithi106.stdout: "osdop_truncate": 0, 2024-03-04T09:41:34.870 INFO:teuthology.orchestra.run.smithi106.stdout: "osdop_delete": 6, 2024-03-04T09:41:34.870 INFO:teuthology.orchestra.run.smithi106.stdout: "osdop_mapext": 0, 2024-03-04T09:41:34.870 INFO:teuthology.orchestra.run.smithi106.stdout: "osdop_sparse_read": 0, 2024-03-04T09:41:34.870 INFO:teuthology.orchestra.run.smithi106.stdout: "osdop_clonerange": 0, 2024-03-04T09:41:34.870 INFO:teuthology.orchestra.run.smithi106.stdout: "osdop_getxattr": 0, 2024-03-04T09:41:34.871 INFO:teuthology.orchestra.run.smithi106.stdout: "osdop_setxattr": 20, 2024-03-04T09:41:34.871 INFO:teuthology.orchestra.run.smithi106.stdout: "osdop_cmpxattr": 0, 2024-03-04T09:41:34.871 INFO:teuthology.orchestra.run.smithi106.stdout: "osdop_rmxattr": 0, 2024-03-04T09:41:34.871 INFO:teuthology.orchestra.run.smithi106.stdout: "osdop_resetxattrs": 0, 2024-03-04T09:41:34.871 INFO:teuthology.orchestra.run.smithi106.stdout: "osdop_call": 0, 2024-03-04T09:41:34.871 INFO:teuthology.orchestra.run.smithi106.stdout: "osdop_watch": 0, 2024-03-04T09:41:34.871 INFO:teuthology.orchestra.run.smithi106.stdout: "osdop_notify": 0, 2024-03-04T09:41:34.872 INFO:teuthology.orchestra.run.smithi106.stdout: "osdop_src_cmpxattr": 0, 2024-03-04T09:41:34.872 INFO:teuthology.orchestra.run.smithi106.stdout: "osdop_pgls": 0, 2024-03-04T09:41:34.872 INFO:teuthology.orchestra.run.smithi106.stdout: "osdop_pgls_filter": 0, 2024-03-04T09:41:34.872 INFO:teuthology.orchestra.run.smithi106.stdout: "osdop_other": 0, 2024-03-04T09:41:34.872 INFO:teuthology.orchestra.run.smithi106.stdout: "linger_active": 0, 2024-03-04T09:41:34.873 INFO:teuthology.orchestra.run.smithi106.stdout: "linger_send": 0, 2024-03-04T09:41:34.873 INFO:teuthology.orchestra.run.smithi106.stdout: "linger_resend": 0, 2024-03-04T09:41:34.873 INFO:teuthology.orchestra.run.smithi106.stdout: "linger_ping": 0, 2024-03-04T09:41:34.873 INFO:teuthology.orchestra.run.smithi106.stdout: "poolop_active": 0, 2024-03-04T09:41:34.873 INFO:teuthology.orchestra.run.smithi106.stdout: "poolop_send": 0, 2024-03-04T09:41:34.873 INFO:teuthology.orchestra.run.smithi106.stdout: "poolop_resend": 0, 2024-03-04T09:41:34.873 INFO:teuthology.orchestra.run.smithi106.stdout: "poolstat_active": 0, 2024-03-04T09:41:34.874 INFO:teuthology.orchestra.run.smithi106.stdout: "poolstat_send": 0, 2024-03-04T09:41:34.874 INFO:teuthology.orchestra.run.smithi106.stdout: "poolstat_resend": 0, 2024-03-04T09:41:34.874 INFO:teuthology.orchestra.run.smithi106.stdout: "statfs_active": 0, 2024-03-04T09:41:34.874 INFO:teuthology.orchestra.run.smithi106.stdout: "statfs_send": 0, 2024-03-04T09:41:34.874 INFO:teuthology.orchestra.run.smithi106.stdout: "statfs_resend": 0, 2024-03-04T09:41:34.874 INFO:teuthology.orchestra.run.smithi106.stdout: "command_active": 0, 2024-03-04T09:41:34.874 INFO:teuthology.orchestra.run.smithi106.stdout: "command_send": 0, 2024-03-04T09:41:34.875 INFO:teuthology.orchestra.run.smithi106.stdout: "command_resend": 0, 2024-03-04T09:41:34.875 INFO:teuthology.orchestra.run.smithi106.stdout: "map_epoch": 102, 2024-03-04T09:41:34.875 INFO:teuthology.orchestra.run.smithi106.stdout: "map_full": 0, 2024-03-04T09:41:34.875 INFO:teuthology.orchestra.run.smithi106.stdout: "map_inc": 13, 2024-03-04T09:41:34.875 INFO:teuthology.orchestra.run.smithi106.stdout: "osd_sessions": 8, 2024-03-04T09:41:34.875 INFO:teuthology.orchestra.run.smithi106.stdout: "osd_session_open": 8, 2024-03-04T09:41:34.875 INFO:teuthology.orchestra.run.smithi106.stdout: "osd_session_close": 0, 2024-03-04T09:41:34.876 INFO:teuthology.orchestra.run.smithi106.stdout: "osd_laggy": 0, 2024-03-04T09:41:34.876 INFO:teuthology.orchestra.run.smithi106.stdout: "omap_wr": 30, 2024-03-04T09:41:34.876 INFO:teuthology.orchestra.run.smithi106.stdout: "omap_rd": 0, 2024-03-04T09:41:34.876 INFO:teuthology.orchestra.run.smithi106.stdout: "omap_del": 3 2024-03-04T09:41:34.876 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-03-04T09:41:34.876 INFO:teuthology.orchestra.run.smithi106.stdout: "oft": { 2024-03-04T09:41:34.876 INFO:teuthology.orchestra.run.smithi106.stdout: "omap_total_objs": 1, 2024-03-04T09:41:34.877 INFO:teuthology.orchestra.run.smithi106.stdout: "omap_total_kv_pairs": 0, 2024-03-04T09:41:34.877 INFO:teuthology.orchestra.run.smithi106.stdout: "omap_total_updates": 0, 2024-03-04T09:41:34.877 INFO:teuthology.orchestra.run.smithi106.stdout: "omap_total_removes": 0 2024-03-04T09:41:34.877 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-03-04T09:41:34.877 INFO:teuthology.orchestra.run.smithi106.stdout: "osd-slow-ops": { 2024-03-04T09:41:34.877 INFO:teuthology.orchestra.run.smithi106.stdout: "slow_ops_count": 0 2024-03-04T09:41:34.877 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-03-04T09:41:34.878 INFO:teuthology.orchestra.run.smithi106.stdout: "purge_queue": { 2024-03-04T09:41:34.878 INFO:teuthology.orchestra.run.smithi106.stdout: "pq_executing_ops": 0, 2024-03-04T09:41:34.878 INFO:teuthology.orchestra.run.smithi106.stdout: "pq_executing_ops_high_water": 0, 2024-03-04T09:41:34.878 INFO:teuthology.orchestra.run.smithi106.stdout: "pq_executing": 0, 2024-03-04T09:41:34.878 INFO:teuthology.orchestra.run.smithi106.stdout: "pq_executing_high_water": 0, 2024-03-04T09:41:34.878 INFO:teuthology.orchestra.run.smithi106.stdout: "pq_executed_ops": 0, 2024-03-04T09:41:34.878 INFO:teuthology.orchestra.run.smithi106.stdout: "pq_executed": 0, 2024-03-04T09:41:34.879 INFO:teuthology.orchestra.run.smithi106.stdout: "pq_item_in_journal": 0 2024-03-04T09:41:34.879 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-03-04T09:41:34.879 INFO:teuthology.orchestra.run.smithi106.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-04T09:41:34.879 INFO:teuthology.orchestra.run.smithi106.stdout: "val": 0, 2024-03-04T09:41:34.879 INFO:teuthology.orchestra.run.smithi106.stdout: "max": 104857600, 2024-03-04T09:41:34.879 INFO:teuthology.orchestra.run.smithi106.stdout: "get_started": 0, 2024-03-04T09:41:34.879 INFO:teuthology.orchestra.run.smithi106.stdout: "get": 937, 2024-03-04T09:41:34.880 INFO:teuthology.orchestra.run.smithi106.stdout: "get_sum": 1620076, 2024-03-04T09:41:34.880 INFO:teuthology.orchestra.run.smithi106.stdout: "get_or_fail_fail": 0, 2024-03-04T09:41:34.880 INFO:teuthology.orchestra.run.smithi106.stdout: "get_or_fail_success": 937, 2024-03-04T09:41:34.880 INFO:teuthology.orchestra.run.smithi106.stdout: "take": 0, 2024-03-04T09:41:34.880 INFO:teuthology.orchestra.run.smithi106.stdout: "take_sum": 0, 2024-03-04T09:41:34.880 INFO:teuthology.orchestra.run.smithi106.stdout: "put": 937, 2024-03-04T09:41:34.880 INFO:teuthology.orchestra.run.smithi106.stdout: "put_sum": 1620076, 2024-03-04T09:41:34.881 INFO:teuthology.orchestra.run.smithi106.stdout: "wait": { 2024-03-04T09:41:34.881 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 0, 2024-03-04T09:41:34.881 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.000000000, 2024-03-04T09:41:34.881 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000000000 2024-03-04T09:41:34.881 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-03-04T09:41:34.881 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-03-04T09:41:34.881 INFO:teuthology.orchestra.run.smithi106.stdout: "throttle-objecter_bytes": { 2024-03-04T09:41:34.881 INFO:teuthology.orchestra.run.smithi106.stdout: "val": 0, 2024-03-04T09:41:34.882 INFO:teuthology.orchestra.run.smithi106.stdout: "max": 104857600, 2024-03-04T09:41:34.882 INFO:teuthology.orchestra.run.smithi106.stdout: "get_started": 0, 2024-03-04T09:41:34.882 INFO:teuthology.orchestra.run.smithi106.stdout: "get": 0, 2024-03-04T09:41:34.882 INFO:teuthology.orchestra.run.smithi106.stdout: "get_sum": 0, 2024-03-04T09:41:34.882 INFO:teuthology.orchestra.run.smithi106.stdout: "get_or_fail_fail": 0, 2024-03-04T09:41:34.882 INFO:teuthology.orchestra.run.smithi106.stdout: "get_or_fail_success": 0, 2024-03-04T09:41:34.882 INFO:teuthology.orchestra.run.smithi106.stdout: "take": 65, 2024-03-04T09:41:34.882 INFO:teuthology.orchestra.run.smithi106.stdout: "take_sum": 154196, 2024-03-04T09:41:34.883 INFO:teuthology.orchestra.run.smithi106.stdout: "put": 65, 2024-03-04T09:41:34.883 INFO:teuthology.orchestra.run.smithi106.stdout: "put_sum": 154196, 2024-03-04T09:41:34.883 INFO:teuthology.orchestra.run.smithi106.stdout: "wait": { 2024-03-04T09:41:34.883 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 0, 2024-03-04T09:41:34.883 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.000000000, 2024-03-04T09:41:34.883 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000000000 2024-03-04T09:41:34.883 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-03-04T09:41:34.883 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-03-04T09:41:34.884 INFO:teuthology.orchestra.run.smithi106.stdout: "throttle-objecter_ops": { 2024-03-04T09:41:34.884 INFO:teuthology.orchestra.run.smithi106.stdout: "val": 0, 2024-03-04T09:41:34.884 INFO:teuthology.orchestra.run.smithi106.stdout: "max": 1024, 2024-03-04T09:41:34.884 INFO:teuthology.orchestra.run.smithi106.stdout: "get_started": 0, 2024-03-04T09:41:34.884 INFO:teuthology.orchestra.run.smithi106.stdout: "get": 0, 2024-03-04T09:41:34.884 INFO:teuthology.orchestra.run.smithi106.stdout: "get_sum": 0, 2024-03-04T09:41:34.884 INFO:teuthology.orchestra.run.smithi106.stdout: "get_or_fail_fail": 0, 2024-03-04T09:41:34.884 INFO:teuthology.orchestra.run.smithi106.stdout: "get_or_fail_success": 0, 2024-03-04T09:41:34.885 INFO:teuthology.orchestra.run.smithi106.stdout: "take": 65, 2024-03-04T09:41:34.885 INFO:teuthology.orchestra.run.smithi106.stdout: "take_sum": 65, 2024-03-04T09:41:34.885 INFO:teuthology.orchestra.run.smithi106.stdout: "put": 65, 2024-03-04T09:41:34.885 INFO:teuthology.orchestra.run.smithi106.stdout: "put_sum": 65, 2024-03-04T09:41:34.885 INFO:teuthology.orchestra.run.smithi106.stdout: "wait": { 2024-03-04T09:41:34.885 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 0, 2024-03-04T09:41:34.885 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.000000000, 2024-03-04T09:41:34.885 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000000000 2024-03-04T09:41:34.885 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-03-04T09:41:34.886 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-03-04T09:41:34.886 INFO:teuthology.orchestra.run.smithi106.stdout: "throttle-write_buf_throttle": { 2024-03-04T09:41:34.886 INFO:teuthology.orchestra.run.smithi106.stdout: "val": 0, 2024-03-04T09:41:34.886 INFO:teuthology.orchestra.run.smithi106.stdout: "max": 3758096384, 2024-03-04T09:41:34.886 INFO:teuthology.orchestra.run.smithi106.stdout: "get_started": 0, 2024-03-04T09:41:34.886 INFO:teuthology.orchestra.run.smithi106.stdout: "get": 0, 2024-03-04T09:41:34.886 INFO:teuthology.orchestra.run.smithi106.stdout: "get_sum": 0, 2024-03-04T09:41:34.886 INFO:teuthology.orchestra.run.smithi106.stdout: "get_or_fail_fail": 0, 2024-03-04T09:41:34.886 INFO:teuthology.orchestra.run.smithi106.stdout: "get_or_fail_success": 0, 2024-03-04T09:41:34.887 INFO:teuthology.orchestra.run.smithi106.stdout: "take": 0, 2024-03-04T09:41:34.887 INFO:teuthology.orchestra.run.smithi106.stdout: "take_sum": 0, 2024-03-04T09:41:34.887 INFO:teuthology.orchestra.run.smithi106.stdout: "put": 0, 2024-03-04T09:41:34.887 INFO:teuthology.orchestra.run.smithi106.stdout: "put_sum": 0, 2024-03-04T09:41:34.887 INFO:teuthology.orchestra.run.smithi106.stdout: "wait": { 2024-03-04T09:41:34.887 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 0, 2024-03-04T09:41:34.887 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.000000000, 2024-03-04T09:41:34.887 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000000000 2024-03-04T09:41:34.887 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-03-04T09:41:34.888 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-03-04T09:41:34.888 INFO:teuthology.orchestra.run.smithi106.stdout: "throttle-write_buf_throttle-0x559b195e86e0": { 2024-03-04T09:41:34.888 INFO:teuthology.orchestra.run.smithi106.stdout: "val": 0, 2024-03-04T09:41:34.888 INFO:teuthology.orchestra.run.smithi106.stdout: "max": 3758096384, 2024-03-04T09:41:34.888 INFO:teuthology.orchestra.run.smithi106.stdout: "get_started": 0, 2024-03-04T09:41:34.888 INFO:teuthology.orchestra.run.smithi106.stdout: "get": 47, 2024-03-04T09:41:34.888 INFO:teuthology.orchestra.run.smithi106.stdout: "get_sum": 139802, 2024-03-04T09:41:34.888 INFO:teuthology.orchestra.run.smithi106.stdout: "get_or_fail_fail": 0, 2024-03-04T09:41:34.889 INFO:teuthology.orchestra.run.smithi106.stdout: "get_or_fail_success": 47, 2024-03-04T09:41:34.889 INFO:teuthology.orchestra.run.smithi106.stdout: "take": 0, 2024-03-04T09:41:34.889 INFO:teuthology.orchestra.run.smithi106.stdout: "take_sum": 0, 2024-03-04T09:41:34.889 INFO:teuthology.orchestra.run.smithi106.stdout: "put": 17, 2024-03-04T09:41:34.889 INFO:teuthology.orchestra.run.smithi106.stdout: "put_sum": 139802, 2024-03-04T09:41:34.889 INFO:teuthology.orchestra.run.smithi106.stdout: "wait": { 2024-03-04T09:41:34.889 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 0, 2024-03-04T09:41:34.889 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.000000000, 2024-03-04T09:41:34.889 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000000000 2024-03-04T09:41:34.890 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-03-04T09:41:34.890 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-03-04T09:41:34.890 INFO:teuthology.orchestra.run.smithi106.stdout:} 2024-03-04T09:41:35.645 INFO:tasks.check_counter:Daemon mds.h mds.exported=0 2024-03-04T09:41:35.645 INFO:tasks.check_counter:Daemon mds.h mds.imported=5 2024-03-04T09:41:35.645 INFO:tasks.check_counter:Daemon mds.h mds.root_rsnaps=0 2024-03-04T09:41:35.646 INFO:tasks.check_counter:Daemon mds.h mds_server.req_mksnap_latency.avgcount=0 2024-03-04T09:41:35.646 INFO:tasks.check_counter:Daemon mds.h mds_server.req_rmsnap_latency.avgcount=0 2024-03-04T09:41:35.646 INFO:tasks.check_counter:Daemon mds.h mds.dir_split=0 2024-03-04T09:41:35.646 DEBUG:tasks.check_counter:Getting stats from k 2024-03-04T09:41:35.646 DEBUG:teuthology.orchestra.run.smithi106:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd shell --fsid 4e059f08-da08-11ee-95c6-87774f69a715 -- ceph daemon mds.k perf dump 2024-03-04T09:41:35.899 INFO:teuthology.orchestra.run.smithi106.stderr:Inferring config /var/lib/ceph/4e059f08-da08-11ee-95c6-87774f69a715/mon.b/config 2024-03-04T09:41:36.281 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:41:36 smithi106 ceph-mon[36484]: pgmap v324: 129 pgs: 129 active+clean; 78 MiB data, 691 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-04T09:41:36.585 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:41:36 smithi052 ceph-mon[32473]: pgmap v324: 129 pgs: 129 active+clean; 78 MiB data, 691 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-04T09:41:36.587 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:41:36 smithi191 ceph-mon[36816]: pgmap v324: 129 pgs: 129 active+clean; 78 MiB data, 691 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-04T09:41:37.637 INFO:teuthology.orchestra.run.smithi106.stdout:{ 2024-03-04T09:41:37.637 INFO:teuthology.orchestra.run.smithi106.stdout: "AsyncMessenger::Worker-0": { 2024-03-04T09:41:37.637 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_recv_messages": 229, 2024-03-04T09:41:37.637 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_send_messages": 219, 2024-03-04T09:41:37.637 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_recv_bytes": 1434304, 2024-03-04T09:41:37.637 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_send_bytes": 131184, 2024-03-04T09:41:37.638 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_created_connections": 4, 2024-03-04T09:41:37.638 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_active_connections": 0, 2024-03-04T09:41:37.638 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_running_total_time": 0.037076887, 2024-03-04T09:41:37.638 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_running_send_time": 0.008345829, 2024-03-04T09:41:37.638 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_running_recv_time": 0.033393778, 2024-03-04T09:41:37.638 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_running_fast_dispatch_time": 0.003834299, 2024-03-04T09:41:37.638 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_send_messages_queue_lat": { 2024-03-04T09:41:37.638 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 218, 2024-03-04T09:41:37.639 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.006352009, 2024-03-04T09:41:37.639 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000029137 2024-03-04T09:41:37.639 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-03-04T09:41:37.639 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_handle_ack_lat": { 2024-03-04T09:41:37.639 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 0, 2024-03-04T09:41:37.639 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.000000000, 2024-03-04T09:41:37.639 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000000000 2024-03-04T09:41:37.639 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-03-04T09:41:37.639 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_recv_encrypted_bytes": 1434304, 2024-03-04T09:41:37.640 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_send_encrypted_bytes": 131184 2024-03-04T09:41:37.640 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-03-04T09:41:37.640 INFO:teuthology.orchestra.run.smithi106.stdout: "AsyncMessenger::Worker-1": { 2024-03-04T09:41:37.641 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_recv_messages": 6, 2024-03-04T09:41:37.641 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_send_messages": 169, 2024-03-04T09:41:37.641 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_recv_bytes": 3344, 2024-03-04T09:41:37.641 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_send_bytes": 184864, 2024-03-04T09:41:37.641 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_created_connections": 5, 2024-03-04T09:41:37.642 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_active_connections": 3, 2024-03-04T09:41:37.642 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_running_total_time": 0.007749296, 2024-03-04T09:41:37.642 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_running_send_time": 0.004527004, 2024-03-04T09:41:37.642 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_running_recv_time": 0.001756524, 2024-03-04T09:41:37.642 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-03-04T09:41:37.642 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_send_messages_queue_lat": { 2024-03-04T09:41:37.642 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 167, 2024-03-04T09:41:37.642 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.015499534, 2024-03-04T09:41:37.643 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000092811 2024-03-04T09:41:37.643 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-03-04T09:41:37.643 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_handle_ack_lat": { 2024-03-04T09:41:37.643 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 0, 2024-03-04T09:41:37.643 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.000000000, 2024-03-04T09:41:37.643 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000000000 2024-03-04T09:41:37.643 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-03-04T09:41:37.644 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_recv_encrypted_bytes": 3344, 2024-03-04T09:41:37.644 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_send_encrypted_bytes": 184864 2024-03-04T09:41:37.644 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-03-04T09:41:37.644 INFO:teuthology.orchestra.run.smithi106.stdout: "AsyncMessenger::Worker-2": { 2024-03-04T09:41:37.644 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_recv_messages": 0, 2024-03-04T09:41:37.644 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_send_messages": 2, 2024-03-04T09:41:37.644 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_recv_bytes": 0, 2024-03-04T09:41:37.645 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_send_bytes": 0, 2024-03-04T09:41:37.645 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_created_connections": 2, 2024-03-04T09:41:37.645 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_active_connections": 1, 2024-03-04T09:41:37.645 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_running_total_time": 0.000405134, 2024-03-04T09:41:37.645 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_running_send_time": 0.000000000, 2024-03-04T09:41:37.645 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_running_recv_time": 0.000314975, 2024-03-04T09:41:37.645 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-03-04T09:41:37.646 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_send_messages_queue_lat": { 2024-03-04T09:41:37.646 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 0, 2024-03-04T09:41:37.646 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.000000000, 2024-03-04T09:41:37.646 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000000000 2024-03-04T09:41:37.646 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-03-04T09:41:37.646 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_handle_ack_lat": { 2024-03-04T09:41:37.646 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 0, 2024-03-04T09:41:37.647 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.000000000, 2024-03-04T09:41:37.647 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000000000 2024-03-04T09:41:37.647 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-03-04T09:41:37.647 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_recv_encrypted_bytes": 0, 2024-03-04T09:41:37.647 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_send_encrypted_bytes": 0 2024-03-04T09:41:37.647 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-03-04T09:41:37.647 INFO:teuthology.orchestra.run.smithi106.stdout: "cct": { 2024-03-04T09:41:37.648 INFO:teuthology.orchestra.run.smithi106.stdout: "total_workers": 0, 2024-03-04T09:41:37.648 INFO:teuthology.orchestra.run.smithi106.stdout: "unhealthy_workers": 0 2024-03-04T09:41:37.648 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-03-04T09:41:37.648 INFO:teuthology.orchestra.run.smithi106.stdout: "mempool": { 2024-03-04T09:41:37.648 INFO:teuthology.orchestra.run.smithi106.stdout: "bloom_filter_bytes": 0, 2024-03-04T09:41:37.648 INFO:teuthology.orchestra.run.smithi106.stdout: "bloom_filter_items": 0, 2024-03-04T09:41:37.648 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_alloc_bytes": 0, 2024-03-04T09:41:37.649 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_alloc_items": 0, 2024-03-04T09:41:37.649 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_cache_data_bytes": 0, 2024-03-04T09:41:37.649 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_cache_data_items": 0, 2024-03-04T09:41:37.649 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-04T09:41:37.649 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_cache_onode_items": 0, 2024-03-04T09:41:37.649 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-04T09:41:37.649 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_cache_meta_items": 0, 2024-03-04T09:41:37.649 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_cache_other_bytes": 0, 2024-03-04T09:41:37.650 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_cache_other_items": 0, 2024-03-04T09:41:37.650 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-04T09:41:37.650 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_cache_buffer_items": 0, 2024-03-04T09:41:37.650 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_extent_bytes": 0, 2024-03-04T09:41:37.650 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_extent_items": 0, 2024-03-04T09:41:37.650 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_blob_bytes": 0, 2024-03-04T09:41:37.650 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_blob_items": 0, 2024-03-04T09:41:37.651 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-04T09:41:37.651 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_shared_blob_items": 0, 2024-03-04T09:41:37.651 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-04T09:41:37.651 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_inline_bl_items": 0, 2024-03-04T09:41:37.651 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_fsck_bytes": 0, 2024-03-04T09:41:37.651 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_fsck_items": 0, 2024-03-04T09:41:37.651 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_txc_bytes": 0, 2024-03-04T09:41:37.651 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_txc_items": 0, 2024-03-04T09:41:37.652 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-04T09:41:37.652 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_writing_deferred_items": 0, 2024-03-04T09:41:37.652 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_writing_bytes": 0, 2024-03-04T09:41:37.652 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_writing_items": 0, 2024-03-04T09:41:37.652 INFO:teuthology.orchestra.run.smithi106.stdout: "bluefs_bytes": 0, 2024-03-04T09:41:37.652 INFO:teuthology.orchestra.run.smithi106.stdout: "bluefs_items": 0, 2024-03-04T09:41:37.652 INFO:teuthology.orchestra.run.smithi106.stdout: "bluefs_file_reader_bytes": 0, 2024-03-04T09:41:37.653 INFO:teuthology.orchestra.run.smithi106.stdout: "bluefs_file_reader_items": 0, 2024-03-04T09:41:37.653 INFO:teuthology.orchestra.run.smithi106.stdout: "bluefs_file_writer_bytes": 0, 2024-03-04T09:41:37.653 INFO:teuthology.orchestra.run.smithi106.stdout: "bluefs_file_writer_items": 0, 2024-03-04T09:41:37.653 INFO:teuthology.orchestra.run.smithi106.stdout: "buffer_anon_bytes": 148794, 2024-03-04T09:41:37.653 INFO:teuthology.orchestra.run.smithi106.stdout: "buffer_anon_items": 32, 2024-03-04T09:41:37.653 INFO:teuthology.orchestra.run.smithi106.stdout: "buffer_meta_bytes": 0, 2024-03-04T09:41:37.653 INFO:teuthology.orchestra.run.smithi106.stdout: "buffer_meta_items": 0, 2024-03-04T09:41:37.654 INFO:teuthology.orchestra.run.smithi106.stdout: "osd_bytes": 0, 2024-03-04T09:41:37.654 INFO:teuthology.orchestra.run.smithi106.stdout: "osd_items": 0, 2024-03-04T09:41:37.654 INFO:teuthology.orchestra.run.smithi106.stdout: "osd_mapbl_bytes": 0, 2024-03-04T09:41:37.654 INFO:teuthology.orchestra.run.smithi106.stdout: "osd_mapbl_items": 0, 2024-03-04T09:41:37.654 INFO:teuthology.orchestra.run.smithi106.stdout: "osd_pglog_bytes": 0, 2024-03-04T09:41:37.654 INFO:teuthology.orchestra.run.smithi106.stdout: "osd_pglog_items": 0, 2024-03-04T09:41:37.655 INFO:teuthology.orchestra.run.smithi106.stdout: "osdmap_bytes": 0, 2024-03-04T09:41:37.655 INFO:teuthology.orchestra.run.smithi106.stdout: "osdmap_items": 0, 2024-03-04T09:41:37.655 INFO:teuthology.orchestra.run.smithi106.stdout: "osdmap_mapping_bytes": 0, 2024-03-04T09:41:37.655 INFO:teuthology.orchestra.run.smithi106.stdout: "osdmap_mapping_items": 0, 2024-03-04T09:41:37.655 INFO:teuthology.orchestra.run.smithi106.stdout: "pgmap_bytes": 0, 2024-03-04T09:41:37.655 INFO:teuthology.orchestra.run.smithi106.stdout: "pgmap_items": 0, 2024-03-04T09:41:37.655 INFO:teuthology.orchestra.run.smithi106.stdout: "mds_co_bytes": 928, 2024-03-04T09:41:37.656 INFO:teuthology.orchestra.run.smithi106.stdout: "mds_co_items": 2, 2024-03-04T09:41:37.656 INFO:teuthology.orchestra.run.smithi106.stdout: "unittest_1_bytes": 0, 2024-03-04T09:41:37.656 INFO:teuthology.orchestra.run.smithi106.stdout: "unittest_1_items": 0, 2024-03-04T09:41:37.656 INFO:teuthology.orchestra.run.smithi106.stdout: "unittest_2_bytes": 0, 2024-03-04T09:41:37.656 INFO:teuthology.orchestra.run.smithi106.stdout: "unittest_2_items": 0 2024-03-04T09:41:37.656 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-03-04T09:41:37.656 INFO:teuthology.orchestra.run.smithi106.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-04T09:41:37.657 INFO:teuthology.orchestra.run.smithi106.stdout: "val": 0, 2024-03-04T09:41:37.657 INFO:teuthology.orchestra.run.smithi106.stdout: "max": 104857600, 2024-03-04T09:41:37.657 INFO:teuthology.orchestra.run.smithi106.stdout: "get_started": 0, 2024-03-04T09:41:37.657 INFO:teuthology.orchestra.run.smithi106.stdout: "get": 231, 2024-03-04T09:41:37.657 INFO:teuthology.orchestra.run.smithi106.stdout: "get_sum": 1402008, 2024-03-04T09:41:37.657 INFO:teuthology.orchestra.run.smithi106.stdout: "get_or_fail_fail": 0, 2024-03-04T09:41:37.657 INFO:teuthology.orchestra.run.smithi106.stdout: "get_or_fail_success": 231, 2024-03-04T09:41:37.657 INFO:teuthology.orchestra.run.smithi106.stdout: "take": 0, 2024-03-04T09:41:37.658 INFO:teuthology.orchestra.run.smithi106.stdout: "take_sum": 0, 2024-03-04T09:41:37.658 INFO:teuthology.orchestra.run.smithi106.stdout: "put": 231, 2024-03-04T09:41:37.658 INFO:teuthology.orchestra.run.smithi106.stdout: "put_sum": 1402008, 2024-03-04T09:41:37.658 INFO:teuthology.orchestra.run.smithi106.stdout: "wait": { 2024-03-04T09:41:37.658 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 0, 2024-03-04T09:41:37.658 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.000000000, 2024-03-04T09:41:37.658 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000000000 2024-03-04T09:41:37.659 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-03-04T09:41:37.659 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-03-04T09:41:37.659 INFO:teuthology.orchestra.run.smithi106.stdout:} 2024-03-04T09:41:38.354 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:41:38 smithi106 ceph-mon[36484]: pgmap v325: 129 pgs: 129 active+clean; 78 MiB data, 691 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-04T09:41:38.380 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.k 2024-03-04T09:41:38.381 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.k 2024-03-04T09:41:38.381 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.k 2024-03-04T09:41:38.381 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.k 2024-03-04T09:41:38.381 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.k 2024-03-04T09:41:38.381 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.k 2024-03-04T09:41:38.381 DEBUG:tasks.check_counter:Getting stats from c 2024-03-04T09:41:38.381 DEBUG:teuthology.orchestra.run.smithi191:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd shell --fsid 4e059f08-da08-11ee-95c6-87774f69a715 -- ceph daemon mds.c perf dump 2024-03-04T09:41:38.407 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:41:38 smithi191 ceph-mon[36816]: pgmap v325: 129 pgs: 129 active+clean; 78 MiB data, 691 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-04T09:41:38.585 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:41:38 smithi052 ceph-mon[32473]: pgmap v325: 129 pgs: 129 active+clean; 78 MiB data, 691 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-04T09:41:38.666 INFO:teuthology.orchestra.run.smithi191.stderr:Inferring config /var/lib/ceph/4e059f08-da08-11ee-95c6-87774f69a715/mon.c/config 2024-03-04T09:41:40.382 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:41:40 smithi106 ceph-mon[36484]: pgmap v326: 129 pgs: 129 active+clean; 78 MiB data, 691 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-04T09:41:40.445 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:41:40 smithi191 ceph-mon[36816]: pgmap v326: 129 pgs: 129 active+clean; 78 MiB data, 691 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-04T09:41:40.579 INFO:teuthology.orchestra.run.smithi191.stdout:{ 2024-03-04T09:41:40.580 INFO:teuthology.orchestra.run.smithi191.stdout: "AsyncMessenger::Worker-0": { 2024-03-04T09:41:40.580 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_recv_messages": 1609, 2024-03-04T09:41:40.580 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_send_messages": 1609, 2024-03-04T09:41:40.580 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_recv_bytes": 23739388, 2024-03-04T09:41:40.580 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_send_bytes": 510848, 2024-03-04T09:41:40.580 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_created_connections": 7, 2024-03-04T09:41:40.580 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_active_connections": 3, 2024-03-04T09:41:40.581 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_running_total_time": 0.108511087, 2024-03-04T09:41:40.581 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_running_send_time": 0.031423263, 2024-03-04T09:41:40.581 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_running_recv_time": 0.035659403, 2024-03-04T09:41:40.581 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_running_fast_dispatch_time": 0.032318408, 2024-03-04T09:41:40.581 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_send_messages_queue_lat": { 2024-03-04T09:41:40.581 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 1608, 2024-03-04T09:41:40.581 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.036984443, 2024-03-04T09:41:40.581 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000023000 2024-03-04T09:41:40.581 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-03-04T09:41:40.582 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_handle_ack_lat": { 2024-03-04T09:41:40.582 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-03-04T09:41:40.582 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-03-04T09:41:40.582 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-03-04T09:41:40.582 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-03-04T09:41:40.582 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_recv_encrypted_bytes": 3024, 2024-03-04T09:41:40.582 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_send_encrypted_bytes": 448 2024-03-04T09:41:40.582 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-03-04T09:41:40.583 INFO:teuthology.orchestra.run.smithi191.stdout: "AsyncMessenger::Worker-1": { 2024-03-04T09:41:40.583 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_recv_messages": 86, 2024-03-04T09:41:40.583 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_send_messages": 249, 2024-03-04T09:41:40.583 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_recv_bytes": 20996020, 2024-03-04T09:41:40.583 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_send_bytes": 470650, 2024-03-04T09:41:40.583 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_created_connections": 8, 2024-03-04T09:41:40.583 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_active_connections": 6, 2024-03-04T09:41:40.583 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_running_total_time": 0.022439467, 2024-03-04T09:41:40.583 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_running_send_time": 0.005895460, 2024-03-04T09:41:40.584 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_running_recv_time": 0.013101211, 2024-03-04T09:41:40.584 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_running_fast_dispatch_time": 0.001640406, 2024-03-04T09:41:40.584 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_send_messages_queue_lat": { 2024-03-04T09:41:40.584 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 246, 2024-03-04T09:41:40.584 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.017169088, 2024-03-04T09:41:40.584 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000069793 2024-03-04T09:41:40.584 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-03-04T09:41:40.584 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_handle_ack_lat": { 2024-03-04T09:41:40.585 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-03-04T09:41:40.585 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-03-04T09:41:40.585 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-03-04T09:41:40.585 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-03-04T09:41:40.585 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_recv_encrypted_bytes": 320, 2024-03-04T09:41:40.585 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_send_encrypted_bytes": 443872 2024-03-04T09:41:40.585 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-03-04T09:41:40.585 INFO:teuthology.orchestra.run.smithi191.stdout: "AsyncMessenger::Worker-2": { 2024-03-04T09:41:40.586 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_recv_messages": 995, 2024-03-04T09:41:40.586 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_send_messages": 981, 2024-03-04T09:41:40.586 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_recv_bytes": 10047824, 2024-03-04T09:41:40.586 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_send_bytes": 369821, 2024-03-04T09:41:40.586 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_created_connections": 4, 2024-03-04T09:41:40.586 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_active_connections": 3, 2024-03-04T09:41:40.586 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_running_total_time": 0.087473709, 2024-03-04T09:41:40.586 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_running_send_time": 0.025472672, 2024-03-04T09:41:40.586 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_running_recv_time": 0.045048914, 2024-03-04T09:41:40.587 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_running_fast_dispatch_time": 0.022308795, 2024-03-04T09:41:40.587 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_send_messages_queue_lat": { 2024-03-04T09:41:40.587 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 980, 2024-03-04T09:41:40.587 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.027018502, 2024-03-04T09:41:40.587 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000027569 2024-03-04T09:41:40.587 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-03-04T09:41:40.587 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_handle_ack_lat": { 2024-03-04T09:41:40.587 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-03-04T09:41:40.588 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-03-04T09:41:40.588 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-03-04T09:41:40.588 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-03-04T09:41:40.588 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_recv_encrypted_bytes": 1473936, 2024-03-04T09:41:40.588 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_send_encrypted_bytes": 132592 2024-03-04T09:41:40.588 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-03-04T09:41:40.588 INFO:teuthology.orchestra.run.smithi191.stdout: "cct": { 2024-03-04T09:41:40.588 INFO:teuthology.orchestra.run.smithi191.stdout: "total_workers": 1, 2024-03-04T09:41:40.589 INFO:teuthology.orchestra.run.smithi191.stdout: "unhealthy_workers": 0 2024-03-04T09:41:40.589 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-03-04T09:41:40.589 INFO:teuthology.orchestra.run.smithi191.stdout: "finisher-MDSRank": { 2024-03-04T09:41:40.589 INFO:teuthology.orchestra.run.smithi191.stdout: "queue_len": 0, 2024-03-04T09:41:40.589 INFO:teuthology.orchestra.run.smithi191.stdout: "complete_latency": { 2024-03-04T09:41:40.589 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 4003, 2024-03-04T09:41:40.589 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.076239916, 2024-03-04T09:41:40.589 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000019045 2024-03-04T09:41:40.590 INFO:teuthology.orchestra.run.smithi191.stdout: } 2024-03-04T09:41:40.590 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-03-04T09:41:40.590 INFO:teuthology.orchestra.run.smithi191.stdout: "finisher-PurgeQueue": { 2024-03-04T09:41:40.590 INFO:teuthology.orchestra.run.smithi191.stdout: "queue_len": 0, 2024-03-04T09:41:40.590 INFO:teuthology.orchestra.run.smithi191.stdout: "complete_latency": { 2024-03-04T09:41:40.590 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-03-04T09:41:40.590 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-03-04T09:41:40.590 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-03-04T09:41:40.591 INFO:teuthology.orchestra.run.smithi191.stdout: } 2024-03-04T09:41:40.591 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-03-04T09:41:40.591 INFO:teuthology.orchestra.run.smithi191.stdout: "mds": { 2024-03-04T09:41:40.591 INFO:teuthology.orchestra.run.smithi191.stdout: "request": 0, 2024-03-04T09:41:40.591 INFO:teuthology.orchestra.run.smithi191.stdout: "reply": 0, 2024-03-04T09:41:40.591 INFO:teuthology.orchestra.run.smithi191.stdout: "reply_latency": { 2024-03-04T09:41:40.591 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-03-04T09:41:40.591 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-03-04T09:41:40.591 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-03-04T09:41:40.592 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-03-04T09:41:40.592 INFO:teuthology.orchestra.run.smithi191.stdout: "slow_reply": 0, 2024-03-04T09:41:40.592 INFO:teuthology.orchestra.run.smithi191.stdout: "forward": 0, 2024-03-04T09:41:40.592 INFO:teuthology.orchestra.run.smithi191.stdout: "dir_fetch_complete": 0, 2024-03-04T09:41:40.592 INFO:teuthology.orchestra.run.smithi191.stdout: "dir_fetch_keys": 0, 2024-03-04T09:41:40.592 INFO:teuthology.orchestra.run.smithi191.stdout: "dir_commit": 0, 2024-03-04T09:41:40.592 INFO:teuthology.orchestra.run.smithi191.stdout: "dir_split": 0, 2024-03-04T09:41:40.592 INFO:teuthology.orchestra.run.smithi191.stdout: "dir_merge": 0, 2024-03-04T09:41:40.593 INFO:teuthology.orchestra.run.smithi191.stdout: "inodes": 9654, 2024-03-04T09:41:40.593 INFO:teuthology.orchestra.run.smithi191.stdout: "inodes_top": 0, 2024-03-04T09:41:40.593 INFO:teuthology.orchestra.run.smithi191.stdout: "inodes_bottom": 9654, 2024-03-04T09:41:40.593 INFO:teuthology.orchestra.run.smithi191.stdout: "inodes_pin_tail": 0, 2024-03-04T09:41:40.593 INFO:teuthology.orchestra.run.smithi191.stdout: "inodes_pinned": 9654, 2024-03-04T09:41:40.593 INFO:teuthology.orchestra.run.smithi191.stdout: "inodes_expired": 0, 2024-03-04T09:41:40.593 INFO:teuthology.orchestra.run.smithi191.stdout: "inodes_with_caps": 0, 2024-03-04T09:41:40.593 INFO:teuthology.orchestra.run.smithi191.stdout: "caps": 0, 2024-03-04T09:41:40.594 INFO:teuthology.orchestra.run.smithi191.stdout: "subtrees": 3, 2024-03-04T09:41:40.594 INFO:teuthology.orchestra.run.smithi191.stdout: "traverse": 0, 2024-03-04T09:41:40.594 INFO:teuthology.orchestra.run.smithi191.stdout: "traverse_hit": 0, 2024-03-04T09:41:40.594 INFO:teuthology.orchestra.run.smithi191.stdout: "traverse_forward": 0, 2024-03-04T09:41:40.594 INFO:teuthology.orchestra.run.smithi191.stdout: "traverse_discover": 0, 2024-03-04T09:41:40.594 INFO:teuthology.orchestra.run.smithi191.stdout: "traverse_dir_fetch": 0, 2024-03-04T09:41:40.594 INFO:teuthology.orchestra.run.smithi191.stdout: "traverse_remote_ino": 0, 2024-03-04T09:41:40.594 INFO:teuthology.orchestra.run.smithi191.stdout: "traverse_lock": 0, 2024-03-04T09:41:40.595 INFO:teuthology.orchestra.run.smithi191.stdout: "load_cent": 0, 2024-03-04T09:41:40.595 INFO:teuthology.orchestra.run.smithi191.stdout: "q": 0, 2024-03-04T09:41:40.595 INFO:teuthology.orchestra.run.smithi191.stdout: "exported": 0, 2024-03-04T09:41:40.595 INFO:teuthology.orchestra.run.smithi191.stdout: "exported_inodes": 0, 2024-03-04T09:41:40.595 INFO:teuthology.orchestra.run.smithi191.stdout: "imported": 0, 2024-03-04T09:41:40.595 INFO:teuthology.orchestra.run.smithi191.stdout: "imported_inodes": 0, 2024-03-04T09:41:40.595 INFO:teuthology.orchestra.run.smithi191.stdout: "openino_dir_fetch": 0, 2024-03-04T09:41:40.595 INFO:teuthology.orchestra.run.smithi191.stdout: "openino_backtrace_fetch": 0, 2024-03-04T09:41:40.595 INFO:teuthology.orchestra.run.smithi191.stdout: "openino_peer_discover": 0, 2024-03-04T09:41:40.596 INFO:teuthology.orchestra.run.smithi191.stdout: "root_rfiles": 3451, 2024-03-04T09:41:40.596 INFO:teuthology.orchestra.run.smithi191.stdout: "root_rbytes": 26478731, 2024-03-04T09:41:40.596 INFO:teuthology.orchestra.run.smithi191.stdout: "root_rsnaps": 6, 2024-03-04T09:41:40.596 INFO:teuthology.orchestra.run.smithi191.stdout: "scrub_backtrace_fetch": 0, 2024-03-04T09:41:40.596 INFO:teuthology.orchestra.run.smithi191.stdout: "scrub_set_tag": 0, 2024-03-04T09:41:40.596 INFO:teuthology.orchestra.run.smithi191.stdout: "scrub_backtrace_repaired": 0, 2024-03-04T09:41:40.596 INFO:teuthology.orchestra.run.smithi191.stdout: "scrub_inotable_repaired": 0, 2024-03-04T09:41:40.596 INFO:teuthology.orchestra.run.smithi191.stdout: "scrub_dir_inodes": 0, 2024-03-04T09:41:40.597 INFO:teuthology.orchestra.run.smithi191.stdout: "scrub_dir_base_inodes": 0, 2024-03-04T09:41:40.597 INFO:teuthology.orchestra.run.smithi191.stdout: "scrub_dirfrag_rstats": 0, 2024-03-04T09:41:40.597 INFO:teuthology.orchestra.run.smithi191.stdout: "scrub_file_inodes": 0, 2024-03-04T09:41:40.597 INFO:teuthology.orchestra.run.smithi191.stdout: "handle_inode_file_caps": 0, 2024-03-04T09:41:40.597 INFO:teuthology.orchestra.run.smithi191.stdout: "ceph_cap_op_revoke": 0, 2024-03-04T09:41:40.597 INFO:teuthology.orchestra.run.smithi191.stdout: "ceph_cap_op_grant": 0, 2024-03-04T09:41:40.597 INFO:teuthology.orchestra.run.smithi191.stdout: "ceph_cap_op_trunc": 0, 2024-03-04T09:41:40.597 INFO:teuthology.orchestra.run.smithi191.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-03-04T09:41:40.598 INFO:teuthology.orchestra.run.smithi191.stdout: "ceph_cap_op_flush_ack": 0, 2024-03-04T09:41:40.598 INFO:teuthology.orchestra.run.smithi191.stdout: "handle_client_caps": 0, 2024-03-04T09:41:40.598 INFO:teuthology.orchestra.run.smithi191.stdout: "handle_client_caps_dirty": 0, 2024-03-04T09:41:40.598 INFO:teuthology.orchestra.run.smithi191.stdout: "handle_client_cap_release": 0, 2024-03-04T09:41:40.598 INFO:teuthology.orchestra.run.smithi191.stdout: "process_request_cap_release": 0 2024-03-04T09:41:40.598 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-03-04T09:41:40.598 INFO:teuthology.orchestra.run.smithi191.stdout: "mds_cache": { 2024-03-04T09:41:40.598 INFO:teuthology.orchestra.run.smithi191.stdout: "dir_update": 0, 2024-03-04T09:41:40.599 INFO:teuthology.orchestra.run.smithi191.stdout: "dir_update_receipt": 0, 2024-03-04T09:41:40.599 INFO:teuthology.orchestra.run.smithi191.stdout: "dir_try_discover": 0, 2024-03-04T09:41:40.599 INFO:teuthology.orchestra.run.smithi191.stdout: "dir_send_discover": 0, 2024-03-04T09:41:40.599 INFO:teuthology.orchestra.run.smithi191.stdout: "dir_handle_discover": 0, 2024-03-04T09:41:40.599 INFO:teuthology.orchestra.run.smithi191.stdout: "num_strays": 0, 2024-03-04T09:41:40.599 INFO:teuthology.orchestra.run.smithi191.stdout: "num_strays_delayed": 0, 2024-03-04T09:41:40.599 INFO:teuthology.orchestra.run.smithi191.stdout: "num_strays_enqueuing": 0, 2024-03-04T09:41:40.599 INFO:teuthology.orchestra.run.smithi191.stdout: "strays_created": 0, 2024-03-04T09:41:40.600 INFO:teuthology.orchestra.run.smithi191.stdout: "strays_enqueued": 0, 2024-03-04T09:41:40.600 INFO:teuthology.orchestra.run.smithi191.stdout: "strays_reintegrated": 0, 2024-03-04T09:41:40.600 INFO:teuthology.orchestra.run.smithi191.stdout: "strays_migrated": 0, 2024-03-04T09:41:40.600 INFO:teuthology.orchestra.run.smithi191.stdout: "num_recovering_processing": 0, 2024-03-04T09:41:40.600 INFO:teuthology.orchestra.run.smithi191.stdout: "num_recovering_enqueued": 0, 2024-03-04T09:41:40.600 INFO:teuthology.orchestra.run.smithi191.stdout: "num_recovering_prioritized": 0, 2024-03-04T09:41:40.600 INFO:teuthology.orchestra.run.smithi191.stdout: "recovery_started": 0, 2024-03-04T09:41:40.600 INFO:teuthology.orchestra.run.smithi191.stdout: "recovery_completed": 0, 2024-03-04T09:41:40.601 INFO:teuthology.orchestra.run.smithi191.stdout: "ireq_enqueue_scrub": 0, 2024-03-04T09:41:40.601 INFO:teuthology.orchestra.run.smithi191.stdout: "ireq_exportdir": 0, 2024-03-04T09:41:40.601 INFO:teuthology.orchestra.run.smithi191.stdout: "ireq_flush": 0, 2024-03-04T09:41:40.601 INFO:teuthology.orchestra.run.smithi191.stdout: "ireq_fragmentdir": 0, 2024-03-04T09:41:40.601 INFO:teuthology.orchestra.run.smithi191.stdout: "ireq_fragstats": 0, 2024-03-04T09:41:40.601 INFO:teuthology.orchestra.run.smithi191.stdout: "ireq_inodestats": 0 2024-03-04T09:41:40.601 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-03-04T09:41:40.601 INFO:teuthology.orchestra.run.smithi191.stdout: "mds_log": { 2024-03-04T09:41:40.601 INFO:teuthology.orchestra.run.smithi191.stdout: "evlrg": 0, 2024-03-04T09:41:40.602 INFO:teuthology.orchestra.run.smithi191.stdout: "evadd": 0, 2024-03-04T09:41:40.602 INFO:teuthology.orchestra.run.smithi191.stdout: "evex": 0, 2024-03-04T09:41:40.602 INFO:teuthology.orchestra.run.smithi191.stdout: "evtrm": 0, 2024-03-04T09:41:40.602 INFO:teuthology.orchestra.run.smithi191.stdout: "ev": 22598, 2024-03-04T09:41:40.602 INFO:teuthology.orchestra.run.smithi191.stdout: "evexg": 0, 2024-03-04T09:41:40.602 INFO:teuthology.orchestra.run.smithi191.stdout: "evexd": 0, 2024-03-04T09:41:40.602 INFO:teuthology.orchestra.run.smithi191.stdout: "segadd": 0, 2024-03-04T09:41:40.602 INFO:teuthology.orchestra.run.smithi191.stdout: "segex": 0, 2024-03-04T09:41:40.603 INFO:teuthology.orchestra.run.smithi191.stdout: "segtrm": 0, 2024-03-04T09:41:40.603 INFO:teuthology.orchestra.run.smithi191.stdout: "seg": 29, 2024-03-04T09:41:40.603 INFO:teuthology.orchestra.run.smithi191.stdout: "segmjr": 3, 2024-03-04T09:41:40.603 INFO:teuthology.orchestra.run.smithi191.stdout: "segexg": 0, 2024-03-04T09:41:40.603 INFO:teuthology.orchestra.run.smithi191.stdout: "segexd": 0, 2024-03-04T09:41:40.603 INFO:teuthology.orchestra.run.smithi191.stdout: "expos": 4194304, 2024-03-04T09:41:40.603 INFO:teuthology.orchestra.run.smithi191.stdout: "wrpos": 56812162, 2024-03-04T09:41:40.603 INFO:teuthology.orchestra.run.smithi191.stdout: "rdpos": 56812026, 2024-03-04T09:41:40.604 INFO:teuthology.orchestra.run.smithi191.stdout: "jlat": { 2024-03-04T09:41:40.604 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-03-04T09:41:40.604 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-03-04T09:41:40.604 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-03-04T09:41:40.604 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-03-04T09:41:40.604 INFO:teuthology.orchestra.run.smithi191.stdout: "replayed": 22598 2024-03-04T09:41:40.604 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-03-04T09:41:40.604 INFO:teuthology.orchestra.run.smithi191.stdout: "mds_mem": { 2024-03-04T09:41:40.605 INFO:teuthology.orchestra.run.smithi191.stdout: "ino": 5317, 2024-03-04T09:41:40.605 INFO:teuthology.orchestra.run.smithi191.stdout: "ino+": 8673, 2024-03-04T09:41:40.605 INFO:teuthology.orchestra.run.smithi191.stdout: "ino-": 3356, 2024-03-04T09:41:40.605 INFO:teuthology.orchestra.run.smithi191.stdout: "dir": 227, 2024-03-04T09:41:40.605 INFO:teuthology.orchestra.run.smithi191.stdout: "dir+": 273, 2024-03-04T09:41:40.605 INFO:teuthology.orchestra.run.smithi191.stdout: "dir-": 46, 2024-03-04T09:41:40.605 INFO:teuthology.orchestra.run.smithi191.stdout: "dn": 9654, 2024-03-04T09:41:40.605 INFO:teuthology.orchestra.run.smithi191.stdout: "dn+": 9848, 2024-03-04T09:41:40.606 INFO:teuthology.orchestra.run.smithi191.stdout: "dn-": 194, 2024-03-04T09:41:40.606 INFO:teuthology.orchestra.run.smithi191.stdout: "cap": 0, 2024-03-04T09:41:40.606 INFO:teuthology.orchestra.run.smithi191.stdout: "cap+": 0, 2024-03-04T09:41:40.606 INFO:teuthology.orchestra.run.smithi191.stdout: "cap-": 0, 2024-03-04T09:41:40.606 INFO:teuthology.orchestra.run.smithi191.stdout: "rss": 74576, 2024-03-04T09:41:40.606 INFO:teuthology.orchestra.run.smithi191.stdout: "heap": 215420 2024-03-04T09:41:40.606 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-03-04T09:41:40.606 INFO:teuthology.orchestra.run.smithi191.stdout: "mds_server": { 2024-03-04T09:41:40.606 INFO:teuthology.orchestra.run.smithi191.stdout: "dispatch_client_request": 0, 2024-03-04T09:41:40.607 INFO:teuthology.orchestra.run.smithi191.stdout: "dispatch_server_request": 0, 2024-03-04T09:41:40.607 INFO:teuthology.orchestra.run.smithi191.stdout: "handle_client_request": 0, 2024-03-04T09:41:40.607 INFO:teuthology.orchestra.run.smithi191.stdout: "handle_client_session": 0, 2024-03-04T09:41:40.607 INFO:teuthology.orchestra.run.smithi191.stdout: "handle_peer_request": 0, 2024-03-04T09:41:40.607 INFO:teuthology.orchestra.run.smithi191.stdout: "req_create_latency": { 2024-03-04T09:41:40.607 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-03-04T09:41:40.607 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-03-04T09:41:40.607 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-03-04T09:41:40.608 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-03-04T09:41:40.608 INFO:teuthology.orchestra.run.smithi191.stdout: "req_getattr_latency": { 2024-03-04T09:41:40.608 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-03-04T09:41:40.608 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-03-04T09:41:40.608 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-03-04T09:41:40.608 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-03-04T09:41:40.608 INFO:teuthology.orchestra.run.smithi191.stdout: "req_getfilelock_latency": { 2024-03-04T09:41:40.608 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-03-04T09:41:40.609 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-03-04T09:41:40.609 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-03-04T09:41:40.609 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-03-04T09:41:40.609 INFO:teuthology.orchestra.run.smithi191.stdout: "req_link_latency": { 2024-03-04T09:41:40.609 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-03-04T09:41:40.609 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-03-04T09:41:40.609 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-03-04T09:41:40.609 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-03-04T09:41:40.609 INFO:teuthology.orchestra.run.smithi191.stdout: "req_lookup_latency": { 2024-03-04T09:41:40.610 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-03-04T09:41:40.610 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-03-04T09:41:40.610 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-03-04T09:41:40.610 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-03-04T09:41:40.610 INFO:teuthology.orchestra.run.smithi191.stdout: "req_lookuphash_latency": { 2024-03-04T09:41:40.610 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-03-04T09:41:40.610 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-03-04T09:41:40.610 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-03-04T09:41:40.611 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-03-04T09:41:40.611 INFO:teuthology.orchestra.run.smithi191.stdout: "req_lookupino_latency": { 2024-03-04T09:41:40.611 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-03-04T09:41:40.611 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-03-04T09:41:40.611 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-03-04T09:41:40.611 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-03-04T09:41:40.611 INFO:teuthology.orchestra.run.smithi191.stdout: "req_lookupname_latency": { 2024-03-04T09:41:40.611 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-03-04T09:41:40.612 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-03-04T09:41:40.612 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-03-04T09:41:40.612 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-03-04T09:41:40.612 INFO:teuthology.orchestra.run.smithi191.stdout: "req_lookupparent_latency": { 2024-03-04T09:41:40.612 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-03-04T09:41:40.612 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-03-04T09:41:40.612 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-03-04T09:41:40.612 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-03-04T09:41:40.613 INFO:teuthology.orchestra.run.smithi191.stdout: "req_lookupsnap_latency": { 2024-03-04T09:41:40.613 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-03-04T09:41:40.613 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-03-04T09:41:40.613 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-03-04T09:41:40.613 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-03-04T09:41:40.613 INFO:teuthology.orchestra.run.smithi191.stdout: "req_lssnap_latency": { 2024-03-04T09:41:40.613 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-03-04T09:41:40.613 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-03-04T09:41:40.613 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-03-04T09:41:40.614 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-03-04T09:41:40.614 INFO:teuthology.orchestra.run.smithi191.stdout: "req_mkdir_latency": { 2024-03-04T09:41:40.614 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-03-04T09:41:40.615 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:41:40 smithi052 ceph-mon[32473]: pgmap v326: 129 pgs: 129 active+clean; 78 MiB data, 691 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-04T09:41:40.615 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-03-04T09:41:40.615 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-03-04T09:41:40.615 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-03-04T09:41:40.615 INFO:teuthology.orchestra.run.smithi191.stdout: "req_mknod_latency": { 2024-03-04T09:41:40.616 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-03-04T09:41:40.616 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-03-04T09:41:40.616 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-03-04T09:41:40.616 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-03-04T09:41:40.616 INFO:teuthology.orchestra.run.smithi191.stdout: "req_mksnap_latency": { 2024-03-04T09:41:40.616 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-03-04T09:41:40.616 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-03-04T09:41:40.617 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-03-04T09:41:40.617 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-03-04T09:41:40.617 INFO:teuthology.orchestra.run.smithi191.stdout: "req_open_latency": { 2024-03-04T09:41:40.617 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-03-04T09:41:40.617 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-03-04T09:41:40.617 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-03-04T09:41:40.617 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-03-04T09:41:40.617 INFO:teuthology.orchestra.run.smithi191.stdout: "req_readdir_latency": { 2024-03-04T09:41:40.618 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-03-04T09:41:40.618 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-03-04T09:41:40.618 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-03-04T09:41:40.618 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-03-04T09:41:40.618 INFO:teuthology.orchestra.run.smithi191.stdout: "req_rename_latency": { 2024-03-04T09:41:40.618 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-03-04T09:41:40.618 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-03-04T09:41:40.618 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-03-04T09:41:40.618 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-03-04T09:41:40.619 INFO:teuthology.orchestra.run.smithi191.stdout: "req_renamesnap_latency": { 2024-03-04T09:41:40.619 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-03-04T09:41:40.619 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-03-04T09:41:40.619 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-03-04T09:41:40.619 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-03-04T09:41:40.619 INFO:teuthology.orchestra.run.smithi191.stdout: "req_snapdiff_latency": { 2024-03-04T09:41:40.619 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-03-04T09:41:40.619 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-03-04T09:41:40.619 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-03-04T09:41:40.620 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-03-04T09:41:40.620 INFO:teuthology.orchestra.run.smithi191.stdout: "req_rmdir_latency": { 2024-03-04T09:41:40.620 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-03-04T09:41:40.620 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-03-04T09:41:40.620 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-03-04T09:41:40.620 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-03-04T09:41:40.620 INFO:teuthology.orchestra.run.smithi191.stdout: "req_rmsnap_latency": { 2024-03-04T09:41:40.621 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-03-04T09:41:40.621 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-03-04T09:41:40.621 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-03-04T09:41:40.621 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-03-04T09:41:40.621 INFO:teuthology.orchestra.run.smithi191.stdout: "req_rmxattr_latency": { 2024-03-04T09:41:40.621 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-03-04T09:41:40.621 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-03-04T09:41:40.621 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-03-04T09:41:40.621 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-03-04T09:41:40.622 INFO:teuthology.orchestra.run.smithi191.stdout: "req_setattr_latency": { 2024-03-04T09:41:40.622 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-03-04T09:41:40.622 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-03-04T09:41:40.622 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-03-04T09:41:40.622 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-03-04T09:41:40.622 INFO:teuthology.orchestra.run.smithi191.stdout: "req_setdirlayout_latency": { 2024-03-04T09:41:40.622 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-03-04T09:41:40.622 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-03-04T09:41:40.623 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-03-04T09:41:40.623 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-03-04T09:41:40.623 INFO:teuthology.orchestra.run.smithi191.stdout: "req_setfilelock_latency": { 2024-03-04T09:41:40.623 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-03-04T09:41:40.623 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-03-04T09:41:40.623 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-03-04T09:41:40.623 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-03-04T09:41:40.623 INFO:teuthology.orchestra.run.smithi191.stdout: "req_setlayout_latency": { 2024-03-04T09:41:40.623 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-03-04T09:41:40.624 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-03-04T09:41:40.624 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-03-04T09:41:40.624 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-03-04T09:41:40.624 INFO:teuthology.orchestra.run.smithi191.stdout: "req_setxattr_latency": { 2024-03-04T09:41:40.624 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-03-04T09:41:40.624 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-03-04T09:41:40.624 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-03-04T09:41:40.624 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-03-04T09:41:40.625 INFO:teuthology.orchestra.run.smithi191.stdout: "req_symlink_latency": { 2024-03-04T09:41:40.625 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-03-04T09:41:40.625 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-03-04T09:41:40.625 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-03-04T09:41:40.625 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-03-04T09:41:40.625 INFO:teuthology.orchestra.run.smithi191.stdout: "req_unlink_latency": { 2024-03-04T09:41:40.625 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-03-04T09:41:40.625 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-03-04T09:41:40.626 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-03-04T09:41:40.626 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-03-04T09:41:40.626 INFO:teuthology.orchestra.run.smithi191.stdout: "cap_revoke_eviction": 0, 2024-03-04T09:41:40.626 INFO:teuthology.orchestra.run.smithi191.stdout: "cap_acquisition_throttle": 0, 2024-03-04T09:41:40.626 INFO:teuthology.orchestra.run.smithi191.stdout: "req_getvxattr_latency": { 2024-03-04T09:41:40.626 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-03-04T09:41:40.626 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-03-04T09:41:40.626 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-03-04T09:41:40.626 INFO:teuthology.orchestra.run.smithi191.stdout: } 2024-03-04T09:41:40.627 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-03-04T09:41:40.627 INFO:teuthology.orchestra.run.smithi191.stdout: "mds_sessions": { 2024-03-04T09:41:40.627 INFO:teuthology.orchestra.run.smithi191.stdout: "session_count": 2, 2024-03-04T09:41:40.627 INFO:teuthology.orchestra.run.smithi191.stdout: "session_add": 7, 2024-03-04T09:41:40.627 INFO:teuthology.orchestra.run.smithi191.stdout: "session_remove": 5, 2024-03-04T09:41:40.627 INFO:teuthology.orchestra.run.smithi191.stdout: "sessions_open": 4, 2024-03-04T09:41:40.627 INFO:teuthology.orchestra.run.smithi191.stdout: "sessions_stale": 0, 2024-03-04T09:41:40.627 INFO:teuthology.orchestra.run.smithi191.stdout: "total_load": 0, 2024-03-04T09:41:40.628 INFO:teuthology.orchestra.run.smithi191.stdout: "average_load": 0, 2024-03-04T09:41:40.628 INFO:teuthology.orchestra.run.smithi191.stdout: "avg_session_uptime": 6164, 2024-03-04T09:41:40.628 INFO:teuthology.orchestra.run.smithi191.stdout: "mdthresh_evicted": 0 2024-03-04T09:41:40.628 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-03-04T09:41:40.628 INFO:teuthology.orchestra.run.smithi191.stdout: "mempool": { 2024-03-04T09:41:40.628 INFO:teuthology.orchestra.run.smithi191.stdout: "bloom_filter_bytes": 0, 2024-03-04T09:41:40.628 INFO:teuthology.orchestra.run.smithi191.stdout: "bloom_filter_items": 0, 2024-03-04T09:41:40.628 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_alloc_bytes": 0, 2024-03-04T09:41:40.628 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_alloc_items": 0, 2024-03-04T09:41:40.629 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_cache_data_bytes": 0, 2024-03-04T09:41:40.629 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_cache_data_items": 0, 2024-03-04T09:41:40.629 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-04T09:41:40.629 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_cache_onode_items": 0, 2024-03-04T09:41:40.629 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-04T09:41:40.629 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_cache_meta_items": 0, 2024-03-04T09:41:40.629 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_cache_other_bytes": 0, 2024-03-04T09:41:40.629 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_cache_other_items": 0, 2024-03-04T09:41:40.629 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-04T09:41:40.630 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_cache_buffer_items": 0, 2024-03-04T09:41:40.630 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_extent_bytes": 0, 2024-03-04T09:41:40.630 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_extent_items": 0, 2024-03-04T09:41:40.630 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_blob_bytes": 0, 2024-03-04T09:41:40.630 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_blob_items": 0, 2024-03-04T09:41:40.630 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-04T09:41:40.630 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_shared_blob_items": 0, 2024-03-04T09:41:40.630 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-04T09:41:40.630 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_inline_bl_items": 0, 2024-03-04T09:41:40.631 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_fsck_bytes": 0, 2024-03-04T09:41:40.631 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_fsck_items": 0, 2024-03-04T09:41:40.631 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_txc_bytes": 0, 2024-03-04T09:41:40.631 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_txc_items": 0, 2024-03-04T09:41:40.631 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-04T09:41:40.631 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_writing_deferred_items": 0, 2024-03-04T09:41:40.631 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_writing_bytes": 0, 2024-03-04T09:41:40.631 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_writing_items": 0, 2024-03-04T09:41:40.632 INFO:teuthology.orchestra.run.smithi191.stdout: "bluefs_bytes": 0, 2024-03-04T09:41:40.632 INFO:teuthology.orchestra.run.smithi191.stdout: "bluefs_items": 0, 2024-03-04T09:41:40.632 INFO:teuthology.orchestra.run.smithi191.stdout: "bluefs_file_reader_bytes": 0, 2024-03-04T09:41:40.632 INFO:teuthology.orchestra.run.smithi191.stdout: "bluefs_file_reader_items": 0, 2024-03-04T09:41:40.632 INFO:teuthology.orchestra.run.smithi191.stdout: "bluefs_file_writer_bytes": 0, 2024-03-04T09:41:40.632 INFO:teuthology.orchestra.run.smithi191.stdout: "bluefs_file_writer_items": 0, 2024-03-04T09:41:40.632 INFO:teuthology.orchestra.run.smithi191.stdout: "buffer_anon_bytes": 412518, 2024-03-04T09:41:40.632 INFO:teuthology.orchestra.run.smithi191.stdout: "buffer_anon_items": 5419, 2024-03-04T09:41:40.633 INFO:teuthology.orchestra.run.smithi191.stdout: "buffer_meta_bytes": 0, 2024-03-04T09:41:40.633 INFO:teuthology.orchestra.run.smithi191.stdout: "buffer_meta_items": 0, 2024-03-04T09:41:40.633 INFO:teuthology.orchestra.run.smithi191.stdout: "osd_bytes": 0, 2024-03-04T09:41:40.633 INFO:teuthology.orchestra.run.smithi191.stdout: "osd_items": 0, 2024-03-04T09:41:40.633 INFO:teuthology.orchestra.run.smithi191.stdout: "osd_mapbl_bytes": 0, 2024-03-04T09:41:40.633 INFO:teuthology.orchestra.run.smithi191.stdout: "osd_mapbl_items": 0, 2024-03-04T09:41:40.633 INFO:teuthology.orchestra.run.smithi191.stdout: "osd_pglog_bytes": 0, 2024-03-04T09:41:40.633 INFO:teuthology.orchestra.run.smithi191.stdout: "osd_pglog_items": 0, 2024-03-04T09:41:40.634 INFO:teuthology.orchestra.run.smithi191.stdout: "osdmap_bytes": 9744, 2024-03-04T09:41:40.634 INFO:teuthology.orchestra.run.smithi191.stdout: "osdmap_items": 204, 2024-03-04T09:41:40.634 INFO:teuthology.orchestra.run.smithi191.stdout: "osdmap_mapping_bytes": 0, 2024-03-04T09:41:40.634 INFO:teuthology.orchestra.run.smithi191.stdout: "osdmap_mapping_items": 0, 2024-03-04T09:41:40.634 INFO:teuthology.orchestra.run.smithi191.stdout: "pgmap_bytes": 0, 2024-03-04T09:41:40.634 INFO:teuthology.orchestra.run.smithi191.stdout: "pgmap_items": 0, 2024-03-04T09:41:40.634 INFO:teuthology.orchestra.run.smithi191.stdout: "mds_co_bytes": 19206532, 2024-03-04T09:41:40.634 INFO:teuthology.orchestra.run.smithi191.stdout: "mds_co_items": 170364, 2024-03-04T09:41:40.635 INFO:teuthology.orchestra.run.smithi191.stdout: "unittest_1_bytes": 0, 2024-03-04T09:41:40.635 INFO:teuthology.orchestra.run.smithi191.stdout: "unittest_1_items": 0, 2024-03-04T09:41:40.635 INFO:teuthology.orchestra.run.smithi191.stdout: "unittest_2_bytes": 0, 2024-03-04T09:41:40.635 INFO:teuthology.orchestra.run.smithi191.stdout: "unittest_2_items": 0 2024-03-04T09:41:40.635 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-03-04T09:41:40.635 INFO:teuthology.orchestra.run.smithi191.stdout: "objecter": { 2024-03-04T09:41:40.635 INFO:teuthology.orchestra.run.smithi191.stdout: "op_active": 0, 2024-03-04T09:41:40.635 INFO:teuthology.orchestra.run.smithi191.stdout: "op_laggy": 0, 2024-03-04T09:41:40.636 INFO:teuthology.orchestra.run.smithi191.stdout: "op_send": 2435, 2024-03-04T09:41:40.636 INFO:teuthology.orchestra.run.smithi191.stdout: "op_send_bytes": 26, 2024-03-04T09:41:40.636 INFO:teuthology.orchestra.run.smithi191.stdout: "op_resend": 4, 2024-03-04T09:41:40.636 INFO:teuthology.orchestra.run.smithi191.stdout: "op_reply": 2435, 2024-03-04T09:41:40.636 INFO:teuthology.orchestra.run.smithi191.stdout: "op_latency": { 2024-03-04T09:41:40.636 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 2435, 2024-03-04T09:41:40.636 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 1.435975249, 2024-03-04T09:41:40.636 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000589722 2024-03-04T09:41:40.637 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-03-04T09:41:40.637 INFO:teuthology.orchestra.run.smithi191.stdout: "op_inflight": 0, 2024-03-04T09:41:40.637 INFO:teuthology.orchestra.run.smithi191.stdout: "oplen_avg": { 2024-03-04T09:41:40.637 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 2435, 2024-03-04T09:41:40.637 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 2436 2024-03-04T09:41:40.637 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-03-04T09:41:40.637 INFO:teuthology.orchestra.run.smithi191.stdout: "op": 2435, 2024-03-04T09:41:40.638 INFO:teuthology.orchestra.run.smithi191.stdout: "op_r": 2435, 2024-03-04T09:41:40.638 INFO:teuthology.orchestra.run.smithi191.stdout: "op_w": 0, 2024-03-04T09:41:40.638 INFO:teuthology.orchestra.run.smithi191.stdout: "op_rmw": 0, 2024-03-04T09:41:40.638 INFO:teuthology.orchestra.run.smithi191.stdout: "op_pg": 0, 2024-03-04T09:41:40.638 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_stat": 1572, 2024-03-04T09:41:40.638 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_create": 0, 2024-03-04T09:41:40.638 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_read": 860, 2024-03-04T09:41:40.638 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_write": 0, 2024-03-04T09:41:40.638 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_writefull": 0, 2024-03-04T09:41:40.638 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_writesame": 0, 2024-03-04T09:41:40.639 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_append": 0, 2024-03-04T09:41:40.639 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_zero": 0, 2024-03-04T09:41:40.639 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_truncate": 0, 2024-03-04T09:41:40.639 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_delete": 0, 2024-03-04T09:41:40.639 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_mapext": 0, 2024-03-04T09:41:40.639 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_sparse_read": 0, 2024-03-04T09:41:40.639 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_clonerange": 0, 2024-03-04T09:41:40.639 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_getxattr": 2, 2024-03-04T09:41:40.639 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_setxattr": 0, 2024-03-04T09:41:40.640 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_cmpxattr": 0, 2024-03-04T09:41:40.640 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_rmxattr": 0, 2024-03-04T09:41:40.640 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_resetxattrs": 0, 2024-03-04T09:41:40.640 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_call": 0, 2024-03-04T09:41:40.640 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_watch": 0, 2024-03-04T09:41:40.640 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_notify": 0, 2024-03-04T09:41:40.640 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_src_cmpxattr": 0, 2024-03-04T09:41:40.641 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_pgls": 0, 2024-03-04T09:41:40.641 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_pgls_filter": 0, 2024-03-04T09:41:40.641 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_other": 0, 2024-03-04T09:41:40.641 INFO:teuthology.orchestra.run.smithi191.stdout: "linger_active": 0, 2024-03-04T09:41:40.641 INFO:teuthology.orchestra.run.smithi191.stdout: "linger_send": 0, 2024-03-04T09:41:40.641 INFO:teuthology.orchestra.run.smithi191.stdout: "linger_resend": 0, 2024-03-04T09:41:40.641 INFO:teuthology.orchestra.run.smithi191.stdout: "linger_ping": 0, 2024-03-04T09:41:40.641 INFO:teuthology.orchestra.run.smithi191.stdout: "poolop_active": 0, 2024-03-04T09:41:40.641 INFO:teuthology.orchestra.run.smithi191.stdout: "poolop_send": 0, 2024-03-04T09:41:40.642 INFO:teuthology.orchestra.run.smithi191.stdout: "poolop_resend": 0, 2024-03-04T09:41:40.642 INFO:teuthology.orchestra.run.smithi191.stdout: "poolstat_active": 0, 2024-03-04T09:41:40.642 INFO:teuthology.orchestra.run.smithi191.stdout: "poolstat_send": 0, 2024-03-04T09:41:40.642 INFO:teuthology.orchestra.run.smithi191.stdout: "poolstat_resend": 0, 2024-03-04T09:41:40.642 INFO:teuthology.orchestra.run.smithi191.stdout: "statfs_active": 0, 2024-03-04T09:41:40.642 INFO:teuthology.orchestra.run.smithi191.stdout: "statfs_send": 0, 2024-03-04T09:41:40.642 INFO:teuthology.orchestra.run.smithi191.stdout: "statfs_resend": 0, 2024-03-04T09:41:40.642 INFO:teuthology.orchestra.run.smithi191.stdout: "command_active": 0, 2024-03-04T09:41:40.642 INFO:teuthology.orchestra.run.smithi191.stdout: "command_send": 0, 2024-03-04T09:41:40.643 INFO:teuthology.orchestra.run.smithi191.stdout: "command_resend": 0, 2024-03-04T09:41:40.643 INFO:teuthology.orchestra.run.smithi191.stdout: "map_epoch": 102, 2024-03-04T09:41:40.643 INFO:teuthology.orchestra.run.smithi191.stdout: "map_full": 0, 2024-03-04T09:41:40.643 INFO:teuthology.orchestra.run.smithi191.stdout: "map_inc": 13, 2024-03-04T09:41:40.643 INFO:teuthology.orchestra.run.smithi191.stdout: "osd_sessions": 8, 2024-03-04T09:41:40.643 INFO:teuthology.orchestra.run.smithi191.stdout: "osd_session_open": 8, 2024-03-04T09:41:40.643 INFO:teuthology.orchestra.run.smithi191.stdout: "osd_session_close": 0, 2024-03-04T09:41:40.643 INFO:teuthology.orchestra.run.smithi191.stdout: "osd_laggy": 0, 2024-03-04T09:41:40.643 INFO:teuthology.orchestra.run.smithi191.stdout: "omap_wr": 0, 2024-03-04T09:41:40.644 INFO:teuthology.orchestra.run.smithi191.stdout: "omap_rd": 2, 2024-03-04T09:41:40.644 INFO:teuthology.orchestra.run.smithi191.stdout: "omap_del": 0 2024-03-04T09:41:40.644 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-03-04T09:41:40.644 INFO:teuthology.orchestra.run.smithi191.stdout: "oft": { 2024-03-04T09:41:40.644 INFO:teuthology.orchestra.run.smithi191.stdout: "omap_total_objs": 0, 2024-03-04T09:41:40.644 INFO:teuthology.orchestra.run.smithi191.stdout: "omap_total_kv_pairs": 0, 2024-03-04T09:41:40.644 INFO:teuthology.orchestra.run.smithi191.stdout: "omap_total_updates": 0, 2024-03-04T09:41:40.645 INFO:teuthology.orchestra.run.smithi191.stdout: "omap_total_removes": 0 2024-03-04T09:41:40.645 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-03-04T09:41:40.645 INFO:teuthology.orchestra.run.smithi191.stdout: "osd-slow-ops": { 2024-03-04T09:41:40.645 INFO:teuthology.orchestra.run.smithi191.stdout: "slow_ops_count": 0 2024-03-04T09:41:40.645 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-03-04T09:41:40.645 INFO:teuthology.orchestra.run.smithi191.stdout: "purge_queue": { 2024-03-04T09:41:40.645 INFO:teuthology.orchestra.run.smithi191.stdout: "pq_executing_ops": 0, 2024-03-04T09:41:40.645 INFO:teuthology.orchestra.run.smithi191.stdout: "pq_executing_ops_high_water": 0, 2024-03-04T09:41:40.645 INFO:teuthology.orchestra.run.smithi191.stdout: "pq_executing": 0, 2024-03-04T09:41:40.646 INFO:teuthology.orchestra.run.smithi191.stdout: "pq_executing_high_water": 0, 2024-03-04T09:41:40.646 INFO:teuthology.orchestra.run.smithi191.stdout: "pq_executed_ops": 0, 2024-03-04T09:41:40.646 INFO:teuthology.orchestra.run.smithi191.stdout: "pq_executed": 0, 2024-03-04T09:41:40.646 INFO:teuthology.orchestra.run.smithi191.stdout: "pq_item_in_journal": 0 2024-03-04T09:41:40.646 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-03-04T09:41:40.646 INFO:teuthology.orchestra.run.smithi191.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-04T09:41:40.646 INFO:teuthology.orchestra.run.smithi191.stdout: "val": 0, 2024-03-04T09:41:40.646 INFO:teuthology.orchestra.run.smithi191.stdout: "max": 104857600, 2024-03-04T09:41:40.647 INFO:teuthology.orchestra.run.smithi191.stdout: "get_started": 0, 2024-03-04T09:41:40.647 INFO:teuthology.orchestra.run.smithi191.stdout: "get": 2686, 2024-03-04T09:41:40.647 INFO:teuthology.orchestra.run.smithi191.stdout: "get_sum": 54525726, 2024-03-04T09:41:40.647 INFO:teuthology.orchestra.run.smithi191.stdout: "get_or_fail_fail": 0, 2024-03-04T09:41:40.647 INFO:teuthology.orchestra.run.smithi191.stdout: "get_or_fail_success": 2686, 2024-03-04T09:41:40.647 INFO:teuthology.orchestra.run.smithi191.stdout: "take": 0, 2024-03-04T09:41:40.648 INFO:teuthology.orchestra.run.smithi191.stdout: "take_sum": 0, 2024-03-04T09:41:40.648 INFO:teuthology.orchestra.run.smithi191.stdout: "put": 2686, 2024-03-04T09:41:40.648 INFO:teuthology.orchestra.run.smithi191.stdout: "put_sum": 54525726, 2024-03-04T09:41:40.648 INFO:teuthology.orchestra.run.smithi191.stdout: "wait": { 2024-03-04T09:41:40.648 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-03-04T09:41:40.648 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-03-04T09:41:40.648 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-03-04T09:41:40.648 INFO:teuthology.orchestra.run.smithi191.stdout: } 2024-03-04T09:41:40.649 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-03-04T09:41:40.649 INFO:teuthology.orchestra.run.smithi191.stdout: "throttle-objecter_bytes": { 2024-03-04T09:41:40.649 INFO:teuthology.orchestra.run.smithi191.stdout: "val": 0, 2024-03-04T09:41:40.649 INFO:teuthology.orchestra.run.smithi191.stdout: "max": 104857600, 2024-03-04T09:41:40.649 INFO:teuthology.orchestra.run.smithi191.stdout: "get_started": 0, 2024-03-04T09:41:40.649 INFO:teuthology.orchestra.run.smithi191.stdout: "get": 0, 2024-03-04T09:41:40.649 INFO:teuthology.orchestra.run.smithi191.stdout: "get_sum": 0, 2024-03-04T09:41:40.649 INFO:teuthology.orchestra.run.smithi191.stdout: "get_or_fail_fail": 0, 2024-03-04T09:41:40.650 INFO:teuthology.orchestra.run.smithi191.stdout: "get_or_fail_success": 0, 2024-03-04T09:41:40.650 INFO:teuthology.orchestra.run.smithi191.stdout: "take": 2435, 2024-03-04T09:41:40.650 INFO:teuthology.orchestra.run.smithi191.stdout: "take_sum": 52624160, 2024-03-04T09:41:40.650 INFO:teuthology.orchestra.run.smithi191.stdout: "put": 2435, 2024-03-04T09:41:40.650 INFO:teuthology.orchestra.run.smithi191.stdout: "put_sum": 52624160, 2024-03-04T09:41:40.650 INFO:teuthology.orchestra.run.smithi191.stdout: "wait": { 2024-03-04T09:41:40.650 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-03-04T09:41:40.650 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-03-04T09:41:40.651 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-03-04T09:41:40.651 INFO:teuthology.orchestra.run.smithi191.stdout: } 2024-03-04T09:41:40.651 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-03-04T09:41:40.651 INFO:teuthology.orchestra.run.smithi191.stdout: "throttle-objecter_ops": { 2024-03-04T09:41:40.651 INFO:teuthology.orchestra.run.smithi191.stdout: "val": 0, 2024-03-04T09:41:40.651 INFO:teuthology.orchestra.run.smithi191.stdout: "max": 1024, 2024-03-04T09:41:40.651 INFO:teuthology.orchestra.run.smithi191.stdout: "get_started": 0, 2024-03-04T09:41:40.651 INFO:teuthology.orchestra.run.smithi191.stdout: "get": 0, 2024-03-04T09:41:40.651 INFO:teuthology.orchestra.run.smithi191.stdout: "get_sum": 0, 2024-03-04T09:41:40.652 INFO:teuthology.orchestra.run.smithi191.stdout: "get_or_fail_fail": 0, 2024-03-04T09:41:40.652 INFO:teuthology.orchestra.run.smithi191.stdout: "get_or_fail_success": 0, 2024-03-04T09:41:40.652 INFO:teuthology.orchestra.run.smithi191.stdout: "take": 2435, 2024-03-04T09:41:40.652 INFO:teuthology.orchestra.run.smithi191.stdout: "take_sum": 2435, 2024-03-04T09:41:40.652 INFO:teuthology.orchestra.run.smithi191.stdout: "put": 2435, 2024-03-04T09:41:40.652 INFO:teuthology.orchestra.run.smithi191.stdout: "put_sum": 2435, 2024-03-04T09:41:40.652 INFO:teuthology.orchestra.run.smithi191.stdout: "wait": { 2024-03-04T09:41:40.653 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-03-04T09:41:40.653 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-03-04T09:41:40.653 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-03-04T09:41:40.653 INFO:teuthology.orchestra.run.smithi191.stdout: } 2024-03-04T09:41:40.653 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-03-04T09:41:40.653 INFO:teuthology.orchestra.run.smithi191.stdout: "throttle-write_buf_throttle": { 2024-03-04T09:41:40.653 INFO:teuthology.orchestra.run.smithi191.stdout: "val": 0, 2024-03-04T09:41:40.653 INFO:teuthology.orchestra.run.smithi191.stdout: "max": 3758096384, 2024-03-04T09:41:40.653 INFO:teuthology.orchestra.run.smithi191.stdout: "get_started": 0, 2024-03-04T09:41:40.654 INFO:teuthology.orchestra.run.smithi191.stdout: "get": 0, 2024-03-04T09:41:40.654 INFO:teuthology.orchestra.run.smithi191.stdout: "get_sum": 0, 2024-03-04T09:41:40.654 INFO:teuthology.orchestra.run.smithi191.stdout: "get_or_fail_fail": 0, 2024-03-04T09:41:40.654 INFO:teuthology.orchestra.run.smithi191.stdout: "get_or_fail_success": 0, 2024-03-04T09:41:40.654 INFO:teuthology.orchestra.run.smithi191.stdout: "take": 0, 2024-03-04T09:41:40.654 INFO:teuthology.orchestra.run.smithi191.stdout: "take_sum": 0, 2024-03-04T09:41:40.654 INFO:teuthology.orchestra.run.smithi191.stdout: "put": 0, 2024-03-04T09:41:40.654 INFO:teuthology.orchestra.run.smithi191.stdout: "put_sum": 0, 2024-03-04T09:41:40.655 INFO:teuthology.orchestra.run.smithi191.stdout: "wait": { 2024-03-04T09:41:40.655 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-03-04T09:41:40.655 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-03-04T09:41:40.655 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-03-04T09:41:40.655 INFO:teuthology.orchestra.run.smithi191.stdout: } 2024-03-04T09:41:40.655 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-03-04T09:41:40.655 INFO:teuthology.orchestra.run.smithi191.stdout: "throttle-write_buf_throttle-0x55cc06bde960": { 2024-03-04T09:41:40.655 INFO:teuthology.orchestra.run.smithi191.stdout: "val": 0, 2024-03-04T09:41:40.656 INFO:teuthology.orchestra.run.smithi191.stdout: "max": 3758096384, 2024-03-04T09:41:40.656 INFO:teuthology.orchestra.run.smithi191.stdout: "get_started": 0, 2024-03-04T09:41:40.656 INFO:teuthology.orchestra.run.smithi191.stdout: "get": 0, 2024-03-04T09:41:40.656 INFO:teuthology.orchestra.run.smithi191.stdout: "get_sum": 0, 2024-03-04T09:41:40.656 INFO:teuthology.orchestra.run.smithi191.stdout: "get_or_fail_fail": 0, 2024-03-04T09:41:40.656 INFO:teuthology.orchestra.run.smithi191.stdout: "get_or_fail_success": 0, 2024-03-04T09:41:40.656 INFO:teuthology.orchestra.run.smithi191.stdout: "take": 0, 2024-03-04T09:41:40.656 INFO:teuthology.orchestra.run.smithi191.stdout: "take_sum": 0, 2024-03-04T09:41:40.657 INFO:teuthology.orchestra.run.smithi191.stdout: "put": 0, 2024-03-04T09:41:40.657 INFO:teuthology.orchestra.run.smithi191.stdout: "put_sum": 0, 2024-03-04T09:41:40.657 INFO:teuthology.orchestra.run.smithi191.stdout: "wait": { 2024-03-04T09:41:40.657 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-03-04T09:41:40.657 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-03-04T09:41:40.657 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-03-04T09:41:40.657 INFO:teuthology.orchestra.run.smithi191.stdout: } 2024-03-04T09:41:40.657 INFO:teuthology.orchestra.run.smithi191.stdout: } 2024-03-04T09:41:40.658 INFO:teuthology.orchestra.run.smithi191.stdout:} 2024-03-04T09:41:41.430 INFO:tasks.check_counter:Daemon mds.c mds.exported=0 2024-03-04T09:41:41.431 INFO:tasks.check_counter:Daemon mds.c mds.imported=0 2024-03-04T09:41:41.431 INFO:tasks.check_counter:Daemon mds.c mds.root_rsnaps=6 2024-03-04T09:41:41.431 INFO:tasks.check_counter:Daemon mds.c mds_server.req_mksnap_latency.avgcount=0 2024-03-04T09:41:41.431 INFO:tasks.check_counter:Daemon mds.c mds_server.req_rmsnap_latency.avgcount=0 2024-03-04T09:41:41.431 INFO:tasks.check_counter:Daemon mds.c mds.dir_split=0 2024-03-04T09:41:41.431 DEBUG:tasks.check_counter:Getting stats from f 2024-03-04T09:41:41.432 DEBUG:teuthology.orchestra.run.smithi191:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd shell --fsid 4e059f08-da08-11ee-95c6-87774f69a715 -- ceph daemon mds.f perf dump 2024-03-04T09:41:41.719 INFO:teuthology.orchestra.run.smithi191.stderr:Inferring config /var/lib/ceph/4e059f08-da08-11ee-95c6-87774f69a715/mon.c/config 2024-03-04T09:41:42.337 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:41:42 smithi191 ceph-mon[36816]: pgmap v327: 129 pgs: 129 active+clean; 78 MiB data, 691 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-04T09:41:42.585 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:41:42 smithi052 ceph-mon[32473]: pgmap v327: 129 pgs: 129 active+clean; 78 MiB data, 691 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-04T09:41:42.632 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:41:42 smithi106 ceph-mon[36484]: pgmap v327: 129 pgs: 129 active+clean; 78 MiB data, 691 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-04T09:41:43.572 INFO:teuthology.orchestra.run.smithi191.stdout:{ 2024-03-04T09:41:43.573 INFO:teuthology.orchestra.run.smithi191.stdout: "AsyncMessenger::Worker-0": { 2024-03-04T09:41:43.573 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_recv_messages": 2181, 2024-03-04T09:41:43.573 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_send_messages": 2181, 2024-03-04T09:41:43.573 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_recv_bytes": 987822, 2024-03-04T09:41:43.573 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_send_bytes": 692741, 2024-03-04T09:41:43.573 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_created_connections": 6, 2024-03-04T09:41:43.573 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_active_connections": 2, 2024-03-04T09:41:43.574 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_running_total_time": 0.121295981, 2024-03-04T09:41:43.574 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_running_send_time": 0.036206273, 2024-03-04T09:41:43.574 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_running_recv_time": 0.031436872, 2024-03-04T09:41:43.574 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_running_fast_dispatch_time": 0.044722019, 2024-03-04T09:41:43.574 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_send_messages_queue_lat": { 2024-03-04T09:41:43.574 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 2180, 2024-03-04T09:41:43.574 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.048397858, 2024-03-04T09:41:43.574 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000022200 2024-03-04T09:41:43.575 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-03-04T09:41:43.575 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_handle_ack_lat": { 2024-03-04T09:41:43.575 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-03-04T09:41:43.575 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-03-04T09:41:43.575 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-03-04T09:41:43.575 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-03-04T09:41:43.575 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_recv_encrypted_bytes": 3024, 2024-03-04T09:41:43.575 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_send_encrypted_bytes": 448 2024-03-04T09:41:43.575 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-03-04T09:41:43.576 INFO:teuthology.orchestra.run.smithi191.stdout: "AsyncMessenger::Worker-1": { 2024-03-04T09:41:43.576 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_recv_messages": 4, 2024-03-04T09:41:43.576 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_send_messages": 167, 2024-03-04T09:41:43.576 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_recv_bytes": 876, 2024-03-04T09:41:43.576 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_send_bytes": 442970, 2024-03-04T09:41:43.576 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_created_connections": 6, 2024-03-04T09:41:43.576 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_active_connections": 4, 2024-03-04T09:41:43.576 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_running_total_time": 0.008287221, 2024-03-04T09:41:43.577 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_running_send_time": 0.004787139, 2024-03-04T09:41:43.577 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_running_recv_time": 0.001911107, 2024-03-04T09:41:43.577 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_running_fast_dispatch_time": 0.000111741, 2024-03-04T09:41:43.577 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_send_messages_queue_lat": { 2024-03-04T09:41:43.577 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 164, 2024-03-04T09:41:43.577 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.012496948, 2024-03-04T09:41:43.577 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000076200 2024-03-04T09:41:43.577 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-03-04T09:41:43.578 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_handle_ack_lat": { 2024-03-04T09:41:43.578 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-03-04T09:41:43.578 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-03-04T09:41:43.578 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-03-04T09:41:43.578 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-03-04T09:41:43.578 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_recv_encrypted_bytes": 320, 2024-03-04T09:41:43.578 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_send_encrypted_bytes": 442272 2024-03-04T09:41:43.578 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-03-04T09:41:43.578 INFO:teuthology.orchestra.run.smithi191.stdout: "AsyncMessenger::Worker-2": { 2024-03-04T09:41:43.579 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_recv_messages": 250, 2024-03-04T09:41:43.579 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_send_messages": 236, 2024-03-04T09:41:43.579 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_recv_bytes": 1474316, 2024-03-04T09:41:43.579 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_send_bytes": 133374, 2024-03-04T09:41:43.579 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_created_connections": 3, 2024-03-04T09:41:43.579 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_active_connections": 2, 2024-03-04T09:41:43.579 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_running_total_time": 0.033498140, 2024-03-04T09:41:43.579 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_running_send_time": 0.008375861, 2024-03-04T09:41:43.580 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_running_recv_time": 0.031676484, 2024-03-04T09:41:43.580 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_running_fast_dispatch_time": 0.003709120, 2024-03-04T09:41:43.580 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_send_messages_queue_lat": { 2024-03-04T09:41:43.580 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 235, 2024-03-04T09:41:43.580 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.007735709, 2024-03-04T09:41:43.580 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000032917 2024-03-04T09:41:43.580 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-03-04T09:41:43.580 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_handle_ack_lat": { 2024-03-04T09:41:43.580 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-03-04T09:41:43.581 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-03-04T09:41:43.581 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-03-04T09:41:43.581 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-03-04T09:41:43.582 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_recv_encrypted_bytes": 1474048, 2024-03-04T09:41:43.582 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_send_encrypted_bytes": 133056 2024-03-04T09:41:43.582 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-03-04T09:41:43.582 INFO:teuthology.orchestra.run.smithi191.stdout: "cct": { 2024-03-04T09:41:43.582 INFO:teuthology.orchestra.run.smithi191.stdout: "total_workers": 1, 2024-03-04T09:41:43.582 INFO:teuthology.orchestra.run.smithi191.stdout: "unhealthy_workers": 0 2024-03-04T09:41:43.582 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-03-04T09:41:43.582 INFO:teuthology.orchestra.run.smithi191.stdout: "finisher-MDSRank": { 2024-03-04T09:41:43.582 INFO:teuthology.orchestra.run.smithi191.stdout: "queue_len": 0, 2024-03-04T09:41:43.583 INFO:teuthology.orchestra.run.smithi191.stdout: "complete_latency": { 2024-03-04T09:41:43.583 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 3710, 2024-03-04T09:41:43.583 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.068684477, 2024-03-04T09:41:43.583 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000018513 2024-03-04T09:41:43.583 INFO:teuthology.orchestra.run.smithi191.stdout: } 2024-03-04T09:41:43.583 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-03-04T09:41:43.583 INFO:teuthology.orchestra.run.smithi191.stdout: "finisher-PurgeQueue": { 2024-03-04T09:41:43.583 INFO:teuthology.orchestra.run.smithi191.stdout: "queue_len": 0, 2024-03-04T09:41:43.584 INFO:teuthology.orchestra.run.smithi191.stdout: "complete_latency": { 2024-03-04T09:41:43.584 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-03-04T09:41:43.584 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-03-04T09:41:43.584 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-03-04T09:41:43.584 INFO:teuthology.orchestra.run.smithi191.stdout: } 2024-03-04T09:41:43.584 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-03-04T09:41:43.584 INFO:teuthology.orchestra.run.smithi191.stdout: "mds": { 2024-03-04T09:41:43.584 INFO:teuthology.orchestra.run.smithi191.stdout: "request": 0, 2024-03-04T09:41:43.584 INFO:teuthology.orchestra.run.smithi191.stdout: "reply": 0, 2024-03-04T09:41:43.585 INFO:teuthology.orchestra.run.smithi191.stdout: "reply_latency": { 2024-03-04T09:41:43.585 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-03-04T09:41:43.585 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-03-04T09:41:43.585 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-03-04T09:41:43.585 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-03-04T09:41:43.585 INFO:teuthology.orchestra.run.smithi191.stdout: "slow_reply": 0, 2024-03-04T09:41:43.585 INFO:teuthology.orchestra.run.smithi191.stdout: "forward": 0, 2024-03-04T09:41:43.585 INFO:teuthology.orchestra.run.smithi191.stdout: "dir_fetch_complete": 0, 2024-03-04T09:41:43.586 INFO:teuthology.orchestra.run.smithi191.stdout: "dir_fetch_keys": 0, 2024-03-04T09:41:43.586 INFO:teuthology.orchestra.run.smithi191.stdout: "dir_commit": 0, 2024-03-04T09:41:43.586 INFO:teuthology.orchestra.run.smithi191.stdout: "dir_split": 0, 2024-03-04T09:41:43.586 INFO:teuthology.orchestra.run.smithi191.stdout: "dir_merge": 0, 2024-03-04T09:41:43.586 INFO:teuthology.orchestra.run.smithi191.stdout: "inodes": 130, 2024-03-04T09:41:43.586 INFO:teuthology.orchestra.run.smithi191.stdout: "inodes_top": 7, 2024-03-04T09:41:43.586 INFO:teuthology.orchestra.run.smithi191.stdout: "inodes_bottom": 123, 2024-03-04T09:41:43.586 INFO:teuthology.orchestra.run.smithi191.stdout: "inodes_pin_tail": 0, 2024-03-04T09:41:43.586 INFO:teuthology.orchestra.run.smithi191.stdout: "inodes_pinned": 120, 2024-03-04T09:41:43.587 INFO:teuthology.orchestra.run.smithi191.stdout: "inodes_expired": 0, 2024-03-04T09:41:43.587 INFO:teuthology.orchestra.run.smithi191.stdout: "inodes_with_caps": 0, 2024-03-04T09:41:43.587 INFO:teuthology.orchestra.run.smithi191.stdout: "caps": 0, 2024-03-04T09:41:43.587 INFO:teuthology.orchestra.run.smithi191.stdout: "subtrees": 5, 2024-03-04T09:41:43.587 INFO:teuthology.orchestra.run.smithi191.stdout: "traverse": 0, 2024-03-04T09:41:43.587 INFO:teuthology.orchestra.run.smithi191.stdout: "traverse_hit": 0, 2024-03-04T09:41:43.587 INFO:teuthology.orchestra.run.smithi191.stdout: "traverse_forward": 0, 2024-03-04T09:41:43.587 INFO:teuthology.orchestra.run.smithi191.stdout: "traverse_discover": 0, 2024-03-04T09:41:43.587 INFO:teuthology.orchestra.run.smithi191.stdout: "traverse_dir_fetch": 0, 2024-03-04T09:41:43.588 INFO:teuthology.orchestra.run.smithi191.stdout: "traverse_remote_ino": 0, 2024-03-04T09:41:43.588 INFO:teuthology.orchestra.run.smithi191.stdout: "traverse_lock": 0, 2024-03-04T09:41:43.588 INFO:teuthology.orchestra.run.smithi191.stdout: "load_cent": 0, 2024-03-04T09:41:43.588 INFO:teuthology.orchestra.run.smithi191.stdout: "q": 0, 2024-03-04T09:41:43.588 INFO:teuthology.orchestra.run.smithi191.stdout: "exported": 0, 2024-03-04T09:41:43.588 INFO:teuthology.orchestra.run.smithi191.stdout: "exported_inodes": 0, 2024-03-04T09:41:43.588 INFO:teuthology.orchestra.run.smithi191.stdout: "imported": 0, 2024-03-04T09:41:43.588 INFO:teuthology.orchestra.run.smithi191.stdout: "imported_inodes": 0, 2024-03-04T09:41:43.589 INFO:teuthology.orchestra.run.smithi191.stdout: "openino_dir_fetch": 0, 2024-03-04T09:41:43.589 INFO:teuthology.orchestra.run.smithi191.stdout: "openino_backtrace_fetch": 0, 2024-03-04T09:41:43.589 INFO:teuthology.orchestra.run.smithi191.stdout: "openino_peer_discover": 0, 2024-03-04T09:41:43.589 INFO:teuthology.orchestra.run.smithi191.stdout: "root_rfiles": 0, 2024-03-04T09:41:43.589 INFO:teuthology.orchestra.run.smithi191.stdout: "root_rbytes": 0, 2024-03-04T09:41:43.589 INFO:teuthology.orchestra.run.smithi191.stdout: "root_rsnaps": 0, 2024-03-04T09:41:43.589 INFO:teuthology.orchestra.run.smithi191.stdout: "scrub_backtrace_fetch": 0, 2024-03-04T09:41:43.589 INFO:teuthology.orchestra.run.smithi191.stdout: "scrub_set_tag": 0, 2024-03-04T09:41:43.590 INFO:teuthology.orchestra.run.smithi191.stdout: "scrub_backtrace_repaired": 0, 2024-03-04T09:41:43.590 INFO:teuthology.orchestra.run.smithi191.stdout: "scrub_inotable_repaired": 0, 2024-03-04T09:41:43.590 INFO:teuthology.orchestra.run.smithi191.stdout: "scrub_dir_inodes": 0, 2024-03-04T09:41:43.590 INFO:teuthology.orchestra.run.smithi191.stdout: "scrub_dir_base_inodes": 0, 2024-03-04T09:41:43.590 INFO:teuthology.orchestra.run.smithi191.stdout: "scrub_dirfrag_rstats": 0, 2024-03-04T09:41:43.590 INFO:teuthology.orchestra.run.smithi191.stdout: "scrub_file_inodes": 0, 2024-03-04T09:41:43.590 INFO:teuthology.orchestra.run.smithi191.stdout: "handle_inode_file_caps": 0, 2024-03-04T09:41:43.590 INFO:teuthology.orchestra.run.smithi191.stdout: "ceph_cap_op_revoke": 0, 2024-03-04T09:41:43.590 INFO:teuthology.orchestra.run.smithi191.stdout: "ceph_cap_op_grant": 0, 2024-03-04T09:41:43.591 INFO:teuthology.orchestra.run.smithi191.stdout: "ceph_cap_op_trunc": 0, 2024-03-04T09:41:43.591 INFO:teuthology.orchestra.run.smithi191.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-03-04T09:41:43.591 INFO:teuthology.orchestra.run.smithi191.stdout: "ceph_cap_op_flush_ack": 0, 2024-03-04T09:41:43.591 INFO:teuthology.orchestra.run.smithi191.stdout: "handle_client_caps": 0, 2024-03-04T09:41:43.591 INFO:teuthology.orchestra.run.smithi191.stdout: "handle_client_caps_dirty": 0, 2024-03-04T09:41:43.591 INFO:teuthology.orchestra.run.smithi191.stdout: "handle_client_cap_release": 0, 2024-03-04T09:41:43.591 INFO:teuthology.orchestra.run.smithi191.stdout: "process_request_cap_release": 0 2024-03-04T09:41:43.591 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-03-04T09:41:43.592 INFO:teuthology.orchestra.run.smithi191.stdout: "mds_cache": { 2024-03-04T09:41:43.592 INFO:teuthology.orchestra.run.smithi191.stdout: "dir_update": 0, 2024-03-04T09:41:43.592 INFO:teuthology.orchestra.run.smithi191.stdout: "dir_update_receipt": 0, 2024-03-04T09:41:43.592 INFO:teuthology.orchestra.run.smithi191.stdout: "dir_try_discover": 0, 2024-03-04T09:41:43.592 INFO:teuthology.orchestra.run.smithi191.stdout: "dir_send_discover": 0, 2024-03-04T09:41:43.592 INFO:teuthology.orchestra.run.smithi191.stdout: "dir_handle_discover": 0, 2024-03-04T09:41:43.592 INFO:teuthology.orchestra.run.smithi191.stdout: "num_strays": 0, 2024-03-04T09:41:43.592 INFO:teuthology.orchestra.run.smithi191.stdout: "num_strays_delayed": 0, 2024-03-04T09:41:43.592 INFO:teuthology.orchestra.run.smithi191.stdout: "num_strays_enqueuing": 0, 2024-03-04T09:41:43.593 INFO:teuthology.orchestra.run.smithi191.stdout: "strays_created": 0, 2024-03-04T09:41:43.593 INFO:teuthology.orchestra.run.smithi191.stdout: "strays_enqueued": 0, 2024-03-04T09:41:43.593 INFO:teuthology.orchestra.run.smithi191.stdout: "strays_reintegrated": 0, 2024-03-04T09:41:43.593 INFO:teuthology.orchestra.run.smithi191.stdout: "strays_migrated": 0, 2024-03-04T09:41:43.593 INFO:teuthology.orchestra.run.smithi191.stdout: "num_recovering_processing": 0, 2024-03-04T09:41:43.593 INFO:teuthology.orchestra.run.smithi191.stdout: "num_recovering_enqueued": 0, 2024-03-04T09:41:43.593 INFO:teuthology.orchestra.run.smithi191.stdout: "num_recovering_prioritized": 0, 2024-03-04T09:41:43.593 INFO:teuthology.orchestra.run.smithi191.stdout: "recovery_started": 0, 2024-03-04T09:41:43.594 INFO:teuthology.orchestra.run.smithi191.stdout: "recovery_completed": 0, 2024-03-04T09:41:43.594 INFO:teuthology.orchestra.run.smithi191.stdout: "ireq_enqueue_scrub": 0, 2024-03-04T09:41:43.594 INFO:teuthology.orchestra.run.smithi191.stdout: "ireq_exportdir": 0, 2024-03-04T09:41:43.594 INFO:teuthology.orchestra.run.smithi191.stdout: "ireq_flush": 0, 2024-03-04T09:41:43.594 INFO:teuthology.orchestra.run.smithi191.stdout: "ireq_fragmentdir": 0, 2024-03-04T09:41:43.594 INFO:teuthology.orchestra.run.smithi191.stdout: "ireq_fragstats": 0, 2024-03-04T09:41:43.594 INFO:teuthology.orchestra.run.smithi191.stdout: "ireq_inodestats": 0 2024-03-04T09:41:43.595 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-03-04T09:41:43.595 INFO:teuthology.orchestra.run.smithi191.stdout: "mds_log": { 2024-03-04T09:41:43.595 INFO:teuthology.orchestra.run.smithi191.stdout: "evlrg": 0, 2024-03-04T09:41:43.595 INFO:teuthology.orchestra.run.smithi191.stdout: "evadd": 0, 2024-03-04T09:41:43.595 INFO:teuthology.orchestra.run.smithi191.stdout: "evex": 0, 2024-03-04T09:41:43.595 INFO:teuthology.orchestra.run.smithi191.stdout: "evtrm": 0, 2024-03-04T09:41:43.595 INFO:teuthology.orchestra.run.smithi191.stdout: "ev": 134, 2024-03-04T09:41:43.595 INFO:teuthology.orchestra.run.smithi191.stdout: "evexg": 0, 2024-03-04T09:41:43.595 INFO:teuthology.orchestra.run.smithi191.stdout: "evexd": 0, 2024-03-04T09:41:43.596 INFO:teuthology.orchestra.run.smithi191.stdout: "segadd": 0, 2024-03-04T09:41:43.596 INFO:teuthology.orchestra.run.smithi191.stdout: "segex": 0, 2024-03-04T09:41:43.596 INFO:teuthology.orchestra.run.smithi191.stdout: "segtrm": 0, 2024-03-04T09:41:43.596 INFO:teuthology.orchestra.run.smithi191.stdout: "seg": 2, 2024-03-04T09:41:43.596 INFO:teuthology.orchestra.run.smithi191.stdout: "segmjr": 2, 2024-03-04T09:41:43.596 INFO:teuthology.orchestra.run.smithi191.stdout: "segexg": 0, 2024-03-04T09:41:43.596 INFO:teuthology.orchestra.run.smithi191.stdout: "segexd": 0, 2024-03-04T09:41:43.596 INFO:teuthology.orchestra.run.smithi191.stdout: "expos": 4194304, 2024-03-04T09:41:43.597 INFO:teuthology.orchestra.run.smithi191.stdout: "wrpos": 4562483, 2024-03-04T09:41:43.597 INFO:teuthology.orchestra.run.smithi191.stdout: "rdpos": 4561909, 2024-03-04T09:41:43.597 INFO:teuthology.orchestra.run.smithi191.stdout: "jlat": { 2024-03-04T09:41:43.597 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-03-04T09:41:43.597 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-03-04T09:41:43.597 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-03-04T09:41:43.597 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-03-04T09:41:43.597 INFO:teuthology.orchestra.run.smithi191.stdout: "replayed": 134 2024-03-04T09:41:43.597 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-03-04T09:41:43.598 INFO:teuthology.orchestra.run.smithi191.stdout: "mds_mem": { 2024-03-04T09:41:43.598 INFO:teuthology.orchestra.run.smithi191.stdout: "ino": 123, 2024-03-04T09:41:43.598 INFO:teuthology.orchestra.run.smithi191.stdout: "ino+": 123, 2024-03-04T09:41:43.598 INFO:teuthology.orchestra.run.smithi191.stdout: "ino-": 0, 2024-03-04T09:41:43.598 INFO:teuthology.orchestra.run.smithi191.stdout: "dir": 20, 2024-03-04T09:41:43.598 INFO:teuthology.orchestra.run.smithi191.stdout: "dir+": 20, 2024-03-04T09:41:43.598 INFO:teuthology.orchestra.run.smithi191.stdout: "dir-": 0, 2024-03-04T09:41:43.598 INFO:teuthology.orchestra.run.smithi191.stdout: "dn": 130, 2024-03-04T09:41:43.598 INFO:teuthology.orchestra.run.smithi191.stdout: "dn+": 130, 2024-03-04T09:41:43.599 INFO:teuthology.orchestra.run.smithi191.stdout: "dn-": 0, 2024-03-04T09:41:43.599 INFO:teuthology.orchestra.run.smithi191.stdout: "cap": 0, 2024-03-04T09:41:43.599 INFO:teuthology.orchestra.run.smithi191.stdout: "cap+": 0, 2024-03-04T09:41:43.599 INFO:teuthology.orchestra.run.smithi191.stdout: "cap-": 0, 2024-03-04T09:41:43.599 INFO:teuthology.orchestra.run.smithi191.stdout: "rss": 50592, 2024-03-04T09:41:43.599 INFO:teuthology.orchestra.run.smithi191.stdout: "heap": 215420 2024-03-04T09:41:43.599 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-03-04T09:41:43.599 INFO:teuthology.orchestra.run.smithi191.stdout: "mds_server": { 2024-03-04T09:41:43.600 INFO:teuthology.orchestra.run.smithi191.stdout: "dispatch_client_request": 0, 2024-03-04T09:41:43.600 INFO:teuthology.orchestra.run.smithi191.stdout: "dispatch_server_request": 0, 2024-03-04T09:41:43.600 INFO:teuthology.orchestra.run.smithi191.stdout: "handle_client_request": 0, 2024-03-04T09:41:43.600 INFO:teuthology.orchestra.run.smithi191.stdout: "handle_client_session": 0, 2024-03-04T09:41:43.600 INFO:teuthology.orchestra.run.smithi191.stdout: "handle_peer_request": 0, 2024-03-04T09:41:43.600 INFO:teuthology.orchestra.run.smithi191.stdout: "req_create_latency": { 2024-03-04T09:41:43.600 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-03-04T09:41:43.600 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-03-04T09:41:43.600 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-03-04T09:41:43.601 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-03-04T09:41:43.601 INFO:teuthology.orchestra.run.smithi191.stdout: "req_getattr_latency": { 2024-03-04T09:41:43.601 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-03-04T09:41:43.601 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-03-04T09:41:43.601 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-03-04T09:41:43.601 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-03-04T09:41:43.601 INFO:teuthology.orchestra.run.smithi191.stdout: "req_getfilelock_latency": { 2024-03-04T09:41:43.601 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-03-04T09:41:43.602 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-03-04T09:41:43.602 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-03-04T09:41:43.602 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-03-04T09:41:43.602 INFO:teuthology.orchestra.run.smithi191.stdout: "req_link_latency": { 2024-03-04T09:41:43.602 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-03-04T09:41:43.602 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-03-04T09:41:43.602 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-03-04T09:41:43.602 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-03-04T09:41:43.603 INFO:teuthology.orchestra.run.smithi191.stdout: "req_lookup_latency": { 2024-03-04T09:41:43.603 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-03-04T09:41:43.603 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-03-04T09:41:43.603 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-03-04T09:41:43.603 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-03-04T09:41:43.603 INFO:teuthology.orchestra.run.smithi191.stdout: "req_lookuphash_latency": { 2024-03-04T09:41:43.603 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-03-04T09:41:43.603 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-03-04T09:41:43.604 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-03-04T09:41:43.604 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-03-04T09:41:43.604 INFO:teuthology.orchestra.run.smithi191.stdout: "req_lookupino_latency": { 2024-03-04T09:41:43.604 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-03-04T09:41:43.604 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-03-04T09:41:43.604 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-03-04T09:41:43.604 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-03-04T09:41:43.604 INFO:teuthology.orchestra.run.smithi191.stdout: "req_lookupname_latency": { 2024-03-04T09:41:43.605 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-03-04T09:41:43.605 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-03-04T09:41:43.605 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-03-04T09:41:43.605 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-03-04T09:41:43.605 INFO:teuthology.orchestra.run.smithi191.stdout: "req_lookupparent_latency": { 2024-03-04T09:41:43.605 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-03-04T09:41:43.605 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-03-04T09:41:43.606 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-03-04T09:41:43.606 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-03-04T09:41:43.606 INFO:teuthology.orchestra.run.smithi191.stdout: "req_lookupsnap_latency": { 2024-03-04T09:41:43.606 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-03-04T09:41:43.606 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-03-04T09:41:43.606 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-03-04T09:41:43.606 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-03-04T09:41:43.606 INFO:teuthology.orchestra.run.smithi191.stdout: "req_lssnap_latency": { 2024-03-04T09:41:43.607 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-03-04T09:41:43.607 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-03-04T09:41:43.607 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-03-04T09:41:43.607 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-03-04T09:41:43.607 INFO:teuthology.orchestra.run.smithi191.stdout: "req_mkdir_latency": { 2024-03-04T09:41:43.607 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-03-04T09:41:43.607 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-03-04T09:41:43.608 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-03-04T09:41:43.608 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-03-04T09:41:43.609 INFO:teuthology.orchestra.run.smithi191.stdout: "req_mknod_latency": { 2024-03-04T09:41:43.610 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-03-04T09:41:43.610 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-03-04T09:41:43.610 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-03-04T09:41:43.610 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-03-04T09:41:43.610 INFO:teuthology.orchestra.run.smithi191.stdout: "req_mksnap_latency": { 2024-03-04T09:41:43.610 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-03-04T09:41:43.610 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-03-04T09:41:43.610 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-03-04T09:41:43.611 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-03-04T09:41:43.611 INFO:teuthology.orchestra.run.smithi191.stdout: "req_open_latency": { 2024-03-04T09:41:43.611 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-03-04T09:41:43.611 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-03-04T09:41:43.611 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-03-04T09:41:43.611 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-03-04T09:41:43.611 INFO:teuthology.orchestra.run.smithi191.stdout: "req_readdir_latency": { 2024-03-04T09:41:43.612 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-03-04T09:41:43.612 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-03-04T09:41:43.612 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-03-04T09:41:43.612 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-03-04T09:41:43.612 INFO:teuthology.orchestra.run.smithi191.stdout: "req_rename_latency": { 2024-03-04T09:41:43.612 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-03-04T09:41:43.612 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-03-04T09:41:43.612 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-03-04T09:41:43.613 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-03-04T09:41:43.613 INFO:teuthology.orchestra.run.smithi191.stdout: "req_renamesnap_latency": { 2024-03-04T09:41:43.613 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-03-04T09:41:43.613 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-03-04T09:41:43.613 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-03-04T09:41:43.613 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-03-04T09:41:43.613 INFO:teuthology.orchestra.run.smithi191.stdout: "req_snapdiff_latency": { 2024-03-04T09:41:43.614 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-03-04T09:41:43.614 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-03-04T09:41:43.614 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-03-04T09:41:43.614 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-03-04T09:41:43.614 INFO:teuthology.orchestra.run.smithi191.stdout: "req_rmdir_latency": { 2024-03-04T09:41:43.614 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-03-04T09:41:43.614 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-03-04T09:41:43.615 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-03-04T09:41:43.615 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-03-04T09:41:43.615 INFO:teuthology.orchestra.run.smithi191.stdout: "req_rmsnap_latency": { 2024-03-04T09:41:43.615 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-03-04T09:41:43.615 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-03-04T09:41:43.615 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-03-04T09:41:43.615 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-03-04T09:41:43.615 INFO:teuthology.orchestra.run.smithi191.stdout: "req_rmxattr_latency": { 2024-03-04T09:41:43.616 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-03-04T09:41:43.616 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-03-04T09:41:43.616 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-03-04T09:41:43.616 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-03-04T09:41:43.616 INFO:teuthology.orchestra.run.smithi191.stdout: "req_setattr_latency": { 2024-03-04T09:41:43.616 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-03-04T09:41:43.616 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-03-04T09:41:43.617 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-03-04T09:41:43.617 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-03-04T09:41:43.617 INFO:teuthology.orchestra.run.smithi191.stdout: "req_setdirlayout_latency": { 2024-03-04T09:41:43.617 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-03-04T09:41:43.617 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-03-04T09:41:43.617 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-03-04T09:41:43.617 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-03-04T09:41:43.617 INFO:teuthology.orchestra.run.smithi191.stdout: "req_setfilelock_latency": { 2024-03-04T09:41:43.617 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-03-04T09:41:43.618 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-03-04T09:41:43.618 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-03-04T09:41:43.618 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-03-04T09:41:43.618 INFO:teuthology.orchestra.run.smithi191.stdout: "req_setlayout_latency": { 2024-03-04T09:41:43.618 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-03-04T09:41:43.618 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-03-04T09:41:43.618 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-03-04T09:41:43.618 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-03-04T09:41:43.619 INFO:teuthology.orchestra.run.smithi191.stdout: "req_setxattr_latency": { 2024-03-04T09:41:43.619 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-03-04T09:41:43.619 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-03-04T09:41:43.619 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-03-04T09:41:43.619 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-03-04T09:41:43.619 INFO:teuthology.orchestra.run.smithi191.stdout: "req_symlink_latency": { 2024-03-04T09:41:43.619 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-03-04T09:41:43.619 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-03-04T09:41:43.619 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-03-04T09:41:43.620 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-03-04T09:41:43.620 INFO:teuthology.orchestra.run.smithi191.stdout: "req_unlink_latency": { 2024-03-04T09:41:43.620 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-03-04T09:41:43.620 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-03-04T09:41:43.620 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-03-04T09:41:43.620 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-03-04T09:41:43.620 INFO:teuthology.orchestra.run.smithi191.stdout: "cap_revoke_eviction": 0, 2024-03-04T09:41:43.621 INFO:teuthology.orchestra.run.smithi191.stdout: "cap_acquisition_throttle": 0, 2024-03-04T09:41:43.621 INFO:teuthology.orchestra.run.smithi191.stdout: "req_getvxattr_latency": { 2024-03-04T09:41:43.621 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-03-04T09:41:43.621 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-03-04T09:41:43.621 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-03-04T09:41:43.621 INFO:teuthology.orchestra.run.smithi191.stdout: } 2024-03-04T09:41:43.621 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-03-04T09:41:43.621 INFO:teuthology.orchestra.run.smithi191.stdout: "mds_sessions": { 2024-03-04T09:41:43.621 INFO:teuthology.orchestra.run.smithi191.stdout: "session_count": 2, 2024-03-04T09:41:43.622 INFO:teuthology.orchestra.run.smithi191.stdout: "session_add": 4, 2024-03-04T09:41:43.622 INFO:teuthology.orchestra.run.smithi191.stdout: "session_remove": 2, 2024-03-04T09:41:43.622 INFO:teuthology.orchestra.run.smithi191.stdout: "sessions_open": 2, 2024-03-04T09:41:43.622 INFO:teuthology.orchestra.run.smithi191.stdout: "sessions_stale": 0, 2024-03-04T09:41:43.622 INFO:teuthology.orchestra.run.smithi191.stdout: "total_load": 0, 2024-03-04T09:41:43.622 INFO:teuthology.orchestra.run.smithi191.stdout: "average_load": 0, 2024-03-04T09:41:43.622 INFO:teuthology.orchestra.run.smithi191.stdout: "avg_session_uptime": 3407, 2024-03-04T09:41:43.622 INFO:teuthology.orchestra.run.smithi191.stdout: "mdthresh_evicted": 0 2024-03-04T09:41:43.622 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-03-04T09:41:43.623 INFO:teuthology.orchestra.run.smithi191.stdout: "mempool": { 2024-03-04T09:41:43.623 INFO:teuthology.orchestra.run.smithi191.stdout: "bloom_filter_bytes": 0, 2024-03-04T09:41:43.623 INFO:teuthology.orchestra.run.smithi191.stdout: "bloom_filter_items": 0, 2024-03-04T09:41:43.623 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_alloc_bytes": 0, 2024-03-04T09:41:43.623 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_alloc_items": 0, 2024-03-04T09:41:43.623 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_cache_data_bytes": 0, 2024-03-04T09:41:43.623 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_cache_data_items": 0, 2024-03-04T09:41:43.623 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-04T09:41:43.624 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_cache_onode_items": 0, 2024-03-04T09:41:43.624 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-04T09:41:43.624 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_cache_meta_items": 0, 2024-03-04T09:41:43.624 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_cache_other_bytes": 0, 2024-03-04T09:41:43.624 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_cache_other_items": 0, 2024-03-04T09:41:43.624 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-04T09:41:43.624 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_cache_buffer_items": 0, 2024-03-04T09:41:43.624 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_extent_bytes": 0, 2024-03-04T09:41:43.624 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_extent_items": 0, 2024-03-04T09:41:43.625 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_blob_bytes": 0, 2024-03-04T09:41:43.625 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_blob_items": 0, 2024-03-04T09:41:43.625 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-04T09:41:43.625 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_shared_blob_items": 0, 2024-03-04T09:41:43.625 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-04T09:41:43.625 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_inline_bl_items": 0, 2024-03-04T09:41:43.625 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_fsck_bytes": 0, 2024-03-04T09:41:43.625 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_fsck_items": 0, 2024-03-04T09:41:43.626 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_txc_bytes": 0, 2024-03-04T09:41:43.626 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_txc_items": 0, 2024-03-04T09:41:43.626 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-04T09:41:43.626 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_writing_deferred_items": 0, 2024-03-04T09:41:43.626 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_writing_bytes": 0, 2024-03-04T09:41:43.626 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_writing_items": 0, 2024-03-04T09:41:43.626 INFO:teuthology.orchestra.run.smithi191.stdout: "bluefs_bytes": 0, 2024-03-04T09:41:43.626 INFO:teuthology.orchestra.run.smithi191.stdout: "bluefs_items": 0, 2024-03-04T09:41:43.627 INFO:teuthology.orchestra.run.smithi191.stdout: "bluefs_file_reader_bytes": 0, 2024-03-04T09:41:43.627 INFO:teuthology.orchestra.run.smithi191.stdout: "bluefs_file_reader_items": 0, 2024-03-04T09:41:43.627 INFO:teuthology.orchestra.run.smithi191.stdout: "bluefs_file_writer_bytes": 0, 2024-03-04T09:41:43.627 INFO:teuthology.orchestra.run.smithi191.stdout: "bluefs_file_writer_items": 0, 2024-03-04T09:41:43.627 INFO:teuthology.orchestra.run.smithi191.stdout: "buffer_anon_bytes": 185833, 2024-03-04T09:41:43.627 INFO:teuthology.orchestra.run.smithi191.stdout: "buffer_anon_items": 169, 2024-03-04T09:41:43.627 INFO:teuthology.orchestra.run.smithi191.stdout: "buffer_meta_bytes": 0, 2024-03-04T09:41:43.627 INFO:teuthology.orchestra.run.smithi191.stdout: "buffer_meta_items": 0, 2024-03-04T09:41:43.627 INFO:teuthology.orchestra.run.smithi191.stdout: "osd_bytes": 0, 2024-03-04T09:41:43.628 INFO:teuthology.orchestra.run.smithi191.stdout: "osd_items": 0, 2024-03-04T09:41:43.628 INFO:teuthology.orchestra.run.smithi191.stdout: "osd_mapbl_bytes": 0, 2024-03-04T09:41:43.628 INFO:teuthology.orchestra.run.smithi191.stdout: "osd_mapbl_items": 0, 2024-03-04T09:41:43.628 INFO:teuthology.orchestra.run.smithi191.stdout: "osd_pglog_bytes": 0, 2024-03-04T09:41:43.628 INFO:teuthology.orchestra.run.smithi191.stdout: "osd_pglog_items": 0, 2024-03-04T09:41:43.628 INFO:teuthology.orchestra.run.smithi191.stdout: "osdmap_bytes": 9744, 2024-03-04T09:41:43.628 INFO:teuthology.orchestra.run.smithi191.stdout: "osdmap_items": 204, 2024-03-04T09:41:43.628 INFO:teuthology.orchestra.run.smithi191.stdout: "osdmap_mapping_bytes": 0, 2024-03-04T09:41:43.629 INFO:teuthology.orchestra.run.smithi191.stdout: "osdmap_mapping_items": 0, 2024-03-04T09:41:43.629 INFO:teuthology.orchestra.run.smithi191.stdout: "pgmap_bytes": 0, 2024-03-04T09:41:43.629 INFO:teuthology.orchestra.run.smithi191.stdout: "pgmap_items": 0, 2024-03-04T09:41:43.629 INFO:teuthology.orchestra.run.smithi191.stdout: "mds_co_bytes": 403212, 2024-03-04T09:41:43.629 INFO:teuthology.orchestra.run.smithi191.stdout: "mds_co_items": 3967, 2024-03-04T09:41:43.629 INFO:teuthology.orchestra.run.smithi191.stdout: "unittest_1_bytes": 0, 2024-03-04T09:41:43.629 INFO:teuthology.orchestra.run.smithi191.stdout: "unittest_1_items": 0, 2024-03-04T09:41:43.629 INFO:teuthology.orchestra.run.smithi191.stdout: "unittest_2_bytes": 0, 2024-03-04T09:41:43.630 INFO:teuthology.orchestra.run.smithi191.stdout: "unittest_2_items": 0 2024-03-04T09:41:43.630 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-03-04T09:41:43.630 INFO:teuthology.orchestra.run.smithi191.stdout: "objecter": { 2024-03-04T09:41:43.630 INFO:teuthology.orchestra.run.smithi191.stdout: "op_active": 0, 2024-03-04T09:41:43.630 INFO:teuthology.orchestra.run.smithi191.stdout: "op_laggy": 0, 2024-03-04T09:41:43.630 INFO:teuthology.orchestra.run.smithi191.stdout: "op_send": 2180, 2024-03-04T09:41:43.630 INFO:teuthology.orchestra.run.smithi191.stdout: "op_send_bytes": 21, 2024-03-04T09:41:43.631 INFO:teuthology.orchestra.run.smithi191.stdout: "op_resend": 3, 2024-03-04T09:41:43.631 INFO:teuthology.orchestra.run.smithi191.stdout: "op_reply": 2180, 2024-03-04T09:41:43.631 INFO:teuthology.orchestra.run.smithi191.stdout: "op_latency": { 2024-03-04T09:41:43.631 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 2180, 2024-03-04T09:41:43.631 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.899984537, 2024-03-04T09:41:43.631 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000412836 2024-03-04T09:41:43.631 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-03-04T09:41:43.631 INFO:teuthology.orchestra.run.smithi191.stdout: "op_inflight": 0, 2024-03-04T09:41:43.632 INFO:teuthology.orchestra.run.smithi191.stdout: "oplen_avg": { 2024-03-04T09:41:43.632 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 2180, 2024-03-04T09:41:43.632 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 2181 2024-03-04T09:41:43.632 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-03-04T09:41:43.632 INFO:teuthology.orchestra.run.smithi191.stdout: "op": 2180, 2024-03-04T09:41:43.632 INFO:teuthology.orchestra.run.smithi191.stdout: "op_r": 2180, 2024-03-04T09:41:43.632 INFO:teuthology.orchestra.run.smithi191.stdout: "op_w": 0, 2024-03-04T09:41:43.632 INFO:teuthology.orchestra.run.smithi191.stdout: "op_rmw": 0, 2024-03-04T09:41:43.632 INFO:teuthology.orchestra.run.smithi191.stdout: "op_pg": 0, 2024-03-04T09:41:43.633 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_stat": 1401, 2024-03-04T09:41:43.633 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_create": 0, 2024-03-04T09:41:43.633 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_read": 777, 2024-03-04T09:41:43.633 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_write": 0, 2024-03-04T09:41:43.633 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_writefull": 0, 2024-03-04T09:41:43.633 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_writesame": 0, 2024-03-04T09:41:43.633 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_append": 0, 2024-03-04T09:41:43.633 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_zero": 0, 2024-03-04T09:41:43.633 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_truncate": 0, 2024-03-04T09:41:43.634 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_delete": 0, 2024-03-04T09:41:43.634 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_mapext": 0, 2024-03-04T09:41:43.634 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_sparse_read": 0, 2024-03-04T09:41:43.634 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_clonerange": 0, 2024-03-04T09:41:43.634 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_getxattr": 1, 2024-03-04T09:41:43.634 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_setxattr": 0, 2024-03-04T09:41:43.634 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_cmpxattr": 0, 2024-03-04T09:41:43.634 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_rmxattr": 0, 2024-03-04T09:41:43.634 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_resetxattrs": 0, 2024-03-04T09:41:43.635 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_call": 0, 2024-03-04T09:41:43.635 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_watch": 0, 2024-03-04T09:41:43.635 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_notify": 0, 2024-03-04T09:41:43.635 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_src_cmpxattr": 0, 2024-03-04T09:41:43.635 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_pgls": 0, 2024-03-04T09:41:43.635 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_pgls_filter": 0, 2024-03-04T09:41:43.635 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_other": 0, 2024-03-04T09:41:43.635 INFO:teuthology.orchestra.run.smithi191.stdout: "linger_active": 0, 2024-03-04T09:41:43.635 INFO:teuthology.orchestra.run.smithi191.stdout: "linger_send": 0, 2024-03-04T09:41:43.636 INFO:teuthology.orchestra.run.smithi191.stdout: "linger_resend": 0, 2024-03-04T09:41:43.636 INFO:teuthology.orchestra.run.smithi191.stdout: "linger_ping": 0, 2024-03-04T09:41:43.636 INFO:teuthology.orchestra.run.smithi191.stdout: "poolop_active": 0, 2024-03-04T09:41:43.636 INFO:teuthology.orchestra.run.smithi191.stdout: "poolop_send": 0, 2024-03-04T09:41:43.636 INFO:teuthology.orchestra.run.smithi191.stdout: "poolop_resend": 0, 2024-03-04T09:41:43.636 INFO:teuthology.orchestra.run.smithi191.stdout: "poolstat_active": 0, 2024-03-04T09:41:43.637 INFO:teuthology.orchestra.run.smithi191.stdout: "poolstat_send": 0, 2024-03-04T09:41:43.637 INFO:teuthology.orchestra.run.smithi191.stdout: "poolstat_resend": 0, 2024-03-04T09:41:43.637 INFO:teuthology.orchestra.run.smithi191.stdout: "statfs_active": 0, 2024-03-04T09:41:43.637 INFO:teuthology.orchestra.run.smithi191.stdout: "statfs_send": 0, 2024-03-04T09:41:43.637 INFO:teuthology.orchestra.run.smithi191.stdout: "statfs_resend": 0, 2024-03-04T09:41:43.637 INFO:teuthology.orchestra.run.smithi191.stdout: "command_active": 0, 2024-03-04T09:41:43.638 INFO:teuthology.orchestra.run.smithi191.stdout: "command_send": 0, 2024-03-04T09:41:43.638 INFO:teuthology.orchestra.run.smithi191.stdout: "command_resend": 0, 2024-03-04T09:41:43.638 INFO:teuthology.orchestra.run.smithi191.stdout: "map_epoch": 102, 2024-03-04T09:41:43.638 INFO:teuthology.orchestra.run.smithi191.stdout: "map_full": 0, 2024-03-04T09:41:43.638 INFO:teuthology.orchestra.run.smithi191.stdout: "map_inc": 13, 2024-03-04T09:41:43.638 INFO:teuthology.orchestra.run.smithi191.stdout: "osd_sessions": 4, 2024-03-04T09:41:43.638 INFO:teuthology.orchestra.run.smithi191.stdout: "osd_session_open": 4, 2024-03-04T09:41:43.638 INFO:teuthology.orchestra.run.smithi191.stdout: "osd_session_close": 0, 2024-03-04T09:41:43.638 INFO:teuthology.orchestra.run.smithi191.stdout: "osd_laggy": 0, 2024-03-04T09:41:43.639 INFO:teuthology.orchestra.run.smithi191.stdout: "omap_wr": 0, 2024-03-04T09:41:43.639 INFO:teuthology.orchestra.run.smithi191.stdout: "omap_rd": 2, 2024-03-04T09:41:43.639 INFO:teuthology.orchestra.run.smithi191.stdout: "omap_del": 0 2024-03-04T09:41:43.639 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-03-04T09:41:43.639 INFO:teuthology.orchestra.run.smithi191.stdout: "oft": { 2024-03-04T09:41:43.639 INFO:teuthology.orchestra.run.smithi191.stdout: "omap_total_objs": 0, 2024-03-04T09:41:43.639 INFO:teuthology.orchestra.run.smithi191.stdout: "omap_total_kv_pairs": 0, 2024-03-04T09:41:43.639 INFO:teuthology.orchestra.run.smithi191.stdout: "omap_total_updates": 0, 2024-03-04T09:41:43.639 INFO:teuthology.orchestra.run.smithi191.stdout: "omap_total_removes": 0 2024-03-04T09:41:43.640 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-03-04T09:41:43.640 INFO:teuthology.orchestra.run.smithi191.stdout: "osd-slow-ops": { 2024-03-04T09:41:43.640 INFO:teuthology.orchestra.run.smithi191.stdout: "slow_ops_count": 0 2024-03-04T09:41:43.640 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-03-04T09:41:43.640 INFO:teuthology.orchestra.run.smithi191.stdout: "purge_queue": { 2024-03-04T09:41:43.640 INFO:teuthology.orchestra.run.smithi191.stdout: "pq_executing_ops": 0, 2024-03-04T09:41:43.640 INFO:teuthology.orchestra.run.smithi191.stdout: "pq_executing_ops_high_water": 0, 2024-03-04T09:41:43.640 INFO:teuthology.orchestra.run.smithi191.stdout: "pq_executing": 0, 2024-03-04T09:41:43.641 INFO:teuthology.orchestra.run.smithi191.stdout: "pq_executing_high_water": 0, 2024-03-04T09:41:43.641 INFO:teuthology.orchestra.run.smithi191.stdout: "pq_executed_ops": 0, 2024-03-04T09:41:43.641 INFO:teuthology.orchestra.run.smithi191.stdout: "pq_executed": 0, 2024-03-04T09:41:43.641 INFO:teuthology.orchestra.run.smithi191.stdout: "pq_item_in_journal": 0 2024-03-04T09:41:43.641 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-03-04T09:41:43.641 INFO:teuthology.orchestra.run.smithi191.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-04T09:41:43.641 INFO:teuthology.orchestra.run.smithi191.stdout: "val": 0, 2024-03-04T09:41:43.641 INFO:teuthology.orchestra.run.smithi191.stdout: "max": 104857600, 2024-03-04T09:41:43.641 INFO:teuthology.orchestra.run.smithi191.stdout: "get_started": 0, 2024-03-04T09:41:43.642 INFO:teuthology.orchestra.run.smithi191.stdout: "get": 2431, 2024-03-04T09:41:43.642 INFO:teuthology.orchestra.run.smithi191.stdout: "get_sum": 2228452, 2024-03-04T09:41:43.642 INFO:teuthology.orchestra.run.smithi191.stdout: "get_or_fail_fail": 0, 2024-03-04T09:41:43.642 INFO:teuthology.orchestra.run.smithi191.stdout: "get_or_fail_success": 2431, 2024-03-04T09:41:43.642 INFO:teuthology.orchestra.run.smithi191.stdout: "take": 0, 2024-03-04T09:41:43.642 INFO:teuthology.orchestra.run.smithi191.stdout: "take_sum": 0, 2024-03-04T09:41:43.642 INFO:teuthology.orchestra.run.smithi191.stdout: "put": 2431, 2024-03-04T09:41:43.642 INFO:teuthology.orchestra.run.smithi191.stdout: "put_sum": 2228452, 2024-03-04T09:41:43.643 INFO:teuthology.orchestra.run.smithi191.stdout: "wait": { 2024-03-04T09:41:43.643 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-03-04T09:41:43.643 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-03-04T09:41:43.643 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-03-04T09:41:43.643 INFO:teuthology.orchestra.run.smithi191.stdout: } 2024-03-04T09:41:43.643 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-03-04T09:41:43.644 INFO:teuthology.orchestra.run.smithi191.stdout: "throttle-objecter_bytes": { 2024-03-04T09:41:43.644 INFO:teuthology.orchestra.run.smithi191.stdout: "val": 0, 2024-03-04T09:41:43.644 INFO:teuthology.orchestra.run.smithi191.stdout: "max": 104857600, 2024-03-04T09:41:43.644 INFO:teuthology.orchestra.run.smithi191.stdout: "get_started": 0, 2024-03-04T09:41:43.644 INFO:teuthology.orchestra.run.smithi191.stdout: "get": 0, 2024-03-04T09:41:43.644 INFO:teuthology.orchestra.run.smithi191.stdout: "get_sum": 0, 2024-03-04T09:41:43.644 INFO:teuthology.orchestra.run.smithi191.stdout: "get_or_fail_fail": 0, 2024-03-04T09:41:43.644 INFO:teuthology.orchestra.run.smithi191.stdout: "get_or_fail_success": 0, 2024-03-04T09:41:43.645 INFO:teuthology.orchestra.run.smithi191.stdout: "take": 2180, 2024-03-04T09:41:43.645 INFO:teuthology.orchestra.run.smithi191.stdout: "take_sum": 368184, 2024-03-04T09:41:43.645 INFO:teuthology.orchestra.run.smithi191.stdout: "put": 2180, 2024-03-04T09:41:43.645 INFO:teuthology.orchestra.run.smithi191.stdout: "put_sum": 368184, 2024-03-04T09:41:43.645 INFO:teuthology.orchestra.run.smithi191.stdout: "wait": { 2024-03-04T09:41:43.645 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-03-04T09:41:43.645 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-03-04T09:41:43.645 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-03-04T09:41:43.646 INFO:teuthology.orchestra.run.smithi191.stdout: } 2024-03-04T09:41:43.646 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-03-04T09:41:43.646 INFO:teuthology.orchestra.run.smithi191.stdout: "throttle-objecter_ops": { 2024-03-04T09:41:43.646 INFO:teuthology.orchestra.run.smithi191.stdout: "val": 0, 2024-03-04T09:41:43.646 INFO:teuthology.orchestra.run.smithi191.stdout: "max": 1024, 2024-03-04T09:41:43.646 INFO:teuthology.orchestra.run.smithi191.stdout: "get_started": 0, 2024-03-04T09:41:43.646 INFO:teuthology.orchestra.run.smithi191.stdout: "get": 0, 2024-03-04T09:41:43.646 INFO:teuthology.orchestra.run.smithi191.stdout: "get_sum": 0, 2024-03-04T09:41:43.646 INFO:teuthology.orchestra.run.smithi191.stdout: "get_or_fail_fail": 0, 2024-03-04T09:41:43.647 INFO:teuthology.orchestra.run.smithi191.stdout: "get_or_fail_success": 0, 2024-03-04T09:41:43.647 INFO:teuthology.orchestra.run.smithi191.stdout: "take": 2180, 2024-03-04T09:41:43.647 INFO:teuthology.orchestra.run.smithi191.stdout: "take_sum": 2180, 2024-03-04T09:41:43.647 INFO:teuthology.orchestra.run.smithi191.stdout: "put": 2180, 2024-03-04T09:41:43.647 INFO:teuthology.orchestra.run.smithi191.stdout: "put_sum": 2180, 2024-03-04T09:41:43.647 INFO:teuthology.orchestra.run.smithi191.stdout: "wait": { 2024-03-04T09:41:43.647 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-03-04T09:41:43.647 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-03-04T09:41:43.647 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-03-04T09:41:43.648 INFO:teuthology.orchestra.run.smithi191.stdout: } 2024-03-04T09:41:43.648 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-03-04T09:41:43.648 INFO:teuthology.orchestra.run.smithi191.stdout: "throttle-write_buf_throttle": { 2024-03-04T09:41:43.648 INFO:teuthology.orchestra.run.smithi191.stdout: "val": 0, 2024-03-04T09:41:43.648 INFO:teuthology.orchestra.run.smithi191.stdout: "max": 3758096384, 2024-03-04T09:41:43.648 INFO:teuthology.orchestra.run.smithi191.stdout: "get_started": 0, 2024-03-04T09:41:43.648 INFO:teuthology.orchestra.run.smithi191.stdout: "get": 0, 2024-03-04T09:41:43.648 INFO:teuthology.orchestra.run.smithi191.stdout: "get_sum": 0, 2024-03-04T09:41:43.649 INFO:teuthology.orchestra.run.smithi191.stdout: "get_or_fail_fail": 0, 2024-03-04T09:41:43.649 INFO:teuthology.orchestra.run.smithi191.stdout: "get_or_fail_success": 0, 2024-03-04T09:41:43.649 INFO:teuthology.orchestra.run.smithi191.stdout: "take": 0, 2024-03-04T09:41:43.649 INFO:teuthology.orchestra.run.smithi191.stdout: "take_sum": 0, 2024-03-04T09:41:43.649 INFO:teuthology.orchestra.run.smithi191.stdout: "put": 0, 2024-03-04T09:41:43.649 INFO:teuthology.orchestra.run.smithi191.stdout: "put_sum": 0, 2024-03-04T09:41:43.649 INFO:teuthology.orchestra.run.smithi191.stdout: "wait": { 2024-03-04T09:41:43.649 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-03-04T09:41:43.649 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-03-04T09:41:43.650 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-03-04T09:41:43.650 INFO:teuthology.orchestra.run.smithi191.stdout: } 2024-03-04T09:41:43.650 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-03-04T09:41:43.650 INFO:teuthology.orchestra.run.smithi191.stdout: "throttle-write_buf_throttle-0x561b42e6a960": { 2024-03-04T09:41:43.650 INFO:teuthology.orchestra.run.smithi191.stdout: "val": 0, 2024-03-04T09:41:43.650 INFO:teuthology.orchestra.run.smithi191.stdout: "max": 3758096384, 2024-03-04T09:41:43.650 INFO:teuthology.orchestra.run.smithi191.stdout: "get_started": 0, 2024-03-04T09:41:43.650 INFO:teuthology.orchestra.run.smithi191.stdout: "get": 0, 2024-03-04T09:41:43.650 INFO:teuthology.orchestra.run.smithi191.stdout: "get_sum": 0, 2024-03-04T09:41:43.651 INFO:teuthology.orchestra.run.smithi191.stdout: "get_or_fail_fail": 0, 2024-03-04T09:41:43.651 INFO:teuthology.orchestra.run.smithi191.stdout: "get_or_fail_success": 0, 2024-03-04T09:41:43.651 INFO:teuthology.orchestra.run.smithi191.stdout: "take": 0, 2024-03-04T09:41:43.651 INFO:teuthology.orchestra.run.smithi191.stdout: "take_sum": 0, 2024-03-04T09:41:43.651 INFO:teuthology.orchestra.run.smithi191.stdout: "put": 0, 2024-03-04T09:41:43.651 INFO:teuthology.orchestra.run.smithi191.stdout: "put_sum": 0, 2024-03-04T09:41:43.651 INFO:teuthology.orchestra.run.smithi191.stdout: "wait": { 2024-03-04T09:41:43.651 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-03-04T09:41:43.652 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-03-04T09:41:43.652 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-03-04T09:41:43.652 INFO:teuthology.orchestra.run.smithi191.stdout: } 2024-03-04T09:41:43.652 INFO:teuthology.orchestra.run.smithi191.stdout: } 2024-03-04T09:41:43.652 INFO:teuthology.orchestra.run.smithi191.stdout:} 2024-03-04T09:41:44.337 INFO:tasks.check_counter:Daemon mds.f mds.exported=0 2024-03-04T09:41:44.338 INFO:tasks.check_counter:Daemon mds.f mds.imported=0 2024-03-04T09:41:44.338 INFO:tasks.check_counter:Daemon mds.f mds.root_rsnaps=0 2024-03-04T09:41:44.338 INFO:tasks.check_counter:Daemon mds.f mds_server.req_mksnap_latency.avgcount=0 2024-03-04T09:41:44.338 INFO:tasks.check_counter:Daemon mds.f mds_server.req_rmsnap_latency.avgcount=0 2024-03-04T09:41:44.338 INFO:tasks.check_counter:Daemon mds.f mds.dir_split=0 2024-03-04T09:41:44.338 DEBUG:tasks.check_counter:Getting stats from i 2024-03-04T09:41:44.339 DEBUG:teuthology.orchestra.run.smithi191:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd shell --fsid 4e059f08-da08-11ee-95c6-87774f69a715 -- ceph daemon mds.i perf dump 2024-03-04T09:41:44.585 INFO:teuthology.orchestra.run.smithi191.stderr:Inferring config /var/lib/ceph/4e059f08-da08-11ee-95c6-87774f69a715/mon.c/config 2024-03-04T09:41:44.585 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:41:44 smithi052 ceph-mon[32473]: pgmap v328: 129 pgs: 129 active+clean; 78 MiB data, 691 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-04T09:41:44.587 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:41:44 smithi191 ceph-mon[36816]: pgmap v328: 129 pgs: 129 active+clean; 78 MiB data, 691 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-04T09:41:44.632 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:41:44 smithi106 ceph-mon[36484]: pgmap v328: 129 pgs: 129 active+clean; 78 MiB data, 691 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-04T09:41:46.369 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:41:46 smithi191 ceph-mon[36816]: pgmap v329: 129 pgs: 129 active+clean; 78 MiB data, 691 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-04T09:41:46.424 INFO:teuthology.orchestra.run.smithi191.stdout:{ 2024-03-04T09:41:46.425 INFO:teuthology.orchestra.run.smithi191.stdout: "AsyncMessenger::Worker-0": { 2024-03-04T09:41:46.425 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_recv_messages": 191, 2024-03-04T09:41:46.425 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_send_messages": 341, 2024-03-04T09:41:46.426 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_recv_bytes": 57783, 2024-03-04T09:41:46.426 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_send_bytes": 544660, 2024-03-04T09:41:46.426 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_created_connections": 16, 2024-03-04T09:41:46.426 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_active_connections": 5, 2024-03-04T09:41:46.426 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_running_total_time": 0.021566249, 2024-03-04T09:41:46.426 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_running_send_time": 0.006362251, 2024-03-04T09:41:46.426 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_running_recv_time": 0.013464198, 2024-03-04T09:41:46.426 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_running_fast_dispatch_time": 0.001229881, 2024-03-04T09:41:46.427 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_send_messages_queue_lat": { 2024-03-04T09:41:46.427 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 335, 2024-03-04T09:41:46.427 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.051669321, 2024-03-04T09:41:46.427 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000154236 2024-03-04T09:41:46.427 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-03-04T09:41:46.427 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_handle_ack_lat": { 2024-03-04T09:41:46.427 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 229, 2024-03-04T09:41:46.427 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000059843, 2024-03-04T09:41:46.427 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000261 2024-03-04T09:41:46.428 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-03-04T09:41:46.428 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_recv_encrypted_bytes": 3120, 2024-03-04T09:41:46.428 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_send_encrypted_bytes": 4128 2024-03-04T09:41:46.428 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-03-04T09:41:46.428 INFO:teuthology.orchestra.run.smithi191.stdout: "AsyncMessenger::Worker-1": { 2024-03-04T09:41:46.428 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_recv_messages": 531, 2024-03-04T09:41:46.428 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_send_messages": 785, 2024-03-04T09:41:46.428 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_recv_bytes": 285718, 2024-03-04T09:41:46.429 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_send_bytes": 758655, 2024-03-04T09:41:46.429 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_created_connections": 9, 2024-03-04T09:41:46.429 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_active_connections": 4, 2024-03-04T09:41:46.429 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_running_total_time": 0.046509116, 2024-03-04T09:41:46.429 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_running_send_time": 0.019588937, 2024-03-04T09:41:46.429 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_running_recv_time": 0.022988871, 2024-03-04T09:41:46.429 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_running_fast_dispatch_time": 0.002885413, 2024-03-04T09:41:46.429 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_send_messages_queue_lat": { 2024-03-04T09:41:46.430 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 787, 2024-03-04T09:41:46.430 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.039694359, 2024-03-04T09:41:46.430 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000050437 2024-03-04T09:41:46.430 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-03-04T09:41:46.430 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_handle_ack_lat": { 2024-03-04T09:41:46.430 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 1105, 2024-03-04T09:41:46.430 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000180378, 2024-03-04T09:41:46.430 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000163 2024-03-04T09:41:46.430 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-03-04T09:41:46.431 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_recv_encrypted_bytes": 3344, 2024-03-04T09:41:46.431 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_send_encrypted_bytes": 445920 2024-03-04T09:41:46.431 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-03-04T09:41:46.431 INFO:teuthology.orchestra.run.smithi191.stdout: "AsyncMessenger::Worker-2": { 2024-03-04T09:41:46.431 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_recv_messages": 306, 2024-03-04T09:41:46.431 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_send_messages": 302, 2024-03-04T09:41:46.431 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_recv_bytes": 1493513, 2024-03-04T09:41:46.431 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_send_bytes": 164891, 2024-03-04T09:41:46.431 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_created_connections": 6, 2024-03-04T09:41:46.432 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_active_connections": 5, 2024-03-04T09:41:46.432 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_running_total_time": 0.036787316, 2024-03-04T09:41:46.432 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_running_send_time": 0.010090734, 2024-03-04T09:41:46.432 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_running_recv_time": 0.032246799, 2024-03-04T09:41:46.432 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_running_fast_dispatch_time": 0.004207526, 2024-03-04T09:41:46.432 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_send_messages_queue_lat": { 2024-03-04T09:41:46.432 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 301, 2024-03-04T09:41:46.433 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.015942214, 2024-03-04T09:41:46.433 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000052964 2024-03-04T09:41:46.433 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-03-04T09:41:46.433 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_handle_ack_lat": { 2024-03-04T09:41:46.433 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 30, 2024-03-04T09:41:46.433 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000005947, 2024-03-04T09:41:46.433 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000198 2024-03-04T09:41:46.434 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-03-04T09:41:46.434 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_recv_encrypted_bytes": 1489200, 2024-03-04T09:41:46.434 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_send_encrypted_bytes": 143008 2024-03-04T09:41:46.434 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-03-04T09:41:46.434 INFO:teuthology.orchestra.run.smithi191.stdout: "cct": { 2024-03-04T09:41:46.434 INFO:teuthology.orchestra.run.smithi191.stdout: "total_workers": 1, 2024-03-04T09:41:46.434 INFO:teuthology.orchestra.run.smithi191.stdout: "unhealthy_workers": 0 2024-03-04T09:41:46.435 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-03-04T09:41:46.435 INFO:teuthology.orchestra.run.smithi191.stdout: "finisher-MDSRank": { 2024-03-04T09:41:46.435 INFO:teuthology.orchestra.run.smithi191.stdout: "queue_len": 0, 2024-03-04T09:41:46.435 INFO:teuthology.orchestra.run.smithi191.stdout: "complete_latency": { 2024-03-04T09:41:46.435 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 91, 2024-03-04T09:41:46.435 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.048512992, 2024-03-04T09:41:46.435 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000533109 2024-03-04T09:41:46.436 INFO:teuthology.orchestra.run.smithi191.stdout: } 2024-03-04T09:41:46.436 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-03-04T09:41:46.436 INFO:teuthology.orchestra.run.smithi191.stdout: "finisher-PurgeQueue": { 2024-03-04T09:41:46.436 INFO:teuthology.orchestra.run.smithi191.stdout: "queue_len": 0, 2024-03-04T09:41:46.436 INFO:teuthology.orchestra.run.smithi191.stdout: "complete_latency": { 2024-03-04T09:41:46.436 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 2, 2024-03-04T09:41:46.436 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000006025, 2024-03-04T09:41:46.437 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000003012 2024-03-04T09:41:46.437 INFO:teuthology.orchestra.run.smithi191.stdout: } 2024-03-04T09:41:46.437 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-03-04T09:41:46.437 INFO:teuthology.orchestra.run.smithi191.stdout: "mds": { 2024-03-04T09:41:46.437 INFO:teuthology.orchestra.run.smithi191.stdout: "request": 6, 2024-03-04T09:41:46.437 INFO:teuthology.orchestra.run.smithi191.stdout: "reply": 3, 2024-03-04T09:41:46.438 INFO:teuthology.orchestra.run.smithi191.stdout: "reply_latency": { 2024-03-04T09:41:46.438 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 3, 2024-03-04T09:41:46.438 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.001930126, 2024-03-04T09:41:46.438 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000643375 2024-03-04T09:41:46.438 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-03-04T09:41:46.438 INFO:teuthology.orchestra.run.smithi191.stdout: "slow_reply": 0, 2024-03-04T09:41:46.439 INFO:teuthology.orchestra.run.smithi191.stdout: "forward": 3, 2024-03-04T09:41:46.439 INFO:teuthology.orchestra.run.smithi191.stdout: "dir_fetch_complete": 0, 2024-03-04T09:41:46.439 INFO:teuthology.orchestra.run.smithi191.stdout: "dir_fetch_keys": 0, 2024-03-04T09:41:46.439 INFO:teuthology.orchestra.run.smithi191.stdout: "dir_commit": 11, 2024-03-04T09:41:46.439 INFO:teuthology.orchestra.run.smithi191.stdout: "dir_split": 0, 2024-03-04T09:41:46.439 INFO:teuthology.orchestra.run.smithi191.stdout: "dir_merge": 0, 2024-03-04T09:41:46.439 INFO:teuthology.orchestra.run.smithi191.stdout: "inodes": 140, 2024-03-04T09:41:46.440 INFO:teuthology.orchestra.run.smithi191.stdout: "inodes_top": 6, 2024-03-04T09:41:46.440 INFO:teuthology.orchestra.run.smithi191.stdout: "inodes_bottom": 134, 2024-03-04T09:41:46.440 INFO:teuthology.orchestra.run.smithi191.stdout: "inodes_pin_tail": 0, 2024-03-04T09:41:46.440 INFO:teuthology.orchestra.run.smithi191.stdout: "inodes_pinned": 130, 2024-03-04T09:41:46.440 INFO:teuthology.orchestra.run.smithi191.stdout: "inodes_expired": 0, 2024-03-04T09:41:46.440 INFO:teuthology.orchestra.run.smithi191.stdout: "inodes_with_caps": 72, 2024-03-04T09:41:46.440 INFO:teuthology.orchestra.run.smithi191.stdout: "caps": 72, 2024-03-04T09:41:46.441 INFO:teuthology.orchestra.run.smithi191.stdout: "subtrees": 5, 2024-03-04T09:41:46.441 INFO:teuthology.orchestra.run.smithi191.stdout: "traverse": 14, 2024-03-04T09:41:46.441 INFO:teuthology.orchestra.run.smithi191.stdout: "traverse_hit": 6, 2024-03-04T09:41:46.441 INFO:teuthology.orchestra.run.smithi191.stdout: "traverse_forward": 1, 2024-03-04T09:41:46.441 INFO:teuthology.orchestra.run.smithi191.stdout: "traverse_discover": 7, 2024-03-04T09:41:46.441 INFO:teuthology.orchestra.run.smithi191.stdout: "traverse_dir_fetch": 0, 2024-03-04T09:41:46.442 INFO:teuthology.orchestra.run.smithi191.stdout: "traverse_remote_ino": 0, 2024-03-04T09:41:46.442 INFO:teuthology.orchestra.run.smithi191.stdout: "traverse_lock": 0, 2024-03-04T09:41:46.442 INFO:teuthology.orchestra.run.smithi191.stdout: "load_cent": 0, 2024-03-04T09:41:46.442 INFO:teuthology.orchestra.run.smithi191.stdout: "q": 0, 2024-03-04T09:41:46.442 INFO:teuthology.orchestra.run.smithi191.stdout: "exported": 0, 2024-03-04T09:41:46.442 INFO:teuthology.orchestra.run.smithi191.stdout: "exported_inodes": 0, 2024-03-04T09:41:46.442 INFO:teuthology.orchestra.run.smithi191.stdout: "imported": 6, 2024-03-04T09:41:46.443 INFO:teuthology.orchestra.run.smithi191.stdout: "imported_inodes": 121, 2024-03-04T09:41:46.443 INFO:teuthology.orchestra.run.smithi191.stdout: "openino_dir_fetch": 0, 2024-03-04T09:41:46.443 INFO:teuthology.orchestra.run.smithi191.stdout: "openino_backtrace_fetch": 0, 2024-03-04T09:41:46.443 INFO:teuthology.orchestra.run.smithi191.stdout: "openino_peer_discover": 0, 2024-03-04T09:41:46.443 INFO:teuthology.orchestra.run.smithi191.stdout: "root_rfiles": 4, 2024-03-04T09:41:46.443 INFO:teuthology.orchestra.run.smithi191.stdout: "root_rbytes": 234, 2024-03-04T09:41:46.443 INFO:teuthology.orchestra.run.smithi191.stdout: "root_rsnaps": 0, 2024-03-04T09:41:46.444 INFO:teuthology.orchestra.run.smithi191.stdout: "scrub_backtrace_fetch": 0, 2024-03-04T09:41:46.444 INFO:teuthology.orchestra.run.smithi191.stdout: "scrub_set_tag": 0, 2024-03-04T09:41:46.444 INFO:teuthology.orchestra.run.smithi191.stdout: "scrub_backtrace_repaired": 0, 2024-03-04T09:41:46.444 INFO:teuthology.orchestra.run.smithi191.stdout: "scrub_inotable_repaired": 0, 2024-03-04T09:41:46.444 INFO:teuthology.orchestra.run.smithi191.stdout: "scrub_dir_inodes": 0, 2024-03-04T09:41:46.444 INFO:teuthology.orchestra.run.smithi191.stdout: "scrub_dir_base_inodes": 0, 2024-03-04T09:41:46.445 INFO:teuthology.orchestra.run.smithi191.stdout: "scrub_dirfrag_rstats": 0, 2024-03-04T09:41:46.445 INFO:teuthology.orchestra.run.smithi191.stdout: "scrub_file_inodes": 0, 2024-03-04T09:41:46.445 INFO:teuthology.orchestra.run.smithi191.stdout: "handle_inode_file_caps": 0, 2024-03-04T09:41:46.445 INFO:teuthology.orchestra.run.smithi191.stdout: "ceph_cap_op_revoke": 0, 2024-03-04T09:41:46.445 INFO:teuthology.orchestra.run.smithi191.stdout: "ceph_cap_op_grant": 4, 2024-03-04T09:41:46.445 INFO:teuthology.orchestra.run.smithi191.stdout: "ceph_cap_op_trunc": 0, 2024-03-04T09:41:46.445 INFO:teuthology.orchestra.run.smithi191.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-03-04T09:41:46.446 INFO:teuthology.orchestra.run.smithi191.stdout: "ceph_cap_op_flush_ack": 0, 2024-03-04T09:41:46.446 INFO:teuthology.orchestra.run.smithi191.stdout: "handle_client_caps": 106, 2024-03-04T09:41:46.446 INFO:teuthology.orchestra.run.smithi191.stdout: "handle_client_caps_dirty": 102, 2024-03-04T09:41:46.446 INFO:teuthology.orchestra.run.smithi191.stdout: "handle_client_cap_release": 1, 2024-03-04T09:41:46.446 INFO:teuthology.orchestra.run.smithi191.stdout: "process_request_cap_release": 0 2024-03-04T09:41:46.446 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-03-04T09:41:46.447 INFO:teuthology.orchestra.run.smithi191.stdout: "mds_cache": { 2024-03-04T09:41:46.447 INFO:teuthology.orchestra.run.smithi191.stdout: "dir_update": 0, 2024-03-04T09:41:46.447 INFO:teuthology.orchestra.run.smithi191.stdout: "dir_update_receipt": 0, 2024-03-04T09:41:46.447 INFO:teuthology.orchestra.run.smithi191.stdout: "dir_try_discover": 0, 2024-03-04T09:41:46.447 INFO:teuthology.orchestra.run.smithi191.stdout: "dir_send_discover": 8, 2024-03-04T09:41:46.447 INFO:teuthology.orchestra.run.smithi191.stdout: "dir_handle_discover": 1, 2024-03-04T09:41:46.447 INFO:teuthology.orchestra.run.smithi191.stdout: "num_strays": 0, 2024-03-04T09:41:46.448 INFO:teuthology.orchestra.run.smithi191.stdout: "num_strays_delayed": 0, 2024-03-04T09:41:46.448 INFO:teuthology.orchestra.run.smithi191.stdout: "num_strays_enqueuing": 0, 2024-03-04T09:41:46.448 INFO:teuthology.orchestra.run.smithi191.stdout: "strays_created": 0, 2024-03-04T09:41:46.448 INFO:teuthology.orchestra.run.smithi191.stdout: "strays_enqueued": 0, 2024-03-04T09:41:46.448 INFO:teuthology.orchestra.run.smithi191.stdout: "strays_reintegrated": 0, 2024-03-04T09:41:46.448 INFO:teuthology.orchestra.run.smithi191.stdout: "strays_migrated": 0, 2024-03-04T09:41:46.449 INFO:teuthology.orchestra.run.smithi191.stdout: "num_recovering_processing": 0, 2024-03-04T09:41:46.449 INFO:teuthology.orchestra.run.smithi191.stdout: "num_recovering_enqueued": 0, 2024-03-04T09:41:46.449 INFO:teuthology.orchestra.run.smithi191.stdout: "num_recovering_prioritized": 0, 2024-03-04T09:41:46.449 INFO:teuthology.orchestra.run.smithi191.stdout: "recovery_started": 0, 2024-03-04T09:41:46.449 INFO:teuthology.orchestra.run.smithi191.stdout: "recovery_completed": 0, 2024-03-04T09:41:46.449 INFO:teuthology.orchestra.run.smithi191.stdout: "ireq_enqueue_scrub": 0, 2024-03-04T09:41:46.449 INFO:teuthology.orchestra.run.smithi191.stdout: "ireq_exportdir": 0, 2024-03-04T09:41:46.450 INFO:teuthology.orchestra.run.smithi191.stdout: "ireq_flush": 0, 2024-03-04T09:41:46.450 INFO:teuthology.orchestra.run.smithi191.stdout: "ireq_fragmentdir": 0, 2024-03-04T09:41:46.450 INFO:teuthology.orchestra.run.smithi191.stdout: "ireq_fragstats": 0, 2024-03-04T09:41:46.450 INFO:teuthology.orchestra.run.smithi191.stdout: "ireq_inodestats": 0 2024-03-04T09:41:46.450 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-03-04T09:41:46.450 INFO:teuthology.orchestra.run.smithi191.stdout: "mds_log": { 2024-03-04T09:41:46.450 INFO:teuthology.orchestra.run.smithi191.stdout: "evlrg": 0, 2024-03-04T09:41:46.451 INFO:teuthology.orchestra.run.smithi191.stdout: "evadd": 134, 2024-03-04T09:41:46.451 INFO:teuthology.orchestra.run.smithi191.stdout: "evex": 0, 2024-03-04T09:41:46.451 INFO:teuthology.orchestra.run.smithi191.stdout: "evtrm": 0, 2024-03-04T09:41:46.451 INFO:teuthology.orchestra.run.smithi191.stdout: "ev": 134, 2024-03-04T09:41:46.451 INFO:teuthology.orchestra.run.smithi191.stdout: "evexg": 0, 2024-03-04T09:41:46.451 INFO:teuthology.orchestra.run.smithi191.stdout: "evexd": 0, 2024-03-04T09:41:46.451 INFO:teuthology.orchestra.run.smithi191.stdout: "segadd": 2, 2024-03-04T09:41:46.451 INFO:teuthology.orchestra.run.smithi191.stdout: "segex": 0, 2024-03-04T09:41:46.452 INFO:teuthology.orchestra.run.smithi191.stdout: "segtrm": 0, 2024-03-04T09:41:46.452 INFO:teuthology.orchestra.run.smithi191.stdout: "seg": 2, 2024-03-04T09:41:46.452 INFO:teuthology.orchestra.run.smithi191.stdout: "segmjr": 2, 2024-03-04T09:41:46.452 INFO:teuthology.orchestra.run.smithi191.stdout: "segexg": 0, 2024-03-04T09:41:46.452 INFO:teuthology.orchestra.run.smithi191.stdout: "segexd": 0, 2024-03-04T09:41:46.452 INFO:teuthology.orchestra.run.smithi191.stdout: "expos": 4194304, 2024-03-04T09:41:46.452 INFO:teuthology.orchestra.run.smithi191.stdout: "wrpos": 4562483, 2024-03-04T09:41:46.452 INFO:teuthology.orchestra.run.smithi191.stdout: "rdpos": 0, 2024-03-04T09:41:46.453 INFO:teuthology.orchestra.run.smithi191.stdout: "jlat": { 2024-03-04T09:41:46.453 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 26, 2024-03-04T09:41:46.453 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.542152179, 2024-03-04T09:41:46.453 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.020852006 2024-03-04T09:41:46.453 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-03-04T09:41:46.453 INFO:teuthology.orchestra.run.smithi191.stdout: "replayed": 0 2024-03-04T09:41:46.453 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-03-04T09:41:46.453 INFO:teuthology.orchestra.run.smithi191.stdout: "mds_mem": { 2024-03-04T09:41:46.453 INFO:teuthology.orchestra.run.smithi191.stdout: "ino": 133, 2024-03-04T09:41:46.454 INFO:teuthology.orchestra.run.smithi191.stdout: "ino+": 133, 2024-03-04T09:41:46.454 INFO:teuthology.orchestra.run.smithi191.stdout: "ino-": 0, 2024-03-04T09:41:46.454 INFO:teuthology.orchestra.run.smithi191.stdout: "dir": 30, 2024-03-04T09:41:46.454 INFO:teuthology.orchestra.run.smithi191.stdout: "dir+": 31, 2024-03-04T09:41:46.454 INFO:teuthology.orchestra.run.smithi191.stdout: "dir-": 1, 2024-03-04T09:41:46.454 INFO:teuthology.orchestra.run.smithi191.stdout: "dn": 140, 2024-03-04T09:41:46.454 INFO:teuthology.orchestra.run.smithi191.stdout: "dn+": 140, 2024-03-04T09:41:46.455 INFO:teuthology.orchestra.run.smithi191.stdout: "dn-": 0, 2024-03-04T09:41:46.455 INFO:teuthology.orchestra.run.smithi191.stdout: "cap": 72, 2024-03-04T09:41:46.455 INFO:teuthology.orchestra.run.smithi191.stdout: "cap+": 114, 2024-03-04T09:41:46.455 INFO:teuthology.orchestra.run.smithi191.stdout: "cap-": 42, 2024-03-04T09:41:46.455 INFO:teuthology.orchestra.run.smithi191.stdout: "rss": 53940, 2024-03-04T09:41:46.455 INFO:teuthology.orchestra.run.smithi191.stdout: "heap": 223612 2024-03-04T09:41:46.455 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-03-04T09:41:46.455 INFO:teuthology.orchestra.run.smithi191.stdout: "mds_server": { 2024-03-04T09:41:46.455 INFO:teuthology.orchestra.run.smithi191.stdout: "dispatch_client_request": 11, 2024-03-04T09:41:46.456 INFO:teuthology.orchestra.run.smithi191.stdout: "dispatch_server_request": 0, 2024-03-04T09:41:46.456 INFO:teuthology.orchestra.run.smithi191.stdout: "handle_client_request": 6, 2024-03-04T09:41:46.456 INFO:teuthology.orchestra.run.smithi191.stdout: "handle_client_session": 22, 2024-03-04T09:41:46.456 INFO:teuthology.orchestra.run.smithi191.stdout: "handle_peer_request": 1, 2024-03-04T09:41:46.456 INFO:teuthology.orchestra.run.smithi191.stdout: "req_create_latency": { 2024-03-04T09:41:46.456 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-03-04T09:41:46.456 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-03-04T09:41:46.456 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-03-04T09:41:46.457 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-03-04T09:41:46.457 INFO:teuthology.orchestra.run.smithi191.stdout: "req_getattr_latency": { 2024-03-04T09:41:46.457 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 1, 2024-03-04T09:41:46.457 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.001147278, 2024-03-04T09:41:46.457 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.001147278 2024-03-04T09:41:46.457 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-03-04T09:41:46.457 INFO:teuthology.orchestra.run.smithi191.stdout: "req_getfilelock_latency": { 2024-03-04T09:41:46.457 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-03-04T09:41:46.457 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-03-04T09:41:46.458 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-03-04T09:41:46.458 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-03-04T09:41:46.458 INFO:teuthology.orchestra.run.smithi191.stdout: "req_link_latency": { 2024-03-04T09:41:46.458 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-03-04T09:41:46.458 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-03-04T09:41:46.458 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-03-04T09:41:46.458 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-03-04T09:41:46.458 INFO:teuthology.orchestra.run.smithi191.stdout: "req_lookup_latency": { 2024-03-04T09:41:46.459 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 2, 2024-03-04T09:41:46.459 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000782848, 2024-03-04T09:41:46.459 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000391424 2024-03-04T09:41:46.459 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-03-04T09:41:46.459 INFO:teuthology.orchestra.run.smithi191.stdout: "req_lookuphash_latency": { 2024-03-04T09:41:46.459 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-03-04T09:41:46.459 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-03-04T09:41:46.459 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-03-04T09:41:46.459 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-03-04T09:41:46.460 INFO:teuthology.orchestra.run.smithi191.stdout: "req_lookupino_latency": { 2024-03-04T09:41:46.460 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-03-04T09:41:46.460 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-03-04T09:41:46.460 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-03-04T09:41:46.460 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-03-04T09:41:46.460 INFO:teuthology.orchestra.run.smithi191.stdout: "req_lookupname_latency": { 2024-03-04T09:41:46.460 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-03-04T09:41:46.461 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-03-04T09:41:46.461 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-03-04T09:41:46.461 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-03-04T09:41:46.461 INFO:teuthology.orchestra.run.smithi191.stdout: "req_lookupparent_latency": { 2024-03-04T09:41:46.461 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-03-04T09:41:46.461 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-03-04T09:41:46.461 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-03-04T09:41:46.462 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-03-04T09:41:46.462 INFO:teuthology.orchestra.run.smithi191.stdout: "req_lookupsnap_latency": { 2024-03-04T09:41:46.462 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-03-04T09:41:46.462 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-03-04T09:41:46.462 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-03-04T09:41:46.462 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-03-04T09:41:46.462 INFO:teuthology.orchestra.run.smithi191.stdout: "req_lssnap_latency": { 2024-03-04T09:41:46.463 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-03-04T09:41:46.463 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-03-04T09:41:46.463 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-03-04T09:41:46.463 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-03-04T09:41:46.463 INFO:teuthology.orchestra.run.smithi191.stdout: "req_mkdir_latency": { 2024-03-04T09:41:46.463 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-03-04T09:41:46.463 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-03-04T09:41:46.464 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-03-04T09:41:46.465 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-03-04T09:41:46.465 INFO:teuthology.orchestra.run.smithi191.stdout: "req_mknod_latency": { 2024-03-04T09:41:46.465 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-03-04T09:41:46.465 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-03-04T09:41:46.465 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-03-04T09:41:46.465 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-03-04T09:41:46.465 INFO:teuthology.orchestra.run.smithi191.stdout: "req_mksnap_latency": { 2024-03-04T09:41:46.466 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-03-04T09:41:46.466 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-03-04T09:41:46.466 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-03-04T09:41:46.466 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-03-04T09:41:46.466 INFO:teuthology.orchestra.run.smithi191.stdout: "req_open_latency": { 2024-03-04T09:41:46.466 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-03-04T09:41:46.466 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-03-04T09:41:46.466 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-03-04T09:41:46.466 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-03-04T09:41:46.467 INFO:teuthology.orchestra.run.smithi191.stdout: "req_readdir_latency": { 2024-03-04T09:41:46.467 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-03-04T09:41:46.467 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-03-04T09:41:46.467 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-03-04T09:41:46.467 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-03-04T09:41:46.467 INFO:teuthology.orchestra.run.smithi191.stdout: "req_rename_latency": { 2024-03-04T09:41:46.467 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-03-04T09:41:46.467 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-03-04T09:41:46.468 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-03-04T09:41:46.468 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-03-04T09:41:46.468 INFO:teuthology.orchestra.run.smithi191.stdout: "req_renamesnap_latency": { 2024-03-04T09:41:46.468 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-03-04T09:41:46.468 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-03-04T09:41:46.468 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-03-04T09:41:46.468 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-03-04T09:41:46.469 INFO:teuthology.orchestra.run.smithi191.stdout: "req_snapdiff_latency": { 2024-03-04T09:41:46.469 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-03-04T09:41:46.469 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-03-04T09:41:46.469 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-03-04T09:41:46.469 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-03-04T09:41:46.469 INFO:teuthology.orchestra.run.smithi191.stdout: "req_rmdir_latency": { 2024-03-04T09:41:46.469 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-03-04T09:41:46.469 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-03-04T09:41:46.470 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-03-04T09:41:46.470 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-03-04T09:41:46.470 INFO:teuthology.orchestra.run.smithi191.stdout: "req_rmsnap_latency": { 2024-03-04T09:41:46.470 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-03-04T09:41:46.470 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-03-04T09:41:46.470 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-03-04T09:41:46.470 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-03-04T09:41:46.470 INFO:teuthology.orchestra.run.smithi191.stdout: "req_rmxattr_latency": { 2024-03-04T09:41:46.471 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-03-04T09:41:46.471 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-03-04T09:41:46.471 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-03-04T09:41:46.471 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-03-04T09:41:46.471 INFO:teuthology.orchestra.run.smithi191.stdout: "req_setattr_latency": { 2024-03-04T09:41:46.471 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-03-04T09:41:46.472 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-03-04T09:41:46.472 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-03-04T09:41:46.472 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-03-04T09:41:46.472 INFO:teuthology.orchestra.run.smithi191.stdout: "req_setdirlayout_latency": { 2024-03-04T09:41:46.472 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-03-04T09:41:46.472 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-03-04T09:41:46.472 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-03-04T09:41:46.473 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-03-04T09:41:46.473 INFO:teuthology.orchestra.run.smithi191.stdout: "req_setfilelock_latency": { 2024-03-04T09:41:46.473 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-03-04T09:41:46.473 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-03-04T09:41:46.473 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-03-04T09:41:46.473 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-03-04T09:41:46.473 INFO:teuthology.orchestra.run.smithi191.stdout: "req_setlayout_latency": { 2024-03-04T09:41:46.474 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-03-04T09:41:46.474 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-03-04T09:41:46.474 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-03-04T09:41:46.474 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-03-04T09:41:46.474 INFO:teuthology.orchestra.run.smithi191.stdout: "req_setxattr_latency": { 2024-03-04T09:41:46.474 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-03-04T09:41:46.475 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-03-04T09:41:46.475 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-03-04T09:41:46.475 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-03-04T09:41:46.475 INFO:teuthology.orchestra.run.smithi191.stdout: "req_symlink_latency": { 2024-03-04T09:41:46.475 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-03-04T09:41:46.475 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-03-04T09:41:46.476 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-03-04T09:41:46.476 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-03-04T09:41:46.476 INFO:teuthology.orchestra.run.smithi191.stdout: "req_unlink_latency": { 2024-03-04T09:41:46.476 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-03-04T09:41:46.476 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-03-04T09:41:46.477 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-03-04T09:41:46.477 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-03-04T09:41:46.477 INFO:teuthology.orchestra.run.smithi191.stdout: "cap_revoke_eviction": 0, 2024-03-04T09:41:46.477 INFO:teuthology.orchestra.run.smithi191.stdout: "cap_acquisition_throttle": 0, 2024-03-04T09:41:46.477 INFO:teuthology.orchestra.run.smithi191.stdout: "req_getvxattr_latency": { 2024-03-04T09:41:46.477 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-03-04T09:41:46.477 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-03-04T09:41:46.478 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-03-04T09:41:46.478 INFO:teuthology.orchestra.run.smithi191.stdout: } 2024-03-04T09:41:46.478 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-03-04T09:41:46.478 INFO:teuthology.orchestra.run.smithi191.stdout: "mds_sessions": { 2024-03-04T09:41:46.478 INFO:teuthology.orchestra.run.smithi191.stdout: "session_count": 2, 2024-03-04T09:41:46.478 INFO:teuthology.orchestra.run.smithi191.stdout: "session_add": 4, 2024-03-04T09:41:46.479 INFO:teuthology.orchestra.run.smithi191.stdout: "session_remove": 2, 2024-03-04T09:41:46.479 INFO:teuthology.orchestra.run.smithi191.stdout: "sessions_open": 2, 2024-03-04T09:41:46.479 INFO:teuthology.orchestra.run.smithi191.stdout: "sessions_stale": 0, 2024-03-04T09:41:46.479 INFO:teuthology.orchestra.run.smithi191.stdout: "total_load": 2, 2024-03-04T09:41:46.479 INFO:teuthology.orchestra.run.smithi191.stdout: "average_load": 2, 2024-03-04T09:41:46.479 INFO:teuthology.orchestra.run.smithi191.stdout: "avg_session_uptime": 584, 2024-03-04T09:41:46.479 INFO:teuthology.orchestra.run.smithi191.stdout: "mdthresh_evicted": 0 2024-03-04T09:41:46.480 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-03-04T09:41:46.480 INFO:teuthology.orchestra.run.smithi191.stdout: "mempool": { 2024-03-04T09:41:46.480 INFO:teuthology.orchestra.run.smithi191.stdout: "bloom_filter_bytes": 0, 2024-03-04T09:41:46.480 INFO:teuthology.orchestra.run.smithi191.stdout: "bloom_filter_items": 0, 2024-03-04T09:41:46.480 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_alloc_bytes": 0, 2024-03-04T09:41:46.480 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_alloc_items": 0, 2024-03-04T09:41:46.480 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_cache_data_bytes": 0, 2024-03-04T09:41:46.481 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_cache_data_items": 0, 2024-03-04T09:41:46.481 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-04T09:41:46.481 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_cache_onode_items": 0, 2024-03-04T09:41:46.481 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-04T09:41:46.481 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_cache_meta_items": 0, 2024-03-04T09:41:46.481 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_cache_other_bytes": 0, 2024-03-04T09:41:46.482 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_cache_other_items": 0, 2024-03-04T09:41:46.482 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-04T09:41:46.482 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_cache_buffer_items": 0, 2024-03-04T09:41:46.482 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_extent_bytes": 0, 2024-03-04T09:41:46.482 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_extent_items": 0, 2024-03-04T09:41:46.482 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_blob_bytes": 0, 2024-03-04T09:41:46.483 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_blob_items": 0, 2024-03-04T09:41:46.483 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-04T09:41:46.483 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_shared_blob_items": 0, 2024-03-04T09:41:46.483 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-04T09:41:46.483 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_inline_bl_items": 0, 2024-03-04T09:41:46.483 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_fsck_bytes": 0, 2024-03-04T09:41:46.483 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_fsck_items": 0, 2024-03-04T09:41:46.483 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_txc_bytes": 0, 2024-03-04T09:41:46.484 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_txc_items": 0, 2024-03-04T09:41:46.484 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-04T09:41:46.484 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_writing_deferred_items": 0, 2024-03-04T09:41:46.484 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_writing_bytes": 0, 2024-03-04T09:41:46.484 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_writing_items": 0, 2024-03-04T09:41:46.484 INFO:teuthology.orchestra.run.smithi191.stdout: "bluefs_bytes": 0, 2024-03-04T09:41:46.484 INFO:teuthology.orchestra.run.smithi191.stdout: "bluefs_items": 0, 2024-03-04T09:41:46.484 INFO:teuthology.orchestra.run.smithi191.stdout: "bluefs_file_reader_bytes": 0, 2024-03-04T09:41:46.485 INFO:teuthology.orchestra.run.smithi191.stdout: "bluefs_file_reader_items": 0, 2024-03-04T09:41:46.485 INFO:teuthology.orchestra.run.smithi191.stdout: "bluefs_file_writer_bytes": 0, 2024-03-04T09:41:46.485 INFO:teuthology.orchestra.run.smithi191.stdout: "bluefs_file_writer_items": 0, 2024-03-04T09:41:46.485 INFO:teuthology.orchestra.run.smithi191.stdout: "buffer_anon_bytes": 268199, 2024-03-04T09:41:46.485 INFO:teuthology.orchestra.run.smithi191.stdout: "buffer_anon_items": 214, 2024-03-04T09:41:46.485 INFO:teuthology.orchestra.run.smithi191.stdout: "buffer_meta_bytes": 176, 2024-03-04T09:41:46.485 INFO:teuthology.orchestra.run.smithi191.stdout: "buffer_meta_items": 2, 2024-03-04T09:41:46.485 INFO:teuthology.orchestra.run.smithi191.stdout: "osd_bytes": 0, 2024-03-04T09:41:46.485 INFO:teuthology.orchestra.run.smithi191.stdout: "osd_items": 0, 2024-03-04T09:41:46.486 INFO:teuthology.orchestra.run.smithi191.stdout: "osd_mapbl_bytes": 0, 2024-03-04T09:41:46.486 INFO:teuthology.orchestra.run.smithi191.stdout: "osd_mapbl_items": 0, 2024-03-04T09:41:46.486 INFO:teuthology.orchestra.run.smithi191.stdout: "osd_pglog_bytes": 0, 2024-03-04T09:41:46.486 INFO:teuthology.orchestra.run.smithi191.stdout: "osd_pglog_items": 0, 2024-03-04T09:41:46.486 INFO:teuthology.orchestra.run.smithi191.stdout: "osdmap_bytes": 9744, 2024-03-04T09:41:46.486 INFO:teuthology.orchestra.run.smithi191.stdout: "osdmap_items": 204, 2024-03-04T09:41:46.486 INFO:teuthology.orchestra.run.smithi191.stdout: "osdmap_mapping_bytes": 0, 2024-03-04T09:41:46.486 INFO:teuthology.orchestra.run.smithi191.stdout: "osdmap_mapping_items": 0, 2024-03-04T09:41:46.487 INFO:teuthology.orchestra.run.smithi191.stdout: "pgmap_bytes": 0, 2024-03-04T09:41:46.487 INFO:teuthology.orchestra.run.smithi191.stdout: "pgmap_items": 0, 2024-03-04T09:41:46.487 INFO:teuthology.orchestra.run.smithi191.stdout: "mds_co_bytes": 480820, 2024-03-04T09:41:46.487 INFO:teuthology.orchestra.run.smithi191.stdout: "mds_co_items": 5271, 2024-03-04T09:41:46.487 INFO:teuthology.orchestra.run.smithi191.stdout: "unittest_1_bytes": 0, 2024-03-04T09:41:46.487 INFO:teuthology.orchestra.run.smithi191.stdout: "unittest_1_items": 0, 2024-03-04T09:41:46.487 INFO:teuthology.orchestra.run.smithi191.stdout: "unittest_2_bytes": 0, 2024-03-04T09:41:46.488 INFO:teuthology.orchestra.run.smithi191.stdout: "unittest_2_items": 0 2024-03-04T09:41:46.488 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-03-04T09:41:46.488 INFO:teuthology.orchestra.run.smithi191.stdout: "objecter": { 2024-03-04T09:41:46.488 INFO:teuthology.orchestra.run.smithi191.stdout: "op_active": 0, 2024-03-04T09:41:46.488 INFO:teuthology.orchestra.run.smithi191.stdout: "op_laggy": 0, 2024-03-04T09:41:46.488 INFO:teuthology.orchestra.run.smithi191.stdout: "op_send": 73, 2024-03-04T09:41:46.488 INFO:teuthology.orchestra.run.smithi191.stdout: "op_send_bytes": 389107, 2024-03-04T09:41:46.488 INFO:teuthology.orchestra.run.smithi191.stdout: "op_resend": 33, 2024-03-04T09:41:46.489 INFO:teuthology.orchestra.run.smithi191.stdout: "op_reply": 73, 2024-03-04T09:41:46.489 INFO:teuthology.orchestra.run.smithi191.stdout: "op_latency": { 2024-03-04T09:41:46.489 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 73, 2024-03-04T09:41:46.489 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.777986462, 2024-03-04T09:41:46.489 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.010657348 2024-03-04T09:41:46.489 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-03-04T09:41:46.489 INFO:teuthology.orchestra.run.smithi191.stdout: "op_inflight": 0, 2024-03-04T09:41:46.490 INFO:teuthology.orchestra.run.smithi191.stdout: "oplen_avg": { 2024-03-04T09:41:46.490 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 73, 2024-03-04T09:41:46.490 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 103 2024-03-04T09:41:46.490 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-03-04T09:41:46.490 INFO:teuthology.orchestra.run.smithi191.stdout: "op": 73, 2024-03-04T09:41:46.490 INFO:teuthology.orchestra.run.smithi191.stdout: "op_r": 0, 2024-03-04T09:41:46.490 INFO:teuthology.orchestra.run.smithi191.stdout: "op_w": 73, 2024-03-04T09:41:46.490 INFO:teuthology.orchestra.run.smithi191.stdout: "op_rmw": 0, 2024-03-04T09:41:46.490 INFO:teuthology.orchestra.run.smithi191.stdout: "op_pg": 0, 2024-03-04T09:41:46.491 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_stat": 0, 2024-03-04T09:41:46.491 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_create": 10, 2024-03-04T09:41:46.491 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_read": 0, 2024-03-04T09:41:46.491 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_write": 26, 2024-03-04T09:41:46.491 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_writefull": 11, 2024-03-04T09:41:46.491 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_writesame": 0, 2024-03-04T09:41:46.491 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_append": 0, 2024-03-04T09:41:46.491 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_zero": 0, 2024-03-04T09:41:46.491 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_truncate": 0, 2024-03-04T09:41:46.492 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_delete": 6, 2024-03-04T09:41:46.492 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_mapext": 0, 2024-03-04T09:41:46.492 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_sparse_read": 0, 2024-03-04T09:41:46.492 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_clonerange": 0, 2024-03-04T09:41:46.492 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_getxattr": 0, 2024-03-04T09:41:46.492 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_setxattr": 20, 2024-03-04T09:41:46.492 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_cmpxattr": 0, 2024-03-04T09:41:46.492 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_rmxattr": 0, 2024-03-04T09:41:46.493 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_resetxattrs": 0, 2024-03-04T09:41:46.493 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_call": 0, 2024-03-04T09:41:46.493 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_watch": 0, 2024-03-04T09:41:46.493 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_notify": 0, 2024-03-04T09:41:46.493 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_src_cmpxattr": 0, 2024-03-04T09:41:46.493 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_pgls": 0, 2024-03-04T09:41:46.493 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_pgls_filter": 0, 2024-03-04T09:41:46.493 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_other": 0, 2024-03-04T09:41:46.494 INFO:teuthology.orchestra.run.smithi191.stdout: "linger_active": 0, 2024-03-04T09:41:46.494 INFO:teuthology.orchestra.run.smithi191.stdout: "linger_send": 0, 2024-03-04T09:41:46.494 INFO:teuthology.orchestra.run.smithi191.stdout: "linger_resend": 0, 2024-03-04T09:41:46.494 INFO:teuthology.orchestra.run.smithi191.stdout: "linger_ping": 0, 2024-03-04T09:41:46.494 INFO:teuthology.orchestra.run.smithi191.stdout: "poolop_active": 0, 2024-03-04T09:41:46.494 INFO:teuthology.orchestra.run.smithi191.stdout: "poolop_send": 0, 2024-03-04T09:41:46.494 INFO:teuthology.orchestra.run.smithi191.stdout: "poolop_resend": 0, 2024-03-04T09:41:46.495 INFO:teuthology.orchestra.run.smithi191.stdout: "poolstat_active": 0, 2024-03-04T09:41:46.495 INFO:teuthology.orchestra.run.smithi191.stdout: "poolstat_send": 0, 2024-03-04T09:41:46.495 INFO:teuthology.orchestra.run.smithi191.stdout: "poolstat_resend": 0, 2024-03-04T09:41:46.495 INFO:teuthology.orchestra.run.smithi191.stdout: "statfs_active": 0, 2024-03-04T09:41:46.495 INFO:teuthology.orchestra.run.smithi191.stdout: "statfs_send": 0, 2024-03-04T09:41:46.495 INFO:teuthology.orchestra.run.smithi191.stdout: "statfs_resend": 0, 2024-03-04T09:41:46.495 INFO:teuthology.orchestra.run.smithi191.stdout: "command_active": 0, 2024-03-04T09:41:46.496 INFO:teuthology.orchestra.run.smithi191.stdout: "command_send": 0, 2024-03-04T09:41:46.496 INFO:teuthology.orchestra.run.smithi191.stdout: "command_resend": 0, 2024-03-04T09:41:46.496 INFO:teuthology.orchestra.run.smithi191.stdout: "map_epoch": 102, 2024-03-04T09:41:46.496 INFO:teuthology.orchestra.run.smithi191.stdout: "map_full": 0, 2024-03-04T09:41:46.496 INFO:teuthology.orchestra.run.smithi191.stdout: "map_inc": 13, 2024-03-04T09:41:46.496 INFO:teuthology.orchestra.run.smithi191.stdout: "osd_sessions": 9, 2024-03-04T09:41:46.496 INFO:teuthology.orchestra.run.smithi191.stdout: "osd_session_open": 9, 2024-03-04T09:41:46.497 INFO:teuthology.orchestra.run.smithi191.stdout: "osd_session_close": 0, 2024-03-04T09:41:46.497 INFO:teuthology.orchestra.run.smithi191.stdout: "osd_laggy": 0, 2024-03-04T09:41:46.497 INFO:teuthology.orchestra.run.smithi191.stdout: "omap_wr": 25, 2024-03-04T09:41:46.497 INFO:teuthology.orchestra.run.smithi191.stdout: "omap_rd": 0, 2024-03-04T09:41:46.497 INFO:teuthology.orchestra.run.smithi191.stdout: "omap_del": 5 2024-03-04T09:41:46.497 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-03-04T09:41:46.497 INFO:teuthology.orchestra.run.smithi191.stdout: "oft": { 2024-03-04T09:41:46.497 INFO:teuthology.orchestra.run.smithi191.stdout: "omap_total_objs": 1, 2024-03-04T09:41:46.498 INFO:teuthology.orchestra.run.smithi191.stdout: "omap_total_kv_pairs": 0, 2024-03-04T09:41:46.498 INFO:teuthology.orchestra.run.smithi191.stdout: "omap_total_updates": 0, 2024-03-04T09:41:46.498 INFO:teuthology.orchestra.run.smithi191.stdout: "omap_total_removes": 0 2024-03-04T09:41:46.498 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-03-04T09:41:46.498 INFO:teuthology.orchestra.run.smithi191.stdout: "osd-slow-ops": { 2024-03-04T09:41:46.498 INFO:teuthology.orchestra.run.smithi191.stdout: "slow_ops_count": 0 2024-03-04T09:41:46.499 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-03-04T09:41:46.499 INFO:teuthology.orchestra.run.smithi191.stdout: "purge_queue": { 2024-03-04T09:41:46.499 INFO:teuthology.orchestra.run.smithi191.stdout: "pq_executing_ops": 0, 2024-03-04T09:41:46.499 INFO:teuthology.orchestra.run.smithi191.stdout: "pq_executing_ops_high_water": 0, 2024-03-04T09:41:46.499 INFO:teuthology.orchestra.run.smithi191.stdout: "pq_executing": 0, 2024-03-04T09:41:46.499 INFO:teuthology.orchestra.run.smithi191.stdout: "pq_executing_high_water": 0, 2024-03-04T09:41:46.500 INFO:teuthology.orchestra.run.smithi191.stdout: "pq_executed_ops": 0, 2024-03-04T09:41:46.500 INFO:teuthology.orchestra.run.smithi191.stdout: "pq_executed": 0, 2024-03-04T09:41:46.500 INFO:teuthology.orchestra.run.smithi191.stdout: "pq_item_in_journal": 0 2024-03-04T09:41:46.500 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-03-04T09:41:46.500 INFO:teuthology.orchestra.run.smithi191.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-04T09:41:46.500 INFO:teuthology.orchestra.run.smithi191.stdout: "val": 0, 2024-03-04T09:41:46.501 INFO:teuthology.orchestra.run.smithi191.stdout: "max": 104857600, 2024-03-04T09:41:46.501 INFO:teuthology.orchestra.run.smithi191.stdout: "get_started": 0, 2024-03-04T09:41:46.501 INFO:teuthology.orchestra.run.smithi191.stdout: "get": 1024, 2024-03-04T09:41:46.501 INFO:teuthology.orchestra.run.smithi191.stdout: "get_sum": 1728527, 2024-03-04T09:41:46.501 INFO:teuthology.orchestra.run.smithi191.stdout: "get_or_fail_fail": 0, 2024-03-04T09:41:46.501 INFO:teuthology.orchestra.run.smithi191.stdout: "get_or_fail_success": 1024, 2024-03-04T09:41:46.501 INFO:teuthology.orchestra.run.smithi191.stdout: "take": 0, 2024-03-04T09:41:46.502 INFO:teuthology.orchestra.run.smithi191.stdout: "take_sum": 0, 2024-03-04T09:41:46.502 INFO:teuthology.orchestra.run.smithi191.stdout: "put": 1024, 2024-03-04T09:41:46.502 INFO:teuthology.orchestra.run.smithi191.stdout: "put_sum": 1728527, 2024-03-04T09:41:46.502 INFO:teuthology.orchestra.run.smithi191.stdout: "wait": { 2024-03-04T09:41:46.502 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-03-04T09:41:46.502 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-03-04T09:41:46.502 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-03-04T09:41:46.503 INFO:teuthology.orchestra.run.smithi191.stdout: } 2024-03-04T09:41:46.503 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-03-04T09:41:46.503 INFO:teuthology.orchestra.run.smithi191.stdout: "throttle-objecter_bytes": { 2024-03-04T09:41:46.503 INFO:teuthology.orchestra.run.smithi191.stdout: "val": 0, 2024-03-04T09:41:46.503 INFO:teuthology.orchestra.run.smithi191.stdout: "max": 104857600, 2024-03-04T09:41:46.503 INFO:teuthology.orchestra.run.smithi191.stdout: "get_started": 0, 2024-03-04T09:41:46.504 INFO:teuthology.orchestra.run.smithi191.stdout: "get": 0, 2024-03-04T09:41:46.504 INFO:teuthology.orchestra.run.smithi191.stdout: "get_sum": 0, 2024-03-04T09:41:46.504 INFO:teuthology.orchestra.run.smithi191.stdout: "get_or_fail_fail": 0, 2024-03-04T09:41:46.504 INFO:teuthology.orchestra.run.smithi191.stdout: "get_or_fail_success": 0, 2024-03-04T09:41:46.504 INFO:teuthology.orchestra.run.smithi191.stdout: "take": 73, 2024-03-04T09:41:46.504 INFO:teuthology.orchestra.run.smithi191.stdout: "take_sum": 389107, 2024-03-04T09:41:46.504 INFO:teuthology.orchestra.run.smithi191.stdout: "put": 73, 2024-03-04T09:41:46.505 INFO:teuthology.orchestra.run.smithi191.stdout: "put_sum": 389107, 2024-03-04T09:41:46.505 INFO:teuthology.orchestra.run.smithi191.stdout: "wait": { 2024-03-04T09:41:46.505 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-03-04T09:41:46.505 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-03-04T09:41:46.505 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-03-04T09:41:46.505 INFO:teuthology.orchestra.run.smithi191.stdout: } 2024-03-04T09:41:46.506 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-03-04T09:41:46.506 INFO:teuthology.orchestra.run.smithi191.stdout: "throttle-objecter_ops": { 2024-03-04T09:41:46.506 INFO:teuthology.orchestra.run.smithi191.stdout: "val": 0, 2024-03-04T09:41:46.506 INFO:teuthology.orchestra.run.smithi191.stdout: "max": 1024, 2024-03-04T09:41:46.506 INFO:teuthology.orchestra.run.smithi191.stdout: "get_started": 0, 2024-03-04T09:41:46.506 INFO:teuthology.orchestra.run.smithi191.stdout: "get": 0, 2024-03-04T09:41:46.506 INFO:teuthology.orchestra.run.smithi191.stdout: "get_sum": 0, 2024-03-04T09:41:46.506 INFO:teuthology.orchestra.run.smithi191.stdout: "get_or_fail_fail": 0, 2024-03-04T09:41:46.507 INFO:teuthology.orchestra.run.smithi191.stdout: "get_or_fail_success": 0, 2024-03-04T09:41:46.507 INFO:teuthology.orchestra.run.smithi191.stdout: "take": 73, 2024-03-04T09:41:46.507 INFO:teuthology.orchestra.run.smithi191.stdout: "take_sum": 73, 2024-03-04T09:41:46.507 INFO:teuthology.orchestra.run.smithi191.stdout: "put": 73, 2024-03-04T09:41:46.507 INFO:teuthology.orchestra.run.smithi191.stdout: "put_sum": 73, 2024-03-04T09:41:46.507 INFO:teuthology.orchestra.run.smithi191.stdout: "wait": { 2024-03-04T09:41:46.507 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-03-04T09:41:46.508 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-03-04T09:41:46.508 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-03-04T09:41:46.508 INFO:teuthology.orchestra.run.smithi191.stdout: } 2024-03-04T09:41:46.508 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-03-04T09:41:46.508 INFO:teuthology.orchestra.run.smithi191.stdout: "throttle-write_buf_throttle": { 2024-03-04T09:41:46.508 INFO:teuthology.orchestra.run.smithi191.stdout: "val": 0, 2024-03-04T09:41:46.508 INFO:teuthology.orchestra.run.smithi191.stdout: "max": 3758096384, 2024-03-04T09:41:46.509 INFO:teuthology.orchestra.run.smithi191.stdout: "get_started": 0, 2024-03-04T09:41:46.509 INFO:teuthology.orchestra.run.smithi191.stdout: "get": 0, 2024-03-04T09:41:46.509 INFO:teuthology.orchestra.run.smithi191.stdout: "get_sum": 0, 2024-03-04T09:41:46.509 INFO:teuthology.orchestra.run.smithi191.stdout: "get_or_fail_fail": 0, 2024-03-04T09:41:46.509 INFO:teuthology.orchestra.run.smithi191.stdout: "get_or_fail_success": 0, 2024-03-04T09:41:46.509 INFO:teuthology.orchestra.run.smithi191.stdout: "take": 0, 2024-03-04T09:41:46.510 INFO:teuthology.orchestra.run.smithi191.stdout: "take_sum": 0, 2024-03-04T09:41:46.510 INFO:teuthology.orchestra.run.smithi191.stdout: "put": 0, 2024-03-04T09:41:46.510 INFO:teuthology.orchestra.run.smithi191.stdout: "put_sum": 0, 2024-03-04T09:41:46.510 INFO:teuthology.orchestra.run.smithi191.stdout: "wait": { 2024-03-04T09:41:46.510 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-03-04T09:41:46.510 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-03-04T09:41:46.510 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-03-04T09:41:46.510 INFO:teuthology.orchestra.run.smithi191.stdout: } 2024-03-04T09:41:46.511 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-03-04T09:41:46.511 INFO:teuthology.orchestra.run.smithi191.stdout: "throttle-write_buf_throttle-0x5575020646e0": { 2024-03-04T09:41:46.511 INFO:teuthology.orchestra.run.smithi191.stdout: "val": 0, 2024-03-04T09:41:46.511 INFO:teuthology.orchestra.run.smithi191.stdout: "max": 3758096384, 2024-03-04T09:41:46.511 INFO:teuthology.orchestra.run.smithi191.stdout: "get_started": 0, 2024-03-04T09:41:46.511 INFO:teuthology.orchestra.run.smithi191.stdout: "get": 134, 2024-03-04T09:41:46.511 INFO:teuthology.orchestra.run.smithi191.stdout: "get_sum": 368179, 2024-03-04T09:41:46.511 INFO:teuthology.orchestra.run.smithi191.stdout: "get_or_fail_fail": 0, 2024-03-04T09:41:46.512 INFO:teuthology.orchestra.run.smithi191.stdout: "get_or_fail_success": 134, 2024-03-04T09:41:46.512 INFO:teuthology.orchestra.run.smithi191.stdout: "take": 0, 2024-03-04T09:41:46.512 INFO:teuthology.orchestra.run.smithi191.stdout: "take_sum": 0, 2024-03-04T09:41:46.512 INFO:teuthology.orchestra.run.smithi191.stdout: "put": 26, 2024-03-04T09:41:46.512 INFO:teuthology.orchestra.run.smithi191.stdout: "put_sum": 368179, 2024-03-04T09:41:46.512 INFO:teuthology.orchestra.run.smithi191.stdout: "wait": { 2024-03-04T09:41:46.512 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-03-04T09:41:46.512 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-03-04T09:41:46.513 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-03-04T09:41:46.513 INFO:teuthology.orchestra.run.smithi191.stdout: } 2024-03-04T09:41:46.513 INFO:teuthology.orchestra.run.smithi191.stdout: } 2024-03-04T09:41:46.513 INFO:teuthology.orchestra.run.smithi191.stdout:} 2024-03-04T09:41:46.586 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:41:46 smithi052 ceph-mon[32473]: pgmap v329: 129 pgs: 129 active+clean; 78 MiB data, 691 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-04T09:41:46.632 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:41:46 smithi106 ceph-mon[36484]: pgmap v329: 129 pgs: 129 active+clean; 78 MiB data, 691 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-04T09:41:47.074 INFO:tasks.check_counter:Daemon mds.i mds.exported=0 2024-03-04T09:41:47.074 INFO:tasks.check_counter:Daemon mds.i mds.imported=6 2024-03-04T09:41:47.074 INFO:tasks.check_counter:Daemon mds.i mds.root_rsnaps=0 2024-03-04T09:41:47.074 INFO:tasks.check_counter:Daemon mds.i mds_server.req_mksnap_latency.avgcount=0 2024-03-04T09:41:47.074 INFO:tasks.check_counter:Daemon mds.i mds_server.req_rmsnap_latency.avgcount=0 2024-03-04T09:41:47.075 INFO:tasks.check_counter:Daemon mds.i mds.dir_split=0 2024-03-04T09:41:47.075 DEBUG:tasks.check_counter:Getting stats from l 2024-03-04T09:41:47.075 DEBUG:teuthology.orchestra.run.smithi191:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd shell --fsid 4e059f08-da08-11ee-95c6-87774f69a715 -- ceph daemon mds.l perf dump 2024-03-04T09:41:47.330 INFO:teuthology.orchestra.run.smithi191.stderr:Inferring config /var/lib/ceph/4e059f08-da08-11ee-95c6-87774f69a715/mon.c/config 2024-03-04T09:41:48.481 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:41:48 smithi191 ceph-mon[36816]: pgmap v330: 129 pgs: 129 active+clean; 78 MiB data, 691 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-04T09:41:48.585 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:41:48 smithi052 ceph-mon[32473]: pgmap v330: 129 pgs: 129 active+clean; 78 MiB data, 691 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-04T09:41:48.632 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:41:48 smithi106 ceph-mon[36484]: pgmap v330: 129 pgs: 129 active+clean; 78 MiB data, 691 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-04T09:41:49.118 INFO:teuthology.orchestra.run.smithi191.stdout:{ 2024-03-04T09:41:49.118 INFO:teuthology.orchestra.run.smithi191.stdout: "AsyncMessenger::Worker-0": { 2024-03-04T09:41:49.119 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_recv_messages": 0, 2024-03-04T09:41:49.119 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_send_messages": 2, 2024-03-04T09:41:49.119 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_recv_bytes": 0, 2024-03-04T09:41:49.119 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_send_bytes": 0, 2024-03-04T09:41:49.119 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_created_connections": 4, 2024-03-04T09:41:49.119 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_active_connections": 0, 2024-03-04T09:41:49.119 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_running_total_time": 0.000972877, 2024-03-04T09:41:49.119 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_running_send_time": 0.000000000, 2024-03-04T09:41:49.119 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_running_recv_time": 0.000178938, 2024-03-04T09:41:49.120 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-03-04T09:41:49.120 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_send_messages_queue_lat": { 2024-03-04T09:41:49.120 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-03-04T09:41:49.120 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-03-04T09:41:49.120 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-03-04T09:41:49.120 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-03-04T09:41:49.120 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_handle_ack_lat": { 2024-03-04T09:41:49.120 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-03-04T09:41:49.121 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-03-04T09:41:49.121 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-03-04T09:41:49.121 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-03-04T09:41:49.121 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_recv_encrypted_bytes": 0, 2024-03-04T09:41:49.121 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_send_encrypted_bytes": 0 2024-03-04T09:41:49.121 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-03-04T09:41:49.121 INFO:teuthology.orchestra.run.smithi191.stdout: "AsyncMessenger::Worker-1": { 2024-03-04T09:41:49.121 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_recv_messages": 236, 2024-03-04T09:41:49.122 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_send_messages": 224, 2024-03-04T09:41:49.122 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_recv_bytes": 1438144, 2024-03-04T09:41:49.122 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_send_bytes": 133504, 2024-03-04T09:41:49.122 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_created_connections": 2, 2024-03-04T09:41:49.122 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_active_connections": 1, 2024-03-04T09:41:49.122 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_running_total_time": 0.035776747, 2024-03-04T09:41:49.122 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_running_send_time": 0.008548926, 2024-03-04T09:41:49.122 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_running_recv_time": 0.032943563, 2024-03-04T09:41:49.123 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_running_fast_dispatch_time": 0.003792182, 2024-03-04T09:41:49.123 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_send_messages_queue_lat": { 2024-03-04T09:41:49.123 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 224, 2024-03-04T09:41:49.123 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.008393686, 2024-03-04T09:41:49.123 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000037471 2024-03-04T09:41:49.123 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-03-04T09:41:49.123 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_handle_ack_lat": { 2024-03-04T09:41:49.123 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-03-04T09:41:49.123 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-03-04T09:41:49.124 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-03-04T09:41:49.124 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-03-04T09:41:49.124 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_recv_encrypted_bytes": 1438144, 2024-03-04T09:41:49.124 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_send_encrypted_bytes": 133504 2024-03-04T09:41:49.124 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-03-04T09:41:49.124 INFO:teuthology.orchestra.run.smithi191.stdout: "AsyncMessenger::Worker-2": { 2024-03-04T09:41:49.124 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_recv_messages": 2, 2024-03-04T09:41:49.124 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_send_messages": 169, 2024-03-04T09:41:49.125 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_recv_bytes": 320, 2024-03-04T09:41:49.125 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_send_bytes": 184768, 2024-03-04T09:41:49.125 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_created_connections": 5, 2024-03-04T09:41:49.125 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_active_connections": 3, 2024-03-04T09:41:49.125 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_running_total_time": 0.007885214, 2024-03-04T09:41:49.125 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_running_send_time": 0.004786044, 2024-03-04T09:41:49.125 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_running_recv_time": 0.001563575, 2024-03-04T09:41:49.125 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-03-04T09:41:49.126 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_send_messages_queue_lat": { 2024-03-04T09:41:49.126 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 166, 2024-03-04T09:41:49.126 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.016284486, 2024-03-04T09:41:49.126 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000098099 2024-03-04T09:41:49.126 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-03-04T09:41:49.126 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_handle_ack_lat": { 2024-03-04T09:41:49.126 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-03-04T09:41:49.126 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-03-04T09:41:49.126 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-03-04T09:41:49.127 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-03-04T09:41:49.127 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_recv_encrypted_bytes": 320, 2024-03-04T09:41:49.127 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_send_encrypted_bytes": 184768 2024-03-04T09:41:49.127 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-03-04T09:41:49.127 INFO:teuthology.orchestra.run.smithi191.stdout: "cct": { 2024-03-04T09:41:49.127 INFO:teuthology.orchestra.run.smithi191.stdout: "total_workers": 0, 2024-03-04T09:41:49.127 INFO:teuthology.orchestra.run.smithi191.stdout: "unhealthy_workers": 0 2024-03-04T09:41:49.127 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-03-04T09:41:49.128 INFO:teuthology.orchestra.run.smithi191.stdout: "mempool": { 2024-03-04T09:41:49.128 INFO:teuthology.orchestra.run.smithi191.stdout: "bloom_filter_bytes": 0, 2024-03-04T09:41:49.128 INFO:teuthology.orchestra.run.smithi191.stdout: "bloom_filter_items": 0, 2024-03-04T09:41:49.128 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_alloc_bytes": 0, 2024-03-04T09:41:49.128 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_alloc_items": 0, 2024-03-04T09:41:49.128 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_cache_data_bytes": 0, 2024-03-04T09:41:49.128 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_cache_data_items": 0, 2024-03-04T09:41:49.128 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-04T09:41:49.128 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_cache_onode_items": 0, 2024-03-04T09:41:49.129 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-04T09:41:49.129 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_cache_meta_items": 0, 2024-03-04T09:41:49.129 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_cache_other_bytes": 0, 2024-03-04T09:41:49.129 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_cache_other_items": 0, 2024-03-04T09:41:49.129 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-04T09:41:49.129 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_cache_buffer_items": 0, 2024-03-04T09:41:49.129 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_extent_bytes": 0, 2024-03-04T09:41:49.129 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_extent_items": 0, 2024-03-04T09:41:49.130 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_blob_bytes": 0, 2024-03-04T09:41:49.130 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_blob_items": 0, 2024-03-04T09:41:49.130 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-04T09:41:49.130 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_shared_blob_items": 0, 2024-03-04T09:41:49.130 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-04T09:41:49.130 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_inline_bl_items": 0, 2024-03-04T09:41:49.130 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_fsck_bytes": 0, 2024-03-04T09:41:49.130 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_fsck_items": 0, 2024-03-04T09:41:49.130 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_txc_bytes": 0, 2024-03-04T09:41:49.131 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_txc_items": 0, 2024-03-04T09:41:49.131 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-04T09:41:49.131 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_writing_deferred_items": 0, 2024-03-04T09:41:49.131 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_writing_bytes": 0, 2024-03-04T09:41:49.131 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_writing_items": 0, 2024-03-04T09:41:49.131 INFO:teuthology.orchestra.run.smithi191.stdout: "bluefs_bytes": 0, 2024-03-04T09:41:49.131 INFO:teuthology.orchestra.run.smithi191.stdout: "bluefs_items": 0, 2024-03-04T09:41:49.131 INFO:teuthology.orchestra.run.smithi191.stdout: "bluefs_file_reader_bytes": 0, 2024-03-04T09:41:49.132 INFO:teuthology.orchestra.run.smithi191.stdout: "bluefs_file_reader_items": 0, 2024-03-04T09:41:49.132 INFO:teuthology.orchestra.run.smithi191.stdout: "bluefs_file_writer_bytes": 0, 2024-03-04T09:41:49.132 INFO:teuthology.orchestra.run.smithi191.stdout: "bluefs_file_writer_items": 0, 2024-03-04T09:41:49.132 INFO:teuthology.orchestra.run.smithi191.stdout: "buffer_anon_bytes": 148754, 2024-03-04T09:41:49.132 INFO:teuthology.orchestra.run.smithi191.stdout: "buffer_anon_items": 31, 2024-03-04T09:41:49.132 INFO:teuthology.orchestra.run.smithi191.stdout: "buffer_meta_bytes": 0, 2024-03-04T09:41:49.132 INFO:teuthology.orchestra.run.smithi191.stdout: "buffer_meta_items": 0, 2024-03-04T09:41:49.132 INFO:teuthology.orchestra.run.smithi191.stdout: "osd_bytes": 0, 2024-03-04T09:41:49.133 INFO:teuthology.orchestra.run.smithi191.stdout: "osd_items": 0, 2024-03-04T09:41:49.133 INFO:teuthology.orchestra.run.smithi191.stdout: "osd_mapbl_bytes": 0, 2024-03-04T09:41:49.133 INFO:teuthology.orchestra.run.smithi191.stdout: "osd_mapbl_items": 0, 2024-03-04T09:41:49.133 INFO:teuthology.orchestra.run.smithi191.stdout: "osd_pglog_bytes": 0, 2024-03-04T09:41:49.133 INFO:teuthology.orchestra.run.smithi191.stdout: "osd_pglog_items": 0, 2024-03-04T09:41:49.133 INFO:teuthology.orchestra.run.smithi191.stdout: "osdmap_bytes": 0, 2024-03-04T09:41:49.133 INFO:teuthology.orchestra.run.smithi191.stdout: "osdmap_items": 0, 2024-03-04T09:41:49.133 INFO:teuthology.orchestra.run.smithi191.stdout: "osdmap_mapping_bytes": 0, 2024-03-04T09:41:49.133 INFO:teuthology.orchestra.run.smithi191.stdout: "osdmap_mapping_items": 0, 2024-03-04T09:41:49.134 INFO:teuthology.orchestra.run.smithi191.stdout: "pgmap_bytes": 0, 2024-03-04T09:41:49.134 INFO:teuthology.orchestra.run.smithi191.stdout: "pgmap_items": 0, 2024-03-04T09:41:49.134 INFO:teuthology.orchestra.run.smithi191.stdout: "mds_co_bytes": 928, 2024-03-04T09:41:49.134 INFO:teuthology.orchestra.run.smithi191.stdout: "mds_co_items": 2, 2024-03-04T09:41:49.134 INFO:teuthology.orchestra.run.smithi191.stdout: "unittest_1_bytes": 0, 2024-03-04T09:41:49.134 INFO:teuthology.orchestra.run.smithi191.stdout: "unittest_1_items": 0, 2024-03-04T09:41:49.134 INFO:teuthology.orchestra.run.smithi191.stdout: "unittest_2_bytes": 0, 2024-03-04T09:41:49.134 INFO:teuthology.orchestra.run.smithi191.stdout: "unittest_2_items": 0 2024-03-04T09:41:49.135 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-03-04T09:41:49.135 INFO:teuthology.orchestra.run.smithi191.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-04T09:41:49.135 INFO:teuthology.orchestra.run.smithi191.stdout: "val": 0, 2024-03-04T09:41:49.135 INFO:teuthology.orchestra.run.smithi191.stdout: "max": 104857600, 2024-03-04T09:41:49.135 INFO:teuthology.orchestra.run.smithi191.stdout: "get_started": 0, 2024-03-04T09:41:49.135 INFO:teuthology.orchestra.run.smithi191.stdout: "get": 234, 2024-03-04T09:41:49.135 INFO:teuthology.orchestra.run.smithi191.stdout: "get_sum": 1402398, 2024-03-04T09:41:49.135 INFO:teuthology.orchestra.run.smithi191.stdout: "get_or_fail_fail": 0, 2024-03-04T09:41:49.135 INFO:teuthology.orchestra.run.smithi191.stdout: "get_or_fail_success": 234, 2024-03-04T09:41:49.136 INFO:teuthology.orchestra.run.smithi191.stdout: "take": 0, 2024-03-04T09:41:49.136 INFO:teuthology.orchestra.run.smithi191.stdout: "take_sum": 0, 2024-03-04T09:41:49.136 INFO:teuthology.orchestra.run.smithi191.stdout: "put": 234, 2024-03-04T09:41:49.136 INFO:teuthology.orchestra.run.smithi191.stdout: "put_sum": 1402398, 2024-03-04T09:41:49.136 INFO:teuthology.orchestra.run.smithi191.stdout: "wait": { 2024-03-04T09:41:49.136 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-03-04T09:41:49.136 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-03-04T09:41:49.136 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-03-04T09:41:49.137 INFO:teuthology.orchestra.run.smithi191.stdout: } 2024-03-04T09:41:49.137 INFO:teuthology.orchestra.run.smithi191.stdout: } 2024-03-04T09:41:49.137 INFO:teuthology.orchestra.run.smithi191.stdout:} 2024-03-04T09:41:49.799 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.l 2024-03-04T09:41:49.800 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.l 2024-03-04T09:41:49.800 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.l 2024-03-04T09:41:49.800 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.l 2024-03-04T09:41:49.800 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.l 2024-03-04T09:41:49.800 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.l 2024-03-04T09:41:49.801 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-03-04T09:41:49.812 INFO:tasks.kclient:Unmounting kernel clients... 2024-03-04T09:41:49.813 DEBUG:teuthology.orchestra.run.smithi052:> set -ex 2024-03-04T09:41:49.813 DEBUG:teuthology.orchestra.run.smithi052:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-04T09:41:49.833 DEBUG:teuthology.orchestra.run.smithi052:> set -ex 2024-03-04T09:41:49.833 DEBUG:teuthology.orchestra.run.smithi052:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-04T09:41:49.895 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-04T09:41:50.050 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:50.049+0000 7fccff866640 1 -- 172.21.15.52:0/1892006021 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fccf81004d0 msgr2=0x7fccf8100950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:41:50.050 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:50.049+0000 7fccff866640 1 --2- 172.21.15.52:0/1892006021 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fccf81004d0 0x7fccf8100950 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fcce8009890 tx=0x7fcce802bb70 comp rx=0 tx=0).stop 2024-03-04T09:41:50.050 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:50.049+0000 7fccff866640 1 -- 172.21.15.52:0/1892006021 shutdown_connections 2024-03-04T09:41:50.050 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:50.049+0000 7fccff866640 1 --2- 172.21.15.52:0/1892006021 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fccf8100e90 0x7fccf810d370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:41:50.050 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:50.049+0000 7fccff866640 1 --2- 172.21.15.52:0/1892006021 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fccf81004d0 0x7fccf8100950 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:41:50.051 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:50.049+0000 7fccff866640 1 --2- 172.21.15.52:0/1892006021 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fccf8106660 0x7fccf8106a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:41:50.051 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:50.049+0000 7fccff866640 1 -- 172.21.15.52:0/1892006021 >> 172.21.15.52:0/1892006021 conn(0x7fccf80775d0 msgr2=0x7fccf80779e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:41:50.051 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:50.050+0000 7fccff866640 1 -- 172.21.15.52:0/1892006021 shutdown_connections 2024-03-04T09:41:50.051 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:50.050+0000 7fccff866640 1 -- 172.21.15.52:0/1892006021 wait complete. 2024-03-04T09:41:50.051 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:50.050+0000 7fccff866640 1 Processor -- start 2024-03-04T09:41:50.051 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:50.050+0000 7fccff866640 1 -- start start 2024-03-04T09:41:50.051 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:50.050+0000 7fccff866640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fccf81004d0 0x7fccf810dd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:41:50.052 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:50.050+0000 7fccff866640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fccf8100e90 0x7fccf810e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:41:50.052 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:50.050+0000 7fccff866640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fccf8106660 0x7fccf810e800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:41:50.052 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:50.050+0000 7fccff866640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccf81119f0 con 0x7fccf8106660 2024-03-04T09:41:50.052 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:50.050+0000 7fccff866640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccf8111870 con 0x7fccf81004d0 2024-03-04T09:41:50.052 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:50.050+0000 7fccff866640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccf8111b70 con 0x7fccf8100e90 2024-03-04T09:41:50.052 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:50.051+0000 7fccfde03640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fccf8106660 0x7fccf810e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:41:50.053 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:50.051+0000 7fccfd602640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fccf81004d0 0x7fccf810dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:41:50.053 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:50.051+0000 7fccfd602640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fccf81004d0 0x7fccf810dd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.52:51710/0 (socket says 172.21.15.52:51710) 2024-03-04T09:41:50.053 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:50.051+0000 7fccfde03640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fccf8106660 0x7fccf810e800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:54004/0 (socket says 172.21.15.52:54004) 2024-03-04T09:41:50.053 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:50.051+0000 7fccfd602640 1 -- 172.21.15.52:0/3396110437 learned_addr learned my addr 172.21.15.52:0/3396110437 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-04T09:41:50.053 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:50.051+0000 7fccfce01640 1 --2- 172.21.15.52:0/3396110437 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fccf8100e90 0x7fccf810e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:41:50.053 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:50.051+0000 7fccfde03640 1 -- 172.21.15.52:0/3396110437 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fccf8100e90 msgr2=0x7fccf810e2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:41:50.053 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:50.051+0000 7fccfde03640 1 --2- 172.21.15.52:0/3396110437 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fccf8100e90 0x7fccf810e2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:41:50.053 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:50.051+0000 7fccfde03640 1 -- 172.21.15.52:0/3396110437 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fccf81004d0 msgr2=0x7fccf810dd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:41:50.053 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:50.051+0000 7fccfde03640 1 --2- 172.21.15.52:0/3396110437 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fccf81004d0 0x7fccf810dd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:41:50.054 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:50.051+0000 7fccfde03640 1 -- 172.21.15.52:0/3396110437 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fccf81a6350 con 0x7fccf8106660 2024-03-04T09:41:50.054 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:50.051+0000 7fccfd602640 1 --2- 172.21.15.52:0/3396110437 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fccf81004d0 0x7fccf810dd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-04T09:41:50.054 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:50.051+0000 7fccfce01640 1 --2- 172.21.15.52:0/3396110437 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fccf8100e90 0x7fccf810e2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-04T09:41:50.054 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:50.051+0000 7fccfde03640 1 --2- 172.21.15.52:0/3396110437 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fccf8106660 0x7fccf810e800 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7fccf000ef00 tx=0x7fccf000c740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:41:50.054 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:50.051+0000 7fccee7fc640 1 -- 172.21.15.52:0/3396110437 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccf0004b20 con 0x7fccf8106660 2024-03-04T09:41:50.054 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:50.051+0000 7fccff866640 1 -- 172.21.15.52:0/3396110437 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fccf81a6640 con 0x7fccf8106660 2024-03-04T09:41:50.055 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:50.051+0000 7fccff866640 1 -- 172.21.15.52:0/3396110437 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fccf81a6bb0 con 0x7fccf8106660 2024-03-04T09:41:50.055 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:50.051+0000 7fccee7fc640 1 -- 172.21.15.52:0/3396110437 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fccf0003e50 con 0x7fccf8106660 2024-03-04T09:41:50.055 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:50.051+0000 7fccee7fc640 1 -- 172.21.15.52:0/3396110437 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccf001a020 con 0x7fccf8106660 2024-03-04T09:41:50.055 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:50.052+0000 7fccff866640 1 -- 172.21.15.52:0/3396110437 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fccdc005290 con 0x7fccf8106660 2024-03-04T09:41:50.056 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:50.053+0000 7fccee7fc640 1 -- 172.21.15.52:0/3396110437 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 25) v1 ==== 136887+0+0 (secure 0 0 0) 0x7fccf001d020 con 0x7fccf8106660 2024-03-04T09:41:50.056 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:50.054+0000 7fccee7fc640 1 --2- 172.21.15.52:0/3396110437 >> [v2:172.21.15.52:6800/478031757,v1:172.21.15.52:6801/478031757] conn(0x7fcce40a1ef0 0x7fcce40a43b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:41:50.056 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:50.054+0000 7fccee7fc640 1 -- 172.21.15.52:0/3396110437 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 9547+0+0 (secure 0 0 0) 0x7fccf00cea90 con 0x7fccf8106660 2024-03-04T09:41:50.057 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:50.054+0000 7fccfd602640 1 --2- 172.21.15.52:0/3396110437 >> [v2:172.21.15.52:6800/478031757,v1:172.21.15.52:6801/478031757] conn(0x7fcce40a1ef0 0x7fcce40a43b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:41:50.057 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:50.054+0000 7fccfd602640 1 --2- 172.21.15.52:0/3396110437 >> [v2:172.21.15.52:6800/478031757,v1:172.21.15.52:6801/478031757] conn(0x7fcce40a1ef0 0x7fcce40a43b0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fccf4011440 tx=0x7fccf4005f90 comp rx=0 tx=0).ready entity=mgr.15486 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:41:50.057 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:50.058+0000 7fccee7fc640 1 -- 172.21.15.52:0/3396110437 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fccf0024070 con 0x7fccf8106660 2024-03-04T09:41:50.239 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:50.241+0000 7fccff866640 1 -- 172.21.15.52:0/3396110437 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fccdc005850 con 0x7fccf8106660 2024-03-04T09:41:50.241 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:50.243+0000 7fccee7fc640 1 -- 172.21.15.52:0/3396110437 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v102) v1 ==== 74+0+25427 (secure 0 0 0) 0x7fccf009b720 con 0x7fccf8106660 2024-03-04T09:41:50.242 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-03-04T09:41:50.242 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":102,"fsid":"4e059f08-da08-11ee-95c6-87774f69a715","created":"2024-03-04T09:20:46.104963+0000","modified":"2024-03-04T09:41:15.999027+0000","last_up_change":"2024-03-04T09:27:49.261365+0000","last_in_change":"2024-03-04T09:27:30.024964+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-04T09:24:01.742461+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-04T09:28:35.835600+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-04T09:28:36.453099+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"101","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":15,"snap_epoch":101,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-04T09:28:40.168831+0000","flags":8197,"flags_names":"hashpspool,ec_overwrites,selfmanaged_snaps","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"101","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":15,"snap_epoch":101,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"583172be-602f-4bf4-bfcb-42663cf5b277","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6802","nonce":1371221397},{"type":"v1","addr":"172.21.15.52:6803","nonce":1371221397}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6804","nonce":1371221397},{"type":"v1","addr":"172.21.15.52:6805","nonce":1371221397}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6808","nonce":1371221397},{"type":"v1","addr":"172.21.15.52:6809","nonce":1371221397}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6806","nonce":1371221397},{"type":"v1","addr":"172.21.15.52:6807","nonce":1371221397}]},"public_addr":"172.21.15.52:6803/1371221397","cluster_addr":"172.21.15.52:6805/1371221397","heartbeat_back_addr":"172.21.15.52:6809/1371221397","heartbeat_front_addr":"172.21.15.52:6807/1371221397","state":["exists","up"]},{"osd":1,"uuid":"5ae0081c-a2f6-46eb-ace0-0368208377b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":2540865537},{"type":"v1","addr":"172.21.15.106:6801","nonce":2540865537}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":2540865537},{"type":"v1","addr":"172.21.15.106:6803","nonce":2540865537}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":2540865537},{"type":"v1","addr":"172.21.15.106:6807","nonce":2540865537}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":2540865537},{"type":"v1","addr":"172.21.15.106:6805","nonce":2540865537}]},"public_addr":"172.21.15.106:6801/2540865537","cluster_addr":"172.21.15.106:6803/2540865537","heartbeat_back_addr":"172.21.15.106:6807/2540865537","heartbeat_front_addr":"172.21.15.106:6805/2540865537","state":["exists","up"]},{"osd":2,"uuid":"1948bac7-2bb2-4218-8b93-4e4743a0de04","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":1377083164},{"type":"v1","addr":"172.21.15.191:6801","nonce":1377083164}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":1377083164},{"type":"v1","addr":"172.21.15.191:6803","nonce":1377083164}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":1377083164},{"type":"v1","addr":"172.21.15.191:6807","nonce":1377083164}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":1377083164},{"type":"v1","addr":"172.21.15.191:6805","nonce":1377083164}]},"public_addr":"172.21.15.191:6801/1377083164","cluster_addr":"172.21.15.191:6803/1377083164","heartbeat_back_addr":"172.21.15.191:6807/1377083164","heartbeat_front_addr":"172.21.15.191:6805/1377083164","state":["exists","up"]},{"osd":3,"uuid":"16c9e9fe-7167-4992-bf12-1a7398e0498a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6810","nonce":2388571810},{"type":"v1","addr":"172.21.15.52:6811","nonce":2388571810}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6812","nonce":2388571810},{"type":"v1","addr":"172.21.15.52:6813","nonce":2388571810}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6816","nonce":2388571810},{"type":"v1","addr":"172.21.15.52:6817","nonce":2388571810}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6814","nonce":2388571810},{"type":"v1","addr":"172.21.15.52:6815","nonce":2388571810}]},"public_addr":"172.21.15.52:6811/2388571810","cluster_addr":"172.21.15.52:6813/2388571810","heartbeat_back_addr":"172.21.15.52:6817/2388571810","heartbeat_front_addr":"172.21.15.52:6815/2388571810","state":["exists","up"]},{"osd":4,"uuid":"c8ef48c7-7c78-44d3-a9e5-99ec750abd3d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6808","nonce":3465868015},{"type":"v1","addr":"172.21.15.106:6809","nonce":3465868015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6810","nonce":3465868015},{"type":"v1","addr":"172.21.15.106:6811","nonce":3465868015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6814","nonce":3465868015},{"type":"v1","addr":"172.21.15.106:6815","nonce":3465868015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6812","nonce":3465868015},{"type":"v1","addr":"172.21.15.106:6813","nonce":3465868015}]},"public_addr":"172.21.15.106:6809/3465868015","cluster_addr":"172.21.15.106:6811/3465868015","heartbeat_back_addr":"172.21.15.106:6815/3465868015","heartbeat_front_addr":"172.21.15.106:6813/3465868015","state":["exists","up"]},{"osd":5,"uuid":"9e41824a-c5a2-4f88-be50-2ba1dceb3220","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":1485092643},{"type":"v1","addr":"172.21.15.191:6809","nonce":1485092643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":1485092643},{"type":"v1","addr":"172.21.15.191:6811","nonce":1485092643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":1485092643},{"type":"v1","addr":"172.21.15.191:6815","nonce":1485092643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":1485092643},{"type":"v1","addr":"172.21.15.191:6813","nonce":1485092643}]},"public_addr":"172.21.15.191:6809/1485092643","cluster_addr":"172.21.15.191:6811/1485092643","heartbeat_back_addr":"172.21.15.191:6815/1485092643","heartbeat_front_addr":"172.21.15.191:6813/1485092643","state":["exists","up"]},{"osd":6,"uuid":"b35dccee-ec16-452e-ac69-3ac2fb7b27fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6818","nonce":1989797856},{"type":"v1","addr":"172.21.15.52:6819","nonce":1989797856}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6820","nonce":1989797856},{"type":"v1","addr":"172.21.15.52:6821","nonce":1989797856}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6824","nonce":1989797856},{"type":"v1","addr":"172.21.15.52:6825","nonce":1989797856}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6822","nonce":1989797856},{"type":"v1","addr":"172.21.15.52:6823","nonce":1989797856}]},"public_addr":"172.21.15.52:6819/1989797856","cluster_addr":"172.21.15.52:6821/1989797856","heartbeat_back_addr":"172.21.15.52:6825/1989797856","heartbeat_front_addr":"172.21.15.52:6823/1989797856","state":["exists","up"]},{"osd":7,"uuid":"71303e3d-14ec-4cbd-b19c-67645a1de3c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6816","nonce":1450870329},{"type":"v1","addr":"172.21.15.106:6817","nonce":1450870329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":1450870329},{"type":"v1","addr":"172.21.15.106:6819","nonce":1450870329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":1450870329},{"type":"v1","addr":"172.21.15.106:6823","nonce":1450870329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6820","nonce":1450870329},{"type":"v1","addr":"172.21.15.106:6821","nonce":1450870329}]},"public_addr":"172.21.15.106:6817/1450870329","cluster_addr":"172.21.15.106:6819/1450870329","heartbeat_back_addr":"172.21.15.106:6823/1450870329","heartbeat_front_addr":"172.21.15.106:6821/1450870329","state":["exists","up"]},{"osd":8,"uuid":"6f5786c4-4402-435a-bdad-adff46fe9473","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":1114119072},{"type":"v1","addr":"172.21.15.191:6817","nonce":1114119072}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6818","nonce":1114119072},{"type":"v1","addr":"172.21.15.191:6819","nonce":1114119072}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":1114119072},{"type":"v1","addr":"172.21.15.191:6823","nonce":1114119072}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":1114119072},{"type":"v1","addr":"172.21.15.191:6821","nonce":1114119072}]},"public_addr":"172.21.15.191:6817/1114119072","cluster_addr":"172.21.15.191:6819/1114119072","heartbeat_back_addr":"172.21.15.191:6823/1114119072","heartbeat_front_addr":"172.21.15.191:6821/1114119072","state":["exists","up"]},{"osd":9,"uuid":"062bbaa5-1227-4684-b859-450f5594a90c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6826","nonce":3485821914},{"type":"v1","addr":"172.21.15.52:6827","nonce":3485821914}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6828","nonce":3485821914},{"type":"v1","addr":"172.21.15.52:6829","nonce":3485821914}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6832","nonce":3485821914},{"type":"v1","addr":"172.21.15.52:6833","nonce":3485821914}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6830","nonce":3485821914},{"type":"v1","addr":"172.21.15.52:6831","nonce":3485821914}]},"public_addr":"172.21.15.52:6827/3485821914","cluster_addr":"172.21.15.52:6829/3485821914","heartbeat_back_addr":"172.21.15.52:6833/3485821914","heartbeat_front_addr":"172.21.15.52:6831/3485821914","state":["exists","up"]},{"osd":10,"uuid":"1a285a5a-f793-48da-a8eb-5326d9528cd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":3815586576},{"type":"v1","addr":"172.21.15.106:6825","nonce":3815586576}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6826","nonce":3815586576},{"type":"v1","addr":"172.21.15.106:6827","nonce":3815586576}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":3815586576},{"type":"v1","addr":"172.21.15.106:6831","nonce":3815586576}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":3815586576},{"type":"v1","addr":"172.21.15.106:6829","nonce":3815586576}]},"public_addr":"172.21.15.106:6825/3815586576","cluster_addr":"172.21.15.106:6827/3815586576","heartbeat_back_addr":"172.21.15.106:6831/3815586576","heartbeat_front_addr":"172.21.15.106:6829/3815586576","state":["exists","up"]},{"osd":11,"uuid":"1f8b9a43-57fa-41f4-9969-3b803a74bf45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":2009632843},{"type":"v1","addr":"172.21.15.191:6825","nonce":2009632843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":2009632843},{"type":"v1","addr":"172.21.15.191:6827","nonce":2009632843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":2009632843},{"type":"v1","addr":"172.21.15.191:6831","nonce":2009632843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":2009632843},{"type":"v1","addr":"172.21.15.191:6829","nonce":2009632843}]},"public_addr":"172.21.15.191:6825/2009632843","cluster_addr":"172.21.15.191:6827/2009632843","heartbeat_back_addr":"172.21.15.191:6831/2009632843","heartbeat_front_addr":"172.21.15.191:6829/2009632843","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T09:23:10.894763+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T09:23:34.420055+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T09:23:58.288324+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T09:24:23.673457+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T09:24:48.208561+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T09:25:13.405473+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T09:25:38.670915+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T09:26:04.451257+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T09:26:30.281380+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T09:26:56.543143+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T09:27:22.412930+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T09:27:48.003866+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.1d","mappings":[{"from":2,"to":1},{"from":8,"to":11}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.20","mappings":[{"from":5,"to":9}]},{"pgid":"3.21","mappings":[{"from":4,"to":6}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]},{"pgid":"4.9","mappings":[{"from":5,"to":8}]},{"pgid":"4.17","mappings":[{"from":9,"to":8}]},{"pgid":"4.1b","mappings":[{"from":9,"to":10}]},{"pgid":"4.1f","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.52:0/1035878034":"2024-03-05T09:21:12.760938+0000","172.21.15.52:0/2743084243":"2024-03-05T09:21:39.401935+0000","172.21.15.52:0/1094295190":"2024-03-05T09:31:08.422018+0000","172.21.15.52:0/568242577":"2024-03-05T09:21:12.760938+0000","172.21.15.52:0/2080168931":"2024-03-05T09:31:08.422018+0000","172.21.15.52:6801/2425730636":"2024-03-05T09:21:39.401935+0000","172.21.15.52:6800/2379361333":"2024-03-05T09:21:12.760938+0000","172.21.15.52:6800/3036033923":"2024-03-05T09:31:08.422018+0000","172.21.15.52:0/497075075":"2024-03-05T09:21:12.760938+0000","172.21.15.52:6800/2425730636":"2024-03-05T09:21:39.401935+0000","172.21.15.52:6801/2379361333":"2024-03-05T09:21:12.760938+0000","172.21.15.52:0/1271949988":"2024-03-05T09:21:39.401935+0000","172.21.15.52:0/2491451071":"2024-03-05T09:31:08.422018+0000","172.21.15.52:0/2466768":"2024-03-05T09:31:08.422018+0000","172.21.15.52:0/4082645718":"2024-03-05T09:21:39.401935+0000","172.21.15.52:0/2935914357":"2024-03-05T09:31:08.422018+0000","172.21.15.52:6801/3036033923":"2024-03-05T09:31:08.422018+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":5,"length":1},{"begin":15,"length":1}]},{"pool":4,"snaps":[{"begin":5,"length":1},{"begin":15,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-04T09:41:50.244 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:50.245+0000 7fccff866640 1 -- 172.21.15.52:0/3396110437 >> [v2:172.21.15.52:6800/478031757,v1:172.21.15.52:6801/478031757] conn(0x7fcce40a1ef0 msgr2=0x7fcce40a43b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:41:50.244 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:50.245+0000 7fccff866640 1 --2- 172.21.15.52:0/3396110437 >> [v2:172.21.15.52:6800/478031757,v1:172.21.15.52:6801/478031757] conn(0x7fcce40a1ef0 0x7fcce40a43b0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fccf4011440 tx=0x7fccf4005f90 comp rx=0 tx=0).stop 2024-03-04T09:41:50.244 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:50.245+0000 7fccff866640 1 -- 172.21.15.52:0/3396110437 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fccf8106660 msgr2=0x7fccf810e800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:41:50.245 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:50.245+0000 7fccff866640 1 --2- 172.21.15.52:0/3396110437 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fccf8106660 0x7fccf810e800 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7fccf000ef00 tx=0x7fccf000c740 comp rx=0 tx=0).stop 2024-03-04T09:41:50.245 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:50.245+0000 7fccff866640 1 -- 172.21.15.52:0/3396110437 shutdown_connections 2024-03-04T09:41:50.245 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:50.245+0000 7fccff866640 1 --2- 172.21.15.52:0/3396110437 >> [v2:172.21.15.52:6800/478031757,v1:172.21.15.52:6801/478031757] conn(0x7fcce40a1ef0 0x7fcce40a43b0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:41:50.245 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:50.245+0000 7fccff866640 1 --2- 172.21.15.52:0/3396110437 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fccf8106660 0x7fccf810e800 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:41:50.246 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:50.245+0000 7fccff866640 1 --2- 172.21.15.52:0/3396110437 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fccf8100e90 0x7fccf810e2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:41:50.246 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:50.245+0000 7fccff866640 1 --2- 172.21.15.52:0/3396110437 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fccf81004d0 0x7fccf810dd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:41:50.246 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:50.245+0000 7fccff866640 1 -- 172.21.15.52:0/3396110437 >> 172.21.15.52:0/3396110437 conn(0x7fccf80775d0 msgr2=0x7fccf80ff090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:41:50.246 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:50.245+0000 7fccff866640 1 -- 172.21.15.52:0/3396110437 shutdown_connections 2024-03-04T09:41:50.246 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:50.245+0000 7fccff866640 1 -- 172.21.15.52:0/3396110437 wait complete. 2024-03-04T09:41:50.252 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:41:50 smithi052 ceph-mon[32473]: pgmap v331: 129 pgs: 129 active+clean; 78 MiB data, 691 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-04T09:41:50.254 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-04T09:24:01.742461+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-04T09:28:35.835600+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-04T09:28:36.453099+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '101', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 15, 'snap_epoch': 101, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-04T09:28:40.168831+0000', 'flags': 8197, 'flags_names': 'hashpspool,ec_overwrites,selfmanaged_snaps', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '101', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 15, 'snap_epoch': 101, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-04T09:41:50.254 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-04T09:41:50.362 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:50.362+0000 7fb969c81640 1 -- 172.21.15.52:0/1524307513 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb964102840 msgr2=0x7fb964102c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:41:50.363 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:50.362+0000 7fb969c81640 1 --2- 172.21.15.52:0/1524307513 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb964102840 0x7fb964102c20 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fb954009900 tx=0x7fb95402bcd0 comp rx=0 tx=0).stop 2024-03-04T09:41:50.363 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:50.362+0000 7fb969c81640 1 -- 172.21.15.52:0/1524307513 shutdown_connections 2024-03-04T09:41:50.363 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:50.362+0000 7fb969c81640 1 --2- 172.21.15.52:0/1524307513 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb96410bc10 0x7fb96410e080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:41:50.363 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:50.362+0000 7fb969c81640 1 --2- 172.21.15.52:0/1524307513 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb964103160 0x7fb96410b640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:41:50.363 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:50.362+0000 7fb969c81640 1 --2- 172.21.15.52:0/1524307513 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb964102840 0x7fb964102c20 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:41:50.363 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:50.362+0000 7fb969c81640 1 -- 172.21.15.52:0/1524307513 >> 172.21.15.52:0/1524307513 conn(0x7fb964077630 msgr2=0x7fb964077a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:41:50.364 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:50.362+0000 7fb969c81640 1 -- 172.21.15.52:0/1524307513 shutdown_connections 2024-03-04T09:41:50.364 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:50.362+0000 7fb969c81640 1 -- 172.21.15.52:0/1524307513 wait complete. 2024-03-04T09:41:50.364 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:50.363+0000 7fb969c81640 1 Processor -- start 2024-03-04T09:41:50.364 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:50.363+0000 7fb969c81640 1 -- start start 2024-03-04T09:41:50.364 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:50.363+0000 7fb969c81640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb964102840 0x7fb964197f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:41:50.364 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:50.363+0000 7fb969c81640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb964103160 0x7fb9641984a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:41:50.364 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:50.363+0000 7fb969c81640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb96410bc10 0x7fb96419c820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:41:50.364 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:50.363+0000 7fb969c81640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9641119e0 con 0x7fb96410bc10 2024-03-04T09:41:50.365 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:50.363+0000 7fb969c81640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb964111860 con 0x7fb964102840 2024-03-04T09:41:50.365 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:50.363+0000 7fb969c81640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb964111b60 con 0x7fb964103160 2024-03-04T09:41:50.365 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:50.363+0000 7fb963fff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb96410bc10 0x7fb96419c820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:41:50.365 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:50.363+0000 7fb963fff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb96410bc10 0x7fb96419c820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:54022/0 (socket says 172.21.15.52:54022) 2024-03-04T09:41:50.365 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:50.363+0000 7fb963fff640 1 -- 172.21.15.52:0/3664999091 learned_addr learned my addr 172.21.15.52:0/3664999091 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-04T09:41:50.365 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:50.363+0000 7fb9637fe640 1 --2- 172.21.15.52:0/3664999091 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb964102840 0x7fb964197f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:41:50.365 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:50.363+0000 7fb962ffd640 1 --2- 172.21.15.52:0/3664999091 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb964103160 0x7fb9641984a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:41:50.365 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:50.364+0000 7fb963fff640 1 -- 172.21.15.52:0/3664999091 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb964103160 msgr2=0x7fb9641984a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:41:50.366 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:50.364+0000 7fb963fff640 1 --2- 172.21.15.52:0/3664999091 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb964103160 0x7fb9641984a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:41:50.366 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:50.364+0000 7fb963fff640 1 -- 172.21.15.52:0/3664999091 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb964102840 msgr2=0x7fb964197f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:41:50.366 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:50.364+0000 7fb963fff640 1 --2- 172.21.15.52:0/3664999091 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb964102840 0x7fb964197f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:41:50.366 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:50.364+0000 7fb963fff640 1 -- 172.21.15.52:0/3664999091 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb96419cf50 con 0x7fb96410bc10 2024-03-04T09:41:50.366 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:50.364+0000 7fb963fff640 1 --2- 172.21.15.52:0/3664999091 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb96410bc10 0x7fb96419c820 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fb95c00eb10 tx=0x7fb95c00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:41:50.366 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:50.364+0000 7fb960ff9640 1 -- 172.21.15.52:0/3664999091 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb95c004b20 con 0x7fb96410bc10 2024-03-04T09:41:50.366 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:50.364+0000 7fb960ff9640 1 -- 172.21.15.52:0/3664999091 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb95c004cc0 con 0x7fb96410bc10 2024-03-04T09:41:50.366 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:50.364+0000 7fb960ff9640 1 -- 172.21.15.52:0/3664999091 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb95c010760 con 0x7fb96410bc10 2024-03-04T09:41:50.366 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:50.364+0000 7fb969c81640 1 -- 172.21.15.52:0/3664999091 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb96419d240 con 0x7fb96410bc10 2024-03-04T09:41:50.367 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:50.364+0000 7fb969c81640 1 -- 172.21.15.52:0/3664999091 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9641a4ad0 con 0x7fb96410bc10 2024-03-04T09:41:50.367 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:50.365+0000 7fb969c81640 1 -- 172.21.15.52:0/3664999091 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb964068810 con 0x7fb96410bc10 2024-03-04T09:41:50.368 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:50.366+0000 7fb960ff9640 1 -- 172.21.15.52:0/3664999091 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 25) v1 ==== 136887+0+0 (secure 0 0 0) 0x7fb95c005020 con 0x7fb96410bc10 2024-03-04T09:41:50.368 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:50.367+0000 7fb960ff9640 1 --2- 172.21.15.52:0/3664999091 >> [v2:172.21.15.52:6800/478031757,v1:172.21.15.52:6801/478031757] conn(0x7fb94c0a1ef0 0x7fb94c0a43b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:41:50.368 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:50.367+0000 7fb9637fe640 1 --2- 172.21.15.52:0/3664999091 >> [v2:172.21.15.52:6800/478031757,v1:172.21.15.52:6801/478031757] conn(0x7fb94c0a1ef0 0x7fb94c0a43b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:41:50.368 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:50.367+0000 7fb960ff9640 1 -- 172.21.15.52:0/3664999091 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 9547+0+0 (secure 0 0 0) 0x7fb95c0ceb40 con 0x7fb96410bc10 2024-03-04T09:41:50.368 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:50.367+0000 7fb9637fe640 1 --2- 172.21.15.52:0/3664999091 >> [v2:172.21.15.52:6800/478031757,v1:172.21.15.52:6801/478031757] conn(0x7fb94c0a1ef0 0x7fb94c0a43b0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fb95402efd0 tx=0x7fb95402cf90 comp rx=0 tx=0).ready entity=mgr.15486 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:41:50.369 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:50.371+0000 7fb960ff9640 1 -- 172.21.15.52:0/3664999091 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fb95c09b7d0 con 0x7fb96410bc10 2024-03-04T09:41:50.550 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:50.552+0000 7fb969c81640 1 -- 172.21.15.52:0/3664999091 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fb964106d60 con 0x7fb96410bc10 2024-03-04T09:41:50.550 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:50.553+0000 7fb960ff9640 1 -- 172.21.15.52:0/3664999091 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v102) v1 ==== 93+0+10 (secure 0 0 0) 0x7fb95c09b0d0 con 0x7fb96410bc10 2024-03-04T09:41:50.551 INFO:teuthology.orchestra.run.smithi052.stdout:pg_num: 1 2024-03-04T09:41:50.553 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:50.554+0000 7fb969c81640 1 -- 172.21.15.52:0/3664999091 >> [v2:172.21.15.52:6800/478031757,v1:172.21.15.52:6801/478031757] conn(0x7fb94c0a1ef0 msgr2=0x7fb94c0a43b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:41:50.553 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:50.554+0000 7fb969c81640 1 --2- 172.21.15.52:0/3664999091 >> [v2:172.21.15.52:6800/478031757,v1:172.21.15.52:6801/478031757] conn(0x7fb94c0a1ef0 0x7fb94c0a43b0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fb95402efd0 tx=0x7fb95402cf90 comp rx=0 tx=0).stop 2024-03-04T09:41:50.553 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:50.554+0000 7fb969c81640 1 -- 172.21.15.52:0/3664999091 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb96410bc10 msgr2=0x7fb96419c820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:41:50.553 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:50.554+0000 7fb969c81640 1 --2- 172.21.15.52:0/3664999091 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb96410bc10 0x7fb96419c820 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fb95c00eb10 tx=0x7fb95c00c2d0 comp rx=0 tx=0).stop 2024-03-04T09:41:50.553 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:50.555+0000 7fb969c81640 1 -- 172.21.15.52:0/3664999091 shutdown_connections 2024-03-04T09:41:50.553 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:50.555+0000 7fb969c81640 1 --2- 172.21.15.52:0/3664999091 >> [v2:172.21.15.52:6800/478031757,v1:172.21.15.52:6801/478031757] conn(0x7fb94c0a1ef0 0x7fb94c0a43b0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:41:50.554 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:50.555+0000 7fb969c81640 1 --2- 172.21.15.52:0/3664999091 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb96410bc10 0x7fb96419c820 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:41:50.554 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:50.555+0000 7fb969c81640 1 --2- 172.21.15.52:0/3664999091 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb964103160 0x7fb9641984a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:41:50.554 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:50.555+0000 7fb969c81640 1 --2- 172.21.15.52:0/3664999091 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb964102840 0x7fb964197f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:41:50.554 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:50.555+0000 7fb969c81640 1 -- 172.21.15.52:0/3664999091 >> 172.21.15.52:0/3664999091 conn(0x7fb964077630 msgr2=0x7fb9640fd680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:41:50.554 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:50.555+0000 7fb969c81640 1 -- 172.21.15.52:0/3664999091 shutdown_connections 2024-03-04T09:41:50.554 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:50.555+0000 7fb969c81640 1 -- 172.21.15.52:0/3664999091 wait complete. 2024-03-04T09:41:50.563 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-04T09:41:50.587 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:41:50 smithi191 ceph-mon[36816]: pgmap v331: 129 pgs: 129 active+clean; 78 MiB data, 691 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-04T09:41:50.632 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:41:50 smithi106 ceph-mon[36484]: pgmap v331: 129 pgs: 129 active+clean; 78 MiB data, 691 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-04T09:41:50.672 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:50.671+0000 7f0e35732640 1 -- 172.21.15.52:0/3573233700 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0e30107590 msgr2=0x7f0e30107970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:41:50.672 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:50.671+0000 7f0e35732640 1 --2- 172.21.15.52:0/3573233700 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0e30107590 0x7f0e30107970 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f0e20009cd0 tx=0x7f0e2002c0f0 comp rx=0 tx=0).stop 2024-03-04T09:41:50.672 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:50.671+0000 7f0e35732640 1 -- 172.21.15.52:0/3573233700 shutdown_connections 2024-03-04T09:41:50.672 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:50.671+0000 7f0e35732640 1 --2- 172.21.15.52:0/3573233700 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0e30100470 0x7f0e3010e450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:41:50.672 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:50.671+0000 7f0e35732640 1 --2- 172.21.15.52:0/3573233700 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0e300ffad0 0x7f0e300fff30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:41:50.673 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:50.671+0000 7f0e35732640 1 --2- 172.21.15.52:0/3573233700 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0e30107590 0x7f0e30107970 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:41:50.673 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:50.671+0000 7f0e35732640 1 -- 172.21.15.52:0/3573233700 >> 172.21.15.52:0/3573233700 conn(0x7f0e30076a90 msgr2=0x7f0e300fdde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:41:50.673 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:50.671+0000 7f0e35732640 1 -- 172.21.15.52:0/3573233700 shutdown_connections 2024-03-04T09:41:50.673 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:50.671+0000 7f0e35732640 1 -- 172.21.15.52:0/3573233700 wait complete. 2024-03-04T09:41:50.673 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:50.672+0000 7f0e35732640 1 Processor -- start 2024-03-04T09:41:50.673 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:50.672+0000 7f0e35732640 1 -- start start 2024-03-04T09:41:50.674 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:50.672+0000 7f0e35732640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0e300ffad0 0x7f0e30100cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:41:50.674 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:50.672+0000 7f0e35732640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0e30100470 0x7f0e301011f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:41:50.674 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:50.672+0000 7f0e35732640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0e30107590 0x7f0e30101730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:41:50.674 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:50.672+0000 7f0e35732640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e301119d0 con 0x7f0e300ffad0 2024-03-04T09:41:50.674 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:50.672+0000 7f0e35732640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e30111850 con 0x7f0e30100470 2024-03-04T09:41:50.675 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:50.672+0000 7f0e35732640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e30111b50 con 0x7f0e30107590 2024-03-04T09:41:50.675 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:50.672+0000 7f0e2effd640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0e300ffad0 0x7f0e30100cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:41:50.675 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:50.672+0000 7f0e2effd640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0e300ffad0 0x7f0e30100cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:54038/0 (socket says 172.21.15.52:54038) 2024-03-04T09:41:50.675 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:50.672+0000 7f0e2effd640 1 -- 172.21.15.52:0/3964243874 learned_addr learned my addr 172.21.15.52:0/3964243874 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-04T09:41:50.675 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:50.672+0000 7f0e2e7fc640 1 --2- 172.21.15.52:0/3964243874 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0e30100470 0x7f0e301011f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:41:50.675 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:50.672+0000 7f0e2f7fe640 1 --2- 172.21.15.52:0/3964243874 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0e30107590 0x7f0e30101730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:41:50.676 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:50.672+0000 7f0e2effd640 1 -- 172.21.15.52:0/3964243874 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0e30107590 msgr2=0x7f0e30101730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:41:50.676 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:50.672+0000 7f0e2effd640 1 --2- 172.21.15.52:0/3964243874 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0e30107590 0x7f0e30101730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:41:50.676 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:50.672+0000 7f0e2effd640 1 -- 172.21.15.52:0/3964243874 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0e30100470 msgr2=0x7f0e301011f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:41:50.676 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:50.672+0000 7f0e2effd640 1 --2- 172.21.15.52:0/3964243874 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0e30100470 0x7f0e301011f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:41:50.676 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:50.672+0000 7f0e2effd640 1 -- 172.21.15.52:0/3964243874 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e301a6370 con 0x7f0e300ffad0 2024-03-04T09:41:50.676 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:50.673+0000 7f0e2effd640 1 --2- 172.21.15.52:0/3964243874 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0e300ffad0 0x7f0e30100cb0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f0e20004af0 tx=0x7f0e20032ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:41:50.676 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:50.673+0000 7f0e27fff640 1 -- 172.21.15.52:0/3964243874 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e20041020 con 0x7f0e300ffad0 2024-03-04T09:41:50.677 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:50.673+0000 7f0e27fff640 1 -- 172.21.15.52:0/3964243874 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0e20034cd0 con 0x7f0e300ffad0 2024-03-04T09:41:50.677 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:50.673+0000 7f0e35732640 1 -- 172.21.15.52:0/3964243874 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e301a6600 con 0x7f0e300ffad0 2024-03-04T09:41:50.677 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:50.673+0000 7f0e27fff640 1 -- 172.21.15.52:0/3964243874 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e20039040 con 0x7f0e300ffad0 2024-03-04T09:41:50.677 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:50.673+0000 7f0e35732640 1 -- 172.21.15.52:0/3964243874 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e301a6af0 con 0x7f0e300ffad0 2024-03-04T09:41:50.677 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:50.674+0000 7f0e35732640 1 -- 172.21.15.52:0/3964243874 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e30068810 con 0x7f0e300ffad0 2024-03-04T09:41:50.678 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:50.675+0000 7f0e27fff640 1 -- 172.21.15.52:0/3964243874 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 25) v1 ==== 136887+0+0 (secure 0 0 0) 0x7f0e20044020 con 0x7f0e300ffad0 2024-03-04T09:41:50.678 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:50.676+0000 7f0e27fff640 1 --2- 172.21.15.52:0/3964243874 >> [v2:172.21.15.52:6800/478031757,v1:172.21.15.52:6801/478031757] conn(0x7f0e1c0a1f20 0x7f0e1c0a43e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:41:50.679 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:50.676+0000 7f0e2e7fc640 1 --2- 172.21.15.52:0/3964243874 >> [v2:172.21.15.52:6800/478031757,v1:172.21.15.52:6801/478031757] conn(0x7f0e1c0a1f20 0x7f0e1c0a43e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:41:50.679 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:50.676+0000 7f0e27fff640 1 -- 172.21.15.52:0/3964243874 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 9547+0+0 (secure 0 0 0) 0x7f0e200f03b0 con 0x7f0e300ffad0 2024-03-04T09:41:50.679 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:50.676+0000 7f0e2e7fc640 1 --2- 172.21.15.52:0/3964243874 >> [v2:172.21.15.52:6800/478031757,v1:172.21.15.52:6801/478031757] conn(0x7f0e1c0a1f20 0x7f0e1c0a43e0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f0e301026f0 tx=0x7f0e18005f90 comp rx=0 tx=0).ready entity=mgr.15486 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:41:50.679 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:50.680+0000 7f0e27fff640 1 -- 172.21.15.52:0/3964243874 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f0e2004b020 con 0x7f0e300ffad0 2024-03-04T09:41:50.858 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:50.861+0000 7f0e35732640 1 -- 172.21.15.52:0/3964243874 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f0e30109b10 con 0x7f0e300ffad0 2024-03-04T09:41:50.859 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:50.861+0000 7f0e27fff640 1 -- 172.21.15.52:0/3964243874 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v102) v1 ==== 104+0+11 (secure 0 0 0) 0x7f0e200bd1b0 con 0x7f0e300ffad0 2024-03-04T09:41:50.859 INFO:teuthology.orchestra.run.smithi052.stdout:pg_num: 32 2024-03-04T09:41:50.861 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:50.863+0000 7f0e35732640 1 -- 172.21.15.52:0/3964243874 >> [v2:172.21.15.52:6800/478031757,v1:172.21.15.52:6801/478031757] conn(0x7f0e1c0a1f20 msgr2=0x7f0e1c0a43e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:41:50.861 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:50.863+0000 7f0e35732640 1 --2- 172.21.15.52:0/3964243874 >> [v2:172.21.15.52:6800/478031757,v1:172.21.15.52:6801/478031757] conn(0x7f0e1c0a1f20 0x7f0e1c0a43e0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f0e301026f0 tx=0x7f0e18005f90 comp rx=0 tx=0).stop 2024-03-04T09:41:50.861 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:50.863+0000 7f0e35732640 1 -- 172.21.15.52:0/3964243874 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0e300ffad0 msgr2=0x7f0e30100cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:41:50.862 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:50.863+0000 7f0e35732640 1 --2- 172.21.15.52:0/3964243874 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0e300ffad0 0x7f0e30100cb0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f0e20004af0 tx=0x7f0e20032ea0 comp rx=0 tx=0).stop 2024-03-04T09:41:50.862 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:50.863+0000 7f0e35732640 1 -- 172.21.15.52:0/3964243874 shutdown_connections 2024-03-04T09:41:50.862 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:50.863+0000 7f0e35732640 1 --2- 172.21.15.52:0/3964243874 >> [v2:172.21.15.52:6800/478031757,v1:172.21.15.52:6801/478031757] conn(0x7f0e1c0a1f20 0x7f0e1c0a43e0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:41:50.862 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:50.863+0000 7f0e35732640 1 --2- 172.21.15.52:0/3964243874 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0e30107590 0x7f0e30101730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:41:50.862 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:50.863+0000 7f0e35732640 1 --2- 172.21.15.52:0/3964243874 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0e30100470 0x7f0e301011f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:41:50.862 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:50.863+0000 7f0e35732640 1 --2- 172.21.15.52:0/3964243874 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0e300ffad0 0x7f0e30100cb0 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:41:50.862 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:50.863+0000 7f0e35732640 1 -- 172.21.15.52:0/3964243874 >> 172.21.15.52:0/3964243874 conn(0x7f0e30076a90 msgr2=0x7f0e300fe1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:41:50.862 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:50.863+0000 7f0e35732640 1 -- 172.21.15.52:0/3964243874 shutdown_connections 2024-03-04T09:41:50.863 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:50.863+0000 7f0e35732640 1 -- 172.21.15.52:0/3964243874 wait complete. 2024-03-04T09:41:50.872 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-04T09:41:50.980 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:50.980+0000 7fb559cc2640 1 -- 172.21.15.52:0/2527218960 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb5541065d0 msgr2=0x7fb5541069b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:41:50.980 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:50.980+0000 7fb559cc2640 1 --2- 172.21.15.52:0/2527218960 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb5541065d0 0x7fb5541069b0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fb544009900 tx=0x7fb54402bc30 comp rx=0 tx=0).stop 2024-03-04T09:41:50.981 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:50.980+0000 7fb559cc2640 1 -- 172.21.15.52:0/2527218960 shutdown_connections 2024-03-04T09:41:50.981 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:50.980+0000 7fb559cc2640 1 --2- 172.21.15.52:0/2527218960 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb554100ed0 0x7fb55410d390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:41:50.981 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:50.980+0000 7fb559cc2640 1 --2- 172.21.15.52:0/2527218960 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb554100510 0x7fb554100990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:41:50.981 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:50.980+0000 7fb559cc2640 1 --2- 172.21.15.52:0/2527218960 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb5541065d0 0x7fb5541069b0 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:41:50.981 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:50.980+0000 7fb559cc2640 1 -- 172.21.15.52:0/2527218960 >> 172.21.15.52:0/2527218960 conn(0x7fb554077630 msgr2=0x7fb554077a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:41:50.981 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:50.980+0000 7fb559cc2640 1 -- 172.21.15.52:0/2527218960 shutdown_connections 2024-03-04T09:41:50.981 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:50.980+0000 7fb559cc2640 1 -- 172.21.15.52:0/2527218960 wait complete. 2024-03-04T09:41:50.982 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:50.980+0000 7fb559cc2640 1 Processor -- start 2024-03-04T09:41:50.982 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:50.980+0000 7fb559cc2640 1 -- start start 2024-03-04T09:41:50.982 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:50.981+0000 7fb559cc2640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb554100510 0x7fb554196d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:41:50.982 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:50.981+0000 7fb559cc2640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb554100ed0 0x7fb554197290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:41:50.982 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:50.981+0000 7fb559cc2640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb5541065d0 0x7fb55419b610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:41:50.982 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:50.981+0000 7fb559cc2640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb554110890 con 0x7fb5541065d0 2024-03-04T09:41:50.982 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:50.981+0000 7fb559cc2640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb554110710 con 0x7fb554100ed0 2024-03-04T09:41:50.982 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:50.981+0000 7fb559cc2640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb554110a10 con 0x7fb554100510 2024-03-04T09:41:50.983 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:50.981+0000 7fb553fff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb5541065d0 0x7fb55419b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:41:50.983 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:50.981+0000 7fb553fff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb5541065d0 0x7fb55419b610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:54062/0 (socket says 172.21.15.52:54062) 2024-03-04T09:41:50.983 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:50.981+0000 7fb553fff640 1 -- 172.21.15.52:0/2087662425 learned_addr learned my addr 172.21.15.52:0/2087662425 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-04T09:41:50.983 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:50.981+0000 7fb552ffd640 1 --2- 172.21.15.52:0/2087662425 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb554100ed0 0x7fb554197290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:41:50.983 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:50.981+0000 7fb5537fe640 1 --2- 172.21.15.52:0/2087662425 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb554100510 0x7fb554196d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:41:50.983 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:50.981+0000 7fb553fff640 1 -- 172.21.15.52:0/2087662425 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb554100510 msgr2=0x7fb554196d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:41:50.983 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:50.981+0000 7fb553fff640 1 --2- 172.21.15.52:0/2087662425 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb554100510 0x7fb554196d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:41:50.983 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:50.981+0000 7fb553fff640 1 -- 172.21.15.52:0/2087662425 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb554100ed0 msgr2=0x7fb554197290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:41:50.984 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:50.981+0000 7fb553fff640 1 --2- 172.21.15.52:0/2087662425 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb554100ed0 0x7fb554197290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:41:50.984 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:50.981+0000 7fb553fff640 1 -- 172.21.15.52:0/2087662425 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb55419bd40 con 0x7fb5541065d0 2024-03-04T09:41:50.984 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:50.981+0000 7fb553fff640 1 --2- 172.21.15.52:0/2087662425 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb5541065d0 0x7fb55419b610 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fb54c00eb10 tx=0x7fb54c00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:41:50.984 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:50.981+0000 7fb550ff9640 1 -- 172.21.15.52:0/2087662425 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb54c004b20 con 0x7fb5541065d0 2024-03-04T09:41:50.984 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:50.981+0000 7fb550ff9640 1 -- 172.21.15.52:0/2087662425 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb54c004cc0 con 0x7fb5541065d0 2024-03-04T09:41:50.984 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:50.981+0000 7fb550ff9640 1 -- 172.21.15.52:0/2087662425 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb54c010760 con 0x7fb5541065d0 2024-03-04T09:41:50.984 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:50.981+0000 7fb559cc2640 1 -- 172.21.15.52:0/2087662425 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb55419c030 con 0x7fb5541065d0 2024-03-04T09:41:50.984 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:50.981+0000 7fb559cc2640 1 -- 172.21.15.52:0/2087662425 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5541a38c0 con 0x7fb5541065d0 2024-03-04T09:41:50.984 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:50.982+0000 7fb559cc2640 1 -- 172.21.15.52:0/2087662425 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb554068810 con 0x7fb5541065d0 2024-03-04T09:41:50.985 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:50.983+0000 7fb550ff9640 1 -- 172.21.15.52:0/2087662425 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 25) v1 ==== 136887+0+0 (secure 0 0 0) 0x7fb54c005020 con 0x7fb5541065d0 2024-03-04T09:41:50.985 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:50.984+0000 7fb550ff9640 1 --2- 172.21.15.52:0/2087662425 >> [v2:172.21.15.52:6800/478031757,v1:172.21.15.52:6801/478031757] conn(0x7fb53c0a1ef0 0x7fb53c0a43b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:41:50.986 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:50.984+0000 7fb5537fe640 1 --2- 172.21.15.52:0/2087662425 >> [v2:172.21.15.52:6800/478031757,v1:172.21.15.52:6801/478031757] conn(0x7fb53c0a1ef0 0x7fb53c0a43b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:41:50.986 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:50.984+0000 7fb550ff9640 1 -- 172.21.15.52:0/2087662425 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 9547+0+0 (secure 0 0 0) 0x7fb54c0ceae0 con 0x7fb5541065d0 2024-03-04T09:41:50.986 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:50.984+0000 7fb5537fe640 1 --2- 172.21.15.52:0/2087662425 >> [v2:172.21.15.52:6800/478031757,v1:172.21.15.52:6801/478031757] conn(0x7fb53c0a1ef0 0x7fb53c0a43b0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fb544009230 tx=0x7fb544005fd0 comp rx=0 tx=0).ready entity=mgr.15486 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:41:50.986 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:50.988+0000 7fb550ff9640 1 -- 172.21.15.52:0/2087662425 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fb54c09b7d0 con 0x7fb5541065d0 2024-03-04T09:41:51.167 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:51.169+0000 7fb559cc2640 1 -- 172.21.15.52:0/2087662425 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fb554108a20 con 0x7fb5541065d0 2024-03-04T09:41:51.168 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:51.170+0000 7fb550ff9640 1 -- 172.21.15.52:0/2087662425 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v102) v1 ==== 100+0+11 (secure 0 0 0) 0x7fb54c09b0d0 con 0x7fb5541065d0 2024-03-04T09:41:51.169 INFO:teuthology.orchestra.run.smithi052.stdout:pg_num: 64 2024-03-04T09:41:51.171 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:51.172+0000 7fb559cc2640 1 -- 172.21.15.52:0/2087662425 >> [v2:172.21.15.52:6800/478031757,v1:172.21.15.52:6801/478031757] conn(0x7fb53c0a1ef0 msgr2=0x7fb53c0a43b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:41:51.171 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:51.172+0000 7fb559cc2640 1 --2- 172.21.15.52:0/2087662425 >> [v2:172.21.15.52:6800/478031757,v1:172.21.15.52:6801/478031757] conn(0x7fb53c0a1ef0 0x7fb53c0a43b0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fb544009230 tx=0x7fb544005fd0 comp rx=0 tx=0).stop 2024-03-04T09:41:51.171 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:51.172+0000 7fb559cc2640 1 -- 172.21.15.52:0/2087662425 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb5541065d0 msgr2=0x7fb55419b610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:41:51.171 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:51.172+0000 7fb559cc2640 1 --2- 172.21.15.52:0/2087662425 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb5541065d0 0x7fb55419b610 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fb54c00eb10 tx=0x7fb54c00c2d0 comp rx=0 tx=0).stop 2024-03-04T09:41:51.171 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:51.172+0000 7fb559cc2640 1 -- 172.21.15.52:0/2087662425 shutdown_connections 2024-03-04T09:41:51.171 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:51.172+0000 7fb559cc2640 1 --2- 172.21.15.52:0/2087662425 >> [v2:172.21.15.52:6800/478031757,v1:172.21.15.52:6801/478031757] conn(0x7fb53c0a1ef0 0x7fb53c0a43b0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:41:51.172 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:51.172+0000 7fb559cc2640 1 --2- 172.21.15.52:0/2087662425 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb5541065d0 0x7fb55419b610 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:41:51.172 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:51.172+0000 7fb559cc2640 1 --2- 172.21.15.52:0/2087662425 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb554100ed0 0x7fb554197290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:41:51.172 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:51.172+0000 7fb559cc2640 1 --2- 172.21.15.52:0/2087662425 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb554100510 0x7fb554196d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:41:51.172 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:51.172+0000 7fb559cc2640 1 -- 172.21.15.52:0/2087662425 >> 172.21.15.52:0/2087662425 conn(0x7fb554077630 msgr2=0x7fb5540ff060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:41:51.172 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:51.173+0000 7fb559cc2640 1 -- 172.21.15.52:0/2087662425 shutdown_connections 2024-03-04T09:41:51.172 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:51.173+0000 7fb559cc2640 1 -- 172.21.15.52:0/2087662425 wait complete. 2024-03-04T09:41:51.181 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-04T09:41:51.289 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:51.289+0000 7f1425dc9640 1 -- 172.21.15.52:0/1037130542 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1420107640 msgr2=0x7f1420107a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:41:51.289 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:51.289+0000 7f1425dc9640 1 --2- 172.21.15.52:0/1037130542 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1420107640 0x7f1420107a20 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f1410009930 tx=0x7f141002bc30 comp rx=0 tx=0).stop 2024-03-04T09:41:51.290 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:51.289+0000 7f1425dc9640 1 -- 172.21.15.52:0/1037130542 shutdown_connections 2024-03-04T09:41:51.290 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:51.289+0000 7f1425dc9640 1 --2- 172.21.15.52:0/1037130542 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1420101f40 0x7f142010e400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:41:51.290 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:51.289+0000 7f1425dc9640 1 --2- 172.21.15.52:0/1037130542 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1420101580 0x7f1420101a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:41:51.290 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:51.289+0000 7f1425dc9640 1 --2- 172.21.15.52:0/1037130542 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1420107640 0x7f1420107a20 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:41:51.290 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:51.289+0000 7f1425dc9640 1 -- 172.21.15.52:0/1037130542 >> 172.21.15.52:0/1037130542 conn(0x7f14200789d0 msgr2=0x7f1420078de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:41:51.290 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:51.289+0000 7f1425dc9640 1 -- 172.21.15.52:0/1037130542 shutdown_connections 2024-03-04T09:41:51.290 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:51.289+0000 7f1425dc9640 1 -- 172.21.15.52:0/1037130542 wait complete. 2024-03-04T09:41:51.291 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:51.290+0000 7f1425dc9640 1 Processor -- start 2024-03-04T09:41:51.291 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:51.290+0000 7f1425dc9640 1 -- start start 2024-03-04T09:41:51.291 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:51.290+0000 7f1425dc9640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1420101580 0x7f1420197e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:41:51.291 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:51.290+0000 7f1425dc9640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1420101f40 0x7f14201983c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:41:51.291 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:51.290+0000 7f1425dc9640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1420107640 0x7f142019c740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:41:51.291 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:51.290+0000 7f1425dc9640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1420111900 con 0x7f1420101f40 2024-03-04T09:41:51.291 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:51.290+0000 7f1425dc9640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1420111780 con 0x7f1420101580 2024-03-04T09:41:51.292 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:51.290+0000 7f1425dc9640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1420111a80 con 0x7f1420107640 2024-03-04T09:41:51.292 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:51.290+0000 7f141effd640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1420101f40 0x7f14201983c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:41:51.292 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:51.290+0000 7f141f7fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1420101580 0x7f1420197e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:41:51.292 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:51.290+0000 7f141effd640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1420101f40 0x7f14201983c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:54084/0 (socket says 172.21.15.52:54084) 2024-03-04T09:41:51.292 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:51.290+0000 7f141f7fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1420101580 0x7f1420197e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.52:51796/0 (socket says 172.21.15.52:51796) 2024-03-04T09:41:51.292 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:51.290+0000 7f141effd640 1 -- 172.21.15.52:0/1696869035 learned_addr learned my addr 172.21.15.52:0/1696869035 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-04T09:41:51.292 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:51.290+0000 7f141ffff640 1 --2- 172.21.15.52:0/1696869035 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1420107640 0x7f142019c740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:41:51.293 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:51.290+0000 7f141effd640 1 -- 172.21.15.52:0/1696869035 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1420107640 msgr2=0x7f142019c740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:41:51.293 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:51.290+0000 7f141effd640 1 --2- 172.21.15.52:0/1696869035 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1420107640 0x7f142019c740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:41:51.293 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:51.290+0000 7f141effd640 1 -- 172.21.15.52:0/1696869035 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1420101580 msgr2=0x7f1420197e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:41:51.293 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:51.290+0000 7f141effd640 1 --2- 172.21.15.52:0/1696869035 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1420101580 0x7f1420197e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:41:51.293 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:51.290+0000 7f141effd640 1 -- 172.21.15.52:0/1696869035 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f142019ce70 con 0x7f1420101f40 2024-03-04T09:41:51.293 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:51.291+0000 7f141effd640 1 --2- 172.21.15.52:0/1696869035 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1420101f40 0x7f14201983c0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f141400ea60 tx=0x7f141400ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:41:51.293 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:51.291+0000 7f141cff9640 1 -- 172.21.15.52:0/1696869035 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f141400cde0 con 0x7f1420101f40 2024-03-04T09:41:51.293 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:51.291+0000 7f1425dc9640 1 -- 172.21.15.52:0/1696869035 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f142019d160 con 0x7f1420101f40 2024-03-04T09:41:51.294 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:51.291+0000 7f141cff9640 1 -- 172.21.15.52:0/1696869035 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f14140035f0 con 0x7f1420101f40 2024-03-04T09:41:51.294 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:51.291+0000 7f141cff9640 1 -- 172.21.15.52:0/1696869035 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1414010670 con 0x7f1420101f40 2024-03-04T09:41:51.294 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:51.291+0000 7f1425dc9640 1 -- 172.21.15.52:0/1696869035 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14201a49f0 con 0x7f1420101f40 2024-03-04T09:41:51.295 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:51.292+0000 7f1425dc9640 1 -- 172.21.15.52:0/1696869035 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1420068810 con 0x7f1420101f40 2024-03-04T09:41:51.295 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:51.293+0000 7f141cff9640 1 -- 172.21.15.52:0/1696869035 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 25) v1 ==== 136887+0+0 (secure 0 0 0) 0x7f14140037a0 con 0x7f1420101f40 2024-03-04T09:41:51.295 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:51.293+0000 7f141cff9640 1 --2- 172.21.15.52:0/1696869035 >> [v2:172.21.15.52:6800/478031757,v1:172.21.15.52:6801/478031757] conn(0x7f14080a1ef0 0x7f14080a43b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:41:51.296 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:51.294+0000 7f141f7fe640 1 --2- 172.21.15.52:0/1696869035 >> [v2:172.21.15.52:6800/478031757,v1:172.21.15.52:6801/478031757] conn(0x7f14080a1ef0 0x7f14080a43b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:41:51.296 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:51.294+0000 7f141cff9640 1 -- 172.21.15.52:0/1696869035 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 9547+0+0 (secure 0 0 0) 0x7f14140ce9d0 con 0x7f1420101f40 2024-03-04T09:41:51.296 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:51.294+0000 7f141f7fe640 1 --2- 172.21.15.52:0/1696869035 >> [v2:172.21.15.52:6800/478031757,v1:172.21.15.52:6801/478031757] conn(0x7f14080a1ef0 0x7f14080a43b0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f1410005ab0 tx=0x7f1410005fd0 comp rx=0 tx=0).ready entity=mgr.15486 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:41:51.296 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:51.297+0000 7f141cff9640 1 -- 172.21.15.52:0/1696869035 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f141409b6f0 con 0x7f1420101f40 2024-03-04T09:41:51.476 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:51.478+0000 7f1425dc9640 1 -- 172.21.15.52:0/1696869035 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f14201094a0 con 0x7f1420101f40 2024-03-04T09:41:51.477 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:51.479+0000 7f141cff9640 1 -- 172.21.15.52:0/1696869035 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v102) v1 ==== 103+0+11 (secure 0 0 0) 0x7f141409aff0 con 0x7f1420101f40 2024-03-04T09:41:51.477 INFO:teuthology.orchestra.run.smithi052.stdout:pg_num: 32 2024-03-04T09:41:51.479 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:51.481+0000 7f1425dc9640 1 -- 172.21.15.52:0/1696869035 >> [v2:172.21.15.52:6800/478031757,v1:172.21.15.52:6801/478031757] conn(0x7f14080a1ef0 msgr2=0x7f14080a43b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:41:51.479 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:51.481+0000 7f1425dc9640 1 --2- 172.21.15.52:0/1696869035 >> [v2:172.21.15.52:6800/478031757,v1:172.21.15.52:6801/478031757] conn(0x7f14080a1ef0 0x7f14080a43b0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f1410005ab0 tx=0x7f1410005fd0 comp rx=0 tx=0).stop 2024-03-04T09:41:51.479 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:51.481+0000 7f1425dc9640 1 -- 172.21.15.52:0/1696869035 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1420101f40 msgr2=0x7f14201983c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:41:51.479 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:51.481+0000 7f1425dc9640 1 --2- 172.21.15.52:0/1696869035 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1420101f40 0x7f14201983c0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f141400ea60 tx=0x7f141400ef20 comp rx=0 tx=0).stop 2024-03-04T09:41:51.480 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:51.481+0000 7f1425dc9640 1 -- 172.21.15.52:0/1696869035 shutdown_connections 2024-03-04T09:41:51.480 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:51.481+0000 7f1425dc9640 1 --2- 172.21.15.52:0/1696869035 >> [v2:172.21.15.52:6800/478031757,v1:172.21.15.52:6801/478031757] conn(0x7f14080a1ef0 0x7f14080a43b0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:41:51.480 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:51.481+0000 7f1425dc9640 1 --2- 172.21.15.52:0/1696869035 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1420107640 0x7f142019c740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:41:51.480 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:51.481+0000 7f1425dc9640 1 --2- 172.21.15.52:0/1696869035 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1420101f40 0x7f14201983c0 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:41:51.480 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:51.481+0000 7f1425dc9640 1 --2- 172.21.15.52:0/1696869035 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1420101580 0x7f1420197e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:41:51.480 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:51.481+0000 7f1425dc9640 1 -- 172.21.15.52:0/1696869035 >> 172.21.15.52:0/1696869035 conn(0x7f14200789d0 msgr2=0x7f14201001e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:41:51.480 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:51.481+0000 7f1425dc9640 1 -- 172.21.15.52:0/1696869035 shutdown_connections 2024-03-04T09:41:51.481 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:51.481+0000 7f1425dc9640 1 -- 172.21.15.52:0/1696869035 wait complete. 2024-03-04T09:41:51.488 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:41:51 smithi052 ceph-mon[32473]: from='client.? 172.21.15.52:0/3396110437' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-04T09:41:51.489 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:41:51 smithi052 ceph-mon[32473]: from='client.? 172.21.15.52:0/3664999091' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-04T09:41:51.489 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:41:51 smithi052 ceph-mon[32473]: from='client.? 172.21.15.52:0/3964243874' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-04T09:41:51.490 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-04T09:41:51.587 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:41:51 smithi191 ceph-mon[36816]: from='client.? 172.21.15.52:0/3396110437' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-04T09:41:51.587 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:41:51 smithi191 ceph-mon[36816]: from='client.? 172.21.15.52:0/3664999091' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-04T09:41:51.587 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:41:51 smithi191 ceph-mon[36816]: from='client.? 172.21.15.52:0/3964243874' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-04T09:41:51.595 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:51.595+0000 7f7a5fde9640 1 -- 172.21.15.52:0/1600500477 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7a581064e0 msgr2=0x7f7a581068c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:41:51.595 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:51.595+0000 7f7a5fde9640 1 --2- 172.21.15.52:0/1600500477 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7a581064e0 0x7f7a581068c0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f7a54009860 tx=0x7f7a5402bb70 comp rx=0 tx=0).stop 2024-03-04T09:41:51.596 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:51.596+0000 7f7a5fde9640 1 -- 172.21.15.52:0/1600500477 shutdown_connections 2024-03-04T09:41:51.596 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:51.596+0000 7f7a5fde9640 1 --2- 172.21.15.52:0/1600500477 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7a58100e00 0x7f7a5810d310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:41:51.596 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:51.596+0000 7f7a5fde9640 1 --2- 172.21.15.52:0/1600500477 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7a58100440 0x7f7a581008c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:41:51.596 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:51.596+0000 7f7a5fde9640 1 --2- 172.21.15.52:0/1600500477 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7a581064e0 0x7f7a581068c0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:41:51.596 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:51.596+0000 7f7a5fde9640 1 -- 172.21.15.52:0/1600500477 >> 172.21.15.52:0/1600500477 conn(0x7f7a580775d0 msgr2=0x7f7a580779e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:41:51.596 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:51.596+0000 7f7a5fde9640 1 -- 172.21.15.52:0/1600500477 shutdown_connections 2024-03-04T09:41:51.596 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:51.596+0000 7f7a5fde9640 1 -- 172.21.15.52:0/1600500477 wait complete. 2024-03-04T09:41:51.597 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:51.596+0000 7f7a5fde9640 1 Processor -- start 2024-03-04T09:41:51.597 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:51.596+0000 7f7a5fde9640 1 -- start start 2024-03-04T09:41:51.597 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:51.596+0000 7f7a5fde9640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7a58100440 0x7f7a58198ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:41:51.597 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:51.596+0000 7f7a5fde9640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7a58100e00 0x7f7a58199220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:41:51.597 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:51.596+0000 7f7a5fde9640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7a581064e0 0x7f7a581a3d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:41:51.597 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:51.596+0000 7f7a5fde9640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a581107e0 con 0x7f7a581064e0 2024-03-04T09:41:51.597 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:51.596+0000 7f7a5fde9640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a58110660 con 0x7f7a58100e00 2024-03-04T09:41:51.597 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:51.596+0000 7f7a5fde9640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a58110960 con 0x7f7a58100440 2024-03-04T09:41:51.597 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:51.596+0000 7f7a5e386640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7a581064e0 0x7f7a581a3d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:41:51.598 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:51.596+0000 7f7a5e386640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7a581064e0 0x7f7a581a3d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:54106/0 (socket says 172.21.15.52:54106) 2024-03-04T09:41:51.598 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:51.597+0000 7f7a5e386640 1 -- 172.21.15.52:0/1165256000 learned_addr learned my addr 172.21.15.52:0/1165256000 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-04T09:41:51.598 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:51.597+0000 7f7a5db85640 1 --2- 172.21.15.52:0/1165256000 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7a58100440 0x7f7a58198ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:41:51.598 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:51.597+0000 7f7a5d384640 1 --2- 172.21.15.52:0/1165256000 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7a58100e00 0x7f7a58199220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:41:51.598 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:51.597+0000 7f7a5e386640 1 -- 172.21.15.52:0/1165256000 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7a58100440 msgr2=0x7f7a58198ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:41:51.598 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:51.597+0000 7f7a5e386640 1 --2- 172.21.15.52:0/1165256000 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7a58100440 0x7f7a58198ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:41:51.598 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:51.597+0000 7f7a5e386640 1 -- 172.21.15.52:0/1165256000 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7a58100e00 msgr2=0x7f7a58199220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:41:51.598 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:51.597+0000 7f7a5e386640 1 --2- 172.21.15.52:0/1165256000 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7a58100e00 0x7f7a58199220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:41:51.599 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:51.597+0000 7f7a5e386640 1 -- 172.21.15.52:0/1165256000 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a581a42a0 con 0x7f7a581064e0 2024-03-04T09:41:51.599 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:51.597+0000 7f7a5e386640 1 --2- 172.21.15.52:0/1165256000 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7a581064e0 0x7f7a581a3d60 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f7a5000eb10 tx=0x7f7a5000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:41:51.599 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:51.597+0000 7f7a4effd640 1 -- 172.21.15.52:0/1165256000 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a50004b20 con 0x7f7a581064e0 2024-03-04T09:41:51.599 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:51.597+0000 7f7a5fde9640 1 -- 172.21.15.52:0/1165256000 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a581a4470 con 0x7f7a581064e0 2024-03-04T09:41:51.599 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:51.597+0000 7f7a4effd640 1 -- 172.21.15.52:0/1165256000 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7a50004cc0 con 0x7f7a581064e0 2024-03-04T09:41:51.599 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:51.597+0000 7f7a4effd640 1 -- 172.21.15.52:0/1165256000 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a50010760 con 0x7f7a581064e0 2024-03-04T09:41:51.599 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:51.597+0000 7f7a5fde9640 1 -- 172.21.15.52:0/1165256000 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a581a4930 con 0x7f7a581064e0 2024-03-04T09:41:51.600 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:51.598+0000 7f7a5fde9640 1 -- 172.21.15.52:0/1165256000 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a580683c0 con 0x7f7a581064e0 2024-03-04T09:41:51.600 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:51.599+0000 7f7a4effd640 1 -- 172.21.15.52:0/1165256000 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 25) v1 ==== 136887+0+0 (secure 0 0 0) 0x7f7a50005000 con 0x7f7a581064e0 2024-03-04T09:41:51.601 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:51.600+0000 7f7a4effd640 1 --2- 172.21.15.52:0/1165256000 >> [v2:172.21.15.52:6800/478031757,v1:172.21.15.52:6801/478031757] conn(0x7f7a440a1ef0 0x7f7a440a43b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:41:51.601 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:51.600+0000 7f7a5db85640 1 --2- 172.21.15.52:0/1165256000 >> [v2:172.21.15.52:6800/478031757,v1:172.21.15.52:6801/478031757] conn(0x7f7a440a1ef0 0x7f7a440a43b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:41:51.601 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:51.600+0000 7f7a4effd640 1 -- 172.21.15.52:0/1165256000 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 9547+0+0 (secure 0 0 0) 0x7f7a500ced60 con 0x7f7a581064e0 2024-03-04T09:41:51.601 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:51.600+0000 7f7a5db85640 1 --2- 172.21.15.52:0/1165256000 >> [v2:172.21.15.52:6800/478031757,v1:172.21.15.52:6801/478031757] conn(0x7f7a440a1ef0 0x7f7a440a43b0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f7a5402c180 tx=0x7f7a540056d0 comp rx=0 tx=0).ready entity=mgr.15486 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:41:51.601 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:51.603+0000 7f7a4effd640 1 -- 172.21.15.52:0/1165256000 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f7a50053280 con 0x7f7a581064e0 2024-03-04T09:41:51.632 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:41:51 smithi106 ceph-mon[36484]: from='client.? 172.21.15.52:0/3396110437' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-04T09:41:51.632 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:41:51 smithi106 ceph-mon[36484]: from='client.? 172.21.15.52:0/3664999091' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-04T09:41:51.632 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:41:51 smithi106 ceph-mon[36484]: from='client.? 172.21.15.52:0/3964243874' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-04T09:41:51.794 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:51.796+0000 7f7a5fde9640 1 -- 172.21.15.52:0/1165256000 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f7a581089b0 con 0x7f7a581064e0 2024-03-04T09:41:51.794 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:51.796+0000 7f7a4effd640 1 -- 172.21.15.52:0/1165256000 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 17 entries v102) v1 ==== 81+0+999 (secure 0 0 0) 0x7f7a5009b9f0 con 0x7f7a581064e0 2024-03-04T09:41:51.795 INFO:teuthology.orchestra.run.smithi052.stdout:172.21.15.52:0/1035878034 2024-03-05T09:21:12.760938+0000 2024-03-04T09:41:51.795 INFO:teuthology.orchestra.run.smithi052.stdout:172.21.15.52:0/2743084243 2024-03-05T09:21:39.401935+0000 2024-03-04T09:41:51.795 INFO:teuthology.orchestra.run.smithi052.stdout:172.21.15.52:0/1094295190 2024-03-05T09:31:08.422018+0000 2024-03-04T09:41:51.795 INFO:teuthology.orchestra.run.smithi052.stdout:172.21.15.52:0/568242577 2024-03-05T09:21:12.760938+0000 2024-03-04T09:41:51.795 INFO:teuthology.orchestra.run.smithi052.stdout:172.21.15.52:0/2080168931 2024-03-05T09:31:08.422018+0000 2024-03-04T09:41:51.795 INFO:teuthology.orchestra.run.smithi052.stdout:172.21.15.52:6801/2425730636 2024-03-05T09:21:39.401935+0000 2024-03-04T09:41:51.796 INFO:teuthology.orchestra.run.smithi052.stdout:172.21.15.52:6800/2379361333 2024-03-05T09:21:12.760938+0000 2024-03-04T09:41:51.796 INFO:teuthology.orchestra.run.smithi052.stdout:172.21.15.52:6800/3036033923 2024-03-05T09:31:08.422018+0000 2024-03-04T09:41:51.796 INFO:teuthology.orchestra.run.smithi052.stdout:172.21.15.52:0/497075075 2024-03-05T09:21:12.760938+0000 2024-03-04T09:41:51.796 INFO:teuthology.orchestra.run.smithi052.stdout:172.21.15.52:6800/2425730636 2024-03-05T09:21:39.401935+0000 2024-03-04T09:41:51.796 INFO:teuthology.orchestra.run.smithi052.stdout:172.21.15.52:6801/2379361333 2024-03-05T09:21:12.760938+0000 2024-03-04T09:41:51.796 INFO:teuthology.orchestra.run.smithi052.stdout:172.21.15.52:0/1271949988 2024-03-05T09:21:39.401935+0000 2024-03-04T09:41:51.796 INFO:teuthology.orchestra.run.smithi052.stdout:172.21.15.52:0/2491451071 2024-03-05T09:31:08.422018+0000 2024-03-04T09:41:51.797 INFO:teuthology.orchestra.run.smithi052.stdout:172.21.15.52:0/2466768 2024-03-05T09:31:08.422018+0000 2024-03-04T09:41:51.797 INFO:teuthology.orchestra.run.smithi052.stdout:172.21.15.52:0/4082645718 2024-03-05T09:21:39.401935+0000 2024-03-04T09:41:51.797 INFO:teuthology.orchestra.run.smithi052.stdout:172.21.15.52:0/2935914357 2024-03-05T09:31:08.422018+0000 2024-03-04T09:41:51.797 INFO:teuthology.orchestra.run.smithi052.stdout:172.21.15.52:6801/3036033923 2024-03-05T09:31:08.422018+0000 2024-03-04T09:41:51.798 INFO:teuthology.orchestra.run.smithi052.stderr:listed 17 entries 2024-03-04T09:41:51.798 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:51.798+0000 7f7a5fde9640 1 -- 172.21.15.52:0/1165256000 >> [v2:172.21.15.52:6800/478031757,v1:172.21.15.52:6801/478031757] conn(0x7f7a440a1ef0 msgr2=0x7f7a440a43b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:41:51.798 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:51.798+0000 7f7a5fde9640 1 --2- 172.21.15.52:0/1165256000 >> [v2:172.21.15.52:6800/478031757,v1:172.21.15.52:6801/478031757] conn(0x7f7a440a1ef0 0x7f7a440a43b0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f7a5402c180 tx=0x7f7a540056d0 comp rx=0 tx=0).stop 2024-03-04T09:41:51.798 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:51.798+0000 7f7a5fde9640 1 -- 172.21.15.52:0/1165256000 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7a581064e0 msgr2=0x7f7a581a3d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:41:51.798 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:51.798+0000 7f7a5fde9640 1 --2- 172.21.15.52:0/1165256000 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7a581064e0 0x7f7a581a3d60 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f7a5000eb10 tx=0x7f7a5000c2d0 comp rx=0 tx=0).stop 2024-03-04T09:41:51.799 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:51.798+0000 7f7a5fde9640 1 -- 172.21.15.52:0/1165256000 shutdown_connections 2024-03-04T09:41:51.799 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:51.798+0000 7f7a5fde9640 1 --2- 172.21.15.52:0/1165256000 >> [v2:172.21.15.52:6800/478031757,v1:172.21.15.52:6801/478031757] conn(0x7f7a440a1ef0 0x7f7a440a43b0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:41:51.799 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:51.798+0000 7f7a5fde9640 1 --2- 172.21.15.52:0/1165256000 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7a581064e0 0x7f7a581a3d60 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:41:51.799 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:51.798+0000 7f7a5fde9640 1 --2- 172.21.15.52:0/1165256000 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7a58100e00 0x7f7a58199220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:41:51.799 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:51.798+0000 7f7a5fde9640 1 --2- 172.21.15.52:0/1165256000 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7a58100440 0x7f7a58198ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:41:51.799 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:51.798+0000 7f7a5fde9640 1 -- 172.21.15.52:0/1165256000 >> 172.21.15.52:0/1165256000 conn(0x7f7a580775d0 msgr2=0x7f7a58102ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:41:51.799 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:51.799+0000 7f7a5fde9640 1 -- 172.21.15.52:0/1165256000 shutdown_connections 2024-03-04T09:41:51.799 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:51.799+0000 7f7a5fde9640 1 -- 172.21.15.52:0/1165256000 wait complete. 2024-03-04T09:41:51.807 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-03-04T09:41:51.807 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-04T09:41:51.808 DEBUG:teuthology.orchestra.run.smithi052:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-03-04T09:41:51.912 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi052.front.sepia.ceph.com 2024-03-04T09:41:51.913 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-04T09:41:51.913 DEBUG:teuthology.orchestra.run.smithi052:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-03-04T09:41:51.928 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-04T09:41:51.928 DEBUG:teuthology.orchestra.run.smithi052:> ip netns list 2024-03-04T09:41:51.982 INFO:teuthology.orchestra.run.smithi052.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-03-04T09:41:51.983 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-04T09:41:51.983 DEBUG:teuthology.orchestra.run.smithi052:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-04T09:41:52.050 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-04T09:41:52.051 DEBUG:teuthology.orchestra.run.smithi052:> sudo ip link delete ceph-brx 2024-03-04T09:41:52.150 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-03-04T09:41:52.161 INFO:tasks.cephadm:Teardown begin 2024-03-04T09:41:52.162 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-03-04T09:41:52.162 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T09:41:52.269 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:52.269+0000 7f1a0ae37640 1 -- 172.21.15.52:0/3000734207 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1a04105500 msgr2=0x7f1a041058e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:41:52.269 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:52.269+0000 7f1a0ae37640 1 --2- 172.21.15.52:0/3000734207 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1a04105500 0x7f1a041058e0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f19fc009cd0 tx=0x7f19fc02c0f0 comp rx=0 tx=0).stop 2024-03-04T09:41:52.270 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:52.269+0000 7f1a0ae37640 1 -- 172.21.15.52:0/3000734207 shutdown_connections 2024-03-04T09:41:52.270 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:52.269+0000 7f1a0ae37640 1 --2- 172.21.15.52:0/3000734207 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1a04101a30 0x7f1a0410d3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:41:52.270 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:52.269+0000 7f1a0ae37640 1 --2- 172.21.15.52:0/3000734207 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1a04101070 0x7f1a041014f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:41:52.270 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:52.269+0000 7f1a0ae37640 1 --2- 172.21.15.52:0/3000734207 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1a04105500 0x7f1a041058e0 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:41:52.270 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:52.269+0000 7f1a0ae37640 1 -- 172.21.15.52:0/3000734207 >> 172.21.15.52:0/3000734207 conn(0x7f1a040757e0 msgr2=0x7f1a04075bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:41:52.270 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:52.269+0000 7f1a0ae37640 1 -- 172.21.15.52:0/3000734207 shutdown_connections 2024-03-04T09:41:52.270 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:52.269+0000 7f1a0ae37640 1 -- 172.21.15.52:0/3000734207 wait complete. 2024-03-04T09:41:52.271 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:52.269+0000 7f1a0ae37640 1 Processor -- start 2024-03-04T09:41:52.271 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:52.269+0000 7f1a0ae37640 1 -- start start 2024-03-04T09:41:52.271 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:52.269+0000 7f1a0ae37640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1a04101070 0x7f1a04196e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:41:52.271 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:52.269+0000 7f1a0ae37640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1a04101a30 0x7f1a04197350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:41:52.271 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:52.269+0000 7f1a0ae37640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1a04105500 0x7f1a0419b6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:41:52.271 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:52.270+0000 7f1a0ae37640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a041108d0 con 0x7f1a04101a30 2024-03-04T09:41:52.272 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:52.270+0000 7f1a0ae37640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a04110750 con 0x7f1a04105500 2024-03-04T09:41:52.272 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:52.270+0000 7f1a0ae37640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a04110a50 con 0x7f1a04101070 2024-03-04T09:41:52.272 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:52.270+0000 7f1a03fff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1a04101a30 0x7f1a04197350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:41:52.272 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:52.270+0000 7f1a08bd3640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1a04101070 0x7f1a04196e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:41:52.272 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:52.270+0000 7f1a093d4640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1a04105500 0x7f1a0419b6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:41:52.272 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:52.270+0000 7f1a03fff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1a04101a30 0x7f1a04197350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:54128/0 (socket says 172.21.15.52:54128) 2024-03-04T09:41:52.272 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:52.270+0000 7f1a03fff640 1 -- 172.21.15.52:0/1553528380 learned_addr learned my addr 172.21.15.52:0/1553528380 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-04T09:41:52.273 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:52.270+0000 7f1a03fff640 1 -- 172.21.15.52:0/1553528380 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1a04101070 msgr2=0x7f1a04196e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:41:52.273 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:52.270+0000 7f1a03fff640 1 --2- 172.21.15.52:0/1553528380 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1a04101070 0x7f1a04196e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:41:52.273 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:52.270+0000 7f1a03fff640 1 -- 172.21.15.52:0/1553528380 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1a04105500 msgr2=0x7f1a0419b6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:41:52.273 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:52.270+0000 7f1a03fff640 1 --2- 172.21.15.52:0/1553528380 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1a04105500 0x7f1a0419b6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:41:52.273 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:52.270+0000 7f1a03fff640 1 -- 172.21.15.52:0/1553528380 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a0419be00 con 0x7f1a04101a30 2024-03-04T09:41:52.273 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:52.270+0000 7f1a093d4640 1 --2- 172.21.15.52:0/1553528380 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1a04105500 0x7f1a0419b6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-04T09:41:52.273 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:52.270+0000 7f1a03fff640 1 --2- 172.21.15.52:0/1553528380 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1a04101a30 0x7f1a04197350 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f19f400e9f0 tx=0x7f19f400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:41:52.273 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:52.270+0000 7f1a01ffb640 1 -- 172.21.15.52:0/1553528380 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19f400cd80 con 0x7f1a04101a30 2024-03-04T09:41:52.273 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:52.270+0000 7f1a0ae37640 1 -- 172.21.15.52:0/1553528380 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a0419c0f0 con 0x7f1a04101a30 2024-03-04T09:41:52.274 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:52.270+0000 7f1a01ffb640 1 -- 172.21.15.52:0/1553528380 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f19f4005500 con 0x7f1a04101a30 2024-03-04T09:41:52.274 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:52.270+0000 7f1a01ffb640 1 -- 172.21.15.52:0/1553528380 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19f4010690 con 0x7f1a04101a30 2024-03-04T09:41:52.274 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:52.270+0000 7f1a0ae37640 1 -- 172.21.15.52:0/1553528380 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a041a3980 con 0x7f1a04101a30 2024-03-04T09:41:52.275 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:52.272+0000 7f1a0ae37640 1 -- 172.21.15.52:0/1553528380 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a04068810 con 0x7f1a04101a30 2024-03-04T09:41:52.275 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:52.272+0000 7f1a01ffb640 1 -- 172.21.15.52:0/1553528380 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 25) v1 ==== 136887+0+0 (secure 0 0 0) 0x7f19f40046c0 con 0x7f1a04101a30 2024-03-04T09:41:52.275 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:52.273+0000 7f1a01ffb640 1 --2- 172.21.15.52:0/1553528380 >> [v2:172.21.15.52:6800/478031757,v1:172.21.15.52:6801/478031757] conn(0x7f19f00a1ef0 0x7f19f00a43b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:41:52.276 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:52.273+0000 7f1a08bd3640 1 --2- 172.21.15.52:0/1553528380 >> [v2:172.21.15.52:6800/478031757,v1:172.21.15.52:6801/478031757] conn(0x7f19f00a1ef0 0x7f19f00a43b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:41:52.276 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:52.273+0000 7f1a01ffb640 1 -- 172.21.15.52:0/1553528380 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 9547+0+0 (secure 0 0 0) 0x7f19f40cea10 con 0x7f1a04101a30 2024-03-04T09:41:52.276 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:52.274+0000 7f1a08bd3640 1 --2- 172.21.15.52:0/1553528380 >> [v2:172.21.15.52:6800/478031757,v1:172.21.15.52:6801/478031757] conn(0x7f19f00a1ef0 0x7f19f00a43b0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f19fc005cc0 tx=0x7f19fc005c50 comp rx=0 tx=0).ready entity=mgr.15486 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:41:52.276 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:52.277+0000 7f1a01ffb640 1 -- 172.21.15.52:0/1553528380 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f19f409b6a0 con 0x7f1a04101a30 2024-03-04T09:41:52.335 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:41:52 smithi052 ceph-mon[32473]: pgmap v332: 129 pgs: 129 active+clean; 78 MiB data, 691 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-04T09:41:52.336 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:41:52 smithi052 ceph-mon[32473]: from='client.? 172.21.15.52:0/2087662425' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-04T09:41:52.336 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:41:52 smithi052 ceph-mon[32473]: from='client.? 172.21.15.52:0/1696869035' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-04T09:41:52.336 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:41:52 smithi052 ceph-mon[32473]: from='client.? 172.21.15.52:0/1165256000' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-03-04T09:41:52.494 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:52.496+0000 7f1a0ae37640 1 -- 172.21.15.52:0/1553528380 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1a04108a40 con 0x7f1a04101a30 2024-03-04T09:41:52.494 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:52.497+0000 7f1a01ffb640 1 -- 172.21.15.52:0/1553528380 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) v1 ==== 94+0+11406 (secure 0 0 0) 0x7f19f409afa0 con 0x7f1a04101a30 2024-03-04T09:41:52.495 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-03-04T09:41:52.495 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6835/1007961019","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6834","nonce":1007961019},{"type":"v1","addr":"172.21.15.52:6835","nonce":1007961019}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6837/3753881283","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6836","nonce":3753881283},{"type":"v1","addr":"172.21.15.52:6837","nonce":3753881283}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14628,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6839/3467897808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6838","nonce":3467897808},{"type":"v1","addr":"172.21.15.52:6839","nonce":3467897808}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14640,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6841/2804292291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6840","nonce":2804292291},{"type":"v1","addr":"172.21.15.52:6841","nonce":2804292291}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24407,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6833/1818480323","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6832","nonce":1818480323},{"type":"v1","addr":"172.21.15.106:6833","nonce":1818480323}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24412,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6833/765768204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":765768204},{"type":"v1","addr":"172.21.15.191:6833","nonce":765768204}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-04T09:28:37.264970+0000","modified":"2024-03-04T09:35:38.670247+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24506,"mds_1":24490,"mds_2":24473},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24440":{"gid":24440,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.106:6835/1541069897","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6834","nonce":1541069897},{"type":"v1","addr":"172.21.15.106:6835","nonce":1541069897}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24451":{"gid":24451,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.191:6835/105048699","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":105048699},{"type":"v1","addr":"172.21.15.191:6835","nonce":105048699}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24463":{"gid":24463,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.191:6837/4071064225","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6836","nonce":4071064225},{"type":"v1","addr":"172.21.15.191:6837","nonce":4071064225}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24473":{"gid":24473,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.106:6837/2901755066","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6836","nonce":2901755066},{"type":"v1","addr":"172.21.15.106:6837","nonce":2901755066}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24490":{"gid":24490,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.191:6839/2194324891","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6838","nonce":2194324891},{"type":"v1","addr":"172.21.15.191:6839","nonce":2194324891}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24506":{"gid":24506,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.106:6839/1995853093","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6838","nonce":1995853093},{"type":"v1","addr":"172.21.15.106:6839","nonce":1995853093}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24506,"qdb_cluster":[24506,24473,24490]},"id":1}]} 2024-03-04T09:41:52.496 INFO:teuthology.orchestra.run.smithi052.stderr:dumped fsmap epoch 28 2024-03-04T09:41:52.497 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:52.499+0000 7f1a0ae37640 1 -- 172.21.15.52:0/1553528380 >> [v2:172.21.15.52:6800/478031757,v1:172.21.15.52:6801/478031757] conn(0x7f19f00a1ef0 msgr2=0x7f19f00a43b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:41:52.497 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:52.499+0000 7f1a0ae37640 1 --2- 172.21.15.52:0/1553528380 >> [v2:172.21.15.52:6800/478031757,v1:172.21.15.52:6801/478031757] conn(0x7f19f00a1ef0 0x7f19f00a43b0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f19fc005cc0 tx=0x7f19fc005c50 comp rx=0 tx=0).stop 2024-03-04T09:41:52.497 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:52.499+0000 7f1a0ae37640 1 -- 172.21.15.52:0/1553528380 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1a04101a30 msgr2=0x7f1a04197350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:41:52.498 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:52.499+0000 7f1a0ae37640 1 --2- 172.21.15.52:0/1553528380 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1a04101a30 0x7f1a04197350 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f19f400e9f0 tx=0x7f19f400edf0 comp rx=0 tx=0).stop 2024-03-04T09:41:52.498 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:52.499+0000 7f1a0ae37640 1 -- 172.21.15.52:0/1553528380 shutdown_connections 2024-03-04T09:41:52.498 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:52.499+0000 7f1a0ae37640 1 --2- 172.21.15.52:0/1553528380 >> [v2:172.21.15.52:6800/478031757,v1:172.21.15.52:6801/478031757] conn(0x7f19f00a1ef0 0x7f19f00a43b0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:41:52.498 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:52.499+0000 7f1a0ae37640 1 --2- 172.21.15.52:0/1553528380 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1a04105500 0x7f1a0419b6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:41:52.498 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:52.499+0000 7f1a0ae37640 1 --2- 172.21.15.52:0/1553528380 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1a04101a30 0x7f1a04197350 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:41:52.498 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:52.499+0000 7f1a0ae37640 1 --2- 172.21.15.52:0/1553528380 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1a04101070 0x7f1a04196e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:41:52.498 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:52.499+0000 7f1a0ae37640 1 -- 172.21.15.52:0/1553528380 >> 172.21.15.52:0/1553528380 conn(0x7f1a040757e0 msgr2=0x7f1a040fde00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:41:52.498 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:52.499+0000 7f1a0ae37640 1 -- 172.21.15.52:0/1553528380 shutdown_connections 2024-03-04T09:41:52.499 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:52.499+0000 7f1a0ae37640 1 -- 172.21.15.52:0/1553528380 wait complete. 2024-03-04T09:41:52.508 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T09:41:52.587 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:41:52 smithi191 ceph-mon[36816]: pgmap v332: 129 pgs: 129 active+clean; 78 MiB data, 691 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-04T09:41:52.588 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:41:52 smithi191 ceph-mon[36816]: from='client.? 172.21.15.52:0/2087662425' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-04T09:41:52.588 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:41:52 smithi191 ceph-mon[36816]: from='client.? 172.21.15.52:0/1696869035' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-04T09:41:52.589 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:41:52 smithi191 ceph-mon[36816]: from='client.? 172.21.15.52:0/1165256000' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-03-04T09:41:52.618 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:52.614+0000 7fc4515ae640 1 -- 172.21.15.52:0/923073532 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc44c101580 msgr2=0x7fc44c101a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:41:52.618 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:52.614+0000 7fc4515ae640 1 --2- 172.21.15.52:0/923073532 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc44c101580 0x7fc44c101a00 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fc440009900 tx=0x7fc44002bc30 comp rx=0 tx=0).stop 2024-03-04T09:41:52.618 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:52.614+0000 7fc4515ae640 1 -- 172.21.15.52:0/923073532 shutdown_connections 2024-03-04T09:41:52.619 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:52.614+0000 7fc4515ae640 1 --2- 172.21.15.52:0/923073532 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc44c101f40 0x7fc44c10e400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:41:52.619 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:52.614+0000 7fc4515ae640 1 --2- 172.21.15.52:0/923073532 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc44c101580 0x7fc44c101a00 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:41:52.619 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:52.614+0000 7fc4515ae640 1 --2- 172.21.15.52:0/923073532 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc44c107640 0x7fc44c107a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:41:52.620 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:52.614+0000 7fc4515ae640 1 -- 172.21.15.52:0/923073532 >> 172.21.15.52:0/923073532 conn(0x7fc44c0fd070 msgr2=0x7fc44c0ff490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:41:52.620 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:52.614+0000 7fc4515ae640 1 -- 172.21.15.52:0/923073532 shutdown_connections 2024-03-04T09:41:52.620 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:52.614+0000 7fc4515ae640 1 -- 172.21.15.52:0/923073532 wait complete. 2024-03-04T09:41:52.621 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:52.614+0000 7fc4515ae640 1 Processor -- start 2024-03-04T09:41:52.621 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:52.615+0000 7fc4515ae640 1 -- start start 2024-03-04T09:41:52.621 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:52.615+0000 7fc4515ae640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc44c101580 0x7fc44c197e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:41:52.621 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:52.615+0000 7fc4515ae640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc44c101f40 0x7fc44c1983a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:41:52.622 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:52.615+0000 7fc4515ae640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc44c107640 0x7fc44c19c720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:41:52.622 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:52.615+0000 7fc4515ae640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc44c111900 con 0x7fc44c101580 2024-03-04T09:41:52.622 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:52.615+0000 7fc4515ae640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc44c111780 con 0x7fc44c107640 2024-03-04T09:41:52.623 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:52.615+0000 7fc4515ae640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc44c111a80 con 0x7fc44c101f40 2024-03-04T09:41:52.623 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:52.615+0000 7fc44affd640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc44c101580 0x7fc44c197e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:41:52.623 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:52.615+0000 7fc44affd640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc44c101580 0x7fc44c197e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:54150/0 (socket says 172.21.15.52:54150) 2024-03-04T09:41:52.623 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:52.615+0000 7fc44affd640 1 -- 172.21.15.52:0/49253365 learned_addr learned my addr 172.21.15.52:0/49253365 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-04T09:41:52.624 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:52.615+0000 7fc44b7fe640 1 --2- 172.21.15.52:0/49253365 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc44c107640 0x7fc44c19c720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:41:52.624 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:52.615+0000 7fc44a7fc640 1 --2- 172.21.15.52:0/49253365 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc44c101f40 0x7fc44c1983a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:41:52.624 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:52.615+0000 7fc44affd640 1 -- 172.21.15.52:0/49253365 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc44c101f40 msgr2=0x7fc44c1983a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:41:52.625 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:52.615+0000 7fc44affd640 1 --2- 172.21.15.52:0/49253365 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc44c101f40 0x7fc44c1983a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:41:52.625 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:52.615+0000 7fc44affd640 1 -- 172.21.15.52:0/49253365 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc44c107640 msgr2=0x7fc44c19c720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:41:52.625 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:52.615+0000 7fc44affd640 1 --2- 172.21.15.52:0/49253365 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc44c107640 0x7fc44c19c720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:41:52.625 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:52.615+0000 7fc44affd640 1 -- 172.21.15.52:0/49253365 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc44c19ce50 con 0x7fc44c101580 2024-03-04T09:41:52.626 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:52.615+0000 7fc44affd640 1 --2- 172.21.15.52:0/49253365 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc44c101580 0x7fc44c197e60 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fc43c00e9f0 tx=0x7fc43c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:41:52.626 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:52.615+0000 7fc433fff640 1 -- 172.21.15.52:0/49253365 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc43c00cd80 con 0x7fc44c101580 2024-03-04T09:41:52.626 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:52.616+0000 7fc433fff640 1 -- 172.21.15.52:0/49253365 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc43c010910 con 0x7fc44c101580 2024-03-04T09:41:52.626 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:52.616+0000 7fc4515ae640 1 -- 172.21.15.52:0/49253365 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc44c19d140 con 0x7fc44c101580 2024-03-04T09:41:52.627 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:52.616+0000 7fc433fff640 1 -- 172.21.15.52:0/49253365 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc43c018990 con 0x7fc44c101580 2024-03-04T09:41:52.627 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:52.616+0000 7fc4515ae640 1 -- 172.21.15.52:0/49253365 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc44c1a49d0 con 0x7fc44c101580 2024-03-04T09:41:52.627 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:52.616+0000 7fc4515ae640 1 -- 172.21.15.52:0/49253365 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc44c068810 con 0x7fc44c101580 2024-03-04T09:41:52.628 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:52.618+0000 7fc433fff640 1 -- 172.21.15.52:0/49253365 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 25) v1 ==== 136887+0+0 (secure 0 0 0) 0x7fc43c010ac0 con 0x7fc44c101580 2024-03-04T09:41:52.628 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:52.618+0000 7fc433fff640 1 --2- 172.21.15.52:0/49253365 >> [v2:172.21.15.52:6800/478031757,v1:172.21.15.52:6801/478031757] conn(0x7fc4340a1ef0 0x7fc4340a43b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:41:52.628 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:52.618+0000 7fc44a7fc640 1 --2- 172.21.15.52:0/49253365 >> [v2:172.21.15.52:6800/478031757,v1:172.21.15.52:6801/478031757] conn(0x7fc4340a1ef0 0x7fc4340a43b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:41:52.628 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:52.618+0000 7fc433fff640 1 -- 172.21.15.52:0/49253365 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 9547+0+0 (secure 0 0 0) 0x7fc43c0ce840 con 0x7fc44c101580 2024-03-04T09:41:52.629 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:52.619+0000 7fc44a7fc640 1 --2- 172.21.15.52:0/49253365 >> [v2:172.21.15.52:6800/478031757,v1:172.21.15.52:6801/478031757] conn(0x7fc4340a1ef0 0x7fc4340a43b0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fc440009230 tx=0x7fc440005fd0 comp rx=0 tx=0).ready entity=mgr.15486 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:41:52.629 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:52.623+0000 7fc433fff640 1 -- 172.21.15.52:0/49253365 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7fc43c09aee0 con 0x7fc44c101580 2024-03-04T09:41:52.632 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:41:52 smithi106 ceph-mon[36484]: pgmap v332: 129 pgs: 129 active+clean; 78 MiB data, 691 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-04T09:41:52.632 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:41:52 smithi106 ceph-mon[36484]: from='client.? 172.21.15.52:0/2087662425' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-04T09:41:52.632 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:41:52 smithi106 ceph-mon[36484]: from='client.? 172.21.15.52:0/1696869035' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-04T09:41:52.633 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:41:52 smithi106 ceph-mon[36484]: from='client.? 172.21.15.52:0/1165256000' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-03-04T09:41:52.842 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:52.845+0000 7fc4515ae640 1 -- 172.21.15.52:0/49253365 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc44c1991b0 con 0x7fc44c101580 2024-03-04T09:41:52.843 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:52.845+0000 7fc433fff640 1 -- 172.21.15.52:0/49253365 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) v1 ==== 94+0+11406 (secure 0 0 0) 0x7fc43c09ef80 con 0x7fc44c101580 2024-03-04T09:41:52.843 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-03-04T09:41:52.844 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6835/1007961019","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6834","nonce":1007961019},{"type":"v1","addr":"172.21.15.52:6835","nonce":1007961019}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6837/3753881283","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6836","nonce":3753881283},{"type":"v1","addr":"172.21.15.52:6837","nonce":3753881283}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14628,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6839/3467897808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6838","nonce":3467897808},{"type":"v1","addr":"172.21.15.52:6839","nonce":3467897808}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14640,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6841/2804292291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6840","nonce":2804292291},{"type":"v1","addr":"172.21.15.52:6841","nonce":2804292291}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24407,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6833/1818480323","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6832","nonce":1818480323},{"type":"v1","addr":"172.21.15.106:6833","nonce":1818480323}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24412,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6833/765768204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":765768204},{"type":"v1","addr":"172.21.15.191:6833","nonce":765768204}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-04T09:28:37.264970+0000","modified":"2024-03-04T09:35:38.670247+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24506,"mds_1":24490,"mds_2":24473},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24440":{"gid":24440,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.106:6835/1541069897","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6834","nonce":1541069897},{"type":"v1","addr":"172.21.15.106:6835","nonce":1541069897}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24451":{"gid":24451,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.191:6835/105048699","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":105048699},{"type":"v1","addr":"172.21.15.191:6835","nonce":105048699}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24463":{"gid":24463,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.191:6837/4071064225","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6836","nonce":4071064225},{"type":"v1","addr":"172.21.15.191:6837","nonce":4071064225}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24473":{"gid":24473,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.106:6837/2901755066","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6836","nonce":2901755066},{"type":"v1","addr":"172.21.15.106:6837","nonce":2901755066}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24490":{"gid":24490,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.191:6839/2194324891","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6838","nonce":2194324891},{"type":"v1","addr":"172.21.15.191:6839","nonce":2194324891}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24506":{"gid":24506,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.106:6839/1995853093","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6838","nonce":1995853093},{"type":"v1","addr":"172.21.15.106:6839","nonce":1995853093}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24506,"qdb_cluster":[24506,24473,24490]},"id":1}]} 2024-03-04T09:41:52.844 INFO:teuthology.orchestra.run.smithi052.stderr:dumped fsmap epoch 28 2024-03-04T09:41:52.845 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:52.847+0000 7fc4515ae640 1 -- 172.21.15.52:0/49253365 >> [v2:172.21.15.52:6800/478031757,v1:172.21.15.52:6801/478031757] conn(0x7fc4340a1ef0 msgr2=0x7fc4340a43b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:41:52.845 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:52.847+0000 7fc4515ae640 1 --2- 172.21.15.52:0/49253365 >> [v2:172.21.15.52:6800/478031757,v1:172.21.15.52:6801/478031757] conn(0x7fc4340a1ef0 0x7fc4340a43b0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fc440009230 tx=0x7fc440005fd0 comp rx=0 tx=0).stop 2024-03-04T09:41:52.846 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:52.847+0000 7fc4515ae640 1 -- 172.21.15.52:0/49253365 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc44c101580 msgr2=0x7fc44c197e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:41:52.846 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:52.847+0000 7fc4515ae640 1 --2- 172.21.15.52:0/49253365 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc44c101580 0x7fc44c197e60 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fc43c00e9f0 tx=0x7fc43c00eeb0 comp rx=0 tx=0).stop 2024-03-04T09:41:52.846 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:52.847+0000 7fc4515ae640 1 -- 172.21.15.52:0/49253365 shutdown_connections 2024-03-04T09:41:52.846 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:52.847+0000 7fc4515ae640 1 --2- 172.21.15.52:0/49253365 >> [v2:172.21.15.52:6800/478031757,v1:172.21.15.52:6801/478031757] conn(0x7fc4340a1ef0 0x7fc4340a43b0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:41:52.846 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:52.847+0000 7fc4515ae640 1 --2- 172.21.15.52:0/49253365 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc44c107640 0x7fc44c19c720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:41:52.846 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:52.847+0000 7fc4515ae640 1 --2- 172.21.15.52:0/49253365 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc44c101f40 0x7fc44c1983a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:41:52.846 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:52.847+0000 7fc4515ae640 1 --2- 172.21.15.52:0/49253365 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc44c101580 0x7fc44c197e60 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:41:52.847 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:52.847+0000 7fc4515ae640 1 -- 172.21.15.52:0/49253365 >> 172.21.15.52:0/49253365 conn(0x7fc44c0fd070 msgr2=0x7fc44c0fea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:41:52.847 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:52.847+0000 7fc4515ae640 1 -- 172.21.15.52:0/49253365 shutdown_connections 2024-03-04T09:41:52.847 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:52.847+0000 7fc4515ae640 1 -- 172.21.15.52:0/49253365 wait complete. 2024-03-04T09:41:52.857 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-04T09:41:52.961 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:52.961+0000 7f21c5688640 1 -- 172.21.15.52:0/343819138 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f21c0107640 msgr2=0x7f21c0107a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:41:52.962 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:52.961+0000 7f21c5688640 1 --2- 172.21.15.52:0/343819138 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f21c0107640 0x7f21c0107a20 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f21b0009900 tx=0x7f21b002bc30 comp rx=0 tx=0).stop 2024-03-04T09:41:52.962 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:52.961+0000 7f21c5688640 1 -- 172.21.15.52:0/343819138 shutdown_connections 2024-03-04T09:41:52.962 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:52.961+0000 7f21c5688640 1 --2- 172.21.15.52:0/343819138 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f21c0101f40 0x7f21c010e400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:41:52.962 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:52.961+0000 7f21c5688640 1 --2- 172.21.15.52:0/343819138 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f21c0101580 0x7f21c0101a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:41:52.962 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:52.961+0000 7f21c5688640 1 --2- 172.21.15.52:0/343819138 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f21c0107640 0x7f21c0107a20 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:41:52.962 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:52.961+0000 7f21c5688640 1 -- 172.21.15.52:0/343819138 >> 172.21.15.52:0/343819138 conn(0x7f21c00fd070 msgr2=0x7f21c00ff490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:41:52.962 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:52.961+0000 7f21c5688640 1 -- 172.21.15.52:0/343819138 shutdown_connections 2024-03-04T09:41:52.963 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:52.961+0000 7f21c5688640 1 -- 172.21.15.52:0/343819138 wait complete. 2024-03-04T09:41:52.963 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:52.962+0000 7f21c5688640 1 Processor -- start 2024-03-04T09:41:52.963 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:52.962+0000 7f21c5688640 1 -- start start 2024-03-04T09:41:52.963 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:52.962+0000 7f21c5688640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f21c0101580 0x7f21c0197ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:41:52.963 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:52.962+0000 7f21c5688640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f21c0101f40 0x7f21c01983e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:41:52.963 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:52.962+0000 7f21c5688640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f21c0107640 0x7f21c019c710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:41:52.963 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:52.962+0000 7f21c5688640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21c0111920 con 0x7f21c0107640 2024-03-04T09:41:52.963 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:52.962+0000 7f21c5688640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21c01117a0 con 0x7f21c0101580 2024-03-04T09:41:52.964 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:52.962+0000 7f21c5688640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21c0111aa0 con 0x7f21c0101f40 2024-03-04T09:41:52.964 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:52.962+0000 7f21bf7fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f21c0107640 0x7f21c019c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:41:52.964 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:52.962+0000 7f21bf7fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f21c0107640 0x7f21c019c710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:54170/0 (socket says 172.21.15.52:54170) 2024-03-04T09:41:52.964 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:52.962+0000 7f21bf7fe640 1 -- 172.21.15.52:0/1551426807 learned_addr learned my addr 172.21.15.52:0/1551426807 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-04T09:41:52.964 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:52.962+0000 7f21beffd640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f21c0101580 0x7f21c0197ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:41:52.964 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:52.962+0000 7f21b7fff640 1 --2- 172.21.15.52:0/1551426807 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f21c0101f40 0x7f21c01983e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:41:52.964 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:52.962+0000 7f21bf7fe640 1 -- 172.21.15.52:0/1551426807 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f21c0101f40 msgr2=0x7f21c01983e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:41:52.964 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:52.962+0000 7f21bf7fe640 1 --2- 172.21.15.52:0/1551426807 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f21c0101f40 0x7f21c01983e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:41:52.964 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:52.962+0000 7f21bf7fe640 1 -- 172.21.15.52:0/1551426807 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f21c0101580 msgr2=0x7f21c0197ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:41:52.965 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:52.962+0000 7f21bf7fe640 1 --2- 172.21.15.52:0/1551426807 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f21c0101580 0x7f21c0197ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:41:52.965 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:52.962+0000 7f21bf7fe640 1 -- 172.21.15.52:0/1551426807 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21c019ce40 con 0x7f21c0107640 2024-03-04T09:41:52.965 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:52.962+0000 7f21beffd640 1 --2- 172.21.15.52:0/1551426807 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f21c0101580 0x7f21c0197ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-04T09:41:52.965 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:52.963+0000 7f21bf7fe640 1 --2- 172.21.15.52:0/1551426807 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f21c0107640 0x7f21c019c710 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f21b800eb10 tx=0x7f21b800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:41:52.965 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:52.963+0000 7f21bcff9640 1 -- 172.21.15.52:0/1551426807 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21b8004b20 con 0x7f21c0107640 2024-03-04T09:41:52.965 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:52.963+0000 7f21bcff9640 1 -- 172.21.15.52:0/1551426807 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f21b8004cc0 con 0x7f21c0107640 2024-03-04T09:41:52.965 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:52.963+0000 7f21bcff9640 1 -- 172.21.15.52:0/1551426807 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21b8010760 con 0x7f21c0107640 2024-03-04T09:41:52.965 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:52.963+0000 7f21c5688640 1 -- 172.21.15.52:0/1551426807 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21c019d130 con 0x7f21c0107640 2024-03-04T09:41:52.965 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:52.963+0000 7f21c5688640 1 -- 172.21.15.52:0/1551426807 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21c01a49c0 con 0x7f21c0107640 2024-03-04T09:41:52.966 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:52.964+0000 7f21c5688640 1 -- 172.21.15.52:0/1551426807 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f21c0068810 con 0x7f21c0107640 2024-03-04T09:41:52.967 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:52.965+0000 7f21bcff9640 1 -- 172.21.15.52:0/1551426807 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 25) v1 ==== 136887+0+0 (secure 0 0 0) 0x7f21b8005020 con 0x7f21c0107640 2024-03-04T09:41:52.967 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:52.966+0000 7f21bcff9640 1 --2- 172.21.15.52:0/1551426807 >> [v2:172.21.15.52:6800/478031757,v1:172.21.15.52:6801/478031757] conn(0x7f21ac0a1ef0 0x7f21ac0a43b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:41:52.967 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:52.966+0000 7f21beffd640 1 --2- 172.21.15.52:0/1551426807 >> [v2:172.21.15.52:6800/478031757,v1:172.21.15.52:6801/478031757] conn(0x7f21ac0a1ef0 0x7f21ac0a43b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:41:52.967 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:52.966+0000 7f21bcff9640 1 -- 172.21.15.52:0/1551426807 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 9547+0+0 (secure 0 0 0) 0x7f21b80cec20 con 0x7f21c0107640 2024-03-04T09:41:52.967 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:52.966+0000 7f21beffd640 1 --2- 172.21.15.52:0/1551426807 >> [v2:172.21.15.52:6800/478031757,v1:172.21.15.52:6801/478031757] conn(0x7f21ac0a1ef0 0x7f21ac0a43b0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f21b0009230 tx=0x7f21b0005fd0 comp rx=0 tx=0).ready entity=mgr.15486 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:41:52.968 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:52.970+0000 7f21bcff9640 1 -- 172.21.15.52:0/1551426807 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f21b809b910 con 0x7f21c0107640 2024-03-04T09:41:53.156 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:53.158+0000 7f21c5688640 1 -- 172.21.15.52:0/1551426807 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f21c01094a0 con 0x7f21c0107640 2024-03-04T09:41:53.159 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:53.160+0000 7f21bcff9640 1 -- 172.21.15.52:0/1551426807 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v102) v1 ==== 74+0+25427 (secure 0 0 0) 0x7f21b809b210 con 0x7f21c0107640 2024-03-04T09:41:53.160 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-03-04T09:41:53.160 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":102,"fsid":"4e059f08-da08-11ee-95c6-87774f69a715","created":"2024-03-04T09:20:46.104963+0000","modified":"2024-03-04T09:41:15.999027+0000","last_up_change":"2024-03-04T09:27:49.261365+0000","last_in_change":"2024-03-04T09:27:30.024964+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-04T09:24:01.742461+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-04T09:28:35.835600+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-04T09:28:36.453099+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"101","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":15,"snap_epoch":101,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-04T09:28:40.168831+0000","flags":8197,"flags_names":"hashpspool,ec_overwrites,selfmanaged_snaps","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"101","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":15,"snap_epoch":101,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"583172be-602f-4bf4-bfcb-42663cf5b277","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6802","nonce":1371221397},{"type":"v1","addr":"172.21.15.52:6803","nonce":1371221397}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6804","nonce":1371221397},{"type":"v1","addr":"172.21.15.52:6805","nonce":1371221397}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6808","nonce":1371221397},{"type":"v1","addr":"172.21.15.52:6809","nonce":1371221397}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6806","nonce":1371221397},{"type":"v1","addr":"172.21.15.52:6807","nonce":1371221397}]},"public_addr":"172.21.15.52:6803/1371221397","cluster_addr":"172.21.15.52:6805/1371221397","heartbeat_back_addr":"172.21.15.52:6809/1371221397","heartbeat_front_addr":"172.21.15.52:6807/1371221397","state":["exists","up"]},{"osd":1,"uuid":"5ae0081c-a2f6-46eb-ace0-0368208377b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":2540865537},{"type":"v1","addr":"172.21.15.106:6801","nonce":2540865537}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":2540865537},{"type":"v1","addr":"172.21.15.106:6803","nonce":2540865537}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":2540865537},{"type":"v1","addr":"172.21.15.106:6807","nonce":2540865537}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":2540865537},{"type":"v1","addr":"172.21.15.106:6805","nonce":2540865537}]},"public_addr":"172.21.15.106:6801/2540865537","cluster_addr":"172.21.15.106:6803/2540865537","heartbeat_back_addr":"172.21.15.106:6807/2540865537","heartbeat_front_addr":"172.21.15.106:6805/2540865537","state":["exists","up"]},{"osd":2,"uuid":"1948bac7-2bb2-4218-8b93-4e4743a0de04","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":1377083164},{"type":"v1","addr":"172.21.15.191:6801","nonce":1377083164}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":1377083164},{"type":"v1","addr":"172.21.15.191:6803","nonce":1377083164}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":1377083164},{"type":"v1","addr":"172.21.15.191:6807","nonce":1377083164}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":1377083164},{"type":"v1","addr":"172.21.15.191:6805","nonce":1377083164}]},"public_addr":"172.21.15.191:6801/1377083164","cluster_addr":"172.21.15.191:6803/1377083164","heartbeat_back_addr":"172.21.15.191:6807/1377083164","heartbeat_front_addr":"172.21.15.191:6805/1377083164","state":["exists","up"]},{"osd":3,"uuid":"16c9e9fe-7167-4992-bf12-1a7398e0498a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6810","nonce":2388571810},{"type":"v1","addr":"172.21.15.52:6811","nonce":2388571810}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6812","nonce":2388571810},{"type":"v1","addr":"172.21.15.52:6813","nonce":2388571810}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6816","nonce":2388571810},{"type":"v1","addr":"172.21.15.52:6817","nonce":2388571810}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6814","nonce":2388571810},{"type":"v1","addr":"172.21.15.52:6815","nonce":2388571810}]},"public_addr":"172.21.15.52:6811/2388571810","cluster_addr":"172.21.15.52:6813/2388571810","heartbeat_back_addr":"172.21.15.52:6817/2388571810","heartbeat_front_addr":"172.21.15.52:6815/2388571810","state":["exists","up"]},{"osd":4,"uuid":"c8ef48c7-7c78-44d3-a9e5-99ec750abd3d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6808","nonce":3465868015},{"type":"v1","addr":"172.21.15.106:6809","nonce":3465868015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6810","nonce":3465868015},{"type":"v1","addr":"172.21.15.106:6811","nonce":3465868015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6814","nonce":3465868015},{"type":"v1","addr":"172.21.15.106:6815","nonce":3465868015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6812","nonce":3465868015},{"type":"v1","addr":"172.21.15.106:6813","nonce":3465868015}]},"public_addr":"172.21.15.106:6809/3465868015","cluster_addr":"172.21.15.106:6811/3465868015","heartbeat_back_addr":"172.21.15.106:6815/3465868015","heartbeat_front_addr":"172.21.15.106:6813/3465868015","state":["exists","up"]},{"osd":5,"uuid":"9e41824a-c5a2-4f88-be50-2ba1dceb3220","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":1485092643},{"type":"v1","addr":"172.21.15.191:6809","nonce":1485092643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":1485092643},{"type":"v1","addr":"172.21.15.191:6811","nonce":1485092643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":1485092643},{"type":"v1","addr":"172.21.15.191:6815","nonce":1485092643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":1485092643},{"type":"v1","addr":"172.21.15.191:6813","nonce":1485092643}]},"public_addr":"172.21.15.191:6809/1485092643","cluster_addr":"172.21.15.191:6811/1485092643","heartbeat_back_addr":"172.21.15.191:6815/1485092643","heartbeat_front_addr":"172.21.15.191:6813/1485092643","state":["exists","up"]},{"osd":6,"uuid":"b35dccee-ec16-452e-ac69-3ac2fb7b27fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6818","nonce":1989797856},{"type":"v1","addr":"172.21.15.52:6819","nonce":1989797856}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6820","nonce":1989797856},{"type":"v1","addr":"172.21.15.52:6821","nonce":1989797856}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6824","nonce":1989797856},{"type":"v1","addr":"172.21.15.52:6825","nonce":1989797856}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6822","nonce":1989797856},{"type":"v1","addr":"172.21.15.52:6823","nonce":1989797856}]},"public_addr":"172.21.15.52:6819/1989797856","cluster_addr":"172.21.15.52:6821/1989797856","heartbeat_back_addr":"172.21.15.52:6825/1989797856","heartbeat_front_addr":"172.21.15.52:6823/1989797856","state":["exists","up"]},{"osd":7,"uuid":"71303e3d-14ec-4cbd-b19c-67645a1de3c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6816","nonce":1450870329},{"type":"v1","addr":"172.21.15.106:6817","nonce":1450870329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":1450870329},{"type":"v1","addr":"172.21.15.106:6819","nonce":1450870329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":1450870329},{"type":"v1","addr":"172.21.15.106:6823","nonce":1450870329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6820","nonce":1450870329},{"type":"v1","addr":"172.21.15.106:6821","nonce":1450870329}]},"public_addr":"172.21.15.106:6817/1450870329","cluster_addr":"172.21.15.106:6819/1450870329","heartbeat_back_addr":"172.21.15.106:6823/1450870329","heartbeat_front_addr":"172.21.15.106:6821/1450870329","state":["exists","up"]},{"osd":8,"uuid":"6f5786c4-4402-435a-bdad-adff46fe9473","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":1114119072},{"type":"v1","addr":"172.21.15.191:6817","nonce":1114119072}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6818","nonce":1114119072},{"type":"v1","addr":"172.21.15.191:6819","nonce":1114119072}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":1114119072},{"type":"v1","addr":"172.21.15.191:6823","nonce":1114119072}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":1114119072},{"type":"v1","addr":"172.21.15.191:6821","nonce":1114119072}]},"public_addr":"172.21.15.191:6817/1114119072","cluster_addr":"172.21.15.191:6819/1114119072","heartbeat_back_addr":"172.21.15.191:6823/1114119072","heartbeat_front_addr":"172.21.15.191:6821/1114119072","state":["exists","up"]},{"osd":9,"uuid":"062bbaa5-1227-4684-b859-450f5594a90c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6826","nonce":3485821914},{"type":"v1","addr":"172.21.15.52:6827","nonce":3485821914}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6828","nonce":3485821914},{"type":"v1","addr":"172.21.15.52:6829","nonce":3485821914}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6832","nonce":3485821914},{"type":"v1","addr":"172.21.15.52:6833","nonce":3485821914}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6830","nonce":3485821914},{"type":"v1","addr":"172.21.15.52:6831","nonce":3485821914}]},"public_addr":"172.21.15.52:6827/3485821914","cluster_addr":"172.21.15.52:6829/3485821914","heartbeat_back_addr":"172.21.15.52:6833/3485821914","heartbeat_front_addr":"172.21.15.52:6831/3485821914","state":["exists","up"]},{"osd":10,"uuid":"1a285a5a-f793-48da-a8eb-5326d9528cd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":3815586576},{"type":"v1","addr":"172.21.15.106:6825","nonce":3815586576}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6826","nonce":3815586576},{"type":"v1","addr":"172.21.15.106:6827","nonce":3815586576}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":3815586576},{"type":"v1","addr":"172.21.15.106:6831","nonce":3815586576}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":3815586576},{"type":"v1","addr":"172.21.15.106:6829","nonce":3815586576}]},"public_addr":"172.21.15.106:6825/3815586576","cluster_addr":"172.21.15.106:6827/3815586576","heartbeat_back_addr":"172.21.15.106:6831/3815586576","heartbeat_front_addr":"172.21.15.106:6829/3815586576","state":["exists","up"]},{"osd":11,"uuid":"1f8b9a43-57fa-41f4-9969-3b803a74bf45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":2009632843},{"type":"v1","addr":"172.21.15.191:6825","nonce":2009632843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":2009632843},{"type":"v1","addr":"172.21.15.191:6827","nonce":2009632843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":2009632843},{"type":"v1","addr":"172.21.15.191:6831","nonce":2009632843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":2009632843},{"type":"v1","addr":"172.21.15.191:6829","nonce":2009632843}]},"public_addr":"172.21.15.191:6825/2009632843","cluster_addr":"172.21.15.191:6827/2009632843","heartbeat_back_addr":"172.21.15.191:6831/2009632843","heartbeat_front_addr":"172.21.15.191:6829/2009632843","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T09:23:10.894763+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T09:23:34.420055+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T09:23:58.288324+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T09:24:23.673457+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T09:24:48.208561+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T09:25:13.405473+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T09:25:38.670915+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T09:26:04.451257+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T09:26:30.281380+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T09:26:56.543143+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T09:27:22.412930+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T09:27:48.003866+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.1d","mappings":[{"from":2,"to":1},{"from":8,"to":11}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.20","mappings":[{"from":5,"to":9}]},{"pgid":"3.21","mappings":[{"from":4,"to":6}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]},{"pgid":"4.9","mappings":[{"from":5,"to":8}]},{"pgid":"4.17","mappings":[{"from":9,"to":8}]},{"pgid":"4.1b","mappings":[{"from":9,"to":10}]},{"pgid":"4.1f","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.52:0/1035878034":"2024-03-05T09:21:12.760938+0000","172.21.15.52:0/2743084243":"2024-03-05T09:21:39.401935+0000","172.21.15.52:0/1094295190":"2024-03-05T09:31:08.422018+0000","172.21.15.52:0/568242577":"2024-03-05T09:21:12.760938+0000","172.21.15.52:0/2080168931":"2024-03-05T09:31:08.422018+0000","172.21.15.52:6801/2425730636":"2024-03-05T09:21:39.401935+0000","172.21.15.52:6800/2379361333":"2024-03-05T09:21:12.760938+0000","172.21.15.52:6800/3036033923":"2024-03-05T09:31:08.422018+0000","172.21.15.52:0/497075075":"2024-03-05T09:21:12.760938+0000","172.21.15.52:6800/2425730636":"2024-03-05T09:21:39.401935+0000","172.21.15.52:6801/2379361333":"2024-03-05T09:21:12.760938+0000","172.21.15.52:0/1271949988":"2024-03-05T09:21:39.401935+0000","172.21.15.52:0/2491451071":"2024-03-05T09:31:08.422018+0000","172.21.15.52:0/2466768":"2024-03-05T09:31:08.422018+0000","172.21.15.52:0/4082645718":"2024-03-05T09:21:39.401935+0000","172.21.15.52:0/2935914357":"2024-03-05T09:31:08.422018+0000","172.21.15.52:6801/3036033923":"2024-03-05T09:31:08.422018+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":5,"length":1},{"begin":15,"length":1}]},{"pool":4,"snaps":[{"begin":5,"length":1},{"begin":15,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-04T09:41:53.161 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:53.162+0000 7f21c5688640 1 -- 172.21.15.52:0/1551426807 >> [v2:172.21.15.52:6800/478031757,v1:172.21.15.52:6801/478031757] conn(0x7f21ac0a1ef0 msgr2=0x7f21ac0a43b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:41:53.161 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:53.162+0000 7f21c5688640 1 --2- 172.21.15.52:0/1551426807 >> [v2:172.21.15.52:6800/478031757,v1:172.21.15.52:6801/478031757] conn(0x7f21ac0a1ef0 0x7f21ac0a43b0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f21b0009230 tx=0x7f21b0005fd0 comp rx=0 tx=0).stop 2024-03-04T09:41:53.161 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:53.162+0000 7f21c5688640 1 -- 172.21.15.52:0/1551426807 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f21c0107640 msgr2=0x7f21c019c710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:41:53.161 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:53.162+0000 7f21c5688640 1 --2- 172.21.15.52:0/1551426807 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f21c0107640 0x7f21c019c710 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f21b800eb10 tx=0x7f21b800c2d0 comp rx=0 tx=0).stop 2024-03-04T09:41:53.162 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:53.162+0000 7f21c5688640 1 -- 172.21.15.52:0/1551426807 shutdown_connections 2024-03-04T09:41:53.162 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:53.162+0000 7f21c5688640 1 --2- 172.21.15.52:0/1551426807 >> [v2:172.21.15.52:6800/478031757,v1:172.21.15.52:6801/478031757] conn(0x7f21ac0a1ef0 0x7f21ac0a43b0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:41:53.162 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:53.162+0000 7f21c5688640 1 --2- 172.21.15.52:0/1551426807 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f21c0107640 0x7f21c019c710 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:41:53.162 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:53.162+0000 7f21c5688640 1 --2- 172.21.15.52:0/1551426807 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f21c0101f40 0x7f21c01983e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:41:53.162 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:53.162+0000 7f21c5688640 1 --2- 172.21.15.52:0/1551426807 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f21c0101580 0x7f21c0197ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:41:53.162 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:53.162+0000 7f21c5688640 1 -- 172.21.15.52:0/1551426807 >> 172.21.15.52:0/1551426807 conn(0x7f21c00fd070 msgr2=0x7f21c00fe870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:41:53.162 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:53.162+0000 7f21c5688640 1 -- 172.21.15.52:0/1551426807 shutdown_connections 2024-03-04T09:41:53.162 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:53.162+0000 7f21c5688640 1 -- 172.21.15.52:0/1551426807 wait complete. 2024-03-04T09:41:53.172 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-03-04T09:41:53.278 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:53.278+0000 7f08f8d93640 1 -- 172.21.15.52:0/125916873 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f08f4069c80 msgr2=0x7f08f406a100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:41:53.279 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:53.278+0000 7f08f8d93640 1 --2- 172.21.15.52:0/125916873 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f08f4069c80 0x7f08f406a100 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f08e8009900 tx=0x7f08e802bc30 comp rx=0 tx=0).stop 2024-03-04T09:41:53.279 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:53.278+0000 7f08f8d93640 1 -- 172.21.15.52:0/125916873 shutdown_connections 2024-03-04T09:41:53.279 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:53.278+0000 7f08f8d93640 1 --2- 172.21.15.52:0/125916873 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f08f406a640 0x7f08f410e400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:41:53.279 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:53.278+0000 7f08f8d93640 1 --2- 172.21.15.52:0/125916873 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f08f4069c80 0x7f08f406a100 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:41:53.279 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:53.278+0000 7f08f8d93640 1 --2- 172.21.15.52:0/125916873 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f08f4107640 0x7f08f4107a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:41:53.279 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:53.278+0000 7f08f8d93640 1 -- 172.21.15.52:0/125916873 >> 172.21.15.52:0/125916873 conn(0x7f08f40775a0 msgr2=0x7f08f40779b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:41:53.279 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:53.278+0000 7f08f8d93640 1 -- 172.21.15.52:0/125916873 shutdown_connections 2024-03-04T09:41:53.280 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:53.278+0000 7f08f8d93640 1 -- 172.21.15.52:0/125916873 wait complete. 2024-03-04T09:41:53.280 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:53.279+0000 7f08f8d93640 1 Processor -- start 2024-03-04T09:41:53.280 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:53.279+0000 7f08f8d93640 1 -- start start 2024-03-04T09:41:53.280 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:53.279+0000 7f08f8d93640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f08f4069c80 0x7f08f4197dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:41:53.280 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:53.279+0000 7f08f8d93640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f08f406a640 0x7f08f4198300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:41:53.280 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:53.279+0000 7f08f8d93640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f08f4107640 0x7f08f419c680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:41:53.280 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:53.279+0000 7f08f8d93640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08f4111900 con 0x7f08f406a640 2024-03-04T09:41:53.281 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:53.279+0000 7f08f8d93640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08f4111780 con 0x7f08f4069c80 2024-03-04T09:41:53.281 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:53.279+0000 7f08f8d93640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08f4111a80 con 0x7f08f4107640 2024-03-04T09:41:53.281 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:53.279+0000 7f08f1d9b640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f08f406a640 0x7f08f4198300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:41:53.281 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:53.279+0000 7f08f1d9b640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f08f406a640 0x7f08f4198300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:54180/0 (socket says 172.21.15.52:54180) 2024-03-04T09:41:53.281 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:53.279+0000 7f08f1d9b640 1 -- 172.21.15.52:0/3896569509 learned_addr learned my addr 172.21.15.52:0/3896569509 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-04T09:41:53.281 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:53.279+0000 7f08f259c640 1 --2- 172.21.15.52:0/3896569509 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f08f4069c80 0x7f08f4197dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:41:53.281 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:53.279+0000 7f08f2d9d640 1 --2- 172.21.15.52:0/3896569509 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f08f4107640 0x7f08f419c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:41:53.281 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:53.279+0000 7f08f1d9b640 1 -- 172.21.15.52:0/3896569509 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f08f4107640 msgr2=0x7f08f419c680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:41:53.281 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:53.279+0000 7f08f1d9b640 1 --2- 172.21.15.52:0/3896569509 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f08f4107640 0x7f08f419c680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:41:53.282 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:53.279+0000 7f08f1d9b640 1 -- 172.21.15.52:0/3896569509 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f08f4069c80 msgr2=0x7f08f4197dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:41:53.282 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:53.279+0000 7f08f1d9b640 1 --2- 172.21.15.52:0/3896569509 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f08f4069c80 0x7f08f4197dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:41:53.282 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:53.279+0000 7f08f1d9b640 1 -- 172.21.15.52:0/3896569509 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08f419cdb0 con 0x7f08f406a640 2024-03-04T09:41:53.282 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:53.279+0000 7f08f259c640 1 --2- 172.21.15.52:0/3896569509 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f08f4069c80 0x7f08f4197dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-04T09:41:53.282 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:53.279+0000 7f08f1d9b640 1 --2- 172.21.15.52:0/3896569509 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f08f406a640 0x7f08f4198300 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f08e8005940 tx=0x7f08e8004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:41:53.282 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:53.280+0000 7f08e37fe640 1 -- 172.21.15.52:0/3896569509 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08e8041070 con 0x7f08f406a640 2024-03-04T09:41:53.282 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:53.280+0000 7f08f8d93640 1 -- 172.21.15.52:0/3896569509 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08f419d040 con 0x7f08f406a640 2024-03-04T09:41:53.282 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:53.280+0000 7f08e37fe640 1 -- 172.21.15.52:0/3896569509 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f08e80054e0 con 0x7f08f406a640 2024-03-04T09:41:53.283 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:53.280+0000 7f08e37fe640 1 -- 172.21.15.52:0/3896569509 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08e8003d00 con 0x7f08f406a640 2024-03-04T09:41:53.283 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:53.280+0000 7f08f8d93640 1 -- 172.21.15.52:0/3896569509 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08f41a4930 con 0x7f08f406a640 2024-03-04T09:41:53.283 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:53.281+0000 7f08f8d93640 1 -- 172.21.15.52:0/3896569509 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f08f4102ca0 con 0x7f08f406a640 2024-03-04T09:41:53.284 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:53.282+0000 7f08e37fe640 1 -- 172.21.15.52:0/3896569509 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 25) v1 ==== 136887+0+0 (secure 0 0 0) 0x7f08e8044070 con 0x7f08f406a640 2024-03-04T09:41:53.284 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:53.282+0000 7f08e37fe640 1 --2- 172.21.15.52:0/3896569509 >> [v2:172.21.15.52:6800/478031757,v1:172.21.15.52:6801/478031757] conn(0x7f08dc0a1ef0 0x7f08dc0a43b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:41:53.284 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:53.282+0000 7f08e37fe640 1 -- 172.21.15.52:0/3896569509 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 9547+0+0 (secure 0 0 0) 0x7f08e80f0370 con 0x7f08f406a640 2024-03-04T09:41:53.284 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:53.282+0000 7f08f259c640 1 --2- 172.21.15.52:0/3896569509 >> [v2:172.21.15.52:6800/478031757,v1:172.21.15.52:6801/478031757] conn(0x7f08dc0a1ef0 0x7f08dc0a43b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:41:53.284 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:53.283+0000 7f08f259c640 1 --2- 172.21.15.52:0/3896569509 >> [v2:172.21.15.52:6800/478031757,v1:172.21.15.52:6801/478031757] conn(0x7f08dc0a1ef0 0x7f08dc0a43b0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f08e4005fd0 tx=0x7f08e4009500 comp rx=0 tx=0).ready entity=mgr.15486 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:41:53.284 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:53.286+0000 7f08e37fe640 1 -- 172.21.15.52:0/3896569509 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f08e80bcb20 con 0x7f08f406a640 2024-03-04T09:41:53.504 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:53.506+0000 7f08f8d93640 1 -- 172.21.15.52:0/3896569509 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f08f4068810 con 0x7f08f406a640 2024-03-04T09:41:53.506 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:53.508+0000 7f08e37fe640 1 -- 172.21.15.52:0/3896569509 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v29) v1 ==== 161+0+0 (secure 0 0 0) 0x7f08f4068810 con 0x7f08f406a640 2024-03-04T09:41:53.506 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:41:53 smithi052 ceph-mon[32473]: from='client.? 172.21.15.52:0/1553528380' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-04T09:41:53.506 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:41:53 smithi052 ceph-mon[32473]: from='client.? 172.21.15.52:0/49253365' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-04T09:41:53.506 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:41:53 smithi052 ceph-mon[32473]: from='client.? 172.21.15.52:0/1551426807' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-04T09:41:53.507 INFO:teuthology.orchestra.run.smithi052.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-03-04T09:41:53.508 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:53.511+0000 7f08f8d93640 1 -- 172.21.15.52:0/3896569509 >> [v2:172.21.15.52:6800/478031757,v1:172.21.15.52:6801/478031757] conn(0x7f08dc0a1ef0 msgr2=0x7f08dc0a43b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:41:53.509 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:53.511+0000 7f08f8d93640 1 --2- 172.21.15.52:0/3896569509 >> [v2:172.21.15.52:6800/478031757,v1:172.21.15.52:6801/478031757] conn(0x7f08dc0a1ef0 0x7f08dc0a43b0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f08e4005fd0 tx=0x7f08e4009500 comp rx=0 tx=0).stop 2024-03-04T09:41:53.509 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:53.511+0000 7f08f8d93640 1 -- 172.21.15.52:0/3896569509 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f08f406a640 msgr2=0x7f08f4198300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:41:53.509 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:53.511+0000 7f08f8d93640 1 --2- 172.21.15.52:0/3896569509 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f08f406a640 0x7f08f4198300 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f08e8005940 tx=0x7f08e8004270 comp rx=0 tx=0).stop 2024-03-04T09:41:53.509 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:53.511+0000 7f08f8d93640 1 -- 172.21.15.52:0/3896569509 shutdown_connections 2024-03-04T09:41:53.509 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:53.511+0000 7f08f8d93640 1 --2- 172.21.15.52:0/3896569509 >> [v2:172.21.15.52:6800/478031757,v1:172.21.15.52:6801/478031757] conn(0x7f08dc0a1ef0 0x7f08dc0a43b0 secure :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f08e4005fd0 tx=0x7f08e4009500 comp rx=0 tx=0).stop 2024-03-04T09:41:53.509 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:53.511+0000 7f08f8d93640 1 --2- 172.21.15.52:0/3896569509 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f08f4107640 0x7f08f419c680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:41:53.509 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:53.511+0000 7f08f8d93640 1 --2- 172.21.15.52:0/3896569509 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f08f406a640 0x7f08f4198300 secure :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f08e8005940 tx=0x7f08e8004270 comp rx=0 tx=0).stop 2024-03-04T09:41:53.509 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:53.511+0000 7f08f8d93640 1 --2- 172.21.15.52:0/3896569509 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f08f4069c80 0x7f08f4197dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:41:53.510 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:53.511+0000 7f08f8d93640 1 -- 172.21.15.52:0/3896569509 >> 172.21.15.52:0/3896569509 conn(0x7f08f40775a0 msgr2=0x7f08f40ff150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:41:53.510 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:53.511+0000 7f08f8d93640 1 -- 172.21.15.52:0/3896569509 shutdown_connections 2024-03-04T09:41:53.510 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:53.512+0000 7f08f8d93640 1 -- 172.21.15.52:0/3896569509 wait complete. 2024-03-04T09:41:53.521 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-03-04T09:41:53.547 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:41:53 smithi106 ceph-mon[36484]: from='client.? 172.21.15.52:0/1553528380' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-04T09:41:53.547 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:41:53 smithi106 ceph-mon[36484]: from='client.? 172.21.15.52:0/49253365' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-04T09:41:53.547 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:41:53 smithi106 ceph-mon[36484]: from='client.? 172.21.15.52:0/1551426807' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-04T09:41:53.549 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:41:53 smithi191 ceph-mon[36816]: from='client.? 172.21.15.52:0/1553528380' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-04T09:41:53.549 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:41:53 smithi191 ceph-mon[36816]: from='client.? 172.21.15.52:0/49253365' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-04T09:41:53.549 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:41:53 smithi191 ceph-mon[36816]: from='client.? 172.21.15.52:0/1551426807' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-04T09:41:53.629 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:53.629+0000 7f8857556640 1 -- 172.21.15.52:0/2400785548 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8850100e00 msgr2=0x7f885010d310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:41:53.629 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:53.629+0000 7f8857556640 1 --2- 172.21.15.52:0/2400785548 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8850100e00 0x7f885010d310 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f8848009cd0 tx=0x7f884802c0f0 comp rx=0 tx=0).stop 2024-03-04T09:41:53.629 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:53.629+0000 7f8857556640 1 -- 172.21.15.52:0/2400785548 shutdown_connections 2024-03-04T09:41:53.629 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:53.629+0000 7f8857556640 1 --2- 172.21.15.52:0/2400785548 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8850100e00 0x7f885010d310 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:41:53.629 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:53.629+0000 7f8857556640 1 --2- 172.21.15.52:0/2400785548 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8850100440 0x7f88501008c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:41:53.629 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:53.629+0000 7f8857556640 1 --2- 172.21.15.52:0/2400785548 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f88501064e0 0x7f88501068c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:41:53.630 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:53.629+0000 7f8857556640 1 -- 172.21.15.52:0/2400785548 >> 172.21.15.52:0/2400785548 conn(0x7f88500775d0 msgr2=0x7f88500779e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:41:53.630 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:53.629+0000 7f8857556640 1 -- 172.21.15.52:0/2400785548 shutdown_connections 2024-03-04T09:41:53.630 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:53.629+0000 7f8857556640 1 -- 172.21.15.52:0/2400785548 wait complete. 2024-03-04T09:41:53.630 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:53.629+0000 7f8857556640 1 Processor -- start 2024-03-04T09:41:53.630 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:53.629+0000 7f8857556640 1 -- start start 2024-03-04T09:41:53.630 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:53.629+0000 7f8857556640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8850100440 0x7f8850197da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:41:53.630 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:53.629+0000 7f8857556640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8850100e00 0x7f88501982e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:41:53.630 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:53.629+0000 7f8857556640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f88501064e0 0x7f885019c660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:41:53.631 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:53.629+0000 7f8857556640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8850111960 con 0x7f88501064e0 2024-03-04T09:41:53.631 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:53.629+0000 7f8857556640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88501117e0 con 0x7f8850100440 2024-03-04T09:41:53.631 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:53.629+0000 7f8857556640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8850111ae0 con 0x7f8850100e00 2024-03-04T09:41:53.631 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:53.630+0000 7f8855af3640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f88501064e0 0x7f885019c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:41:53.631 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:53.630+0000 7f8855af3640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f88501064e0 0x7f885019c660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:54204/0 (socket says 172.21.15.52:54204) 2024-03-04T09:41:53.631 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:53.630+0000 7f8855af3640 1 -- 172.21.15.52:0/13977471 learned_addr learned my addr 172.21.15.52:0/13977471 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-04T09:41:53.631 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:53.630+0000 7f8854af1640 1 --2- 172.21.15.52:0/13977471 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8850100e00 0x7f88501982e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:41:53.631 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:53.630+0000 7f88552f2640 1 --2- 172.21.15.52:0/13977471 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8850100440 0x7f8850197da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:41:53.632 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:53.630+0000 7f8855af3640 1 -- 172.21.15.52:0/13977471 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8850100e00 msgr2=0x7f88501982e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:41:53.632 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:53.630+0000 7f8855af3640 1 --2- 172.21.15.52:0/13977471 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8850100e00 0x7f88501982e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:41:53.632 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:53.630+0000 7f8855af3640 1 -- 172.21.15.52:0/13977471 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8850100440 msgr2=0x7f8850197da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:41:53.632 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:53.630+0000 7f8855af3640 1 --2- 172.21.15.52:0/13977471 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8850100440 0x7f8850197da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:41:53.632 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:53.630+0000 7f8855af3640 1 -- 172.21.15.52:0/13977471 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f885019cd90 con 0x7f88501064e0 2024-03-04T09:41:53.632 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:53.630+0000 7f8854af1640 1 --2- 172.21.15.52:0/13977471 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8850100e00 0x7f88501982e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T09:41:53.632 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:53.630+0000 7f8855af3640 1 --2- 172.21.15.52:0/13977471 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f88501064e0 0x7f885019c660 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f8848004af0 tx=0x7f8848032ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:41:53.633 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:53.630+0000 7f88467fc640 1 -- 172.21.15.52:0/13977471 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8848041020 con 0x7f88501064e0 2024-03-04T09:41:53.633 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:53.630+0000 7f88467fc640 1 -- 172.21.15.52:0/13977471 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8848034cd0 con 0x7f88501064e0 2024-03-04T09:41:53.633 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:53.630+0000 7f8857556640 1 -- 172.21.15.52:0/13977471 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f885019d020 con 0x7f88501064e0 2024-03-04T09:41:53.633 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:53.630+0000 7f88467fc640 1 -- 172.21.15.52:0/13977471 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8848039040 con 0x7f88501064e0 2024-03-04T09:41:53.633 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:53.630+0000 7f8857556640 1 -- 172.21.15.52:0/13977471 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88501a4910 con 0x7f88501064e0 2024-03-04T09:41:53.634 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:53.631+0000 7f8857556640 1 -- 172.21.15.52:0/13977471 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f88500683c0 con 0x7f88501064e0 2024-03-04T09:41:53.634 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:53.632+0000 7f88467fc640 1 -- 172.21.15.52:0/13977471 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 25) v1 ==== 136887+0+0 (secure 0 0 0) 0x7f8848044020 con 0x7f88501064e0 2024-03-04T09:41:53.634 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:53.633+0000 7f88467fc640 1 --2- 172.21.15.52:0/13977471 >> [v2:172.21.15.52:6800/478031757,v1:172.21.15.52:6801/478031757] conn(0x7f883c0a1ef0 0x7f883c0a43b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:41:53.634 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:53.633+0000 7f88467fc640 1 -- 172.21.15.52:0/13977471 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 9717+0+0 (secure 0 0 0) 0x7f88480f03e0 con 0x7f88501064e0 2024-03-04T09:41:53.635 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:53.633+0000 7f88552f2640 1 --2- 172.21.15.52:0/13977471 >> [v2:172.21.15.52:6800/478031757,v1:172.21.15.52:6801/478031757] conn(0x7f883c0a1ef0 0x7f883c0a43b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:41:53.635 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:53.633+0000 7f88552f2640 1 --2- 172.21.15.52:0/13977471 >> [v2:172.21.15.52:6800/478031757,v1:172.21.15.52:6801/478031757] conn(0x7f883c0a1ef0 0x7f883c0a43b0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f884c009a20 tx=0x7f884c005f90 comp rx=0 tx=0).ready entity=mgr.15486 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:41:53.635 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:53.637+0000 7f88467fc640 1 -- 172.21.15.52:0/13977471 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f884804b020 con 0x7f88501064e0 2024-03-04T09:41:53.836 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:41:53 smithi052 ceph-4e059f08-da08-11ee-95c6-87774f69a715-mon-a[32469]: 2024-03-04T09:41:53.507+0000 7ff7cacaa700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-04T09:41:53.837 INFO:journalctl@ceph.mds.f.smithi191.stdout:Mar 04 09:41:53 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-mds-f[66771]: ignoring --setuser ceph since I am not root 2024-03-04T09:41:53.837 INFO:journalctl@ceph.mds.f.smithi191.stdout:Mar 04 09:41:53 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-mds-f[66771]: ignoring --setgroup ceph since I am not root 2024-03-04T09:41:53.838 INFO:journalctl@ceph.mds.f.smithi191.stdout:Mar 04 09:41:53 smithi191 ceph-mds[66775]: -- 172.21.15.191:0/2742178456 <== mon.1 v2:172.21.15.191:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x560dc4319a40 con 0x560dc42ecc00 2024-03-04T09:41:53.838 INFO:journalctl@ceph.mds.f.smithi191.stdout:Mar 04 09:41:53 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-mds-f[66771]: starting mds.f at 2024-03-04T09:41:53.838 INFO:journalctl@ceph.mds.i.smithi191.stdout:Mar 04 09:41:53 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-mds-i[67404]: ignoring --setuser ceph since I am not root 2024-03-04T09:41:53.838 INFO:journalctl@ceph.mds.i.smithi191.stdout:Mar 04 09:41:53 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-mds-i[67404]: ignoring --setgroup ceph since I am not root 2024-03-04T09:41:53.838 INFO:journalctl@ceph.mds.i.smithi191.stdout:Mar 04 09:41:53 smithi191 ceph-mds[67408]: -- 172.21.15.191:0/3669950268 <== mon.0 v2:172.21.15.52:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x56123c4a7a40 con 0x56123c47ac00 2024-03-04T09:41:53.839 INFO:journalctl@ceph.mds.i.smithi191.stdout:Mar 04 09:41:53 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-mds-i[67404]: starting mds.i at 2024-03-04T09:41:53.839 INFO:journalctl@ceph.mds.c.smithi191.stdout:Mar 04 09:41:53 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-mds-c[67083]: ignoring --setuser ceph since I am not root 2024-03-04T09:41:53.839 INFO:journalctl@ceph.mds.c.smithi191.stdout:Mar 04 09:41:53 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-mds-c[67083]: ignoring --setgroup ceph since I am not root 2024-03-04T09:41:53.839 INFO:journalctl@ceph.mds.c.smithi191.stdout:Mar 04 09:41:53 smithi191 ceph-mds[67087]: -- 172.21.15.191:0/505145868 <== mon.0 v2:172.21.15.52:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x562036e35a40 con 0x562036e08800 2024-03-04T09:41:53.839 INFO:journalctl@ceph.mds.c.smithi191.stdout:Mar 04 09:41:53 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-mds-c[67083]: starting mds.c at 2024-03-04T09:41:53.857 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:53.859+0000 7f8857556640 1 -- 172.21.15.52:0/13977471 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f88501089b0 con 0x7f88501064e0 2024-03-04T09:41:53.882 INFO:journalctl@ceph.mds.b.smithi106.stdout:Mar 04 09:41:53 smithi106 ceph-4e059f08-da08-11ee-95c6-87774f69a715-mds-b[66655]: ignoring --setuser ceph since I am not root 2024-03-04T09:41:53.882 INFO:journalctl@ceph.mds.b.smithi106.stdout:Mar 04 09:41:53 smithi106 ceph-4e059f08-da08-11ee-95c6-87774f69a715-mds-b[66655]: ignoring --setgroup ceph since I am not root 2024-03-04T09:41:53.882 INFO:journalctl@ceph.mds.b.smithi106.stdout:Mar 04 09:41:53 smithi106 ceph-mds[66659]: -- 172.21.15.106:0/324914690 <== mon.2 v2:172.21.15.106:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5616e0407a40 con 0x5616e03dac00 2024-03-04T09:41:53.883 INFO:journalctl@ceph.mds.b.smithi106.stdout:Mar 04 09:41:53 smithi106 ceph-4e059f08-da08-11ee-95c6-87774f69a715-mds-b[66655]: starting mds.b at 2024-03-04T09:41:53.883 INFO:journalctl@ceph.mds.e.smithi106.stdout:Mar 04 09:41:53 smithi106 ceph-4e059f08-da08-11ee-95c6-87774f69a715-mds-e[66027]: ignoring --setuser ceph since I am not root 2024-03-04T09:41:53.883 INFO:journalctl@ceph.mds.e.smithi106.stdout:Mar 04 09:41:53 smithi106 ceph-4e059f08-da08-11ee-95c6-87774f69a715-mds-e[66027]: ignoring --setgroup ceph since I am not root 2024-03-04T09:41:53.883 INFO:journalctl@ceph.mds.e.smithi106.stdout:Mar 04 09:41:53 smithi106 ceph-mds[66031]: -- 172.21.15.106:0/2358117859 <== mon.2 v2:172.21.15.106:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55cd75f2fa40 con 0x55cd75f03000 2024-03-04T09:41:53.883 INFO:journalctl@ceph.mds.e.smithi106.stdout:Mar 04 09:41:53 smithi106 ceph-4e059f08-da08-11ee-95c6-87774f69a715-mds-e[66027]: starting mds.e at 2024-03-04T09:41:53.883 INFO:journalctl@ceph.mds.h.smithi106.stdout:Mar 04 09:41:53 smithi106 ceph-4e059f08-da08-11ee-95c6-87774f69a715-mds-h[66334]: ignoring --setuser ceph since I am not root 2024-03-04T09:41:53.884 INFO:journalctl@ceph.mds.h.smithi106.stdout:Mar 04 09:41:53 smithi106 ceph-4e059f08-da08-11ee-95c6-87774f69a715-mds-h[66334]: ignoring --setgroup ceph since I am not root 2024-03-04T09:41:53.884 INFO:journalctl@ceph.mds.h.smithi106.stdout:Mar 04 09:41:53 smithi106 ceph-mds[66338]: -- 172.21.15.106:0/1556166573 <== mon.2 v2:172.21.15.106:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x562fcb671a40 con 0x562fcb644c00 2024-03-04T09:41:53.884 INFO:journalctl@ceph.mds.h.smithi106.stdout:Mar 04 09:41:53 smithi106 ceph-4e059f08-da08-11ee-95c6-87774f69a715-mds-h[66334]: starting mds.h at 2024-03-04T09:41:54.508 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:54.510+0000 7f88467fc640 1 -- 172.21.15.52:0/13977471 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v30) v1 ==== 104+0+0 (secure 0 0 0) 0x7f88480bd0d0 con 0x7f88501064e0 2024-03-04T09:41:54.511 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:54.512+0000 7f8857556640 1 -- 172.21.15.52:0/13977471 >> [v2:172.21.15.52:6800/478031757,v1:172.21.15.52:6801/478031757] conn(0x7f883c0a1ef0 msgr2=0x7f883c0a43b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:41:54.511 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:54.512+0000 7f8857556640 1 --2- 172.21.15.52:0/13977471 >> [v2:172.21.15.52:6800/478031757,v1:172.21.15.52:6801/478031757] conn(0x7f883c0a1ef0 0x7f883c0a43b0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f884c009a20 tx=0x7f884c005f90 comp rx=0 tx=0).stop 2024-03-04T09:41:54.511 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:54.512+0000 7f8857556640 1 -- 172.21.15.52:0/13977471 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f88501064e0 msgr2=0x7f885019c660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:41:54.511 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:54.512+0000 7f8857556640 1 --2- 172.21.15.52:0/13977471 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f88501064e0 0x7f885019c660 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f8848004af0 tx=0x7f8848032ec0 comp rx=0 tx=0).stop 2024-03-04T09:41:54.511 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:54.512+0000 7f8857556640 1 -- 172.21.15.52:0/13977471 shutdown_connections 2024-03-04T09:41:54.511 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:54.512+0000 7f8857556640 1 --2- 172.21.15.52:0/13977471 >> [v2:172.21.15.52:6800/478031757,v1:172.21.15.52:6801/478031757] conn(0x7f883c0a1ef0 0x7f883c0a43b0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:41:54.512 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:54.512+0000 7f8857556640 1 --2- 172.21.15.52:0/13977471 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f88501064e0 0x7f885019c660 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:41:54.512 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:54.512+0000 7f8857556640 1 --2- 172.21.15.52:0/13977471 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8850100e00 0x7f88501982e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:41:54.512 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:54.512+0000 7f8857556640 1 --2- 172.21.15.52:0/13977471 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8850100440 0x7f8850197da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:41:54.512 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:54.512+0000 7f8857556640 1 -- 172.21.15.52:0/13977471 >> 172.21.15.52:0/13977471 conn(0x7f88500775d0 msgr2=0x7f8850102b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:41:54.512 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:54.513+0000 7f8857556640 1 -- 172.21.15.52:0/13977471 shutdown_connections 2024-03-04T09:41:54.512 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:54.513+0000 7f8857556640 1 -- 172.21.15.52:0/13977471 wait complete. 2024-03-04T09:41:54.524 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:41:54 smithi052 ceph-mon[32473]: pgmap v333: 129 pgs: 129 active+clean; 78 MiB data, 691 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-04T09:41:54.525 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:41:54 smithi052 ceph-mon[32473]: from='client.? 172.21.15.52:0/3896569509' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-03-04T09:41:54.525 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:41:54 smithi052 ceph-mon[32473]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-03-04T09:41:54.525 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:41:54 smithi052 ceph-mon[32473]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-04T09:41:54.525 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:41:54 smithi052 ceph-mon[32473]: from='client.? 172.21.15.52:0/3896569509' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-03-04T09:41:54.525 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:41:54 smithi052 ceph-mon[32473]: osdmap e103: 12 total, 12 up, 12 in 2024-03-04T09:41:54.525 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:41:54 smithi052 ceph-mon[32473]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-03-04T09:41:54.525 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:41:54 smithi052 ceph-mon[32473]: from='client.? 172.21.15.52:0/13977471' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-03-04T09:41:54.527 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-03-04T09:41:54.587 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:41:54 smithi191 ceph-mon[36816]: pgmap v333: 129 pgs: 129 active+clean; 78 MiB data, 691 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-04T09:41:54.588 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:41:54 smithi191 ceph-mon[36816]: from='client.? 172.21.15.52:0/3896569509' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-03-04T09:41:54.588 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:41:54 smithi191 ceph-mon[36816]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-03-04T09:41:54.588 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:41:54 smithi191 ceph-mon[36816]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-04T09:41:54.588 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:41:54 smithi191 ceph-mon[36816]: from='client.? 172.21.15.52:0/3896569509' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-03-04T09:41:54.588 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:41:54 smithi191 ceph-mon[36816]: osdmap e103: 12 total, 12 up, 12 in 2024-03-04T09:41:54.589 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:41:54 smithi191 ceph-mon[36816]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-03-04T09:41:54.589 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:41:54 smithi191 ceph-mon[36816]: from='client.? 172.21.15.52:0/13977471' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-03-04T09:41:54.632 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:41:54 smithi106 ceph-mon[36484]: pgmap v333: 129 pgs: 129 active+clean; 78 MiB data, 691 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-04T09:41:55.014 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:41:54 smithi106 ceph-mon[36484]: from='client.? 172.21.15.52:0/3896569509' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-03-04T09:41:55.015 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:41:54 smithi106 ceph-mon[36484]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-03-04T09:41:55.015 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:41:54 smithi106 ceph-mon[36484]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-04T09:41:55.015 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:41:54 smithi106 ceph-mon[36484]: from='client.? 172.21.15.52:0/3896569509' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-03-04T09:41:55.015 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:41:54 smithi106 ceph-mon[36484]: osdmap e103: 12 total, 12 up, 12 in 2024-03-04T09:41:55.016 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:41:54 smithi106 ceph-mon[36484]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-03-04T09:41:55.016 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:41:54 smithi106 ceph-mon[36484]: from='client.? 172.21.15.52:0/13977471' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-03-04T09:41:55.020 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:54.641+0000 7f17f8b16640 1 -- 172.21.15.52:0/4120927832 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f17f41064e0 msgr2=0x7f17f41068c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:41:55.021 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:54.641+0000 7f17f8b16640 1 --2- 172.21.15.52:0/4120927832 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f17f41064e0 0x7f17f41068c0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f17e4009930 tx=0x7f17e402bc30 comp rx=0 tx=0).stop 2024-03-04T09:41:55.021 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:54.641+0000 7f17f8b16640 1 -- 172.21.15.52:0/4120927832 shutdown_connections 2024-03-04T09:41:55.021 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:54.641+0000 7f17f8b16640 1 --2- 172.21.15.52:0/4120927832 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f17f4100e00 0x7f17f410d310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:41:55.021 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:54.641+0000 7f17f8b16640 1 --2- 172.21.15.52:0/4120927832 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f17f4100440 0x7f17f41008c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:41:55.021 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:54.641+0000 7f17f8b16640 1 --2- 172.21.15.52:0/4120927832 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f17f41064e0 0x7f17f41068c0 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:41:55.022 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:54.641+0000 7f17f8b16640 1 -- 172.21.15.52:0/4120927832 >> 172.21.15.52:0/4120927832 conn(0x7f17f40775d0 msgr2=0x7f17f40779e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:41:55.022 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:54.641+0000 7f17f8b16640 1 -- 172.21.15.52:0/4120927832 shutdown_connections 2024-03-04T09:41:55.022 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:54.641+0000 7f17f8b16640 1 -- 172.21.15.52:0/4120927832 wait complete. 2024-03-04T09:41:55.022 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:54.641+0000 7f17f8b16640 1 Processor -- start 2024-03-04T09:41:55.022 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:54.642+0000 7f17f8b16640 1 -- start start 2024-03-04T09:41:55.023 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:54.642+0000 7f17f8b16640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f17f4100440 0x7f17f4199fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:41:55.023 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:54.642+0000 7f17f8b16640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f17f4100e00 0x7f17f419a520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:41:55.023 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:54.642+0000 7f17f8b16640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f17f41064e0 0x7f17f419e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:41:55.023 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:54.642+0000 7f17f8b16640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17f41107e0 con 0x7f17f4100e00 2024-03-04T09:41:55.023 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:54.642+0000 7f17f8b16640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17f4110660 con 0x7f17f4100440 2024-03-04T09:41:55.023 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:54.642+0000 7f17f8b16640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17f4110960 con 0x7f17f41064e0 2024-03-04T09:41:55.023 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:54.642+0000 7f17ebfff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f17f4100e00 0x7f17f419a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:41:55.024 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:54.642+0000 7f17ebfff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f17f4100e00 0x7f17f419a520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:56652/0 (socket says 172.21.15.52:56652) 2024-03-04T09:41:55.024 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:54.642+0000 7f17ebfff640 1 -- 172.21.15.52:0/3799156681 learned_addr learned my addr 172.21.15.52:0/3799156681 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-04T09:41:55.024 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:54.642+0000 7f17f259c640 1 --2- 172.21.15.52:0/3799156681 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f17f4100440 0x7f17f4199fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:41:55.024 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:54.642+0000 7f17f2d9d640 1 --2- 172.21.15.52:0/3799156681 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f17f41064e0 0x7f17f419e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:41:55.024 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:54.642+0000 7f17ebfff640 1 -- 172.21.15.52:0/3799156681 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f17f41064e0 msgr2=0x7f17f419e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:41:55.024 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:54.642+0000 7f17ebfff640 1 --2- 172.21.15.52:0/3799156681 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f17f41064e0 0x7f17f419e8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:41:55.025 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:54.642+0000 7f17ebfff640 1 -- 172.21.15.52:0/3799156681 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f17f4100440 msgr2=0x7f17f4199fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:41:55.025 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:54.642+0000 7f17ebfff640 1 --2- 172.21.15.52:0/3799156681 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f17f4100440 0x7f17f4199fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:41:55.025 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:54.642+0000 7f17ebfff640 1 -- 172.21.15.52:0/3799156681 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17f419efd0 con 0x7f17f4100e00 2024-03-04T09:41:55.025 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:54.642+0000 7f17ebfff640 1 --2- 172.21.15.52:0/3799156681 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f17f4100e00 0x7f17f419a520 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f17dc00d950 tx=0x7f17dc00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:41:55.026 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:54.642+0000 7f17eb7fe640 1 -- 172.21.15.52:0/3799156681 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17dc017070 con 0x7f17f4100e00 2024-03-04T09:41:55.026 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:54.643+0000 7f17f8b16640 1 -- 172.21.15.52:0/3799156681 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17f419f2c0 con 0x7f17f4100e00 2024-03-04T09:41:55.026 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:54.643+0000 7f17eb7fe640 1 -- 172.21.15.52:0/3799156681 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f17dc009c30 con 0x7f17f4100e00 2024-03-04T09:41:55.026 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:54.643+0000 7f17eb7fe640 1 -- 172.21.15.52:0/3799156681 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17dc00f6b0 con 0x7f17f4100e00 2024-03-04T09:41:55.026 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:54.643+0000 7f17f8b16640 1 -- 172.21.15.52:0/3799156681 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17f41a6b50 con 0x7f17f4100e00 2024-03-04T09:41:55.026 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:54.644+0000 7f17f8b16640 1 -- 172.21.15.52:0/3799156681 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f17f40683c0 con 0x7f17f4100e00 2024-03-04T09:41:55.027 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:54.645+0000 7f17eb7fe640 1 -- 172.21.15.52:0/3799156681 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 25) v1 ==== 136887+0+0 (secure 0 0 0) 0x7f17dc0109d0 con 0x7f17f4100e00 2024-03-04T09:41:55.027 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:54.645+0000 7f17eb7fe640 1 --2- 172.21.15.52:0/3799156681 >> [v2:172.21.15.52:6800/478031757,v1:172.21.15.52:6801/478031757] conn(0x7f17e00a1ef0 0x7f17e00a43b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:41:55.027 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:54.645+0000 7f17f259c640 1 --2- 172.21.15.52:0/3799156681 >> [v2:172.21.15.52:6800/478031757,v1:172.21.15.52:6801/478031757] conn(0x7f17e00a1ef0 0x7f17e00a43b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:41:55.027 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:54.645+0000 7f17eb7fe640 1 -- 172.21.15.52:0/3799156681 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 9717+0+0 (secure 0 0 0) 0x7f17dc0ce1e0 con 0x7f17f4100e00 2024-03-04T09:41:55.027 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:54.646+0000 7f17f259c640 1 --2- 172.21.15.52:0/3799156681 >> [v2:172.21.15.52:6800/478031757,v1:172.21.15.52:6801/478031757] conn(0x7f17e00a1ef0 0x7f17e00a43b0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f17e4005ab0 tx=0x7f17e4005fd0 comp rx=0 tx=0).ready entity=mgr.15486 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:41:55.028 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:54.650+0000 7f17eb7fe640 1 -- 172.21.15.52:0/3799156681 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f17dc009de0 con 0x7f17f4100e00 2024-03-04T09:41:55.028 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:54.832+0000 7f17f8b16640 1 -- 172.21.15.52:0/3799156681 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f17f41a6d80 con 0x7f17f4100e00 2024-03-04T09:41:55.513 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:55.515+0000 7f17eb7fe640 1 -- 172.21.15.52:0/3799156681 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v104) v1 ==== 181+0+0 (secure 0 0 0) 0x7f17dc09a7d0 con 0x7f17f4100e00 2024-03-04T09:41:55.513 INFO:teuthology.orchestra.run.smithi052.stderr:pool 'cephfs_metadata' removed 2024-03-04T09:41:55.516 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:55.518+0000 7f17f8b16640 1 -- 172.21.15.52:0/3799156681 >> [v2:172.21.15.52:6800/478031757,v1:172.21.15.52:6801/478031757] conn(0x7f17e00a1ef0 msgr2=0x7f17e00a43b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:41:55.516 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:55.518+0000 7f17f8b16640 1 --2- 172.21.15.52:0/3799156681 >> [v2:172.21.15.52:6800/478031757,v1:172.21.15.52:6801/478031757] conn(0x7f17e00a1ef0 0x7f17e00a43b0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f17e4005ab0 tx=0x7f17e4005fd0 comp rx=0 tx=0).stop 2024-03-04T09:41:55.516 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:55.518+0000 7f17f8b16640 1 -- 172.21.15.52:0/3799156681 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f17f4100e00 msgr2=0x7f17f419a520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:41:55.517 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:55.518+0000 7f17f8b16640 1 --2- 172.21.15.52:0/3799156681 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f17f4100e00 0x7f17f419a520 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f17dc00d950 tx=0x7f17dc00de10 comp rx=0 tx=0).stop 2024-03-04T09:41:55.517 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:55.519+0000 7f17f8b16640 1 -- 172.21.15.52:0/3799156681 shutdown_connections 2024-03-04T09:41:55.517 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:55.519+0000 7f17f8b16640 1 --2- 172.21.15.52:0/3799156681 >> [v2:172.21.15.52:6800/478031757,v1:172.21.15.52:6801/478031757] conn(0x7f17e00a1ef0 0x7f17e00a43b0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:41:55.517 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:55.519+0000 7f17f8b16640 1 --2- 172.21.15.52:0/3799156681 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f17f41064e0 0x7f17f419e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:41:55.517 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:55.519+0000 7f17f8b16640 1 --2- 172.21.15.52:0/3799156681 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f17f4100e00 0x7f17f419a520 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:41:55.517 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:55.519+0000 7f17f8b16640 1 --2- 172.21.15.52:0/3799156681 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f17f4100440 0x7f17f4199fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:41:55.517 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:55.519+0000 7f17f8b16640 1 -- 172.21.15.52:0/3799156681 >> 172.21.15.52:0/3799156681 conn(0x7f17f40775d0 msgr2=0x7f17f410c100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:41:55.518 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:55.519+0000 7f17f8b16640 1 -- 172.21.15.52:0/3799156681 shutdown_connections 2024-03-04T09:41:55.518 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:55.519+0000 7f17f8b16640 1 -- 172.21.15.52:0/3799156681 wait complete. 2024-03-04T09:41:55.531 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:41:55 smithi052 ceph-mon[32473]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-03-04T09:41:55.532 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:41:55 smithi052 ceph-mon[32473]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-03-04T09:41:55.532 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:41:55 smithi052 ceph-mon[32473]: Cluster is now healthy 2024-03-04T09:41:55.532 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:41:55 smithi052 ceph-mon[32473]: from='client.? 172.21.15.52:0/13977471' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-03-04T09:41:55.532 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:41:55 smithi052 ceph-mon[32473]: mds.? [v2:172.21.15.191:6838/919292478,v1:172.21.15.191:6839/919292478] up:boot 2024-03-04T09:41:55.532 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:41:55 smithi052 ceph-mon[32473]: mds.? [v2:172.21.15.106:6836/717946048,v1:172.21.15.106:6837/717946048] up:boot 2024-03-04T09:41:55.532 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:41:55 smithi052 ceph-mon[32473]: mds.? [v2:172.21.15.106:6834/2201601049,v1:172.21.15.106:6835/2201601049] up:boot 2024-03-04T09:41:55.532 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:41:55 smithi052 ceph-mon[32473]: mds.? [v2:172.21.15.191:6834/3318252703,v1:172.21.15.191:6836/3318252703] up:boot 2024-03-04T09:41:55.532 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:41:55 smithi052 ceph-mon[32473]: mds.? [v2:172.21.15.191:6835/425490082,v1:172.21.15.191:6837/425490082] up:boot 2024-03-04T09:41:55.533 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:41:55 smithi052 ceph-mon[32473]: mds.? [v2:172.21.15.106:6838/574975903,v1:172.21.15.106:6839/574975903] up:boot 2024-03-04T09:41:55.533 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:41:55 smithi052 ceph-mon[32473]: fsmap 12 up:standby 2024-03-04T09:41:55.533 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:41:55 smithi052 ceph-mon[32473]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-03-04T09:41:55.533 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:41:55 smithi052 ceph-mon[32473]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-03-04T09:41:55.533 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:41:55 smithi052 ceph-mon[32473]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-03-04T09:41:55.533 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:41:55 smithi052 ceph-mon[32473]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-03-04T09:41:55.533 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:41:55 smithi052 ceph-mon[32473]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-04T09:41:55.533 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:41:55 smithi052 ceph-mon[32473]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-03-04T09:41:55.534 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:41:55 smithi052 ceph-mon[32473]: from='client.? 172.21.15.52:0/3799156681' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-04T09:41:55.535 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-03-04T09:41:55.587 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:41:55 smithi191 ceph-mon[36816]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-03-04T09:41:55.588 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:41:55 smithi191 ceph-mon[36816]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-03-04T09:41:55.588 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:41:55 smithi191 ceph-mon[36816]: Cluster is now healthy 2024-03-04T09:41:55.588 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:41:55 smithi191 ceph-mon[36816]: from='client.? 172.21.15.52:0/13977471' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-03-04T09:41:55.588 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:41:55 smithi191 ceph-mon[36816]: mds.? [v2:172.21.15.191:6838/919292478,v1:172.21.15.191:6839/919292478] up:boot 2024-03-04T09:41:55.588 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:41:55 smithi191 ceph-mon[36816]: mds.? [v2:172.21.15.106:6836/717946048,v1:172.21.15.106:6837/717946048] up:boot 2024-03-04T09:41:55.588 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:41:55 smithi191 ceph-mon[36816]: mds.? [v2:172.21.15.106:6834/2201601049,v1:172.21.15.106:6835/2201601049] up:boot 2024-03-04T09:41:55.588 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:41:55 smithi191 ceph-mon[36816]: mds.? [v2:172.21.15.191:6834/3318252703,v1:172.21.15.191:6836/3318252703] up:boot 2024-03-04T09:41:55.589 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:41:55 smithi191 ceph-mon[36816]: mds.? [v2:172.21.15.191:6835/425490082,v1:172.21.15.191:6837/425490082] up:boot 2024-03-04T09:41:55.589 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:41:55 smithi191 ceph-mon[36816]: mds.? [v2:172.21.15.106:6838/574975903,v1:172.21.15.106:6839/574975903] up:boot 2024-03-04T09:41:55.589 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:41:55 smithi191 ceph-mon[36816]: fsmap 12 up:standby 2024-03-04T09:41:55.589 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:41:55 smithi191 ceph-mon[36816]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-03-04T09:41:55.589 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:41:55 smithi191 ceph-mon[36816]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-03-04T09:41:55.589 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:41:55 smithi191 ceph-mon[36816]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-03-04T09:41:55.589 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:41:55 smithi191 ceph-mon[36816]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-03-04T09:41:55.589 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:41:55 smithi191 ceph-mon[36816]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-04T09:41:55.589 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:41:55 smithi191 ceph-mon[36816]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-03-04T09:41:55.590 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:41:55 smithi191 ceph-mon[36816]: from='client.? 172.21.15.52:0/3799156681' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-04T09:41:55.632 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:41:55 smithi106 ceph-mon[36484]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-03-04T09:41:55.632 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:41:55 smithi106 ceph-mon[36484]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-03-04T09:41:55.632 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:41:55 smithi106 ceph-mon[36484]: Cluster is now healthy 2024-03-04T09:41:55.633 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:41:55 smithi106 ceph-mon[36484]: from='client.? 172.21.15.52:0/13977471' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-03-04T09:41:55.633 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:41:55 smithi106 ceph-mon[36484]: mds.? [v2:172.21.15.191:6838/919292478,v1:172.21.15.191:6839/919292478] up:boot 2024-03-04T09:41:55.633 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:41:55 smithi106 ceph-mon[36484]: mds.? [v2:172.21.15.106:6836/717946048,v1:172.21.15.106:6837/717946048] up:boot 2024-03-04T09:41:55.633 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:41:55 smithi106 ceph-mon[36484]: mds.? [v2:172.21.15.106:6834/2201601049,v1:172.21.15.106:6835/2201601049] up:boot 2024-03-04T09:41:55.633 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:41:55 smithi106 ceph-mon[36484]: mds.? [v2:172.21.15.191:6834/3318252703,v1:172.21.15.191:6836/3318252703] up:boot 2024-03-04T09:41:55.633 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:41:55 smithi106 ceph-mon[36484]: mds.? [v2:172.21.15.191:6835/425490082,v1:172.21.15.191:6837/425490082] up:boot 2024-03-04T09:41:55.633 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:41:55 smithi106 ceph-mon[36484]: mds.? [v2:172.21.15.106:6838/574975903,v1:172.21.15.106:6839/574975903] up:boot 2024-03-04T09:41:55.633 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:41:55 smithi106 ceph-mon[36484]: fsmap 12 up:standby 2024-03-04T09:41:55.634 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:41:55 smithi106 ceph-mon[36484]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-03-04T09:41:55.634 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:41:55 smithi106 ceph-mon[36484]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-03-04T09:41:55.634 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:41:55 smithi106 ceph-mon[36484]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-03-04T09:41:55.634 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:41:55 smithi106 ceph-mon[36484]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-03-04T09:41:55.634 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:41:55 smithi106 ceph-mon[36484]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-04T09:41:55.634 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:41:55 smithi106 ceph-mon[36484]: from='mgr.15486 172.21.15.52:0/3877536348' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-03-04T09:41:55.634 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:41:55 smithi106 ceph-mon[36484]: from='client.? 172.21.15.52:0/3799156681' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-04T09:41:55.650 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:55.650+0000 7f9a2fb32640 1 -- 172.21.15.52:0/188831308 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9a28100630 msgr2=0x7f9a28100a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:41:55.650 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:55.650+0000 7f9a2fb32640 1 --2- 172.21.15.52:0/188831308 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9a28100630 0x7f9a28100a10 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f9a24009960 tx=0x7f9a2402bca0 comp rx=0 tx=0).stop 2024-03-04T09:41:55.651 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:55.650+0000 7f9a2fb32640 1 -- 172.21.15.52:0/188831308 shutdown_connections 2024-03-04T09:41:55.651 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:55.650+0000 7f9a2fb32640 1 --2- 172.21.15.52:0/188831308 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9a28105a30 0x7f9a28105ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:41:55.651 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:55.650+0000 7f9a2fb32640 1 --2- 172.21.15.52:0/188831308 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9a28100f50 0x7f9a28105410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:41:55.651 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:55.650+0000 7f9a2fb32640 1 --2- 172.21.15.52:0/188831308 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9a28100630 0x7f9a28100a10 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:41:55.651 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:55.650+0000 7f9a2fb32640 1 -- 172.21.15.52:0/188831308 >> 172.21.15.52:0/188831308 conn(0x7f9a28077630 msgr2=0x7f9a28077a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:41:55.651 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:55.650+0000 7f9a2fb32640 1 -- 172.21.15.52:0/188831308 shutdown_connections 2024-03-04T09:41:55.651 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:55.650+0000 7f9a2fb32640 1 -- 172.21.15.52:0/188831308 wait complete. 2024-03-04T09:41:55.652 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:55.651+0000 7f9a2fb32640 1 Processor -- start 2024-03-04T09:41:55.652 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:55.651+0000 7f9a2fb32640 1 -- start start 2024-03-04T09:41:55.652 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:55.651+0000 7f9a2fb32640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9a28100630 0x7f9a2819e530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:41:55.652 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:55.651+0000 7f9a2fb32640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9a28100f50 0x7f9a2819ea70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:41:55.652 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:55.651+0000 7f9a2fb32640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9a28105a30 0x7f9a281a2df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:41:55.652 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:55.651+0000 7f9a2fb32640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a28118030 con 0x7f9a28100f50 2024-03-04T09:41:55.652 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:55.651+0000 7f9a2fb32640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a28117eb0 con 0x7f9a28100630 2024-03-04T09:41:55.652 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:55.651+0000 7f9a2fb32640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a281181b0 con 0x7f9a28105a30 2024-03-04T09:41:55.653 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:55.651+0000 7f9a2d0cd640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9a28100f50 0x7f9a2819ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:41:55.653 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:55.651+0000 7f9a2d0cd640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9a28100f50 0x7f9a2819ea70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:56676/0 (socket says 172.21.15.52:56676) 2024-03-04T09:41:55.653 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:55.651+0000 7f9a2d0cd640 1 -- 172.21.15.52:0/1740833807 learned_addr learned my addr 172.21.15.52:0/1740833807 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-04T09:41:55.653 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:55.651+0000 7f9a2d8ce640 1 --2- 172.21.15.52:0/1740833807 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9a28100630 0x7f9a2819e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:41:55.653 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:55.651+0000 7f9a2e0cf640 1 --2- 172.21.15.52:0/1740833807 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9a28105a30 0x7f9a281a2df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:41:55.653 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:55.652+0000 7f9a2d0cd640 1 -- 172.21.15.52:0/1740833807 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9a28105a30 msgr2=0x7f9a281a2df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:41:55.653 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:55.652+0000 7f9a2d0cd640 1 --2- 172.21.15.52:0/1740833807 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9a28105a30 0x7f9a281a2df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:41:55.653 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:55.652+0000 7f9a2d0cd640 1 -- 172.21.15.52:0/1740833807 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9a28100630 msgr2=0x7f9a2819e530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:41:55.654 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:55.652+0000 7f9a2d0cd640 1 --2- 172.21.15.52:0/1740833807 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9a28100630 0x7f9a2819e530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:41:55.654 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:55.652+0000 7f9a2d0cd640 1 -- 172.21.15.52:0/1740833807 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a281a3520 con 0x7f9a28100f50 2024-03-04T09:41:55.654 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:55.652+0000 7f9a2e0cf640 1 --2- 172.21.15.52:0/1740833807 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9a28105a30 0x7f9a281a2df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T09:41:55.654 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:55.652+0000 7f9a2d0cd640 1 --2- 172.21.15.52:0/1740833807 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9a28100f50 0x7f9a2819ea70 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f9a1800d950 tx=0x7f9a1800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:41:55.654 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:55.652+0000 7f9a1effd640 1 -- 172.21.15.52:0/1740833807 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a18017070 con 0x7f9a28100f50 2024-03-04T09:41:55.654 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:55.652+0000 7f9a1effd640 1 -- 172.21.15.52:0/1740833807 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9a18009c30 con 0x7f9a28100f50 2024-03-04T09:41:55.654 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:55.652+0000 7f9a2fb32640 1 -- 172.21.15.52:0/1740833807 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a281a3810 con 0x7f9a28100f50 2024-03-04T09:41:55.654 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:55.652+0000 7f9a1effd640 1 -- 172.21.15.52:0/1740833807 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a1800f6b0 con 0x7f9a28100f50 2024-03-04T09:41:55.655 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:55.652+0000 7f9a2fb32640 1 -- 172.21.15.52:0/1740833807 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a281ab0a0 con 0x7f9a28100f50 2024-03-04T09:41:55.655 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:55.653+0000 7f9a2fb32640 1 -- 172.21.15.52:0/1740833807 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a28068810 con 0x7f9a28100f50 2024-03-04T09:41:55.656 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:55.654+0000 7f9a1effd640 1 -- 172.21.15.52:0/1740833807 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 25) v1 ==== 136887+0+0 (secure 0 0 0) 0x7f9a180109d0 con 0x7f9a28100f50 2024-03-04T09:41:55.656 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:55.655+0000 7f9a1effd640 1 --2- 172.21.15.52:0/1740833807 >> [v2:172.21.15.52:6800/478031757,v1:172.21.15.52:6801/478031757] conn(0x7f9a140a1ef0 0x7f9a140a43b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:41:55.656 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:55.655+0000 7f9a2d8ce640 1 --2- 172.21.15.52:0/1740833807 >> [v2:172.21.15.52:6800/478031757,v1:172.21.15.52:6801/478031757] conn(0x7f9a140a1ef0 0x7f9a140a43b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:41:55.656 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:55.655+0000 7f9a1effd640 1 -- 172.21.15.52:0/1740833807 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 9184+0+0 (secure 0 0 0) 0x7f9a180ce0b0 con 0x7f9a28100f50 2024-03-04T09:41:55.656 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:55.655+0000 7f9a2d8ce640 1 --2- 172.21.15.52:0/1740833807 >> [v2:172.21.15.52:6800/478031757,v1:172.21.15.52:6801/478031757] conn(0x7f9a140a1ef0 0x7f9a140a43b0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f9a240096c0 tx=0x7f9a24000bc0 comp rx=0 tx=0).ready entity=mgr.15486 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:41:55.657 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:55.659+0000 7f9a1effd640 1 -- 172.21.15.52:0/1740833807 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7f9a18009de0 con 0x7f9a28100f50 2024-03-04T09:41:55.850 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:55.852+0000 7f9a2fb32640 1 -- 172.21.15.52:0/1740833807 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f9a281ab2d0 con 0x7f9a28100f50 2024-03-04T09:41:56.489 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:41:56 smithi191 ceph-mon[36816]: pgmap v335: 129 pgs: 129 active+clean; 78 MiB data, 695 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 716 B/s wr, 6 op/s 2024-03-04T09:41:56.489 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:41:56 smithi191 ceph-mon[36816]: from='client.? 172.21.15.52:0/3799156681' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-03-04T09:41:56.489 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:41:56 smithi191 ceph-mon[36816]: osdmap e104: 12 total, 12 up, 12 in 2024-03-04T09:41:56.489 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:41:56 smithi191 ceph-mon[36816]: from='client.? 172.21.15.52:0/1740833807' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-04T09:41:56.514 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:56.516+0000 7f9a1effd640 1 -- 172.21.15.52:0/1740833807 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v105) v1 ==== 169+0+0 (secure 0 0 0) 0x7f9a1809aed0 con 0x7f9a28100f50 2024-03-04T09:41:56.515 INFO:teuthology.orchestra.run.smithi052.stderr:pool 'cephfs_data' removed 2024-03-04T09:41:56.519 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:56.520+0000 7f9a2fb32640 1 -- 172.21.15.52:0/1740833807 >> [v2:172.21.15.52:6800/478031757,v1:172.21.15.52:6801/478031757] conn(0x7f9a140a1ef0 msgr2=0x7f9a140a43b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:41:56.519 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:56.520+0000 7f9a2fb32640 1 --2- 172.21.15.52:0/1740833807 >> [v2:172.21.15.52:6800/478031757,v1:172.21.15.52:6801/478031757] conn(0x7f9a140a1ef0 0x7f9a140a43b0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f9a240096c0 tx=0x7f9a24000bc0 comp rx=0 tx=0).stop 2024-03-04T09:41:56.519 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:56.521+0000 7f9a2fb32640 1 -- 172.21.15.52:0/1740833807 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9a28100f50 msgr2=0x7f9a2819ea70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:41:56.519 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:56.521+0000 7f9a2fb32640 1 --2- 172.21.15.52:0/1740833807 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9a28100f50 0x7f9a2819ea70 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f9a1800d950 tx=0x7f9a1800de10 comp rx=0 tx=0).stop 2024-03-04T09:41:56.519 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:56.521+0000 7f9a2fb32640 1 -- 172.21.15.52:0/1740833807 shutdown_connections 2024-03-04T09:41:56.519 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:56.521+0000 7f9a2fb32640 1 --2- 172.21.15.52:0/1740833807 >> [v2:172.21.15.52:6800/478031757,v1:172.21.15.52:6801/478031757] conn(0x7f9a140a1ef0 0x7f9a140a43b0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:41:56.519 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:56.521+0000 7f9a2fb32640 1 --2- 172.21.15.52:0/1740833807 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9a28105a30 0x7f9a281a2df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:41:56.520 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:56.521+0000 7f9a2fb32640 1 --2- 172.21.15.52:0/1740833807 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9a28100f50 0x7f9a2819ea70 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:41:56.520 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:56.521+0000 7f9a2fb32640 1 --2- 172.21.15.52:0/1740833807 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9a28100630 0x7f9a2819e530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:41:56.520 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:56.521+0000 7f9a2fb32640 1 -- 172.21.15.52:0/1740833807 >> 172.21.15.52:0/1740833807 conn(0x7f9a28077630 msgr2=0x7f9a281033e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:41:56.520 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:56.521+0000 7f9a2fb32640 1 -- 172.21.15.52:0/1740833807 shutdown_connections 2024-03-04T09:41:56.520 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:56.521+0000 7f9a2fb32640 1 -- 172.21.15.52:0/1740833807 wait complete. 2024-03-04T09:41:56.533 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:41:56 smithi052 ceph-mon[32473]: pgmap v335: 129 pgs: 129 active+clean; 78 MiB data, 695 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 716 B/s wr, 6 op/s 2024-03-04T09:41:56.533 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:41:56 smithi052 ceph-mon[32473]: from='client.? 172.21.15.52:0/3799156681' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-03-04T09:41:56.533 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:41:56 smithi052 ceph-mon[32473]: osdmap e104: 12 total, 12 up, 12 in 2024-03-04T09:41:56.533 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:41:56 smithi052 ceph-mon[32473]: from='client.? 172.21.15.52:0/1740833807' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-04T09:41:56.534 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data_ec cephfs_data_ec --yes-i-really-really-mean-it 2024-03-04T09:41:56.632 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:41:56 smithi106 ceph-mon[36484]: pgmap v335: 129 pgs: 129 active+clean; 78 MiB data, 695 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 716 B/s wr, 6 op/s 2024-03-04T09:41:56.632 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:41:56 smithi106 ceph-mon[36484]: from='client.? 172.21.15.52:0/3799156681' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-03-04T09:41:56.632 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:41:56 smithi106 ceph-mon[36484]: osdmap e104: 12 total, 12 up, 12 in 2024-03-04T09:41:56.633 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:41:56 smithi106 ceph-mon[36484]: from='client.? 172.21.15.52:0/1740833807' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-04T09:41:56.648 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:56.650+0000 7ff51933a640 1 -- 172.21.15.52:0/3985965749 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff5140ffed0 msgr2=0x7ff514100350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:41:56.648 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:56.650+0000 7ff51933a640 1 --2- 172.21.15.52:0/3985965749 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff5140ffed0 0x7ff514100350 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7ff4fc009900 tx=0x7ff4fc02bc30 comp rx=0 tx=0).stop 2024-03-04T09:41:56.649 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:56.650+0000 7ff51933a640 1 -- 172.21.15.52:0/3985965749 shutdown_connections 2024-03-04T09:41:56.649 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:56.650+0000 7ff51933a640 1 --2- 172.21.15.52:0/3985965749 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff514100890 0x7ff51410e470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:41:56.649 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:56.650+0000 7ff51933a640 1 --2- 172.21.15.52:0/3985965749 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff5140ffed0 0x7ff514100350 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:41:56.649 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:56.650+0000 7ff51933a640 1 --2- 172.21.15.52:0/3985965749 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff5141065d0 0x7ff5141069b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:41:56.649 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:56.650+0000 7ff51933a640 1 -- 172.21.15.52:0/3985965749 >> 172.21.15.52:0/3985965749 conn(0x7ff5140757c0 msgr2=0x7ff514075bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:41:56.649 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:56.650+0000 7ff51933a640 1 -- 172.21.15.52:0/3985965749 shutdown_connections 2024-03-04T09:41:56.649 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:56.650+0000 7ff51933a640 1 -- 172.21.15.52:0/3985965749 wait complete. 2024-03-04T09:41:56.652 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:56.651+0000 7ff51933a640 1 Processor -- start 2024-03-04T09:41:56.652 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:56.651+0000 7ff51933a640 1 -- start start 2024-03-04T09:41:56.652 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:56.651+0000 7ff51933a640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff5140ffed0 0x7ff514197e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:41:56.652 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:56.651+0000 7ff51933a640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff514100890 0x7ff514198370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:41:56.652 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:56.651+0000 7ff51933a640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff5141065d0 0x7ff51419c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:41:56.652 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:56.651+0000 7ff51933a640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff514111990 con 0x7ff5140ffed0 2024-03-04T09:41:56.653 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:56.651+0000 7ff51933a640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff514111810 con 0x7ff5141065d0 2024-03-04T09:41:56.653 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:56.651+0000 7ff51933a640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff514111b10 con 0x7ff514100890 2024-03-04T09:41:56.653 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:56.651+0000 7ff512ffd640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff5140ffed0 0x7ff514197e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:41:56.653 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:56.651+0000 7ff512ffd640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff5140ffed0 0x7ff514197e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:56688/0 (socket says 172.21.15.52:56688) 2024-03-04T09:41:56.653 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:56.651+0000 7ff512ffd640 1 -- 172.21.15.52:0/3693357053 learned_addr learned my addr 172.21.15.52:0/3693357053 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-03-04T09:41:56.653 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:56.651+0000 7ff50bfff640 1 --2- 172.21.15.52:0/3693357053 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff514100890 0x7ff514198370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:41:56.653 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:56.651+0000 7ff5137fe640 1 --2- 172.21.15.52:0/3693357053 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff5141065d0 0x7ff51419c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:41:56.653 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:56.651+0000 7ff512ffd640 1 -- 172.21.15.52:0/3693357053 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff514100890 msgr2=0x7ff514198370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:41:56.654 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:56.651+0000 7ff512ffd640 1 --2- 172.21.15.52:0/3693357053 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff514100890 0x7ff514198370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:41:56.654 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:56.652+0000 7ff512ffd640 1 -- 172.21.15.52:0/3693357053 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff5141065d0 msgr2=0x7ff51419c6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:41:56.654 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:56.652+0000 7ff512ffd640 1 --2- 172.21.15.52:0/3693357053 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff5141065d0 0x7ff51419c6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:41:56.654 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:56.652+0000 7ff512ffd640 1 -- 172.21.15.52:0/3693357053 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff51419ce20 con 0x7ff5140ffed0 2024-03-04T09:41:56.654 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:56.652+0000 7ff50bfff640 1 --2- 172.21.15.52:0/3693357053 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff514100890 0x7ff514198370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T09:41:56.654 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:56.652+0000 7ff512ffd640 1 --2- 172.21.15.52:0/3693357053 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff5140ffed0 0x7ff514197e30 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7ff50400e9f0 tx=0x7ff50400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:41:56.654 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:56.652+0000 7ff510ff9640 1 -- 172.21.15.52:0/3693357053 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff50400cd80 con 0x7ff5140ffed0 2024-03-04T09:41:56.654 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:56.652+0000 7ff510ff9640 1 -- 172.21.15.52:0/3693357053 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff504005500 con 0x7ff5140ffed0 2024-03-04T09:41:56.655 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:56.652+0000 7ff51933a640 1 -- 172.21.15.52:0/3693357053 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff51419d110 con 0x7ff5140ffed0 2024-03-04T09:41:56.655 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:56.652+0000 7ff510ff9640 1 -- 172.21.15.52:0/3693357053 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff504010690 con 0x7ff5140ffed0 2024-03-04T09:41:56.655 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:56.652+0000 7ff51933a640 1 -- 172.21.15.52:0/3693357053 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5141a49a0 con 0x7ff5140ffed0 2024-03-04T09:41:56.655 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:56.653+0000 7ff51933a640 1 -- 172.21.15.52:0/3693357053 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff514068810 con 0x7ff5140ffed0 2024-03-04T09:41:56.655 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:56.654+0000 7ff510ff9640 1 -- 172.21.15.52:0/3693357053 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 25) v1 ==== 136887+0+0 (secure 0 0 0) 0x7ff5040046c0 con 0x7ff5140ffed0 2024-03-04T09:41:56.656 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:56.655+0000 7ff510ff9640 1 --2- 172.21.15.52:0/3693357053 >> [v2:172.21.15.52:6800/478031757,v1:172.21.15.52:6801/478031757] conn(0x7ff5000a1ef0 0x7ff5000a43b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T09:41:56.656 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:56.655+0000 7ff50bfff640 1 --2- 172.21.15.52:0/3693357053 >> [v2:172.21.15.52:6800/478031757,v1:172.21.15.52:6801/478031757] conn(0x7ff5000a1ef0 0x7ff5000a43b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T09:41:56.656 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:56.655+0000 7ff510ff9640 1 -- 172.21.15.52:0/3693357053 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 8625+0+0 (secure 0 0 0) 0x7ff5040ce9c0 con 0x7ff5140ffed0 2024-03-04T09:41:56.656 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:56.655+0000 7ff50bfff640 1 --2- 172.21.15.52:0/3693357053 >> [v2:172.21.15.52:6800/478031757,v1:172.21.15.52:6801/478031757] conn(0x7ff5000a1ef0 0x7ff5000a43b0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7ff5140ff9d0 tx=0x7ff4fc005a20 comp rx=0 tx=0).ready entity=mgr.15486 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T09:41:56.657 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:56.658+0000 7ff510ff9640 1 -- 172.21.15.52:0/3693357053 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184932 (secure 0 0 0) 0x7ff504053280 con 0x7ff5140ffed0 2024-03-04T09:41:56.838 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:56.840+0000 7ff51933a640 1 -- 172.21.15.52:0/3693357053 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7ff514109b20 con 0x7ff5140ffed0 2024-03-04T09:41:57.520 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:57.522+0000 7ff510ff9640 1 -- 172.21.15.52:0/3693357053 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data_ec' removed v106) v1 ==== 178+0+0 (secure 0 0 0) 0x7ff50409b9e0 con 0x7ff5140ffed0 2024-03-04T09:41:57.520 INFO:teuthology.orchestra.run.smithi052.stderr:pool 'cephfs_data_ec' removed 2024-03-04T09:41:57.524 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:57.526+0000 7ff51933a640 1 -- 172.21.15.52:0/3693357053 >> [v2:172.21.15.52:6800/478031757,v1:172.21.15.52:6801/478031757] conn(0x7ff5000a1ef0 msgr2=0x7ff5000a43b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:41:57.524 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:57.526+0000 7ff51933a640 1 --2- 172.21.15.52:0/3693357053 >> [v2:172.21.15.52:6800/478031757,v1:172.21.15.52:6801/478031757] conn(0x7ff5000a1ef0 0x7ff5000a43b0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7ff5140ff9d0 tx=0x7ff4fc005a20 comp rx=0 tx=0).stop 2024-03-04T09:41:57.524 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:57.526+0000 7ff51933a640 1 -- 172.21.15.52:0/3693357053 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff5140ffed0 msgr2=0x7ff514197e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T09:41:57.524 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:57.526+0000 7ff51933a640 1 --2- 172.21.15.52:0/3693357053 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff5140ffed0 0x7ff514197e30 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7ff50400e9f0 tx=0x7ff50400edf0 comp rx=0 tx=0).stop 2024-03-04T09:41:57.525 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:57.526+0000 7ff51933a640 1 -- 172.21.15.52:0/3693357053 shutdown_connections 2024-03-04T09:41:57.525 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:57.526+0000 7ff51933a640 1 --2- 172.21.15.52:0/3693357053 >> [v2:172.21.15.52:6800/478031757,v1:172.21.15.52:6801/478031757] conn(0x7ff5000a1ef0 0x7ff5000a43b0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:41:57.525 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:57.526+0000 7ff51933a640 1 --2- 172.21.15.52:0/3693357053 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff5141065d0 0x7ff51419c6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:41:57.525 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:57.526+0000 7ff51933a640 1 --2- 172.21.15.52:0/3693357053 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff514100890 0x7ff514198370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:41:57.525 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:57.526+0000 7ff51933a640 1 --2- 172.21.15.52:0/3693357053 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff5140ffed0 0x7ff514197e30 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T09:41:57.525 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:57.526+0000 7ff51933a640 1 -- 172.21.15.52:0/3693357053 >> 172.21.15.52:0/3693357053 conn(0x7ff5140757c0 msgr2=0x7ff5140fdbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T09:41:57.525 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:57.527+0000 7ff51933a640 1 -- 172.21.15.52:0/3693357053 shutdown_connections 2024-03-04T09:41:57.525 INFO:teuthology.orchestra.run.smithi052.stderr:2024-03-04T09:41:57.527+0000 7ff51933a640 1 -- 172.21.15.52:0/3693357053 wait complete. 2024-03-04T09:41:57.541 DEBUG:teuthology.orchestra.run.smithi052:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-03-04T09:41:57.570 DEBUG:teuthology.orchestra.run.smithi106:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-03-04T09:41:57.605 DEBUG:teuthology.orchestra.run.smithi191:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-03-04T09:41:57.638 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-03-04T09:41:57.638 DEBUG:teuthology.orchestra.run.smithi052:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-03-04T09:41:57.654 DEBUG:teuthology.orchestra.run.smithi106:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-03-04T09:41:57.671 DEBUG:teuthology.orchestra.run.smithi191:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-03-04T09:41:57.692 INFO:tasks.cephadm:Stopping all daemons... 2024-03-04T09:41:57.692 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-03-04T09:41:57.692 DEBUG:teuthology.orchestra.run.smithi052:> sudo systemctl stop ceph-4e059f08-da08-11ee-95c6-87774f69a715@mon.a 2024-03-04T09:41:57.835 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:41:57 smithi052 ceph-mon[32473]: from='client.? 172.21.15.52:0/1740833807' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-03-04T09:41:57.836 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:41:57 smithi052 ceph-mon[32473]: osdmap e105: 12 total, 12 up, 12 in 2024-03-04T09:41:57.836 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:41:57 smithi052 ceph-mon[32473]: pgmap v338: 33 pgs: 33 active+clean; 26 MiB data, 695 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-04T09:41:57.836 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:41:57 smithi052 ceph-mon[32473]: from='client.? 172.21.15.52:0/3693357053' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-04T09:41:57.836 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:41:57 smithi052 systemd[1]: Stopping Ceph mon.a for 4e059f08-da08-11ee-95c6-87774f69a715... 2024-03-04T09:41:57.837 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:41:57 smithi191 ceph-mon[36816]: from='client.? 172.21.15.52:0/1740833807' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-03-04T09:41:57.837 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:41:57 smithi191 ceph-mon[36816]: osdmap e105: 12 total, 12 up, 12 in 2024-03-04T09:41:57.837 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:41:57 smithi191 ceph-mon[36816]: pgmap v338: 33 pgs: 33 active+clean; 26 MiB data, 695 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-04T09:41:57.837 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:41:57 smithi191 ceph-mon[36816]: from='client.? 172.21.15.52:0/3693357053' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-04T09:41:57.882 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:41:57 smithi106 ceph-mon[36484]: from='client.? 172.21.15.52:0/1740833807' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-03-04T09:41:57.882 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:41:57 smithi106 ceph-mon[36484]: osdmap e105: 12 total, 12 up, 12 in 2024-03-04T09:41:57.882 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:41:57 smithi106 ceph-mon[36484]: pgmap v338: 33 pgs: 33 active+clean; 26 MiB data, 695 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-04T09:41:57.883 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:41:57 smithi106 ceph-mon[36484]: from='client.? 172.21.15.52:0/3693357053' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-04T09:41:58.302 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:41:57 smithi052 ceph-4e059f08-da08-11ee-95c6-87774f69a715-mon-a[32469]: 2024-03-04T09:41:57.994+0000 7ff7d04b5700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-03-04T09:41:58.302 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:41:57 smithi052 ceph-4e059f08-da08-11ee-95c6-87774f69a715-mon-a[32469]: 2024-03-04T09:41:57.994+0000 7ff7d04b5700 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-03-04T09:41:58.302 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:41:58 smithi052 podman[95728]: 2024-03-04 09:41:58.036152918 +0000 UTC m=+0.146385570 container died b840a3470a2b8f2bf6ea9a6f5ee27a387058780843cce4dbfba12d501c25b2b0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd, name=ceph-4e059f08-da08-11ee-95c6-87774f69a715-mon-a, org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240222, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-qdb-1bc250c, GIT_BRANCH=HEAD) 2024-03-04T09:41:58.585 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:41:58 smithi052 podman[95728]: 2024-03-04 09:41:58.404331041 +0000 UTC m=+0.514563702 container cleanup b840a3470a2b8f2bf6ea9a6f5ee27a387058780843cce4dbfba12d501c25b2b0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd, name=ceph-4e059f08-da08-11ee-95c6-87774f69a715-mon-a, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-qdb-1bc250c, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240222, GIT_CLEAN=True) 2024-03-04T09:41:58.586 INFO:journalctl@ceph.mon.a.smithi052.stdout:Mar 04 09:41:58 smithi052 bash[95728]: ceph-4e059f08-da08-11ee-95c6-87774f69a715-mon-a 2024-03-04T09:41:58.876 DEBUG:teuthology.orchestra.run.smithi052:> sudo pkill -f 'journalctl -f -n 0 -u ceph-4e059f08-da08-11ee-95c6-87774f69a715@mon.a.service' 2024-03-04T09:41:58.927 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-04T09:41:58.927 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-03-04T09:41:58.927 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2024-03-04T09:41:58.928 DEBUG:teuthology.orchestra.run.smithi106:> sudo systemctl stop ceph-4e059f08-da08-11ee-95c6-87774f69a715@mon.b 2024-03-04T09:41:59.258 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:41:58 smithi106 systemd[1]: Stopping Ceph mon.b for 4e059f08-da08-11ee-95c6-87774f69a715... 2024-03-04T09:41:59.584 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:41:59 smithi106 ceph-4e059f08-da08-11ee-95c6-87774f69a715-mon-b[36480]: 2024-03-04T09:41:59.259+0000 7ffbb9b41700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-03-04T09:41:59.584 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:41:59 smithi106 ceph-4e059f08-da08-11ee-95c6-87774f69a715-mon-b[36480]: 2024-03-04T09:41:59.259+0000 7ffbb9b41700 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2024-03-04T09:41:59.584 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:41:59 smithi106 podman[71942]: 2024-03-04 09:41:59.281203397 +0000 UTC m=+0.152817233 container died d74eb04dccfbc3c1eb96bff258e17a1d8839af147c51fa5babd7f8a293c4b8de (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5, name=ceph-4e059f08-da08-11ee-95c6-87774f69a715-mon-b, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=wip-lusov-qdb-1bc250c, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240222, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, GIT_CLEAN=True) 2024-03-04T09:41:59.882 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:41:59 smithi106 podman[71942]: 2024-03-04 09:41:59.669507514 +0000 UTC m=+0.541121360 container cleanup d74eb04dccfbc3c1eb96bff258e17a1d8839af147c51fa5babd7f8a293c4b8de (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5, name=ceph-4e059f08-da08-11ee-95c6-87774f69a715-mon-b, RELEASE=wip-lusov-qdb-1bc250c, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240222) 2024-03-04T09:41:59.883 INFO:journalctl@ceph.mon.b.smithi106.stdout:Mar 04 09:41:59 smithi106 bash[71942]: ceph-4e059f08-da08-11ee-95c6-87774f69a715-mon-b 2024-03-04T09:42:00.082 DEBUG:teuthology.orchestra.run.smithi106:> sudo pkill -f 'journalctl -f -n 0 -u ceph-4e059f08-da08-11ee-95c6-87774f69a715@mon.b.service' 2024-03-04T09:42:00.125 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-04T09:42:00.126 INFO:tasks.cephadm.mon.c:Stopped mon.b 2024-03-04T09:42:00.126 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2024-03-04T09:42:00.126 DEBUG:teuthology.orchestra.run.smithi191:> sudo systemctl stop ceph-4e059f08-da08-11ee-95c6-87774f69a715@mon.c 2024-03-04T09:42:00.444 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:42:00 smithi191 systemd[1]: Stopping Ceph mon.c for 4e059f08-da08-11ee-95c6-87774f69a715... 2024-03-04T09:42:00.837 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:42:00 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-mon-c[36812]: 2024-03-04T09:42:00.446+0000 7fbe5689b700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-03-04T09:42:00.838 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:42:00 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-mon-c[36812]: 2024-03-04T09:42:00.446+0000 7fbe5689b700 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2024-03-04T09:42:00.838 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:42:00 smithi191 podman[72936]: 2024-03-04 09:42:00.619268775 +0000 UTC m=+0.297980201 container died d08b1b732845c8c5fd819d7b38a6a2598472d997186d7270900b57b281951066 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5, name=ceph-4e059f08-da08-11ee-95c6-87774f69a715-mon-c, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, RELEASE=wip-lusov-qdb-1bc250c, org.label-schema.build-date=20240222, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, ceph=True) 2024-03-04T09:42:01.086 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Mar 04 09:42:00 smithi052 ceph-4e059f08-da08-11ee-95c6-87774f69a715-mgr-x[32705]: 2024-03-04T09:42:00.811+0000 7f5f5c2d8700 -1 client.15516: SimpleRADOSStriper: lock: snap_db_v0.db: lock failed: (2) No such file or directory 2024-03-04T09:42:01.295 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:42:00 smithi191 podman[72936]: 2024-03-04 09:42:00.988323692 +0000 UTC m=+0.667035117 container cleanup d08b1b732845c8c5fd819d7b38a6a2598472d997186d7270900b57b281951066 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5, name=ceph-4e059f08-da08-11ee-95c6-87774f69a715-mon-c, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, io.buildah.version=1.33.5, RELEASE=wip-lusov-qdb-1bc250c, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.build-date=20240222, org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-03-04T09:42:01.296 INFO:journalctl@ceph.mon.c.smithi191.stdout:Mar 04 09:42:00 smithi191 bash[72936]: ceph-4e059f08-da08-11ee-95c6-87774f69a715-mon-c 2024-03-04T09:42:01.450 DEBUG:teuthology.orchestra.run.smithi191:> sudo pkill -f 'journalctl -f -n 0 -u ceph-4e059f08-da08-11ee-95c6-87774f69a715@mon.c.service' 2024-03-04T09:42:01.498 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-04T09:42:01.498 INFO:tasks.cephadm.mon.c:Stopped mon.c 2024-03-04T09:42:01.498 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-03-04T09:42:01.499 DEBUG:teuthology.orchestra.run.smithi052:> sudo systemctl stop ceph-4e059f08-da08-11ee-95c6-87774f69a715@mgr.x 2024-03-04T09:42:01.836 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Mar 04 09:42:01 smithi052 systemd[1]: Stopping Ceph mgr.x for 4e059f08-da08-11ee-95c6-87774f69a715... 2024-03-04T09:42:02.175 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Mar 04 09:42:01 smithi052 ceph-4e059f08-da08-11ee-95c6-87774f69a715-mgr-x[32705]: 2024-03-04T09:42:01.901+0000 7f5ff731b700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mgr -n mgr.x -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-04T09:42:02.176 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Mar 04 09:42:01 smithi052 podman[95851]: 2024-03-04 09:42:01.932437806 +0000 UTC m=+0.234462241 container died 4852946a56df23e775c98f79c6a3ff5f22eecdc45a7f257119b0a1b7426f11c6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd, name=ceph-4e059f08-da08-11ee-95c6-87774f69a715-mgr-x, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-qdb-1bc250c, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, GIT_CLEAN=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240222) 2024-03-04T09:42:02.476 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Mar 04 09:42:02 smithi052 podman[95851]: 2024-03-04 09:42:02.269171402 +0000 UTC m=+0.571195846 container cleanup 4852946a56df23e775c98f79c6a3ff5f22eecdc45a7f257119b0a1b7426f11c6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd, name=ceph-4e059f08-da08-11ee-95c6-87774f69a715-mgr-x, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240222, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, RELEASE=wip-lusov-qdb-1bc250c) 2024-03-04T09:42:02.476 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Mar 04 09:42:02 smithi052 bash[95851]: ceph-4e059f08-da08-11ee-95c6-87774f69a715-mgr-x 2024-03-04T09:42:02.704 DEBUG:teuthology.orchestra.run.smithi052:> sudo pkill -f 'journalctl -f -n 0 -u ceph-4e059f08-da08-11ee-95c6-87774f69a715@mgr.x.service' 2024-03-04T09:42:02.727 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Mar 04 09:42:02 smithi052 podman[95865]: 2024-03-04 09:42:02.479319986 +0000 UTC m=+0.544380173 container remove 4852946a56df23e775c98f79c6a3ff5f22eecdc45a7f257119b0a1b7426f11c6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd, name=ceph-4e059f08-da08-11ee-95c6-87774f69a715-mgr-x, org.label-schema.schema-version=1.0, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240222, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=wip-lusov-qdb-1bc250c) 2024-03-04T09:42:02.728 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Mar 04 09:42:02 smithi052 systemd[1]: ceph-4e059f08-da08-11ee-95c6-87774f69a715@mgr.x.service: Deactivated successfully. 2024-03-04T09:42:02.728 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Mar 04 09:42:02 smithi052 systemd[1]: Stopped Ceph mgr.x for 4e059f08-da08-11ee-95c6-87774f69a715. 2024-03-04T09:42:02.728 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Mar 04 09:42:02 smithi052 systemd[1]: ceph-4e059f08-da08-11ee-95c6-87774f69a715@mgr.x.service: Consumed 1min 27.366s CPU time. 2024-03-04T09:42:02.741 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-04T09:42:02.741 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-03-04T09:42:02.742 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-03-04T09:42:02.742 DEBUG:teuthology.orchestra.run.smithi106:> sudo systemctl stop ceph-4e059f08-da08-11ee-95c6-87774f69a715@mgr.y 2024-03-04T09:42:03.035 INFO:journalctl@ceph.mgr.y.smithi106.stdout:Mar 04 09:42:02 smithi106 systemd[1]: Stopping Ceph mgr.y for 4e059f08-da08-11ee-95c6-87774f69a715... 2024-03-04T09:42:03.382 INFO:journalctl@ceph.mgr.y.smithi106.stdout:Mar 04 09:42:03 smithi106 podman[72065]: 2024-03-04 09:42:03.170648624 +0000 UTC m=+0.219929419 container died d19f286d79b1d9e97c75aabccbdf8a9c1c8dbe4a179c0ad2dae8c8767e248a6c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5, name=ceph-4e059f08-da08-11ee-95c6-87774f69a715-mgr-y, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, org.label-schema.build-date=20240222, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, RELEASE=wip-lusov-qdb-1bc250c) 2024-03-04T09:42:03.825 INFO:journalctl@ceph.mgr.y.smithi106.stdout:Mar 04 09:42:03 smithi106 podman[72065]: 2024-03-04 09:42:03.608917614 +0000 UTC m=+0.658198402 container cleanup d19f286d79b1d9e97c75aabccbdf8a9c1c8dbe4a179c0ad2dae8c8767e248a6c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5, name=ceph-4e059f08-da08-11ee-95c6-87774f69a715-mgr-y, org.label-schema.build-date=20240222, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, RELEASE=wip-lusov-qdb-1bc250c, GIT_BRANCH=HEAD, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, GIT_CLEAN=True) 2024-03-04T09:42:03.825 INFO:journalctl@ceph.mgr.y.smithi106.stdout:Mar 04 09:42:03 smithi106 bash[72065]: ceph-4e059f08-da08-11ee-95c6-87774f69a715-mgr-y 2024-03-04T09:42:04.065 DEBUG:teuthology.orchestra.run.smithi106:> sudo pkill -f 'journalctl -f -n 0 -u ceph-4e059f08-da08-11ee-95c6-87774f69a715@mgr.y.service' 2024-03-04T09:42:04.089 INFO:journalctl@ceph.mgr.y.smithi106.stdout:Mar 04 09:42:03 smithi106 podman[72082]: 2024-03-04 09:42:03.827210045 +0000 UTC m=+0.652479135 container remove d19f286d79b1d9e97c75aabccbdf8a9c1c8dbe4a179c0ad2dae8c8767e248a6c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5, name=ceph-4e059f08-da08-11ee-95c6-87774f69a715-mgr-y, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-qdb-1bc250c, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240222, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-04T09:42:04.089 INFO:journalctl@ceph.mgr.y.smithi106.stdout:Mar 04 09:42:03 smithi106 systemd[1]: ceph-4e059f08-da08-11ee-95c6-87774f69a715@mgr.y.service: Main process exited, code=exited, status=143/n/a 2024-03-04T09:42:04.090 INFO:journalctl@ceph.mgr.y.smithi106.stdout:Mar 04 09:42:04 smithi106 systemd[1]: ceph-4e059f08-da08-11ee-95c6-87774f69a715@mgr.y.service: Failed with result 'exit-code'. 2024-03-04T09:42:04.090 INFO:journalctl@ceph.mgr.y.smithi106.stdout:Mar 04 09:42:04 smithi106 systemd[1]: Stopped Ceph mgr.y for 4e059f08-da08-11ee-95c6-87774f69a715. 2024-03-04T09:42:04.090 INFO:journalctl@ceph.mgr.y.smithi106.stdout:Mar 04 09:42:04 smithi106 systemd[1]: ceph-4e059f08-da08-11ee-95c6-87774f69a715@mgr.y.service: Consumed 19.323s CPU time. 2024-03-04T09:42:04.104 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-04T09:42:04.104 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-03-04T09:42:04.104 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2024-03-04T09:42:04.104 DEBUG:teuthology.orchestra.run.smithi191:> sudo systemctl stop ceph-4e059f08-da08-11ee-95c6-87774f69a715@mgr.z 2024-03-04T09:42:04.469 INFO:journalctl@ceph.mgr.z.smithi191.stdout:Mar 04 09:42:04 smithi191 systemd[1]: Stopping Ceph mgr.z for 4e059f08-da08-11ee-95c6-87774f69a715... 2024-03-04T09:42:04.757 INFO:journalctl@ceph.mgr.z.smithi191.stdout:Mar 04 09:42:04 smithi191 podman[73058]: 2024-03-04 09:42:04.472126457 +0000 UTC m=+0.171601503 container died f5a6a77297b8bdcfba5bc2d61a06446a1ad68d6593dc6fccacc4b3095948d61b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5, name=ceph-4e059f08-da08-11ee-95c6-87774f69a715-mgr-z, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=wip-lusov-qdb-1bc250c, io.buildah.version=1.33.5, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, ceph=True, org.label-schema.build-date=20240222, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-03-04T09:42:05.087 INFO:journalctl@ceph.mgr.z.smithi191.stdout:Mar 04 09:42:04 smithi191 podman[73058]: 2024-03-04 09:42:04.876856305 +0000 UTC m=+0.576331344 container cleanup f5a6a77297b8bdcfba5bc2d61a06446a1ad68d6593dc6fccacc4b3095948d61b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5, name=ceph-4e059f08-da08-11ee-95c6-87774f69a715-mgr-z, RELEASE=wip-lusov-qdb-1bc250c, GIT_CLEAN=True, org.label-schema.build-date=20240222, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-03-04T09:42:05.088 INFO:journalctl@ceph.mgr.z.smithi191.stdout:Mar 04 09:42:04 smithi191 bash[73058]: ceph-4e059f08-da08-11ee-95c6-87774f69a715-mgr-z 2024-03-04T09:42:05.292 DEBUG:teuthology.orchestra.run.smithi191:> sudo pkill -f 'journalctl -f -n 0 -u ceph-4e059f08-da08-11ee-95c6-87774f69a715@mgr.z.service' 2024-03-04T09:42:05.330 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-04T09:42:05.330 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2024-03-04T09:42:05.331 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-03-04T09:42:05.331 DEBUG:teuthology.orchestra.run.smithi052:> sudo systemctl stop ceph-4e059f08-da08-11ee-95c6-87774f69a715@osd.0 2024-03-04T09:42:05.654 INFO:journalctl@ceph.osd.0.smithi052.stdout:Mar 04 09:42:05 smithi052 systemd[1]: Stopping Ceph osd.0 for 4e059f08-da08-11ee-95c6-87774f69a715... 2024-03-04T09:42:06.086 INFO:journalctl@ceph.osd.0.smithi052.stdout:Mar 04 09:42:05 smithi052 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-0[44607]: 2024-03-04T09:42:05.656+0000 7fab29252700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-04T09:42:06.086 INFO:journalctl@ceph.osd.0.smithi052.stdout:Mar 04 09:42:05 smithi052 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-0[44607]: 2024-03-04T09:42:05.656+0000 7fab29252700 -1 osd.0 106 *** Got signal Terminated *** 2024-03-04T09:42:06.086 INFO:journalctl@ceph.osd.0.smithi052.stdout:Mar 04 09:42:05 smithi052 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-0[44607]: 2024-03-04T09:42:05.656+0000 7fab29252700 -1 osd.0 106 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-04T09:42:11.306 INFO:journalctl@ceph.osd.0.smithi052.stdout:Mar 04 09:42:11 smithi052 podman[95972]: 2024-03-04 09:42:11.022655807 +0000 UTC m=+5.489385328 container died d30d901a490a12504505ad93175ebdac749676bb68718777cce06ac6ac7db5f5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5, name=ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, ceph=True, RELEASE=wip-lusov-qdb-1bc250c, org.label-schema.build-date=20240222, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.33.5) 2024-03-04T09:42:11.586 INFO:journalctl@ceph.osd.0.smithi052.stdout:Mar 04 09:42:11 smithi052 podman[95972]: 2024-03-04 09:42:11.408075961 +0000 UTC m=+5.874805481 container cleanup d30d901a490a12504505ad93175ebdac749676bb68718777cce06ac6ac7db5f5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5, name=ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-qdb-1bc250c, io.buildah.version=1.33.5, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240222, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-03-04T09:42:11.586 INFO:journalctl@ceph.osd.0.smithi052.stdout:Mar 04 09:42:11 smithi052 bash[95972]: ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-0 2024-03-04T09:42:12.025 INFO:journalctl@ceph.osd.0.smithi052.stdout:Mar 04 09:42:11 smithi052 podman[95989]: 2024-03-04 09:42:11.668155866 +0000 UTC m=+0.643131375 container remove d30d901a490a12504505ad93175ebdac749676bb68718777cce06ac6ac7db5f5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5, name=ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-0, ceph=True, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-qdb-1bc250c, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.build-date=20240222) 2024-03-04T09:42:12.319 INFO:journalctl@ceph.osd.0.smithi052.stdout:Mar 04 09:42:12 smithi052 podman[96078]: 2024-03-04 09:42:11.930795971 +0000 UTC m=+0.018400538 image pull f88ffb32aa6a7d5dfdc954d9f096f0c9cc47981d05ec8d117ba859a22d8185ba quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5 2024-03-04T09:42:12.320 INFO:journalctl@ceph.osd.0.smithi052.stdout:Mar 04 09:42:12 smithi052 podman[96078]: 2024-03-04 09:42:12.043997492 +0000 UTC m=+0.131602043 container create 7068cfd2cf9c4ea3c7e15e8180958d5f95d3b8cb0d0dfa77b8b70b5827974c27 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5, name=ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-0-deactivate, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=wip-lusov-qdb-1bc250c, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.build-date=20240222, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8) 2024-03-04T09:42:12.320 INFO:journalctl@ceph.osd.0.smithi052.stdout:Mar 04 09:42:12 smithi052 podman[96078]: 2024-03-04 09:42:12.32189008 +0000 UTC m=+0.409494634 container init 7068cfd2cf9c4ea3c7e15e8180958d5f95d3b8cb0d0dfa77b8b70b5827974c27 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5, name=ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-0-deactivate, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240222, RELEASE=wip-lusov-qdb-1bc250c, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-04T09:42:12.586 INFO:journalctl@ceph.osd.0.smithi052.stdout:Mar 04 09:42:12 smithi052 podman[96078]: 2024-03-04 09:42:12.325481354 +0000 UTC m=+0.413085906 container start 7068cfd2cf9c4ea3c7e15e8180958d5f95d3b8cb0d0dfa77b8b70b5827974c27 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5, name=ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-0-deactivate, org.label-schema.build-date=20240222, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, RELEASE=wip-lusov-qdb-1bc250c, org.label-schema.license=GPLv2, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=) 2024-03-04T09:42:12.586 INFO:journalctl@ceph.osd.0.smithi052.stdout:Mar 04 09:42:12 smithi052 podman[96078]: 2024-03-04 09:42:12.430333845 +0000 UTC m=+0.517938414 container attach 7068cfd2cf9c4ea3c7e15e8180958d5f95d3b8cb0d0dfa77b8b70b5827974c27 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5, name=ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-0-deactivate, maintainer=Guillaume Abrioux , GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD, RELEASE=wip-lusov-qdb-1bc250c, org.label-schema.build-date=20240222, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0) 2024-03-04T09:42:12.922 INFO:journalctl@ceph.osd.0.smithi052.stdout:Mar 04 09:42:12 smithi052 podman[96078]: 2024-03-04 09:42:12.61762855 +0000 UTC m=+0.705233102 container died 7068cfd2cf9c4ea3c7e15e8180958d5f95d3b8cb0d0dfa77b8b70b5827974c27 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5, name=ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-0-deactivate, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=wip-lusov-qdb-1bc250c, org.label-schema.build-date=20240222, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True) 2024-03-04T09:42:13.239 DEBUG:teuthology.orchestra.run.smithi052:> sudo pkill -f 'journalctl -f -n 0 -u ceph-4e059f08-da08-11ee-95c6-87774f69a715@osd.0.service' 2024-03-04T09:42:13.285 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-04T09:42:13.286 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-03-04T09:42:13.286 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-03-04T09:42:13.286 DEBUG:teuthology.orchestra.run.smithi106:> sudo systemctl stop ceph-4e059f08-da08-11ee-95c6-87774f69a715@osd.1 2024-03-04T09:42:13.595 INFO:journalctl@ceph.osd.1.smithi106.stdout:Mar 04 09:42:13 smithi106 systemd[1]: Stopping Ceph osd.1 for 4e059f08-da08-11ee-95c6-87774f69a715... 2024-03-04T09:42:13.882 INFO:journalctl@ceph.osd.1.smithi106.stdout:Mar 04 09:42:13 smithi106 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-1[41081]: 2024-03-04T09:42:13.596+0000 7f14aa809700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-04T09:42:13.882 INFO:journalctl@ceph.osd.1.smithi106.stdout:Mar 04 09:42:13 smithi106 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-1[41081]: 2024-03-04T09:42:13.596+0000 7f14aa809700 -1 osd.1 106 *** Got signal Terminated *** 2024-03-04T09:42:13.883 INFO:journalctl@ceph.osd.1.smithi106.stdout:Mar 04 09:42:13 smithi106 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-1[41081]: 2024-03-04T09:42:13.596+0000 7f14aa809700 -1 osd.1 106 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-04T09:42:19.296 INFO:journalctl@ceph.osd.1.smithi106.stdout:Mar 04 09:42:18 smithi106 podman[72202]: 2024-03-04 09:42:18.996768417 +0000 UTC m=+5.513300185 container died d0c65575d03e082a7ef32e8a89ab1c7045ba7e2b33b531b005fb699c2234c8ee (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5, name=ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-1, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240222, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-qdb-1bc250c, org.label-schema.license=GPLv2, io.buildah.version=1.33.5) 2024-03-04T09:42:19.632 INFO:journalctl@ceph.osd.1.smithi106.stdout:Mar 04 09:42:19 smithi106 podman[72202]: 2024-03-04 09:42:19.388845278 +0000 UTC m=+5.905377062 container cleanup d0c65575d03e082a7ef32e8a89ab1c7045ba7e2b33b531b005fb699c2234c8ee (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5, name=ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-1, maintainer=Guillaume Abrioux , GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, GIT_BRANCH=HEAD, RELEASE=wip-lusov-qdb-1bc250c, org.label-schema.license=GPLv2, org.label-schema.build-date=20240222, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-04T09:42:19.633 INFO:journalctl@ceph.osd.1.smithi106.stdout:Mar 04 09:42:19 smithi106 bash[72202]: ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-1 2024-03-04T09:42:19.997 INFO:journalctl@ceph.osd.1.smithi106.stdout:Mar 04 09:42:19 smithi106 podman[72219]: 2024-03-04 09:42:19.665522037 +0000 UTC m=+0.665913271 container remove d0c65575d03e082a7ef32e8a89ab1c7045ba7e2b33b531b005fb699c2234c8ee (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5, name=ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-1, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240222, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-qdb-1bc250c, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8) 2024-03-04T09:42:20.342 INFO:journalctl@ceph.osd.1.smithi106.stdout:Mar 04 09:42:19 smithi106 podman[72311]: 2024-03-04 09:42:19.999608131 +0000 UTC m=+0.113856842 container create a2c65e6d5aa9a070746b801e8c5eb3201476c1b165fa1525eb4bacec353b33be (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5, name=ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-1-deactivate, org.label-schema.build-date=20240222, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-qdb-1bc250c, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True) 2024-03-04T09:42:20.342 INFO:journalctl@ceph.osd.1.smithi106.stdout:Mar 04 09:42:20 smithi106 podman[72311]: 2024-03-04 09:42:19.90323526 +0000 UTC m=+0.017483983 image pull f88ffb32aa6a7d5dfdc954d9f096f0c9cc47981d05ec8d117ba859a22d8185ba quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5 2024-03-04T09:42:20.342 INFO:journalctl@ceph.osd.1.smithi106.stdout:Mar 04 09:42:20 smithi106 podman[72311]: 2024-03-04 09:42:20.244298105 +0000 UTC m=+0.358546821 container init a2c65e6d5aa9a070746b801e8c5eb3201476c1b165fa1525eb4bacec353b33be (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5, name=ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-1-deactivate, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240222, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-qdb-1bc250c, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8) 2024-03-04T09:42:20.342 INFO:journalctl@ceph.osd.1.smithi106.stdout:Mar 04 09:42:20 smithi106 podman[72311]: 2024-03-04 09:42:20.247587708 +0000 UTC m=+0.361836417 container start a2c65e6d5aa9a070746b801e8c5eb3201476c1b165fa1525eb4bacec353b33be (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5, name=ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-1-deactivate, io.buildah.version=1.33.5, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240222, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, RELEASE=wip-lusov-qdb-1bc250c, CEPH_POINT_RELEASE=) 2024-03-04T09:42:20.342 INFO:journalctl@ceph.osd.1.smithi106.stdout:Mar 04 09:42:20 smithi106 podman[72311]: 2024-03-04 09:42:20.343986175 +0000 UTC m=+0.458234898 container attach a2c65e6d5aa9a070746b801e8c5eb3201476c1b165fa1525eb4bacec353b33be (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5, name=ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-1-deactivate, RELEASE=wip-lusov-qdb-1bc250c, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240222, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-04T09:42:20.632 INFO:journalctl@ceph.osd.1.smithi106.stdout:Mar 04 09:42:20 smithi106 podman[72311]: 2024-03-04 09:42:20.541888978 +0000 UTC m=+0.656137686 container died a2c65e6d5aa9a070746b801e8c5eb3201476c1b165fa1525eb4bacec353b33be (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5, name=ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-1-deactivate, RELEASE=wip-lusov-qdb-1bc250c, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240222, org.label-schema.vendor=CentOS, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-04T09:42:21.195 DEBUG:teuthology.orchestra.run.smithi106:> sudo pkill -f 'journalctl -f -n 0 -u ceph-4e059f08-da08-11ee-95c6-87774f69a715@osd.1.service' 2024-03-04T09:42:21.236 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-04T09:42:21.236 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-03-04T09:42:21.237 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-03-04T09:42:21.237 DEBUG:teuthology.orchestra.run.smithi191:> sudo systemctl stop ceph-4e059f08-da08-11ee-95c6-87774f69a715@osd.2 2024-03-04T09:42:21.554 INFO:journalctl@ceph.osd.2.smithi191.stdout:Mar 04 09:42:21 smithi191 systemd[1]: Stopping Ceph osd.2 for 4e059f08-da08-11ee-95c6-87774f69a715... 2024-03-04T09:42:21.837 INFO:journalctl@ceph.osd.2.smithi191.stdout:Mar 04 09:42:21 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-2[41970]: 2024-03-04T09:42:21.555+0000 7f4819643700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-04T09:42:21.838 INFO:journalctl@ceph.osd.2.smithi191.stdout:Mar 04 09:42:21 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-2[41970]: 2024-03-04T09:42:21.555+0000 7f4819643700 -1 osd.2 106 *** Got signal Terminated *** 2024-03-04T09:42:21.838 INFO:journalctl@ceph.osd.2.smithi191.stdout:Mar 04 09:42:21 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-2[41970]: 2024-03-04T09:42:21.555+0000 7f4819643700 -1 osd.2 106 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-04T09:42:27.172 INFO:journalctl@ceph.osd.2.smithi191.stdout:Mar 04 09:42:26 smithi191 podman[73182]: 2024-03-04 09:42:26.896783823 +0000 UTC m=+5.459309139 container died da5629b7b7442f7aabc0b57b60f9cf1618f6b4f721e3cf6caf7b2437805972cc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5, name=ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-2, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240222, RELEASE=wip-lusov-qdb-1bc250c, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8) 2024-03-04T09:42:27.540 INFO:journalctl@ceph.osd.2.smithi191.stdout:Mar 04 09:42:27 smithi191 podman[73182]: 2024-03-04 09:42:27.298078068 +0000 UTC m=+5.860603384 container cleanup da5629b7b7442f7aabc0b57b60f9cf1618f6b4f721e3cf6caf7b2437805972cc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5, name=ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-2, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-qdb-1bc250c, ceph=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240222, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8) 2024-03-04T09:42:27.540 INFO:journalctl@ceph.osd.2.smithi191.stdout:Mar 04 09:42:27 smithi191 bash[73182]: ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-2 2024-03-04T09:42:27.837 INFO:journalctl@ceph.osd.2.smithi191.stdout:Mar 04 09:42:27 smithi191 podman[73199]: 2024-03-04 09:42:27.54275253 +0000 UTC m=+0.646616026 container remove da5629b7b7442f7aabc0b57b60f9cf1618f6b4f721e3cf6caf7b2437805972cc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5, name=ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-2, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240222, ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-qdb-1bc250c, GIT_BRANCH=HEAD) 2024-03-04T09:42:28.220 INFO:journalctl@ceph.osd.2.smithi191.stdout:Mar 04 09:42:27 smithi191 podman[73290]: 2024-03-04 09:42:27.873273818 +0000 UTC m=+0.018186402 image pull f88ffb32aa6a7d5dfdc954d9f096f0c9cc47981d05ec8d117ba859a22d8185ba quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5 2024-03-04T09:42:28.220 INFO:journalctl@ceph.osd.2.smithi191.stdout:Mar 04 09:42:27 smithi191 podman[73290]: 2024-03-04 09:42:27.986098372 +0000 UTC m=+0.131010981 container create 42688cda573e741f0a9c84e9042ba6f6d9d38ace3cbb536b41b83c1bac03b663 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5, name=ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-2-deactivate, org.label-schema.build-date=20240222, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-qdb-1bc250c, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-03-04T09:42:28.220 INFO:journalctl@ceph.osd.2.smithi191.stdout:Mar 04 09:42:28 smithi191 podman[73290]: 2024-03-04 09:42:28.222436502 +0000 UTC m=+0.367349080 container init 42688cda573e741f0a9c84e9042ba6f6d9d38ace3cbb536b41b83c1bac03b663 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5, name=ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-2-deactivate, CEPH_POINT_RELEASE=, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, RELEASE=wip-lusov-qdb-1bc250c, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240222, ceph=True, org.label-schema.license=GPLv2) 2024-03-04T09:42:28.509 INFO:journalctl@ceph.osd.2.smithi191.stdout:Mar 04 09:42:28 smithi191 podman[73290]: 2024-03-04 09:42:28.225775536 +0000 UTC m=+0.370688115 container start 42688cda573e741f0a9c84e9042ba6f6d9d38ace3cbb536b41b83c1bac03b663 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5, name=ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-2-deactivate, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-qdb-1bc250c, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240222, org.label-schema.license=GPLv2, ceph=True) 2024-03-04T09:42:28.509 INFO:journalctl@ceph.osd.2.smithi191.stdout:Mar 04 09:42:28 smithi191 podman[73290]: 2024-03-04 09:42:28.322532259 +0000 UTC m=+0.467444830 container attach 42688cda573e741f0a9c84e9042ba6f6d9d38ace3cbb536b41b83c1bac03b663 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5, name=ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-2-deactivate, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, RELEASE=wip-lusov-qdb-1bc250c, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240222, GIT_CLEAN=True) 2024-03-04T09:42:28.807 INFO:journalctl@ceph.osd.2.smithi191.stdout:Mar 04 09:42:28 smithi191 podman[73290]: 2024-03-04 09:42:28.511961387 +0000 UTC m=+0.656873961 container died 42688cda573e741f0a9c84e9042ba6f6d9d38ace3cbb536b41b83c1bac03b663 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5, name=ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-2-deactivate, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240222, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-qdb-1bc250c, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_CLEAN=True) 2024-03-04T09:42:29.096 DEBUG:teuthology.orchestra.run.smithi191:> sudo pkill -f 'journalctl -f -n 0 -u ceph-4e059f08-da08-11ee-95c6-87774f69a715@osd.2.service' 2024-03-04T09:42:29.137 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-04T09:42:29.138 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-03-04T09:42:29.138 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-03-04T09:42:29.138 DEBUG:teuthology.orchestra.run.smithi052:> sudo systemctl stop ceph-4e059f08-da08-11ee-95c6-87774f69a715@osd.3 2024-03-04T09:42:29.450 INFO:journalctl@ceph.osd.3.smithi052.stdout:Mar 04 09:42:29 smithi052 systemd[1]: Stopping Ceph osd.3 for 4e059f08-da08-11ee-95c6-87774f69a715... 2024-03-04T09:42:29.836 INFO:journalctl@ceph.osd.3.smithi052.stdout:Mar 04 09:42:29 smithi052 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-3[51542]: 2024-03-04T09:42:29.451+0000 7f159c209700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-04T09:42:29.836 INFO:journalctl@ceph.osd.3.smithi052.stdout:Mar 04 09:42:29 smithi052 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-3[51542]: 2024-03-04T09:42:29.451+0000 7f159c209700 -1 osd.3 106 *** Got signal Terminated *** 2024-03-04T09:42:29.836 INFO:journalctl@ceph.osd.3.smithi052.stdout:Mar 04 09:42:29 smithi052 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-3[51542]: 2024-03-04T09:42:29.451+0000 7f159c209700 -1 osd.3 106 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-04T09:42:32.836 INFO:journalctl@ceph.osd.9.smithi052.stdout:Mar 04 09:42:32 smithi052 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-9[65396]: 2024-03-04T09:42:32.476+0000 7fd1f1e8e700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-03-04T09:42:09.861821+0000 front 2024-03-04T09:42:09.861761+0000 (oldest deadline 2024-03-04T09:42:32.161859+0000) 2024-03-04T09:42:32.918 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 04 09:42:32 smithi106 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-10[60969]: 2024-03-04T09:42:32.662+0000 7f10f0d39700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-03-04T09:42:08.466029+0000 front 2024-03-04T09:42:08.465963+0000 (oldest deadline 2024-03-04T09:42:32.566054+0000) 2024-03-04T09:42:33.588 INFO:journalctl@ceph.osd.5.smithi191.stdout:Mar 04 09:42:33 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-5[48947]: 2024-03-04T09:42:33.106+0000 7ff1de4cc700 -1 osd.5 106 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-03-04T09:42:08.534684+0000 front 2024-03-04T09:42:08.534615+0000 (oldest deadline 2024-03-04T09:42:32.634794+0000) 2024-03-04T09:42:33.588 INFO:journalctl@ceph.osd.11.smithi191.stdout:Mar 04 09:42:33 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-11[62160]: 2024-03-04T09:42:33.181+0000 7fdceac97700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-03-04T09:42:08.553313+0000 front 2024-03-04T09:42:08.553176+0000 (oldest deadline 2024-03-04T09:42:32.653011+0000) 2024-03-04T09:42:33.836 INFO:journalctl@ceph.osd.9.smithi052.stdout:Mar 04 09:42:33 smithi052 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-9[65396]: 2024-03-04T09:42:33.479+0000 7fd1f1e8e700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-03-04T09:42:09.861821+0000 front 2024-03-04T09:42:09.861761+0000 (oldest deadline 2024-03-04T09:42:32.161859+0000) 2024-03-04T09:42:34.132 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 04 09:42:33 smithi106 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-10[60969]: 2024-03-04T09:42:33.701+0000 7f10f0d39700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-03-04T09:42:08.466029+0000 front 2024-03-04T09:42:08.465963+0000 (oldest deadline 2024-03-04T09:42:32.566054+0000) 2024-03-04T09:42:34.337 INFO:journalctl@ceph.osd.5.smithi191.stdout:Mar 04 09:42:34 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-5[48947]: 2024-03-04T09:42:34.079+0000 7ff1de4cc700 -1 osd.5 106 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-03-04T09:42:08.534684+0000 front 2024-03-04T09:42:08.534615+0000 (oldest deadline 2024-03-04T09:42:32.634794+0000) 2024-03-04T09:42:34.338 INFO:journalctl@ceph.osd.11.smithi191.stdout:Mar 04 09:42:34 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-11[62160]: 2024-03-04T09:42:34.230+0000 7fdceac97700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-03-04T09:42:08.553313+0000 front 2024-03-04T09:42:08.553176+0000 (oldest deadline 2024-03-04T09:42:32.653011+0000) 2024-03-04T09:42:34.836 INFO:journalctl@ceph.osd.9.smithi052.stdout:Mar 04 09:42:34 smithi052 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-9[65396]: 2024-03-04T09:42:34.508+0000 7fd1f1e8e700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-03-04T09:42:09.861821+0000 front 2024-03-04T09:42:09.861761+0000 (oldest deadline 2024-03-04T09:42:32.161859+0000) 2024-03-04T09:42:35.117 INFO:journalctl@ceph.osd.3.smithi052.stdout:Mar 04 09:42:34 smithi052 podman[96198]: 2024-03-04 09:42:34.848428704 +0000 UTC m=+5.506253585 container died 42b56a6f5c48a32e7f8772d74997e8f741a07c6ad521a57e59d30e0af99ea2ca (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5, name=ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-3, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-qdb-1bc250c, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.build-date=20240222, GIT_BRANCH=HEAD, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-03-04T09:42:35.132 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 04 09:42:34 smithi106 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-10[60969]: 2024-03-04T09:42:34.674+0000 7f10f0d39700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-03-04T09:42:08.466029+0000 front 2024-03-04T09:42:08.465963+0000 (oldest deadline 2024-03-04T09:42:32.566054+0000) 2024-03-04T09:42:35.511 INFO:journalctl@ceph.osd.3.smithi052.stdout:Mar 04 09:42:35 smithi052 podman[96198]: 2024-03-04 09:42:35.219173479 +0000 UTC m=+5.876998359 container cleanup 42b56a6f5c48a32e7f8772d74997e8f741a07c6ad521a57e59d30e0af99ea2ca (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5, name=ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-3, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240222, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, RELEASE=wip-lusov-qdb-1bc250c, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-03-04T09:42:35.511 INFO:journalctl@ceph.osd.3.smithi052.stdout:Mar 04 09:42:35 smithi052 bash[96198]: ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-3 2024-03-04T09:42:35.587 INFO:journalctl@ceph.osd.5.smithi191.stdout:Mar 04 09:42:35 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-5[48947]: 2024-03-04T09:42:35.125+0000 7ff1de4cc700 -1 osd.5 106 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-03-04T09:42:08.534684+0000 front 2024-03-04T09:42:08.534615+0000 (oldest deadline 2024-03-04T09:42:32.634794+0000) 2024-03-04T09:42:35.588 INFO:journalctl@ceph.osd.11.smithi191.stdout:Mar 04 09:42:35 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-11[62160]: 2024-03-04T09:42:35.234+0000 7fdceac97700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-03-04T09:42:08.553313+0000 front 2024-03-04T09:42:08.553176+0000 (oldest deadline 2024-03-04T09:42:32.653011+0000) 2024-03-04T09:42:35.836 INFO:journalctl@ceph.osd.3.smithi052.stdout:Mar 04 09:42:35 smithi052 podman[96215]: 2024-03-04 09:42:35.541157438 +0000 UTC m=+0.684961817 container remove 42b56a6f5c48a32e7f8772d74997e8f741a07c6ad521a57e59d30e0af99ea2ca (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5, name=ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-3, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.build-date=20240222, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, RELEASE=wip-lusov-qdb-1bc250c, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-03-04T09:42:35.836 INFO:journalctl@ceph.osd.9.smithi052.stdout:Mar 04 09:42:35 smithi052 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-9[65396]: 2024-03-04T09:42:35.513+0000 7fd1f1e8e700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-03-04T09:42:09.861821+0000 front 2024-03-04T09:42:09.861761+0000 (oldest deadline 2024-03-04T09:42:32.161859+0000) 2024-03-04T09:42:35.882 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 04 09:42:35 smithi106 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-10[60969]: 2024-03-04T09:42:35.633+0000 7f10f0d39700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-03-04T09:42:08.466029+0000 front 2024-03-04T09:42:08.465963+0000 (oldest deadline 2024-03-04T09:42:32.566054+0000) 2024-03-04T09:42:36.205 INFO:journalctl@ceph.osd.3.smithi052.stdout:Mar 04 09:42:35 smithi052 podman[96303]: 2024-03-04 09:42:35.871508399 +0000 UTC m=+0.098648130 container create 6fbe317a651e49fb3b0b07390270d05ced5598a42b793cb21afe13a714079916 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5, name=ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-3-deactivate, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=wip-lusov-qdb-1bc250c, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240222, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-03-04T09:42:36.205 INFO:journalctl@ceph.osd.3.smithi052.stdout:Mar 04 09:42:35 smithi052 podman[96303]: 2024-03-04 09:42:35.791226073 +0000 UTC m=+0.018365826 image pull f88ffb32aa6a7d5dfdc954d9f096f0c9cc47981d05ec8d117ba859a22d8185ba quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5 2024-03-04T09:42:36.205 INFO:journalctl@ceph.osd.3.smithi052.stdout:Mar 04 09:42:36 smithi052 podman[96303]: 2024-03-04 09:42:36.116241467 +0000 UTC m=+0.343381199 container init 6fbe317a651e49fb3b0b07390270d05ced5598a42b793cb21afe13a714079916 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5, name=ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-3-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240222, ceph=True, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-qdb-1bc250c, GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-03-04T09:42:36.206 INFO:journalctl@ceph.osd.3.smithi052.stdout:Mar 04 09:42:36 smithi052 podman[96303]: 2024-03-04 09:42:36.119313619 +0000 UTC m=+0.346453357 container start 6fbe317a651e49fb3b0b07390270d05ced5598a42b793cb21afe13a714079916 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5, name=ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-3-deactivate, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240222, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=wip-lusov-qdb-1bc250c, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-04T09:42:36.206 INFO:journalctl@ceph.osd.3.smithi052.stdout:Mar 04 09:42:36 smithi052 podman[96303]: 2024-03-04 09:42:36.207857407 +0000 UTC m=+0.434997165 container attach 6fbe317a651e49fb3b0b07390270d05ced5598a42b793cb21afe13a714079916 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5, name=ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-3-deactivate, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=wip-lusov-qdb-1bc250c, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240222, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5) 2024-03-04T09:42:36.548 INFO:journalctl@ceph.osd.3.smithi052.stdout:Mar 04 09:42:36 smithi052 podman[96303]: 2024-03-04 09:42:36.417337157 +0000 UTC m=+0.644476887 container died 6fbe317a651e49fb3b0b07390270d05ced5598a42b793cb21afe13a714079916 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5, name=ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-3-deactivate, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, RELEASE=wip-lusov-qdb-1bc250c, org.label-schema.license=GPLv2, org.label-schema.build-date=20240222, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, org.label-schema.schema-version=1.0) 2024-03-04T09:42:36.587 INFO:journalctl@ceph.osd.5.smithi191.stdout:Mar 04 09:42:36 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-5[48947]: 2024-03-04T09:42:36.104+0000 7ff1de4cc700 -1 osd.5 106 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-03-04T09:42:08.534684+0000 front 2024-03-04T09:42:08.534615+0000 (oldest deadline 2024-03-04T09:42:32.634794+0000) 2024-03-04T09:42:36.588 INFO:journalctl@ceph.osd.11.smithi191.stdout:Mar 04 09:42:36 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-11[62160]: 2024-03-04T09:42:36.249+0000 7fdceac97700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-03-04T09:42:08.553313+0000 front 2024-03-04T09:42:08.553176+0000 (oldest deadline 2024-03-04T09:42:32.653011+0000) 2024-03-04T09:42:36.836 INFO:journalctl@ceph.osd.9.smithi052.stdout:Mar 04 09:42:36 smithi052 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-9[65396]: 2024-03-04T09:42:36.550+0000 7fd1f1e8e700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-03-04T09:42:09.861821+0000 front 2024-03-04T09:42:09.861761+0000 (oldest deadline 2024-03-04T09:42:32.161859+0000) 2024-03-04T09:42:36.882 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 04 09:42:36 smithi106 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-10[60969]: 2024-03-04T09:42:36.587+0000 7f10f0d39700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-03-04T09:42:08.466029+0000 front 2024-03-04T09:42:08.465963+0000 (oldest deadline 2024-03-04T09:42:32.566054+0000) 2024-03-04T09:42:37.003 DEBUG:teuthology.orchestra.run.smithi052:> sudo pkill -f 'journalctl -f -n 0 -u ceph-4e059f08-da08-11ee-95c6-87774f69a715@osd.3.service' 2024-03-04T09:42:37.051 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-04T09:42:37.051 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-03-04T09:42:37.051 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-03-04T09:42:37.052 DEBUG:teuthology.orchestra.run.smithi106:> sudo systemctl stop ceph-4e059f08-da08-11ee-95c6-87774f69a715@osd.4 2024-03-04T09:42:37.338 INFO:journalctl@ceph.osd.5.smithi191.stdout:Mar 04 09:42:37 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-5[48947]: 2024-03-04T09:42:37.086+0000 7ff1de4cc700 -1 osd.5 106 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-03-04T09:42:08.534684+0000 front 2024-03-04T09:42:08.534615+0000 (oldest deadline 2024-03-04T09:42:32.634794+0000) 2024-03-04T09:42:37.338 INFO:journalctl@ceph.osd.11.smithi191.stdout:Mar 04 09:42:37 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-11[62160]: 2024-03-04T09:42:37.207+0000 7fdceac97700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-03-04T09:42:08.553313+0000 front 2024-03-04T09:42:08.553176+0000 (oldest deadline 2024-03-04T09:42:32.653011+0000) 2024-03-04T09:42:37.374 INFO:journalctl@ceph.osd.4.smithi106.stdout:Mar 04 09:42:37 smithi106 systemd[1]: Stopping Ceph osd.4 for 4e059f08-da08-11ee-95c6-87774f69a715... 2024-03-04T09:42:37.632 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 04 09:42:37 smithi106 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-10[60969]: 2024-03-04T09:42:37.592+0000 7f10f0d39700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-03-04T09:42:08.466029+0000 front 2024-03-04T09:42:08.465963+0000 (oldest deadline 2024-03-04T09:42:32.566054+0000) 2024-03-04T09:42:37.633 INFO:journalctl@ceph.osd.4.smithi106.stdout:Mar 04 09:42:37 smithi106 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-4[47787]: 2024-03-04T09:42:37.375+0000 7f6e5550c700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-04T09:42:37.633 INFO:journalctl@ceph.osd.4.smithi106.stdout:Mar 04 09:42:37 smithi106 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-4[47787]: 2024-03-04T09:42:37.375+0000 7f6e5550c700 -1 osd.4 106 *** Got signal Terminated *** 2024-03-04T09:42:37.633 INFO:journalctl@ceph.osd.4.smithi106.stdout:Mar 04 09:42:37 smithi106 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-4[47787]: 2024-03-04T09:42:37.375+0000 7f6e5550c700 -1 osd.4 106 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-04T09:42:37.836 INFO:journalctl@ceph.osd.9.smithi052.stdout:Mar 04 09:42:37 smithi052 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-9[65396]: 2024-03-04T09:42:37.533+0000 7fd1f1e8e700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-03-04T09:42:09.861821+0000 front 2024-03-04T09:42:09.861761+0000 (oldest deadline 2024-03-04T09:42:32.161859+0000) 2024-03-04T09:42:38.588 INFO:journalctl@ceph.osd.5.smithi191.stdout:Mar 04 09:42:38 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-5[48947]: 2024-03-04T09:42:38.117+0000 7ff1de4cc700 -1 osd.5 106 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-03-04T09:42:08.534684+0000 front 2024-03-04T09:42:08.534615+0000 (oldest deadline 2024-03-04T09:42:32.634794+0000) 2024-03-04T09:42:38.588 INFO:journalctl@ceph.osd.11.smithi191.stdout:Mar 04 09:42:38 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-11[62160]: 2024-03-04T09:42:38.249+0000 7fdceac97700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-03-04T09:42:08.553313+0000 front 2024-03-04T09:42:08.553176+0000 (oldest deadline 2024-03-04T09:42:32.653011+0000) 2024-03-04T09:42:38.836 INFO:journalctl@ceph.osd.9.smithi052.stdout:Mar 04 09:42:38 smithi052 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-9[65396]: 2024-03-04T09:42:38.488+0000 7fd1f1e8e700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-03-04T09:42:09.861821+0000 front 2024-03-04T09:42:09.861761+0000 (oldest deadline 2024-03-04T09:42:32.161859+0000) 2024-03-04T09:42:38.882 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 04 09:42:38 smithi106 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-10[60969]: 2024-03-04T09:42:38.564+0000 7f10f0d39700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-03-04T09:42:08.466029+0000 front 2024-03-04T09:42:08.465963+0000 (oldest deadline 2024-03-04T09:42:32.566054+0000) 2024-03-04T09:42:39.588 INFO:journalctl@ceph.osd.5.smithi191.stdout:Mar 04 09:42:39 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-5[48947]: 2024-03-04T09:42:39.145+0000 7ff1de4cc700 -1 osd.5 106 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-03-04T09:42:08.534684+0000 front 2024-03-04T09:42:08.534615+0000 (oldest deadline 2024-03-04T09:42:32.634794+0000) 2024-03-04T09:42:39.588 INFO:journalctl@ceph.osd.11.smithi191.stdout:Mar 04 09:42:39 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-11[62160]: 2024-03-04T09:42:39.282+0000 7fdceac97700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-03-04T09:42:08.553313+0000 front 2024-03-04T09:42:08.553176+0000 (oldest deadline 2024-03-04T09:42:32.653011+0000) 2024-03-04T09:42:39.714 INFO:journalctl@ceph.osd.9.smithi052.stdout:Mar 04 09:42:39 smithi052 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-9[65396]: 2024-03-04T09:42:39.464+0000 7fd1f1e8e700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-03-04T09:42:09.861821+0000 front 2024-03-04T09:42:09.861761+0000 (oldest deadline 2024-03-04T09:42:32.161859+0000) 2024-03-04T09:42:39.882 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 04 09:42:39 smithi106 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-10[60969]: 2024-03-04T09:42:39.593+0000 7f10f0d39700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-03-04T09:42:08.466029+0000 front 2024-03-04T09:42:08.465963+0000 (oldest deadline 2024-03-04T09:42:32.566054+0000) 2024-03-04T09:42:39.883 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 04 09:42:39 smithi106 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-10[60969]: 2024-03-04T09:42:39.593+0000 7f10f0d39700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-04T09:42:15.467191+0000 front 2024-03-04T09:42:15.467152+0000 (oldest deadline 2024-03-04T09:42:39.567351+0000) 2024-03-04T09:42:40.086 INFO:journalctl@ceph.osd.6.smithi052.stdout:Mar 04 09:42:39 smithi052 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-6[58363]: 2024-03-04T09:42:39.716+0000 7fa32e856700 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-04T09:42:18.565964+0000 front 2024-03-04T09:42:18.566095+0000 (oldest deadline 2024-03-04T09:42:39.666005+0000) 2024-03-04T09:42:40.588 INFO:journalctl@ceph.osd.11.smithi191.stdout:Mar 04 09:42:40 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-11[62160]: 2024-03-04T09:42:40.300+0000 7fdceac97700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-03-04T09:42:08.553313+0000 front 2024-03-04T09:42:08.553176+0000 (oldest deadline 2024-03-04T09:42:32.653011+0000) 2024-03-04T09:42:40.588 INFO:journalctl@ceph.osd.5.smithi191.stdout:Mar 04 09:42:40 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-5[48947]: 2024-03-04T09:42:40.187+0000 7ff1de4cc700 -1 osd.5 106 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-03-04T09:42:08.534684+0000 front 2024-03-04T09:42:08.534615+0000 (oldest deadline 2024-03-04T09:42:32.634794+0000) 2024-03-04T09:42:40.836 INFO:journalctl@ceph.osd.6.smithi052.stdout:Mar 04 09:42:40 smithi052 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-6[58363]: 2024-03-04T09:42:40.688+0000 7fa32e856700 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-04T09:42:18.565964+0000 front 2024-03-04T09:42:18.566095+0000 (oldest deadline 2024-03-04T09:42:39.666005+0000) 2024-03-04T09:42:40.836 INFO:journalctl@ceph.osd.9.smithi052.stdout:Mar 04 09:42:40 smithi052 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-9[65396]: 2024-03-04T09:42:40.480+0000 7fd1f1e8e700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-03-04T09:42:09.861821+0000 front 2024-03-04T09:42:09.861761+0000 (oldest deadline 2024-03-04T09:42:32.161859+0000) 2024-03-04T09:42:40.882 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 04 09:42:40 smithi106 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-10[60969]: 2024-03-04T09:42:40.601+0000 7f10f0d39700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-03-04T09:42:08.466029+0000 front 2024-03-04T09:42:08.465963+0000 (oldest deadline 2024-03-04T09:42:32.566054+0000) 2024-03-04T09:42:40.883 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 04 09:42:40 smithi106 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-10[60969]: 2024-03-04T09:42:40.601+0000 7f10f0d39700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-04T09:42:15.467191+0000 front 2024-03-04T09:42:15.467152+0000 (oldest deadline 2024-03-04T09:42:39.567351+0000) 2024-03-04T09:42:41.383 INFO:journalctl@ceph.osd.4.smithi106.stdout:Mar 04 09:42:40 smithi106 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-4[47787]: 2024-03-04T09:42:40.960+0000 7f6e51b00700 -1 osd.4 106 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-04T09:42:18.019490+0000 front 2024-03-04T09:42:18.019511+0000 (oldest deadline 2024-03-04T09:42:40.319683+0000) 2024-03-04T09:42:41.383 INFO:journalctl@ceph.osd.7.smithi106.stdout:Mar 04 09:42:40 smithi106 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-7[54346]: 2024-03-04T09:42:40.943+0000 7f3be1c52700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-04T09:42:17.071420+0000 front 2024-03-04T09:42:17.071488+0000 (oldest deadline 2024-03-04T09:42:40.472167+0000) 2024-03-04T09:42:41.588 INFO:journalctl@ceph.osd.11.smithi191.stdout:Mar 04 09:42:41 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-11[62160]: 2024-03-04T09:42:41.277+0000 7fdceac97700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-03-04T09:42:08.553313+0000 front 2024-03-04T09:42:08.553176+0000 (oldest deadline 2024-03-04T09:42:32.653011+0000) 2024-03-04T09:42:41.588 INFO:journalctl@ceph.osd.5.smithi191.stdout:Mar 04 09:42:41 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-5[48947]: 2024-03-04T09:42:41.216+0000 7ff1de4cc700 -1 osd.5 106 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-03-04T09:42:08.534684+0000 front 2024-03-04T09:42:08.534615+0000 (oldest deadline 2024-03-04T09:42:32.634794+0000) 2024-03-04T09:42:41.588 INFO:journalctl@ceph.osd.5.smithi191.stdout:Mar 04 09:42:41 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-5[48947]: 2024-03-04T09:42:41.216+0000 7ff1de4cc700 -1 osd.5 106 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-04T09:42:17.236674+0000 front 2024-03-04T09:42:17.236482+0000 (oldest deadline 2024-03-04T09:42:40.736400+0000) 2024-03-04T09:42:41.589 INFO:journalctl@ceph.osd.8.smithi191.stdout:Mar 04 09:42:41 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-8[55524]: 2024-03-04T09:42:41.454+0000 7f436df85700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-04T09:42:17.524704+0000 front 2024-03-04T09:42:17.524866+0000 (oldest deadline 2024-03-04T09:42:41.024775+0000) 2024-03-04T09:42:41.737 INFO:journalctl@ceph.osd.6.smithi052.stdout:Mar 04 09:42:41 smithi052 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-6[58363]: 2024-03-04T09:42:41.666+0000 7fa32e856700 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-04T09:42:18.565964+0000 front 2024-03-04T09:42:18.566095+0000 (oldest deadline 2024-03-04T09:42:39.666005+0000) 2024-03-04T09:42:41.738 INFO:journalctl@ceph.osd.9.smithi052.stdout:Mar 04 09:42:41 smithi052 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-9[65396]: 2024-03-04T09:42:41.457+0000 7fd1f1e8e700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-03-04T09:42:09.861821+0000 front 2024-03-04T09:42:09.861761+0000 (oldest deadline 2024-03-04T09:42:32.161859+0000) 2024-03-04T09:42:41.882 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 04 09:42:41 smithi106 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-10[60969]: 2024-03-04T09:42:41.562+0000 7f10f0d39700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-03-04T09:42:08.466029+0000 front 2024-03-04T09:42:08.465963+0000 (oldest deadline 2024-03-04T09:42:32.566054+0000) 2024-03-04T09:42:41.882 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 04 09:42:41 smithi106 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-10[60969]: 2024-03-04T09:42:41.562+0000 7f10f0d39700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-04T09:42:15.467191+0000 front 2024-03-04T09:42:15.467152+0000 (oldest deadline 2024-03-04T09:42:39.567351+0000) 2024-03-04T09:42:42.382 INFO:journalctl@ceph.osd.4.smithi106.stdout:Mar 04 09:42:41 smithi106 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-4[47787]: 2024-03-04T09:42:41.987+0000 7f6e51b00700 -1 osd.4 106 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-04T09:42:18.019490+0000 front 2024-03-04T09:42:18.019511+0000 (oldest deadline 2024-03-04T09:42:40.319683+0000) 2024-03-04T09:42:42.383 INFO:journalctl@ceph.osd.7.smithi106.stdout:Mar 04 09:42:41 smithi106 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-7[54346]: 2024-03-04T09:42:41.907+0000 7f3be1c52700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-04T09:42:17.071420+0000 front 2024-03-04T09:42:17.071488+0000 (oldest deadline 2024-03-04T09:42:40.472167+0000) 2024-03-04T09:42:42.442 INFO:journalctl@ceph.osd.11.smithi191.stdout:Mar 04 09:42:42 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-11[62160]: 2024-03-04T09:42:42.279+0000 7fdceac97700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-03-04T09:42:08.553313+0000 front 2024-03-04T09:42:08.553176+0000 (oldest deadline 2024-03-04T09:42:32.653011+0000) 2024-03-04T09:42:42.443 INFO:journalctl@ceph.osd.5.smithi191.stdout:Mar 04 09:42:42 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-5[48947]: 2024-03-04T09:42:42.175+0000 7ff1de4cc700 -1 osd.5 106 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-03-04T09:42:08.534684+0000 front 2024-03-04T09:42:08.534615+0000 (oldest deadline 2024-03-04T09:42:32.634794+0000) 2024-03-04T09:42:42.443 INFO:journalctl@ceph.osd.5.smithi191.stdout:Mar 04 09:42:42 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-5[48947]: 2024-03-04T09:42:42.175+0000 7ff1de4cc700 -1 osd.5 106 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-04T09:42:17.236674+0000 front 2024-03-04T09:42:17.236482+0000 (oldest deadline 2024-03-04T09:42:40.736400+0000) 2024-03-04T09:42:42.781 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 04 09:42:42 smithi106 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-10[60969]: 2024-03-04T09:42:42.526+0000 7f10f0d39700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-03-04T09:42:08.466029+0000 front 2024-03-04T09:42:08.465963+0000 (oldest deadline 2024-03-04T09:42:32.566054+0000) 2024-03-04T09:42:42.781 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 04 09:42:42 smithi106 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-10[60969]: 2024-03-04T09:42:42.526+0000 7f10f0d39700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-04T09:42:15.467191+0000 front 2024-03-04T09:42:15.467152+0000 (oldest deadline 2024-03-04T09:42:39.567351+0000) 2024-03-04T09:42:42.836 INFO:journalctl@ceph.osd.6.smithi052.stdout:Mar 04 09:42:42 smithi052 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-6[58363]: 2024-03-04T09:42:42.652+0000 7fa32e856700 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-04T09:42:18.565964+0000 front 2024-03-04T09:42:18.566095+0000 (oldest deadline 2024-03-04T09:42:39.666005+0000) 2024-03-04T09:42:42.836 INFO:journalctl@ceph.osd.9.smithi052.stdout:Mar 04 09:42:42 smithi052 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-9[65396]: 2024-03-04T09:42:42.415+0000 7fd1f1e8e700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-03-04T09:42:09.861821+0000 front 2024-03-04T09:42:09.861761+0000 (oldest deadline 2024-03-04T09:42:32.161859+0000) 2024-03-04T09:42:42.837 INFO:journalctl@ceph.osd.9.smithi052.stdout:Mar 04 09:42:42 smithi052 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-9[65396]: 2024-03-04T09:42:42.415+0000 7fd1f1e8e700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-04T09:42:17.363956+0000 front 2024-03-04T09:42:17.363876+0000 (oldest deadline 2024-03-04T09:42:41.463546+0000) 2024-03-04T09:42:42.837 INFO:journalctl@ceph.osd.8.smithi191.stdout:Mar 04 09:42:42 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-8[55524]: 2024-03-04T09:42:42.444+0000 7f436df85700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-04T09:42:17.524704+0000 front 2024-03-04T09:42:17.524866+0000 (oldest deadline 2024-03-04T09:42:41.024775+0000) 2024-03-04T09:42:43.084 INFO:journalctl@ceph.osd.7.smithi106.stdout:Mar 04 09:42:42 smithi106 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-7[54346]: 2024-03-04T09:42:42.878+0000 7f3be1c52700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-04T09:42:17.071420+0000 front 2024-03-04T09:42:17.071488+0000 (oldest deadline 2024-03-04T09:42:40.472167+0000) 2024-03-04T09:42:43.084 INFO:journalctl@ceph.osd.4.smithi106.stdout:Mar 04 09:42:42 smithi106 podman[72432]: 2024-03-04 09:42:42.783618114 +0000 UTC m=+5.527941419 container died 703e793e68e47ee1d3c2c8d74485a9724c84d6340b0de96ac317a71d6b31322f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5, name=ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-4, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, RELEASE=wip-lusov-qdb-1bc250c, org.label-schema.build-date=20240222) 2024-03-04T09:42:43.382 INFO:journalctl@ceph.osd.4.smithi106.stdout:Mar 04 09:42:43 smithi106 podman[72432]: 2024-03-04 09:42:43.176894926 +0000 UTC m=+5.921218238 container cleanup 703e793e68e47ee1d3c2c8d74485a9724c84d6340b0de96ac317a71d6b31322f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5, name=ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-4, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-qdb-1bc250c, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240222, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-04T09:42:43.383 INFO:journalctl@ceph.osd.4.smithi106.stdout:Mar 04 09:42:43 smithi106 bash[72432]: ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-4 2024-03-04T09:42:43.588 INFO:journalctl@ceph.osd.8.smithi191.stdout:Mar 04 09:42:43 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-8[55524]: 2024-03-04T09:42:43.406+0000 7f436df85700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-04T09:42:17.524704+0000 front 2024-03-04T09:42:17.524866+0000 (oldest deadline 2024-03-04T09:42:41.024775+0000) 2024-03-04T09:42:43.588 INFO:journalctl@ceph.osd.5.smithi191.stdout:Mar 04 09:42:43 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-5[48947]: 2024-03-04T09:42:43.168+0000 7ff1de4cc700 -1 osd.5 106 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-03-04T09:42:08.534684+0000 front 2024-03-04T09:42:08.534615+0000 (oldest deadline 2024-03-04T09:42:32.634794+0000) 2024-03-04T09:42:43.588 INFO:journalctl@ceph.osd.5.smithi191.stdout:Mar 04 09:42:43 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-5[48947]: 2024-03-04T09:42:43.168+0000 7ff1de4cc700 -1 osd.5 106 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-04T09:42:17.236674+0000 front 2024-03-04T09:42:17.236482+0000 (oldest deadline 2024-03-04T09:42:40.736400+0000) 2024-03-04T09:42:43.588 INFO:journalctl@ceph.osd.11.smithi191.stdout:Mar 04 09:42:43 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-11[62160]: 2024-03-04T09:42:43.262+0000 7fdceac97700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-03-04T09:42:08.553313+0000 front 2024-03-04T09:42:08.553176+0000 (oldest deadline 2024-03-04T09:42:32.653011+0000) 2024-03-04T09:42:43.651 INFO:journalctl@ceph.osd.9.smithi052.stdout:Mar 04 09:42:43 smithi052 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-9[65396]: 2024-03-04T09:42:43.398+0000 7fd1f1e8e700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-03-04T09:42:09.861821+0000 front 2024-03-04T09:42:09.861761+0000 (oldest deadline 2024-03-04T09:42:32.161859+0000) 2024-03-04T09:42:43.651 INFO:journalctl@ceph.osd.9.smithi052.stdout:Mar 04 09:42:43 smithi052 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-9[65396]: 2024-03-04T09:42:43.398+0000 7fd1f1e8e700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-04T09:42:17.363956+0000 front 2024-03-04T09:42:17.363876+0000 (oldest deadline 2024-03-04T09:42:41.463546+0000) 2024-03-04T09:42:43.747 INFO:journalctl@ceph.osd.4.smithi106.stdout:Mar 04 09:42:43 smithi106 podman[72447]: 2024-03-04 09:42:43.419838718 +0000 UTC m=+0.637940241 container remove 703e793e68e47ee1d3c2c8d74485a9724c84d6340b0de96ac317a71d6b31322f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5, name=ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-4, RELEASE=wip-lusov-qdb-1bc250c, ceph=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240222, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2) 2024-03-04T09:42:43.748 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 04 09:42:43 smithi106 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-10[60969]: 2024-03-04T09:42:43.518+0000 7f10f0d39700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-03-04T09:42:08.466029+0000 front 2024-03-04T09:42:08.465963+0000 (oldest deadline 2024-03-04T09:42:32.566054+0000) 2024-03-04T09:42:43.748 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 04 09:42:43 smithi106 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-10[60969]: 2024-03-04T09:42:43.518+0000 7f10f0d39700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-04T09:42:15.467191+0000 front 2024-03-04T09:42:15.467152+0000 (oldest deadline 2024-03-04T09:42:39.567351+0000) 2024-03-04T09:42:44.064 INFO:journalctl@ceph.osd.7.smithi106.stdout:Mar 04 09:42:43 smithi106 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-7[54346]: 2024-03-04T09:42:43.915+0000 7f3be1c52700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-04T09:42:17.071420+0000 front 2024-03-04T09:42:17.071488+0000 (oldest deadline 2024-03-04T09:42:40.472167+0000) 2024-03-04T09:42:44.064 INFO:journalctl@ceph.osd.4.smithi106.stdout:Mar 04 09:42:43 smithi106 podman[72542]: 2024-03-04 09:42:43.653280679 +0000 UTC m=+0.017500216 image pull f88ffb32aa6a7d5dfdc954d9f096f0c9cc47981d05ec8d117ba859a22d8185ba quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5 2024-03-04T09:42:44.065 INFO:journalctl@ceph.osd.4.smithi106.stdout:Mar 04 09:42:43 smithi106 podman[72542]: 2024-03-04 09:42:43.788933695 +0000 UTC m=+0.153153225 container create eddf2fdd5fe58b6f87c82db58d63fd09354f09f7a2cd157b2788a12f0d85d0bd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5, name=ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-4-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.build-date=20240222, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-qdb-1bc250c, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-03-04T09:42:44.065 INFO:journalctl@ceph.osd.4.smithi106.stdout:Mar 04 09:42:44 smithi106 podman[72542]: 2024-03-04 09:42:44.06525701 +0000 UTC m=+0.429476559 container init eddf2fdd5fe58b6f87c82db58d63fd09354f09f7a2cd157b2788a12f0d85d0bd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5, name=ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-4-deactivate, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240222, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-qdb-1bc250c, io.buildah.version=1.33.5, ceph=True, CEPH_POINT_RELEASE=) 2024-03-04T09:42:44.086 INFO:journalctl@ceph.osd.6.smithi052.stdout:Mar 04 09:42:43 smithi052 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-6[58363]: 2024-03-04T09:42:43.652+0000 7fa32e856700 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-04T09:42:18.565964+0000 front 2024-03-04T09:42:18.566095+0000 (oldest deadline 2024-03-04T09:42:39.666005+0000) 2024-03-04T09:42:44.353 INFO:journalctl@ceph.osd.4.smithi106.stdout:Mar 04 09:42:44 smithi106 podman[72542]: 2024-03-04 09:42:44.068368774 +0000 UTC m=+0.432588306 container start eddf2fdd5fe58b6f87c82db58d63fd09354f09f7a2cd157b2788a12f0d85d0bd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5, name=ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-4-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240222, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-qdb-1bc250c, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, ceph=True) 2024-03-04T09:42:44.354 INFO:journalctl@ceph.osd.4.smithi106.stdout:Mar 04 09:42:44 smithi106 podman[72542]: 2024-03-04 09:42:44.165224407 +0000 UTC m=+0.529443967 container attach eddf2fdd5fe58b6f87c82db58d63fd09354f09f7a2cd157b2788a12f0d85d0bd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5, name=ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-4-deactivate, org.label-schema.schema-version=1.0, ceph=True, RELEASE=wip-lusov-qdb-1bc250c, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240222, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-04T09:42:44.587 INFO:journalctl@ceph.osd.8.smithi191.stdout:Mar 04 09:42:44 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-8[55524]: 2024-03-04T09:42:44.384+0000 7f436df85700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-04T09:42:17.524704+0000 front 2024-03-04T09:42:17.524866+0000 (oldest deadline 2024-03-04T09:42:41.024775+0000) 2024-03-04T09:42:44.588 INFO:journalctl@ceph.osd.11.smithi191.stdout:Mar 04 09:42:44 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-11[62160]: 2024-03-04T09:42:44.243+0000 7fdceac97700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-03-04T09:42:08.553313+0000 front 2024-03-04T09:42:08.553176+0000 (oldest deadline 2024-03-04T09:42:32.653011+0000) 2024-03-04T09:42:44.588 INFO:journalctl@ceph.osd.5.smithi191.stdout:Mar 04 09:42:44 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-5[48947]: 2024-03-04T09:42:44.149+0000 7ff1de4cc700 -1 osd.5 106 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-03-04T09:42:08.534684+0000 front 2024-03-04T09:42:08.534615+0000 (oldest deadline 2024-03-04T09:42:32.634794+0000) 2024-03-04T09:42:44.588 INFO:journalctl@ceph.osd.5.smithi191.stdout:Mar 04 09:42:44 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-5[48947]: 2024-03-04T09:42:44.149+0000 7ff1de4cc700 -1 osd.5 106 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-04T09:42:17.236674+0000 front 2024-03-04T09:42:17.236482+0000 (oldest deadline 2024-03-04T09:42:40.736400+0000) 2024-03-04T09:42:44.632 INFO:journalctl@ceph.osd.4.smithi106.stdout:Mar 04 09:42:44 smithi106 podman[72542]: 2024-03-04 09:42:44.356464357 +0000 UTC m=+0.720683902 container died eddf2fdd5fe58b6f87c82db58d63fd09354f09f7a2cd157b2788a12f0d85d0bd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5, name=ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-4-deactivate, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.build-date=20240222, RELEASE=wip-lusov-qdb-1bc250c, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-03-04T09:42:44.633 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 04 09:42:44 smithi106 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-10[60969]: 2024-03-04T09:42:44.560+0000 7f10f0d39700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-03-04T09:42:08.466029+0000 front 2024-03-04T09:42:08.465963+0000 (oldest deadline 2024-03-04T09:42:32.566054+0000) 2024-03-04T09:42:44.633 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 04 09:42:44 smithi106 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-10[60969]: 2024-03-04T09:42:44.560+0000 7f10f0d39700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-04T09:42:15.467191+0000 front 2024-03-04T09:42:15.467152+0000 (oldest deadline 2024-03-04T09:42:39.567351+0000) 2024-03-04T09:42:44.694 INFO:journalctl@ceph.osd.9.smithi052.stdout:Mar 04 09:42:44 smithi052 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-9[65396]: 2024-03-04T09:42:44.405+0000 7fd1f1e8e700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-03-04T09:42:09.861821+0000 front 2024-03-04T09:42:09.861761+0000 (oldest deadline 2024-03-04T09:42:32.161859+0000) 2024-03-04T09:42:44.695 INFO:journalctl@ceph.osd.9.smithi052.stdout:Mar 04 09:42:44 smithi052 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-9[65396]: 2024-03-04T09:42:44.405+0000 7fd1f1e8e700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-04T09:42:17.363956+0000 front 2024-03-04T09:42:17.363876+0000 (oldest deadline 2024-03-04T09:42:41.463546+0000) 2024-03-04T09:42:44.966 INFO:journalctl@ceph.osd.7.smithi106.stdout:Mar 04 09:42:44 smithi106 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-7[54346]: 2024-03-04T09:42:44.905+0000 7f3be1c52700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-04T09:42:17.071420+0000 front 2024-03-04T09:42:17.071488+0000 (oldest deadline 2024-03-04T09:42:40.472167+0000) 2024-03-04T09:42:44.988 DEBUG:teuthology.orchestra.run.smithi106:> sudo pkill -f 'journalctl -f -n 0 -u ceph-4e059f08-da08-11ee-95c6-87774f69a715@osd.4.service' 2024-03-04T09:42:45.029 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-04T09:42:45.029 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-03-04T09:42:45.030 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-03-04T09:42:45.030 DEBUG:teuthology.orchestra.run.smithi191:> sudo systemctl stop ceph-4e059f08-da08-11ee-95c6-87774f69a715@osd.5 2024-03-04T09:42:45.086 INFO:journalctl@ceph.osd.6.smithi052.stdout:Mar 04 09:42:44 smithi052 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-6[58363]: 2024-03-04T09:42:44.696+0000 7fa32e856700 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-04T09:42:18.565964+0000 front 2024-03-04T09:42:18.566095+0000 (oldest deadline 2024-03-04T09:42:39.666005+0000) 2024-03-04T09:42:45.338 INFO:journalctl@ceph.osd.5.smithi191.stdout:Mar 04 09:42:45 smithi191 systemd[1]: Stopping Ceph osd.5 for 4e059f08-da08-11ee-95c6-87774f69a715... 2024-03-04T09:42:45.338 INFO:journalctl@ceph.osd.5.smithi191.stdout:Mar 04 09:42:45 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-5[48947]: 2024-03-04T09:42:45.164+0000 7ff1de4cc700 -1 osd.5 106 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-03-04T09:42:08.534684+0000 front 2024-03-04T09:42:08.534615+0000 (oldest deadline 2024-03-04T09:42:32.634794+0000) 2024-03-04T09:42:45.338 INFO:journalctl@ceph.osd.5.smithi191.stdout:Mar 04 09:42:45 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-5[48947]: 2024-03-04T09:42:45.164+0000 7ff1de4cc700 -1 osd.5 106 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-04T09:42:17.236674+0000 front 2024-03-04T09:42:17.236482+0000 (oldest deadline 2024-03-04T09:42:40.736400+0000) 2024-03-04T09:42:45.338 INFO:journalctl@ceph.osd.11.smithi191.stdout:Mar 04 09:42:45 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-11[62160]: 2024-03-04T09:42:45.194+0000 7fdceac97700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-03-04T09:42:08.553313+0000 front 2024-03-04T09:42:08.553176+0000 (oldest deadline 2024-03-04T09:42:32.653011+0000) 2024-03-04T09:42:45.339 INFO:journalctl@ceph.osd.11.smithi191.stdout:Mar 04 09:42:45 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-11[62160]: 2024-03-04T09:42:45.194+0000 7fdceac97700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-04T09:42:18.355200+0000 front 2024-03-04T09:42:18.355189+0000 (oldest deadline 2024-03-04T09:42:44.254936+0000) 2024-03-04T09:42:45.653 INFO:journalctl@ceph.osd.9.smithi052.stdout:Mar 04 09:42:45 smithi052 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-9[65396]: 2024-03-04T09:42:45.360+0000 7fd1f1e8e700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-03-04T09:42:09.861821+0000 front 2024-03-04T09:42:09.861761+0000 (oldest deadline 2024-03-04T09:42:32.161859+0000) 2024-03-04T09:42:45.653 INFO:journalctl@ceph.osd.9.smithi052.stdout:Mar 04 09:42:45 smithi052 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-9[65396]: 2024-03-04T09:42:45.360+0000 7fd1f1e8e700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-04T09:42:17.363956+0000 front 2024-03-04T09:42:17.363876+0000 (oldest deadline 2024-03-04T09:42:41.463546+0000) 2024-03-04T09:42:45.837 INFO:journalctl@ceph.osd.8.smithi191.stdout:Mar 04 09:42:45 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-8[55524]: 2024-03-04T09:42:45.365+0000 7f436df85700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-04T09:42:17.524704+0000 front 2024-03-04T09:42:17.524866+0000 (oldest deadline 2024-03-04T09:42:41.024775+0000) 2024-03-04T09:42:45.838 INFO:journalctl@ceph.osd.5.smithi191.stdout:Mar 04 09:42:45 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-5[48947]: 2024-03-04T09:42:45.352+0000 7ff1e5ee0700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-04T09:42:45.838 INFO:journalctl@ceph.osd.5.smithi191.stdout:Mar 04 09:42:45 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-5[48947]: 2024-03-04T09:42:45.352+0000 7ff1e5ee0700 -1 osd.5 106 *** Got signal Terminated *** 2024-03-04T09:42:45.838 INFO:journalctl@ceph.osd.5.smithi191.stdout:Mar 04 09:42:45 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-5[48947]: 2024-03-04T09:42:45.352+0000 7ff1e5ee0700 -1 osd.5 106 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-04T09:42:45.866 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 04 09:42:45 smithi106 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-10[60969]: 2024-03-04T09:42:45.588+0000 7f10f0d39700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-03-04T09:42:08.466029+0000 front 2024-03-04T09:42:08.465963+0000 (oldest deadline 2024-03-04T09:42:32.566054+0000) 2024-03-04T09:42:45.866 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 04 09:42:45 smithi106 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-10[60969]: 2024-03-04T09:42:45.588+0000 7f10f0d39700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-04T09:42:15.467191+0000 front 2024-03-04T09:42:15.467152+0000 (oldest deadline 2024-03-04T09:42:39.567351+0000) 2024-03-04T09:42:46.086 INFO:journalctl@ceph.osd.6.smithi052.stdout:Mar 04 09:42:45 smithi052 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-6[58363]: 2024-03-04T09:42:45.655+0000 7fa32e856700 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-04T09:42:18.565964+0000 front 2024-03-04T09:42:18.566095+0000 (oldest deadline 2024-03-04T09:42:39.666005+0000) 2024-03-04T09:42:46.132 INFO:journalctl@ceph.osd.7.smithi106.stdout:Mar 04 09:42:45 smithi106 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-7[54346]: 2024-03-04T09:42:45.868+0000 7f3be1c52700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-04T09:42:17.071420+0000 front 2024-03-04T09:42:17.071488+0000 (oldest deadline 2024-03-04T09:42:40.472167+0000) 2024-03-04T09:42:46.588 INFO:journalctl@ceph.osd.8.smithi191.stdout:Mar 04 09:42:46 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-8[55524]: 2024-03-04T09:42:46.383+0000 7f436df85700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-04T09:42:17.524704+0000 front 2024-03-04T09:42:17.524866+0000 (oldest deadline 2024-03-04T09:42:41.024775+0000) 2024-03-04T09:42:46.588 INFO:journalctl@ceph.osd.5.smithi191.stdout:Mar 04 09:42:46 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-5[48947]: 2024-03-04T09:42:46.175+0000 7ff1de4cc700 -1 osd.5 106 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-03-04T09:42:08.534684+0000 front 2024-03-04T09:42:08.534615+0000 (oldest deadline 2024-03-04T09:42:32.634794+0000) 2024-03-04T09:42:46.588 INFO:journalctl@ceph.osd.5.smithi191.stdout:Mar 04 09:42:46 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-5[48947]: 2024-03-04T09:42:46.175+0000 7ff1de4cc700 -1 osd.5 106 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-04T09:42:17.236674+0000 front 2024-03-04T09:42:17.236482+0000 (oldest deadline 2024-03-04T09:42:40.736400+0000) 2024-03-04T09:42:46.589 INFO:journalctl@ceph.osd.11.smithi191.stdout:Mar 04 09:42:46 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-11[62160]: 2024-03-04T09:42:46.210+0000 7fdceac97700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-03-04T09:42:08.553313+0000 front 2024-03-04T09:42:08.553176+0000 (oldest deadline 2024-03-04T09:42:32.653011+0000) 2024-03-04T09:42:46.589 INFO:journalctl@ceph.osd.11.smithi191.stdout:Mar 04 09:42:46 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-11[62160]: 2024-03-04T09:42:46.210+0000 7fdceac97700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-04T09:42:18.355200+0000 front 2024-03-04T09:42:18.355189+0000 (oldest deadline 2024-03-04T09:42:44.254936+0000) 2024-03-04T09:42:46.699 INFO:journalctl@ceph.osd.9.smithi052.stdout:Mar 04 09:42:46 smithi052 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-9[65396]: 2024-03-04T09:42:46.386+0000 7fd1f1e8e700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-03-04T09:42:09.861821+0000 front 2024-03-04T09:42:09.861761+0000 (oldest deadline 2024-03-04T09:42:32.161859+0000) 2024-03-04T09:42:46.700 INFO:journalctl@ceph.osd.9.smithi052.stdout:Mar 04 09:42:46 smithi052 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-9[65396]: 2024-03-04T09:42:46.386+0000 7fd1f1e8e700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-04T09:42:17.363956+0000 front 2024-03-04T09:42:17.363876+0000 (oldest deadline 2024-03-04T09:42:41.463546+0000) 2024-03-04T09:42:46.853 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 04 09:42:46 smithi106 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-10[60969]: 2024-03-04T09:42:46.546+0000 7f10f0d39700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-03-04T09:42:08.466029+0000 front 2024-03-04T09:42:08.465963+0000 (oldest deadline 2024-03-04T09:42:32.566054+0000) 2024-03-04T09:42:46.854 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 04 09:42:46 smithi106 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-10[60969]: 2024-03-04T09:42:46.546+0000 7f10f0d39700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-04T09:42:15.467191+0000 front 2024-03-04T09:42:15.467152+0000 (oldest deadline 2024-03-04T09:42:39.567351+0000) 2024-03-04T09:42:47.086 INFO:journalctl@ceph.osd.6.smithi052.stdout:Mar 04 09:42:46 smithi052 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-6[58363]: 2024-03-04T09:42:46.701+0000 7fa32e856700 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-04T09:42:18.565964+0000 front 2024-03-04T09:42:18.566095+0000 (oldest deadline 2024-03-04T09:42:39.666005+0000) 2024-03-04T09:42:47.132 INFO:journalctl@ceph.osd.7.smithi106.stdout:Mar 04 09:42:46 smithi106 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-7[54346]: 2024-03-04T09:42:46.855+0000 7f3be1c52700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-04T09:42:17.071420+0000 front 2024-03-04T09:42:17.071488+0000 (oldest deadline 2024-03-04T09:42:40.472167+0000) 2024-03-04T09:42:47.587 INFO:journalctl@ceph.osd.8.smithi191.stdout:Mar 04 09:42:47 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-8[55524]: 2024-03-04T09:42:47.379+0000 7f436df85700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-04T09:42:17.524704+0000 front 2024-03-04T09:42:17.524866+0000 (oldest deadline 2024-03-04T09:42:41.024775+0000) 2024-03-04T09:42:47.588 INFO:journalctl@ceph.osd.11.smithi191.stdout:Mar 04 09:42:47 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-11[62160]: 2024-03-04T09:42:47.228+0000 7fdceac97700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-03-04T09:42:08.553313+0000 front 2024-03-04T09:42:08.553176+0000 (oldest deadline 2024-03-04T09:42:32.653011+0000) 2024-03-04T09:42:47.588 INFO:journalctl@ceph.osd.11.smithi191.stdout:Mar 04 09:42:47 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-11[62160]: 2024-03-04T09:42:47.228+0000 7fdceac97700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-04T09:42:18.355200+0000 front 2024-03-04T09:42:18.355189+0000 (oldest deadline 2024-03-04T09:42:44.254936+0000) 2024-03-04T09:42:47.588 INFO:journalctl@ceph.osd.11.smithi191.stdout:Mar 04 09:42:47 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-11[62160]: 2024-03-04T09:42:47.228+0000 7fdceac97700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-03-04T09:42:24.255361+0000 front 2024-03-04T09:42:24.255246+0000 (oldest deadline 2024-03-04T09:42:47.155465+0000) 2024-03-04T09:42:47.589 INFO:journalctl@ceph.osd.5.smithi191.stdout:Mar 04 09:42:47 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-5[48947]: 2024-03-04T09:42:47.212+0000 7ff1de4cc700 -1 osd.5 106 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-03-04T09:42:08.534684+0000 front 2024-03-04T09:42:08.534615+0000 (oldest deadline 2024-03-04T09:42:32.634794+0000) 2024-03-04T09:42:47.589 INFO:journalctl@ceph.osd.5.smithi191.stdout:Mar 04 09:42:47 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-5[48947]: 2024-03-04T09:42:47.212+0000 7ff1de4cc700 -1 osd.5 106 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-04T09:42:17.236674+0000 front 2024-03-04T09:42:17.236482+0000 (oldest deadline 2024-03-04T09:42:40.736400+0000) 2024-03-04T09:42:47.670 INFO:journalctl@ceph.osd.9.smithi052.stdout:Mar 04 09:42:47 smithi052 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-9[65396]: 2024-03-04T09:42:47.396+0000 7fd1f1e8e700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-03-04T09:42:09.861821+0000 front 2024-03-04T09:42:09.861761+0000 (oldest deadline 2024-03-04T09:42:32.161859+0000) 2024-03-04T09:42:47.670 INFO:journalctl@ceph.osd.9.smithi052.stdout:Mar 04 09:42:47 smithi052 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-9[65396]: 2024-03-04T09:42:47.396+0000 7fd1f1e8e700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-04T09:42:17.363956+0000 front 2024-03-04T09:42:17.363876+0000 (oldest deadline 2024-03-04T09:42:41.463546+0000) 2024-03-04T09:42:47.670 INFO:journalctl@ceph.osd.9.smithi052.stdout:Mar 04 09:42:47 smithi052 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-9[65396]: 2024-03-04T09:42:47.396+0000 7fd1f1e8e700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-03-04T09:42:21.464200+0000 front 2024-03-04T09:42:21.464139+0000 (oldest deadline 2024-03-04T09:42:47.363942+0000) 2024-03-04T09:42:47.810 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 04 09:42:47 smithi106 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-10[60969]: 2024-03-04T09:42:47.525+0000 7f10f0d39700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-03-04T09:42:08.466029+0000 front 2024-03-04T09:42:08.465963+0000 (oldest deadline 2024-03-04T09:42:32.566054+0000) 2024-03-04T09:42:47.811 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 04 09:42:47 smithi106 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-10[60969]: 2024-03-04T09:42:47.525+0000 7f10f0d39700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-04T09:42:15.467191+0000 front 2024-03-04T09:42:15.467152+0000 (oldest deadline 2024-03-04T09:42:39.567351+0000) 2024-03-04T09:42:48.086 INFO:journalctl@ceph.osd.6.smithi052.stdout:Mar 04 09:42:47 smithi052 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-6[58363]: 2024-03-04T09:42:47.672+0000 7fa32e856700 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-04T09:42:18.565964+0000 front 2024-03-04T09:42:18.566095+0000 (oldest deadline 2024-03-04T09:42:39.666005+0000) 2024-03-04T09:42:48.132 INFO:journalctl@ceph.osd.7.smithi106.stdout:Mar 04 09:42:47 smithi106 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-7[54346]: 2024-03-04T09:42:47.812+0000 7f3be1c52700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-04T09:42:17.071420+0000 front 2024-03-04T09:42:17.071488+0000 (oldest deadline 2024-03-04T09:42:40.472167+0000) 2024-03-04T09:42:48.586 INFO:journalctl@ceph.osd.9.smithi052.stdout:Mar 04 09:42:48 smithi052 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-9[65396]: 2024-03-04T09:42:48.417+0000 7fd1f1e8e700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-03-04T09:42:09.861821+0000 front 2024-03-04T09:42:09.861761+0000 (oldest deadline 2024-03-04T09:42:32.161859+0000) 2024-03-04T09:42:48.586 INFO:journalctl@ceph.osd.9.smithi052.stdout:Mar 04 09:42:48 smithi052 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-9[65396]: 2024-03-04T09:42:48.417+0000 7fd1f1e8e700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-04T09:42:17.363956+0000 front 2024-03-04T09:42:17.363876+0000 (oldest deadline 2024-03-04T09:42:41.463546+0000) 2024-03-04T09:42:48.587 INFO:journalctl@ceph.osd.9.smithi052.stdout:Mar 04 09:42:48 smithi052 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-9[65396]: 2024-03-04T09:42:48.417+0000 7fd1f1e8e700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-03-04T09:42:21.464200+0000 front 2024-03-04T09:42:21.464139+0000 (oldest deadline 2024-03-04T09:42:47.363942+0000) 2024-03-04T09:42:48.587 INFO:journalctl@ceph.osd.5.smithi191.stdout:Mar 04 09:42:48 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-5[48947]: 2024-03-04T09:42:48.194+0000 7ff1de4cc700 -1 osd.5 106 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-03-04T09:42:08.534684+0000 front 2024-03-04T09:42:08.534615+0000 (oldest deadline 2024-03-04T09:42:32.634794+0000) 2024-03-04T09:42:48.587 INFO:journalctl@ceph.osd.5.smithi191.stdout:Mar 04 09:42:48 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-5[48947]: 2024-03-04T09:42:48.194+0000 7ff1de4cc700 -1 osd.5 106 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-04T09:42:17.236674+0000 front 2024-03-04T09:42:17.236482+0000 (oldest deadline 2024-03-04T09:42:40.736400+0000) 2024-03-04T09:42:48.588 INFO:journalctl@ceph.osd.8.smithi191.stdout:Mar 04 09:42:48 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-8[55524]: 2024-03-04T09:42:48.388+0000 7f436df85700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-04T09:42:17.524704+0000 front 2024-03-04T09:42:17.524866+0000 (oldest deadline 2024-03-04T09:42:41.024775+0000) 2024-03-04T09:42:48.588 INFO:journalctl@ceph.osd.8.smithi191.stdout:Mar 04 09:42:48 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-8[55524]: 2024-03-04T09:42:48.388+0000 7f436df85700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-03-04T09:42:25.125775+0000 front 2024-03-04T09:42:25.125737+0000 (oldest deadline 2024-03-04T09:42:48.025913+0000) 2024-03-04T09:42:48.588 INFO:journalctl@ceph.osd.11.smithi191.stdout:Mar 04 09:42:48 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-11[62160]: 2024-03-04T09:42:48.189+0000 7fdceac97700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-03-04T09:42:08.553313+0000 front 2024-03-04T09:42:08.553176+0000 (oldest deadline 2024-03-04T09:42:32.653011+0000) 2024-03-04T09:42:48.588 INFO:journalctl@ceph.osd.11.smithi191.stdout:Mar 04 09:42:48 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-11[62160]: 2024-03-04T09:42:48.189+0000 7fdceac97700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-04T09:42:18.355200+0000 front 2024-03-04T09:42:18.355189+0000 (oldest deadline 2024-03-04T09:42:44.254936+0000) 2024-03-04T09:42:48.589 INFO:journalctl@ceph.osd.11.smithi191.stdout:Mar 04 09:42:48 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-11[62160]: 2024-03-04T09:42:48.189+0000 7fdceac97700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-03-04T09:42:24.255361+0000 front 2024-03-04T09:42:24.255246+0000 (oldest deadline 2024-03-04T09:42:47.155465+0000) 2024-03-04T09:42:48.809 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 04 09:42:48 smithi106 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-10[60969]: 2024-03-04T09:42:48.536+0000 7f10f0d39700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-03-04T09:42:08.466029+0000 front 2024-03-04T09:42:08.465963+0000 (oldest deadline 2024-03-04T09:42:32.566054+0000) 2024-03-04T09:42:48.810 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 04 09:42:48 smithi106 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-10[60969]: 2024-03-04T09:42:48.536+0000 7f10f0d39700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-04T09:42:15.467191+0000 front 2024-03-04T09:42:15.467152+0000 (oldest deadline 2024-03-04T09:42:39.567351+0000) 2024-03-04T09:42:49.086 INFO:journalctl@ceph.osd.6.smithi052.stdout:Mar 04 09:42:48 smithi052 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-6[58363]: 2024-03-04T09:42:48.663+0000 7fa32e856700 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-04T09:42:18.565964+0000 front 2024-03-04T09:42:18.566095+0000 (oldest deadline 2024-03-04T09:42:39.666005+0000) 2024-03-04T09:42:49.132 INFO:journalctl@ceph.osd.7.smithi106.stdout:Mar 04 09:42:48 smithi106 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-7[54346]: 2024-03-04T09:42:48.811+0000 7f3be1c52700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-04T09:42:17.071420+0000 front 2024-03-04T09:42:17.071488+0000 (oldest deadline 2024-03-04T09:42:40.472167+0000) 2024-03-04T09:42:49.133 INFO:journalctl@ceph.osd.7.smithi106.stdout:Mar 04 09:42:48 smithi106 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-7[54346]: 2024-03-04T09:42:48.811+0000 7f3be1c52700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-03-04T09:42:25.173267+0000 front 2024-03-04T09:42:25.173398+0000 (oldest deadline 2024-03-04T09:42:48.673397+0000) 2024-03-04T09:42:49.587 INFO:journalctl@ceph.osd.8.smithi191.stdout:Mar 04 09:42:49 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-8[55524]: 2024-03-04T09:42:49.412+0000 7f436df85700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-04T09:42:17.524704+0000 front 2024-03-04T09:42:17.524866+0000 (oldest deadline 2024-03-04T09:42:41.024775+0000) 2024-03-04T09:42:49.588 INFO:journalctl@ceph.osd.8.smithi191.stdout:Mar 04 09:42:49 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-8[55524]: 2024-03-04T09:42:49.412+0000 7f436df85700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-03-04T09:42:25.125775+0000 front 2024-03-04T09:42:25.125737+0000 (oldest deadline 2024-03-04T09:42:48.025913+0000) 2024-03-04T09:42:49.591 INFO:journalctl@ceph.osd.5.smithi191.stdout:Mar 04 09:42:49 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-5[48947]: 2024-03-04T09:42:49.202+0000 7ff1de4cc700 -1 osd.5 106 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-03-04T09:42:08.534684+0000 front 2024-03-04T09:42:08.534615+0000 (oldest deadline 2024-03-04T09:42:32.634794+0000) 2024-03-04T09:42:49.591 INFO:journalctl@ceph.osd.5.smithi191.stdout:Mar 04 09:42:49 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-5[48947]: 2024-03-04T09:42:49.202+0000 7ff1de4cc700 -1 osd.5 106 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-04T09:42:17.236674+0000 front 2024-03-04T09:42:17.236482+0000 (oldest deadline 2024-03-04T09:42:40.736400+0000) 2024-03-04T09:42:49.591 INFO:journalctl@ceph.osd.5.smithi191.stdout:Mar 04 09:42:49 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-5[48947]: 2024-03-04T09:42:49.202+0000 7ff1de4cc700 -1 osd.5 106 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-03-04T09:42:20.736784+0000 front 2024-03-04T09:42:20.736866+0000 (oldest deadline 2024-03-04T09:42:48.337522+0000) 2024-03-04T09:42:49.591 INFO:journalctl@ceph.osd.11.smithi191.stdout:Mar 04 09:42:49 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-11[62160]: 2024-03-04T09:42:49.224+0000 7fdceac97700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-03-04T09:42:08.553313+0000 front 2024-03-04T09:42:08.553176+0000 (oldest deadline 2024-03-04T09:42:32.653011+0000) 2024-03-04T09:42:49.591 INFO:journalctl@ceph.osd.11.smithi191.stdout:Mar 04 09:42:49 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-11[62160]: 2024-03-04T09:42:49.224+0000 7fdceac97700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-04T09:42:18.355200+0000 front 2024-03-04T09:42:18.355189+0000 (oldest deadline 2024-03-04T09:42:44.254936+0000) 2024-03-04T09:42:49.592 INFO:journalctl@ceph.osd.11.smithi191.stdout:Mar 04 09:42:49 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-11[62160]: 2024-03-04T09:42:49.224+0000 7fdceac97700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-03-04T09:42:24.255361+0000 front 2024-03-04T09:42:24.255246+0000 (oldest deadline 2024-03-04T09:42:47.155465+0000) 2024-03-04T09:42:49.836 INFO:journalctl@ceph.osd.6.smithi052.stdout:Mar 04 09:42:49 smithi052 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-6[58363]: 2024-03-04T09:42:49.619+0000 7fa32e856700 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-04T09:42:18.565964+0000 front 2024-03-04T09:42:18.566095+0000 (oldest deadline 2024-03-04T09:42:39.666005+0000) 2024-03-04T09:42:49.836 INFO:journalctl@ceph.osd.9.smithi052.stdout:Mar 04 09:42:49 smithi052 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-9[65396]: 2024-03-04T09:42:49.431+0000 7fd1f1e8e700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-03-04T09:42:09.861821+0000 front 2024-03-04T09:42:09.861761+0000 (oldest deadline 2024-03-04T09:42:32.161859+0000) 2024-03-04T09:42:49.836 INFO:journalctl@ceph.osd.9.smithi052.stdout:Mar 04 09:42:49 smithi052 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-9[65396]: 2024-03-04T09:42:49.431+0000 7fd1f1e8e700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-04T09:42:17.363956+0000 front 2024-03-04T09:42:17.363876+0000 (oldest deadline 2024-03-04T09:42:41.463546+0000) 2024-03-04T09:42:49.837 INFO:journalctl@ceph.osd.9.smithi052.stdout:Mar 04 09:42:49 smithi052 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-9[65396]: 2024-03-04T09:42:49.431+0000 7fd1f1e8e700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-03-04T09:42:21.464200+0000 front 2024-03-04T09:42:21.464139+0000 (oldest deadline 2024-03-04T09:42:47.363942+0000) 2024-03-04T09:42:49.837 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 04 09:42:49 smithi106 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-10[60969]: 2024-03-04T09:42:49.586+0000 7f10f0d39700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-03-04T09:42:08.466029+0000 front 2024-03-04T09:42:08.465963+0000 (oldest deadline 2024-03-04T09:42:32.566054+0000) 2024-03-04T09:42:49.837 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 04 09:42:49 smithi106 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-10[60969]: 2024-03-04T09:42:49.586+0000 7f10f0d39700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-04T09:42:15.467191+0000 front 2024-03-04T09:42:15.467152+0000 (oldest deadline 2024-03-04T09:42:39.567351+0000) 2024-03-04T09:42:49.837 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 04 09:42:49 smithi106 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-10[60969]: 2024-03-04T09:42:49.586+0000 7f10f0d39700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-03-04T09:42:25.368854+0000 front 2024-03-04T09:42:25.368833+0000 (oldest deadline 2024-03-04T09:42:48.868990+0000) 2024-03-04T09:42:50.132 INFO:journalctl@ceph.osd.7.smithi106.stdout:Mar 04 09:42:49 smithi106 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-7[54346]: 2024-03-04T09:42:49.837+0000 7f3be1c52700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-04T09:42:17.071420+0000 front 2024-03-04T09:42:17.071488+0000 (oldest deadline 2024-03-04T09:42:40.472167+0000) 2024-03-04T09:42:50.132 INFO:journalctl@ceph.osd.7.smithi106.stdout:Mar 04 09:42:49 smithi106 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-7[54346]: 2024-03-04T09:42:49.837+0000 7f3be1c52700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-03-04T09:42:25.173267+0000 front 2024-03-04T09:42:25.173398+0000 (oldest deadline 2024-03-04T09:42:48.673397+0000) 2024-03-04T09:42:50.588 INFO:journalctl@ceph.osd.8.smithi191.stdout:Mar 04 09:42:50 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-8[55524]: 2024-03-04T09:42:50.398+0000 7f436df85700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-04T09:42:17.524704+0000 front 2024-03-04T09:42:17.524866+0000 (oldest deadline 2024-03-04T09:42:41.024775+0000) 2024-03-04T09:42:50.588 INFO:journalctl@ceph.osd.8.smithi191.stdout:Mar 04 09:42:50 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-8[55524]: 2024-03-04T09:42:50.398+0000 7f436df85700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-03-04T09:42:25.125775+0000 front 2024-03-04T09:42:25.125737+0000 (oldest deadline 2024-03-04T09:42:48.025913+0000) 2024-03-04T09:42:50.588 INFO:journalctl@ceph.osd.5.smithi191.stdout:Mar 04 09:42:50 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-5[48947]: 2024-03-04T09:42:50.168+0000 7ff1de4cc700 -1 osd.5 106 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-03-04T09:42:08.534684+0000 front 2024-03-04T09:42:08.534615+0000 (oldest deadline 2024-03-04T09:42:32.634794+0000) 2024-03-04T09:42:50.589 INFO:journalctl@ceph.osd.5.smithi191.stdout:Mar 04 09:42:50 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-5[48947]: 2024-03-04T09:42:50.168+0000 7ff1de4cc700 -1 osd.5 106 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-04T09:42:17.236674+0000 front 2024-03-04T09:42:17.236482+0000 (oldest deadline 2024-03-04T09:42:40.736400+0000) 2024-03-04T09:42:50.589 INFO:journalctl@ceph.osd.5.smithi191.stdout:Mar 04 09:42:50 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-5[48947]: 2024-03-04T09:42:50.168+0000 7ff1de4cc700 -1 osd.5 106 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-03-04T09:42:20.736784+0000 front 2024-03-04T09:42:20.736866+0000 (oldest deadline 2024-03-04T09:42:48.337522+0000) 2024-03-04T09:42:50.589 INFO:journalctl@ceph.osd.11.smithi191.stdout:Mar 04 09:42:50 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-11[62160]: 2024-03-04T09:42:50.230+0000 7fdceac97700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-03-04T09:42:08.553313+0000 front 2024-03-04T09:42:08.553176+0000 (oldest deadline 2024-03-04T09:42:32.653011+0000) 2024-03-04T09:42:50.589 INFO:journalctl@ceph.osd.11.smithi191.stdout:Mar 04 09:42:50 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-11[62160]: 2024-03-04T09:42:50.230+0000 7fdceac97700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-04T09:42:18.355200+0000 front 2024-03-04T09:42:18.355189+0000 (oldest deadline 2024-03-04T09:42:44.254936+0000) 2024-03-04T09:42:50.589 INFO:journalctl@ceph.osd.11.smithi191.stdout:Mar 04 09:42:50 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-11[62160]: 2024-03-04T09:42:50.230+0000 7fdceac97700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-03-04T09:42:24.255361+0000 front 2024-03-04T09:42:24.255246+0000 (oldest deadline 2024-03-04T09:42:47.155465+0000) 2024-03-04T09:42:50.836 INFO:journalctl@ceph.osd.6.smithi052.stdout:Mar 04 09:42:50 smithi052 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-6[58363]: 2024-03-04T09:42:50.638+0000 7fa32e856700 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-04T09:42:18.565964+0000 front 2024-03-04T09:42:18.566095+0000 (oldest deadline 2024-03-04T09:42:39.666005+0000) 2024-03-04T09:42:50.836 INFO:journalctl@ceph.osd.6.smithi052.stdout:Mar 04 09:42:50 smithi052 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-6[58363]: 2024-03-04T09:42:50.638+0000 7fa32e856700 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-03-04T09:42:23.767434+0000 front 2024-03-04T09:42:23.767558+0000 (oldest deadline 2024-03-04T09:42:49.667486+0000) 2024-03-04T09:42:50.837 INFO:journalctl@ceph.osd.9.smithi052.stdout:Mar 04 09:42:50 smithi052 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-9[65396]: 2024-03-04T09:42:50.458+0000 7fd1f1e8e700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-03-04T09:42:09.861821+0000 front 2024-03-04T09:42:09.861761+0000 (oldest deadline 2024-03-04T09:42:32.161859+0000) 2024-03-04T09:42:50.837 INFO:journalctl@ceph.osd.9.smithi052.stdout:Mar 04 09:42:50 smithi052 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-9[65396]: 2024-03-04T09:42:50.458+0000 7fd1f1e8e700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-04T09:42:17.363956+0000 front 2024-03-04T09:42:17.363876+0000 (oldest deadline 2024-03-04T09:42:41.463546+0000) 2024-03-04T09:42:50.837 INFO:journalctl@ceph.osd.9.smithi052.stdout:Mar 04 09:42:50 smithi052 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-9[65396]: 2024-03-04T09:42:50.458+0000 7fd1f1e8e700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-03-04T09:42:21.464200+0000 front 2024-03-04T09:42:21.464139+0000 (oldest deadline 2024-03-04T09:42:47.363942+0000) 2024-03-04T09:42:50.873 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 04 09:42:50 smithi106 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-10[60969]: 2024-03-04T09:42:50.586+0000 7f10f0d39700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-03-04T09:42:08.466029+0000 front 2024-03-04T09:42:08.465963+0000 (oldest deadline 2024-03-04T09:42:32.566054+0000) 2024-03-04T09:42:50.873 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 04 09:42:50 smithi106 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-10[60969]: 2024-03-04T09:42:50.586+0000 7f10f0d39700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-04T09:42:15.467191+0000 front 2024-03-04T09:42:15.467152+0000 (oldest deadline 2024-03-04T09:42:39.567351+0000) 2024-03-04T09:42:50.874 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 04 09:42:50 smithi106 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-10[60969]: 2024-03-04T09:42:50.586+0000 7f10f0d39700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-03-04T09:42:25.368854+0000 front 2024-03-04T09:42:25.368833+0000 (oldest deadline 2024-03-04T09:42:48.868990+0000) 2024-03-04T09:42:51.077 INFO:journalctl@ceph.osd.5.smithi191.stdout:Mar 04 09:42:50 smithi191 podman[73395]: 2024-03-04 09:42:50.797131642 +0000 UTC m=+5.572683045 container died e7711eace7bdf85fa26e692f93b5f39cecbade3ca3de7b8c68cdca86309acd53 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5, name=ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-5, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240222, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, RELEASE=wip-lusov-qdb-1bc250c) 2024-03-04T09:42:51.132 INFO:journalctl@ceph.osd.7.smithi106.stdout:Mar 04 09:42:50 smithi106 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-7[54346]: 2024-03-04T09:42:50.874+0000 7f3be1c52700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-04T09:42:17.071420+0000 front 2024-03-04T09:42:17.071488+0000 (oldest deadline 2024-03-04T09:42:40.472167+0000) 2024-03-04T09:42:51.133 INFO:journalctl@ceph.osd.7.smithi106.stdout:Mar 04 09:42:50 smithi106 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-7[54346]: 2024-03-04T09:42:50.874+0000 7f3be1c52700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-03-04T09:42:25.173267+0000 front 2024-03-04T09:42:25.173398+0000 (oldest deadline 2024-03-04T09:42:48.673397+0000) 2024-03-04T09:42:51.338 INFO:journalctl@ceph.osd.5.smithi191.stdout:Mar 04 09:42:51 smithi191 podman[73395]: 2024-03-04 09:42:51.162521393 +0000 UTC m=+5.938072796 container cleanup e7711eace7bdf85fa26e692f93b5f39cecbade3ca3de7b8c68cdca86309acd53 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5, name=ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-5, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-qdb-1bc250c, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240222) 2024-03-04T09:42:51.338 INFO:journalctl@ceph.osd.5.smithi191.stdout:Mar 04 09:42:51 smithi191 bash[73395]: ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-5 2024-03-04T09:42:51.339 INFO:journalctl@ceph.osd.11.smithi191.stdout:Mar 04 09:42:51 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-11[62160]: 2024-03-04T09:42:51.252+0000 7fdceac97700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-03-04T09:42:08.553313+0000 front 2024-03-04T09:42:08.553176+0000 (oldest deadline 2024-03-04T09:42:32.653011+0000) 2024-03-04T09:42:51.339 INFO:journalctl@ceph.osd.11.smithi191.stdout:Mar 04 09:42:51 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-11[62160]: 2024-03-04T09:42:51.252+0000 7fdceac97700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-04T09:42:18.355200+0000 front 2024-03-04T09:42:18.355189+0000 (oldest deadline 2024-03-04T09:42:44.254936+0000) 2024-03-04T09:42:51.339 INFO:journalctl@ceph.osd.11.smithi191.stdout:Mar 04 09:42:51 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-11[62160]: 2024-03-04T09:42:51.252+0000 7fdceac97700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-03-04T09:42:24.255361+0000 front 2024-03-04T09:42:24.255246+0000 (oldest deadline 2024-03-04T09:42:47.155465+0000) 2024-03-04T09:42:51.771 INFO:journalctl@ceph.osd.5.smithi191.stdout:Mar 04 09:42:51 smithi191 podman[73412]: 2024-03-04 09:42:51.45627159 +0000 UTC m=+0.656649236 container remove e7711eace7bdf85fa26e692f93b5f39cecbade3ca3de7b8c68cdca86309acd53 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5, name=ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-5, GIT_BRANCH=HEAD, org.label-schema.build-date=20240222, org.label-schema.schema-version=1.0, ceph=True, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-qdb-1bc250c, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-04T09:42:51.771 INFO:journalctl@ceph.osd.8.smithi191.stdout:Mar 04 09:42:51 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-8[55524]: 2024-03-04T09:42:51.363+0000 7f436df85700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-04T09:42:17.524704+0000 front 2024-03-04T09:42:17.524866+0000 (oldest deadline 2024-03-04T09:42:41.024775+0000) 2024-03-04T09:42:51.772 INFO:journalctl@ceph.osd.8.smithi191.stdout:Mar 04 09:42:51 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-8[55524]: 2024-03-04T09:42:51.363+0000 7f436df85700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-03-04T09:42:25.125775+0000 front 2024-03-04T09:42:25.125737+0000 (oldest deadline 2024-03-04T09:42:48.025913+0000) 2024-03-04T09:42:51.836 INFO:journalctl@ceph.osd.6.smithi052.stdout:Mar 04 09:42:51 smithi052 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-6[58363]: 2024-03-04T09:42:51.680+0000 7fa32e856700 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-04T09:42:18.565964+0000 front 2024-03-04T09:42:18.566095+0000 (oldest deadline 2024-03-04T09:42:39.666005+0000) 2024-03-04T09:42:51.836 INFO:journalctl@ceph.osd.6.smithi052.stdout:Mar 04 09:42:51 smithi052 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-6[58363]: 2024-03-04T09:42:51.680+0000 7fa32e856700 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-03-04T09:42:23.767434+0000 front 2024-03-04T09:42:23.767558+0000 (oldest deadline 2024-03-04T09:42:49.667486+0000) 2024-03-04T09:42:51.837 INFO:journalctl@ceph.osd.9.smithi052.stdout:Mar 04 09:42:51 smithi052 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-9[65396]: 2024-03-04T09:42:51.444+0000 7fd1f1e8e700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-03-04T09:42:09.861821+0000 front 2024-03-04T09:42:09.861761+0000 (oldest deadline 2024-03-04T09:42:32.161859+0000) 2024-03-04T09:42:51.837 INFO:journalctl@ceph.osd.9.smithi052.stdout:Mar 04 09:42:51 smithi052 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-9[65396]: 2024-03-04T09:42:51.444+0000 7fd1f1e8e700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-04T09:42:17.363956+0000 front 2024-03-04T09:42:17.363876+0000 (oldest deadline 2024-03-04T09:42:41.463546+0000) 2024-03-04T09:42:51.837 INFO:journalctl@ceph.osd.9.smithi052.stdout:Mar 04 09:42:51 smithi052 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-9[65396]: 2024-03-04T09:42:51.444+0000 7fd1f1e8e700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-03-04T09:42:21.464200+0000 front 2024-03-04T09:42:21.464139+0000 (oldest deadline 2024-03-04T09:42:47.363942+0000) 2024-03-04T09:42:51.882 INFO:journalctl@ceph.osd.7.smithi106.stdout:Mar 04 09:42:51 smithi106 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-7[54346]: 2024-03-04T09:42:51.826+0000 7f3be1c52700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-04T09:42:17.071420+0000 front 2024-03-04T09:42:17.071488+0000 (oldest deadline 2024-03-04T09:42:40.472167+0000) 2024-03-04T09:42:51.883 INFO:journalctl@ceph.osd.7.smithi106.stdout:Mar 04 09:42:51 smithi106 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-7[54346]: 2024-03-04T09:42:51.826+0000 7f3be1c52700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-03-04T09:42:25.173267+0000 front 2024-03-04T09:42:25.173398+0000 (oldest deadline 2024-03-04T09:42:48.673397+0000) 2024-03-04T09:42:51.883 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 04 09:42:51 smithi106 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-10[60969]: 2024-03-04T09:42:51.595+0000 7f10f0d39700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-03-04T09:42:08.466029+0000 front 2024-03-04T09:42:08.465963+0000 (oldest deadline 2024-03-04T09:42:32.566054+0000) 2024-03-04T09:42:51.884 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 04 09:42:51 smithi106 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-10[60969]: 2024-03-04T09:42:51.595+0000 7f10f0d39700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-04T09:42:15.467191+0000 front 2024-03-04T09:42:15.467152+0000 (oldest deadline 2024-03-04T09:42:39.567351+0000) 2024-03-04T09:42:51.884 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 04 09:42:51 smithi106 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-10[60969]: 2024-03-04T09:42:51.595+0000 7f10f0d39700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-03-04T09:42:25.368854+0000 front 2024-03-04T09:42:25.368833+0000 (oldest deadline 2024-03-04T09:42:48.868990+0000) 2024-03-04T09:42:52.088 INFO:journalctl@ceph.osd.5.smithi191.stdout:Mar 04 09:42:51 smithi191 podman[73506]: 2024-03-04 09:42:51.773831061 +0000 UTC m=+0.111070892 container create b1193cac79ff00c80b2162cd6a03af61b96f4c11eef53ca88b4e4a521571aa43 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5, name=ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-5-deactivate, org.label-schema.build-date=20240222, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True, RELEASE=wip-lusov-qdb-1bc250c, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8) 2024-03-04T09:42:52.088 INFO:journalctl@ceph.osd.5.smithi191.stdout:Mar 04 09:42:51 smithi191 podman[73506]: 2024-03-04 09:42:51.679848714 +0000 UTC m=+0.017088537 image pull f88ffb32aa6a7d5dfdc954d9f096f0c9cc47981d05ec8d117ba859a22d8185ba quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5 2024-03-04T09:42:52.088 INFO:journalctl@ceph.osd.5.smithi191.stdout:Mar 04 09:42:52 smithi191 podman[73506]: 2024-03-04 09:42:52.00139977 +0000 UTC m=+0.338639592 container init b1193cac79ff00c80b2162cd6a03af61b96f4c11eef53ca88b4e4a521571aa43 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5, name=ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-5-deactivate, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240222, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-qdb-1bc250c) 2024-03-04T09:42:52.089 INFO:journalctl@ceph.osd.5.smithi191.stdout:Mar 04 09:42:52 smithi191 podman[73506]: 2024-03-04 09:42:52.005163638 +0000 UTC m=+0.342403454 container start b1193cac79ff00c80b2162cd6a03af61b96f4c11eef53ca88b4e4a521571aa43 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5, name=ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-5-deactivate, GIT_CLEAN=True, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-qdb-1bc250c, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, GIT_BRANCH=HEAD, org.label-schema.build-date=20240222, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=) 2024-03-04T09:42:52.585 INFO:journalctl@ceph.osd.8.smithi191.stdout:Mar 04 09:42:52 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-8[55524]: 2024-03-04T09:42:52.345+0000 7f436df85700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-04T09:42:17.524704+0000 front 2024-03-04T09:42:17.524866+0000 (oldest deadline 2024-03-04T09:42:41.024775+0000) 2024-03-04T09:42:52.585 INFO:journalctl@ceph.osd.8.smithi191.stdout:Mar 04 09:42:52 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-8[55524]: 2024-03-04T09:42:52.345+0000 7f436df85700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-03-04T09:42:25.125775+0000 front 2024-03-04T09:42:25.125737+0000 (oldest deadline 2024-03-04T09:42:48.025913+0000) 2024-03-04T09:42:52.586 INFO:journalctl@ceph.osd.5.smithi191.stdout:Mar 04 09:42:52 smithi191 podman[73506]: 2024-03-04 09:42:52.111054868 +0000 UTC m=+0.448294682 container attach b1193cac79ff00c80b2162cd6a03af61b96f4c11eef53ca88b4e4a521571aa43 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5, name=ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-5-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, RELEASE=wip-lusov-qdb-1bc250c, org.label-schema.schema-version=1.0, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, org.label-schema.build-date=20240222, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=) 2024-03-04T09:42:52.586 INFO:journalctl@ceph.osd.5.smithi191.stdout:Mar 04 09:42:52 smithi191 podman[73506]: 2024-03-04 09:42:52.296186032 +0000 UTC m=+0.633425849 container died b1193cac79ff00c80b2162cd6a03af61b96f4c11eef53ca88b4e4a521571aa43 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5, name=ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-5-deactivate, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240222, org.label-schema.license=GPLv2, RELEASE=wip-lusov-qdb-1bc250c, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-03-04T09:42:52.586 INFO:journalctl@ceph.osd.11.smithi191.stdout:Mar 04 09:42:52 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-11[62160]: 2024-03-04T09:42:52.276+0000 7fdceac97700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-03-04T09:42:08.553313+0000 front 2024-03-04T09:42:08.553176+0000 (oldest deadline 2024-03-04T09:42:32.653011+0000) 2024-03-04T09:42:52.586 INFO:journalctl@ceph.osd.11.smithi191.stdout:Mar 04 09:42:52 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-11[62160]: 2024-03-04T09:42:52.276+0000 7fdceac97700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-04T09:42:18.355200+0000 front 2024-03-04T09:42:18.355189+0000 (oldest deadline 2024-03-04T09:42:44.254936+0000) 2024-03-04T09:42:52.586 INFO:journalctl@ceph.osd.11.smithi191.stdout:Mar 04 09:42:52 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-11[62160]: 2024-03-04T09:42:52.276+0000 7fdceac97700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-03-04T09:42:24.255361+0000 front 2024-03-04T09:42:24.255246+0000 (oldest deadline 2024-03-04T09:42:47.155465+0000) 2024-03-04T09:42:52.729 INFO:journalctl@ceph.osd.9.smithi052.stdout:Mar 04 09:42:52 smithi052 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-9[65396]: 2024-03-04T09:42:52.479+0000 7fd1f1e8e700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-03-04T09:42:09.861821+0000 front 2024-03-04T09:42:09.861761+0000 (oldest deadline 2024-03-04T09:42:32.161859+0000) 2024-03-04T09:42:52.729 INFO:journalctl@ceph.osd.9.smithi052.stdout:Mar 04 09:42:52 smithi052 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-9[65396]: 2024-03-04T09:42:52.479+0000 7fd1f1e8e700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-04T09:42:17.363956+0000 front 2024-03-04T09:42:17.363876+0000 (oldest deadline 2024-03-04T09:42:41.463546+0000) 2024-03-04T09:42:52.729 INFO:journalctl@ceph.osd.9.smithi052.stdout:Mar 04 09:42:52 smithi052 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-9[65396]: 2024-03-04T09:42:52.479+0000 7fd1f1e8e700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-03-04T09:42:21.464200+0000 front 2024-03-04T09:42:21.464139+0000 (oldest deadline 2024-03-04T09:42:47.363942+0000) 2024-03-04T09:42:52.848 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 04 09:42:52 smithi106 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-10[60969]: 2024-03-04T09:42:52.546+0000 7f10f0d39700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-03-04T09:42:08.466029+0000 front 2024-03-04T09:42:08.465963+0000 (oldest deadline 2024-03-04T09:42:32.566054+0000) 2024-03-04T09:42:52.848 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 04 09:42:52 smithi106 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-10[60969]: 2024-03-04T09:42:52.546+0000 7f10f0d39700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-04T09:42:15.467191+0000 front 2024-03-04T09:42:15.467152+0000 (oldest deadline 2024-03-04T09:42:39.567351+0000) 2024-03-04T09:42:52.848 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 04 09:42:52 smithi106 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-10[60969]: 2024-03-04T09:42:52.546+0000 7f10f0d39700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-03-04T09:42:25.368854+0000 front 2024-03-04T09:42:25.368833+0000 (oldest deadline 2024-03-04T09:42:48.868990+0000) 2024-03-04T09:42:52.871 DEBUG:teuthology.orchestra.run.smithi191:> sudo pkill -f 'journalctl -f -n 0 -u ceph-4e059f08-da08-11ee-95c6-87774f69a715@osd.5.service' 2024-03-04T09:42:52.915 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-04T09:42:52.915 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-03-04T09:42:52.915 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-03-04T09:42:52.915 DEBUG:teuthology.orchestra.run.smithi052:> sudo systemctl stop ceph-4e059f08-da08-11ee-95c6-87774f69a715@osd.6 2024-03-04T09:42:53.086 INFO:journalctl@ceph.osd.6.smithi052.stdout:Mar 04 09:42:52 smithi052 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-6[58363]: 2024-03-04T09:42:52.731+0000 7fa32e856700 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-04T09:42:18.565964+0000 front 2024-03-04T09:42:18.566095+0000 (oldest deadline 2024-03-04T09:42:39.666005+0000) 2024-03-04T09:42:53.086 INFO:journalctl@ceph.osd.6.smithi052.stdout:Mar 04 09:42:52 smithi052 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-6[58363]: 2024-03-04T09:42:52.731+0000 7fa32e856700 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-03-04T09:42:23.767434+0000 front 2024-03-04T09:42:23.767558+0000 (oldest deadline 2024-03-04T09:42:49.667486+0000) 2024-03-04T09:42:53.087 INFO:journalctl@ceph.osd.6.smithi052.stdout:Mar 04 09:42:52 smithi052 systemd[1]: Stopping Ceph osd.6 for 4e059f08-da08-11ee-95c6-87774f69a715... 2024-03-04T09:42:53.132 INFO:journalctl@ceph.osd.7.smithi106.stdout:Mar 04 09:42:52 smithi106 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-7[54346]: 2024-03-04T09:42:52.849+0000 7f3be1c52700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-04T09:42:17.071420+0000 front 2024-03-04T09:42:17.071488+0000 (oldest deadline 2024-03-04T09:42:40.472167+0000) 2024-03-04T09:42:53.132 INFO:journalctl@ceph.osd.7.smithi106.stdout:Mar 04 09:42:52 smithi106 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-7[54346]: 2024-03-04T09:42:52.849+0000 7f3be1c52700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-03-04T09:42:25.173267+0000 front 2024-03-04T09:42:25.173398+0000 (oldest deadline 2024-03-04T09:42:48.673397+0000) 2024-03-04T09:42:53.586 INFO:journalctl@ceph.osd.9.smithi052.stdout:Mar 04 09:42:53 smithi052 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-9[65396]: 2024-03-04T09:42:53.430+0000 7fd1f1e8e700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-03-04T09:42:09.861821+0000 front 2024-03-04T09:42:09.861761+0000 (oldest deadline 2024-03-04T09:42:32.161859+0000) 2024-03-04T09:42:53.587 INFO:journalctl@ceph.osd.9.smithi052.stdout:Mar 04 09:42:53 smithi052 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-9[65396]: 2024-03-04T09:42:53.430+0000 7fd1f1e8e700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-04T09:42:17.363956+0000 front 2024-03-04T09:42:17.363876+0000 (oldest deadline 2024-03-04T09:42:41.463546+0000) 2024-03-04T09:42:53.587 INFO:journalctl@ceph.osd.9.smithi052.stdout:Mar 04 09:42:53 smithi052 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-9[65396]: 2024-03-04T09:42:53.430+0000 7fd1f1e8e700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-03-04T09:42:21.464200+0000 front 2024-03-04T09:42:21.464139+0000 (oldest deadline 2024-03-04T09:42:47.363942+0000) 2024-03-04T09:42:53.587 INFO:journalctl@ceph.osd.6.smithi052.stdout:Mar 04 09:42:53 smithi052 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-6[58363]: 2024-03-04T09:42:53.222+0000 7fa332262700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-04T09:42:53.587 INFO:journalctl@ceph.osd.6.smithi052.stdout:Mar 04 09:42:53 smithi052 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-6[58363]: 2024-03-04T09:42:53.222+0000 7fa332262700 -1 osd.6 106 *** Got signal Terminated *** 2024-03-04T09:42:53.587 INFO:journalctl@ceph.osd.6.smithi052.stdout:Mar 04 09:42:53 smithi052 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-6[58363]: 2024-03-04T09:42:53.222+0000 7fa332262700 -1 osd.6 106 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-04T09:42:53.588 INFO:journalctl@ceph.osd.8.smithi191.stdout:Mar 04 09:42:53 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-8[55524]: 2024-03-04T09:42:53.334+0000 7f436df85700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-04T09:42:17.524704+0000 front 2024-03-04T09:42:17.524866+0000 (oldest deadline 2024-03-04T09:42:41.024775+0000) 2024-03-04T09:42:53.588 INFO:journalctl@ceph.osd.8.smithi191.stdout:Mar 04 09:42:53 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-8[55524]: 2024-03-04T09:42:53.334+0000 7f436df85700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-03-04T09:42:25.125775+0000 front 2024-03-04T09:42:25.125737+0000 (oldest deadline 2024-03-04T09:42:48.025913+0000) 2024-03-04T09:42:53.588 INFO:journalctl@ceph.osd.11.smithi191.stdout:Mar 04 09:42:53 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-11[62160]: 2024-03-04T09:42:53.287+0000 7fdceac97700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-03-04T09:42:08.553313+0000 front 2024-03-04T09:42:08.553176+0000 (oldest deadline 2024-03-04T09:42:32.653011+0000) 2024-03-04T09:42:53.588 INFO:journalctl@ceph.osd.11.smithi191.stdout:Mar 04 09:42:53 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-11[62160]: 2024-03-04T09:42:53.288+0000 7fdceac97700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-04T09:42:18.355200+0000 front 2024-03-04T09:42:18.355189+0000 (oldest deadline 2024-03-04T09:42:44.254936+0000) 2024-03-04T09:42:53.589 INFO:journalctl@ceph.osd.11.smithi191.stdout:Mar 04 09:42:53 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-11[62160]: 2024-03-04T09:42:53.288+0000 7fdceac97700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-03-04T09:42:24.255361+0000 front 2024-03-04T09:42:24.255246+0000 (oldest deadline 2024-03-04T09:42:47.155465+0000) 2024-03-04T09:42:53.862 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 04 09:42:53 smithi106 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-10[60969]: 2024-03-04T09:42:53.515+0000 7f10f0d39700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-03-04T09:42:08.466029+0000 front 2024-03-04T09:42:08.465963+0000 (oldest deadline 2024-03-04T09:42:32.566054+0000) 2024-03-04T09:42:53.862 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 04 09:42:53 smithi106 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-10[60969]: 2024-03-04T09:42:53.515+0000 7f10f0d39700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-04T09:42:15.467191+0000 front 2024-03-04T09:42:15.467152+0000 (oldest deadline 2024-03-04T09:42:39.567351+0000) 2024-03-04T09:42:53.863 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 04 09:42:53 smithi106 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-10[60969]: 2024-03-04T09:42:53.515+0000 7f10f0d39700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-03-04T09:42:25.368854+0000 front 2024-03-04T09:42:25.368833+0000 (oldest deadline 2024-03-04T09:42:48.868990+0000) 2024-03-04T09:42:54.086 INFO:journalctl@ceph.osd.6.smithi052.stdout:Mar 04 09:42:53 smithi052 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-6[58363]: 2024-03-04T09:42:53.696+0000 7fa32e856700 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-04T09:42:18.565964+0000 front 2024-03-04T09:42:18.566095+0000 (oldest deadline 2024-03-04T09:42:39.666005+0000) 2024-03-04T09:42:54.086 INFO:journalctl@ceph.osd.6.smithi052.stdout:Mar 04 09:42:53 smithi052 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-6[58363]: 2024-03-04T09:42:53.696+0000 7fa32e856700 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-03-04T09:42:23.767434+0000 front 2024-03-04T09:42:23.767558+0000 (oldest deadline 2024-03-04T09:42:49.667486+0000) 2024-03-04T09:42:54.132 INFO:journalctl@ceph.osd.7.smithi106.stdout:Mar 04 09:42:53 smithi106 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-7[54346]: 2024-03-04T09:42:53.864+0000 7f3be1c52700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-04T09:42:17.071420+0000 front 2024-03-04T09:42:17.071488+0000 (oldest deadline 2024-03-04T09:42:40.472167+0000) 2024-03-04T09:42:54.133 INFO:journalctl@ceph.osd.7.smithi106.stdout:Mar 04 09:42:53 smithi106 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-7[54346]: 2024-03-04T09:42:53.864+0000 7f3be1c52700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-03-04T09:42:25.173267+0000 front 2024-03-04T09:42:25.173398+0000 (oldest deadline 2024-03-04T09:42:48.673397+0000) 2024-03-04T09:42:54.588 INFO:journalctl@ceph.osd.8.smithi191.stdout:Mar 04 09:42:54 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-8[55524]: 2024-03-04T09:42:54.334+0000 7f436df85700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-04T09:42:17.524704+0000 front 2024-03-04T09:42:17.524866+0000 (oldest deadline 2024-03-04T09:42:41.024775+0000) 2024-03-04T09:42:54.588 INFO:journalctl@ceph.osd.8.smithi191.stdout:Mar 04 09:42:54 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-8[55524]: 2024-03-04T09:42:54.334+0000 7f436df85700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-03-04T09:42:25.125775+0000 front 2024-03-04T09:42:25.125737+0000 (oldest deadline 2024-03-04T09:42:48.025913+0000) 2024-03-04T09:42:54.588 INFO:journalctl@ceph.osd.11.smithi191.stdout:Mar 04 09:42:54 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-11[62160]: 2024-03-04T09:42:54.326+0000 7fdceac97700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-03-04T09:42:08.553313+0000 front 2024-03-04T09:42:08.553176+0000 (oldest deadline 2024-03-04T09:42:32.653011+0000) 2024-03-04T09:42:54.588 INFO:journalctl@ceph.osd.11.smithi191.stdout:Mar 04 09:42:54 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-11[62160]: 2024-03-04T09:42:54.326+0000 7fdceac97700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-04T09:42:18.355200+0000 front 2024-03-04T09:42:18.355189+0000 (oldest deadline 2024-03-04T09:42:44.254936+0000) 2024-03-04T09:42:54.589 INFO:journalctl@ceph.osd.11.smithi191.stdout:Mar 04 09:42:54 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-11[62160]: 2024-03-04T09:42:54.326+0000 7fdceac97700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-03-04T09:42:24.255361+0000 front 2024-03-04T09:42:24.255246+0000 (oldest deadline 2024-03-04T09:42:47.155465+0000) 2024-03-04T09:42:54.714 INFO:journalctl@ceph.osd.9.smithi052.stdout:Mar 04 09:42:54 smithi052 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-9[65396]: 2024-03-04T09:42:54.446+0000 7fd1f1e8e700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-03-04T09:42:09.861821+0000 front 2024-03-04T09:42:09.861761+0000 (oldest deadline 2024-03-04T09:42:32.161859+0000) 2024-03-04T09:42:54.714 INFO:journalctl@ceph.osd.9.smithi052.stdout:Mar 04 09:42:54 smithi052 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-9[65396]: 2024-03-04T09:42:54.446+0000 7fd1f1e8e700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-04T09:42:17.363956+0000 front 2024-03-04T09:42:17.363876+0000 (oldest deadline 2024-03-04T09:42:41.463546+0000) 2024-03-04T09:42:54.714 INFO:journalctl@ceph.osd.9.smithi052.stdout:Mar 04 09:42:54 smithi052 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-9[65396]: 2024-03-04T09:42:54.446+0000 7fd1f1e8e700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-03-04T09:42:21.464200+0000 front 2024-03-04T09:42:21.464139+0000 (oldest deadline 2024-03-04T09:42:47.363942+0000) 2024-03-04T09:42:54.819 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 04 09:42:54 smithi106 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-10[60969]: 2024-03-04T09:42:54.552+0000 7f10f0d39700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-03-04T09:42:08.466029+0000 front 2024-03-04T09:42:08.465963+0000 (oldest deadline 2024-03-04T09:42:32.566054+0000) 2024-03-04T09:42:54.820 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 04 09:42:54 smithi106 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-10[60969]: 2024-03-04T09:42:54.552+0000 7f10f0d39700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-04T09:42:15.467191+0000 front 2024-03-04T09:42:15.467152+0000 (oldest deadline 2024-03-04T09:42:39.567351+0000) 2024-03-04T09:42:54.820 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 04 09:42:54 smithi106 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-10[60969]: 2024-03-04T09:42:54.552+0000 7f10f0d39700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-03-04T09:42:25.368854+0000 front 2024-03-04T09:42:25.368833+0000 (oldest deadline 2024-03-04T09:42:48.868990+0000) 2024-03-04T09:42:55.086 INFO:journalctl@ceph.osd.6.smithi052.stdout:Mar 04 09:42:54 smithi052 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-6[58363]: 2024-03-04T09:42:54.716+0000 7fa32e856700 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-04T09:42:18.565964+0000 front 2024-03-04T09:42:18.566095+0000 (oldest deadline 2024-03-04T09:42:39.666005+0000) 2024-03-04T09:42:55.086 INFO:journalctl@ceph.osd.6.smithi052.stdout:Mar 04 09:42:54 smithi052 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-6[58363]: 2024-03-04T09:42:54.716+0000 7fa32e856700 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-03-04T09:42:23.767434+0000 front 2024-03-04T09:42:23.767558+0000 (oldest deadline 2024-03-04T09:42:49.667486+0000) 2024-03-04T09:42:55.132 INFO:journalctl@ceph.osd.7.smithi106.stdout:Mar 04 09:42:54 smithi106 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-7[54346]: 2024-03-04T09:42:54.821+0000 7f3be1c52700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-04T09:42:17.071420+0000 front 2024-03-04T09:42:17.071488+0000 (oldest deadline 2024-03-04T09:42:40.472167+0000) 2024-03-04T09:42:55.133 INFO:journalctl@ceph.osd.7.smithi106.stdout:Mar 04 09:42:54 smithi106 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-7[54346]: 2024-03-04T09:42:54.821+0000 7f3be1c52700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-03-04T09:42:25.173267+0000 front 2024-03-04T09:42:25.173398+0000 (oldest deadline 2024-03-04T09:42:48.673397+0000) 2024-03-04T09:42:55.587 INFO:journalctl@ceph.osd.8.smithi191.stdout:Mar 04 09:42:55 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-8[55524]: 2024-03-04T09:42:55.308+0000 7f436df85700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-04T09:42:17.524704+0000 front 2024-03-04T09:42:17.524866+0000 (oldest deadline 2024-03-04T09:42:41.024775+0000) 2024-03-04T09:42:55.588 INFO:journalctl@ceph.osd.8.smithi191.stdout:Mar 04 09:42:55 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-8[55524]: 2024-03-04T09:42:55.308+0000 7f436df85700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-03-04T09:42:25.125775+0000 front 2024-03-04T09:42:25.125737+0000 (oldest deadline 2024-03-04T09:42:48.025913+0000) 2024-03-04T09:42:55.588 INFO:journalctl@ceph.osd.11.smithi191.stdout:Mar 04 09:42:55 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-11[62160]: 2024-03-04T09:42:55.345+0000 7fdceac97700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-03-04T09:42:08.553313+0000 front 2024-03-04T09:42:08.553176+0000 (oldest deadline 2024-03-04T09:42:32.653011+0000) 2024-03-04T09:42:55.588 INFO:journalctl@ceph.osd.11.smithi191.stdout:Mar 04 09:42:55 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-11[62160]: 2024-03-04T09:42:55.345+0000 7fdceac97700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-04T09:42:18.355200+0000 front 2024-03-04T09:42:18.355189+0000 (oldest deadline 2024-03-04T09:42:44.254936+0000) 2024-03-04T09:42:55.588 INFO:journalctl@ceph.osd.11.smithi191.stdout:Mar 04 09:42:55 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-11[62160]: 2024-03-04T09:42:55.345+0000 7fdceac97700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-03-04T09:42:24.255361+0000 front 2024-03-04T09:42:24.255246+0000 (oldest deadline 2024-03-04T09:42:47.155465+0000) 2024-03-04T09:42:55.712 INFO:journalctl@ceph.osd.9.smithi052.stdout:Mar 04 09:42:55 smithi052 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-9[65396]: 2024-03-04T09:42:55.454+0000 7fd1f1e8e700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-03-04T09:42:09.861821+0000 front 2024-03-04T09:42:09.861761+0000 (oldest deadline 2024-03-04T09:42:32.161859+0000) 2024-03-04T09:42:55.712 INFO:journalctl@ceph.osd.9.smithi052.stdout:Mar 04 09:42:55 smithi052 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-9[65396]: 2024-03-04T09:42:55.454+0000 7fd1f1e8e700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-04T09:42:17.363956+0000 front 2024-03-04T09:42:17.363876+0000 (oldest deadline 2024-03-04T09:42:41.463546+0000) 2024-03-04T09:42:55.712 INFO:journalctl@ceph.osd.9.smithi052.stdout:Mar 04 09:42:55 smithi052 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-9[65396]: 2024-03-04T09:42:55.454+0000 7fd1f1e8e700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-03-04T09:42:21.464200+0000 front 2024-03-04T09:42:21.464139+0000 (oldest deadline 2024-03-04T09:42:47.363942+0000) 2024-03-04T09:42:55.791 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 04 09:42:55 smithi106 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-10[60969]: 2024-03-04T09:42:55.512+0000 7f10f0d39700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-03-04T09:42:08.466029+0000 front 2024-03-04T09:42:08.465963+0000 (oldest deadline 2024-03-04T09:42:32.566054+0000) 2024-03-04T09:42:55.792 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 04 09:42:55 smithi106 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-10[60969]: 2024-03-04T09:42:55.512+0000 7f10f0d39700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-04T09:42:15.467191+0000 front 2024-03-04T09:42:15.467152+0000 (oldest deadline 2024-03-04T09:42:39.567351+0000) 2024-03-04T09:42:55.792 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 04 09:42:55 smithi106 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-10[60969]: 2024-03-04T09:42:55.512+0000 7f10f0d39700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-03-04T09:42:25.368854+0000 front 2024-03-04T09:42:25.368833+0000 (oldest deadline 2024-03-04T09:42:48.868990+0000) 2024-03-04T09:42:55.792 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 04 09:42:55 smithi106 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-10[60969]: 2024-03-04T09:42:55.512+0000 7f10f0d39700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2024-03-04T09:42:28.869679+0000 front 2024-03-04T09:42:28.869692+0000 (oldest deadline 2024-03-04T09:42:55.270190+0000) 2024-03-04T09:42:56.086 INFO:journalctl@ceph.osd.6.smithi052.stdout:Mar 04 09:42:55 smithi052 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-6[58363]: 2024-03-04T09:42:55.714+0000 7fa32e856700 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-04T09:42:18.565964+0000 front 2024-03-04T09:42:18.566095+0000 (oldest deadline 2024-03-04T09:42:39.666005+0000) 2024-03-04T09:42:56.086 INFO:journalctl@ceph.osd.6.smithi052.stdout:Mar 04 09:42:55 smithi052 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-6[58363]: 2024-03-04T09:42:55.714+0000 7fa32e856700 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-03-04T09:42:23.767434+0000 front 2024-03-04T09:42:23.767558+0000 (oldest deadline 2024-03-04T09:42:49.667486+0000) 2024-03-04T09:42:56.086 INFO:journalctl@ceph.osd.6.smithi052.stdout:Mar 04 09:42:55 smithi052 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-6[58363]: 2024-03-04T09:42:55.714+0000 7fa32e856700 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2024-03-04T09:42:30.768689+0000 front 2024-03-04T09:42:30.768712+0000 (oldest deadline 2024-03-04T09:42:54.868822+0000) 2024-03-04T09:42:56.132 INFO:journalctl@ceph.osd.7.smithi106.stdout:Mar 04 09:42:55 smithi106 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-7[54346]: 2024-03-04T09:42:55.793+0000 7f3be1c52700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-04T09:42:17.071420+0000 front 2024-03-04T09:42:17.071488+0000 (oldest deadline 2024-03-04T09:42:40.472167+0000) 2024-03-04T09:42:56.133 INFO:journalctl@ceph.osd.7.smithi106.stdout:Mar 04 09:42:55 smithi106 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-7[54346]: 2024-03-04T09:42:55.793+0000 7f3be1c52700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-03-04T09:42:25.173267+0000 front 2024-03-04T09:42:25.173398+0000 (oldest deadline 2024-03-04T09:42:48.673397+0000) 2024-03-04T09:42:56.589 INFO:journalctl@ceph.osd.8.smithi191.stdout:Mar 04 09:42:56 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-8[55524]: 2024-03-04T09:42:56.275+0000 7f436df85700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-04T09:42:17.524704+0000 front 2024-03-04T09:42:17.524866+0000 (oldest deadline 2024-03-04T09:42:41.024775+0000) 2024-03-04T09:42:56.590 INFO:journalctl@ceph.osd.8.smithi191.stdout:Mar 04 09:42:56 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-8[55524]: 2024-03-04T09:42:56.275+0000 7f436df85700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-03-04T09:42:25.125775+0000 front 2024-03-04T09:42:25.125737+0000 (oldest deadline 2024-03-04T09:42:48.025913+0000) 2024-03-04T09:42:56.590 INFO:journalctl@ceph.osd.8.smithi191.stdout:Mar 04 09:42:56 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-8[55524]: 2024-03-04T09:42:56.275+0000 7f436df85700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2024-03-04T09:42:34.328161+0000 front 2024-03-04T09:42:34.328294+0000 (oldest deadline 2024-03-04T09:42:55.428059+0000) 2024-03-04T09:42:56.591 INFO:journalctl@ceph.osd.11.smithi191.stdout:Mar 04 09:42:56 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-11[62160]: 2024-03-04T09:42:56.371+0000 7fdceac97700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-03-04T09:42:08.553313+0000 front 2024-03-04T09:42:08.553176+0000 (oldest deadline 2024-03-04T09:42:32.653011+0000) 2024-03-04T09:42:56.591 INFO:journalctl@ceph.osd.11.smithi191.stdout:Mar 04 09:42:56 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-11[62160]: 2024-03-04T09:42:56.371+0000 7fdceac97700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-04T09:42:18.355200+0000 front 2024-03-04T09:42:18.355189+0000 (oldest deadline 2024-03-04T09:42:44.254936+0000) 2024-03-04T09:42:56.591 INFO:journalctl@ceph.osd.11.smithi191.stdout:Mar 04 09:42:56 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-11[62160]: 2024-03-04T09:42:56.371+0000 7fdceac97700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-03-04T09:42:24.255361+0000 front 2024-03-04T09:42:24.255246+0000 (oldest deadline 2024-03-04T09:42:47.155465+0000) 2024-03-04T09:42:56.743 INFO:journalctl@ceph.osd.9.smithi052.stdout:Mar 04 09:42:56 smithi052 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-9[65396]: 2024-03-04T09:42:56.482+0000 7fd1f1e8e700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-03-04T09:42:09.861821+0000 front 2024-03-04T09:42:09.861761+0000 (oldest deadline 2024-03-04T09:42:32.161859+0000) 2024-03-04T09:42:56.744 INFO:journalctl@ceph.osd.9.smithi052.stdout:Mar 04 09:42:56 smithi052 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-9[65396]: 2024-03-04T09:42:56.482+0000 7fd1f1e8e700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-04T09:42:17.363956+0000 front 2024-03-04T09:42:17.363876+0000 (oldest deadline 2024-03-04T09:42:41.463546+0000) 2024-03-04T09:42:56.744 INFO:journalctl@ceph.osd.9.smithi052.stdout:Mar 04 09:42:56 smithi052 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-9[65396]: 2024-03-04T09:42:56.482+0000 7fd1f1e8e700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-03-04T09:42:21.464200+0000 front 2024-03-04T09:42:21.464139+0000 (oldest deadline 2024-03-04T09:42:47.363942+0000) 2024-03-04T09:42:56.883 INFO:journalctl@ceph.osd.7.smithi106.stdout:Mar 04 09:42:56 smithi106 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-7[54346]: 2024-03-04T09:42:56.752+0000 7f3be1c52700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-04T09:42:17.071420+0000 front 2024-03-04T09:42:17.071488+0000 (oldest deadline 2024-03-04T09:42:40.472167+0000) 2024-03-04T09:42:56.883 INFO:journalctl@ceph.osd.7.smithi106.stdout:Mar 04 09:42:56 smithi106 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-7[54346]: 2024-03-04T09:42:56.752+0000 7f3be1c52700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-03-04T09:42:25.173267+0000 front 2024-03-04T09:42:25.173398+0000 (oldest deadline 2024-03-04T09:42:48.673397+0000) 2024-03-04T09:42:56.883 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 04 09:42:56 smithi106 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-10[60969]: 2024-03-04T09:42:56.557+0000 7f10f0d39700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-03-04T09:42:08.466029+0000 front 2024-03-04T09:42:08.465963+0000 (oldest deadline 2024-03-04T09:42:32.566054+0000) 2024-03-04T09:42:56.883 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 04 09:42:56 smithi106 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-10[60969]: 2024-03-04T09:42:56.557+0000 7f10f0d39700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-04T09:42:15.467191+0000 front 2024-03-04T09:42:15.467152+0000 (oldest deadline 2024-03-04T09:42:39.567351+0000) 2024-03-04T09:42:56.883 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 04 09:42:56 smithi106 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-10[60969]: 2024-03-04T09:42:56.557+0000 7f10f0d39700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-03-04T09:42:25.368854+0000 front 2024-03-04T09:42:25.368833+0000 (oldest deadline 2024-03-04T09:42:48.868990+0000) 2024-03-04T09:42:56.884 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 04 09:42:56 smithi106 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-10[60969]: 2024-03-04T09:42:56.557+0000 7f10f0d39700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2024-03-04T09:42:28.869679+0000 front 2024-03-04T09:42:28.869692+0000 (oldest deadline 2024-03-04T09:42:55.270190+0000) 2024-03-04T09:42:57.086 INFO:journalctl@ceph.osd.6.smithi052.stdout:Mar 04 09:42:56 smithi052 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-6[58363]: 2024-03-04T09:42:56.746+0000 7fa32e856700 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-04T09:42:18.565964+0000 front 2024-03-04T09:42:18.566095+0000 (oldest deadline 2024-03-04T09:42:39.666005+0000) 2024-03-04T09:42:57.086 INFO:journalctl@ceph.osd.6.smithi052.stdout:Mar 04 09:42:56 smithi052 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-6[58363]: 2024-03-04T09:42:56.746+0000 7fa32e856700 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-03-04T09:42:23.767434+0000 front 2024-03-04T09:42:23.767558+0000 (oldest deadline 2024-03-04T09:42:49.667486+0000) 2024-03-04T09:42:57.086 INFO:journalctl@ceph.osd.6.smithi052.stdout:Mar 04 09:42:56 smithi052 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-6[58363]: 2024-03-04T09:42:56.746+0000 7fa32e856700 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2024-03-04T09:42:30.768689+0000 front 2024-03-04T09:42:30.768712+0000 (oldest deadline 2024-03-04T09:42:54.868822+0000) 2024-03-04T09:42:57.588 INFO:journalctl@ceph.osd.8.smithi191.stdout:Mar 04 09:42:57 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-8[55524]: 2024-03-04T09:42:57.249+0000 7f436df85700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-04T09:42:17.524704+0000 front 2024-03-04T09:42:17.524866+0000 (oldest deadline 2024-03-04T09:42:41.024775+0000) 2024-03-04T09:42:57.588 INFO:journalctl@ceph.osd.8.smithi191.stdout:Mar 04 09:42:57 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-8[55524]: 2024-03-04T09:42:57.249+0000 7f436df85700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-03-04T09:42:25.125775+0000 front 2024-03-04T09:42:25.125737+0000 (oldest deadline 2024-03-04T09:42:48.025913+0000) 2024-03-04T09:42:57.588 INFO:journalctl@ceph.osd.8.smithi191.stdout:Mar 04 09:42:57 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-8[55524]: 2024-03-04T09:42:57.249+0000 7f436df85700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2024-03-04T09:42:34.328161+0000 front 2024-03-04T09:42:34.328294+0000 (oldest deadline 2024-03-04T09:42:55.428059+0000) 2024-03-04T09:42:57.589 INFO:journalctl@ceph.osd.11.smithi191.stdout:Mar 04 09:42:57 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-11[62160]: 2024-03-04T09:42:57.341+0000 7fdceac97700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-03-04T09:42:08.553313+0000 front 2024-03-04T09:42:08.553176+0000 (oldest deadline 2024-03-04T09:42:32.653011+0000) 2024-03-04T09:42:57.589 INFO:journalctl@ceph.osd.11.smithi191.stdout:Mar 04 09:42:57 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-11[62160]: 2024-03-04T09:42:57.341+0000 7fdceac97700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-04T09:42:18.355200+0000 front 2024-03-04T09:42:18.355189+0000 (oldest deadline 2024-03-04T09:42:44.254936+0000) 2024-03-04T09:42:57.589 INFO:journalctl@ceph.osd.11.smithi191.stdout:Mar 04 09:42:57 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-11[62160]: 2024-03-04T09:42:57.341+0000 7fdceac97700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-03-04T09:42:24.255361+0000 front 2024-03-04T09:42:24.255246+0000 (oldest deadline 2024-03-04T09:42:47.155465+0000) 2024-03-04T09:42:57.836 INFO:journalctl@ceph.osd.6.smithi052.stdout:Mar 04 09:42:57 smithi052 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-6[58363]: 2024-03-04T09:42:57.705+0000 7fa32e856700 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-04T09:42:18.565964+0000 front 2024-03-04T09:42:18.566095+0000 (oldest deadline 2024-03-04T09:42:39.666005+0000) 2024-03-04T09:42:57.836 INFO:journalctl@ceph.osd.6.smithi052.stdout:Mar 04 09:42:57 smithi052 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-6[58363]: 2024-03-04T09:42:57.705+0000 7fa32e856700 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-03-04T09:42:23.767434+0000 front 2024-03-04T09:42:23.767558+0000 (oldest deadline 2024-03-04T09:42:49.667486+0000) 2024-03-04T09:42:57.837 INFO:journalctl@ceph.osd.6.smithi052.stdout:Mar 04 09:42:57 smithi052 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-6[58363]: 2024-03-04T09:42:57.705+0000 7fa32e856700 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2024-03-04T09:42:30.768689+0000 front 2024-03-04T09:42:30.768712+0000 (oldest deadline 2024-03-04T09:42:54.868822+0000) 2024-03-04T09:42:57.837 INFO:journalctl@ceph.osd.9.smithi052.stdout:Mar 04 09:42:57 smithi052 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-9[65396]: 2024-03-04T09:42:57.527+0000 7fd1f1e8e700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-03-04T09:42:09.861821+0000 front 2024-03-04T09:42:09.861761+0000 (oldest deadline 2024-03-04T09:42:32.161859+0000) 2024-03-04T09:42:57.837 INFO:journalctl@ceph.osd.9.smithi052.stdout:Mar 04 09:42:57 smithi052 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-9[65396]: 2024-03-04T09:42:57.527+0000 7fd1f1e8e700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-04T09:42:17.363956+0000 front 2024-03-04T09:42:17.363876+0000 (oldest deadline 2024-03-04T09:42:41.463546+0000) 2024-03-04T09:42:57.837 INFO:journalctl@ceph.osd.9.smithi052.stdout:Mar 04 09:42:57 smithi052 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-9[65396]: 2024-03-04T09:42:57.527+0000 7fd1f1e8e700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-03-04T09:42:21.464200+0000 front 2024-03-04T09:42:21.464139+0000 (oldest deadline 2024-03-04T09:42:47.363942+0000) 2024-03-04T09:42:57.882 INFO:journalctl@ceph.osd.7.smithi106.stdout:Mar 04 09:42:57 smithi106 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-7[54346]: 2024-03-04T09:42:57.740+0000 7f3be1c52700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-04T09:42:17.071420+0000 front 2024-03-04T09:42:17.071488+0000 (oldest deadline 2024-03-04T09:42:40.472167+0000) 2024-03-04T09:42:57.883 INFO:journalctl@ceph.osd.7.smithi106.stdout:Mar 04 09:42:57 smithi106 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-7[54346]: 2024-03-04T09:42:57.740+0000 7f3be1c52700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-03-04T09:42:25.173267+0000 front 2024-03-04T09:42:25.173398+0000 (oldest deadline 2024-03-04T09:42:48.673397+0000) 2024-03-04T09:42:57.883 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 04 09:42:57 smithi106 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-10[60969]: 2024-03-04T09:42:57.590+0000 7f10f0d39700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-03-04T09:42:08.466029+0000 front 2024-03-04T09:42:08.465963+0000 (oldest deadline 2024-03-04T09:42:32.566054+0000) 2024-03-04T09:42:57.883 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 04 09:42:57 smithi106 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-10[60969]: 2024-03-04T09:42:57.590+0000 7f10f0d39700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-04T09:42:15.467191+0000 front 2024-03-04T09:42:15.467152+0000 (oldest deadline 2024-03-04T09:42:39.567351+0000) 2024-03-04T09:42:57.884 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 04 09:42:57 smithi106 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-10[60969]: 2024-03-04T09:42:57.590+0000 7f10f0d39700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-03-04T09:42:25.368854+0000 front 2024-03-04T09:42:25.368833+0000 (oldest deadline 2024-03-04T09:42:48.868990+0000) 2024-03-04T09:42:57.884 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 04 09:42:57 smithi106 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-10[60969]: 2024-03-04T09:42:57.591+0000 7f10f0d39700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2024-03-04T09:42:28.869679+0000 front 2024-03-04T09:42:28.869692+0000 (oldest deadline 2024-03-04T09:42:55.270190+0000) 2024-03-04T09:42:58.588 INFO:journalctl@ceph.osd.8.smithi191.stdout:Mar 04 09:42:58 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-8[55524]: 2024-03-04T09:42:58.218+0000 7f436df85700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-04T09:42:17.524704+0000 front 2024-03-04T09:42:17.524866+0000 (oldest deadline 2024-03-04T09:42:41.024775+0000) 2024-03-04T09:42:58.591 INFO:journalctl@ceph.osd.8.smithi191.stdout:Mar 04 09:42:58 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-8[55524]: 2024-03-04T09:42:58.218+0000 7f436df85700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-03-04T09:42:25.125775+0000 front 2024-03-04T09:42:25.125737+0000 (oldest deadline 2024-03-04T09:42:48.025913+0000) 2024-03-04T09:42:58.591 INFO:journalctl@ceph.osd.8.smithi191.stdout:Mar 04 09:42:58 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-8[55524]: 2024-03-04T09:42:58.218+0000 7f436df85700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2024-03-04T09:42:34.328161+0000 front 2024-03-04T09:42:34.328294+0000 (oldest deadline 2024-03-04T09:42:55.428059+0000) 2024-03-04T09:42:58.591 INFO:journalctl@ceph.osd.11.smithi191.stdout:Mar 04 09:42:58 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-11[62160]: 2024-03-04T09:42:58.369+0000 7fdceac97700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-03-04T09:42:08.553313+0000 front 2024-03-04T09:42:08.553176+0000 (oldest deadline 2024-03-04T09:42:32.653011+0000) 2024-03-04T09:42:58.591 INFO:journalctl@ceph.osd.11.smithi191.stdout:Mar 04 09:42:58 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-11[62160]: 2024-03-04T09:42:58.369+0000 7fdceac97700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-04T09:42:18.355200+0000 front 2024-03-04T09:42:18.355189+0000 (oldest deadline 2024-03-04T09:42:44.254936+0000) 2024-03-04T09:42:58.592 INFO:journalctl@ceph.osd.11.smithi191.stdout:Mar 04 09:42:58 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-11[62160]: 2024-03-04T09:42:58.369+0000 7fdceac97700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-03-04T09:42:24.255361+0000 front 2024-03-04T09:42:24.255246+0000 (oldest deadline 2024-03-04T09:42:47.155465+0000) 2024-03-04T09:42:58.836 INFO:journalctl@ceph.osd.6.smithi052.stdout:Mar 04 09:42:58 smithi052 podman[96419]: 2024-03-04 09:42:58.713113301 +0000 UTC m=+5.597987794 container died 95ef300435cc19f7993d454c979822d5603ce256f962f8990ca07591dc896149 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5, name=ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-6, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-qdb-1bc250c, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.build-date=20240222, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True) 2024-03-04T09:42:58.836 INFO:journalctl@ceph.osd.9.smithi052.stdout:Mar 04 09:42:58 smithi052 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-9[65396]: 2024-03-04T09:42:58.512+0000 7fd1f1e8e700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-03-04T09:42:09.861821+0000 front 2024-03-04T09:42:09.861761+0000 (oldest deadline 2024-03-04T09:42:32.161859+0000) 2024-03-04T09:42:58.837 INFO:journalctl@ceph.osd.9.smithi052.stdout:Mar 04 09:42:58 smithi052 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-9[65396]: 2024-03-04T09:42:58.512+0000 7fd1f1e8e700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-04T09:42:17.363956+0000 front 2024-03-04T09:42:17.363876+0000 (oldest deadline 2024-03-04T09:42:41.463546+0000) 2024-03-04T09:42:58.837 INFO:journalctl@ceph.osd.9.smithi052.stdout:Mar 04 09:42:58 smithi052 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-9[65396]: 2024-03-04T09:42:58.512+0000 7fd1f1e8e700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-03-04T09:42:21.464200+0000 front 2024-03-04T09:42:21.464139+0000 (oldest deadline 2024-03-04T09:42:47.363942+0000) 2024-03-04T09:42:58.882 INFO:journalctl@ceph.osd.7.smithi106.stdout:Mar 04 09:42:58 smithi106 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-7[54346]: 2024-03-04T09:42:58.705+0000 7f3be1c52700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-04T09:42:17.071420+0000 front 2024-03-04T09:42:17.071488+0000 (oldest deadline 2024-03-04T09:42:40.472167+0000) 2024-03-04T09:42:58.883 INFO:journalctl@ceph.osd.7.smithi106.stdout:Mar 04 09:42:58 smithi106 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-7[54346]: 2024-03-04T09:42:58.705+0000 7f3be1c52700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-03-04T09:42:25.173267+0000 front 2024-03-04T09:42:25.173398+0000 (oldest deadline 2024-03-04T09:42:48.673397+0000) 2024-03-04T09:42:58.883 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 04 09:42:58 smithi106 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-10[60969]: 2024-03-04T09:42:58.627+0000 7f10f0d39700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-03-04T09:42:08.466029+0000 front 2024-03-04T09:42:08.465963+0000 (oldest deadline 2024-03-04T09:42:32.566054+0000) 2024-03-04T09:42:58.883 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 04 09:42:58 smithi106 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-10[60969]: 2024-03-04T09:42:58.627+0000 7f10f0d39700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-04T09:42:15.467191+0000 front 2024-03-04T09:42:15.467152+0000 (oldest deadline 2024-03-04T09:42:39.567351+0000) 2024-03-04T09:42:58.883 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 04 09:42:58 smithi106 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-10[60969]: 2024-03-04T09:42:58.627+0000 7f10f0d39700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-03-04T09:42:25.368854+0000 front 2024-03-04T09:42:25.368833+0000 (oldest deadline 2024-03-04T09:42:48.868990+0000) 2024-03-04T09:42:58.883 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 04 09:42:58 smithi106 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-10[60969]: 2024-03-04T09:42:58.627+0000 7f10f0d39700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2024-03-04T09:42:28.869679+0000 front 2024-03-04T09:42:28.869692+0000 (oldest deadline 2024-03-04T09:42:55.270190+0000) 2024-03-04T09:42:59.336 INFO:journalctl@ceph.osd.6.smithi052.stdout:Mar 04 09:42:59 smithi052 podman[96419]: 2024-03-04 09:42:59.064482004 +0000 UTC m=+5.949356494 container cleanup 95ef300435cc19f7993d454c979822d5603ce256f962f8990ca07591dc896149 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5, name=ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-6, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240222, GIT_CLEAN=True, RELEASE=wip-lusov-qdb-1bc250c) 2024-03-04T09:42:59.336 INFO:journalctl@ceph.osd.6.smithi052.stdout:Mar 04 09:42:59 smithi052 bash[96419]: ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-6 2024-03-04T09:42:59.588 INFO:journalctl@ceph.osd.8.smithi191.stdout:Mar 04 09:42:59 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-8[55524]: 2024-03-04T09:42:59.251+0000 7f436df85700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-04T09:42:17.524704+0000 front 2024-03-04T09:42:17.524866+0000 (oldest deadline 2024-03-04T09:42:41.024775+0000) 2024-03-04T09:42:59.588 INFO:journalctl@ceph.osd.8.smithi191.stdout:Mar 04 09:42:59 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-8[55524]: 2024-03-04T09:42:59.251+0000 7f436df85700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-03-04T09:42:25.125775+0000 front 2024-03-04T09:42:25.125737+0000 (oldest deadline 2024-03-04T09:42:48.025913+0000) 2024-03-04T09:42:59.588 INFO:journalctl@ceph.osd.8.smithi191.stdout:Mar 04 09:42:59 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-8[55524]: 2024-03-04T09:42:59.251+0000 7f436df85700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2024-03-04T09:42:34.328161+0000 front 2024-03-04T09:42:34.328294+0000 (oldest deadline 2024-03-04T09:42:55.428059+0000) 2024-03-04T09:42:59.588 INFO:journalctl@ceph.osd.11.smithi191.stdout:Mar 04 09:42:59 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-11[62160]: 2024-03-04T09:42:59.418+0000 7fdceac97700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-03-04T09:42:08.553313+0000 front 2024-03-04T09:42:08.553176+0000 (oldest deadline 2024-03-04T09:42:32.653011+0000) 2024-03-04T09:42:59.589 INFO:journalctl@ceph.osd.11.smithi191.stdout:Mar 04 09:42:59 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-11[62160]: 2024-03-04T09:42:59.419+0000 7fdceac97700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-04T09:42:18.355200+0000 front 2024-03-04T09:42:18.355189+0000 (oldest deadline 2024-03-04T09:42:44.254936+0000) 2024-03-04T09:42:59.589 INFO:journalctl@ceph.osd.11.smithi191.stdout:Mar 04 09:42:59 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-11[62160]: 2024-03-04T09:42:59.419+0000 7fdceac97700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-03-04T09:42:24.255361+0000 front 2024-03-04T09:42:24.255246+0000 (oldest deadline 2024-03-04T09:42:47.155465+0000) 2024-03-04T09:42:59.680 INFO:journalctl@ceph.osd.6.smithi052.stdout:Mar 04 09:42:59 smithi052 podman[96436]: 2024-03-04 09:42:59.349003276 +0000 UTC m=+0.635154457 container remove 95ef300435cc19f7993d454c979822d5603ce256f962f8990ca07591dc896149 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5, name=ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-6, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240222, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=wip-lusov-qdb-1bc250c, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, GIT_BRANCH=HEAD) 2024-03-04T09:42:59.681 INFO:journalctl@ceph.osd.9.smithi052.stdout:Mar 04 09:42:59 smithi052 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-9[65396]: 2024-03-04T09:42:59.553+0000 7fd1f1e8e700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-03-04T09:42:09.861821+0000 front 2024-03-04T09:42:09.861761+0000 (oldest deadline 2024-03-04T09:42:32.161859+0000) 2024-03-04T09:42:59.681 INFO:journalctl@ceph.osd.9.smithi052.stdout:Mar 04 09:42:59 smithi052 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-9[65396]: 2024-03-04T09:42:59.553+0000 7fd1f1e8e700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-04T09:42:17.363956+0000 front 2024-03-04T09:42:17.363876+0000 (oldest deadline 2024-03-04T09:42:41.463546+0000) 2024-03-04T09:42:59.681 INFO:journalctl@ceph.osd.9.smithi052.stdout:Mar 04 09:42:59 smithi052 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-9[65396]: 2024-03-04T09:42:59.553+0000 7fd1f1e8e700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-03-04T09:42:21.464200+0000 front 2024-03-04T09:42:21.464139+0000 (oldest deadline 2024-03-04T09:42:47.363942+0000) 2024-03-04T09:42:59.883 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 04 09:42:59 smithi106 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-10[60969]: 2024-03-04T09:42:59.610+0000 7f10f0d39700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-03-04T09:42:08.466029+0000 front 2024-03-04T09:42:08.465963+0000 (oldest deadline 2024-03-04T09:42:32.566054+0000) 2024-03-04T09:42:59.883 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 04 09:42:59 smithi106 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-10[60969]: 2024-03-04T09:42:59.610+0000 7f10f0d39700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-04T09:42:15.467191+0000 front 2024-03-04T09:42:15.467152+0000 (oldest deadline 2024-03-04T09:42:39.567351+0000) 2024-03-04T09:42:59.883 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 04 09:42:59 smithi106 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-10[60969]: 2024-03-04T09:42:59.610+0000 7f10f0d39700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-03-04T09:42:25.368854+0000 front 2024-03-04T09:42:25.368833+0000 (oldest deadline 2024-03-04T09:42:48.868990+0000) 2024-03-04T09:42:59.883 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 04 09:42:59 smithi106 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-10[60969]: 2024-03-04T09:42:59.610+0000 7f10f0d39700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2024-03-04T09:42:28.869679+0000 front 2024-03-04T09:42:28.869692+0000 (oldest deadline 2024-03-04T09:42:55.270190+0000) 2024-03-04T09:42:59.883 INFO:journalctl@ceph.osd.7.smithi106.stdout:Mar 04 09:42:59 smithi106 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-7[54346]: 2024-03-04T09:42:59.660+0000 7f3be1c52700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-04T09:42:17.071420+0000 front 2024-03-04T09:42:17.071488+0000 (oldest deadline 2024-03-04T09:42:40.472167+0000) 2024-03-04T09:42:59.884 INFO:journalctl@ceph.osd.7.smithi106.stdout:Mar 04 09:42:59 smithi106 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-7[54346]: 2024-03-04T09:42:59.660+0000 7f3be1c52700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-03-04T09:42:25.173267+0000 front 2024-03-04T09:42:25.173398+0000 (oldest deadline 2024-03-04T09:42:48.673397+0000) 2024-03-04T09:42:59.884 INFO:journalctl@ceph.osd.7.smithi106.stdout:Mar 04 09:42:59 smithi106 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-7[54346]: 2024-03-04T09:42:59.660+0000 7f3be1c52700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2024-03-04T09:42:33.974526+0000 front 2024-03-04T09:42:33.974586+0000 (oldest deadline 2024-03-04T09:42:59.175048+0000) 2024-03-04T09:42:59.991 INFO:journalctl@ceph.osd.6.smithi052.stdout:Mar 04 09:42:59 smithi052 podman[96527]: 2024-03-04 09:42:59.585408018 +0000 UTC m=+0.017499415 image pull f88ffb32aa6a7d5dfdc954d9f096f0c9cc47981d05ec8d117ba859a22d8185ba quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5 2024-03-04T09:42:59.992 INFO:journalctl@ceph.osd.6.smithi052.stdout:Mar 04 09:42:59 smithi052 podman[96527]: 2024-03-04 09:42:59.717892558 +0000 UTC m=+0.149983993 container create 5c144b4061f8291b716e88b8cff44702805cdd1c39629f9a02f269cd602c4f32 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5, name=ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-6-deactivate, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, org.label-schema.license=GPLv2, RELEASE=wip-lusov-qdb-1bc250c, GIT_CLEAN=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240222, ceph=True) 2024-03-04T09:43:00.292 INFO:journalctl@ceph.osd.6.smithi052.stdout:Mar 04 09:42:59 smithi052 podman[96527]: 2024-03-04 09:42:59.99431989 +0000 UTC m=+0.426411286 container init 5c144b4061f8291b716e88b8cff44702805cdd1c39629f9a02f269cd602c4f32 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5, name=ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-6-deactivate, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240222, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, RELEASE=wip-lusov-qdb-1bc250c, org.label-schema.vendor=CentOS) 2024-03-04T09:43:00.293 INFO:journalctl@ceph.osd.6.smithi052.stdout:Mar 04 09:42:59 smithi052 podman[96527]: 2024-03-04 09:42:59.997679939 +0000 UTC m=+0.429771329 container start 5c144b4061f8291b716e88b8cff44702805cdd1c39629f9a02f269cd602c4f32 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5, name=ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-6-deactivate, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240222, RELEASE=wip-lusov-qdb-1bc250c, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-03-04T09:43:00.293 INFO:journalctl@ceph.osd.6.smithi052.stdout:Mar 04 09:43:00 smithi052 podman[96527]: 2024-03-04 09:43:00.061199266 +0000 UTC m=+0.493290655 container attach 5c144b4061f8291b716e88b8cff44702805cdd1c39629f9a02f269cd602c4f32 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5, name=ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-6-deactivate, GIT_CLEAN=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-qdb-1bc250c, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, org.label-schema.build-date=20240222, ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5) 2024-03-04T09:43:00.555 INFO:journalctl@ceph.osd.6.smithi052.stdout:Mar 04 09:43:00 smithi052 podman[96527]: 2024-03-04 09:43:00.295500798 +0000 UTC m=+0.727592196 container died 5c144b4061f8291b716e88b8cff44702805cdd1c39629f9a02f269cd602c4f32 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5, name=ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-6-deactivate, io.buildah.version=1.33.5, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240222, RELEASE=wip-lusov-qdb-1bc250c, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-03-04T09:43:00.587 INFO:journalctl@ceph.osd.8.smithi191.stdout:Mar 04 09:43:00 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-8[55524]: 2024-03-04T09:43:00.248+0000 7f436df85700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-04T09:42:17.524704+0000 front 2024-03-04T09:42:17.524866+0000 (oldest deadline 2024-03-04T09:42:41.024775+0000) 2024-03-04T09:43:00.588 INFO:journalctl@ceph.osd.8.smithi191.stdout:Mar 04 09:43:00 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-8[55524]: 2024-03-04T09:43:00.248+0000 7f436df85700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-03-04T09:42:25.125775+0000 front 2024-03-04T09:42:25.125737+0000 (oldest deadline 2024-03-04T09:42:48.025913+0000) 2024-03-04T09:43:00.588 INFO:journalctl@ceph.osd.8.smithi191.stdout:Mar 04 09:43:00 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-8[55524]: 2024-03-04T09:43:00.248+0000 7f436df85700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2024-03-04T09:42:34.328161+0000 front 2024-03-04T09:42:34.328294+0000 (oldest deadline 2024-03-04T09:42:55.428059+0000) 2024-03-04T09:43:00.588 INFO:journalctl@ceph.osd.11.smithi191.stdout:Mar 04 09:43:00 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-11[62160]: 2024-03-04T09:43:00.424+0000 7fdceac97700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-03-04T09:42:08.553313+0000 front 2024-03-04T09:42:08.553176+0000 (oldest deadline 2024-03-04T09:42:32.653011+0000) 2024-03-04T09:43:00.588 INFO:journalctl@ceph.osd.11.smithi191.stdout:Mar 04 09:43:00 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-11[62160]: 2024-03-04T09:43:00.424+0000 7fdceac97700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-04T09:42:18.355200+0000 front 2024-03-04T09:42:18.355189+0000 (oldest deadline 2024-03-04T09:42:44.254936+0000) 2024-03-04T09:43:00.588 INFO:journalctl@ceph.osd.11.smithi191.stdout:Mar 04 09:43:00 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-11[62160]: 2024-03-04T09:43:00.424+0000 7fdceac97700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-03-04T09:42:24.255361+0000 front 2024-03-04T09:42:24.255246+0000 (oldest deadline 2024-03-04T09:42:47.155465+0000) 2024-03-04T09:43:00.836 INFO:journalctl@ceph.osd.9.smithi052.stdout:Mar 04 09:43:00 smithi052 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-9[65396]: 2024-03-04T09:43:00.557+0000 7fd1f1e8e700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-03-04T09:42:09.861821+0000 front 2024-03-04T09:42:09.861761+0000 (oldest deadline 2024-03-04T09:42:32.161859+0000) 2024-03-04T09:43:00.836 INFO:journalctl@ceph.osd.9.smithi052.stdout:Mar 04 09:43:00 smithi052 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-9[65396]: 2024-03-04T09:43:00.557+0000 7fd1f1e8e700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-04T09:42:17.363956+0000 front 2024-03-04T09:42:17.363876+0000 (oldest deadline 2024-03-04T09:42:41.463546+0000) 2024-03-04T09:43:00.837 INFO:journalctl@ceph.osd.9.smithi052.stdout:Mar 04 09:43:00 smithi052 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-9[65396]: 2024-03-04T09:43:00.557+0000 7fd1f1e8e700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-03-04T09:42:21.464200+0000 front 2024-03-04T09:42:21.464139+0000 (oldest deadline 2024-03-04T09:42:47.363942+0000) 2024-03-04T09:43:00.837 INFO:journalctl@ceph.osd.9.smithi052.stdout:Mar 04 09:43:00 smithi052 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-9[65396]: 2024-03-04T09:43:00.557+0000 7fd1f1e8e700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2024-03-04T09:42:33.765329+0000 front 2024-03-04T09:42:33.765393+0000 (oldest deadline 2024-03-04T09:42:59.665478+0000) 2024-03-04T09:43:00.882 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 04 09:43:00 smithi106 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-10[60969]: 2024-03-04T09:43:00.569+0000 7f10f0d39700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-03-04T09:42:08.466029+0000 front 2024-03-04T09:42:08.465963+0000 (oldest deadline 2024-03-04T09:42:32.566054+0000) 2024-03-04T09:43:00.883 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 04 09:43:00 smithi106 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-10[60969]: 2024-03-04T09:43:00.569+0000 7f10f0d39700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-04T09:42:15.467191+0000 front 2024-03-04T09:42:15.467152+0000 (oldest deadline 2024-03-04T09:42:39.567351+0000) 2024-03-04T09:43:00.883 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 04 09:43:00 smithi106 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-10[60969]: 2024-03-04T09:43:00.569+0000 7f10f0d39700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-03-04T09:42:25.368854+0000 front 2024-03-04T09:42:25.368833+0000 (oldest deadline 2024-03-04T09:42:48.868990+0000) 2024-03-04T09:43:00.883 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 04 09:43:00 smithi106 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-10[60969]: 2024-03-04T09:43:00.569+0000 7f10f0d39700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2024-03-04T09:42:28.869679+0000 front 2024-03-04T09:42:28.869692+0000 (oldest deadline 2024-03-04T09:42:55.270190+0000) 2024-03-04T09:43:00.883 INFO:journalctl@ceph.osd.7.smithi106.stdout:Mar 04 09:43:00 smithi106 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-7[54346]: 2024-03-04T09:43:00.705+0000 7f3be1c52700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-04T09:42:17.071420+0000 front 2024-03-04T09:42:17.071488+0000 (oldest deadline 2024-03-04T09:42:40.472167+0000) 2024-03-04T09:43:00.884 INFO:journalctl@ceph.osd.7.smithi106.stdout:Mar 04 09:43:00 smithi106 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-7[54346]: 2024-03-04T09:43:00.705+0000 7f3be1c52700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-03-04T09:42:25.173267+0000 front 2024-03-04T09:42:25.173398+0000 (oldest deadline 2024-03-04T09:42:48.673397+0000) 2024-03-04T09:43:00.884 INFO:journalctl@ceph.osd.7.smithi106.stdout:Mar 04 09:43:00 smithi106 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-7[54346]: 2024-03-04T09:43:00.705+0000 7f3be1c52700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2024-03-04T09:42:33.974526+0000 front 2024-03-04T09:42:33.974586+0000 (oldest deadline 2024-03-04T09:42:59.175048+0000) 2024-03-04T09:43:00.917 DEBUG:teuthology.orchestra.run.smithi052:> sudo pkill -f 'journalctl -f -n 0 -u ceph-4e059f08-da08-11ee-95c6-87774f69a715@osd.6.service' 2024-03-04T09:43:00.962 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-04T09:43:00.962 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-03-04T09:43:00.962 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-03-04T09:43:00.963 DEBUG:teuthology.orchestra.run.smithi106:> sudo systemctl stop ceph-4e059f08-da08-11ee-95c6-87774f69a715@osd.7 2024-03-04T09:43:01.269 INFO:journalctl@ceph.osd.7.smithi106.stdout:Mar 04 09:43:01 smithi106 systemd[1]: Stopping Ceph osd.7 for 4e059f08-da08-11ee-95c6-87774f69a715... 2024-03-04T09:43:01.270 INFO:journalctl@ceph.osd.7.smithi106.stdout:Mar 04 09:43:01 smithi106 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-7[54346]: 2024-03-04T09:43:01.270+0000 7f3be565e700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-04T09:43:01.270 INFO:journalctl@ceph.osd.7.smithi106.stdout:Mar 04 09:43:01 smithi106 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-7[54346]: 2024-03-04T09:43:01.270+0000 7f3be565e700 -1 osd.7 106 *** Got signal Terminated *** 2024-03-04T09:43:01.270 INFO:journalctl@ceph.osd.7.smithi106.stdout:Mar 04 09:43:01 smithi106 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-7[54346]: 2024-03-04T09:43:01.270+0000 7f3be565e700 -1 osd.7 106 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-04T09:43:01.587 INFO:journalctl@ceph.osd.8.smithi191.stdout:Mar 04 09:43:01 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-8[55524]: 2024-03-04T09:43:01.201+0000 7f436df85700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-04T09:42:17.524704+0000 front 2024-03-04T09:42:17.524866+0000 (oldest deadline 2024-03-04T09:42:41.024775+0000) 2024-03-04T09:43:01.588 INFO:journalctl@ceph.osd.8.smithi191.stdout:Mar 04 09:43:01 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-8[55524]: 2024-03-04T09:43:01.201+0000 7f436df85700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-03-04T09:42:25.125775+0000 front 2024-03-04T09:42:25.125737+0000 (oldest deadline 2024-03-04T09:42:48.025913+0000) 2024-03-04T09:43:01.588 INFO:journalctl@ceph.osd.8.smithi191.stdout:Mar 04 09:43:01 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-8[55524]: 2024-03-04T09:43:01.201+0000 7f436df85700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2024-03-04T09:42:34.328161+0000 front 2024-03-04T09:42:34.328294+0000 (oldest deadline 2024-03-04T09:42:55.428059+0000) 2024-03-04T09:43:01.588 INFO:journalctl@ceph.osd.11.smithi191.stdout:Mar 04 09:43:01 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-11[62160]: 2024-03-04T09:43:01.425+0000 7fdceac97700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-03-04T09:42:08.553313+0000 front 2024-03-04T09:42:08.553176+0000 (oldest deadline 2024-03-04T09:42:32.653011+0000) 2024-03-04T09:43:01.588 INFO:journalctl@ceph.osd.11.smithi191.stdout:Mar 04 09:43:01 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-11[62160]: 2024-03-04T09:43:01.425+0000 7fdceac97700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-04T09:42:18.355200+0000 front 2024-03-04T09:42:18.355189+0000 (oldest deadline 2024-03-04T09:42:44.254936+0000) 2024-03-04T09:43:01.588 INFO:journalctl@ceph.osd.11.smithi191.stdout:Mar 04 09:43:01 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-11[62160]: 2024-03-04T09:43:01.425+0000 7fdceac97700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-03-04T09:42:24.255361+0000 front 2024-03-04T09:42:24.255246+0000 (oldest deadline 2024-03-04T09:42:47.155465+0000) 2024-03-04T09:43:01.836 INFO:journalctl@ceph.osd.9.smithi052.stdout:Mar 04 09:43:01 smithi052 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-9[65396]: 2024-03-04T09:43:01.569+0000 7fd1f1e8e700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-03-04T09:42:09.861821+0000 front 2024-03-04T09:42:09.861761+0000 (oldest deadline 2024-03-04T09:42:32.161859+0000) 2024-03-04T09:43:01.836 INFO:journalctl@ceph.osd.9.smithi052.stdout:Mar 04 09:43:01 smithi052 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-9[65396]: 2024-03-04T09:43:01.569+0000 7fd1f1e8e700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-04T09:42:17.363956+0000 front 2024-03-04T09:42:17.363876+0000 (oldest deadline 2024-03-04T09:42:41.463546+0000) 2024-03-04T09:43:01.837 INFO:journalctl@ceph.osd.9.smithi052.stdout:Mar 04 09:43:01 smithi052 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-9[65396]: 2024-03-04T09:43:01.569+0000 7fd1f1e8e700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-03-04T09:42:21.464200+0000 front 2024-03-04T09:42:21.464139+0000 (oldest deadline 2024-03-04T09:42:47.363942+0000) 2024-03-04T09:43:01.837 INFO:journalctl@ceph.osd.9.smithi052.stdout:Mar 04 09:43:01 smithi052 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-9[65396]: 2024-03-04T09:43:01.569+0000 7fd1f1e8e700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2024-03-04T09:42:33.765329+0000 front 2024-03-04T09:42:33.765393+0000 (oldest deadline 2024-03-04T09:42:59.665478+0000) 2024-03-04T09:43:01.883 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 04 09:43:01 smithi106 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-10[60969]: 2024-03-04T09:43:01.552+0000 7f10f0d39700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-03-04T09:42:08.466029+0000 front 2024-03-04T09:42:08.465963+0000 (oldest deadline 2024-03-04T09:42:32.566054+0000) 2024-03-04T09:43:01.883 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 04 09:43:01 smithi106 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-10[60969]: 2024-03-04T09:43:01.552+0000 7f10f0d39700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-04T09:42:15.467191+0000 front 2024-03-04T09:42:15.467152+0000 (oldest deadline 2024-03-04T09:42:39.567351+0000) 2024-03-04T09:43:01.883 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 04 09:43:01 smithi106 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-10[60969]: 2024-03-04T09:43:01.552+0000 7f10f0d39700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-03-04T09:42:25.368854+0000 front 2024-03-04T09:42:25.368833+0000 (oldest deadline 2024-03-04T09:42:48.868990+0000) 2024-03-04T09:43:01.883 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 04 09:43:01 smithi106 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-10[60969]: 2024-03-04T09:43:01.552+0000 7f10f0d39700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2024-03-04T09:42:28.869679+0000 front 2024-03-04T09:42:28.869692+0000 (oldest deadline 2024-03-04T09:42:55.270190+0000) 2024-03-04T09:43:01.883 INFO:journalctl@ceph.osd.7.smithi106.stdout:Mar 04 09:43:01 smithi106 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-7[54346]: 2024-03-04T09:43:01.727+0000 7f3be1c52700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-04T09:42:17.071420+0000 front 2024-03-04T09:42:17.071488+0000 (oldest deadline 2024-03-04T09:42:40.472167+0000) 2024-03-04T09:43:01.884 INFO:journalctl@ceph.osd.7.smithi106.stdout:Mar 04 09:43:01 smithi106 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-7[54346]: 2024-03-04T09:43:01.727+0000 7f3be1c52700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-03-04T09:42:25.173267+0000 front 2024-03-04T09:42:25.173398+0000 (oldest deadline 2024-03-04T09:42:48.673397+0000) 2024-03-04T09:43:01.884 INFO:journalctl@ceph.osd.7.smithi106.stdout:Mar 04 09:43:01 smithi106 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-7[54346]: 2024-03-04T09:43:01.727+0000 7f3be1c52700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2024-03-04T09:42:33.974526+0000 front 2024-03-04T09:42:33.974586+0000 (oldest deadline 2024-03-04T09:42:59.175048+0000) 2024-03-04T09:43:02.588 INFO:journalctl@ceph.osd.8.smithi191.stdout:Mar 04 09:43:02 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-8[55524]: 2024-03-04T09:43:02.156+0000 7f436df85700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-04T09:42:17.524704+0000 front 2024-03-04T09:42:17.524866+0000 (oldest deadline 2024-03-04T09:42:41.024775+0000) 2024-03-04T09:43:02.588 INFO:journalctl@ceph.osd.8.smithi191.stdout:Mar 04 09:43:02 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-8[55524]: 2024-03-04T09:43:02.156+0000 7f436df85700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-03-04T09:42:25.125775+0000 front 2024-03-04T09:42:25.125737+0000 (oldest deadline 2024-03-04T09:42:48.025913+0000) 2024-03-04T09:43:02.588 INFO:journalctl@ceph.osd.8.smithi191.stdout:Mar 04 09:43:02 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-8[55524]: 2024-03-04T09:43:02.156+0000 7f436df85700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2024-03-04T09:42:34.328161+0000 front 2024-03-04T09:42:34.328294+0000 (oldest deadline 2024-03-04T09:42:55.428059+0000) 2024-03-04T09:43:02.588 INFO:journalctl@ceph.osd.11.smithi191.stdout:Mar 04 09:43:02 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-11[62160]: 2024-03-04T09:43:02.402+0000 7fdceac97700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-03-04T09:42:08.553313+0000 front 2024-03-04T09:42:08.553176+0000 (oldest deadline 2024-03-04T09:42:32.653011+0000) 2024-03-04T09:43:02.588 INFO:journalctl@ceph.osd.11.smithi191.stdout:Mar 04 09:43:02 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-11[62160]: 2024-03-04T09:43:02.402+0000 7fdceac97700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-04T09:42:18.355200+0000 front 2024-03-04T09:42:18.355189+0000 (oldest deadline 2024-03-04T09:42:44.254936+0000) 2024-03-04T09:43:02.588 INFO:journalctl@ceph.osd.11.smithi191.stdout:Mar 04 09:43:02 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-11[62160]: 2024-03-04T09:43:02.402+0000 7fdceac97700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-03-04T09:42:24.255361+0000 front 2024-03-04T09:42:24.255246+0000 (oldest deadline 2024-03-04T09:42:47.155465+0000) 2024-03-04T09:43:02.785 INFO:journalctl@ceph.osd.7.smithi106.stdout:Mar 04 09:43:02 smithi106 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-7[54346]: 2024-03-04T09:43:02.718+0000 7f3be1c52700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-04T09:42:17.071420+0000 front 2024-03-04T09:42:17.071488+0000 (oldest deadline 2024-03-04T09:42:40.472167+0000) 2024-03-04T09:43:02.785 INFO:journalctl@ceph.osd.7.smithi106.stdout:Mar 04 09:43:02 smithi106 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-7[54346]: 2024-03-04T09:43:02.718+0000 7f3be1c52700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-03-04T09:42:25.173267+0000 front 2024-03-04T09:42:25.173398+0000 (oldest deadline 2024-03-04T09:42:48.673397+0000) 2024-03-04T09:43:02.785 INFO:journalctl@ceph.osd.7.smithi106.stdout:Mar 04 09:43:02 smithi106 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-7[54346]: 2024-03-04T09:43:02.718+0000 7f3be1c52700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2024-03-04T09:42:33.974526+0000 front 2024-03-04T09:42:33.974586+0000 (oldest deadline 2024-03-04T09:42:59.175048+0000) 2024-03-04T09:43:02.785 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 04 09:43:02 smithi106 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-10[60969]: 2024-03-04T09:43:02.516+0000 7f10f0d39700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-03-04T09:42:08.466029+0000 front 2024-03-04T09:42:08.465963+0000 (oldest deadline 2024-03-04T09:42:32.566054+0000) 2024-03-04T09:43:02.786 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 04 09:43:02 smithi106 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-10[60969]: 2024-03-04T09:43:02.516+0000 7f10f0d39700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-04T09:42:15.467191+0000 front 2024-03-04T09:42:15.467152+0000 (oldest deadline 2024-03-04T09:42:39.567351+0000) 2024-03-04T09:43:02.786 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 04 09:43:02 smithi106 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-10[60969]: 2024-03-04T09:43:02.516+0000 7f10f0d39700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-03-04T09:42:25.368854+0000 front 2024-03-04T09:42:25.368833+0000 (oldest deadline 2024-03-04T09:42:48.868990+0000) 2024-03-04T09:43:02.786 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 04 09:43:02 smithi106 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-10[60969]: 2024-03-04T09:43:02.516+0000 7f10f0d39700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2024-03-04T09:42:28.869679+0000 front 2024-03-04T09:42:28.869692+0000 (oldest deadline 2024-03-04T09:42:55.270190+0000) 2024-03-04T09:43:02.836 INFO:journalctl@ceph.osd.9.smithi052.stdout:Mar 04 09:43:02 smithi052 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-9[65396]: 2024-03-04T09:43:02.554+0000 7fd1f1e8e700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-03-04T09:42:09.861821+0000 front 2024-03-04T09:42:09.861761+0000 (oldest deadline 2024-03-04T09:42:32.161859+0000) 2024-03-04T09:43:02.836 INFO:journalctl@ceph.osd.9.smithi052.stdout:Mar 04 09:43:02 smithi052 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-9[65396]: 2024-03-04T09:43:02.554+0000 7fd1f1e8e700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-04T09:42:17.363956+0000 front 2024-03-04T09:42:17.363876+0000 (oldest deadline 2024-03-04T09:42:41.463546+0000) 2024-03-04T09:43:02.837 INFO:journalctl@ceph.osd.9.smithi052.stdout:Mar 04 09:43:02 smithi052 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-9[65396]: 2024-03-04T09:43:02.554+0000 7fd1f1e8e700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-03-04T09:42:21.464200+0000 front 2024-03-04T09:42:21.464139+0000 (oldest deadline 2024-03-04T09:42:47.363942+0000) 2024-03-04T09:43:02.837 INFO:journalctl@ceph.osd.9.smithi052.stdout:Mar 04 09:43:02 smithi052 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-9[65396]: 2024-03-04T09:43:02.554+0000 7fd1f1e8e700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2024-03-04T09:42:33.765329+0000 front 2024-03-04T09:42:33.765393+0000 (oldest deadline 2024-03-04T09:42:59.665478+0000) 2024-03-04T09:43:03.371 INFO:journalctl@ceph.osd.8.smithi191.stdout:Mar 04 09:43:03 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-8[55524]: 2024-03-04T09:43:03.110+0000 7f436df85700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-04T09:42:17.524704+0000 front 2024-03-04T09:42:17.524866+0000 (oldest deadline 2024-03-04T09:42:41.024775+0000) 2024-03-04T09:43:03.371 INFO:journalctl@ceph.osd.8.smithi191.stdout:Mar 04 09:43:03 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-8[55524]: 2024-03-04T09:43:03.110+0000 7f436df85700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-03-04T09:42:25.125775+0000 front 2024-03-04T09:42:25.125737+0000 (oldest deadline 2024-03-04T09:42:48.025913+0000) 2024-03-04T09:43:03.371 INFO:journalctl@ceph.osd.8.smithi191.stdout:Mar 04 09:43:03 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-8[55524]: 2024-03-04T09:43:03.110+0000 7f436df85700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2024-03-04T09:42:34.328161+0000 front 2024-03-04T09:42:34.328294+0000 (oldest deadline 2024-03-04T09:42:55.428059+0000) 2024-03-04T09:43:03.799 INFO:journalctl@ceph.osd.7.smithi106.stdout:Mar 04 09:43:03 smithi106 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-7[54346]: 2024-03-04T09:43:03.725+0000 7f3be1c52700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-04T09:42:17.071420+0000 front 2024-03-04T09:42:17.071488+0000 (oldest deadline 2024-03-04T09:42:40.472167+0000) 2024-03-04T09:43:03.800 INFO:journalctl@ceph.osd.7.smithi106.stdout:Mar 04 09:43:03 smithi106 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-7[54346]: 2024-03-04T09:43:03.725+0000 7f3be1c52700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-03-04T09:42:25.173267+0000 front 2024-03-04T09:42:25.173398+0000 (oldest deadline 2024-03-04T09:42:48.673397+0000) 2024-03-04T09:43:03.800 INFO:journalctl@ceph.osd.7.smithi106.stdout:Mar 04 09:43:03 smithi106 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-7[54346]: 2024-03-04T09:43:03.725+0000 7f3be1c52700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2024-03-04T09:42:33.974526+0000 front 2024-03-04T09:42:33.974586+0000 (oldest deadline 2024-03-04T09:42:59.175048+0000) 2024-03-04T09:43:03.800 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 04 09:43:03 smithi106 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-10[60969]: 2024-03-04T09:43:03.542+0000 7f10f0d39700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-03-04T09:42:08.466029+0000 front 2024-03-04T09:42:08.465963+0000 (oldest deadline 2024-03-04T09:42:32.566054+0000) 2024-03-04T09:43:03.800 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 04 09:43:03 smithi106 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-10[60969]: 2024-03-04T09:43:03.542+0000 7f10f0d39700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-04T09:42:15.467191+0000 front 2024-03-04T09:42:15.467152+0000 (oldest deadline 2024-03-04T09:42:39.567351+0000) 2024-03-04T09:43:03.800 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 04 09:43:03 smithi106 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-10[60969]: 2024-03-04T09:43:03.542+0000 7f10f0d39700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-03-04T09:42:25.368854+0000 front 2024-03-04T09:42:25.368833+0000 (oldest deadline 2024-03-04T09:42:48.868990+0000) 2024-03-04T09:43:03.800 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 04 09:43:03 smithi106 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-10[60969]: 2024-03-04T09:43:03.542+0000 7f10f0d39700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2024-03-04T09:42:28.869679+0000 front 2024-03-04T09:42:28.869692+0000 (oldest deadline 2024-03-04T09:42:55.270190+0000) 2024-03-04T09:43:03.836 INFO:journalctl@ceph.osd.9.smithi052.stdout:Mar 04 09:43:03 smithi052 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-9[65396]: 2024-03-04T09:43:03.513+0000 7fd1f1e8e700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-03-04T09:42:09.861821+0000 front 2024-03-04T09:42:09.861761+0000 (oldest deadline 2024-03-04T09:42:32.161859+0000) 2024-03-04T09:43:03.836 INFO:journalctl@ceph.osd.9.smithi052.stdout:Mar 04 09:43:03 smithi052 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-9[65396]: 2024-03-04T09:43:03.513+0000 7fd1f1e8e700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-04T09:42:17.363956+0000 front 2024-03-04T09:42:17.363876+0000 (oldest deadline 2024-03-04T09:42:41.463546+0000) 2024-03-04T09:43:03.837 INFO:journalctl@ceph.osd.9.smithi052.stdout:Mar 04 09:43:03 smithi052 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-9[65396]: 2024-03-04T09:43:03.513+0000 7fd1f1e8e700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-03-04T09:42:21.464200+0000 front 2024-03-04T09:42:21.464139+0000 (oldest deadline 2024-03-04T09:42:47.363942+0000) 2024-03-04T09:43:03.837 INFO:journalctl@ceph.osd.9.smithi052.stdout:Mar 04 09:43:03 smithi052 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-9[65396]: 2024-03-04T09:43:03.513+0000 7fd1f1e8e700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2024-03-04T09:42:33.765329+0000 front 2024-03-04T09:42:33.765393+0000 (oldest deadline 2024-03-04T09:42:59.665478+0000) 2024-03-04T09:43:03.838 INFO:journalctl@ceph.osd.11.smithi191.stdout:Mar 04 09:43:03 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-11[62160]: 2024-03-04T09:43:03.373+0000 7fdceac97700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-03-04T09:42:08.553313+0000 front 2024-03-04T09:42:08.553176+0000 (oldest deadline 2024-03-04T09:42:32.653011+0000) 2024-03-04T09:43:03.838 INFO:journalctl@ceph.osd.11.smithi191.stdout:Mar 04 09:43:03 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-11[62160]: 2024-03-04T09:43:03.373+0000 7fdceac97700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-04T09:42:18.355200+0000 front 2024-03-04T09:42:18.355189+0000 (oldest deadline 2024-03-04T09:42:44.254936+0000) 2024-03-04T09:43:03.838 INFO:journalctl@ceph.osd.11.smithi191.stdout:Mar 04 09:43:03 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-11[62160]: 2024-03-04T09:43:03.373+0000 7fdceac97700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-03-04T09:42:24.255361+0000 front 2024-03-04T09:42:24.255246+0000 (oldest deadline 2024-03-04T09:42:47.155465+0000) 2024-03-04T09:43:04.388 INFO:journalctl@ceph.osd.8.smithi191.stdout:Mar 04 09:43:04 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-8[55524]: 2024-03-04T09:43:04.130+0000 7f436df85700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-04T09:42:17.524704+0000 front 2024-03-04T09:42:17.524866+0000 (oldest deadline 2024-03-04T09:42:41.024775+0000) 2024-03-04T09:43:04.389 INFO:journalctl@ceph.osd.8.smithi191.stdout:Mar 04 09:43:04 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-8[55524]: 2024-03-04T09:43:04.130+0000 7f436df85700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-03-04T09:42:25.125775+0000 front 2024-03-04T09:42:25.125737+0000 (oldest deadline 2024-03-04T09:42:48.025913+0000) 2024-03-04T09:43:04.389 INFO:journalctl@ceph.osd.8.smithi191.stdout:Mar 04 09:43:04 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-8[55524]: 2024-03-04T09:43:04.130+0000 7f436df85700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2024-03-04T09:42:34.328161+0000 front 2024-03-04T09:42:34.328294+0000 (oldest deadline 2024-03-04T09:42:55.428059+0000) 2024-03-04T09:43:04.389 INFO:journalctl@ceph.osd.8.smithi191.stdout:Mar 04 09:43:04 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-8[55524]: 2024-03-04T09:43:04.130+0000 7f436df85700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-03-04T09:42:41.329036+0000 front 2024-03-04T09:42:41.329257+0000 (oldest deadline 2024-03-04T09:43:03.628955+0000) 2024-03-04T09:43:04.836 INFO:journalctl@ceph.osd.9.smithi052.stdout:Mar 04 09:43:04 smithi052 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-9[65396]: 2024-03-04T09:43:04.486+0000 7fd1f1e8e700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-03-04T09:42:09.861821+0000 front 2024-03-04T09:42:09.861761+0000 (oldest deadline 2024-03-04T09:42:32.161859+0000) 2024-03-04T09:43:04.836 INFO:journalctl@ceph.osd.9.smithi052.stdout:Mar 04 09:43:04 smithi052 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-9[65396]: 2024-03-04T09:43:04.486+0000 7fd1f1e8e700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-04T09:42:17.363956+0000 front 2024-03-04T09:42:17.363876+0000 (oldest deadline 2024-03-04T09:42:41.463546+0000) 2024-03-04T09:43:04.836 INFO:journalctl@ceph.osd.9.smithi052.stdout:Mar 04 09:43:04 smithi052 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-9[65396]: 2024-03-04T09:43:04.486+0000 7fd1f1e8e700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-03-04T09:42:21.464200+0000 front 2024-03-04T09:42:21.464139+0000 (oldest deadline 2024-03-04T09:42:47.363942+0000) 2024-03-04T09:43:04.837 INFO:journalctl@ceph.osd.9.smithi052.stdout:Mar 04 09:43:04 smithi052 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-9[65396]: 2024-03-04T09:43:04.486+0000 7fd1f1e8e700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2024-03-04T09:42:33.765329+0000 front 2024-03-04T09:42:33.765393+0000 (oldest deadline 2024-03-04T09:42:59.665478+0000) 2024-03-04T09:43:04.837 INFO:journalctl@ceph.osd.11.smithi191.stdout:Mar 04 09:43:04 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-11[62160]: 2024-03-04T09:43:04.390+0000 7fdceac97700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-03-04T09:42:08.553313+0000 front 2024-03-04T09:42:08.553176+0000 (oldest deadline 2024-03-04T09:42:32.653011+0000) 2024-03-04T09:43:04.838 INFO:journalctl@ceph.osd.11.smithi191.stdout:Mar 04 09:43:04 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-11[62160]: 2024-03-04T09:43:04.390+0000 7fdceac97700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-04T09:42:18.355200+0000 front 2024-03-04T09:42:18.355189+0000 (oldest deadline 2024-03-04T09:42:44.254936+0000) 2024-03-04T09:43:04.838 INFO:journalctl@ceph.osd.11.smithi191.stdout:Mar 04 09:43:04 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-11[62160]: 2024-03-04T09:43:04.390+0000 7fdceac97700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-03-04T09:42:24.255361+0000 front 2024-03-04T09:42:24.255246+0000 (oldest deadline 2024-03-04T09:42:47.155465+0000) 2024-03-04T09:43:04.838 INFO:journalctl@ceph.osd.11.smithi191.stdout:Mar 04 09:43:04 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-11[62160]: 2024-03-04T09:43:04.390+0000 7fdceac97700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-03-04T09:42:41.857812+0000 front 2024-03-04T09:42:41.857710+0000 (oldest deadline 2024-03-04T09:43:03.557748+0000) 2024-03-04T09:43:04.882 INFO:journalctl@ceph.osd.7.smithi106.stdout:Mar 04 09:43:04 smithi106 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-7[54346]: 2024-03-04T09:43:04.685+0000 7f3be1c52700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-04T09:42:17.071420+0000 front 2024-03-04T09:42:17.071488+0000 (oldest deadline 2024-03-04T09:42:40.472167+0000) 2024-03-04T09:43:04.883 INFO:journalctl@ceph.osd.7.smithi106.stdout:Mar 04 09:43:04 smithi106 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-7[54346]: 2024-03-04T09:43:04.685+0000 7f3be1c52700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-03-04T09:42:25.173267+0000 front 2024-03-04T09:42:25.173398+0000 (oldest deadline 2024-03-04T09:42:48.673397+0000) 2024-03-04T09:43:04.883 INFO:journalctl@ceph.osd.7.smithi106.stdout:Mar 04 09:43:04 smithi106 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-7[54346]: 2024-03-04T09:43:04.685+0000 7f3be1c52700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2024-03-04T09:42:33.974526+0000 front 2024-03-04T09:42:33.974586+0000 (oldest deadline 2024-03-04T09:42:59.175048+0000) 2024-03-04T09:43:04.883 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 04 09:43:04 smithi106 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-10[60969]: 2024-03-04T09:43:04.511+0000 7f10f0d39700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-03-04T09:42:08.466029+0000 front 2024-03-04T09:42:08.465963+0000 (oldest deadline 2024-03-04T09:42:32.566054+0000) 2024-03-04T09:43:04.883 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 04 09:43:04 smithi106 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-10[60969]: 2024-03-04T09:43:04.511+0000 7f10f0d39700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-04T09:42:15.467191+0000 front 2024-03-04T09:42:15.467152+0000 (oldest deadline 2024-03-04T09:42:39.567351+0000) 2024-03-04T09:43:04.883 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 04 09:43:04 smithi106 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-10[60969]: 2024-03-04T09:43:04.511+0000 7f10f0d39700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-03-04T09:42:25.368854+0000 front 2024-03-04T09:42:25.368833+0000 (oldest deadline 2024-03-04T09:42:48.868990+0000) 2024-03-04T09:43:04.884 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 04 09:43:04 smithi106 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-10[60969]: 2024-03-04T09:43:04.511+0000 7f10f0d39700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2024-03-04T09:42:28.869679+0000 front 2024-03-04T09:42:28.869692+0000 (oldest deadline 2024-03-04T09:42:55.270190+0000) 2024-03-04T09:43:05.588 INFO:journalctl@ceph.osd.8.smithi191.stdout:Mar 04 09:43:05 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-8[55524]: 2024-03-04T09:43:05.162+0000 7f436df85700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-04T09:42:17.524704+0000 front 2024-03-04T09:42:17.524866+0000 (oldest deadline 2024-03-04T09:42:41.024775+0000) 2024-03-04T09:43:05.588 INFO:journalctl@ceph.osd.8.smithi191.stdout:Mar 04 09:43:05 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-8[55524]: 2024-03-04T09:43:05.162+0000 7f436df85700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-03-04T09:42:25.125775+0000 front 2024-03-04T09:42:25.125737+0000 (oldest deadline 2024-03-04T09:42:48.025913+0000) 2024-03-04T09:43:05.588 INFO:journalctl@ceph.osd.8.smithi191.stdout:Mar 04 09:43:05 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-8[55524]: 2024-03-04T09:43:05.162+0000 7f436df85700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2024-03-04T09:42:34.328161+0000 front 2024-03-04T09:42:34.328294+0000 (oldest deadline 2024-03-04T09:42:55.428059+0000) 2024-03-04T09:43:05.589 INFO:journalctl@ceph.osd.8.smithi191.stdout:Mar 04 09:43:05 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-8[55524]: 2024-03-04T09:43:05.162+0000 7f436df85700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-03-04T09:42:41.329036+0000 front 2024-03-04T09:42:41.329257+0000 (oldest deadline 2024-03-04T09:43:03.628955+0000) 2024-03-04T09:43:05.589 INFO:journalctl@ceph.osd.11.smithi191.stdout:Mar 04 09:43:05 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-11[62160]: 2024-03-04T09:43:05.381+0000 7fdceac97700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-03-04T09:42:08.553313+0000 front 2024-03-04T09:42:08.553176+0000 (oldest deadline 2024-03-04T09:42:32.653011+0000) 2024-03-04T09:43:05.589 INFO:journalctl@ceph.osd.11.smithi191.stdout:Mar 04 09:43:05 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-11[62160]: 2024-03-04T09:43:05.381+0000 7fdceac97700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-04T09:42:18.355200+0000 front 2024-03-04T09:42:18.355189+0000 (oldest deadline 2024-03-04T09:42:44.254936+0000) 2024-03-04T09:43:05.589 INFO:journalctl@ceph.osd.11.smithi191.stdout:Mar 04 09:43:05 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-11[62160]: 2024-03-04T09:43:05.381+0000 7fdceac97700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-03-04T09:42:24.255361+0000 front 2024-03-04T09:42:24.255246+0000 (oldest deadline 2024-03-04T09:42:47.155465+0000) 2024-03-04T09:43:05.589 INFO:journalctl@ceph.osd.11.smithi191.stdout:Mar 04 09:43:05 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-11[62160]: 2024-03-04T09:43:05.381+0000 7fdceac97700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-03-04T09:42:41.857812+0000 front 2024-03-04T09:42:41.857710+0000 (oldest deadline 2024-03-04T09:43:03.557748+0000) 2024-03-04T09:43:05.836 INFO:journalctl@ceph.osd.9.smithi052.stdout:Mar 04 09:43:05 smithi052 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-9[65396]: 2024-03-04T09:43:05.439+0000 7fd1f1e8e700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-03-04T09:42:09.861821+0000 front 2024-03-04T09:42:09.861761+0000 (oldest deadline 2024-03-04T09:42:32.161859+0000) 2024-03-04T09:43:05.836 INFO:journalctl@ceph.osd.9.smithi052.stdout:Mar 04 09:43:05 smithi052 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-9[65396]: 2024-03-04T09:43:05.439+0000 7fd1f1e8e700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-04T09:42:17.363956+0000 front 2024-03-04T09:42:17.363876+0000 (oldest deadline 2024-03-04T09:42:41.463546+0000) 2024-03-04T09:43:05.836 INFO:journalctl@ceph.osd.9.smithi052.stdout:Mar 04 09:43:05 smithi052 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-9[65396]: 2024-03-04T09:43:05.439+0000 7fd1f1e8e700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-03-04T09:42:21.464200+0000 front 2024-03-04T09:42:21.464139+0000 (oldest deadline 2024-03-04T09:42:47.363942+0000) 2024-03-04T09:43:05.837 INFO:journalctl@ceph.osd.9.smithi052.stdout:Mar 04 09:43:05 smithi052 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-9[65396]: 2024-03-04T09:43:05.439+0000 7fd1f1e8e700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2024-03-04T09:42:33.765329+0000 front 2024-03-04T09:42:33.765393+0000 (oldest deadline 2024-03-04T09:42:59.665478+0000) 2024-03-04T09:43:05.837 INFO:journalctl@ceph.osd.9.smithi052.stdout:Mar 04 09:43:05 smithi052 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-9[65396]: 2024-03-04T09:43:05.439+0000 7fd1f1e8e700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-03-04T09:42:39.666192+0000 front 2024-03-04T09:42:39.666267+0000 (oldest deadline 2024-03-04T09:43:04.966101+0000) 2024-03-04T09:43:05.883 INFO:journalctl@ceph.osd.7.smithi106.stdout:Mar 04 09:43:05 smithi106 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-7[54346]: 2024-03-04T09:43:05.694+0000 7f3be1c52700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-04T09:42:17.071420+0000 front 2024-03-04T09:42:17.071488+0000 (oldest deadline 2024-03-04T09:42:40.472167+0000) 2024-03-04T09:43:05.883 INFO:journalctl@ceph.osd.7.smithi106.stdout:Mar 04 09:43:05 smithi106 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-7[54346]: 2024-03-04T09:43:05.694+0000 7f3be1c52700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-03-04T09:42:25.173267+0000 front 2024-03-04T09:42:25.173398+0000 (oldest deadline 2024-03-04T09:42:48.673397+0000) 2024-03-04T09:43:05.883 INFO:journalctl@ceph.osd.7.smithi106.stdout:Mar 04 09:43:05 smithi106 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-7[54346]: 2024-03-04T09:43:05.694+0000 7f3be1c52700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2024-03-04T09:42:33.974526+0000 front 2024-03-04T09:42:33.974586+0000 (oldest deadline 2024-03-04T09:42:59.175048+0000) 2024-03-04T09:43:05.883 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 04 09:43:05 smithi106 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-10[60969]: 2024-03-04T09:43:05.559+0000 7f10f0d39700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-03-04T09:42:08.466029+0000 front 2024-03-04T09:42:08.465963+0000 (oldest deadline 2024-03-04T09:42:32.566054+0000) 2024-03-04T09:43:05.883 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 04 09:43:05 smithi106 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-10[60969]: 2024-03-04T09:43:05.559+0000 7f10f0d39700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-04T09:42:15.467191+0000 front 2024-03-04T09:42:15.467152+0000 (oldest deadline 2024-03-04T09:42:39.567351+0000) 2024-03-04T09:43:05.884 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 04 09:43:05 smithi106 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-10[60969]: 2024-03-04T09:43:05.559+0000 7f10f0d39700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-03-04T09:42:25.368854+0000 front 2024-03-04T09:42:25.368833+0000 (oldest deadline 2024-03-04T09:42:48.868990+0000) 2024-03-04T09:43:05.884 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 04 09:43:05 smithi106 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-10[60969]: 2024-03-04T09:43:05.559+0000 7f10f0d39700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2024-03-04T09:42:28.869679+0000 front 2024-03-04T09:42:28.869692+0000 (oldest deadline 2024-03-04T09:42:55.270190+0000) 2024-03-04T09:43:06.384 INFO:journalctl@ceph.osd.8.smithi191.stdout:Mar 04 09:43:06 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-8[55524]: 2024-03-04T09:43:06.121+0000 7f436df85700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-04T09:42:17.524704+0000 front 2024-03-04T09:42:17.524866+0000 (oldest deadline 2024-03-04T09:42:41.024775+0000) 2024-03-04T09:43:06.385 INFO:journalctl@ceph.osd.8.smithi191.stdout:Mar 04 09:43:06 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-8[55524]: 2024-03-04T09:43:06.121+0000 7f436df85700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-03-04T09:42:25.125775+0000 front 2024-03-04T09:42:25.125737+0000 (oldest deadline 2024-03-04T09:42:48.025913+0000) 2024-03-04T09:43:06.385 INFO:journalctl@ceph.osd.8.smithi191.stdout:Mar 04 09:43:06 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-8[55524]: 2024-03-04T09:43:06.121+0000 7f436df85700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2024-03-04T09:42:34.328161+0000 front 2024-03-04T09:42:34.328294+0000 (oldest deadline 2024-03-04T09:42:55.428059+0000) 2024-03-04T09:43:06.385 INFO:journalctl@ceph.osd.8.smithi191.stdout:Mar 04 09:43:06 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-8[55524]: 2024-03-04T09:43:06.121+0000 7f436df85700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-03-04T09:42:41.329036+0000 front 2024-03-04T09:42:41.329257+0000 (oldest deadline 2024-03-04T09:43:03.628955+0000) 2024-03-04T09:43:06.836 INFO:journalctl@ceph.osd.9.smithi052.stdout:Mar 04 09:43:06 smithi052 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-9[65396]: 2024-03-04T09:43:06.434+0000 7fd1f1e8e700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-03-04T09:42:09.861821+0000 front 2024-03-04T09:42:09.861761+0000 (oldest deadline 2024-03-04T09:42:32.161859+0000) 2024-03-04T09:43:06.836 INFO:journalctl@ceph.osd.9.smithi052.stdout:Mar 04 09:43:06 smithi052 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-9[65396]: 2024-03-04T09:43:06.435+0000 7fd1f1e8e700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-04T09:42:17.363956+0000 front 2024-03-04T09:42:17.363876+0000 (oldest deadline 2024-03-04T09:42:41.463546+0000) 2024-03-04T09:43:06.837 INFO:journalctl@ceph.osd.9.smithi052.stdout:Mar 04 09:43:06 smithi052 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-9[65396]: 2024-03-04T09:43:06.435+0000 7fd1f1e8e700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-03-04T09:42:21.464200+0000 front 2024-03-04T09:42:21.464139+0000 (oldest deadline 2024-03-04T09:42:47.363942+0000) 2024-03-04T09:43:06.837 INFO:journalctl@ceph.osd.9.smithi052.stdout:Mar 04 09:43:06 smithi052 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-9[65396]: 2024-03-04T09:43:06.435+0000 7fd1f1e8e700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2024-03-04T09:42:33.765329+0000 front 2024-03-04T09:42:33.765393+0000 (oldest deadline 2024-03-04T09:42:59.665478+0000) 2024-03-04T09:43:06.837 INFO:journalctl@ceph.osd.9.smithi052.stdout:Mar 04 09:43:06 smithi052 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-9[65396]: 2024-03-04T09:43:06.435+0000 7fd1f1e8e700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-03-04T09:42:39.666192+0000 front 2024-03-04T09:42:39.666267+0000 (oldest deadline 2024-03-04T09:43:04.966101+0000) 2024-03-04T09:43:06.837 INFO:journalctl@ceph.osd.11.smithi191.stdout:Mar 04 09:43:06 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-11[62160]: 2024-03-04T09:43:06.387+0000 7fdceac97700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-03-04T09:42:08.553313+0000 front 2024-03-04T09:42:08.553176+0000 (oldest deadline 2024-03-04T09:42:32.653011+0000) 2024-03-04T09:43:06.838 INFO:journalctl@ceph.osd.11.smithi191.stdout:Mar 04 09:43:06 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-11[62160]: 2024-03-04T09:43:06.387+0000 7fdceac97700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-04T09:42:18.355200+0000 front 2024-03-04T09:42:18.355189+0000 (oldest deadline 2024-03-04T09:42:44.254936+0000) 2024-03-04T09:43:06.838 INFO:journalctl@ceph.osd.11.smithi191.stdout:Mar 04 09:43:06 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-11[62160]: 2024-03-04T09:43:06.387+0000 7fdceac97700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-03-04T09:42:24.255361+0000 front 2024-03-04T09:42:24.255246+0000 (oldest deadline 2024-03-04T09:42:47.155465+0000) 2024-03-04T09:43:06.838 INFO:journalctl@ceph.osd.11.smithi191.stdout:Mar 04 09:43:06 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-11[62160]: 2024-03-04T09:43:06.387+0000 7fdceac97700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-03-04T09:42:41.857812+0000 front 2024-03-04T09:42:41.857710+0000 (oldest deadline 2024-03-04T09:43:03.557748+0000) 2024-03-04T09:43:06.882 INFO:journalctl@ceph.osd.7.smithi106.stdout:Mar 04 09:43:06 smithi106 podman[72647]: 2024-03-04 09:43:06.613090486 +0000 UTC m=+5.465226959 container died 1c595e2437851ced3b780dc79941952d66c7d1955dcdaaee95556113f28b4ed0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5, name=ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-7, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-qdb-1bc250c, org.label-schema.build-date=20240222, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8) 2024-03-04T09:43:06.883 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 04 09:43:06 smithi106 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-10[60969]: 2024-03-04T09:43:06.521+0000 7f10f0d39700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-03-04T09:42:08.466029+0000 front 2024-03-04T09:42:08.465963+0000 (oldest deadline 2024-03-04T09:42:32.566054+0000) 2024-03-04T09:43:06.883 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 04 09:43:06 smithi106 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-10[60969]: 2024-03-04T09:43:06.521+0000 7f10f0d39700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-04T09:42:15.467191+0000 front 2024-03-04T09:42:15.467152+0000 (oldest deadline 2024-03-04T09:42:39.567351+0000) 2024-03-04T09:43:06.883 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 04 09:43:06 smithi106 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-10[60969]: 2024-03-04T09:43:06.521+0000 7f10f0d39700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-03-04T09:42:25.368854+0000 front 2024-03-04T09:42:25.368833+0000 (oldest deadline 2024-03-04T09:42:48.868990+0000) 2024-03-04T09:43:06.883 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 04 09:43:06 smithi106 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-10[60969]: 2024-03-04T09:43:06.521+0000 7f10f0d39700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2024-03-04T09:42:28.869679+0000 front 2024-03-04T09:42:28.869692+0000 (oldest deadline 2024-03-04T09:42:55.270190+0000) 2024-03-04T09:43:07.270 INFO:journalctl@ceph.osd.7.smithi106.stdout:Mar 04 09:43:07 smithi106 podman[72647]: 2024-03-04 09:43:07.021417852 +0000 UTC m=+5.873554316 container cleanup 1c595e2437851ced3b780dc79941952d66c7d1955dcdaaee95556113f28b4ed0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5, name=ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-7, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-qdb-1bc250c, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240222, GIT_CLEAN=True) 2024-03-04T09:43:07.270 INFO:journalctl@ceph.osd.7.smithi106.stdout:Mar 04 09:43:07 smithi106 bash[72647]: ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-7 2024-03-04T09:43:07.409 INFO:journalctl@ceph.osd.8.smithi191.stdout:Mar 04 09:43:07 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-8[55524]: 2024-03-04T09:43:07.111+0000 7f436df85700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-04T09:42:17.524704+0000 front 2024-03-04T09:42:17.524866+0000 (oldest deadline 2024-03-04T09:42:41.024775+0000) 2024-03-04T09:43:07.410 INFO:journalctl@ceph.osd.8.smithi191.stdout:Mar 04 09:43:07 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-8[55524]: 2024-03-04T09:43:07.111+0000 7f436df85700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-03-04T09:42:25.125775+0000 front 2024-03-04T09:42:25.125737+0000 (oldest deadline 2024-03-04T09:42:48.025913+0000) 2024-03-04T09:43:07.410 INFO:journalctl@ceph.osd.8.smithi191.stdout:Mar 04 09:43:07 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-8[55524]: 2024-03-04T09:43:07.111+0000 7f436df85700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2024-03-04T09:42:34.328161+0000 front 2024-03-04T09:42:34.328294+0000 (oldest deadline 2024-03-04T09:42:55.428059+0000) 2024-03-04T09:43:07.410 INFO:journalctl@ceph.osd.8.smithi191.stdout:Mar 04 09:43:07 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-8[55524]: 2024-03-04T09:43:07.111+0000 7f436df85700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-03-04T09:42:41.329036+0000 front 2024-03-04T09:42:41.329257+0000 (oldest deadline 2024-03-04T09:43:03.628955+0000) 2024-03-04T09:43:07.584 INFO:journalctl@ceph.osd.7.smithi106.stdout:Mar 04 09:43:07 smithi106 podman[72677]: 2024-03-04 09:43:07.272772123 +0000 UTC m=+0.655540408 container remove 1c595e2437851ced3b780dc79941952d66c7d1955dcdaaee95556113f28b4ed0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5, name=ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-7, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-qdb-1bc250c, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, org.label-schema.license=GPLv2, org.label-schema.build-date=20240222, ceph=True, org.label-schema.schema-version=1.0) 2024-03-04T09:43:07.584 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 04 09:43:07 smithi106 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-10[60969]: 2024-03-04T09:43:07.490+0000 7f10f0d39700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-03-04T09:42:08.466029+0000 front 2024-03-04T09:42:08.465963+0000 (oldest deadline 2024-03-04T09:42:32.566054+0000) 2024-03-04T09:43:07.585 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 04 09:43:07 smithi106 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-10[60969]: 2024-03-04T09:43:07.490+0000 7f10f0d39700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-04T09:42:15.467191+0000 front 2024-03-04T09:42:15.467152+0000 (oldest deadline 2024-03-04T09:42:39.567351+0000) 2024-03-04T09:43:07.585 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 04 09:43:07 smithi106 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-10[60969]: 2024-03-04T09:43:07.490+0000 7f10f0d39700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-03-04T09:42:25.368854+0000 front 2024-03-04T09:42:25.368833+0000 (oldest deadline 2024-03-04T09:42:48.868990+0000) 2024-03-04T09:43:07.585 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 04 09:43:07 smithi106 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-10[60969]: 2024-03-04T09:43:07.490+0000 7f10f0d39700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2024-03-04T09:42:28.869679+0000 front 2024-03-04T09:42:28.869692+0000 (oldest deadline 2024-03-04T09:42:55.270190+0000) 2024-03-04T09:43:07.836 INFO:journalctl@ceph.osd.9.smithi052.stdout:Mar 04 09:43:07 smithi052 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-9[65396]: 2024-03-04T09:43:07.448+0000 7fd1f1e8e700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-03-04T09:42:09.861821+0000 front 2024-03-04T09:42:09.861761+0000 (oldest deadline 2024-03-04T09:42:32.161859+0000) 2024-03-04T09:43:07.836 INFO:journalctl@ceph.osd.9.smithi052.stdout:Mar 04 09:43:07 smithi052 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-9[65396]: 2024-03-04T09:43:07.448+0000 7fd1f1e8e700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-04T09:42:17.363956+0000 front 2024-03-04T09:42:17.363876+0000 (oldest deadline 2024-03-04T09:42:41.463546+0000) 2024-03-04T09:43:07.836 INFO:journalctl@ceph.osd.9.smithi052.stdout:Mar 04 09:43:07 smithi052 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-9[65396]: 2024-03-04T09:43:07.448+0000 7fd1f1e8e700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-03-04T09:42:21.464200+0000 front 2024-03-04T09:42:21.464139+0000 (oldest deadline 2024-03-04T09:42:47.363942+0000) 2024-03-04T09:43:07.837 INFO:journalctl@ceph.osd.9.smithi052.stdout:Mar 04 09:43:07 smithi052 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-9[65396]: 2024-03-04T09:43:07.448+0000 7fd1f1e8e700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2024-03-04T09:42:33.765329+0000 front 2024-03-04T09:42:33.765393+0000 (oldest deadline 2024-03-04T09:42:59.665478+0000) 2024-03-04T09:43:07.837 INFO:journalctl@ceph.osd.9.smithi052.stdout:Mar 04 09:43:07 smithi052 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-9[65396]: 2024-03-04T09:43:07.448+0000 7fd1f1e8e700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-03-04T09:42:39.666192+0000 front 2024-03-04T09:42:39.666267+0000 (oldest deadline 2024-03-04T09:43:04.966101+0000) 2024-03-04T09:43:07.837 INFO:journalctl@ceph.osd.11.smithi191.stdout:Mar 04 09:43:07 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-11[62160]: 2024-03-04T09:43:07.411+0000 7fdceac97700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-03-04T09:42:08.553313+0000 front 2024-03-04T09:42:08.553176+0000 (oldest deadline 2024-03-04T09:42:32.653011+0000) 2024-03-04T09:43:07.838 INFO:journalctl@ceph.osd.11.smithi191.stdout:Mar 04 09:43:07 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-11[62160]: 2024-03-04T09:43:07.411+0000 7fdceac97700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-04T09:42:18.355200+0000 front 2024-03-04T09:42:18.355189+0000 (oldest deadline 2024-03-04T09:42:44.254936+0000) 2024-03-04T09:43:07.838 INFO:journalctl@ceph.osd.11.smithi191.stdout:Mar 04 09:43:07 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-11[62160]: 2024-03-04T09:43:07.411+0000 7fdceac97700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-03-04T09:42:24.255361+0000 front 2024-03-04T09:42:24.255246+0000 (oldest deadline 2024-03-04T09:42:47.155465+0000) 2024-03-04T09:43:07.838 INFO:journalctl@ceph.osd.11.smithi191.stdout:Mar 04 09:43:07 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-11[62160]: 2024-03-04T09:43:07.411+0000 7fdceac97700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-03-04T09:42:41.857812+0000 front 2024-03-04T09:42:41.857710+0000 (oldest deadline 2024-03-04T09:43:03.557748+0000) 2024-03-04T09:43:07.848 INFO:journalctl@ceph.osd.7.smithi106.stdout:Mar 04 09:43:07 smithi106 podman[72767]: 2024-03-04 09:43:07.489667261 +0000 UTC m=+0.018158377 image pull f88ffb32aa6a7d5dfdc954d9f096f0c9cc47981d05ec8d117ba859a22d8185ba quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5 2024-03-04T09:43:07.849 INFO:journalctl@ceph.osd.7.smithi106.stdout:Mar 04 09:43:07 smithi106 podman[72767]: 2024-03-04 09:43:07.590278617 +0000 UTC m=+0.118769752 container create 6f276e58e3ac2dcbc128f68f9e0542071e50d6b3021f517a3bb94dda6e6f2b32 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5, name=ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-7-deactivate, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240222, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, RELEASE=wip-lusov-qdb-1bc250c, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-03-04T09:43:07.849 INFO:journalctl@ceph.osd.7.smithi106.stdout:Mar 04 09:43:07 smithi106 podman[72767]: 2024-03-04 09:43:07.851190029 +0000 UTC m=+0.379681147 container init 6f276e58e3ac2dcbc128f68f9e0542071e50d6b3021f517a3bb94dda6e6f2b32 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5, name=ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-7-deactivate, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240222, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, RELEASE=wip-lusov-qdb-1bc250c, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-03-04T09:43:08.133 INFO:journalctl@ceph.osd.7.smithi106.stdout:Mar 04 09:43:07 smithi106 podman[72767]: 2024-03-04 09:43:07.854702673 +0000 UTC m=+0.383193787 container start 6f276e58e3ac2dcbc128f68f9e0542071e50d6b3021f517a3bb94dda6e6f2b32 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5, name=ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-7-deactivate, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240222, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-qdb-1bc250c, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, ceph=True) 2024-03-04T09:43:08.133 INFO:journalctl@ceph.osd.7.smithi106.stdout:Mar 04 09:43:07 smithi106 podman[72767]: 2024-03-04 09:43:07.942904779 +0000 UTC m=+0.471395909 container attach 6f276e58e3ac2dcbc128f68f9e0542071e50d6b3021f517a3bb94dda6e6f2b32 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5, name=ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-7-deactivate, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-qdb-1bc250c, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.build-date=20240222, ceph=True, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-03-04T09:43:08.378 INFO:journalctl@ceph.osd.8.smithi191.stdout:Mar 04 09:43:08 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-8[55524]: 2024-03-04T09:43:08.106+0000 7f436df85700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-04T09:42:17.524704+0000 front 2024-03-04T09:42:17.524866+0000 (oldest deadline 2024-03-04T09:42:41.024775+0000) 2024-03-04T09:43:08.379 INFO:journalctl@ceph.osd.8.smithi191.stdout:Mar 04 09:43:08 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-8[55524]: 2024-03-04T09:43:08.106+0000 7f436df85700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-03-04T09:42:25.125775+0000 front 2024-03-04T09:42:25.125737+0000 (oldest deadline 2024-03-04T09:42:48.025913+0000) 2024-03-04T09:43:08.379 INFO:journalctl@ceph.osd.8.smithi191.stdout:Mar 04 09:43:08 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-8[55524]: 2024-03-04T09:43:08.106+0000 7f436df85700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2024-03-04T09:42:34.328161+0000 front 2024-03-04T09:42:34.328294+0000 (oldest deadline 2024-03-04T09:42:55.428059+0000) 2024-03-04T09:43:08.379 INFO:journalctl@ceph.osd.8.smithi191.stdout:Mar 04 09:43:08 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-8[55524]: 2024-03-04T09:43:08.106+0000 7f436df85700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-03-04T09:42:41.329036+0000 front 2024-03-04T09:42:41.329257+0000 (oldest deadline 2024-03-04T09:43:03.628955+0000) 2024-03-04T09:43:08.427 INFO:journalctl@ceph.osd.7.smithi106.stdout:Mar 04 09:43:08 smithi106 podman[72767]: 2024-03-04 09:43:08.13991715 +0000 UTC m=+0.668408266 container died 6f276e58e3ac2dcbc128f68f9e0542071e50d6b3021f517a3bb94dda6e6f2b32 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5, name=ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-7-deactivate, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240222, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-qdb-1bc250c, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-04T09:43:08.735 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 04 09:43:08 smithi106 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-10[60969]: 2024-03-04T09:43:08.539+0000 7f10f0d39700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-03-04T09:42:08.466029+0000 front 2024-03-04T09:42:08.465963+0000 (oldest deadline 2024-03-04T09:42:32.566054+0000) 2024-03-04T09:43:08.736 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 04 09:43:08 smithi106 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-10[60969]: 2024-03-04T09:43:08.539+0000 7f10f0d39700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-04T09:42:15.467191+0000 front 2024-03-04T09:42:15.467152+0000 (oldest deadline 2024-03-04T09:42:39.567351+0000) 2024-03-04T09:43:08.736 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 04 09:43:08 smithi106 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-10[60969]: 2024-03-04T09:43:08.539+0000 7f10f0d39700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-03-04T09:42:25.368854+0000 front 2024-03-04T09:42:25.368833+0000 (oldest deadline 2024-03-04T09:42:48.868990+0000) 2024-03-04T09:43:08.736 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 04 09:43:08 smithi106 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-10[60969]: 2024-03-04T09:43:08.539+0000 7f10f0d39700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2024-03-04T09:42:28.869679+0000 front 2024-03-04T09:42:28.869692+0000 (oldest deadline 2024-03-04T09:42:55.270190+0000) 2024-03-04T09:43:08.736 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 04 09:43:08 smithi106 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-10[60969]: 2024-03-04T09:43:08.539+0000 7f10f0d39700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-03-04T09:42:42.271376+0000 front 2024-03-04T09:42:42.271331+0000 (oldest deadline 2024-03-04T09:43:07.571444+0000) 2024-03-04T09:43:08.763 DEBUG:teuthology.orchestra.run.smithi106:> sudo pkill -f 'journalctl -f -n 0 -u ceph-4e059f08-da08-11ee-95c6-87774f69a715@osd.7.service' 2024-03-04T09:43:08.804 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-04T09:43:08.804 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-03-04T09:43:08.805 INFO:tasks.cephadm.osd.8:Stopping osd.8... 2024-03-04T09:43:08.805 DEBUG:teuthology.orchestra.run.smithi191:> sudo systemctl stop ceph-4e059f08-da08-11ee-95c6-87774f69a715@osd.8 2024-03-04T09:43:08.834 INFO:journalctl@ceph.osd.11.smithi191.stdout:Mar 04 09:43:08 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-11[62160]: 2024-03-04T09:43:08.380+0000 7fdceac97700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-03-04T09:42:08.553313+0000 front 2024-03-04T09:42:08.553176+0000 (oldest deadline 2024-03-04T09:42:32.653011+0000) 2024-03-04T09:43:08.834 INFO:journalctl@ceph.osd.11.smithi191.stdout:Mar 04 09:43:08 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-11[62160]: 2024-03-04T09:43:08.380+0000 7fdceac97700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-04T09:42:18.355200+0000 front 2024-03-04T09:42:18.355189+0000 (oldest deadline 2024-03-04T09:42:44.254936+0000) 2024-03-04T09:43:08.834 INFO:journalctl@ceph.osd.11.smithi191.stdout:Mar 04 09:43:08 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-11[62160]: 2024-03-04T09:43:08.380+0000 7fdceac97700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-03-04T09:42:24.255361+0000 front 2024-03-04T09:42:24.255246+0000 (oldest deadline 2024-03-04T09:42:47.155465+0000) 2024-03-04T09:43:08.834 INFO:journalctl@ceph.osd.11.smithi191.stdout:Mar 04 09:43:08 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-11[62160]: 2024-03-04T09:43:08.380+0000 7fdceac97700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-03-04T09:42:41.857812+0000 front 2024-03-04T09:42:41.857710+0000 (oldest deadline 2024-03-04T09:43:03.557748+0000) 2024-03-04T09:43:08.836 INFO:journalctl@ceph.osd.9.smithi052.stdout:Mar 04 09:43:08 smithi052 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-9[65396]: 2024-03-04T09:43:08.423+0000 7fd1f1e8e700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-03-04T09:42:09.861821+0000 front 2024-03-04T09:42:09.861761+0000 (oldest deadline 2024-03-04T09:42:32.161859+0000) 2024-03-04T09:43:08.836 INFO:journalctl@ceph.osd.9.smithi052.stdout:Mar 04 09:43:08 smithi052 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-9[65396]: 2024-03-04T09:43:08.423+0000 7fd1f1e8e700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-04T09:42:17.363956+0000 front 2024-03-04T09:42:17.363876+0000 (oldest deadline 2024-03-04T09:42:41.463546+0000) 2024-03-04T09:43:08.836 INFO:journalctl@ceph.osd.9.smithi052.stdout:Mar 04 09:43:08 smithi052 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-9[65396]: 2024-03-04T09:43:08.423+0000 7fd1f1e8e700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-03-04T09:42:21.464200+0000 front 2024-03-04T09:42:21.464139+0000 (oldest deadline 2024-03-04T09:42:47.363942+0000) 2024-03-04T09:43:08.836 INFO:journalctl@ceph.osd.9.smithi052.stdout:Mar 04 09:43:08 smithi052 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-9[65396]: 2024-03-04T09:43:08.423+0000 7fd1f1e8e700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2024-03-04T09:42:33.765329+0000 front 2024-03-04T09:42:33.765393+0000 (oldest deadline 2024-03-04T09:42:59.665478+0000) 2024-03-04T09:43:08.836 INFO:journalctl@ceph.osd.9.smithi052.stdout:Mar 04 09:43:08 smithi052 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-9[65396]: 2024-03-04T09:43:08.423+0000 7fd1f1e8e700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-03-04T09:42:39.666192+0000 front 2024-03-04T09:42:39.666267+0000 (oldest deadline 2024-03-04T09:43:04.966101+0000) 2024-03-04T09:43:09.088 INFO:journalctl@ceph.osd.8.smithi191.stdout:Mar 04 09:43:08 smithi191 systemd[1]: Stopping Ceph osd.8 for 4e059f08-da08-11ee-95c6-87774f69a715... 2024-03-04T09:43:09.400 INFO:journalctl@ceph.osd.8.smithi191.stdout:Mar 04 09:43:09 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-8[55524]: 2024-03-04T09:43:09.116+0000 7f4371991700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.8 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-04T09:43:09.400 INFO:journalctl@ceph.osd.8.smithi191.stdout:Mar 04 09:43:09 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-8[55524]: 2024-03-04T09:43:09.116+0000 7f4371991700 -1 osd.8 106 *** Got signal Terminated *** 2024-03-04T09:43:09.401 INFO:journalctl@ceph.osd.8.smithi191.stdout:Mar 04 09:43:09 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-8[55524]: 2024-03-04T09:43:09.116+0000 7f4371991700 -1 osd.8 106 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-04T09:43:09.401 INFO:journalctl@ceph.osd.8.smithi191.stdout:Mar 04 09:43:09 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-8[55524]: 2024-03-04T09:43:09.153+0000 7f436df85700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-04T09:42:17.524704+0000 front 2024-03-04T09:42:17.524866+0000 (oldest deadline 2024-03-04T09:42:41.024775+0000) 2024-03-04T09:43:09.401 INFO:journalctl@ceph.osd.8.smithi191.stdout:Mar 04 09:43:09 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-8[55524]: 2024-03-04T09:43:09.153+0000 7f436df85700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-03-04T09:42:25.125775+0000 front 2024-03-04T09:42:25.125737+0000 (oldest deadline 2024-03-04T09:42:48.025913+0000) 2024-03-04T09:43:09.401 INFO:journalctl@ceph.osd.8.smithi191.stdout:Mar 04 09:43:09 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-8[55524]: 2024-03-04T09:43:09.153+0000 7f436df85700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2024-03-04T09:42:34.328161+0000 front 2024-03-04T09:42:34.328294+0000 (oldest deadline 2024-03-04T09:42:55.428059+0000) 2024-03-04T09:43:09.401 INFO:journalctl@ceph.osd.8.smithi191.stdout:Mar 04 09:43:09 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-8[55524]: 2024-03-04T09:43:09.153+0000 7f436df85700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-03-04T09:42:41.329036+0000 front 2024-03-04T09:42:41.329257+0000 (oldest deadline 2024-03-04T09:43:03.628955+0000) 2024-03-04T09:43:09.836 INFO:journalctl@ceph.osd.9.smithi052.stdout:Mar 04 09:43:09 smithi052 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-9[65396]: 2024-03-04T09:43:09.376+0000 7fd1f1e8e700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-03-04T09:42:09.861821+0000 front 2024-03-04T09:42:09.861761+0000 (oldest deadline 2024-03-04T09:42:32.161859+0000) 2024-03-04T09:43:09.837 INFO:journalctl@ceph.osd.9.smithi052.stdout:Mar 04 09:43:09 smithi052 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-9[65396]: 2024-03-04T09:43:09.376+0000 7fd1f1e8e700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-04T09:42:17.363956+0000 front 2024-03-04T09:42:17.363876+0000 (oldest deadline 2024-03-04T09:42:41.463546+0000) 2024-03-04T09:43:09.837 INFO:journalctl@ceph.osd.9.smithi052.stdout:Mar 04 09:43:09 smithi052 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-9[65396]: 2024-03-04T09:43:09.376+0000 7fd1f1e8e700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-03-04T09:42:21.464200+0000 front 2024-03-04T09:42:21.464139+0000 (oldest deadline 2024-03-04T09:42:47.363942+0000) 2024-03-04T09:43:09.837 INFO:journalctl@ceph.osd.9.smithi052.stdout:Mar 04 09:43:09 smithi052 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-9[65396]: 2024-03-04T09:43:09.376+0000 7fd1f1e8e700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2024-03-04T09:42:33.765329+0000 front 2024-03-04T09:42:33.765393+0000 (oldest deadline 2024-03-04T09:42:59.665478+0000) 2024-03-04T09:43:09.837 INFO:journalctl@ceph.osd.9.smithi052.stdout:Mar 04 09:43:09 smithi052 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-9[65396]: 2024-03-04T09:43:09.376+0000 7fd1f1e8e700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-03-04T09:42:39.666192+0000 front 2024-03-04T09:42:39.666267+0000 (oldest deadline 2024-03-04T09:43:04.966101+0000) 2024-03-04T09:43:09.837 INFO:journalctl@ceph.osd.11.smithi191.stdout:Mar 04 09:43:09 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-11[62160]: 2024-03-04T09:43:09.402+0000 7fdceac97700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-03-04T09:42:08.553313+0000 front 2024-03-04T09:42:08.553176+0000 (oldest deadline 2024-03-04T09:42:32.653011+0000) 2024-03-04T09:43:09.838 INFO:journalctl@ceph.osd.11.smithi191.stdout:Mar 04 09:43:09 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-11[62160]: 2024-03-04T09:43:09.402+0000 7fdceac97700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-04T09:42:18.355200+0000 front 2024-03-04T09:42:18.355189+0000 (oldest deadline 2024-03-04T09:42:44.254936+0000) 2024-03-04T09:43:09.838 INFO:journalctl@ceph.osd.11.smithi191.stdout:Mar 04 09:43:09 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-11[62160]: 2024-03-04T09:43:09.402+0000 7fdceac97700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-03-04T09:42:24.255361+0000 front 2024-03-04T09:42:24.255246+0000 (oldest deadline 2024-03-04T09:42:47.155465+0000) 2024-03-04T09:43:09.838 INFO:journalctl@ceph.osd.11.smithi191.stdout:Mar 04 09:43:09 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-11[62160]: 2024-03-04T09:43:09.402+0000 7fdceac97700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-03-04T09:42:41.857812+0000 front 2024-03-04T09:42:41.857710+0000 (oldest deadline 2024-03-04T09:43:03.557748+0000) 2024-03-04T09:43:09.883 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 04 09:43:09 smithi106 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-10[60969]: 2024-03-04T09:43:09.513+0000 7f10f0d39700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-03-04T09:42:08.466029+0000 front 2024-03-04T09:42:08.465963+0000 (oldest deadline 2024-03-04T09:42:32.566054+0000) 2024-03-04T09:43:09.883 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 04 09:43:09 smithi106 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-10[60969]: 2024-03-04T09:43:09.513+0000 7f10f0d39700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-04T09:42:15.467191+0000 front 2024-03-04T09:42:15.467152+0000 (oldest deadline 2024-03-04T09:42:39.567351+0000) 2024-03-04T09:43:09.883 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 04 09:43:09 smithi106 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-10[60969]: 2024-03-04T09:43:09.513+0000 7f10f0d39700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-03-04T09:42:25.368854+0000 front 2024-03-04T09:42:25.368833+0000 (oldest deadline 2024-03-04T09:42:48.868990+0000) 2024-03-04T09:43:09.883 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 04 09:43:09 smithi106 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-10[60969]: 2024-03-04T09:43:09.513+0000 7f10f0d39700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2024-03-04T09:42:28.869679+0000 front 2024-03-04T09:42:28.869692+0000 (oldest deadline 2024-03-04T09:42:55.270190+0000) 2024-03-04T09:43:09.884 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 04 09:43:09 smithi106 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-10[60969]: 2024-03-04T09:43:09.513+0000 7f10f0d39700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-03-04T09:42:42.271376+0000 front 2024-03-04T09:42:42.271331+0000 (oldest deadline 2024-03-04T09:43:07.571444+0000) 2024-03-04T09:43:10.588 INFO:journalctl@ceph.osd.8.smithi191.stdout:Mar 04 09:43:10 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-8[55524]: 2024-03-04T09:43:10.158+0000 7f436df85700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-04T09:42:17.524704+0000 front 2024-03-04T09:42:17.524866+0000 (oldest deadline 2024-03-04T09:42:41.024775+0000) 2024-03-04T09:43:10.588 INFO:journalctl@ceph.osd.8.smithi191.stdout:Mar 04 09:43:10 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-8[55524]: 2024-03-04T09:43:10.158+0000 7f436df85700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-03-04T09:42:25.125775+0000 front 2024-03-04T09:42:25.125737+0000 (oldest deadline 2024-03-04T09:42:48.025913+0000) 2024-03-04T09:43:10.588 INFO:journalctl@ceph.osd.8.smithi191.stdout:Mar 04 09:43:10 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-8[55524]: 2024-03-04T09:43:10.158+0000 7f436df85700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2024-03-04T09:42:34.328161+0000 front 2024-03-04T09:42:34.328294+0000 (oldest deadline 2024-03-04T09:42:55.428059+0000) 2024-03-04T09:43:10.589 INFO:journalctl@ceph.osd.8.smithi191.stdout:Mar 04 09:43:10 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-8[55524]: 2024-03-04T09:43:10.158+0000 7f436df85700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-03-04T09:42:41.329036+0000 front 2024-03-04T09:42:41.329257+0000 (oldest deadline 2024-03-04T09:43:03.628955+0000) 2024-03-04T09:43:10.589 INFO:journalctl@ceph.osd.11.smithi191.stdout:Mar 04 09:43:10 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-11[62160]: 2024-03-04T09:43:10.406+0000 7fdceac97700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-03-04T09:42:08.553313+0000 front 2024-03-04T09:42:08.553176+0000 (oldest deadline 2024-03-04T09:42:32.653011+0000) 2024-03-04T09:43:10.589 INFO:journalctl@ceph.osd.11.smithi191.stdout:Mar 04 09:43:10 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-11[62160]: 2024-03-04T09:43:10.406+0000 7fdceac97700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-04T09:42:18.355200+0000 front 2024-03-04T09:42:18.355189+0000 (oldest deadline 2024-03-04T09:42:44.254936+0000) 2024-03-04T09:43:10.589 INFO:journalctl@ceph.osd.11.smithi191.stdout:Mar 04 09:43:10 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-11[62160]: 2024-03-04T09:43:10.406+0000 7fdceac97700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-03-04T09:42:24.255361+0000 front 2024-03-04T09:42:24.255246+0000 (oldest deadline 2024-03-04T09:42:47.155465+0000) 2024-03-04T09:43:10.589 INFO:journalctl@ceph.osd.11.smithi191.stdout:Mar 04 09:43:10 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-11[62160]: 2024-03-04T09:43:10.406+0000 7fdceac97700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-03-04T09:42:41.857812+0000 front 2024-03-04T09:42:41.857710+0000 (oldest deadline 2024-03-04T09:43:03.557748+0000) 2024-03-04T09:43:10.836 INFO:journalctl@ceph.osd.9.smithi052.stdout:Mar 04 09:43:10 smithi052 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-9[65396]: 2024-03-04T09:43:10.374+0000 7fd1f1e8e700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-03-04T09:42:09.861821+0000 front 2024-03-04T09:42:09.861761+0000 (oldest deadline 2024-03-04T09:42:32.161859+0000) 2024-03-04T09:43:10.836 INFO:journalctl@ceph.osd.9.smithi052.stdout:Mar 04 09:43:10 smithi052 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-9[65396]: 2024-03-04T09:43:10.374+0000 7fd1f1e8e700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-04T09:42:17.363956+0000 front 2024-03-04T09:42:17.363876+0000 (oldest deadline 2024-03-04T09:42:41.463546+0000) 2024-03-04T09:43:10.837 INFO:journalctl@ceph.osd.9.smithi052.stdout:Mar 04 09:43:10 smithi052 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-9[65396]: 2024-03-04T09:43:10.374+0000 7fd1f1e8e700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-03-04T09:42:21.464200+0000 front 2024-03-04T09:42:21.464139+0000 (oldest deadline 2024-03-04T09:42:47.363942+0000) 2024-03-04T09:43:10.837 INFO:journalctl@ceph.osd.9.smithi052.stdout:Mar 04 09:43:10 smithi052 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-9[65396]: 2024-03-04T09:43:10.374+0000 7fd1f1e8e700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2024-03-04T09:42:33.765329+0000 front 2024-03-04T09:42:33.765393+0000 (oldest deadline 2024-03-04T09:42:59.665478+0000) 2024-03-04T09:43:10.837 INFO:journalctl@ceph.osd.9.smithi052.stdout:Mar 04 09:43:10 smithi052 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-9[65396]: 2024-03-04T09:43:10.374+0000 7fd1f1e8e700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-03-04T09:42:39.666192+0000 front 2024-03-04T09:42:39.666267+0000 (oldest deadline 2024-03-04T09:43:04.966101+0000) 2024-03-04T09:43:10.883 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 04 09:43:10 smithi106 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-10[60969]: 2024-03-04T09:43:10.492+0000 7f10f0d39700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-03-04T09:42:08.466029+0000 front 2024-03-04T09:42:08.465963+0000 (oldest deadline 2024-03-04T09:42:32.566054+0000) 2024-03-04T09:43:10.883 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 04 09:43:10 smithi106 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-10[60969]: 2024-03-04T09:43:10.492+0000 7f10f0d39700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-04T09:42:15.467191+0000 front 2024-03-04T09:42:15.467152+0000 (oldest deadline 2024-03-04T09:42:39.567351+0000) 2024-03-04T09:43:10.883 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 04 09:43:10 smithi106 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-10[60969]: 2024-03-04T09:43:10.492+0000 7f10f0d39700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-03-04T09:42:25.368854+0000 front 2024-03-04T09:42:25.368833+0000 (oldest deadline 2024-03-04T09:42:48.868990+0000) 2024-03-04T09:43:10.883 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 04 09:43:10 smithi106 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-10[60969]: 2024-03-04T09:43:10.492+0000 7f10f0d39700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2024-03-04T09:42:28.869679+0000 front 2024-03-04T09:42:28.869692+0000 (oldest deadline 2024-03-04T09:42:55.270190+0000) 2024-03-04T09:43:10.883 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 04 09:43:10 smithi106 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-10[60969]: 2024-03-04T09:43:10.492+0000 7f10f0d39700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-03-04T09:42:42.271376+0000 front 2024-03-04T09:42:42.271331+0000 (oldest deadline 2024-03-04T09:43:07.571444+0000) 2024-03-04T09:43:11.410 INFO:journalctl@ceph.osd.8.smithi191.stdout:Mar 04 09:43:11 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-8[55524]: 2024-03-04T09:43:11.109+0000 7f436df85700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-04T09:42:17.524704+0000 front 2024-03-04T09:42:17.524866+0000 (oldest deadline 2024-03-04T09:42:41.024775+0000) 2024-03-04T09:43:11.410 INFO:journalctl@ceph.osd.8.smithi191.stdout:Mar 04 09:43:11 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-8[55524]: 2024-03-04T09:43:11.109+0000 7f436df85700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-03-04T09:42:25.125775+0000 front 2024-03-04T09:42:25.125737+0000 (oldest deadline 2024-03-04T09:42:48.025913+0000) 2024-03-04T09:43:11.411 INFO:journalctl@ceph.osd.8.smithi191.stdout:Mar 04 09:43:11 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-8[55524]: 2024-03-04T09:43:11.109+0000 7f436df85700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2024-03-04T09:42:34.328161+0000 front 2024-03-04T09:42:34.328294+0000 (oldest deadline 2024-03-04T09:42:55.428059+0000) 2024-03-04T09:43:11.411 INFO:journalctl@ceph.osd.8.smithi191.stdout:Mar 04 09:43:11 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-8[55524]: 2024-03-04T09:43:11.109+0000 7f436df85700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-03-04T09:42:41.329036+0000 front 2024-03-04T09:42:41.329257+0000 (oldest deadline 2024-03-04T09:43:03.628955+0000) 2024-03-04T09:43:11.737 INFO:journalctl@ceph.osd.9.smithi052.stdout:Mar 04 09:43:11 smithi052 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-9[65396]: 2024-03-04T09:43:11.395+0000 7fd1f1e8e700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-03-04T09:42:09.861821+0000 front 2024-03-04T09:42:09.861761+0000 (oldest deadline 2024-03-04T09:42:32.161859+0000) 2024-03-04T09:43:11.738 INFO:journalctl@ceph.osd.9.smithi052.stdout:Mar 04 09:43:11 smithi052 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-9[65396]: 2024-03-04T09:43:11.395+0000 7fd1f1e8e700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-04T09:42:17.363956+0000 front 2024-03-04T09:42:17.363876+0000 (oldest deadline 2024-03-04T09:42:41.463546+0000) 2024-03-04T09:43:11.738 INFO:journalctl@ceph.osd.9.smithi052.stdout:Mar 04 09:43:11 smithi052 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-9[65396]: 2024-03-04T09:43:11.395+0000 7fd1f1e8e700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-03-04T09:42:21.464200+0000 front 2024-03-04T09:42:21.464139+0000 (oldest deadline 2024-03-04T09:42:47.363942+0000) 2024-03-04T09:43:11.738 INFO:journalctl@ceph.osd.9.smithi052.stdout:Mar 04 09:43:11 smithi052 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-9[65396]: 2024-03-04T09:43:11.395+0000 7fd1f1e8e700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2024-03-04T09:42:33.765329+0000 front 2024-03-04T09:42:33.765393+0000 (oldest deadline 2024-03-04T09:42:59.665478+0000) 2024-03-04T09:43:11.738 INFO:journalctl@ceph.osd.9.smithi052.stdout:Mar 04 09:43:11 smithi052 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-9[65396]: 2024-03-04T09:43:11.395+0000 7fd1f1e8e700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-03-04T09:42:39.666192+0000 front 2024-03-04T09:42:39.666267+0000 (oldest deadline 2024-03-04T09:43:04.966101+0000) 2024-03-04T09:43:11.838 INFO:journalctl@ceph.osd.11.smithi191.stdout:Mar 04 09:43:11 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-11[62160]: 2024-03-04T09:43:11.412+0000 7fdceac97700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-03-04T09:42:08.553313+0000 front 2024-03-04T09:42:08.553176+0000 (oldest deadline 2024-03-04T09:42:32.653011+0000) 2024-03-04T09:43:11.838 INFO:journalctl@ceph.osd.11.smithi191.stdout:Mar 04 09:43:11 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-11[62160]: 2024-03-04T09:43:11.412+0000 7fdceac97700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-04T09:42:18.355200+0000 front 2024-03-04T09:42:18.355189+0000 (oldest deadline 2024-03-04T09:42:44.254936+0000) 2024-03-04T09:43:11.838 INFO:journalctl@ceph.osd.11.smithi191.stdout:Mar 04 09:43:11 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-11[62160]: 2024-03-04T09:43:11.412+0000 7fdceac97700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-03-04T09:42:24.255361+0000 front 2024-03-04T09:42:24.255246+0000 (oldest deadline 2024-03-04T09:42:47.155465+0000) 2024-03-04T09:43:11.838 INFO:journalctl@ceph.osd.11.smithi191.stdout:Mar 04 09:43:11 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-11[62160]: 2024-03-04T09:43:11.412+0000 7fdceac97700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-03-04T09:42:41.857812+0000 front 2024-03-04T09:42:41.857710+0000 (oldest deadline 2024-03-04T09:43:03.557748+0000) 2024-03-04T09:43:11.882 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 04 09:43:11 smithi106 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-10[60969]: 2024-03-04T09:43:11.524+0000 7f10f0d39700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-03-04T09:42:08.466029+0000 front 2024-03-04T09:42:08.465963+0000 (oldest deadline 2024-03-04T09:42:32.566054+0000) 2024-03-04T09:43:11.883 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 04 09:43:11 smithi106 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-10[60969]: 2024-03-04T09:43:11.524+0000 7f10f0d39700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-04T09:42:15.467191+0000 front 2024-03-04T09:42:15.467152+0000 (oldest deadline 2024-03-04T09:42:39.567351+0000) 2024-03-04T09:43:11.883 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 04 09:43:11 smithi106 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-10[60969]: 2024-03-04T09:43:11.524+0000 7f10f0d39700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-03-04T09:42:25.368854+0000 front 2024-03-04T09:42:25.368833+0000 (oldest deadline 2024-03-04T09:42:48.868990+0000) 2024-03-04T09:43:11.883 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 04 09:43:11 smithi106 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-10[60969]: 2024-03-04T09:43:11.524+0000 7f10f0d39700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2024-03-04T09:42:28.869679+0000 front 2024-03-04T09:42:28.869692+0000 (oldest deadline 2024-03-04T09:42:55.270190+0000) 2024-03-04T09:43:11.883 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 04 09:43:11 smithi106 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-10[60969]: 2024-03-04T09:43:11.524+0000 7f10f0d39700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-03-04T09:42:42.271376+0000 front 2024-03-04T09:42:42.271331+0000 (oldest deadline 2024-03-04T09:43:07.571444+0000) 2024-03-04T09:43:11.883 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 04 09:43:11 smithi106 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-10[60969]: 2024-03-04T09:43:11.524+0000 7f10f0d39700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.191:6812 osd.5 since back 2024-03-04T09:42:49.272525+0000 front 2024-03-04T09:42:49.272394+0000 (oldest deadline 2024-03-04T09:43:10.872863+0000) 2024-03-04T09:43:12.386 INFO:journalctl@ceph.osd.8.smithi191.stdout:Mar 04 09:43:12 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-8[55524]: 2024-03-04T09:43:12.111+0000 7f436df85700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-04T09:42:17.524704+0000 front 2024-03-04T09:42:17.524866+0000 (oldest deadline 2024-03-04T09:42:41.024775+0000) 2024-03-04T09:43:12.386 INFO:journalctl@ceph.osd.8.smithi191.stdout:Mar 04 09:43:12 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-8[55524]: 2024-03-04T09:43:12.111+0000 7f436df85700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-03-04T09:42:25.125775+0000 front 2024-03-04T09:42:25.125737+0000 (oldest deadline 2024-03-04T09:42:48.025913+0000) 2024-03-04T09:43:12.386 INFO:journalctl@ceph.osd.8.smithi191.stdout:Mar 04 09:43:12 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-8[55524]: 2024-03-04T09:43:12.111+0000 7f436df85700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2024-03-04T09:42:34.328161+0000 front 2024-03-04T09:42:34.328294+0000 (oldest deadline 2024-03-04T09:42:55.428059+0000) 2024-03-04T09:43:12.386 INFO:journalctl@ceph.osd.8.smithi191.stdout:Mar 04 09:43:12 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-8[55524]: 2024-03-04T09:43:12.111+0000 7f436df85700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-03-04T09:42:41.329036+0000 front 2024-03-04T09:42:41.329257+0000 (oldest deadline 2024-03-04T09:43:03.628955+0000) 2024-03-04T09:43:12.386 INFO:journalctl@ceph.osd.8.smithi191.stdout:Mar 04 09:43:12 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-8[55524]: 2024-03-04T09:43:12.111+0000 7f436df85700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.191:6812 osd.5 since back 2024-03-04T09:42:45.929893+0000 front 2024-03-04T09:42:45.929916+0000 (oldest deadline 2024-03-04T09:43:11.830000+0000) 2024-03-04T09:43:12.750 INFO:journalctl@ceph.osd.9.smithi052.stdout:Mar 04 09:43:12 smithi052 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-9[65396]: 2024-03-04T09:43:12.443+0000 7fd1f1e8e700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-03-04T09:42:09.861821+0000 front 2024-03-04T09:42:09.861761+0000 (oldest deadline 2024-03-04T09:42:32.161859+0000) 2024-03-04T09:43:12.750 INFO:journalctl@ceph.osd.9.smithi052.stdout:Mar 04 09:43:12 smithi052 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-9[65396]: 2024-03-04T09:43:12.443+0000 7fd1f1e8e700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-04T09:42:17.363956+0000 front 2024-03-04T09:42:17.363876+0000 (oldest deadline 2024-03-04T09:42:41.463546+0000) 2024-03-04T09:43:12.750 INFO:journalctl@ceph.osd.9.smithi052.stdout:Mar 04 09:43:12 smithi052 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-9[65396]: 2024-03-04T09:43:12.443+0000 7fd1f1e8e700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-03-04T09:42:21.464200+0000 front 2024-03-04T09:42:21.464139+0000 (oldest deadline 2024-03-04T09:42:47.363942+0000) 2024-03-04T09:43:12.750 INFO:journalctl@ceph.osd.9.smithi052.stdout:Mar 04 09:43:12 smithi052 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-9[65396]: 2024-03-04T09:43:12.443+0000 7fd1f1e8e700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2024-03-04T09:42:33.765329+0000 front 2024-03-04T09:42:33.765393+0000 (oldest deadline 2024-03-04T09:42:59.665478+0000) 2024-03-04T09:43:12.750 INFO:journalctl@ceph.osd.9.smithi052.stdout:Mar 04 09:43:12 smithi052 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-9[65396]: 2024-03-04T09:43:12.443+0000 7fd1f1e8e700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-03-04T09:42:39.666192+0000 front 2024-03-04T09:42:39.666267+0000 (oldest deadline 2024-03-04T09:43:04.966101+0000) 2024-03-04T09:43:12.751 INFO:journalctl@ceph.osd.9.smithi052.stdout:Mar 04 09:43:12 smithi052 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-9[65396]: 2024-03-04T09:43:12.443+0000 7fd1f1e8e700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.191:6812 osd.5 since back 2024-03-04T09:42:47.167809+0000 front 2024-03-04T09:42:47.167792+0000 (oldest deadline 2024-03-04T09:43:12.368082+0000) 2024-03-04T09:43:12.838 INFO:journalctl@ceph.osd.11.smithi191.stdout:Mar 04 09:43:12 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-11[62160]: 2024-03-04T09:43:12.387+0000 7fdceac97700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-03-04T09:42:08.553313+0000 front 2024-03-04T09:42:08.553176+0000 (oldest deadline 2024-03-04T09:42:32.653011+0000) 2024-03-04T09:43:12.838 INFO:journalctl@ceph.osd.11.smithi191.stdout:Mar 04 09:43:12 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-11[62160]: 2024-03-04T09:43:12.387+0000 7fdceac97700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-04T09:42:18.355200+0000 front 2024-03-04T09:42:18.355189+0000 (oldest deadline 2024-03-04T09:42:44.254936+0000) 2024-03-04T09:43:12.838 INFO:journalctl@ceph.osd.11.smithi191.stdout:Mar 04 09:43:12 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-11[62160]: 2024-03-04T09:43:12.387+0000 7fdceac97700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-03-04T09:42:24.255361+0000 front 2024-03-04T09:42:24.255246+0000 (oldest deadline 2024-03-04T09:42:47.155465+0000) 2024-03-04T09:43:12.838 INFO:journalctl@ceph.osd.11.smithi191.stdout:Mar 04 09:43:12 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-11[62160]: 2024-03-04T09:43:12.387+0000 7fdceac97700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-03-04T09:42:41.857812+0000 front 2024-03-04T09:42:41.857710+0000 (oldest deadline 2024-03-04T09:43:03.557748+0000) 2024-03-04T09:43:12.882 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 04 09:43:12 smithi106 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-10[60969]: 2024-03-04T09:43:12.506+0000 7f10f0d39700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-03-04T09:42:08.466029+0000 front 2024-03-04T09:42:08.465963+0000 (oldest deadline 2024-03-04T09:42:32.566054+0000) 2024-03-04T09:43:12.883 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 04 09:43:12 smithi106 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-10[60969]: 2024-03-04T09:43:12.506+0000 7f10f0d39700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-04T09:42:15.467191+0000 front 2024-03-04T09:42:15.467152+0000 (oldest deadline 2024-03-04T09:42:39.567351+0000) 2024-03-04T09:43:12.883 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 04 09:43:12 smithi106 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-10[60969]: 2024-03-04T09:43:12.506+0000 7f10f0d39700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-03-04T09:42:25.368854+0000 front 2024-03-04T09:42:25.368833+0000 (oldest deadline 2024-03-04T09:42:48.868990+0000) 2024-03-04T09:43:12.883 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 04 09:43:12 smithi106 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-10[60969]: 2024-03-04T09:43:12.506+0000 7f10f0d39700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2024-03-04T09:42:28.869679+0000 front 2024-03-04T09:42:28.869692+0000 (oldest deadline 2024-03-04T09:42:55.270190+0000) 2024-03-04T09:43:12.883 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 04 09:43:12 smithi106 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-10[60969]: 2024-03-04T09:43:12.506+0000 7f10f0d39700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-03-04T09:42:42.271376+0000 front 2024-03-04T09:42:42.271331+0000 (oldest deadline 2024-03-04T09:43:07.571444+0000) 2024-03-04T09:43:12.883 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 04 09:43:12 smithi106 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-10[60969]: 2024-03-04T09:43:12.506+0000 7f10f0d39700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.191:6812 osd.5 since back 2024-03-04T09:42:49.272525+0000 front 2024-03-04T09:42:49.272394+0000 (oldest deadline 2024-03-04T09:43:10.872863+0000) 2024-03-04T09:43:13.338 INFO:journalctl@ceph.osd.8.smithi191.stdout:Mar 04 09:43:13 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-8[55524]: 2024-03-04T09:43:13.067+0000 7f436df85700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-04T09:42:17.524704+0000 front 2024-03-04T09:42:17.524866+0000 (oldest deadline 2024-03-04T09:42:41.024775+0000) 2024-03-04T09:43:13.338 INFO:journalctl@ceph.osd.8.smithi191.stdout:Mar 04 09:43:13 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-8[55524]: 2024-03-04T09:43:13.067+0000 7f436df85700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-03-04T09:42:25.125775+0000 front 2024-03-04T09:42:25.125737+0000 (oldest deadline 2024-03-04T09:42:48.025913+0000) 2024-03-04T09:43:13.338 INFO:journalctl@ceph.osd.8.smithi191.stdout:Mar 04 09:43:13 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-8[55524]: 2024-03-04T09:43:13.067+0000 7f436df85700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2024-03-04T09:42:34.328161+0000 front 2024-03-04T09:42:34.328294+0000 (oldest deadline 2024-03-04T09:42:55.428059+0000) 2024-03-04T09:43:13.338 INFO:journalctl@ceph.osd.8.smithi191.stdout:Mar 04 09:43:13 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-8[55524]: 2024-03-04T09:43:13.067+0000 7f436df85700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-03-04T09:42:41.329036+0000 front 2024-03-04T09:42:41.329257+0000 (oldest deadline 2024-03-04T09:43:03.628955+0000) 2024-03-04T09:43:13.338 INFO:journalctl@ceph.osd.8.smithi191.stdout:Mar 04 09:43:13 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-8[55524]: 2024-03-04T09:43:13.067+0000 7f436df85700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.191:6812 osd.5 since back 2024-03-04T09:42:45.929893+0000 front 2024-03-04T09:42:45.929916+0000 (oldest deadline 2024-03-04T09:43:11.830000+0000) 2024-03-04T09:43:13.836 INFO:journalctl@ceph.osd.9.smithi052.stdout:Mar 04 09:43:13 smithi052 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-9[65396]: 2024-03-04T09:43:13.401+0000 7fd1f1e8e700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-03-04T09:42:09.861821+0000 front 2024-03-04T09:42:09.861761+0000 (oldest deadline 2024-03-04T09:42:32.161859+0000) 2024-03-04T09:43:13.836 INFO:journalctl@ceph.osd.9.smithi052.stdout:Mar 04 09:43:13 smithi052 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-9[65396]: 2024-03-04T09:43:13.401+0000 7fd1f1e8e700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-04T09:42:17.363956+0000 front 2024-03-04T09:42:17.363876+0000 (oldest deadline 2024-03-04T09:42:41.463546+0000) 2024-03-04T09:43:13.837 INFO:journalctl@ceph.osd.9.smithi052.stdout:Mar 04 09:43:13 smithi052 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-9[65396]: 2024-03-04T09:43:13.401+0000 7fd1f1e8e700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-03-04T09:42:21.464200+0000 front 2024-03-04T09:42:21.464139+0000 (oldest deadline 2024-03-04T09:42:47.363942+0000) 2024-03-04T09:43:13.837 INFO:journalctl@ceph.osd.9.smithi052.stdout:Mar 04 09:43:13 smithi052 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-9[65396]: 2024-03-04T09:43:13.401+0000 7fd1f1e8e700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2024-03-04T09:42:33.765329+0000 front 2024-03-04T09:42:33.765393+0000 (oldest deadline 2024-03-04T09:42:59.665478+0000) 2024-03-04T09:43:13.837 INFO:journalctl@ceph.osd.9.smithi052.stdout:Mar 04 09:43:13 smithi052 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-9[65396]: 2024-03-04T09:43:13.401+0000 7fd1f1e8e700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-03-04T09:42:39.666192+0000 front 2024-03-04T09:42:39.666267+0000 (oldest deadline 2024-03-04T09:43:04.966101+0000) 2024-03-04T09:43:13.837 INFO:journalctl@ceph.osd.9.smithi052.stdout:Mar 04 09:43:13 smithi052 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-9[65396]: 2024-03-04T09:43:13.401+0000 7fd1f1e8e700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.191:6812 osd.5 since back 2024-03-04T09:42:47.167809+0000 front 2024-03-04T09:42:47.167792+0000 (oldest deadline 2024-03-04T09:43:12.368082+0000) 2024-03-04T09:43:13.837 INFO:journalctl@ceph.osd.11.smithi191.stdout:Mar 04 09:43:13 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-11[62160]: 2024-03-04T09:43:13.348+0000 7fdceac97700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-03-04T09:42:08.553313+0000 front 2024-03-04T09:42:08.553176+0000 (oldest deadline 2024-03-04T09:42:32.653011+0000) 2024-03-04T09:43:13.838 INFO:journalctl@ceph.osd.11.smithi191.stdout:Mar 04 09:43:13 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-11[62160]: 2024-03-04T09:43:13.348+0000 7fdceac97700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-04T09:42:18.355200+0000 front 2024-03-04T09:42:18.355189+0000 (oldest deadline 2024-03-04T09:42:44.254936+0000) 2024-03-04T09:43:13.838 INFO:journalctl@ceph.osd.11.smithi191.stdout:Mar 04 09:43:13 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-11[62160]: 2024-03-04T09:43:13.348+0000 7fdceac97700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-03-04T09:42:24.255361+0000 front 2024-03-04T09:42:24.255246+0000 (oldest deadline 2024-03-04T09:42:47.155465+0000) 2024-03-04T09:43:13.838 INFO:journalctl@ceph.osd.11.smithi191.stdout:Mar 04 09:43:13 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-11[62160]: 2024-03-04T09:43:13.348+0000 7fdceac97700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-03-04T09:42:41.857812+0000 front 2024-03-04T09:42:41.857710+0000 (oldest deadline 2024-03-04T09:43:03.557748+0000) 2024-03-04T09:43:13.838 INFO:journalctl@ceph.osd.11.smithi191.stdout:Mar 04 09:43:13 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-11[62160]: 2024-03-04T09:43:13.348+0000 7fdceac97700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.191:6812 osd.5 since back 2024-03-04T09:42:46.959192+0000 front 2024-03-04T09:42:46.959105+0000 (oldest deadline 2024-03-04T09:43:12.859230+0000) 2024-03-04T09:43:13.883 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 04 09:43:13 smithi106 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-10[60969]: 2024-03-04T09:43:13.541+0000 7f10f0d39700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-03-04T09:42:08.466029+0000 front 2024-03-04T09:42:08.465963+0000 (oldest deadline 2024-03-04T09:42:32.566054+0000) 2024-03-04T09:43:13.883 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 04 09:43:13 smithi106 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-10[60969]: 2024-03-04T09:43:13.541+0000 7f10f0d39700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-04T09:42:15.467191+0000 front 2024-03-04T09:42:15.467152+0000 (oldest deadline 2024-03-04T09:42:39.567351+0000) 2024-03-04T09:43:13.883 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 04 09:43:13 smithi106 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-10[60969]: 2024-03-04T09:43:13.541+0000 7f10f0d39700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-03-04T09:42:25.368854+0000 front 2024-03-04T09:42:25.368833+0000 (oldest deadline 2024-03-04T09:42:48.868990+0000) 2024-03-04T09:43:13.883 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 04 09:43:13 smithi106 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-10[60969]: 2024-03-04T09:43:13.541+0000 7f10f0d39700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2024-03-04T09:42:28.869679+0000 front 2024-03-04T09:42:28.869692+0000 (oldest deadline 2024-03-04T09:42:55.270190+0000) 2024-03-04T09:43:13.883 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 04 09:43:13 smithi106 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-10[60969]: 2024-03-04T09:43:13.541+0000 7f10f0d39700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-03-04T09:42:42.271376+0000 front 2024-03-04T09:42:42.271331+0000 (oldest deadline 2024-03-04T09:43:07.571444+0000) 2024-03-04T09:43:13.883 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 04 09:43:13 smithi106 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-10[60969]: 2024-03-04T09:43:13.541+0000 7f10f0d39700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.191:6812 osd.5 since back 2024-03-04T09:42:49.272525+0000 front 2024-03-04T09:42:49.272394+0000 (oldest deadline 2024-03-04T09:43:10.872863+0000) 2024-03-04T09:43:14.325 INFO:journalctl@ceph.osd.8.smithi191.stdout:Mar 04 09:43:14 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-8[55524]: 2024-03-04T09:43:14.020+0000 7f436df85700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-04T09:42:17.524704+0000 front 2024-03-04T09:42:17.524866+0000 (oldest deadline 2024-03-04T09:42:41.024775+0000) 2024-03-04T09:43:14.325 INFO:journalctl@ceph.osd.8.smithi191.stdout:Mar 04 09:43:14 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-8[55524]: 2024-03-04T09:43:14.020+0000 7f436df85700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-03-04T09:42:25.125775+0000 front 2024-03-04T09:42:25.125737+0000 (oldest deadline 2024-03-04T09:42:48.025913+0000) 2024-03-04T09:43:14.325 INFO:journalctl@ceph.osd.8.smithi191.stdout:Mar 04 09:43:14 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-8[55524]: 2024-03-04T09:43:14.020+0000 7f436df85700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2024-03-04T09:42:34.328161+0000 front 2024-03-04T09:42:34.328294+0000 (oldest deadline 2024-03-04T09:42:55.428059+0000) 2024-03-04T09:43:14.325 INFO:journalctl@ceph.osd.8.smithi191.stdout:Mar 04 09:43:14 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-8[55524]: 2024-03-04T09:43:14.020+0000 7f436df85700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-03-04T09:42:41.329036+0000 front 2024-03-04T09:42:41.329257+0000 (oldest deadline 2024-03-04T09:43:03.628955+0000) 2024-03-04T09:43:14.325 INFO:journalctl@ceph.osd.8.smithi191.stdout:Mar 04 09:43:14 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-8[55524]: 2024-03-04T09:43:14.020+0000 7f436df85700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.191:6812 osd.5 since back 2024-03-04T09:42:45.929893+0000 front 2024-03-04T09:42:45.929916+0000 (oldest deadline 2024-03-04T09:43:11.830000+0000) 2024-03-04T09:43:14.588 INFO:journalctl@ceph.osd.8.smithi191.stdout:Mar 04 09:43:14 smithi191 podman[73611]: 2024-03-04 09:43:14.468583594 +0000 UTC m=+5.470374145 container died 34223926f7eea1b7d317e41f2a6f0590498d51237de4d7a33470246ad84cf38f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5, name=ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-8, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240222, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, RELEASE=wip-lusov-qdb-1bc250c, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-04T09:43:14.589 INFO:journalctl@ceph.osd.11.smithi191.stdout:Mar 04 09:43:14 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-11[62160]: 2024-03-04T09:43:14.326+0000 7fdceac97700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-03-04T09:42:08.553313+0000 front 2024-03-04T09:42:08.553176+0000 (oldest deadline 2024-03-04T09:42:32.653011+0000) 2024-03-04T09:43:14.589 INFO:journalctl@ceph.osd.11.smithi191.stdout:Mar 04 09:43:14 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-11[62160]: 2024-03-04T09:43:14.326+0000 7fdceac97700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-04T09:42:18.355200+0000 front 2024-03-04T09:42:18.355189+0000 (oldest deadline 2024-03-04T09:42:44.254936+0000) 2024-03-04T09:43:14.589 INFO:journalctl@ceph.osd.11.smithi191.stdout:Mar 04 09:43:14 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-11[62160]: 2024-03-04T09:43:14.326+0000 7fdceac97700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-03-04T09:42:24.255361+0000 front 2024-03-04T09:42:24.255246+0000 (oldest deadline 2024-03-04T09:42:47.155465+0000) 2024-03-04T09:43:14.589 INFO:journalctl@ceph.osd.11.smithi191.stdout:Mar 04 09:43:14 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-11[62160]: 2024-03-04T09:43:14.326+0000 7fdceac97700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-03-04T09:42:41.857812+0000 front 2024-03-04T09:42:41.857710+0000 (oldest deadline 2024-03-04T09:43:03.557748+0000) 2024-03-04T09:43:14.589 INFO:journalctl@ceph.osd.11.smithi191.stdout:Mar 04 09:43:14 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-11[62160]: 2024-03-04T09:43:14.326+0000 7fdceac97700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.191:6812 osd.5 since back 2024-03-04T09:42:46.959192+0000 front 2024-03-04T09:42:46.959105+0000 (oldest deadline 2024-03-04T09:43:12.859230+0000) 2024-03-04T09:43:14.836 INFO:journalctl@ceph.osd.9.smithi052.stdout:Mar 04 09:43:14 smithi052 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-9[65396]: 2024-03-04T09:43:14.407+0000 7fd1f1e8e700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-03-04T09:42:09.861821+0000 front 2024-03-04T09:42:09.861761+0000 (oldest deadline 2024-03-04T09:42:32.161859+0000) 2024-03-04T09:43:14.836 INFO:journalctl@ceph.osd.9.smithi052.stdout:Mar 04 09:43:14 smithi052 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-9[65396]: 2024-03-04T09:43:14.407+0000 7fd1f1e8e700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-04T09:42:17.363956+0000 front 2024-03-04T09:42:17.363876+0000 (oldest deadline 2024-03-04T09:42:41.463546+0000) 2024-03-04T09:43:14.836 INFO:journalctl@ceph.osd.9.smithi052.stdout:Mar 04 09:43:14 smithi052 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-9[65396]: 2024-03-04T09:43:14.407+0000 7fd1f1e8e700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-03-04T09:42:21.464200+0000 front 2024-03-04T09:42:21.464139+0000 (oldest deadline 2024-03-04T09:42:47.363942+0000) 2024-03-04T09:43:14.837 INFO:journalctl@ceph.osd.9.smithi052.stdout:Mar 04 09:43:14 smithi052 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-9[65396]: 2024-03-04T09:43:14.407+0000 7fd1f1e8e700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2024-03-04T09:42:33.765329+0000 front 2024-03-04T09:42:33.765393+0000 (oldest deadline 2024-03-04T09:42:59.665478+0000) 2024-03-04T09:43:14.837 INFO:journalctl@ceph.osd.9.smithi052.stdout:Mar 04 09:43:14 smithi052 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-9[65396]: 2024-03-04T09:43:14.407+0000 7fd1f1e8e700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-03-04T09:42:39.666192+0000 front 2024-03-04T09:42:39.666267+0000 (oldest deadline 2024-03-04T09:43:04.966101+0000) 2024-03-04T09:43:14.837 INFO:journalctl@ceph.osd.9.smithi052.stdout:Mar 04 09:43:14 smithi052 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-9[65396]: 2024-03-04T09:43:14.407+0000 7fd1f1e8e700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.191:6812 osd.5 since back 2024-03-04T09:42:47.167809+0000 front 2024-03-04T09:42:47.167792+0000 (oldest deadline 2024-03-04T09:43:12.368082+0000) 2024-03-04T09:43:14.883 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 04 09:43:14 smithi106 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-10[60969]: 2024-03-04T09:43:14.514+0000 7f10f0d39700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-03-04T09:42:08.466029+0000 front 2024-03-04T09:42:08.465963+0000 (oldest deadline 2024-03-04T09:42:32.566054+0000) 2024-03-04T09:43:14.883 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 04 09:43:14 smithi106 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-10[60969]: 2024-03-04T09:43:14.514+0000 7f10f0d39700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-04T09:42:15.467191+0000 front 2024-03-04T09:42:15.467152+0000 (oldest deadline 2024-03-04T09:42:39.567351+0000) 2024-03-04T09:43:14.883 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 04 09:43:14 smithi106 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-10[60969]: 2024-03-04T09:43:14.514+0000 7f10f0d39700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-03-04T09:42:25.368854+0000 front 2024-03-04T09:42:25.368833+0000 (oldest deadline 2024-03-04T09:42:48.868990+0000) 2024-03-04T09:43:14.883 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 04 09:43:14 smithi106 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-10[60969]: 2024-03-04T09:43:14.514+0000 7f10f0d39700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2024-03-04T09:42:28.869679+0000 front 2024-03-04T09:42:28.869692+0000 (oldest deadline 2024-03-04T09:42:55.270190+0000) 2024-03-04T09:43:14.883 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 04 09:43:14 smithi106 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-10[60969]: 2024-03-04T09:43:14.514+0000 7f10f0d39700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-03-04T09:42:42.271376+0000 front 2024-03-04T09:42:42.271331+0000 (oldest deadline 2024-03-04T09:43:07.571444+0000) 2024-03-04T09:43:14.884 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 04 09:43:14 smithi106 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-10[60969]: 2024-03-04T09:43:14.514+0000 7f10f0d39700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.191:6812 osd.5 since back 2024-03-04T09:42:49.272525+0000 front 2024-03-04T09:42:49.272394+0000 (oldest deadline 2024-03-04T09:43:10.872863+0000) 2024-03-04T09:43:15.088 INFO:journalctl@ceph.osd.8.smithi191.stdout:Mar 04 09:43:14 smithi191 podman[73611]: 2024-03-04 09:43:14.858041998 +0000 UTC m=+5.859832529 container cleanup 34223926f7eea1b7d317e41f2a6f0590498d51237de4d7a33470246ad84cf38f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5, name=ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-8, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240222, GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-qdb-1bc250c, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD) 2024-03-04T09:43:15.088 INFO:journalctl@ceph.osd.8.smithi191.stdout:Mar 04 09:43:14 smithi191 bash[73611]: ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-8 2024-03-04T09:43:15.455 INFO:journalctl@ceph.osd.8.smithi191.stdout:Mar 04 09:43:15 smithi191 podman[73628]: 2024-03-04 09:43:15.134318016 +0000 UTC m=+0.665353465 container remove 34223926f7eea1b7d317e41f2a6f0590498d51237de4d7a33470246ad84cf38f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5, name=ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-8, GIT_CLEAN=True, org.label-schema.build-date=20240222, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, RELEASE=wip-lusov-qdb-1bc250c, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-04T09:43:15.455 INFO:journalctl@ceph.osd.11.smithi191.stdout:Mar 04 09:43:15 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-11[62160]: 2024-03-04T09:43:15.347+0000 7fdceac97700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-03-04T09:42:08.553313+0000 front 2024-03-04T09:42:08.553176+0000 (oldest deadline 2024-03-04T09:42:32.653011+0000) 2024-03-04T09:43:15.456 INFO:journalctl@ceph.osd.11.smithi191.stdout:Mar 04 09:43:15 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-11[62160]: 2024-03-04T09:43:15.347+0000 7fdceac97700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-04T09:42:18.355200+0000 front 2024-03-04T09:42:18.355189+0000 (oldest deadline 2024-03-04T09:42:44.254936+0000) 2024-03-04T09:43:15.456 INFO:journalctl@ceph.osd.11.smithi191.stdout:Mar 04 09:43:15 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-11[62160]: 2024-03-04T09:43:15.347+0000 7fdceac97700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-03-04T09:42:24.255361+0000 front 2024-03-04T09:42:24.255246+0000 (oldest deadline 2024-03-04T09:42:47.155465+0000) 2024-03-04T09:43:15.456 INFO:journalctl@ceph.osd.11.smithi191.stdout:Mar 04 09:43:15 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-11[62160]: 2024-03-04T09:43:15.347+0000 7fdceac97700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-03-04T09:42:41.857812+0000 front 2024-03-04T09:42:41.857710+0000 (oldest deadline 2024-03-04T09:43:03.557748+0000) 2024-03-04T09:43:15.456 INFO:journalctl@ceph.osd.11.smithi191.stdout:Mar 04 09:43:15 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-11[62160]: 2024-03-04T09:43:15.347+0000 7fdceac97700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.191:6812 osd.5 since back 2024-03-04T09:42:46.959192+0000 front 2024-03-04T09:42:46.959105+0000 (oldest deadline 2024-03-04T09:43:12.859230+0000) 2024-03-04T09:43:15.718 INFO:journalctl@ceph.osd.8.smithi191.stdout:Mar 04 09:43:15 smithi191 podman[73718]: 2024-03-04 09:43:15.36059567 +0000 UTC m=+0.018545513 image pull f88ffb32aa6a7d5dfdc954d9f096f0c9cc47981d05ec8d117ba859a22d8185ba quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5 2024-03-04T09:43:15.719 INFO:journalctl@ceph.osd.8.smithi191.stdout:Mar 04 09:43:15 smithi191 podman[73718]: 2024-03-04 09:43:15.476686739 +0000 UTC m=+0.134636566 container create d1bd3ce8b1cacf025b915a11a51b547dbe473f0dd570c0a8c9440fb36169d494 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5, name=ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-8-deactivate, RELEASE=wip-lusov-qdb-1bc250c, org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240222, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8) 2024-03-04T09:43:15.719 INFO:journalctl@ceph.osd.8.smithi191.stdout:Mar 04 09:43:15 smithi191 podman[73718]: 2024-03-04 09:43:15.721033315 +0000 UTC m=+0.378983156 container init d1bd3ce8b1cacf025b915a11a51b547dbe473f0dd570c0a8c9440fb36169d494 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5, name=ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-8-deactivate, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240222, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.33.5, RELEASE=wip-lusov-qdb-1bc250c, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-04T09:43:15.836 INFO:journalctl@ceph.osd.9.smithi052.stdout:Mar 04 09:43:15 smithi052 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-9[65396]: 2024-03-04T09:43:15.434+0000 7fd1f1e8e700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-03-04T09:42:09.861821+0000 front 2024-03-04T09:42:09.861761+0000 (oldest deadline 2024-03-04T09:42:32.161859+0000) 2024-03-04T09:43:15.836 INFO:journalctl@ceph.osd.9.smithi052.stdout:Mar 04 09:43:15 smithi052 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-9[65396]: 2024-03-04T09:43:15.434+0000 7fd1f1e8e700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-04T09:42:17.363956+0000 front 2024-03-04T09:42:17.363876+0000 (oldest deadline 2024-03-04T09:42:41.463546+0000) 2024-03-04T09:43:15.836 INFO:journalctl@ceph.osd.9.smithi052.stdout:Mar 04 09:43:15 smithi052 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-9[65396]: 2024-03-04T09:43:15.434+0000 7fd1f1e8e700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-03-04T09:42:21.464200+0000 front 2024-03-04T09:42:21.464139+0000 (oldest deadline 2024-03-04T09:42:47.363942+0000) 2024-03-04T09:43:15.837 INFO:journalctl@ceph.osd.9.smithi052.stdout:Mar 04 09:43:15 smithi052 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-9[65396]: 2024-03-04T09:43:15.434+0000 7fd1f1e8e700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2024-03-04T09:42:33.765329+0000 front 2024-03-04T09:42:33.765393+0000 (oldest deadline 2024-03-04T09:42:59.665478+0000) 2024-03-04T09:43:15.837 INFO:journalctl@ceph.osd.9.smithi052.stdout:Mar 04 09:43:15 smithi052 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-9[65396]: 2024-03-04T09:43:15.434+0000 7fd1f1e8e700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-03-04T09:42:39.666192+0000 front 2024-03-04T09:42:39.666267+0000 (oldest deadline 2024-03-04T09:43:04.966101+0000) 2024-03-04T09:43:15.837 INFO:journalctl@ceph.osd.9.smithi052.stdout:Mar 04 09:43:15 smithi052 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-9[65396]: 2024-03-04T09:43:15.434+0000 7fd1f1e8e700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.191:6812 osd.5 since back 2024-03-04T09:42:47.167809+0000 front 2024-03-04T09:42:47.167792+0000 (oldest deadline 2024-03-04T09:43:12.368082+0000) 2024-03-04T09:43:15.883 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 04 09:43:15 smithi106 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-10[60969]: 2024-03-04T09:43:15.541+0000 7f10f0d39700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-03-04T09:42:08.466029+0000 front 2024-03-04T09:42:08.465963+0000 (oldest deadline 2024-03-04T09:42:32.566054+0000) 2024-03-04T09:43:15.883 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 04 09:43:15 smithi106 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-10[60969]: 2024-03-04T09:43:15.541+0000 7f10f0d39700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-04T09:42:15.467191+0000 front 2024-03-04T09:42:15.467152+0000 (oldest deadline 2024-03-04T09:42:39.567351+0000) 2024-03-04T09:43:15.883 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 04 09:43:15 smithi106 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-10[60969]: 2024-03-04T09:43:15.541+0000 7f10f0d39700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-03-04T09:42:25.368854+0000 front 2024-03-04T09:42:25.368833+0000 (oldest deadline 2024-03-04T09:42:48.868990+0000) 2024-03-04T09:43:15.883 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 04 09:43:15 smithi106 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-10[60969]: 2024-03-04T09:43:15.541+0000 7f10f0d39700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2024-03-04T09:42:28.869679+0000 front 2024-03-04T09:42:28.869692+0000 (oldest deadline 2024-03-04T09:42:55.270190+0000) 2024-03-04T09:43:15.883 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 04 09:43:15 smithi106 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-10[60969]: 2024-03-04T09:43:15.541+0000 7f10f0d39700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-03-04T09:42:42.271376+0000 front 2024-03-04T09:42:42.271331+0000 (oldest deadline 2024-03-04T09:43:07.571444+0000) 2024-03-04T09:43:15.883 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 04 09:43:15 smithi106 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-10[60969]: 2024-03-04T09:43:15.541+0000 7f10f0d39700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.191:6812 osd.5 since back 2024-03-04T09:42:49.272525+0000 front 2024-03-04T09:42:49.272394+0000 (oldest deadline 2024-03-04T09:43:10.872863+0000) 2024-03-04T09:43:16.010 INFO:journalctl@ceph.osd.8.smithi191.stdout:Mar 04 09:43:15 smithi191 podman[73718]: 2024-03-04 09:43:15.72413937 +0000 UTC m=+0.382089203 container start d1bd3ce8b1cacf025b915a11a51b547dbe473f0dd570c0a8c9440fb36169d494 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5, name=ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-8-deactivate, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.build-date=20240222, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, RELEASE=wip-lusov-qdb-1bc250c) 2024-03-04T09:43:16.010 INFO:journalctl@ceph.osd.8.smithi191.stdout:Mar 04 09:43:15 smithi191 podman[73718]: 2024-03-04 09:43:15.812732582 +0000 UTC m=+0.470682425 container attach d1bd3ce8b1cacf025b915a11a51b547dbe473f0dd570c0a8c9440fb36169d494 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5, name=ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-8-deactivate, GIT_CLEAN=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240222, RELEASE=wip-lusov-qdb-1bc250c, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0) 2024-03-04T09:43:16.320 INFO:journalctl@ceph.osd.8.smithi191.stdout:Mar 04 09:43:16 smithi191 podman[73718]: 2024-03-04 09:43:16.013141146 +0000 UTC m=+0.671090966 container died d1bd3ce8b1cacf025b915a11a51b547dbe473f0dd570c0a8c9440fb36169d494 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5, name=ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-8-deactivate, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240222, org.label-schema.vendor=CentOS, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, RELEASE=wip-lusov-qdb-1bc250c, org.label-schema.license=GPLv2) 2024-03-04T09:43:16.580 INFO:journalctl@ceph.osd.11.smithi191.stdout:Mar 04 09:43:16 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-11[62160]: 2024-03-04T09:43:16.321+0000 7fdceac97700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-03-04T09:42:08.553313+0000 front 2024-03-04T09:42:08.553176+0000 (oldest deadline 2024-03-04T09:42:32.653011+0000) 2024-03-04T09:43:16.581 INFO:journalctl@ceph.osd.11.smithi191.stdout:Mar 04 09:43:16 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-11[62160]: 2024-03-04T09:43:16.321+0000 7fdceac97700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-04T09:42:18.355200+0000 front 2024-03-04T09:42:18.355189+0000 (oldest deadline 2024-03-04T09:42:44.254936+0000) 2024-03-04T09:43:16.581 INFO:journalctl@ceph.osd.11.smithi191.stdout:Mar 04 09:43:16 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-11[62160]: 2024-03-04T09:43:16.321+0000 7fdceac97700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-03-04T09:42:24.255361+0000 front 2024-03-04T09:42:24.255246+0000 (oldest deadline 2024-03-04T09:42:47.155465+0000) 2024-03-04T09:43:16.581 INFO:journalctl@ceph.osd.11.smithi191.stdout:Mar 04 09:43:16 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-11[62160]: 2024-03-04T09:43:16.321+0000 7fdceac97700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-03-04T09:42:41.857812+0000 front 2024-03-04T09:42:41.857710+0000 (oldest deadline 2024-03-04T09:43:03.557748+0000) 2024-03-04T09:43:16.581 INFO:journalctl@ceph.osd.11.smithi191.stdout:Mar 04 09:43:16 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-11[62160]: 2024-03-04T09:43:16.321+0000 7fdceac97700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.191:6812 osd.5 since back 2024-03-04T09:42:46.959192+0000 front 2024-03-04T09:42:46.959105+0000 (oldest deadline 2024-03-04T09:43:12.859230+0000) 2024-03-04T09:43:16.599 DEBUG:teuthology.orchestra.run.smithi191:> sudo pkill -f 'journalctl -f -n 0 -u ceph-4e059f08-da08-11ee-95c6-87774f69a715@osd.8.service' 2024-03-04T09:43:16.679 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-04T09:43:16.680 INFO:tasks.cephadm.osd.8:Stopped osd.8 2024-03-04T09:43:16.680 INFO:tasks.cephadm.osd.9:Stopping osd.9... 2024-03-04T09:43:16.680 DEBUG:teuthology.orchestra.run.smithi052:> sudo systemctl stop ceph-4e059f08-da08-11ee-95c6-87774f69a715@osd.9 2024-03-04T09:43:16.710 INFO:journalctl@ceph.osd.9.smithi052.stdout:Mar 04 09:43:16 smithi052 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-9[65396]: 2024-03-04T09:43:16.454+0000 7fd1f1e8e700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-03-04T09:42:09.861821+0000 front 2024-03-04T09:42:09.861761+0000 (oldest deadline 2024-03-04T09:42:32.161859+0000) 2024-03-04T09:43:16.710 INFO:journalctl@ceph.osd.9.smithi052.stdout:Mar 04 09:43:16 smithi052 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-9[65396]: 2024-03-04T09:43:16.454+0000 7fd1f1e8e700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-04T09:42:17.363956+0000 front 2024-03-04T09:42:17.363876+0000 (oldest deadline 2024-03-04T09:42:41.463546+0000) 2024-03-04T09:43:16.710 INFO:journalctl@ceph.osd.9.smithi052.stdout:Mar 04 09:43:16 smithi052 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-9[65396]: 2024-03-04T09:43:16.454+0000 7fd1f1e8e700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-03-04T09:42:21.464200+0000 front 2024-03-04T09:42:21.464139+0000 (oldest deadline 2024-03-04T09:42:47.363942+0000) 2024-03-04T09:43:16.710 INFO:journalctl@ceph.osd.9.smithi052.stdout:Mar 04 09:43:16 smithi052 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-9[65396]: 2024-03-04T09:43:16.454+0000 7fd1f1e8e700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2024-03-04T09:42:33.765329+0000 front 2024-03-04T09:42:33.765393+0000 (oldest deadline 2024-03-04T09:42:59.665478+0000) 2024-03-04T09:43:16.710 INFO:journalctl@ceph.osd.9.smithi052.stdout:Mar 04 09:43:16 smithi052 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-9[65396]: 2024-03-04T09:43:16.454+0000 7fd1f1e8e700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-03-04T09:42:39.666192+0000 front 2024-03-04T09:42:39.666267+0000 (oldest deadline 2024-03-04T09:43:04.966101+0000) 2024-03-04T09:43:16.710 INFO:journalctl@ceph.osd.9.smithi052.stdout:Mar 04 09:43:16 smithi052 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-9[65396]: 2024-03-04T09:43:16.454+0000 7fd1f1e8e700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.191:6812 osd.5 since back 2024-03-04T09:42:47.167809+0000 front 2024-03-04T09:42:47.167792+0000 (oldest deadline 2024-03-04T09:43:12.368082+0000) 2024-03-04T09:43:16.883 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 04 09:43:16 smithi106 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-10[60969]: 2024-03-04T09:43:16.590+0000 7f10f0d39700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-03-04T09:42:08.466029+0000 front 2024-03-04T09:42:08.465963+0000 (oldest deadline 2024-03-04T09:42:32.566054+0000) 2024-03-04T09:43:16.883 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 04 09:43:16 smithi106 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-10[60969]: 2024-03-04T09:43:16.590+0000 7f10f0d39700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-04T09:42:15.467191+0000 front 2024-03-04T09:42:15.467152+0000 (oldest deadline 2024-03-04T09:42:39.567351+0000) 2024-03-04T09:43:16.883 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 04 09:43:16 smithi106 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-10[60969]: 2024-03-04T09:43:16.590+0000 7f10f0d39700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-03-04T09:42:25.368854+0000 front 2024-03-04T09:42:25.368833+0000 (oldest deadline 2024-03-04T09:42:48.868990+0000) 2024-03-04T09:43:16.883 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 04 09:43:16 smithi106 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-10[60969]: 2024-03-04T09:43:16.590+0000 7f10f0d39700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2024-03-04T09:42:28.869679+0000 front 2024-03-04T09:42:28.869692+0000 (oldest deadline 2024-03-04T09:42:55.270190+0000) 2024-03-04T09:43:16.883 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 04 09:43:16 smithi106 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-10[60969]: 2024-03-04T09:43:16.590+0000 7f10f0d39700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-03-04T09:42:42.271376+0000 front 2024-03-04T09:42:42.271331+0000 (oldest deadline 2024-03-04T09:43:07.571444+0000) 2024-03-04T09:43:16.883 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 04 09:43:16 smithi106 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-10[60969]: 2024-03-04T09:43:16.590+0000 7f10f0d39700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.191:6812 osd.5 since back 2024-03-04T09:42:49.272525+0000 front 2024-03-04T09:42:49.272394+0000 (oldest deadline 2024-03-04T09:43:10.872863+0000) 2024-03-04T09:43:16.981 INFO:journalctl@ceph.osd.9.smithi052.stdout:Mar 04 09:43:16 smithi052 systemd[1]: Stopping Ceph osd.9 for 4e059f08-da08-11ee-95c6-87774f69a715... 2024-03-04T09:43:17.336 INFO:journalctl@ceph.osd.9.smithi052.stdout:Mar 04 09:43:16 smithi052 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-9[65396]: 2024-03-04T09:43:16.983+0000 7fd1f98a2700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.9 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-04T09:43:17.336 INFO:journalctl@ceph.osd.9.smithi052.stdout:Mar 04 09:43:16 smithi052 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-9[65396]: 2024-03-04T09:43:16.983+0000 7fd1f98a2700 -1 osd.9 106 *** Got signal Terminated *** 2024-03-04T09:43:17.336 INFO:journalctl@ceph.osd.9.smithi052.stdout:Mar 04 09:43:16 smithi052 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-9[65396]: 2024-03-04T09:43:16.983+0000 7fd1f98a2700 -1 osd.9 106 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-04T09:43:17.588 INFO:journalctl@ceph.osd.11.smithi191.stdout:Mar 04 09:43:17 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-11[62160]: 2024-03-04T09:43:17.291+0000 7fdceac97700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-03-04T09:42:08.553313+0000 front 2024-03-04T09:42:08.553176+0000 (oldest deadline 2024-03-04T09:42:32.653011+0000) 2024-03-04T09:43:17.588 INFO:journalctl@ceph.osd.11.smithi191.stdout:Mar 04 09:43:17 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-11[62160]: 2024-03-04T09:43:17.291+0000 7fdceac97700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-04T09:42:18.355200+0000 front 2024-03-04T09:42:18.355189+0000 (oldest deadline 2024-03-04T09:42:44.254936+0000) 2024-03-04T09:43:17.588 INFO:journalctl@ceph.osd.11.smithi191.stdout:Mar 04 09:43:17 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-11[62160]: 2024-03-04T09:43:17.291+0000 7fdceac97700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-03-04T09:42:24.255361+0000 front 2024-03-04T09:42:24.255246+0000 (oldest deadline 2024-03-04T09:42:47.155465+0000) 2024-03-04T09:43:17.588 INFO:journalctl@ceph.osd.11.smithi191.stdout:Mar 04 09:43:17 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-11[62160]: 2024-03-04T09:43:17.291+0000 7fdceac97700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-03-04T09:42:41.857812+0000 front 2024-03-04T09:42:41.857710+0000 (oldest deadline 2024-03-04T09:43:03.557748+0000) 2024-03-04T09:43:17.589 INFO:journalctl@ceph.osd.11.smithi191.stdout:Mar 04 09:43:17 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-11[62160]: 2024-03-04T09:43:17.291+0000 7fdceac97700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.191:6812 osd.5 since back 2024-03-04T09:42:46.959192+0000 front 2024-03-04T09:42:46.959105+0000 (oldest deadline 2024-03-04T09:43:12.859230+0000) 2024-03-04T09:43:17.836 INFO:journalctl@ceph.osd.9.smithi052.stdout:Mar 04 09:43:17 smithi052 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-9[65396]: 2024-03-04T09:43:17.504+0000 7fd1f1e8e700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-03-04T09:42:09.861821+0000 front 2024-03-04T09:42:09.861761+0000 (oldest deadline 2024-03-04T09:42:32.161859+0000) 2024-03-04T09:43:17.837 INFO:journalctl@ceph.osd.9.smithi052.stdout:Mar 04 09:43:17 smithi052 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-9[65396]: 2024-03-04T09:43:17.504+0000 7fd1f1e8e700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-04T09:42:17.363956+0000 front 2024-03-04T09:42:17.363876+0000 (oldest deadline 2024-03-04T09:42:41.463546+0000) 2024-03-04T09:43:17.837 INFO:journalctl@ceph.osd.9.smithi052.stdout:Mar 04 09:43:17 smithi052 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-9[65396]: 2024-03-04T09:43:17.504+0000 7fd1f1e8e700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-03-04T09:42:21.464200+0000 front 2024-03-04T09:42:21.464139+0000 (oldest deadline 2024-03-04T09:42:47.363942+0000) 2024-03-04T09:43:17.837 INFO:journalctl@ceph.osd.9.smithi052.stdout:Mar 04 09:43:17 smithi052 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-9[65396]: 2024-03-04T09:43:17.504+0000 7fd1f1e8e700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2024-03-04T09:42:33.765329+0000 front 2024-03-04T09:42:33.765393+0000 (oldest deadline 2024-03-04T09:42:59.665478+0000) 2024-03-04T09:43:17.837 INFO:journalctl@ceph.osd.9.smithi052.stdout:Mar 04 09:43:17 smithi052 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-9[65396]: 2024-03-04T09:43:17.504+0000 7fd1f1e8e700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-03-04T09:42:39.666192+0000 front 2024-03-04T09:42:39.666267+0000 (oldest deadline 2024-03-04T09:43:04.966101+0000) 2024-03-04T09:43:17.837 INFO:journalctl@ceph.osd.9.smithi052.stdout:Mar 04 09:43:17 smithi052 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-9[65396]: 2024-03-04T09:43:17.504+0000 7fd1f1e8e700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.191:6812 osd.5 since back 2024-03-04T09:42:47.167809+0000 front 2024-03-04T09:42:47.167792+0000 (oldest deadline 2024-03-04T09:43:12.368082+0000) 2024-03-04T09:43:17.883 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 04 09:43:17 smithi106 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-10[60969]: 2024-03-04T09:43:17.598+0000 7f10f0d39700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-03-04T09:42:08.466029+0000 front 2024-03-04T09:42:08.465963+0000 (oldest deadline 2024-03-04T09:42:32.566054+0000) 2024-03-04T09:43:17.883 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 04 09:43:17 smithi106 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-10[60969]: 2024-03-04T09:43:17.598+0000 7f10f0d39700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-04T09:42:15.467191+0000 front 2024-03-04T09:42:15.467152+0000 (oldest deadline 2024-03-04T09:42:39.567351+0000) 2024-03-04T09:43:17.883 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 04 09:43:17 smithi106 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-10[60969]: 2024-03-04T09:43:17.598+0000 7f10f0d39700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-03-04T09:42:25.368854+0000 front 2024-03-04T09:42:25.368833+0000 (oldest deadline 2024-03-04T09:42:48.868990+0000) 2024-03-04T09:43:17.883 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 04 09:43:17 smithi106 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-10[60969]: 2024-03-04T09:43:17.598+0000 7f10f0d39700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2024-03-04T09:42:28.869679+0000 front 2024-03-04T09:42:28.869692+0000 (oldest deadline 2024-03-04T09:42:55.270190+0000) 2024-03-04T09:43:17.883 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 04 09:43:17 smithi106 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-10[60969]: 2024-03-04T09:43:17.598+0000 7f10f0d39700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-03-04T09:42:42.271376+0000 front 2024-03-04T09:42:42.271331+0000 (oldest deadline 2024-03-04T09:43:07.571444+0000) 2024-03-04T09:43:17.883 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 04 09:43:17 smithi106 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-10[60969]: 2024-03-04T09:43:17.598+0000 7f10f0d39700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.191:6812 osd.5 since back 2024-03-04T09:42:49.272525+0000 front 2024-03-04T09:42:49.272394+0000 (oldest deadline 2024-03-04T09:43:10.872863+0000) 2024-03-04T09:43:18.588 INFO:journalctl@ceph.osd.11.smithi191.stdout:Mar 04 09:43:18 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-11[62160]: 2024-03-04T09:43:18.331+0000 7fdceac97700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-03-04T09:42:08.553313+0000 front 2024-03-04T09:42:08.553176+0000 (oldest deadline 2024-03-04T09:42:32.653011+0000) 2024-03-04T09:43:18.588 INFO:journalctl@ceph.osd.11.smithi191.stdout:Mar 04 09:43:18 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-11[62160]: 2024-03-04T09:43:18.331+0000 7fdceac97700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-04T09:42:18.355200+0000 front 2024-03-04T09:42:18.355189+0000 (oldest deadline 2024-03-04T09:42:44.254936+0000) 2024-03-04T09:43:18.588 INFO:journalctl@ceph.osd.11.smithi191.stdout:Mar 04 09:43:18 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-11[62160]: 2024-03-04T09:43:18.331+0000 7fdceac97700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-03-04T09:42:24.255361+0000 front 2024-03-04T09:42:24.255246+0000 (oldest deadline 2024-03-04T09:42:47.155465+0000) 2024-03-04T09:43:18.588 INFO:journalctl@ceph.osd.11.smithi191.stdout:Mar 04 09:43:18 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-11[62160]: 2024-03-04T09:43:18.331+0000 7fdceac97700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-03-04T09:42:41.857812+0000 front 2024-03-04T09:42:41.857710+0000 (oldest deadline 2024-03-04T09:43:03.557748+0000) 2024-03-04T09:43:18.588 INFO:journalctl@ceph.osd.11.smithi191.stdout:Mar 04 09:43:18 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-11[62160]: 2024-03-04T09:43:18.331+0000 7fdceac97700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.191:6812 osd.5 since back 2024-03-04T09:42:46.959192+0000 front 2024-03-04T09:42:46.959105+0000 (oldest deadline 2024-03-04T09:43:12.859230+0000) 2024-03-04T09:43:18.836 INFO:journalctl@ceph.osd.9.smithi052.stdout:Mar 04 09:43:18 smithi052 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-9[65396]: 2024-03-04T09:43:18.548+0000 7fd1f1e8e700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-03-04T09:42:09.861821+0000 front 2024-03-04T09:42:09.861761+0000 (oldest deadline 2024-03-04T09:42:32.161859+0000) 2024-03-04T09:43:18.837 INFO:journalctl@ceph.osd.9.smithi052.stdout:Mar 04 09:43:18 smithi052 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-9[65396]: 2024-03-04T09:43:18.548+0000 7fd1f1e8e700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-04T09:42:17.363956+0000 front 2024-03-04T09:42:17.363876+0000 (oldest deadline 2024-03-04T09:42:41.463546+0000) 2024-03-04T09:43:18.837 INFO:journalctl@ceph.osd.9.smithi052.stdout:Mar 04 09:43:18 smithi052 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-9[65396]: 2024-03-04T09:43:18.548+0000 7fd1f1e8e700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-03-04T09:42:21.464200+0000 front 2024-03-04T09:42:21.464139+0000 (oldest deadline 2024-03-04T09:42:47.363942+0000) 2024-03-04T09:43:18.837 INFO:journalctl@ceph.osd.9.smithi052.stdout:Mar 04 09:43:18 smithi052 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-9[65396]: 2024-03-04T09:43:18.548+0000 7fd1f1e8e700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2024-03-04T09:42:33.765329+0000 front 2024-03-04T09:42:33.765393+0000 (oldest deadline 2024-03-04T09:42:59.665478+0000) 2024-03-04T09:43:18.837 INFO:journalctl@ceph.osd.9.smithi052.stdout:Mar 04 09:43:18 smithi052 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-9[65396]: 2024-03-04T09:43:18.548+0000 7fd1f1e8e700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-03-04T09:42:39.666192+0000 front 2024-03-04T09:42:39.666267+0000 (oldest deadline 2024-03-04T09:43:04.966101+0000) 2024-03-04T09:43:18.837 INFO:journalctl@ceph.osd.9.smithi052.stdout:Mar 04 09:43:18 smithi052 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-9[65396]: 2024-03-04T09:43:18.548+0000 7fd1f1e8e700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.191:6812 osd.5 since back 2024-03-04T09:42:47.167809+0000 front 2024-03-04T09:42:47.167792+0000 (oldest deadline 2024-03-04T09:43:12.368082+0000) 2024-03-04T09:43:18.883 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 04 09:43:18 smithi106 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-10[60969]: 2024-03-04T09:43:18.592+0000 7f10f0d39700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-03-04T09:42:08.466029+0000 front 2024-03-04T09:42:08.465963+0000 (oldest deadline 2024-03-04T09:42:32.566054+0000) 2024-03-04T09:43:18.883 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 04 09:43:18 smithi106 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-10[60969]: 2024-03-04T09:43:18.592+0000 7f10f0d39700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-04T09:42:15.467191+0000 front 2024-03-04T09:42:15.467152+0000 (oldest deadline 2024-03-04T09:42:39.567351+0000) 2024-03-04T09:43:18.883 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 04 09:43:18 smithi106 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-10[60969]: 2024-03-04T09:43:18.592+0000 7f10f0d39700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-03-04T09:42:25.368854+0000 front 2024-03-04T09:42:25.368833+0000 (oldest deadline 2024-03-04T09:42:48.868990+0000) 2024-03-04T09:43:18.883 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 04 09:43:18 smithi106 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-10[60969]: 2024-03-04T09:43:18.592+0000 7f10f0d39700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2024-03-04T09:42:28.869679+0000 front 2024-03-04T09:42:28.869692+0000 (oldest deadline 2024-03-04T09:42:55.270190+0000) 2024-03-04T09:43:18.884 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 04 09:43:18 smithi106 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-10[60969]: 2024-03-04T09:43:18.592+0000 7f10f0d39700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-03-04T09:42:42.271376+0000 front 2024-03-04T09:42:42.271331+0000 (oldest deadline 2024-03-04T09:43:07.571444+0000) 2024-03-04T09:43:18.884 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 04 09:43:18 smithi106 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-10[60969]: 2024-03-04T09:43:18.592+0000 7f10f0d39700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.191:6812 osd.5 since back 2024-03-04T09:42:49.272525+0000 front 2024-03-04T09:42:49.272394+0000 (oldest deadline 2024-03-04T09:43:10.872863+0000) 2024-03-04T09:43:19.588 INFO:journalctl@ceph.osd.11.smithi191.stdout:Mar 04 09:43:19 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-11[62160]: 2024-03-04T09:43:19.338+0000 7fdceac97700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-03-04T09:42:08.553313+0000 front 2024-03-04T09:42:08.553176+0000 (oldest deadline 2024-03-04T09:42:32.653011+0000) 2024-03-04T09:43:19.588 INFO:journalctl@ceph.osd.11.smithi191.stdout:Mar 04 09:43:19 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-11[62160]: 2024-03-04T09:43:19.338+0000 7fdceac97700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-04T09:42:18.355200+0000 front 2024-03-04T09:42:18.355189+0000 (oldest deadline 2024-03-04T09:42:44.254936+0000) 2024-03-04T09:43:19.588 INFO:journalctl@ceph.osd.11.smithi191.stdout:Mar 04 09:43:19 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-11[62160]: 2024-03-04T09:43:19.338+0000 7fdceac97700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-03-04T09:42:24.255361+0000 front 2024-03-04T09:42:24.255246+0000 (oldest deadline 2024-03-04T09:42:47.155465+0000) 2024-03-04T09:43:19.588 INFO:journalctl@ceph.osd.11.smithi191.stdout:Mar 04 09:43:19 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-11[62160]: 2024-03-04T09:43:19.338+0000 7fdceac97700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-03-04T09:42:41.857812+0000 front 2024-03-04T09:42:41.857710+0000 (oldest deadline 2024-03-04T09:43:03.557748+0000) 2024-03-04T09:43:19.589 INFO:journalctl@ceph.osd.11.smithi191.stdout:Mar 04 09:43:19 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-11[62160]: 2024-03-04T09:43:19.338+0000 7fdceac97700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.191:6812 osd.5 since back 2024-03-04T09:42:46.959192+0000 front 2024-03-04T09:42:46.959105+0000 (oldest deadline 2024-03-04T09:43:12.859230+0000) 2024-03-04T09:43:19.836 INFO:journalctl@ceph.osd.9.smithi052.stdout:Mar 04 09:43:19 smithi052 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-9[65396]: 2024-03-04T09:43:19.533+0000 7fd1f1e8e700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-03-04T09:42:09.861821+0000 front 2024-03-04T09:42:09.861761+0000 (oldest deadline 2024-03-04T09:42:32.161859+0000) 2024-03-04T09:43:19.837 INFO:journalctl@ceph.osd.9.smithi052.stdout:Mar 04 09:43:19 smithi052 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-9[65396]: 2024-03-04T09:43:19.533+0000 7fd1f1e8e700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-04T09:42:17.363956+0000 front 2024-03-04T09:42:17.363876+0000 (oldest deadline 2024-03-04T09:42:41.463546+0000) 2024-03-04T09:43:19.837 INFO:journalctl@ceph.osd.9.smithi052.stdout:Mar 04 09:43:19 smithi052 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-9[65396]: 2024-03-04T09:43:19.533+0000 7fd1f1e8e700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-03-04T09:42:21.464200+0000 front 2024-03-04T09:42:21.464139+0000 (oldest deadline 2024-03-04T09:42:47.363942+0000) 2024-03-04T09:43:19.837 INFO:journalctl@ceph.osd.9.smithi052.stdout:Mar 04 09:43:19 smithi052 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-9[65396]: 2024-03-04T09:43:19.533+0000 7fd1f1e8e700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2024-03-04T09:42:33.765329+0000 front 2024-03-04T09:42:33.765393+0000 (oldest deadline 2024-03-04T09:42:59.665478+0000) 2024-03-04T09:43:19.837 INFO:journalctl@ceph.osd.9.smithi052.stdout:Mar 04 09:43:19 smithi052 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-9[65396]: 2024-03-04T09:43:19.533+0000 7fd1f1e8e700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-03-04T09:42:39.666192+0000 front 2024-03-04T09:42:39.666267+0000 (oldest deadline 2024-03-04T09:43:04.966101+0000) 2024-03-04T09:43:19.837 INFO:journalctl@ceph.osd.9.smithi052.stdout:Mar 04 09:43:19 smithi052 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-9[65396]: 2024-03-04T09:43:19.533+0000 7fd1f1e8e700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.191:6812 osd.5 since back 2024-03-04T09:42:47.167809+0000 front 2024-03-04T09:42:47.167792+0000 (oldest deadline 2024-03-04T09:43:12.368082+0000) 2024-03-04T09:43:19.883 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 04 09:43:19 smithi106 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-10[60969]: 2024-03-04T09:43:19.543+0000 7f10f0d39700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-03-04T09:42:08.466029+0000 front 2024-03-04T09:42:08.465963+0000 (oldest deadline 2024-03-04T09:42:32.566054+0000) 2024-03-04T09:43:19.883 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 04 09:43:19 smithi106 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-10[60969]: 2024-03-04T09:43:19.543+0000 7f10f0d39700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-04T09:42:15.467191+0000 front 2024-03-04T09:42:15.467152+0000 (oldest deadline 2024-03-04T09:42:39.567351+0000) 2024-03-04T09:43:19.883 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 04 09:43:19 smithi106 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-10[60969]: 2024-03-04T09:43:19.543+0000 7f10f0d39700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-03-04T09:42:25.368854+0000 front 2024-03-04T09:42:25.368833+0000 (oldest deadline 2024-03-04T09:42:48.868990+0000) 2024-03-04T09:43:19.883 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 04 09:43:19 smithi106 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-10[60969]: 2024-03-04T09:43:19.543+0000 7f10f0d39700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2024-03-04T09:42:28.869679+0000 front 2024-03-04T09:42:28.869692+0000 (oldest deadline 2024-03-04T09:42:55.270190+0000) 2024-03-04T09:43:19.883 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 04 09:43:19 smithi106 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-10[60969]: 2024-03-04T09:43:19.543+0000 7f10f0d39700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-03-04T09:42:42.271376+0000 front 2024-03-04T09:42:42.271331+0000 (oldest deadline 2024-03-04T09:43:07.571444+0000) 2024-03-04T09:43:19.883 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 04 09:43:19 smithi106 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-10[60969]: 2024-03-04T09:43:19.543+0000 7f10f0d39700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.191:6812 osd.5 since back 2024-03-04T09:42:49.272525+0000 front 2024-03-04T09:42:49.272394+0000 (oldest deadline 2024-03-04T09:43:10.872863+0000) 2024-03-04T09:43:20.836 INFO:journalctl@ceph.osd.9.smithi052.stdout:Mar 04 09:43:20 smithi052 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-9[65396]: 2024-03-04T09:43:20.570+0000 7fd1f1e8e700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-03-04T09:42:09.861821+0000 front 2024-03-04T09:42:09.861761+0000 (oldest deadline 2024-03-04T09:42:32.161859+0000) 2024-03-04T09:43:20.837 INFO:journalctl@ceph.osd.9.smithi052.stdout:Mar 04 09:43:20 smithi052 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-9[65396]: 2024-03-04T09:43:20.570+0000 7fd1f1e8e700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-04T09:42:17.363956+0000 front 2024-03-04T09:42:17.363876+0000 (oldest deadline 2024-03-04T09:42:41.463546+0000) 2024-03-04T09:43:20.837 INFO:journalctl@ceph.osd.9.smithi052.stdout:Mar 04 09:43:20 smithi052 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-9[65396]: 2024-03-04T09:43:20.570+0000 7fd1f1e8e700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-03-04T09:42:21.464200+0000 front 2024-03-04T09:42:21.464139+0000 (oldest deadline 2024-03-04T09:42:47.363942+0000) 2024-03-04T09:43:20.837 INFO:journalctl@ceph.osd.9.smithi052.stdout:Mar 04 09:43:20 smithi052 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-9[65396]: 2024-03-04T09:43:20.570+0000 7fd1f1e8e700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2024-03-04T09:42:33.765329+0000 front 2024-03-04T09:42:33.765393+0000 (oldest deadline 2024-03-04T09:42:59.665478+0000) 2024-03-04T09:43:20.837 INFO:journalctl@ceph.osd.9.smithi052.stdout:Mar 04 09:43:20 smithi052 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-9[65396]: 2024-03-04T09:43:20.570+0000 7fd1f1e8e700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-03-04T09:42:39.666192+0000 front 2024-03-04T09:42:39.666267+0000 (oldest deadline 2024-03-04T09:43:04.966101+0000) 2024-03-04T09:43:20.837 INFO:journalctl@ceph.osd.9.smithi052.stdout:Mar 04 09:43:20 smithi052 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-9[65396]: 2024-03-04T09:43:20.570+0000 7fd1f1e8e700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.191:6812 osd.5 since back 2024-03-04T09:42:47.167809+0000 front 2024-03-04T09:42:47.167792+0000 (oldest deadline 2024-03-04T09:43:12.368082+0000) 2024-03-04T09:43:20.838 INFO:journalctl@ceph.osd.11.smithi191.stdout:Mar 04 09:43:20 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-11[62160]: 2024-03-04T09:43:20.360+0000 7fdceac97700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-03-04T09:42:08.553313+0000 front 2024-03-04T09:42:08.553176+0000 (oldest deadline 2024-03-04T09:42:32.653011+0000) 2024-03-04T09:43:20.838 INFO:journalctl@ceph.osd.11.smithi191.stdout:Mar 04 09:43:20 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-11[62160]: 2024-03-04T09:43:20.360+0000 7fdceac97700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-04T09:42:18.355200+0000 front 2024-03-04T09:42:18.355189+0000 (oldest deadline 2024-03-04T09:42:44.254936+0000) 2024-03-04T09:43:20.838 INFO:journalctl@ceph.osd.11.smithi191.stdout:Mar 04 09:43:20 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-11[62160]: 2024-03-04T09:43:20.360+0000 7fdceac97700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-03-04T09:42:24.255361+0000 front 2024-03-04T09:42:24.255246+0000 (oldest deadline 2024-03-04T09:42:47.155465+0000) 2024-03-04T09:43:20.838 INFO:journalctl@ceph.osd.11.smithi191.stdout:Mar 04 09:43:20 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-11[62160]: 2024-03-04T09:43:20.360+0000 7fdceac97700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-03-04T09:42:41.857812+0000 front 2024-03-04T09:42:41.857710+0000 (oldest deadline 2024-03-04T09:43:03.557748+0000) 2024-03-04T09:43:20.838 INFO:journalctl@ceph.osd.11.smithi191.stdout:Mar 04 09:43:20 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-11[62160]: 2024-03-04T09:43:20.360+0000 7fdceac97700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.191:6812 osd.5 since back 2024-03-04T09:42:46.959192+0000 front 2024-03-04T09:42:46.959105+0000 (oldest deadline 2024-03-04T09:43:12.859230+0000) 2024-03-04T09:43:20.883 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 04 09:43:20 smithi106 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-10[60969]: 2024-03-04T09:43:20.579+0000 7f10f0d39700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-03-04T09:42:08.466029+0000 front 2024-03-04T09:42:08.465963+0000 (oldest deadline 2024-03-04T09:42:32.566054+0000) 2024-03-04T09:43:20.883 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 04 09:43:20 smithi106 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-10[60969]: 2024-03-04T09:43:20.579+0000 7f10f0d39700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-04T09:42:15.467191+0000 front 2024-03-04T09:42:15.467152+0000 (oldest deadline 2024-03-04T09:42:39.567351+0000) 2024-03-04T09:43:20.883 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 04 09:43:20 smithi106 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-10[60969]: 2024-03-04T09:43:20.579+0000 7f10f0d39700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-03-04T09:42:25.368854+0000 front 2024-03-04T09:42:25.368833+0000 (oldest deadline 2024-03-04T09:42:48.868990+0000) 2024-03-04T09:43:20.884 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 04 09:43:20 smithi106 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-10[60969]: 2024-03-04T09:43:20.579+0000 7f10f0d39700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2024-03-04T09:42:28.869679+0000 front 2024-03-04T09:42:28.869692+0000 (oldest deadline 2024-03-04T09:42:55.270190+0000) 2024-03-04T09:43:20.884 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 04 09:43:20 smithi106 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-10[60969]: 2024-03-04T09:43:20.579+0000 7f10f0d39700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-03-04T09:42:42.271376+0000 front 2024-03-04T09:42:42.271331+0000 (oldest deadline 2024-03-04T09:43:07.571444+0000) 2024-03-04T09:43:20.884 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 04 09:43:20 smithi106 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-10[60969]: 2024-03-04T09:43:20.579+0000 7f10f0d39700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.191:6812 osd.5 since back 2024-03-04T09:42:49.272525+0000 front 2024-03-04T09:42:49.272394+0000 (oldest deadline 2024-03-04T09:43:10.872863+0000) 2024-03-04T09:43:21.838 INFO:journalctl@ceph.osd.11.smithi191.stdout:Mar 04 09:43:21 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-11[62160]: 2024-03-04T09:43:21.381+0000 7fdceac97700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-03-04T09:42:08.553313+0000 front 2024-03-04T09:42:08.553176+0000 (oldest deadline 2024-03-04T09:42:32.653011+0000) 2024-03-04T09:43:21.838 INFO:journalctl@ceph.osd.11.smithi191.stdout:Mar 04 09:43:21 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-11[62160]: 2024-03-04T09:43:21.381+0000 7fdceac97700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-04T09:42:18.355200+0000 front 2024-03-04T09:42:18.355189+0000 (oldest deadline 2024-03-04T09:42:44.254936+0000) 2024-03-04T09:43:21.838 INFO:journalctl@ceph.osd.11.smithi191.stdout:Mar 04 09:43:21 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-11[62160]: 2024-03-04T09:43:21.381+0000 7fdceac97700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-03-04T09:42:24.255361+0000 front 2024-03-04T09:42:24.255246+0000 (oldest deadline 2024-03-04T09:42:47.155465+0000) 2024-03-04T09:43:21.839 INFO:journalctl@ceph.osd.11.smithi191.stdout:Mar 04 09:43:21 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-11[62160]: 2024-03-04T09:43:21.381+0000 7fdceac97700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-03-04T09:42:41.857812+0000 front 2024-03-04T09:42:41.857710+0000 (oldest deadline 2024-03-04T09:43:03.557748+0000) 2024-03-04T09:43:21.839 INFO:journalctl@ceph.osd.11.smithi191.stdout:Mar 04 09:43:21 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-11[62160]: 2024-03-04T09:43:21.381+0000 7fdceac97700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.191:6812 osd.5 since back 2024-03-04T09:42:46.959192+0000 front 2024-03-04T09:42:46.959105+0000 (oldest deadline 2024-03-04T09:43:12.859230+0000) 2024-03-04T09:43:21.883 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 04 09:43:21 smithi106 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-10[60969]: 2024-03-04T09:43:21.617+0000 7f10f0d39700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-03-04T09:42:08.466029+0000 front 2024-03-04T09:42:08.465963+0000 (oldest deadline 2024-03-04T09:42:32.566054+0000) 2024-03-04T09:43:21.883 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 04 09:43:21 smithi106 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-10[60969]: 2024-03-04T09:43:21.617+0000 7f10f0d39700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-04T09:42:15.467191+0000 front 2024-03-04T09:42:15.467152+0000 (oldest deadline 2024-03-04T09:42:39.567351+0000) 2024-03-04T09:43:21.883 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 04 09:43:21 smithi106 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-10[60969]: 2024-03-04T09:43:21.617+0000 7f10f0d39700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-03-04T09:42:25.368854+0000 front 2024-03-04T09:42:25.368833+0000 (oldest deadline 2024-03-04T09:42:48.868990+0000) 2024-03-04T09:43:21.883 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 04 09:43:21 smithi106 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-10[60969]: 2024-03-04T09:43:21.617+0000 7f10f0d39700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2024-03-04T09:42:28.869679+0000 front 2024-03-04T09:42:28.869692+0000 (oldest deadline 2024-03-04T09:42:55.270190+0000) 2024-03-04T09:43:21.883 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 04 09:43:21 smithi106 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-10[60969]: 2024-03-04T09:43:21.617+0000 7f10f0d39700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-03-04T09:42:42.271376+0000 front 2024-03-04T09:42:42.271331+0000 (oldest deadline 2024-03-04T09:43:07.571444+0000) 2024-03-04T09:43:21.884 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 04 09:43:21 smithi106 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-10[60969]: 2024-03-04T09:43:21.617+0000 7f10f0d39700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.191:6812 osd.5 since back 2024-03-04T09:42:49.272525+0000 front 2024-03-04T09:42:49.272394+0000 (oldest deadline 2024-03-04T09:43:10.872863+0000) 2024-03-04T09:43:22.086 INFO:journalctl@ceph.osd.9.smithi052.stdout:Mar 04 09:43:21 smithi052 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-9[65396]: 2024-03-04T09:43:21.596+0000 7fd1f1e8e700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-03-04T09:42:09.861821+0000 front 2024-03-04T09:42:09.861761+0000 (oldest deadline 2024-03-04T09:42:32.161859+0000) 2024-03-04T09:43:22.086 INFO:journalctl@ceph.osd.9.smithi052.stdout:Mar 04 09:43:21 smithi052 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-9[65396]: 2024-03-04T09:43:21.596+0000 7fd1f1e8e700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-04T09:42:17.363956+0000 front 2024-03-04T09:42:17.363876+0000 (oldest deadline 2024-03-04T09:42:41.463546+0000) 2024-03-04T09:43:22.087 INFO:journalctl@ceph.osd.9.smithi052.stdout:Mar 04 09:43:21 smithi052 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-9[65396]: 2024-03-04T09:43:21.596+0000 7fd1f1e8e700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-03-04T09:42:21.464200+0000 front 2024-03-04T09:42:21.464139+0000 (oldest deadline 2024-03-04T09:42:47.363942+0000) 2024-03-04T09:43:22.087 INFO:journalctl@ceph.osd.9.smithi052.stdout:Mar 04 09:43:21 smithi052 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-9[65396]: 2024-03-04T09:43:21.596+0000 7fd1f1e8e700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2024-03-04T09:42:33.765329+0000 front 2024-03-04T09:42:33.765393+0000 (oldest deadline 2024-03-04T09:42:59.665478+0000) 2024-03-04T09:43:22.087 INFO:journalctl@ceph.osd.9.smithi052.stdout:Mar 04 09:43:21 smithi052 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-9[65396]: 2024-03-04T09:43:21.596+0000 7fd1f1e8e700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-03-04T09:42:39.666192+0000 front 2024-03-04T09:42:39.666267+0000 (oldest deadline 2024-03-04T09:43:04.966101+0000) 2024-03-04T09:43:22.087 INFO:journalctl@ceph.osd.9.smithi052.stdout:Mar 04 09:43:21 smithi052 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-9[65396]: 2024-03-04T09:43:21.596+0000 7fd1f1e8e700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.191:6812 osd.5 since back 2024-03-04T09:42:47.167809+0000 front 2024-03-04T09:42:47.167792+0000 (oldest deadline 2024-03-04T09:43:12.368082+0000) 2024-03-04T09:43:22.640 INFO:journalctl@ceph.osd.9.smithi052.stdout:Mar 04 09:43:22 smithi052 podman[96644]: 2024-03-04 09:43:22.377963372 +0000 UTC m=+5.494816043 container died d6717d8248e57ee8dd44fa175f458eb75ca63aad4918b12c49fe6fd55ab23b68 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5, name=ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-9, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, RELEASE=wip-lusov-qdb-1bc250c, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240222, org.label-schema.license=GPLv2, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux ) 2024-03-04T09:43:22.838 INFO:journalctl@ceph.osd.11.smithi191.stdout:Mar 04 09:43:22 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-11[62160]: 2024-03-04T09:43:22.381+0000 7fdceac97700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-03-04T09:42:08.553313+0000 front 2024-03-04T09:42:08.553176+0000 (oldest deadline 2024-03-04T09:42:32.653011+0000) 2024-03-04T09:43:22.838 INFO:journalctl@ceph.osd.11.smithi191.stdout:Mar 04 09:43:22 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-11[62160]: 2024-03-04T09:43:22.381+0000 7fdceac97700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-04T09:42:18.355200+0000 front 2024-03-04T09:42:18.355189+0000 (oldest deadline 2024-03-04T09:42:44.254936+0000) 2024-03-04T09:43:22.838 INFO:journalctl@ceph.osd.11.smithi191.stdout:Mar 04 09:43:22 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-11[62160]: 2024-03-04T09:43:22.381+0000 7fdceac97700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-03-04T09:42:24.255361+0000 front 2024-03-04T09:42:24.255246+0000 (oldest deadline 2024-03-04T09:42:47.155465+0000) 2024-03-04T09:43:22.838 INFO:journalctl@ceph.osd.11.smithi191.stdout:Mar 04 09:43:22 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-11[62160]: 2024-03-04T09:43:22.381+0000 7fdceac97700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-03-04T09:42:41.857812+0000 front 2024-03-04T09:42:41.857710+0000 (oldest deadline 2024-03-04T09:43:03.557748+0000) 2024-03-04T09:43:22.838 INFO:journalctl@ceph.osd.11.smithi191.stdout:Mar 04 09:43:22 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-11[62160]: 2024-03-04T09:43:22.381+0000 7fdceac97700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.191:6812 osd.5 since back 2024-03-04T09:42:46.959192+0000 front 2024-03-04T09:42:46.959105+0000 (oldest deadline 2024-03-04T09:43:12.859230+0000) 2024-03-04T09:43:22.839 INFO:journalctl@ceph.osd.11.smithi191.stdout:Mar 04 09:43:22 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-11[62160]: 2024-03-04T09:43:22.381+0000 7fdceac97700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.52:6822 osd.6 since back 2024-03-04T09:42:56.260599+0000 front 2024-03-04T09:42:56.260647+0000 (oldest deadline 2024-03-04T09:43:22.160568+0000) 2024-03-04T09:43:22.883 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 04 09:43:22 smithi106 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-10[60969]: 2024-03-04T09:43:22.571+0000 7f10f0d39700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-03-04T09:42:08.466029+0000 front 2024-03-04T09:42:08.465963+0000 (oldest deadline 2024-03-04T09:42:32.566054+0000) 2024-03-04T09:43:22.883 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 04 09:43:22 smithi106 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-10[60969]: 2024-03-04T09:43:22.571+0000 7f10f0d39700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-04T09:42:15.467191+0000 front 2024-03-04T09:42:15.467152+0000 (oldest deadline 2024-03-04T09:42:39.567351+0000) 2024-03-04T09:43:22.883 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 04 09:43:22 smithi106 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-10[60969]: 2024-03-04T09:43:22.571+0000 7f10f0d39700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-03-04T09:42:25.368854+0000 front 2024-03-04T09:42:25.368833+0000 (oldest deadline 2024-03-04T09:42:48.868990+0000) 2024-03-04T09:43:22.884 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 04 09:43:22 smithi106 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-10[60969]: 2024-03-04T09:43:22.571+0000 7f10f0d39700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2024-03-04T09:42:28.869679+0000 front 2024-03-04T09:42:28.869692+0000 (oldest deadline 2024-03-04T09:42:55.270190+0000) 2024-03-04T09:43:22.884 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 04 09:43:22 smithi106 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-10[60969]: 2024-03-04T09:43:22.571+0000 7f10f0d39700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-03-04T09:42:42.271376+0000 front 2024-03-04T09:42:42.271331+0000 (oldest deadline 2024-03-04T09:43:07.571444+0000) 2024-03-04T09:43:22.884 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 04 09:43:22 smithi106 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-10[60969]: 2024-03-04T09:43:22.571+0000 7f10f0d39700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.191:6812 osd.5 since back 2024-03-04T09:42:49.272525+0000 front 2024-03-04T09:42:49.272394+0000 (oldest deadline 2024-03-04T09:43:10.872863+0000) 2024-03-04T09:43:22.884 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 04 09:43:22 smithi106 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-10[60969]: 2024-03-04T09:43:22.571+0000 7f10f0d39700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.52:6822 osd.6 since back 2024-03-04T09:42:55.974740+0000 front 2024-03-04T09:42:55.974666+0000 (oldest deadline 2024-03-04T09:43:21.874604+0000) 2024-03-04T09:43:22.924 INFO:journalctl@ceph.osd.9.smithi052.stdout:Mar 04 09:43:22 smithi052 podman[96644]: 2024-03-04 09:43:22.733841106 +0000 UTC m=+5.850693778 container cleanup d6717d8248e57ee8dd44fa175f458eb75ca63aad4918b12c49fe6fd55ab23b68 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5, name=ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-9, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-qdb-1bc250c, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, org.label-schema.build-date=20240222, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-03-04T09:43:22.924 INFO:journalctl@ceph.osd.9.smithi052.stdout:Mar 04 09:43:22 smithi052 bash[96644]: ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-9 2024-03-04T09:43:23.325 INFO:journalctl@ceph.osd.9.smithi052.stdout:Mar 04 09:43:22 smithi052 podman[96662]: 2024-03-04 09:43:22.926724856 +0000 UTC m=+0.544552898 container remove d6717d8248e57ee8dd44fa175f458eb75ca63aad4918b12c49fe6fd55ab23b68 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5, name=ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-9, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-qdb-1bc250c, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240222, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux ) 2024-03-04T09:43:23.586 INFO:journalctl@ceph.osd.9.smithi052.stdout:Mar 04 09:43:23 smithi052 podman[96750]: 2024-03-04 09:43:23.327506525 +0000 UTC m=+0.095882502 container create a6ecac5ff478d31c355119e1421876c0c998cca12fe515f5d8670447cbe785e6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5, name=ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-9-deactivate, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-qdb-1bc250c, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.build-date=20240222, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux ) 2024-03-04T09:43:23.587 INFO:journalctl@ceph.osd.9.smithi052.stdout:Mar 04 09:43:23 smithi052 podman[96750]: 2024-03-04 09:43:23.249383843 +0000 UTC m=+0.017759822 image pull f88ffb32aa6a7d5dfdc954d9f096f0c9cc47981d05ec8d117ba859a22d8185ba quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5 2024-03-04T09:43:23.838 INFO:journalctl@ceph.osd.11.smithi191.stdout:Mar 04 09:43:23 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-11[62160]: 2024-03-04T09:43:23.377+0000 7fdceac97700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-03-04T09:42:08.553313+0000 front 2024-03-04T09:42:08.553176+0000 (oldest deadline 2024-03-04T09:42:32.653011+0000) 2024-03-04T09:43:23.838 INFO:journalctl@ceph.osd.11.smithi191.stdout:Mar 04 09:43:23 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-11[62160]: 2024-03-04T09:43:23.377+0000 7fdceac97700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-04T09:42:18.355200+0000 front 2024-03-04T09:42:18.355189+0000 (oldest deadline 2024-03-04T09:42:44.254936+0000) 2024-03-04T09:43:23.838 INFO:journalctl@ceph.osd.11.smithi191.stdout:Mar 04 09:43:23 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-11[62160]: 2024-03-04T09:43:23.377+0000 7fdceac97700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-03-04T09:42:24.255361+0000 front 2024-03-04T09:42:24.255246+0000 (oldest deadline 2024-03-04T09:42:47.155465+0000) 2024-03-04T09:43:23.838 INFO:journalctl@ceph.osd.11.smithi191.stdout:Mar 04 09:43:23 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-11[62160]: 2024-03-04T09:43:23.377+0000 7fdceac97700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-03-04T09:42:41.857812+0000 front 2024-03-04T09:42:41.857710+0000 (oldest deadline 2024-03-04T09:43:03.557748+0000) 2024-03-04T09:43:23.838 INFO:journalctl@ceph.osd.11.smithi191.stdout:Mar 04 09:43:23 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-11[62160]: 2024-03-04T09:43:23.377+0000 7fdceac97700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.191:6812 osd.5 since back 2024-03-04T09:42:46.959192+0000 front 2024-03-04T09:42:46.959105+0000 (oldest deadline 2024-03-04T09:43:12.859230+0000) 2024-03-04T09:43:23.839 INFO:journalctl@ceph.osd.11.smithi191.stdout:Mar 04 09:43:23 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-11[62160]: 2024-03-04T09:43:23.377+0000 7fdceac97700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.52:6822 osd.6 since back 2024-03-04T09:42:56.260599+0000 front 2024-03-04T09:42:56.260647+0000 (oldest deadline 2024-03-04T09:43:22.160568+0000) 2024-03-04T09:43:23.883 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 04 09:43:23 smithi106 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-10[60969]: 2024-03-04T09:43:23.546+0000 7f10f0d39700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-03-04T09:42:08.466029+0000 front 2024-03-04T09:42:08.465963+0000 (oldest deadline 2024-03-04T09:42:32.566054+0000) 2024-03-04T09:43:23.883 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 04 09:43:23 smithi106 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-10[60969]: 2024-03-04T09:43:23.546+0000 7f10f0d39700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-04T09:42:15.467191+0000 front 2024-03-04T09:42:15.467152+0000 (oldest deadline 2024-03-04T09:42:39.567351+0000) 2024-03-04T09:43:23.883 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 04 09:43:23 smithi106 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-10[60969]: 2024-03-04T09:43:23.546+0000 7f10f0d39700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-03-04T09:42:25.368854+0000 front 2024-03-04T09:42:25.368833+0000 (oldest deadline 2024-03-04T09:42:48.868990+0000) 2024-03-04T09:43:23.883 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 04 09:43:23 smithi106 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-10[60969]: 2024-03-04T09:43:23.546+0000 7f10f0d39700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2024-03-04T09:42:28.869679+0000 front 2024-03-04T09:42:28.869692+0000 (oldest deadline 2024-03-04T09:42:55.270190+0000) 2024-03-04T09:43:23.884 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 04 09:43:23 smithi106 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-10[60969]: 2024-03-04T09:43:23.546+0000 7f10f0d39700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-03-04T09:42:42.271376+0000 front 2024-03-04T09:42:42.271331+0000 (oldest deadline 2024-03-04T09:43:07.571444+0000) 2024-03-04T09:43:23.884 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 04 09:43:23 smithi106 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-10[60969]: 2024-03-04T09:43:23.546+0000 7f10f0d39700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.191:6812 osd.5 since back 2024-03-04T09:42:49.272525+0000 front 2024-03-04T09:42:49.272394+0000 (oldest deadline 2024-03-04T09:43:10.872863+0000) 2024-03-04T09:43:23.884 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 04 09:43:23 smithi106 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-10[60969]: 2024-03-04T09:43:23.546+0000 7f10f0d39700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.52:6822 osd.6 since back 2024-03-04T09:42:55.974740+0000 front 2024-03-04T09:42:55.974666+0000 (oldest deadline 2024-03-04T09:43:21.874604+0000) 2024-03-04T09:43:23.934 INFO:journalctl@ceph.osd.9.smithi052.stdout:Mar 04 09:43:23 smithi052 podman[96750]: 2024-03-04 09:43:23.638360278 +0000 UTC m=+0.406736272 container init a6ecac5ff478d31c355119e1421876c0c998cca12fe515f5d8670447cbe785e6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5, name=ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-9-deactivate, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-qdb-1bc250c, org.label-schema.license=GPLv2, org.label-schema.build-date=20240222, maintainer=Guillaume Abrioux ) 2024-03-04T09:43:23.935 INFO:journalctl@ceph.osd.9.smithi052.stdout:Mar 04 09:43:23 smithi052 podman[96750]: 2024-03-04 09:43:23.641773723 +0000 UTC m=+0.410149702 container start a6ecac5ff478d31c355119e1421876c0c998cca12fe515f5d8670447cbe785e6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5, name=ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-9-deactivate, RELEASE=wip-lusov-qdb-1bc250c, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, io.buildah.version=1.33.5, org.label-schema.build-date=20240222, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-03-04T09:43:23.935 INFO:journalctl@ceph.osd.9.smithi052.stdout:Mar 04 09:43:23 smithi052 podman[96750]: 2024-03-04 09:43:23.696524616 +0000 UTC m=+0.464900611 container attach a6ecac5ff478d31c355119e1421876c0c998cca12fe515f5d8670447cbe785e6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5, name=ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-9-deactivate, ceph=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-qdb-1bc250c, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, org.label-schema.build-date=20240222, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-03-04T09:43:24.256 INFO:journalctl@ceph.osd.9.smithi052.stdout:Mar 04 09:43:23 smithi052 podman[96750]: 2024-03-04 09:43:23.937355807 +0000 UTC m=+0.705731799 container died a6ecac5ff478d31c355119e1421876c0c998cca12fe515f5d8670447cbe785e6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5, name=ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-9-deactivate, org.label-schema.build-date=20240222, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-qdb-1bc250c, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-04T09:43:24.498 DEBUG:teuthology.orchestra.run.smithi052:> sudo pkill -f 'journalctl -f -n 0 -u ceph-4e059f08-da08-11ee-95c6-87774f69a715@osd.9.service' 2024-03-04T09:43:24.563 INFO:journalctl@ceph.osd.9.smithi052.stdout:Mar 04 09:43:24 smithi052 podman[96777]: 2024-03-04 09:43:24.492066473 +0000 UTC m=+0.549290874 container remove a6ecac5ff478d31c355119e1421876c0c998cca12fe515f5d8670447cbe785e6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5, name=ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-9-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240222, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, RELEASE=wip-lusov-qdb-1bc250c, org.label-schema.schema-version=1.0) 2024-03-04T09:43:24.563 INFO:journalctl@ceph.osd.9.smithi052.stdout:Mar 04 09:43:24 smithi052 systemd[1]: ceph-4e059f08-da08-11ee-95c6-87774f69a715@osd.9.service: Deactivated successfully. 2024-03-04T09:43:24.564 INFO:journalctl@ceph.osd.9.smithi052.stdout:Mar 04 09:43:24 smithi052 systemd[1]: Stopped Ceph osd.9 for 4e059f08-da08-11ee-95c6-87774f69a715. 2024-03-04T09:43:24.564 INFO:journalctl@ceph.osd.9.smithi052.stdout:Mar 04 09:43:24 smithi052 systemd[1]: ceph-4e059f08-da08-11ee-95c6-87774f69a715@osd.9.service: Consumed 19.953s CPU time. 2024-03-04T09:43:24.579 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-04T09:43:24.580 INFO:tasks.cephadm.osd.9:Stopped osd.9 2024-03-04T09:43:24.580 INFO:tasks.cephadm.osd.10:Stopping osd.10... 2024-03-04T09:43:24.580 DEBUG:teuthology.orchestra.run.smithi106:> sudo systemctl stop ceph-4e059f08-da08-11ee-95c6-87774f69a715@osd.10 2024-03-04T09:43:24.838 INFO:journalctl@ceph.osd.11.smithi191.stdout:Mar 04 09:43:24 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-11[62160]: 2024-03-04T09:43:24.367+0000 7fdceac97700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-03-04T09:42:08.553313+0000 front 2024-03-04T09:42:08.553176+0000 (oldest deadline 2024-03-04T09:42:32.653011+0000) 2024-03-04T09:43:24.838 INFO:journalctl@ceph.osd.11.smithi191.stdout:Mar 04 09:43:24 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-11[62160]: 2024-03-04T09:43:24.368+0000 7fdceac97700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-04T09:42:18.355200+0000 front 2024-03-04T09:42:18.355189+0000 (oldest deadline 2024-03-04T09:42:44.254936+0000) 2024-03-04T09:43:24.838 INFO:journalctl@ceph.osd.11.smithi191.stdout:Mar 04 09:43:24 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-11[62160]: 2024-03-04T09:43:24.368+0000 7fdceac97700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-03-04T09:42:24.255361+0000 front 2024-03-04T09:42:24.255246+0000 (oldest deadline 2024-03-04T09:42:47.155465+0000) 2024-03-04T09:43:24.839 INFO:journalctl@ceph.osd.11.smithi191.stdout:Mar 04 09:43:24 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-11[62160]: 2024-03-04T09:43:24.368+0000 7fdceac97700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-03-04T09:42:41.857812+0000 front 2024-03-04T09:42:41.857710+0000 (oldest deadline 2024-03-04T09:43:03.557748+0000) 2024-03-04T09:43:24.839 INFO:journalctl@ceph.osd.11.smithi191.stdout:Mar 04 09:43:24 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-11[62160]: 2024-03-04T09:43:24.368+0000 7fdceac97700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.191:6812 osd.5 since back 2024-03-04T09:42:46.959192+0000 front 2024-03-04T09:42:46.959105+0000 (oldest deadline 2024-03-04T09:43:12.859230+0000) 2024-03-04T09:43:24.839 INFO:journalctl@ceph.osd.11.smithi191.stdout:Mar 04 09:43:24 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-11[62160]: 2024-03-04T09:43:24.368+0000 7fdceac97700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.52:6822 osd.6 since back 2024-03-04T09:42:56.260599+0000 front 2024-03-04T09:42:56.260647+0000 (oldest deadline 2024-03-04T09:43:22.160568+0000) 2024-03-04T09:43:24.880 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 04 09:43:24 smithi106 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-10[60969]: 2024-03-04T09:43:24.588+0000 7f10f0d39700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-03-04T09:42:08.466029+0000 front 2024-03-04T09:42:08.465963+0000 (oldest deadline 2024-03-04T09:42:32.566054+0000) 2024-03-04T09:43:24.880 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 04 09:43:24 smithi106 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-10[60969]: 2024-03-04T09:43:24.588+0000 7f10f0d39700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-04T09:42:15.467191+0000 front 2024-03-04T09:42:15.467152+0000 (oldest deadline 2024-03-04T09:42:39.567351+0000) 2024-03-04T09:43:24.880 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 04 09:43:24 smithi106 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-10[60969]: 2024-03-04T09:43:24.588+0000 7f10f0d39700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-03-04T09:42:25.368854+0000 front 2024-03-04T09:42:25.368833+0000 (oldest deadline 2024-03-04T09:42:48.868990+0000) 2024-03-04T09:43:24.881 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 04 09:43:24 smithi106 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-10[60969]: 2024-03-04T09:43:24.588+0000 7f10f0d39700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2024-03-04T09:42:28.869679+0000 front 2024-03-04T09:42:28.869692+0000 (oldest deadline 2024-03-04T09:42:55.270190+0000) 2024-03-04T09:43:24.881 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 04 09:43:24 smithi106 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-10[60969]: 2024-03-04T09:43:24.588+0000 7f10f0d39700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-03-04T09:42:42.271376+0000 front 2024-03-04T09:42:42.271331+0000 (oldest deadline 2024-03-04T09:43:07.571444+0000) 2024-03-04T09:43:24.881 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 04 09:43:24 smithi106 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-10[60969]: 2024-03-04T09:43:24.588+0000 7f10f0d39700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.191:6812 osd.5 since back 2024-03-04T09:42:49.272525+0000 front 2024-03-04T09:42:49.272394+0000 (oldest deadline 2024-03-04T09:43:10.872863+0000) 2024-03-04T09:43:24.881 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 04 09:43:24 smithi106 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-10[60969]: 2024-03-04T09:43:24.588+0000 7f10f0d39700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.52:6822 osd.6 since back 2024-03-04T09:42:55.974740+0000 front 2024-03-04T09:42:55.974666+0000 (oldest deadline 2024-03-04T09:43:21.874604+0000) 2024-03-04T09:43:24.881 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 04 09:43:24 smithi106 systemd[1]: Stopping Ceph osd.10 for 4e059f08-da08-11ee-95c6-87774f69a715... 2024-03-04T09:43:24.881 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 04 09:43:24 smithi106 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-10[60969]: 2024-03-04T09:43:24.880+0000 7f10f874d700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.10 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-04T09:43:24.881 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 04 09:43:24 smithi106 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-10[60969]: 2024-03-04T09:43:24.880+0000 7f10f874d700 -1 osd.10 106 *** Got signal Terminated *** 2024-03-04T09:43:24.882 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 04 09:43:24 smithi106 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-10[60969]: 2024-03-04T09:43:24.880+0000 7f10f874d700 -1 osd.10 106 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-04T09:43:25.838 INFO:journalctl@ceph.osd.11.smithi191.stdout:Mar 04 09:43:25 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-11[62160]: 2024-03-04T09:43:25.368+0000 7fdceac97700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-03-04T09:42:08.553313+0000 front 2024-03-04T09:42:08.553176+0000 (oldest deadline 2024-03-04T09:42:32.653011+0000) 2024-03-04T09:43:25.838 INFO:journalctl@ceph.osd.11.smithi191.stdout:Mar 04 09:43:25 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-11[62160]: 2024-03-04T09:43:25.368+0000 7fdceac97700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-04T09:42:18.355200+0000 front 2024-03-04T09:42:18.355189+0000 (oldest deadline 2024-03-04T09:42:44.254936+0000) 2024-03-04T09:43:25.838 INFO:journalctl@ceph.osd.11.smithi191.stdout:Mar 04 09:43:25 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-11[62160]: 2024-03-04T09:43:25.368+0000 7fdceac97700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-03-04T09:42:24.255361+0000 front 2024-03-04T09:42:24.255246+0000 (oldest deadline 2024-03-04T09:42:47.155465+0000) 2024-03-04T09:43:25.838 INFO:journalctl@ceph.osd.11.smithi191.stdout:Mar 04 09:43:25 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-11[62160]: 2024-03-04T09:43:25.368+0000 7fdceac97700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-03-04T09:42:41.857812+0000 front 2024-03-04T09:42:41.857710+0000 (oldest deadline 2024-03-04T09:43:03.557748+0000) 2024-03-04T09:43:25.839 INFO:journalctl@ceph.osd.11.smithi191.stdout:Mar 04 09:43:25 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-11[62160]: 2024-03-04T09:43:25.368+0000 7fdceac97700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.191:6812 osd.5 since back 2024-03-04T09:42:46.959192+0000 front 2024-03-04T09:42:46.959105+0000 (oldest deadline 2024-03-04T09:43:12.859230+0000) 2024-03-04T09:43:25.839 INFO:journalctl@ceph.osd.11.smithi191.stdout:Mar 04 09:43:25 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-11[62160]: 2024-03-04T09:43:25.368+0000 7fdceac97700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.52:6822 osd.6 since back 2024-03-04T09:42:56.260599+0000 front 2024-03-04T09:42:56.260647+0000 (oldest deadline 2024-03-04T09:43:22.160568+0000) 2024-03-04T09:43:26.133 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 04 09:43:25 smithi106 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-10[60969]: 2024-03-04T09:43:25.636+0000 7f10f0d39700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-03-04T09:42:08.466029+0000 front 2024-03-04T09:42:08.465963+0000 (oldest deadline 2024-03-04T09:42:32.566054+0000) 2024-03-04T09:43:26.133 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 04 09:43:25 smithi106 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-10[60969]: 2024-03-04T09:43:25.636+0000 7f10f0d39700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-04T09:42:15.467191+0000 front 2024-03-04T09:42:15.467152+0000 (oldest deadline 2024-03-04T09:42:39.567351+0000) 2024-03-04T09:43:26.133 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 04 09:43:25 smithi106 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-10[60969]: 2024-03-04T09:43:25.636+0000 7f10f0d39700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-03-04T09:42:25.368854+0000 front 2024-03-04T09:42:25.368833+0000 (oldest deadline 2024-03-04T09:42:48.868990+0000) 2024-03-04T09:43:26.133 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 04 09:43:25 smithi106 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-10[60969]: 2024-03-04T09:43:25.636+0000 7f10f0d39700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2024-03-04T09:42:28.869679+0000 front 2024-03-04T09:42:28.869692+0000 (oldest deadline 2024-03-04T09:42:55.270190+0000) 2024-03-04T09:43:26.133 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 04 09:43:25 smithi106 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-10[60969]: 2024-03-04T09:43:25.636+0000 7f10f0d39700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-03-04T09:42:42.271376+0000 front 2024-03-04T09:42:42.271331+0000 (oldest deadline 2024-03-04T09:43:07.571444+0000) 2024-03-04T09:43:26.133 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 04 09:43:25 smithi106 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-10[60969]: 2024-03-04T09:43:25.636+0000 7f10f0d39700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.191:6812 osd.5 since back 2024-03-04T09:42:49.272525+0000 front 2024-03-04T09:42:49.272394+0000 (oldest deadline 2024-03-04T09:43:10.872863+0000) 2024-03-04T09:43:26.134 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 04 09:43:25 smithi106 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-10[60969]: 2024-03-04T09:43:25.636+0000 7f10f0d39700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.52:6822 osd.6 since back 2024-03-04T09:42:55.974740+0000 front 2024-03-04T09:42:55.974666+0000 (oldest deadline 2024-03-04T09:43:21.874604+0000) 2024-03-04T09:43:26.588 INFO:journalctl@ceph.osd.11.smithi191.stdout:Mar 04 09:43:26 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-11[62160]: 2024-03-04T09:43:26.331+0000 7fdceac97700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-03-04T09:42:08.553313+0000 front 2024-03-04T09:42:08.553176+0000 (oldest deadline 2024-03-04T09:42:32.653011+0000) 2024-03-04T09:43:26.588 INFO:journalctl@ceph.osd.11.smithi191.stdout:Mar 04 09:43:26 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-11[62160]: 2024-03-04T09:43:26.331+0000 7fdceac97700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-04T09:42:18.355200+0000 front 2024-03-04T09:42:18.355189+0000 (oldest deadline 2024-03-04T09:42:44.254936+0000) 2024-03-04T09:43:26.589 INFO:journalctl@ceph.osd.11.smithi191.stdout:Mar 04 09:43:26 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-11[62160]: 2024-03-04T09:43:26.331+0000 7fdceac97700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-03-04T09:42:24.255361+0000 front 2024-03-04T09:42:24.255246+0000 (oldest deadline 2024-03-04T09:42:47.155465+0000) 2024-03-04T09:43:26.589 INFO:journalctl@ceph.osd.11.smithi191.stdout:Mar 04 09:43:26 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-11[62160]: 2024-03-04T09:43:26.331+0000 7fdceac97700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-03-04T09:42:41.857812+0000 front 2024-03-04T09:42:41.857710+0000 (oldest deadline 2024-03-04T09:43:03.557748+0000) 2024-03-04T09:43:26.589 INFO:journalctl@ceph.osd.11.smithi191.stdout:Mar 04 09:43:26 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-11[62160]: 2024-03-04T09:43:26.331+0000 7fdceac97700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.191:6812 osd.5 since back 2024-03-04T09:42:46.959192+0000 front 2024-03-04T09:42:46.959105+0000 (oldest deadline 2024-03-04T09:43:12.859230+0000) 2024-03-04T09:43:26.589 INFO:journalctl@ceph.osd.11.smithi191.stdout:Mar 04 09:43:26 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-11[62160]: 2024-03-04T09:43:26.331+0000 7fdceac97700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.52:6822 osd.6 since back 2024-03-04T09:42:56.260599+0000 front 2024-03-04T09:42:56.260647+0000 (oldest deadline 2024-03-04T09:43:22.160568+0000) 2024-03-04T09:43:26.883 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 04 09:43:26 smithi106 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-10[60969]: 2024-03-04T09:43:26.628+0000 7f10f0d39700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-03-04T09:42:08.466029+0000 front 2024-03-04T09:42:08.465963+0000 (oldest deadline 2024-03-04T09:42:32.566054+0000) 2024-03-04T09:43:26.883 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 04 09:43:26 smithi106 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-10[60969]: 2024-03-04T09:43:26.628+0000 7f10f0d39700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-04T09:42:15.467191+0000 front 2024-03-04T09:42:15.467152+0000 (oldest deadline 2024-03-04T09:42:39.567351+0000) 2024-03-04T09:43:26.883 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 04 09:43:26 smithi106 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-10[60969]: 2024-03-04T09:43:26.628+0000 7f10f0d39700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-03-04T09:42:25.368854+0000 front 2024-03-04T09:42:25.368833+0000 (oldest deadline 2024-03-04T09:42:48.868990+0000) 2024-03-04T09:43:26.883 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 04 09:43:26 smithi106 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-10[60969]: 2024-03-04T09:43:26.628+0000 7f10f0d39700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2024-03-04T09:42:28.869679+0000 front 2024-03-04T09:42:28.869692+0000 (oldest deadline 2024-03-04T09:42:55.270190+0000) 2024-03-04T09:43:26.883 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 04 09:43:26 smithi106 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-10[60969]: 2024-03-04T09:43:26.628+0000 7f10f0d39700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-03-04T09:42:42.271376+0000 front 2024-03-04T09:42:42.271331+0000 (oldest deadline 2024-03-04T09:43:07.571444+0000) 2024-03-04T09:43:26.883 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 04 09:43:26 smithi106 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-10[60969]: 2024-03-04T09:43:26.628+0000 7f10f0d39700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.191:6812 osd.5 since back 2024-03-04T09:42:49.272525+0000 front 2024-03-04T09:42:49.272394+0000 (oldest deadline 2024-03-04T09:43:10.872863+0000) 2024-03-04T09:43:26.883 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 04 09:43:26 smithi106 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-10[60969]: 2024-03-04T09:43:26.628+0000 7f10f0d39700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.52:6822 osd.6 since back 2024-03-04T09:42:55.974740+0000 front 2024-03-04T09:42:55.974666+0000 (oldest deadline 2024-03-04T09:43:21.874604+0000) 2024-03-04T09:43:27.765 INFO:journalctl@ceph.osd.11.smithi191.stdout:Mar 04 09:43:27 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-11[62160]: 2024-03-04T09:43:27.362+0000 7fdceac97700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-03-04T09:42:08.553313+0000 front 2024-03-04T09:42:08.553176+0000 (oldest deadline 2024-03-04T09:42:32.653011+0000) 2024-03-04T09:43:27.766 INFO:journalctl@ceph.osd.11.smithi191.stdout:Mar 04 09:43:27 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-11[62160]: 2024-03-04T09:43:27.362+0000 7fdceac97700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-04T09:42:18.355200+0000 front 2024-03-04T09:42:18.355189+0000 (oldest deadline 2024-03-04T09:42:44.254936+0000) 2024-03-04T09:43:27.766 INFO:journalctl@ceph.osd.11.smithi191.stdout:Mar 04 09:43:27 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-11[62160]: 2024-03-04T09:43:27.362+0000 7fdceac97700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-03-04T09:42:24.255361+0000 front 2024-03-04T09:42:24.255246+0000 (oldest deadline 2024-03-04T09:42:47.155465+0000) 2024-03-04T09:43:27.766 INFO:journalctl@ceph.osd.11.smithi191.stdout:Mar 04 09:43:27 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-11[62160]: 2024-03-04T09:43:27.362+0000 7fdceac97700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-03-04T09:42:41.857812+0000 front 2024-03-04T09:42:41.857710+0000 (oldest deadline 2024-03-04T09:43:03.557748+0000) 2024-03-04T09:43:27.766 INFO:journalctl@ceph.osd.11.smithi191.stdout:Mar 04 09:43:27 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-11[62160]: 2024-03-04T09:43:27.362+0000 7fdceac97700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.191:6812 osd.5 since back 2024-03-04T09:42:46.959192+0000 front 2024-03-04T09:42:46.959105+0000 (oldest deadline 2024-03-04T09:43:12.859230+0000) 2024-03-04T09:43:27.766 INFO:journalctl@ceph.osd.11.smithi191.stdout:Mar 04 09:43:27 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-11[62160]: 2024-03-04T09:43:27.362+0000 7fdceac97700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.52:6822 osd.6 since back 2024-03-04T09:42:56.260599+0000 front 2024-03-04T09:42:56.260647+0000 (oldest deadline 2024-03-04T09:43:22.160568+0000) 2024-03-04T09:43:27.883 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 04 09:43:27 smithi106 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-10[60969]: 2024-03-04T09:43:27.619+0000 7f10f0d39700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-03-04T09:42:08.466029+0000 front 2024-03-04T09:42:08.465963+0000 (oldest deadline 2024-03-04T09:42:32.566054+0000) 2024-03-04T09:43:27.883 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 04 09:43:27 smithi106 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-10[60969]: 2024-03-04T09:43:27.619+0000 7f10f0d39700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-04T09:42:15.467191+0000 front 2024-03-04T09:42:15.467152+0000 (oldest deadline 2024-03-04T09:42:39.567351+0000) 2024-03-04T09:43:27.883 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 04 09:43:27 smithi106 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-10[60969]: 2024-03-04T09:43:27.619+0000 7f10f0d39700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-03-04T09:42:25.368854+0000 front 2024-03-04T09:42:25.368833+0000 (oldest deadline 2024-03-04T09:42:48.868990+0000) 2024-03-04T09:43:27.884 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 04 09:43:27 smithi106 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-10[60969]: 2024-03-04T09:43:27.619+0000 7f10f0d39700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2024-03-04T09:42:28.869679+0000 front 2024-03-04T09:42:28.869692+0000 (oldest deadline 2024-03-04T09:42:55.270190+0000) 2024-03-04T09:43:27.884 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 04 09:43:27 smithi106 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-10[60969]: 2024-03-04T09:43:27.619+0000 7f10f0d39700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-03-04T09:42:42.271376+0000 front 2024-03-04T09:42:42.271331+0000 (oldest deadline 2024-03-04T09:43:07.571444+0000) 2024-03-04T09:43:27.884 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 04 09:43:27 smithi106 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-10[60969]: 2024-03-04T09:43:27.619+0000 7f10f0d39700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.191:6812 osd.5 since back 2024-03-04T09:42:49.272525+0000 front 2024-03-04T09:42:49.272394+0000 (oldest deadline 2024-03-04T09:43:10.872863+0000) 2024-03-04T09:43:27.884 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 04 09:43:27 smithi106 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-10[60969]: 2024-03-04T09:43:27.619+0000 7f10f0d39700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.52:6822 osd.6 since back 2024-03-04T09:42:55.974740+0000 front 2024-03-04T09:42:55.974666+0000 (oldest deadline 2024-03-04T09:43:21.874604+0000) 2024-03-04T09:43:28.688 INFO:journalctl@ceph.osd.11.smithi191.stdout:Mar 04 09:43:28 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-11[62160]: 2024-03-04T09:43:28.388+0000 7fdceac97700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-03-04T09:42:08.553313+0000 front 2024-03-04T09:42:08.553176+0000 (oldest deadline 2024-03-04T09:42:32.653011+0000) 2024-03-04T09:43:28.688 INFO:journalctl@ceph.osd.11.smithi191.stdout:Mar 04 09:43:28 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-11[62160]: 2024-03-04T09:43:28.388+0000 7fdceac97700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-04T09:42:18.355200+0000 front 2024-03-04T09:42:18.355189+0000 (oldest deadline 2024-03-04T09:42:44.254936+0000) 2024-03-04T09:43:28.688 INFO:journalctl@ceph.osd.11.smithi191.stdout:Mar 04 09:43:28 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-11[62160]: 2024-03-04T09:43:28.388+0000 7fdceac97700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-03-04T09:42:24.255361+0000 front 2024-03-04T09:42:24.255246+0000 (oldest deadline 2024-03-04T09:42:47.155465+0000) 2024-03-04T09:43:28.689 INFO:journalctl@ceph.osd.11.smithi191.stdout:Mar 04 09:43:28 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-11[62160]: 2024-03-04T09:43:28.388+0000 7fdceac97700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-03-04T09:42:41.857812+0000 front 2024-03-04T09:42:41.857710+0000 (oldest deadline 2024-03-04T09:43:03.557748+0000) 2024-03-04T09:43:28.689 INFO:journalctl@ceph.osd.11.smithi191.stdout:Mar 04 09:43:28 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-11[62160]: 2024-03-04T09:43:28.388+0000 7fdceac97700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.191:6812 osd.5 since back 2024-03-04T09:42:46.959192+0000 front 2024-03-04T09:42:46.959105+0000 (oldest deadline 2024-03-04T09:43:12.859230+0000) 2024-03-04T09:43:28.689 INFO:journalctl@ceph.osd.11.smithi191.stdout:Mar 04 09:43:28 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-11[62160]: 2024-03-04T09:43:28.388+0000 7fdceac97700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.52:6822 osd.6 since back 2024-03-04T09:42:56.260599+0000 front 2024-03-04T09:42:56.260647+0000 (oldest deadline 2024-03-04T09:43:22.160568+0000) 2024-03-04T09:43:28.689 INFO:journalctl@ceph.osd.11.smithi191.stdout:Mar 04 09:43:28 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-11[62160]: 2024-03-04T09:43:28.388+0000 7fdceac97700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.106:6820 osd.7 since back 2024-03-04T09:43:02.161173+0000 front 2024-03-04T09:43:02.161205+0000 (oldest deadline 2024-03-04T09:43:28.061162+0000) 2024-03-04T09:43:29.133 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 04 09:43:28 smithi106 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-10[60969]: 2024-03-04T09:43:28.651+0000 7f10f0d39700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-03-04T09:42:08.466029+0000 front 2024-03-04T09:42:08.465963+0000 (oldest deadline 2024-03-04T09:42:32.566054+0000) 2024-03-04T09:43:29.133 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 04 09:43:28 smithi106 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-10[60969]: 2024-03-04T09:43:28.651+0000 7f10f0d39700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-04T09:42:15.467191+0000 front 2024-03-04T09:42:15.467152+0000 (oldest deadline 2024-03-04T09:42:39.567351+0000) 2024-03-04T09:43:29.133 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 04 09:43:28 smithi106 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-10[60969]: 2024-03-04T09:43:28.651+0000 7f10f0d39700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-03-04T09:42:25.368854+0000 front 2024-03-04T09:42:25.368833+0000 (oldest deadline 2024-03-04T09:42:48.868990+0000) 2024-03-04T09:43:29.133 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 04 09:43:28 smithi106 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-10[60969]: 2024-03-04T09:43:28.651+0000 7f10f0d39700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2024-03-04T09:42:28.869679+0000 front 2024-03-04T09:42:28.869692+0000 (oldest deadline 2024-03-04T09:42:55.270190+0000) 2024-03-04T09:43:29.134 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 04 09:43:28 smithi106 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-10[60969]: 2024-03-04T09:43:28.651+0000 7f10f0d39700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-03-04T09:42:42.271376+0000 front 2024-03-04T09:42:42.271331+0000 (oldest deadline 2024-03-04T09:43:07.571444+0000) 2024-03-04T09:43:29.134 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 04 09:43:28 smithi106 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-10[60969]: 2024-03-04T09:43:28.651+0000 7f10f0d39700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.191:6812 osd.5 since back 2024-03-04T09:42:49.272525+0000 front 2024-03-04T09:42:49.272394+0000 (oldest deadline 2024-03-04T09:43:10.872863+0000) 2024-03-04T09:43:29.134 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 04 09:43:28 smithi106 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-10[60969]: 2024-03-04T09:43:28.651+0000 7f10f0d39700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.52:6822 osd.6 since back 2024-03-04T09:42:55.974740+0000 front 2024-03-04T09:42:55.974666+0000 (oldest deadline 2024-03-04T09:43:21.874604+0000) 2024-03-04T09:43:29.838 INFO:journalctl@ceph.osd.11.smithi191.stdout:Mar 04 09:43:29 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-11[62160]: 2024-03-04T09:43:29.414+0000 7fdceac97700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-03-04T09:42:08.553313+0000 front 2024-03-04T09:42:08.553176+0000 (oldest deadline 2024-03-04T09:42:32.653011+0000) 2024-03-04T09:43:29.838 INFO:journalctl@ceph.osd.11.smithi191.stdout:Mar 04 09:43:29 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-11[62160]: 2024-03-04T09:43:29.414+0000 7fdceac97700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-04T09:42:18.355200+0000 front 2024-03-04T09:42:18.355189+0000 (oldest deadline 2024-03-04T09:42:44.254936+0000) 2024-03-04T09:43:29.838 INFO:journalctl@ceph.osd.11.smithi191.stdout:Mar 04 09:43:29 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-11[62160]: 2024-03-04T09:43:29.414+0000 7fdceac97700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-03-04T09:42:24.255361+0000 front 2024-03-04T09:42:24.255246+0000 (oldest deadline 2024-03-04T09:42:47.155465+0000) 2024-03-04T09:43:29.839 INFO:journalctl@ceph.osd.11.smithi191.stdout:Mar 04 09:43:29 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-11[62160]: 2024-03-04T09:43:29.414+0000 7fdceac97700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-03-04T09:42:41.857812+0000 front 2024-03-04T09:42:41.857710+0000 (oldest deadline 2024-03-04T09:43:03.557748+0000) 2024-03-04T09:43:29.839 INFO:journalctl@ceph.osd.11.smithi191.stdout:Mar 04 09:43:29 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-11[62160]: 2024-03-04T09:43:29.414+0000 7fdceac97700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.191:6812 osd.5 since back 2024-03-04T09:42:46.959192+0000 front 2024-03-04T09:42:46.959105+0000 (oldest deadline 2024-03-04T09:43:12.859230+0000) 2024-03-04T09:43:29.839 INFO:journalctl@ceph.osd.11.smithi191.stdout:Mar 04 09:43:29 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-11[62160]: 2024-03-04T09:43:29.414+0000 7fdceac97700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.52:6822 osd.6 since back 2024-03-04T09:42:56.260599+0000 front 2024-03-04T09:42:56.260647+0000 (oldest deadline 2024-03-04T09:43:22.160568+0000) 2024-03-04T09:43:29.839 INFO:journalctl@ceph.osd.11.smithi191.stdout:Mar 04 09:43:29 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-11[62160]: 2024-03-04T09:43:29.414+0000 7fdceac97700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.106:6820 osd.7 since back 2024-03-04T09:43:02.161173+0000 front 2024-03-04T09:43:02.161205+0000 (oldest deadline 2024-03-04T09:43:28.061162+0000) 2024-03-04T09:43:30.133 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 04 09:43:29 smithi106 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-10[60969]: 2024-03-04T09:43:29.690+0000 7f10f0d39700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-03-04T09:42:08.466029+0000 front 2024-03-04T09:42:08.465963+0000 (oldest deadline 2024-03-04T09:42:32.566054+0000) 2024-03-04T09:43:30.133 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 04 09:43:29 smithi106 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-10[60969]: 2024-03-04T09:43:29.690+0000 7f10f0d39700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-04T09:42:15.467191+0000 front 2024-03-04T09:42:15.467152+0000 (oldest deadline 2024-03-04T09:42:39.567351+0000) 2024-03-04T09:43:30.133 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 04 09:43:29 smithi106 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-10[60969]: 2024-03-04T09:43:29.690+0000 7f10f0d39700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-03-04T09:42:25.368854+0000 front 2024-03-04T09:42:25.368833+0000 (oldest deadline 2024-03-04T09:42:48.868990+0000) 2024-03-04T09:43:30.133 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 04 09:43:29 smithi106 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-10[60969]: 2024-03-04T09:43:29.690+0000 7f10f0d39700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2024-03-04T09:42:28.869679+0000 front 2024-03-04T09:42:28.869692+0000 (oldest deadline 2024-03-04T09:42:55.270190+0000) 2024-03-04T09:43:30.134 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 04 09:43:29 smithi106 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-10[60969]: 2024-03-04T09:43:29.690+0000 7f10f0d39700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-03-04T09:42:42.271376+0000 front 2024-03-04T09:42:42.271331+0000 (oldest deadline 2024-03-04T09:43:07.571444+0000) 2024-03-04T09:43:30.134 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 04 09:43:29 smithi106 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-10[60969]: 2024-03-04T09:43:29.690+0000 7f10f0d39700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.191:6812 osd.5 since back 2024-03-04T09:42:49.272525+0000 front 2024-03-04T09:42:49.272394+0000 (oldest deadline 2024-03-04T09:43:10.872863+0000) 2024-03-04T09:43:30.134 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 04 09:43:29 smithi106 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-10[60969]: 2024-03-04T09:43:29.690+0000 7f10f0d39700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.52:6822 osd.6 since back 2024-03-04T09:42:55.974740+0000 front 2024-03-04T09:42:55.974666+0000 (oldest deadline 2024-03-04T09:43:21.874604+0000) 2024-03-04T09:43:30.633 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 04 09:43:30 smithi106 podman[72874]: 2024-03-04 09:43:30.360735341 +0000 UTC m=+5.585127567 container died 2cdd66ca240bd75ea42e3b44b3b39d1f1cf698d1fe2a84e30103bcfd390b9ba8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5, name=ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-10, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, org.label-schema.build-date=20240222, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.33.5, RELEASE=wip-lusov-qdb-1bc250c, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-03-04T09:43:30.838 INFO:journalctl@ceph.osd.11.smithi191.stdout:Mar 04 09:43:30 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-11[62160]: 2024-03-04T09:43:30.448+0000 7fdceac97700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-03-04T09:42:08.553313+0000 front 2024-03-04T09:42:08.553176+0000 (oldest deadline 2024-03-04T09:42:32.653011+0000) 2024-03-04T09:43:30.838 INFO:journalctl@ceph.osd.11.smithi191.stdout:Mar 04 09:43:30 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-11[62160]: 2024-03-04T09:43:30.448+0000 7fdceac97700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-04T09:42:18.355200+0000 front 2024-03-04T09:42:18.355189+0000 (oldest deadline 2024-03-04T09:42:44.254936+0000) 2024-03-04T09:43:30.838 INFO:journalctl@ceph.osd.11.smithi191.stdout:Mar 04 09:43:30 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-11[62160]: 2024-03-04T09:43:30.448+0000 7fdceac97700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-03-04T09:42:24.255361+0000 front 2024-03-04T09:42:24.255246+0000 (oldest deadline 2024-03-04T09:42:47.155465+0000) 2024-03-04T09:43:30.838 INFO:journalctl@ceph.osd.11.smithi191.stdout:Mar 04 09:43:30 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-11[62160]: 2024-03-04T09:43:30.448+0000 7fdceac97700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-03-04T09:42:41.857812+0000 front 2024-03-04T09:42:41.857710+0000 (oldest deadline 2024-03-04T09:43:03.557748+0000) 2024-03-04T09:43:30.838 INFO:journalctl@ceph.osd.11.smithi191.stdout:Mar 04 09:43:30 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-11[62160]: 2024-03-04T09:43:30.448+0000 7fdceac97700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.191:6812 osd.5 since back 2024-03-04T09:42:46.959192+0000 front 2024-03-04T09:42:46.959105+0000 (oldest deadline 2024-03-04T09:43:12.859230+0000) 2024-03-04T09:43:30.839 INFO:journalctl@ceph.osd.11.smithi191.stdout:Mar 04 09:43:30 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-11[62160]: 2024-03-04T09:43:30.448+0000 7fdceac97700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.52:6822 osd.6 since back 2024-03-04T09:42:56.260599+0000 front 2024-03-04T09:42:56.260647+0000 (oldest deadline 2024-03-04T09:43:22.160568+0000) 2024-03-04T09:43:30.839 INFO:journalctl@ceph.osd.11.smithi191.stdout:Mar 04 09:43:30 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-11[62160]: 2024-03-04T09:43:30.448+0000 7fdceac97700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.106:6820 osd.7 since back 2024-03-04T09:43:02.161173+0000 front 2024-03-04T09:43:02.161205+0000 (oldest deadline 2024-03-04T09:43:28.061162+0000) 2024-03-04T09:43:31.133 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 04 09:43:30 smithi106 podman[72874]: 2024-03-04 09:43:30.648398232 +0000 UTC m=+5.872790459 container cleanup 2cdd66ca240bd75ea42e3b44b3b39d1f1cf698d1fe2a84e30103bcfd390b9ba8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5, name=ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-10, RELEASE=wip-lusov-qdb-1bc250c, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, org.label-schema.build-date=20240222, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-03-04T09:43:31.133 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 04 09:43:30 smithi106 bash[72874]: ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-10 2024-03-04T09:43:31.133 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 04 09:43:30 smithi106 podman[72891]: 2024-03-04 09:43:30.874510721 +0000 UTC m=+0.514416307 container remove 2cdd66ca240bd75ea42e3b44b3b39d1f1cf698d1fe2a84e30103bcfd390b9ba8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5, name=ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-10, org.label-schema.build-date=20240222, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=wip-lusov-qdb-1bc250c, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-04T09:43:31.467 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 04 09:43:31 smithi106 podman[72980]: 2024-03-04 09:43:31.106860993 +0000 UTC m=+0.017809714 image pull f88ffb32aa6a7d5dfdc954d9f096f0c9cc47981d05ec8d117ba859a22d8185ba quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5 2024-03-04T09:43:31.467 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 04 09:43:31 smithi106 podman[72980]: 2024-03-04 09:43:31.225490685 +0000 UTC m=+0.136439410 container create 0e09775ba9bd46ee8dd51c70bba6a4923dadd1ad4480da343845c41cd8cf08e0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5, name=ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-10-deactivate, maintainer=Guillaume Abrioux , GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, ceph=True, RELEASE=wip-lusov-qdb-1bc250c, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.build-date=20240222, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-03-04T09:43:31.467 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 04 09:43:31 smithi106 podman[72980]: 2024-03-04 09:43:31.469470769 +0000 UTC m=+0.380419478 container init 0e09775ba9bd46ee8dd51c70bba6a4923dadd1ad4480da343845c41cd8cf08e0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5, name=ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-10-deactivate, ceph=True, RELEASE=wip-lusov-qdb-1bc250c, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.build-date=20240222, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-04T09:43:31.759 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 04 09:43:31 smithi106 podman[72980]: 2024-03-04 09:43:31.472701974 +0000 UTC m=+0.383650673 container start 0e09775ba9bd46ee8dd51c70bba6a4923dadd1ad4480da343845c41cd8cf08e0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5, name=ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-10-deactivate, RELEASE=wip-lusov-qdb-1bc250c, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, io.buildah.version=1.33.5, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240222) 2024-03-04T09:43:31.759 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 04 09:43:31 smithi106 podman[72980]: 2024-03-04 09:43:31.560848735 +0000 UTC m=+0.471797453 container attach 0e09775ba9bd46ee8dd51c70bba6a4923dadd1ad4480da343845c41cd8cf08e0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5, name=ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-10-deactivate, RELEASE=wip-lusov-qdb-1bc250c, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240222, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0) 2024-03-04T09:43:31.838 INFO:journalctl@ceph.osd.11.smithi191.stdout:Mar 04 09:43:31 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-11[62160]: 2024-03-04T09:43:31.494+0000 7fdceac97700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-03-04T09:42:08.553313+0000 front 2024-03-04T09:42:08.553176+0000 (oldest deadline 2024-03-04T09:42:32.653011+0000) 2024-03-04T09:43:31.838 INFO:journalctl@ceph.osd.11.smithi191.stdout:Mar 04 09:43:31 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-11[62160]: 2024-03-04T09:43:31.494+0000 7fdceac97700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-04T09:42:18.355200+0000 front 2024-03-04T09:42:18.355189+0000 (oldest deadline 2024-03-04T09:42:44.254936+0000) 2024-03-04T09:43:31.838 INFO:journalctl@ceph.osd.11.smithi191.stdout:Mar 04 09:43:31 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-11[62160]: 2024-03-04T09:43:31.494+0000 7fdceac97700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-03-04T09:42:24.255361+0000 front 2024-03-04T09:42:24.255246+0000 (oldest deadline 2024-03-04T09:42:47.155465+0000) 2024-03-04T09:43:31.838 INFO:journalctl@ceph.osd.11.smithi191.stdout:Mar 04 09:43:31 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-11[62160]: 2024-03-04T09:43:31.494+0000 7fdceac97700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-03-04T09:42:41.857812+0000 front 2024-03-04T09:42:41.857710+0000 (oldest deadline 2024-03-04T09:43:03.557748+0000) 2024-03-04T09:43:31.838 INFO:journalctl@ceph.osd.11.smithi191.stdout:Mar 04 09:43:31 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-11[62160]: 2024-03-04T09:43:31.494+0000 7fdceac97700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.191:6812 osd.5 since back 2024-03-04T09:42:46.959192+0000 front 2024-03-04T09:42:46.959105+0000 (oldest deadline 2024-03-04T09:43:12.859230+0000) 2024-03-04T09:43:31.839 INFO:journalctl@ceph.osd.11.smithi191.stdout:Mar 04 09:43:31 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-11[62160]: 2024-03-04T09:43:31.494+0000 7fdceac97700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.52:6822 osd.6 since back 2024-03-04T09:42:56.260599+0000 front 2024-03-04T09:42:56.260647+0000 (oldest deadline 2024-03-04T09:43:22.160568+0000) 2024-03-04T09:43:31.839 INFO:journalctl@ceph.osd.11.smithi191.stdout:Mar 04 09:43:31 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-11[62160]: 2024-03-04T09:43:31.494+0000 7fdceac97700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.106:6820 osd.7 since back 2024-03-04T09:43:02.161173+0000 front 2024-03-04T09:43:02.161205+0000 (oldest deadline 2024-03-04T09:43:28.061162+0000) 2024-03-04T09:43:32.036 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 04 09:43:31 smithi106 podman[72980]: 2024-03-04 09:43:31.761472943 +0000 UTC m=+0.672421647 container died 0e09775ba9bd46ee8dd51c70bba6a4923dadd1ad4480da343845c41cd8cf08e0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5, name=ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-10-deactivate, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.build-date=20240222, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-qdb-1bc250c) 2024-03-04T09:43:32.278 DEBUG:teuthology.orchestra.run.smithi106:> sudo pkill -f 'journalctl -f -n 0 -u ceph-4e059f08-da08-11ee-95c6-87774f69a715@osd.10.service' 2024-03-04T09:43:32.305 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 04 09:43:32 smithi106 podman[73006]: 2024-03-04 09:43:32.264001316 +0000 UTC m=+0.497309183 container remove 0e09775ba9bd46ee8dd51c70bba6a4923dadd1ad4480da343845c41cd8cf08e0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5, name=ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-10-deactivate, GIT_BRANCH=HEAD, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-qdb-1bc250c, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.build-date=20240222, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-04T09:43:32.305 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 04 09:43:32 smithi106 systemd[1]: ceph-4e059f08-da08-11ee-95c6-87774f69a715@osd.10.service: Deactivated successfully. 2024-03-04T09:43:32.306 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 04 09:43:32 smithi106 systemd[1]: Stopped Ceph osd.10 for 4e059f08-da08-11ee-95c6-87774f69a715. 2024-03-04T09:43:32.306 INFO:journalctl@ceph.osd.10.smithi106.stdout:Mar 04 09:43:32 smithi106 systemd[1]: ceph-4e059f08-da08-11ee-95c6-87774f69a715@osd.10.service: Consumed 13.870s CPU time. 2024-03-04T09:43:32.321 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-04T09:43:32.321 INFO:tasks.cephadm.osd.10:Stopped osd.10 2024-03-04T09:43:32.321 INFO:tasks.cephadm.osd.11:Stopping osd.11... 2024-03-04T09:43:32.322 DEBUG:teuthology.orchestra.run.smithi191:> sudo systemctl stop ceph-4e059f08-da08-11ee-95c6-87774f69a715@osd.11 2024-03-04T09:43:32.632 INFO:journalctl@ceph.osd.11.smithi191.stdout:Mar 04 09:43:32 smithi191 systemd[1]: Stopping Ceph osd.11 for 4e059f08-da08-11ee-95c6-87774f69a715... 2024-03-04T09:43:32.633 INFO:journalctl@ceph.osd.11.smithi191.stdout:Mar 04 09:43:32 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-11[62160]: 2024-03-04T09:43:32.505+0000 7fdceac97700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-03-04T09:42:08.553313+0000 front 2024-03-04T09:42:08.553176+0000 (oldest deadline 2024-03-04T09:42:32.653011+0000) 2024-03-04T09:43:32.633 INFO:journalctl@ceph.osd.11.smithi191.stdout:Mar 04 09:43:32 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-11[62160]: 2024-03-04T09:43:32.505+0000 7fdceac97700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-04T09:42:18.355200+0000 front 2024-03-04T09:42:18.355189+0000 (oldest deadline 2024-03-04T09:42:44.254936+0000) 2024-03-04T09:43:32.633 INFO:journalctl@ceph.osd.11.smithi191.stdout:Mar 04 09:43:32 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-11[62160]: 2024-03-04T09:43:32.505+0000 7fdceac97700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-03-04T09:42:24.255361+0000 front 2024-03-04T09:42:24.255246+0000 (oldest deadline 2024-03-04T09:42:47.155465+0000) 2024-03-04T09:43:32.633 INFO:journalctl@ceph.osd.11.smithi191.stdout:Mar 04 09:43:32 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-11[62160]: 2024-03-04T09:43:32.505+0000 7fdceac97700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-03-04T09:42:41.857812+0000 front 2024-03-04T09:42:41.857710+0000 (oldest deadline 2024-03-04T09:43:03.557748+0000) 2024-03-04T09:43:32.633 INFO:journalctl@ceph.osd.11.smithi191.stdout:Mar 04 09:43:32 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-11[62160]: 2024-03-04T09:43:32.505+0000 7fdceac97700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.191:6812 osd.5 since back 2024-03-04T09:42:46.959192+0000 front 2024-03-04T09:42:46.959105+0000 (oldest deadline 2024-03-04T09:43:12.859230+0000) 2024-03-04T09:43:32.633 INFO:journalctl@ceph.osd.11.smithi191.stdout:Mar 04 09:43:32 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-11[62160]: 2024-03-04T09:43:32.505+0000 7fdceac97700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.52:6822 osd.6 since back 2024-03-04T09:42:56.260599+0000 front 2024-03-04T09:42:56.260647+0000 (oldest deadline 2024-03-04T09:43:22.160568+0000) 2024-03-04T09:43:32.634 INFO:journalctl@ceph.osd.11.smithi191.stdout:Mar 04 09:43:32 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-11[62160]: 2024-03-04T09:43:32.505+0000 7fdceac97700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.106:6820 osd.7 since back 2024-03-04T09:43:02.161173+0000 front 2024-03-04T09:43:02.161205+0000 (oldest deadline 2024-03-04T09:43:28.061162+0000) 2024-03-04T09:43:32.634 INFO:journalctl@ceph.osd.11.smithi191.stdout:Mar 04 09:43:32 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-11[62160]: 2024-03-04T09:43:32.633+0000 7fdcee6a3700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.11 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-04T09:43:32.634 INFO:journalctl@ceph.osd.11.smithi191.stdout:Mar 04 09:43:32 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-11[62160]: 2024-03-04T09:43:32.633+0000 7fdcee6a3700 -1 osd.11 106 *** Got signal Terminated *** 2024-03-04T09:43:32.634 INFO:journalctl@ceph.osd.11.smithi191.stdout:Mar 04 09:43:32 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-11[62160]: 2024-03-04T09:43:32.633+0000 7fdcee6a3700 -1 osd.11 106 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-04T09:43:33.838 INFO:journalctl@ceph.osd.11.smithi191.stdout:Mar 04 09:43:33 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-11[62160]: 2024-03-04T09:43:33.507+0000 7fdceac97700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-03-04T09:42:08.553313+0000 front 2024-03-04T09:42:08.553176+0000 (oldest deadline 2024-03-04T09:42:32.653011+0000) 2024-03-04T09:43:33.838 INFO:journalctl@ceph.osd.11.smithi191.stdout:Mar 04 09:43:33 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-11[62160]: 2024-03-04T09:43:33.507+0000 7fdceac97700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-04T09:42:18.355200+0000 front 2024-03-04T09:42:18.355189+0000 (oldest deadline 2024-03-04T09:42:44.254936+0000) 2024-03-04T09:43:33.839 INFO:journalctl@ceph.osd.11.smithi191.stdout:Mar 04 09:43:33 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-11[62160]: 2024-03-04T09:43:33.507+0000 7fdceac97700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-03-04T09:42:24.255361+0000 front 2024-03-04T09:42:24.255246+0000 (oldest deadline 2024-03-04T09:42:47.155465+0000) 2024-03-04T09:43:33.839 INFO:journalctl@ceph.osd.11.smithi191.stdout:Mar 04 09:43:33 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-11[62160]: 2024-03-04T09:43:33.507+0000 7fdceac97700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-03-04T09:42:41.857812+0000 front 2024-03-04T09:42:41.857710+0000 (oldest deadline 2024-03-04T09:43:03.557748+0000) 2024-03-04T09:43:33.839 INFO:journalctl@ceph.osd.11.smithi191.stdout:Mar 04 09:43:33 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-11[62160]: 2024-03-04T09:43:33.507+0000 7fdceac97700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.191:6812 osd.5 since back 2024-03-04T09:42:46.959192+0000 front 2024-03-04T09:42:46.959105+0000 (oldest deadline 2024-03-04T09:43:12.859230+0000) 2024-03-04T09:43:33.839 INFO:journalctl@ceph.osd.11.smithi191.stdout:Mar 04 09:43:33 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-11[62160]: 2024-03-04T09:43:33.507+0000 7fdceac97700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.52:6822 osd.6 since back 2024-03-04T09:42:56.260599+0000 front 2024-03-04T09:42:56.260647+0000 (oldest deadline 2024-03-04T09:43:22.160568+0000) 2024-03-04T09:43:33.839 INFO:journalctl@ceph.osd.11.smithi191.stdout:Mar 04 09:43:33 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-11[62160]: 2024-03-04T09:43:33.507+0000 7fdceac97700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.106:6820 osd.7 since back 2024-03-04T09:43:02.161173+0000 front 2024-03-04T09:43:02.161205+0000 (oldest deadline 2024-03-04T09:43:28.061162+0000) 2024-03-04T09:43:34.838 INFO:journalctl@ceph.osd.11.smithi191.stdout:Mar 04 09:43:34 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-11[62160]: 2024-03-04T09:43:34.511+0000 7fdceac97700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-03-04T09:42:08.553313+0000 front 2024-03-04T09:42:08.553176+0000 (oldest deadline 2024-03-04T09:42:32.653011+0000) 2024-03-04T09:43:34.838 INFO:journalctl@ceph.osd.11.smithi191.stdout:Mar 04 09:43:34 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-11[62160]: 2024-03-04T09:43:34.511+0000 7fdceac97700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-04T09:42:18.355200+0000 front 2024-03-04T09:42:18.355189+0000 (oldest deadline 2024-03-04T09:42:44.254936+0000) 2024-03-04T09:43:34.838 INFO:journalctl@ceph.osd.11.smithi191.stdout:Mar 04 09:43:34 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-11[62160]: 2024-03-04T09:43:34.511+0000 7fdceac97700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-03-04T09:42:24.255361+0000 front 2024-03-04T09:42:24.255246+0000 (oldest deadline 2024-03-04T09:42:47.155465+0000) 2024-03-04T09:43:34.838 INFO:journalctl@ceph.osd.11.smithi191.stdout:Mar 04 09:43:34 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-11[62160]: 2024-03-04T09:43:34.511+0000 7fdceac97700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-03-04T09:42:41.857812+0000 front 2024-03-04T09:42:41.857710+0000 (oldest deadline 2024-03-04T09:43:03.557748+0000) 2024-03-04T09:43:34.839 INFO:journalctl@ceph.osd.11.smithi191.stdout:Mar 04 09:43:34 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-11[62160]: 2024-03-04T09:43:34.511+0000 7fdceac97700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.191:6812 osd.5 since back 2024-03-04T09:42:46.959192+0000 front 2024-03-04T09:42:46.959105+0000 (oldest deadline 2024-03-04T09:43:12.859230+0000) 2024-03-04T09:43:34.839 INFO:journalctl@ceph.osd.11.smithi191.stdout:Mar 04 09:43:34 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-11[62160]: 2024-03-04T09:43:34.511+0000 7fdceac97700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.52:6822 osd.6 since back 2024-03-04T09:42:56.260599+0000 front 2024-03-04T09:42:56.260647+0000 (oldest deadline 2024-03-04T09:43:22.160568+0000) 2024-03-04T09:43:34.839 INFO:journalctl@ceph.osd.11.smithi191.stdout:Mar 04 09:43:34 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-11[62160]: 2024-03-04T09:43:34.511+0000 7fdceac97700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.106:6820 osd.7 since back 2024-03-04T09:43:02.161173+0000 front 2024-03-04T09:43:02.161205+0000 (oldest deadline 2024-03-04T09:43:28.061162+0000) 2024-03-04T09:43:35.838 INFO:journalctl@ceph.osd.11.smithi191.stdout:Mar 04 09:43:35 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-11[62160]: 2024-03-04T09:43:35.497+0000 7fdceac97700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-03-04T09:42:08.553313+0000 front 2024-03-04T09:42:08.553176+0000 (oldest deadline 2024-03-04T09:42:32.653011+0000) 2024-03-04T09:43:35.838 INFO:journalctl@ceph.osd.11.smithi191.stdout:Mar 04 09:43:35 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-11[62160]: 2024-03-04T09:43:35.497+0000 7fdceac97700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-04T09:42:18.355200+0000 front 2024-03-04T09:42:18.355189+0000 (oldest deadline 2024-03-04T09:42:44.254936+0000) 2024-03-04T09:43:35.838 INFO:journalctl@ceph.osd.11.smithi191.stdout:Mar 04 09:43:35 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-11[62160]: 2024-03-04T09:43:35.497+0000 7fdceac97700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-03-04T09:42:24.255361+0000 front 2024-03-04T09:42:24.255246+0000 (oldest deadline 2024-03-04T09:42:47.155465+0000) 2024-03-04T09:43:35.838 INFO:journalctl@ceph.osd.11.smithi191.stdout:Mar 04 09:43:35 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-11[62160]: 2024-03-04T09:43:35.497+0000 7fdceac97700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-03-04T09:42:41.857812+0000 front 2024-03-04T09:42:41.857710+0000 (oldest deadline 2024-03-04T09:43:03.557748+0000) 2024-03-04T09:43:35.839 INFO:journalctl@ceph.osd.11.smithi191.stdout:Mar 04 09:43:35 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-11[62160]: 2024-03-04T09:43:35.497+0000 7fdceac97700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.191:6812 osd.5 since back 2024-03-04T09:42:46.959192+0000 front 2024-03-04T09:42:46.959105+0000 (oldest deadline 2024-03-04T09:43:12.859230+0000) 2024-03-04T09:43:35.839 INFO:journalctl@ceph.osd.11.smithi191.stdout:Mar 04 09:43:35 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-11[62160]: 2024-03-04T09:43:35.497+0000 7fdceac97700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.52:6822 osd.6 since back 2024-03-04T09:42:56.260599+0000 front 2024-03-04T09:42:56.260647+0000 (oldest deadline 2024-03-04T09:43:22.160568+0000) 2024-03-04T09:43:35.839 INFO:journalctl@ceph.osd.11.smithi191.stdout:Mar 04 09:43:35 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-11[62160]: 2024-03-04T09:43:35.497+0000 7fdceac97700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.106:6820 osd.7 since back 2024-03-04T09:43:02.161173+0000 front 2024-03-04T09:43:02.161205+0000 (oldest deadline 2024-03-04T09:43:28.061162+0000) 2024-03-04T09:43:36.838 INFO:journalctl@ceph.osd.11.smithi191.stdout:Mar 04 09:43:36 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-11[62160]: 2024-03-04T09:43:36.535+0000 7fdceac97700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-03-04T09:42:08.553313+0000 front 2024-03-04T09:42:08.553176+0000 (oldest deadline 2024-03-04T09:42:32.653011+0000) 2024-03-04T09:43:36.838 INFO:journalctl@ceph.osd.11.smithi191.stdout:Mar 04 09:43:36 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-11[62160]: 2024-03-04T09:43:36.535+0000 7fdceac97700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-04T09:42:18.355200+0000 front 2024-03-04T09:42:18.355189+0000 (oldest deadline 2024-03-04T09:42:44.254936+0000) 2024-03-04T09:43:36.838 INFO:journalctl@ceph.osd.11.smithi191.stdout:Mar 04 09:43:36 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-11[62160]: 2024-03-04T09:43:36.535+0000 7fdceac97700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-03-04T09:42:24.255361+0000 front 2024-03-04T09:42:24.255246+0000 (oldest deadline 2024-03-04T09:42:47.155465+0000) 2024-03-04T09:43:36.839 INFO:journalctl@ceph.osd.11.smithi191.stdout:Mar 04 09:43:36 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-11[62160]: 2024-03-04T09:43:36.535+0000 7fdceac97700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-03-04T09:42:41.857812+0000 front 2024-03-04T09:42:41.857710+0000 (oldest deadline 2024-03-04T09:43:03.557748+0000) 2024-03-04T09:43:36.839 INFO:journalctl@ceph.osd.11.smithi191.stdout:Mar 04 09:43:36 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-11[62160]: 2024-03-04T09:43:36.535+0000 7fdceac97700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.191:6812 osd.5 since back 2024-03-04T09:42:46.959192+0000 front 2024-03-04T09:42:46.959105+0000 (oldest deadline 2024-03-04T09:43:12.859230+0000) 2024-03-04T09:43:36.839 INFO:journalctl@ceph.osd.11.smithi191.stdout:Mar 04 09:43:36 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-11[62160]: 2024-03-04T09:43:36.535+0000 7fdceac97700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.52:6822 osd.6 since back 2024-03-04T09:42:56.260599+0000 front 2024-03-04T09:42:56.260647+0000 (oldest deadline 2024-03-04T09:43:22.160568+0000) 2024-03-04T09:43:36.839 INFO:journalctl@ceph.osd.11.smithi191.stdout:Mar 04 09:43:36 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-11[62160]: 2024-03-04T09:43:36.535+0000 7fdceac97700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.106:6820 osd.7 since back 2024-03-04T09:43:02.161173+0000 front 2024-03-04T09:43:02.161205+0000 (oldest deadline 2024-03-04T09:43:28.061162+0000) 2024-03-04T09:43:36.839 INFO:journalctl@ceph.osd.11.smithi191.stdout:Mar 04 09:43:36 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-11[62160]: 2024-03-04T09:43:36.535+0000 7fdceac97700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.191:6820 osd.8 since back 2024-03-04T09:43:12.761944+0000 front 2024-03-04T09:43:12.761977+0000 (oldest deadline 2024-03-04T09:43:35.661951+0000) 2024-03-04T09:43:37.838 INFO:journalctl@ceph.osd.11.smithi191.stdout:Mar 04 09:43:37 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-11[62160]: 2024-03-04T09:43:37.510+0000 7fdceac97700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-03-04T09:42:08.553313+0000 front 2024-03-04T09:42:08.553176+0000 (oldest deadline 2024-03-04T09:42:32.653011+0000) 2024-03-04T09:43:37.839 INFO:journalctl@ceph.osd.11.smithi191.stdout:Mar 04 09:43:37 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-11[62160]: 2024-03-04T09:43:37.510+0000 7fdceac97700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-03-04T09:42:18.355200+0000 front 2024-03-04T09:42:18.355189+0000 (oldest deadline 2024-03-04T09:42:44.254936+0000) 2024-03-04T09:43:37.839 INFO:journalctl@ceph.osd.11.smithi191.stdout:Mar 04 09:43:37 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-11[62160]: 2024-03-04T09:43:37.510+0000 7fdceac97700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.191:6804 osd.2 since back 2024-03-04T09:42:24.255361+0000 front 2024-03-04T09:42:24.255246+0000 (oldest deadline 2024-03-04T09:42:47.155465+0000) 2024-03-04T09:43:37.839 INFO:journalctl@ceph.osd.11.smithi191.stdout:Mar 04 09:43:37 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-11[62160]: 2024-03-04T09:43:37.510+0000 7fdceac97700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-03-04T09:42:41.857812+0000 front 2024-03-04T09:42:41.857710+0000 (oldest deadline 2024-03-04T09:43:03.557748+0000) 2024-03-04T09:43:37.839 INFO:journalctl@ceph.osd.11.smithi191.stdout:Mar 04 09:43:37 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-11[62160]: 2024-03-04T09:43:37.510+0000 7fdceac97700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.191:6812 osd.5 since back 2024-03-04T09:42:46.959192+0000 front 2024-03-04T09:42:46.959105+0000 (oldest deadline 2024-03-04T09:43:12.859230+0000) 2024-03-04T09:43:37.839 INFO:journalctl@ceph.osd.11.smithi191.stdout:Mar 04 09:43:37 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-11[62160]: 2024-03-04T09:43:37.510+0000 7fdceac97700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.52:6822 osd.6 since back 2024-03-04T09:42:56.260599+0000 front 2024-03-04T09:42:56.260647+0000 (oldest deadline 2024-03-04T09:43:22.160568+0000) 2024-03-04T09:43:37.839 INFO:journalctl@ceph.osd.11.smithi191.stdout:Mar 04 09:43:37 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-11[62160]: 2024-03-04T09:43:37.510+0000 7fdceac97700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.106:6820 osd.7 since back 2024-03-04T09:43:02.161173+0000 front 2024-03-04T09:43:02.161205+0000 (oldest deadline 2024-03-04T09:43:28.061162+0000) 2024-03-04T09:43:37.839 INFO:journalctl@ceph.osd.11.smithi191.stdout:Mar 04 09:43:37 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-11[62160]: 2024-03-04T09:43:37.510+0000 7fdceac97700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.191:6820 osd.8 since back 2024-03-04T09:43:12.761944+0000 front 2024-03-04T09:43:12.761977+0000 (oldest deadline 2024-03-04T09:43:35.661951+0000) 2024-03-04T09:43:38.306 INFO:journalctl@ceph.osd.11.smithi191.stdout:Mar 04 09:43:38 smithi191 podman[73836]: 2024-03-04 09:43:38.03101217 +0000 UTC m=+5.509857358 container died 1cb2ddf0e1ef4cfeb54c9fab214ded2f743821bc3038d372657c5efa9cb6fa20 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5, name=ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-11, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-qdb-1bc250c, io.buildah.version=1.33.5, org.label-schema.build-date=20240222, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8) 2024-03-04T09:43:38.306 INFO:journalctl@ceph.osd.11.smithi191.stdout:Mar 04 09:43:38 smithi191 podman[73836]: 2024-03-04 09:43:38.308689367 +0000 UTC m=+5.787534564 container cleanup 1cb2ddf0e1ef4cfeb54c9fab214ded2f743821bc3038d372657c5efa9cb6fa20 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5, name=ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-11, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-qdb-1bc250c, org.label-schema.build-date=20240222, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, org.label-schema.license=GPLv2, ceph=True) 2024-03-04T09:43:38.566 INFO:journalctl@ceph.osd.11.smithi191.stdout:Mar 04 09:43:38 smithi191 bash[73836]: ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-11 2024-03-04T09:43:38.838 INFO:journalctl@ceph.osd.11.smithi191.stdout:Mar 04 09:43:38 smithi191 podman[73853]: 2024-03-04 09:43:38.568566412 +0000 UTC m=+0.531521398 container remove 1cb2ddf0e1ef4cfeb54c9fab214ded2f743821bc3038d372657c5efa9cb6fa20 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5, name=ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-11, org.label-schema.build-date=20240222, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, RELEASE=wip-lusov-qdb-1bc250c, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-03-04T09:43:39.172 INFO:journalctl@ceph.osd.11.smithi191.stdout:Mar 04 09:43:38 smithi191 podman[73946]: 2024-03-04 09:43:38.826452046 +0000 UTC m=+0.017994918 image pull f88ffb32aa6a7d5dfdc954d9f096f0c9cc47981d05ec8d117ba859a22d8185ba quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5 2024-03-04T09:43:39.172 INFO:journalctl@ceph.osd.11.smithi191.stdout:Mar 04 09:43:38 smithi191 podman[73946]: 2024-03-04 09:43:38.94438198 +0000 UTC m=+0.135924885 container create 20407855f343e6b567c1936a0ceca72b052d19518e929244c6ea67ed5af52e13 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5, name=ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-11-deactivate, GIT_CLEAN=True, RELEASE=wip-lusov-qdb-1bc250c, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240222, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, org.label-schema.vendor=CentOS) 2024-03-04T09:43:39.172 INFO:journalctl@ceph.osd.11.smithi191.stdout:Mar 04 09:43:39 smithi191 podman[73946]: 2024-03-04 09:43:39.171528811 +0000 UTC m=+0.363071691 container init 20407855f343e6b567c1936a0ceca72b052d19518e929244c6ea67ed5af52e13 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5, name=ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-11-deactivate, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, org.label-schema.build-date=20240222, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, RELEASE=wip-lusov-qdb-1bc250c, ceph=True, CEPH_POINT_RELEASE=) 2024-03-04T09:43:39.172 INFO:journalctl@ceph.osd.11.smithi191.stdout:Mar 04 09:43:39 smithi191 podman[73946]: 2024-03-04 09:43:39.174519228 +0000 UTC m=+0.366062090 container start 20407855f343e6b567c1936a0ceca72b052d19518e929244c6ea67ed5af52e13 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5, name=ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-11-deactivate, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-qdb-1bc250c, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240222, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.33.5) 2024-03-04T09:43:39.468 INFO:journalctl@ceph.osd.11.smithi191.stdout:Mar 04 09:43:39 smithi191 podman[73946]: 2024-03-04 09:43:39.26304798 +0000 UTC m=+0.454590851 container attach 20407855f343e6b567c1936a0ceca72b052d19518e929244c6ea67ed5af52e13 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5, name=ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-11-deactivate, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240222, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, GIT_BRANCH=HEAD, RELEASE=wip-lusov-qdb-1bc250c, ceph=True, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux ) 2024-03-04T09:43:39.468 INFO:journalctl@ceph.osd.11.smithi191.stdout:Mar 04 09:43:39 smithi191 podman[73946]: 2024-03-04 09:43:39.470262069 +0000 UTC m=+0.661804941 container died 20407855f343e6b567c1936a0ceca72b052d19518e929244c6ea67ed5af52e13 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5, name=ceph-4e059f08-da08-11ee-95c6-87774f69a715-osd-11-deactivate, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-qdb-1bc250c, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240222) 2024-03-04T09:43:40.029 DEBUG:teuthology.orchestra.run.smithi191:> sudo pkill -f 'journalctl -f -n 0 -u ceph-4e059f08-da08-11ee-95c6-87774f69a715@osd.11.service' 2024-03-04T09:43:40.070 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-04T09:43:40.071 INFO:tasks.cephadm.osd.11:Stopped osd.11 2024-03-04T09:43:40.071 INFO:tasks.cephadm.mds.a:Stopping mds.a... 2024-03-04T09:43:40.071 DEBUG:teuthology.orchestra.run.smithi052:> sudo systemctl stop ceph-4e059f08-da08-11ee-95c6-87774f69a715@mds.a 2024-03-04T09:43:40.351 INFO:journalctl@ceph.mds.a.smithi052.stdout:Mar 04 09:43:40 smithi052 systemd[1]: Stopping Ceph mds.a for 4e059f08-da08-11ee-95c6-87774f69a715... 2024-03-04T09:43:40.352 INFO:journalctl@ceph.mds.a.smithi052.stdout:Mar 04 09:43:40 smithi052 ceph-4e059f08-da08-11ee-95c6-87774f69a715-mds-a[71773]: 2024-03-04T09:43:40.352+0000 7f54a3418700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-04T09:43:40.352 INFO:journalctl@ceph.mds.a.smithi052.stdout:Mar 04 09:43:40 smithi052 ceph-4e059f08-da08-11ee-95c6-87774f69a715-mds-a[71773]: 2024-03-04T09:43:40.352+0000 7f54a3418700 -1 mds.a *** got signal Terminated *** 2024-03-04T09:43:43.336 INFO:journalctl@ceph.mds.a.smithi052.stdout:Mar 04 09:43:42 smithi052 podman[96869]: 2024-03-04 09:43:42.94727695 +0000 UTC m=+2.674502244 container died f1c2020db07bcf3b2c1b71521436149f1ce55ddae48dd6c52698c0bec66355ac (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5, name=ceph-4e059f08-da08-11ee-95c6-87774f69a715-mds-a, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240222, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-qdb-1bc250c, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_CLEAN=True) 2024-03-04T09:43:43.673 INFO:journalctl@ceph.mds.a.smithi052.stdout:Mar 04 09:43:43 smithi052 podman[96869]: 2024-03-04 09:43:43.343578203 +0000 UTC m=+3.070803498 container cleanup f1c2020db07bcf3b2c1b71521436149f1ce55ddae48dd6c52698c0bec66355ac (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5, name=ceph-4e059f08-da08-11ee-95c6-87774f69a715-mds-a, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-qdb-1bc250c, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240222, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-03-04T09:43:43.674 INFO:journalctl@ceph.mds.a.smithi052.stdout:Mar 04 09:43:43 smithi052 bash[96869]: ceph-4e059f08-da08-11ee-95c6-87774f69a715-mds-a 2024-03-04T09:43:43.674 INFO:journalctl@ceph.mds.a.smithi052.stdout:Mar 04 09:43:43 smithi052 podman[96881]: 2024-03-04 09:43:43.561582693 +0000 UTC m=+0.608985371 container remove f1c2020db07bcf3b2c1b71521436149f1ce55ddae48dd6c52698c0bec66355ac (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5, name=ceph-4e059f08-da08-11ee-95c6-87774f69a715-mds-a, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-qdb-1bc250c, org.label-schema.build-date=20240222, CEPH_POINT_RELEASE=, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, GIT_CLEAN=True, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_BRANCH=HEAD) 2024-03-04T09:43:43.688 DEBUG:teuthology.orchestra.run.smithi052:> sudo pkill -f 'journalctl -f -n 0 -u ceph-4e059f08-da08-11ee-95c6-87774f69a715@mds.a.service' 2024-03-04T09:43:43.725 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-04T09:43:43.725 INFO:tasks.cephadm.mds.a:Stopped mds.a 2024-03-04T09:43:43.725 INFO:tasks.cephadm.mds.d:Stopping mds.d... 2024-03-04T09:43:43.726 DEBUG:teuthology.orchestra.run.smithi052:> sudo systemctl stop ceph-4e059f08-da08-11ee-95c6-87774f69a715@mds.d 2024-03-04T09:43:44.045 INFO:journalctl@ceph.mds.d.smithi052.stdout:Mar 04 09:43:43 smithi052 systemd[1]: Stopping Ceph mds.d for 4e059f08-da08-11ee-95c6-87774f69a715... 2024-03-04T09:43:44.336 INFO:journalctl@ceph.mds.d.smithi052.stdout:Mar 04 09:43:44 smithi052 ceph-4e059f08-da08-11ee-95c6-87774f69a715-mds-d[70814]: 2024-03-04T09:43:44.047+0000 7f09b68f1700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.d -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-04T09:43:44.336 INFO:journalctl@ceph.mds.d.smithi052.stdout:Mar 04 09:43:44 smithi052 ceph-4e059f08-da08-11ee-95c6-87774f69a715-mds-d[70814]: 2024-03-04T09:43:44.047+0000 7f09b68f1700 -1 mds.d *** got signal Terminated *** 2024-03-04T09:43:47.212 INFO:journalctl@ceph.mds.d.smithi052.stdout:Mar 04 09:43:46 smithi052 podman[96992]: 2024-03-04 09:43:46.938762412 +0000 UTC m=+2.994673369 container died 4e33c573a820f491556588421c2e1c00ea5ab0cd73053969449f3f411791166e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5, name=ceph-4e059f08-da08-11ee-95c6-87774f69a715-mds-d, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, org.label-schema.build-date=20240222, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=wip-lusov-qdb-1bc250c, org.label-schema.license=GPLv2, io.buildah.version=1.33.5) 2024-03-04T09:43:47.521 INFO:journalctl@ceph.mds.d.smithi052.stdout:Mar 04 09:43:47 smithi052 podman[96992]: 2024-03-04 09:43:47.3062131 +0000 UTC m=+3.362124056 container cleanup 4e33c573a820f491556588421c2e1c00ea5ab0cd73053969449f3f411791166e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5, name=ceph-4e059f08-da08-11ee-95c6-87774f69a715-mds-d, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240222, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-qdb-1bc250c, io.buildah.version=1.33.5) 2024-03-04T09:43:47.522 INFO:journalctl@ceph.mds.d.smithi052.stdout:Mar 04 09:43:47 smithi052 bash[96992]: ceph-4e059f08-da08-11ee-95c6-87774f69a715-mds-d 2024-03-04T09:43:47.656 DEBUG:teuthology.orchestra.run.smithi052:> sudo pkill -f 'journalctl -f -n 0 -u ceph-4e059f08-da08-11ee-95c6-87774f69a715@mds.d.service' 2024-03-04T09:43:47.694 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-04T09:43:47.694 INFO:tasks.cephadm.mds.d:Stopped mds.d 2024-03-04T09:43:47.694 INFO:tasks.cephadm.mds.g:Stopping mds.g... 2024-03-04T09:43:47.694 DEBUG:teuthology.orchestra.run.smithi052:> sudo systemctl stop ceph-4e059f08-da08-11ee-95c6-87774f69a715@mds.g 2024-03-04T09:43:48.086 INFO:journalctl@ceph.mds.g.smithi052.stdout:Mar 04 09:43:47 smithi052 systemd[1]: Stopping Ceph mds.g for 4e059f08-da08-11ee-95c6-87774f69a715... 2024-03-04T09:43:48.586 INFO:journalctl@ceph.mds.g.smithi052.stdout:Mar 04 09:43:48 smithi052 ceph-4e059f08-da08-11ee-95c6-87774f69a715-mds-g[71410]: 2024-03-04T09:43:48.126+0000 7f9f978d7700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.g -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-04T09:43:48.587 INFO:journalctl@ceph.mds.g.smithi052.stdout:Mar 04 09:43:48 smithi052 ceph-4e059f08-da08-11ee-95c6-87774f69a715-mds-g[71410]: 2024-03-04T09:43:48.126+0000 7f9f978d7700 -1 mds.g *** got signal Terminated *** 2024-03-04T09:43:52.433 INFO:journalctl@ceph.mds.g.smithi052.stdout:Mar 04 09:43:52 smithi052 podman[97124]: 2024-03-04 09:43:52.13723201 +0000 UTC m=+4.215315087 container died 828128f1c5012dfee78709bdb7e0b2cce64a1ac65dbaab27e016c4ef89236fde (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5, name=ceph-4e059f08-da08-11ee-95c6-87774f69a715-mds-g, RELEASE=wip-lusov-qdb-1bc250c, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240222, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-03-04T09:43:52.725 INFO:journalctl@ceph.mds.g.smithi052.stdout:Mar 04 09:43:52 smithi052 podman[97124]: 2024-03-04 09:43:52.518004158 +0000 UTC m=+4.596087242 container cleanup 828128f1c5012dfee78709bdb7e0b2cce64a1ac65dbaab27e016c4ef89236fde (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5, name=ceph-4e059f08-da08-11ee-95c6-87774f69a715-mds-g, org.label-schema.vendor=CentOS, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-qdb-1bc250c, GIT_CLEAN=True, org.label-schema.build-date=20240222, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-03-04T09:43:52.725 INFO:journalctl@ceph.mds.g.smithi052.stdout:Mar 04 09:43:52 smithi052 bash[97124]: ceph-4e059f08-da08-11ee-95c6-87774f69a715-mds-g 2024-03-04T09:43:52.869 DEBUG:teuthology.orchestra.run.smithi052:> sudo pkill -f 'journalctl -f -n 0 -u ceph-4e059f08-da08-11ee-95c6-87774f69a715@mds.g.service' 2024-03-04T09:43:52.904 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-04T09:43:52.904 INFO:tasks.cephadm.mds.g:Stopped mds.g 2024-03-04T09:43:52.905 INFO:tasks.cephadm.mds.j:Stopping mds.j... 2024-03-04T09:43:52.905 DEBUG:teuthology.orchestra.run.smithi052:> sudo systemctl stop ceph-4e059f08-da08-11ee-95c6-87774f69a715@mds.j 2024-03-04T09:43:53.336 INFO:journalctl@ceph.mds.j.smithi052.stdout:Mar 04 09:43:52 smithi052 systemd[1]: Stopping Ceph mds.j for 4e059f08-da08-11ee-95c6-87774f69a715... 2024-03-04T09:43:53.337 INFO:journalctl@ceph.mds.j.smithi052.stdout:Mar 04 09:43:53 smithi052 ceph-4e059f08-da08-11ee-95c6-87774f69a715-mds-j[70434]: 2024-03-04T09:43:53.220+0000 7fbb6bf1f700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.j -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-04T09:43:53.337 INFO:journalctl@ceph.mds.j.smithi052.stdout:Mar 04 09:43:53 smithi052 ceph-4e059f08-da08-11ee-95c6-87774f69a715-mds-j[70434]: 2024-03-04T09:43:53.220+0000 7fbb6bf1f700 -1 mds.j *** got signal Terminated *** 2024-03-04T09:43:56.369 INFO:journalctl@ceph.mds.j.smithi052.stdout:Mar 04 09:43:56 smithi052 podman[97244]: 2024-03-04 09:43:56.093558495 +0000 UTC m=+2.971483212 container died 942ac769c5f25b486442cf5e61daacf8570415d4721ef8df4785a5d62bf29e57 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5, name=ceph-4e059f08-da08-11ee-95c6-87774f69a715-mds-j, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240222, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-qdb-1bc250c, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, ceph=True) 2024-03-04T09:43:56.370 INFO:journalctl@ceph.mds.j.smithi052.stdout:Mar 04 09:43:56 smithi052 podman[97244]: 2024-03-04 09:43:56.371814465 +0000 UTC m=+3.249739188 container cleanup 942ac769c5f25b486442cf5e61daacf8570415d4721ef8df4785a5d62bf29e57 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5, name=ceph-4e059f08-da08-11ee-95c6-87774f69a715-mds-j, org.label-schema.build-date=20240222, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-qdb-1bc250c, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-03-04T09:43:56.728 INFO:journalctl@ceph.mds.j.smithi052.stdout:Mar 04 09:43:56 smithi052 bash[97244]: ceph-4e059f08-da08-11ee-95c6-87774f69a715-mds-j 2024-03-04T09:43:56.940 DEBUG:teuthology.orchestra.run.smithi052:> sudo pkill -f 'journalctl -f -n 0 -u ceph-4e059f08-da08-11ee-95c6-87774f69a715@mds.j.service' 2024-03-04T09:43:56.974 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-04T09:43:56.974 INFO:tasks.cephadm.mds.j:Stopped mds.j 2024-03-04T09:43:56.974 INFO:tasks.cephadm.mds.b:Stopping mds.b... 2024-03-04T09:43:56.975 DEBUG:teuthology.orchestra.run.smithi106:> sudo systemctl stop ceph-4e059f08-da08-11ee-95c6-87774f69a715@mds.b 2024-03-04T09:43:57.339 INFO:journalctl@ceph.mds.b.smithi106.stdout:Mar 04 09:43:57 smithi106 systemd[1]: Stopping Ceph mds.b for 4e059f08-da08-11ee-95c6-87774f69a715... 2024-03-04T09:43:57.633 INFO:journalctl@ceph.mds.b.smithi106.stdout:Mar 04 09:43:57 smithi106 ceph-4e059f08-da08-11ee-95c6-87774f69a715-mds-b[66655]: 2024-03-04T09:43:57.341+0000 7f173b0a7700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-04T09:43:57.633 INFO:journalctl@ceph.mds.b.smithi106.stdout:Mar 04 09:43:57 smithi106 ceph-4e059f08-da08-11ee-95c6-87774f69a715-mds-b[66655]: 2024-03-04T09:43:57.341+0000 7f173b0a7700 -1 mds.b *** got signal Terminated *** 2024-03-04T09:44:01.622 INFO:journalctl@ceph.mds.b.smithi106.stdout:Mar 04 09:44:01 smithi106 podman[73096]: 2024-03-04 09:44:01.353618061 +0000 UTC m=+4.189124350 container died a2424e9e9412a364f435c365e29cd1799fd06f352a296a66de3b493add01da77 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5, name=ceph-4e059f08-da08-11ee-95c6-87774f69a715-mds-b, RELEASE=wip-lusov-qdb-1bc250c, GIT_BRANCH=HEAD, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, org.label-schema.build-date=20240222, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-03-04T09:44:01.883 INFO:journalctl@ceph.mds.b.smithi106.stdout:Mar 04 09:44:01 smithi106 podman[73096]: 2024-03-04 09:44:01.723291279 +0000 UTC m=+4.558797565 container cleanup a2424e9e9412a364f435c365e29cd1799fd06f352a296a66de3b493add01da77 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5, name=ceph-4e059f08-da08-11ee-95c6-87774f69a715-mds-b, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux , ceph=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, RELEASE=wip-lusov-qdb-1bc250c, org.label-schema.license=GPLv2, org.label-schema.build-date=20240222, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-04T09:44:01.883 INFO:journalctl@ceph.mds.b.smithi106.stdout:Mar 04 09:44:01 smithi106 bash[73096]: ceph-4e059f08-da08-11ee-95c6-87774f69a715-mds-b 2024-03-04T09:44:02.084 DEBUG:teuthology.orchestra.run.smithi106:> sudo pkill -f 'journalctl -f -n 0 -u ceph-4e059f08-da08-11ee-95c6-87774f69a715@mds.b.service' 2024-03-04T09:44:02.121 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-04T09:44:02.121 INFO:tasks.cephadm.mds.b:Stopped mds.b 2024-03-04T09:44:02.121 INFO:tasks.cephadm.mds.e:Stopping mds.e... 2024-03-04T09:44:02.121 DEBUG:teuthology.orchestra.run.smithi106:> sudo systemctl stop ceph-4e059f08-da08-11ee-95c6-87774f69a715@mds.e 2024-03-04T09:44:02.383 INFO:journalctl@ceph.mds.e.smithi106.stdout:Mar 04 09:44:02 smithi106 systemd[1]: Stopping Ceph mds.e for 4e059f08-da08-11ee-95c6-87774f69a715... 2024-03-04T09:44:02.883 INFO:journalctl@ceph.mds.e.smithi106.stdout:Mar 04 09:44:02 smithi106 ceph-4e059f08-da08-11ee-95c6-87774f69a715-mds-e[66027]: 2024-03-04T09:44:02.443+0000 7feddeebc700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.e -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-04T09:44:02.883 INFO:journalctl@ceph.mds.e.smithi106.stdout:Mar 04 09:44:02 smithi106 ceph-4e059f08-da08-11ee-95c6-87774f69a715-mds-e[66027]: 2024-03-04T09:44:02.443+0000 7feddeebc700 -1 mds.e *** got signal Terminated *** 2024-03-04T09:44:05.883 INFO:journalctl@ceph.mds.e.smithi106.stdout:Mar 04 09:44:05 smithi106 podman[73217]: 2024-03-04 09:44:05.584736872 +0000 UTC m=+3.243713203 container died 2bbb1b9af83dec2c854defb30c77bfdc68abf11068f3aa49fd499116ce7870f9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5, name=ceph-4e059f08-da08-11ee-95c6-87774f69a715-mds-e, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240222, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-qdb-1bc250c, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2) 2024-03-04T09:44:06.209 INFO:journalctl@ceph.mds.e.smithi106.stdout:Mar 04 09:44:05 smithi106 podman[73217]: 2024-03-04 09:44:05.968565294 +0000 UTC m=+3.627541630 container cleanup 2bbb1b9af83dec2c854defb30c77bfdc68abf11068f3aa49fd499116ce7870f9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5, name=ceph-4e059f08-da08-11ee-95c6-87774f69a715-mds-e, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.33.5, RELEASE=wip-lusov-qdb-1bc250c, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, org.label-schema.build-date=20240222) 2024-03-04T09:44:06.209 INFO:journalctl@ceph.mds.e.smithi106.stdout:Mar 04 09:44:05 smithi106 bash[73217]: ceph-4e059f08-da08-11ee-95c6-87774f69a715-mds-e 2024-03-04T09:44:06.337 DEBUG:teuthology.orchestra.run.smithi106:> sudo pkill -f 'journalctl -f -n 0 -u ceph-4e059f08-da08-11ee-95c6-87774f69a715@mds.e.service' 2024-03-04T09:44:06.374 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-04T09:44:06.374 INFO:tasks.cephadm.mds.e:Stopped mds.e 2024-03-04T09:44:06.375 INFO:tasks.cephadm.mds.h:Stopping mds.h... 2024-03-04T09:44:06.375 DEBUG:teuthology.orchestra.run.smithi106:> sudo systemctl stop ceph-4e059f08-da08-11ee-95c6-87774f69a715@mds.h 2024-03-04T09:44:06.633 INFO:journalctl@ceph.mds.h.smithi106.stdout:Mar 04 09:44:06 smithi106 systemd[1]: Stopping Ceph mds.h for 4e059f08-da08-11ee-95c6-87774f69a715... 2024-03-04T09:44:07.133 INFO:journalctl@ceph.mds.h.smithi106.stdout:Mar 04 09:44:06 smithi106 ceph-4e059f08-da08-11ee-95c6-87774f69a715-mds-h[66334]: 2024-03-04T09:44:06.721+0000 7f0ca248e700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.h -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-04T09:44:07.133 INFO:journalctl@ceph.mds.h.smithi106.stdout:Mar 04 09:44:06 smithi106 ceph-4e059f08-da08-11ee-95c6-87774f69a715-mds-h[66334]: 2024-03-04T09:44:06.721+0000 7f0ca248e700 -1 mds.h *** got signal Terminated *** 2024-03-04T09:44:09.862 INFO:journalctl@ceph.mds.h.smithi106.stdout:Mar 04 09:44:09 smithi106 podman[73349]: 2024-03-04 09:44:09.568597221 +0000 UTC m=+2.977078170 container died 7a759fae8efb0fb8b3808f050eb58c5ec7ba9f3afe57f08007faaad34b1ad3d9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5, name=ceph-4e059f08-da08-11ee-95c6-87774f69a715-mds-h, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, ceph=True, org.label-schema.build-date=20240222, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-qdb-1bc250c, org.label-schema.license=GPLv2, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8) 2024-03-04T09:44:10.133 INFO:journalctl@ceph.mds.h.smithi106.stdout:Mar 04 09:44:09 smithi106 podman[73349]: 2024-03-04 09:44:09.947467889 +0000 UTC m=+3.355948829 container cleanup 7a759fae8efb0fb8b3808f050eb58c5ec7ba9f3afe57f08007faaad34b1ad3d9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5, name=ceph-4e059f08-da08-11ee-95c6-87774f69a715-mds-h, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.build-date=20240222, GIT_CLEAN=True, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-qdb-1bc250c, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=) 2024-03-04T09:44:10.133 INFO:journalctl@ceph.mds.h.smithi106.stdout:Mar 04 09:44:09 smithi106 bash[73349]: ceph-4e059f08-da08-11ee-95c6-87774f69a715-mds-h 2024-03-04T09:44:10.320 DEBUG:teuthology.orchestra.run.smithi106:> sudo pkill -f 'journalctl -f -n 0 -u ceph-4e059f08-da08-11ee-95c6-87774f69a715@mds.h.service' 2024-03-04T09:44:10.355 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-04T09:44:10.355 INFO:tasks.cephadm.mds.h:Stopped mds.h 2024-03-04T09:44:10.355 INFO:tasks.cephadm.mds.k:Stopping mds.k... 2024-03-04T09:44:10.356 DEBUG:teuthology.orchestra.run.smithi106:> sudo systemctl stop ceph-4e059f08-da08-11ee-95c6-87774f69a715@mds.k 2024-03-04T09:44:10.633 INFO:journalctl@ceph.mds.k.smithi106.stdout:Mar 04 09:44:10 smithi106 systemd[1]: Stopping Ceph mds.k for 4e059f08-da08-11ee-95c6-87774f69a715... 2024-03-04T09:44:11.133 INFO:journalctl@ceph.mds.k.smithi106.stdout:Mar 04 09:44:10 smithi106 ceph-4e059f08-da08-11ee-95c6-87774f69a715-mds-k[65704]: 2024-03-04T09:44:10.684+0000 7f70d843f700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.k -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-04T09:44:11.133 INFO:journalctl@ceph.mds.k.smithi106.stdout:Mar 04 09:44:10 smithi106 ceph-4e059f08-da08-11ee-95c6-87774f69a715-mds-k[65704]: 2024-03-04T09:44:10.684+0000 7f70d843f700 -1 mds.k *** got signal Terminated *** 2024-03-04T09:44:13.549 INFO:journalctl@ceph.mds.k.smithi106.stdout:Mar 04 09:44:13 smithi106 podman[73468]: 2024-03-04 09:44:13.296978303 +0000 UTC m=+2.726589599 container died a3f695264ac9030fc69e3fe0c14043664f858181b11765e394b01d500dac2d4b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5, name=ceph-4e059f08-da08-11ee-95c6-87774f69a715-mds-k, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-qdb-1bc250c, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240222) 2024-03-04T09:44:13.841 INFO:journalctl@ceph.mds.k.smithi106.stdout:Mar 04 09:44:13 smithi106 podman[73468]: 2024-03-04 09:44:13.642689785 +0000 UTC m=+3.072301073 container cleanup a3f695264ac9030fc69e3fe0c14043664f858181b11765e394b01d500dac2d4b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5, name=ceph-4e059f08-da08-11ee-95c6-87774f69a715-mds-k, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240222, ceph=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-qdb-1bc250c, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=) 2024-03-04T09:44:13.842 INFO:journalctl@ceph.mds.k.smithi106.stdout:Mar 04 09:44:13 smithi106 bash[73468]: ceph-4e059f08-da08-11ee-95c6-87774f69a715-mds-k 2024-03-04T09:44:14.053 DEBUG:teuthology.orchestra.run.smithi106:> sudo pkill -f 'journalctl -f -n 0 -u ceph-4e059f08-da08-11ee-95c6-87774f69a715@mds.k.service' 2024-03-04T09:44:14.088 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-04T09:44:14.088 INFO:tasks.cephadm.mds.k:Stopped mds.k 2024-03-04T09:44:14.089 INFO:tasks.cephadm.mds.c:Stopping mds.c... 2024-03-04T09:44:14.089 DEBUG:teuthology.orchestra.run.smithi191:> sudo systemctl stop ceph-4e059f08-da08-11ee-95c6-87774f69a715@mds.c 2024-03-04T09:44:14.388 INFO:journalctl@ceph.mds.c.smithi191.stdout:Mar 04 09:44:14 smithi191 systemd[1]: Stopping Ceph mds.c for 4e059f08-da08-11ee-95c6-87774f69a715... 2024-03-04T09:44:14.656 INFO:journalctl@ceph.mds.c.smithi191.stdout:Mar 04 09:44:14 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-mds-c[67083]: 2024-03-04T09:44:14.389+0000 7fe5dae4f700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-04T09:44:14.656 INFO:journalctl@ceph.mds.c.smithi191.stdout:Mar 04 09:44:14 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-mds-c[67083]: 2024-03-04T09:44:14.389+0000 7fe5dae4f700 -1 mds.c *** got signal Terminated *** 2024-03-04T09:44:17.838 INFO:journalctl@ceph.mds.c.smithi191.stdout:Mar 04 09:44:17 smithi191 podman[74051]: 2024-03-04 09:44:17.569972374 +0000 UTC m=+3.288431203 container died ea63df47849d8faa6ca475ebbe5a1343b821d97aa01c4b1d060a5659bd3fa733 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5, name=ceph-4e059f08-da08-11ee-95c6-87774f69a715-mds-c, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240222, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=wip-lusov-qdb-1bc250c) 2024-03-04T09:44:18.088 INFO:journalctl@ceph.mds.c.smithi191.stdout:Mar 04 09:44:17 smithi191 podman[74051]: 2024-03-04 09:44:17.914163809 +0000 UTC m=+3.632622620 container cleanup ea63df47849d8faa6ca475ebbe5a1343b821d97aa01c4b1d060a5659bd3fa733 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5, name=ceph-4e059f08-da08-11ee-95c6-87774f69a715-mds-c, GIT_CLEAN=True, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, io.buildah.version=1.33.5, org.label-schema.build-date=20240222, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-qdb-1bc250c, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-03-04T09:44:18.088 INFO:journalctl@ceph.mds.c.smithi191.stdout:Mar 04 09:44:17 smithi191 bash[74051]: ceph-4e059f08-da08-11ee-95c6-87774f69a715-mds-c 2024-03-04T09:44:18.342 DEBUG:teuthology.orchestra.run.smithi191:> sudo pkill -f 'journalctl -f -n 0 -u ceph-4e059f08-da08-11ee-95c6-87774f69a715@mds.c.service' 2024-03-04T09:44:18.366 INFO:journalctl@ceph.mds.c.smithi191.stdout:Mar 04 09:44:18 smithi191 podman[74077]: 2024-03-04 09:44:18.109257849 +0000 UTC m=+0.533151244 container remove ea63df47849d8faa6ca475ebbe5a1343b821d97aa01c4b1d060a5659bd3fa733 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5, name=ceph-4e059f08-da08-11ee-95c6-87774f69a715-mds-c, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, org.label-schema.build-date=20240222, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, io.buildah.version=1.33.5, ceph=True, RELEASE=wip-lusov-qdb-1bc250c) 2024-03-04T09:44:18.366 INFO:journalctl@ceph.mds.c.smithi191.stdout:Mar 04 09:44:18 smithi191 systemd[1]: ceph-4e059f08-da08-11ee-95c6-87774f69a715@mds.c.service: Deactivated successfully. 2024-03-04T09:44:18.366 INFO:journalctl@ceph.mds.c.smithi191.stdout:Mar 04 09:44:18 smithi191 systemd[1]: Stopped Ceph mds.c for 4e059f08-da08-11ee-95c6-87774f69a715. 2024-03-04T09:44:18.366 INFO:journalctl@ceph.mds.c.smithi191.stdout:Mar 04 09:44:18 smithi191 systemd[1]: ceph-4e059f08-da08-11ee-95c6-87774f69a715@mds.c.service: Consumed 5.498s CPU time. 2024-03-04T09:44:18.379 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-04T09:44:18.379 INFO:tasks.cephadm.mds.c:Stopped mds.c 2024-03-04T09:44:18.380 INFO:tasks.cephadm.mds.f:Stopping mds.f... 2024-03-04T09:44:18.380 DEBUG:teuthology.orchestra.run.smithi191:> sudo systemctl stop ceph-4e059f08-da08-11ee-95c6-87774f69a715@mds.f 2024-03-04T09:44:18.691 INFO:journalctl@ceph.mds.f.smithi191.stdout:Mar 04 09:44:18 smithi191 systemd[1]: Stopping Ceph mds.f for 4e059f08-da08-11ee-95c6-87774f69a715... 2024-03-04T09:44:19.088 INFO:journalctl@ceph.mds.f.smithi191.stdout:Mar 04 09:44:18 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-mds-f[66771]: 2024-03-04T09:44:18.691+0000 7f8ad5217700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.f -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-04T09:44:19.088 INFO:journalctl@ceph.mds.f.smithi191.stdout:Mar 04 09:44:18 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-mds-f[66771]: 2024-03-04T09:44:18.691+0000 7f8ad5217700 -1 mds.f *** got signal Terminated *** 2024-03-04T09:44:21.838 INFO:journalctl@ceph.mds.f.smithi191.stdout:Mar 04 09:44:21 smithi191 podman[74183]: 2024-03-04 09:44:21.581417579 +0000 UTC m=+2.984938377 container died 701812dde8795ec976caad0e1f41395fb4d9513389a4202224d7e1b403636a6b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5, name=ceph-4e059f08-da08-11ee-95c6-87774f69a715-mds-f, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240222, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-qdb-1bc250c, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, GIT_CLEAN=True) 2024-03-04T09:44:22.133 INFO:journalctl@ceph.mds.f.smithi191.stdout:Mar 04 09:44:21 smithi191 podman[74183]: 2024-03-04 09:44:21.925275595 +0000 UTC m=+3.328796368 container cleanup 701812dde8795ec976caad0e1f41395fb4d9513389a4202224d7e1b403636a6b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5, name=ceph-4e059f08-da08-11ee-95c6-87774f69a715-mds-f, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240222, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.33.5, GIT_CLEAN=True, RELEASE=wip-lusov-qdb-1bc250c, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-04T09:44:22.133 INFO:journalctl@ceph.mds.f.smithi191.stdout:Mar 04 09:44:21 smithi191 bash[74183]: ceph-4e059f08-da08-11ee-95c6-87774f69a715-mds-f 2024-03-04T09:44:22.328 DEBUG:teuthology.orchestra.run.smithi191:> sudo pkill -f 'journalctl -f -n 0 -u ceph-4e059f08-da08-11ee-95c6-87774f69a715@mds.f.service' 2024-03-04T09:44:22.363 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-04T09:44:22.363 INFO:tasks.cephadm.mds.f:Stopped mds.f 2024-03-04T09:44:22.363 INFO:tasks.cephadm.mds.i:Stopping mds.i... 2024-03-04T09:44:22.364 DEBUG:teuthology.orchestra.run.smithi191:> sudo systemctl stop ceph-4e059f08-da08-11ee-95c6-87774f69a715@mds.i 2024-03-04T09:44:22.838 INFO:journalctl@ceph.mds.i.smithi191.stdout:Mar 04 09:44:22 smithi191 systemd[1]: Stopping Ceph mds.i for 4e059f08-da08-11ee-95c6-87774f69a715... 2024-03-04T09:44:22.838 INFO:journalctl@ceph.mds.i.smithi191.stdout:Mar 04 09:44:22 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-mds-i[67404]: 2024-03-04T09:44:22.669+0000 7f5385701700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.i -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-04T09:44:22.838 INFO:journalctl@ceph.mds.i.smithi191.stdout:Mar 04 09:44:22 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-mds-i[67404]: 2024-03-04T09:44:22.669+0000 7f5385701700 -1 mds.i *** got signal Terminated *** 2024-03-04T09:44:25.836 INFO:journalctl@ceph.mds.i.smithi191.stdout:Mar 04 09:44:25 smithi191 podman[74303]: 2024-03-04 09:44:25.569253967 +0000 UTC m=+2.985957312 container died a07ca64ad21d83b913c50c069d97f5a3dad12b121405c9a545b65e2cc2e5a2f5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5, name=ceph-4e059f08-da08-11ee-95c6-87774f69a715-mds-i, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, RELEASE=wip-lusov-qdb-1bc250c, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240222, io.buildah.version=1.33.5, ceph=True) 2024-03-04T09:44:26.088 INFO:journalctl@ceph.mds.i.smithi191.stdout:Mar 04 09:44:25 smithi191 podman[74303]: 2024-03-04 09:44:25.946036282 +0000 UTC m=+3.362739666 container cleanup a07ca64ad21d83b913c50c069d97f5a3dad12b121405c9a545b65e2cc2e5a2f5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5, name=ceph-4e059f08-da08-11ee-95c6-87774f69a715-mds-i, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240222, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-qdb-1bc250c, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-04T09:44:26.088 INFO:journalctl@ceph.mds.i.smithi191.stdout:Mar 04 09:44:25 smithi191 bash[74303]: ceph-4e059f08-da08-11ee-95c6-87774f69a715-mds-i 2024-03-04T09:44:26.356 DEBUG:teuthology.orchestra.run.smithi191:> sudo pkill -f 'journalctl -f -n 0 -u ceph-4e059f08-da08-11ee-95c6-87774f69a715@mds.i.service' 2024-03-04T09:44:26.390 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-04T09:44:26.390 INFO:tasks.cephadm.mds.i:Stopped mds.i 2024-03-04T09:44:26.390 INFO:tasks.cephadm.mds.l:Stopping mds.l... 2024-03-04T09:44:26.390 DEBUG:teuthology.orchestra.run.smithi191:> sudo systemctl stop ceph-4e059f08-da08-11ee-95c6-87774f69a715@mds.l 2024-03-04T09:44:26.714 INFO:journalctl@ceph.mds.l.smithi191.stdout:Mar 04 09:44:26 smithi191 systemd[1]: Stopping Ceph mds.l for 4e059f08-da08-11ee-95c6-87774f69a715... 2024-03-04T09:44:27.088 INFO:journalctl@ceph.mds.l.smithi191.stdout:Mar 04 09:44:26 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-mds-l[66458]: 2024-03-04T09:44:26.715+0000 7fa9b8f74700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.l -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-04T09:44:27.088 INFO:journalctl@ceph.mds.l.smithi191.stdout:Mar 04 09:44:26 smithi191 ceph-4e059f08-da08-11ee-95c6-87774f69a715-mds-l[66458]: 2024-03-04T09:44:26.715+0000 7fa9b8f74700 -1 mds.l *** got signal Terminated *** 2024-03-04T09:44:30.838 INFO:journalctl@ceph.mds.l.smithi191.stdout:Mar 04 09:44:30 smithi191 podman[74422]: 2024-03-04 09:44:30.487501274 +0000 UTC m=+3.876405474 container died 08126d739d3a2e2f9c3140f21255c28092412552931135a4033cb0be331b541d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5, name=ceph-4e059f08-da08-11ee-95c6-87774f69a715-mds-l, org.label-schema.build-date=20240222, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=wip-lusov-qdb-1bc250c, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-03-04T09:44:31.243 INFO:journalctl@ceph.mds.l.smithi191.stdout:Mar 04 09:44:30 smithi191 podman[74422]: 2024-03-04 09:44:30.923824057 +0000 UTC m=+4.312728256 container cleanup 08126d739d3a2e2f9c3140f21255c28092412552931135a4033cb0be331b541d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5, name=ceph-4e059f08-da08-11ee-95c6-87774f69a715-mds-l, org.label-schema.build-date=20240222, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, RELEASE=wip-lusov-qdb-1bc250c, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-03-04T09:44:31.243 INFO:journalctl@ceph.mds.l.smithi191.stdout:Mar 04 09:44:30 smithi191 bash[74422]: ceph-4e059f08-da08-11ee-95c6-87774f69a715-mds-l 2024-03-04T09:44:31.487 DEBUG:teuthology.orchestra.run.smithi191:> sudo pkill -f 'journalctl -f -n 0 -u ceph-4e059f08-da08-11ee-95c6-87774f69a715@mds.l.service' 2024-03-04T09:44:31.510 INFO:journalctl@ceph.mds.l.smithi191.stdout:Mar 04 09:44:31 smithi191 podman[74436]: 2024-03-04 09:44:31.241534962 +0000 UTC m=+0.745150601 container remove 08126d739d3a2e2f9c3140f21255c28092412552931135a4033cb0be331b541d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3011ef4e883c5a20dd92da8f81055baaa8abf2682670e25e08f983b156727fe5, name=ceph-4e059f08-da08-11ee-95c6-87774f69a715-mds-l, org.label-schema.build-date=20240222, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=918ebec1af6f745255c974e3941e386838a94fd8, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-qdb-1bc250c, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True) 2024-03-04T09:44:31.510 INFO:journalctl@ceph.mds.l.smithi191.stdout:Mar 04 09:44:31 smithi191 systemd[1]: ceph-4e059f08-da08-11ee-95c6-87774f69a715@mds.l.service: Deactivated successfully. 2024-03-04T09:44:31.511 INFO:journalctl@ceph.mds.l.smithi191.stdout:Mar 04 09:44:31 smithi191 systemd[1]: Stopped Ceph mds.l for 4e059f08-da08-11ee-95c6-87774f69a715. 2024-03-04T09:44:31.511 INFO:journalctl@ceph.mds.l.smithi191.stdout:Mar 04 09:44:31 smithi191 systemd[1]: ceph-4e059f08-da08-11ee-95c6-87774f69a715@mds.l.service: Consumed 1.005s CPU time. 2024-03-04T09:44:31.521 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-04T09:44:31.521 INFO:tasks.cephadm.mds.l:Stopped mds.l 2024-03-04T09:44:31.521 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 4e059f08-da08-11ee-95c6-87774f69a715 --force --keep-logs 2024-03-04T09:44:31.718 INFO:teuthology.orchestra.run.smithi052.stdout:Deleting cluster with fsid: 4e059f08-da08-11ee-95c6-87774f69a715 2024-03-04T09:44:34.048 DEBUG:teuthology.orchestra.run.smithi106:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 4e059f08-da08-11ee-95c6-87774f69a715 --force --keep-logs 2024-03-04T09:44:34.238 INFO:teuthology.orchestra.run.smithi106.stdout:Deleting cluster with fsid: 4e059f08-da08-11ee-95c6-87774f69a715 2024-03-04T09:44:36.723 DEBUG:teuthology.orchestra.run.smithi191:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 4e059f08-da08-11ee-95c6-87774f69a715 --force --keep-logs 2024-03-04T09:44:36.910 INFO:teuthology.orchestra.run.smithi191.stdout:Deleting cluster with fsid: 4e059f08-da08-11ee-95c6-87774f69a715 2024-03-04T09:44:39.317 DEBUG:teuthology.orchestra.run.smithi052:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-03-04T09:44:39.351 DEBUG:teuthology.orchestra.run.smithi106:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-03-04T09:44:39.386 DEBUG:teuthology.orchestra.run.smithi191:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-03-04T09:44:39.414 INFO:tasks.cephadm:Archiving crash dumps... 2024-03-04T09:44:39.416 DEBUG:teuthology.misc:Transferring archived files from smithi052:/var/lib/ceph/4e059f08-da08-11ee-95c6-87774f69a715/crash to /home/teuthworker/archive/leonidus-2024-03-04_07:05:03-fs-wip-lusov-qdb-distro-default-smithi/7580854/remote/smithi052/crash 2024-03-04T09:44:39.417 DEBUG:teuthology.orchestra.run.smithi052:> sudo tar c -f - -C /var/lib/ceph/4e059f08-da08-11ee-95c6-87774f69a715/crash -- . 2024-03-04T09:44:39.455 INFO:teuthology.orchestra.run.smithi052.stderr:tar: /var/lib/ceph/4e059f08-da08-11ee-95c6-87774f69a715/crash: Cannot open: No such file or directory 2024-03-04T09:44:39.455 INFO:teuthology.orchestra.run.smithi052.stderr:tar: Error is not recoverable: exiting now 2024-03-04T09:44:39.457 DEBUG:teuthology.misc:Transferring archived files from smithi106:/var/lib/ceph/4e059f08-da08-11ee-95c6-87774f69a715/crash to /home/teuthworker/archive/leonidus-2024-03-04_07:05:03-fs-wip-lusov-qdb-distro-default-smithi/7580854/remote/smithi106/crash 2024-03-04T09:44:39.458 DEBUG:teuthology.orchestra.run.smithi106:> sudo tar c -f - -C /var/lib/ceph/4e059f08-da08-11ee-95c6-87774f69a715/crash -- . 2024-03-04T09:44:39.508 INFO:teuthology.orchestra.run.smithi106.stderr:tar: /var/lib/ceph/4e059f08-da08-11ee-95c6-87774f69a715/crash: Cannot open: No such file or directory 2024-03-04T09:44:39.508 INFO:teuthology.orchestra.run.smithi106.stderr:tar: Error is not recoverable: exiting now 2024-03-04T09:44:39.510 DEBUG:teuthology.misc:Transferring archived files from smithi191:/var/lib/ceph/4e059f08-da08-11ee-95c6-87774f69a715/crash to /home/teuthworker/archive/leonidus-2024-03-04_07:05:03-fs-wip-lusov-qdb-distro-default-smithi/7580854/remote/smithi191/crash 2024-03-04T09:44:39.511 DEBUG:teuthology.orchestra.run.smithi191:> sudo tar c -f - -C /var/lib/ceph/4e059f08-da08-11ee-95c6-87774f69a715/crash -- . 2024-03-04T09:44:39.544 INFO:teuthology.orchestra.run.smithi191.stderr:tar: /var/lib/ceph/4e059f08-da08-11ee-95c6-87774f69a715/crash: Cannot open: No such file or directory 2024-03-04T09:44:39.544 INFO:teuthology.orchestra.run.smithi191.stderr:tar: Error is not recoverable: exiting now 2024-03-04T09:44:39.545 INFO:tasks.cephadm:Checking cluster log for badness... 2024-03-04T09:44:39.546 DEBUG:teuthology.orchestra.run.smithi052:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/4e059f08-da08-11ee-95c6-87774f69a715/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | head -n 1 2024-03-04T09:44:39.581 INFO:teuthology.orchestra.run.smithi052.stdout:2024-03-04T09:22:37.309449+0000 mon.a (mon.0) 191 : cluster 3 [WRN] MON_DOWN: 1/3 mons down, quorum a,c 2024-03-04T09:44:39.582 WARNING:tasks.cephadm:Found errors (ERR|WRN|SEC) in cluster log 2024-03-04T09:44:39.583 INFO:tasks.cephadm:Compressing logs... 2024-03-04T09:44:39.583 DEBUG:teuthology.orchestra.run.smithi052:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-03-04T09:44:39.624 DEBUG:teuthology.orchestra.run.smithi106:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-03-04T09:44:39.626 DEBUG:teuthology.orchestra.run.smithi191:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-03-04T09:44:39.652 INFO:teuthology.orchestra.run.smithi052.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-03-04T09:44:39.652 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-03-04T09:44:39.652 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.log 2024-03-04T09:44:39.664 INFO:teuthology.orchestra.run.smithi106.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-03-04T09:44:39.664 INFO:teuthology.orchestra.run.smithi106.stderr:: No such file or directory 2024-03-04T09:44:39.665 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/4e059f08-da08-11ee-95c6-87774f69a715/ceph-osd.10.log 2024-03-04T09:44:39.665 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/4e059f08-da08-11ee-95c6-87774f69a715/ceph.audit.log 2024-03-04T09:44:39.665 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/cephadm.log: /var/log/ceph/4e059f08-da08-11ee-95c6-87774f69a715/ceph-osd.10.log: gzip -5 --verbose -- /var/log/ceph/4e059f08-da08-11ee-95c6-87774f69a715/ceph-mds.b.log 2024-03-04T09:44:39.665 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/4e059f08-da08-11ee-95c6-87774f69a715/ceph-osd.4.log 2024-03-04T09:44:39.665 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/4e059f08-da08-11ee-95c6-87774f69a715/ceph-osd.1.log 2024-03-04T09:44:39.665 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/4e059f08-da08-11ee-95c6-87774f69a715/ceph-mds.k.log 2024-03-04T09:44:39.665 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/4e059f08-da08-11ee-95c6-87774f69a715/ceph.audit.log: 92.1% -- replaced with /var/log/ceph/cephadm.log.gz 2024-03-04T09:44:39.665 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/4e059f08-da08-11ee-95c6-87774f69a715/ceph-mds.b.log: /var/log/ceph/4e059f08-da08-11ee-95c6-87774f69a715/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/4e059f08-da08-11ee-95c6-87774f69a715/ceph-mds.h.log 2024-03-04T09:44:39.666 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/4e059f08-da08-11ee-95c6-87774f69a715/ceph-osd.4.log: /var/log/ceph/4e059f08-da08-11ee-95c6-87774f69a715/ceph-mds.k.log: gzip -5 --verbose -- /var/log/ceph/4e059f08-da08-11ee-95c6-87774f69a715/ceph-osd.7.log 2024-03-04T09:44:39.666 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/4e059f08-da08-11ee-95c6-87774f69a715/ceph-mds.e.log 2024-03-04T09:44:39.666 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/4e059f08-da08-11ee-95c6-87774f69a715/ceph-mds.h.log: /var/log/ceph/4e059f08-da08-11ee-95c6-87774f69a715/ceph-osd.7.log: 90.3% -- replaced with /var/log/ceph/4e059f08-da08-11ee-95c6-87774f69a715/ceph.audit.log.gz 2024-03-04T09:44:39.666 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/4e059f08-da08-11ee-95c6-87774f69a715/ceph-mgr.y.log 2024-03-04T09:44:39.666 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/4e059f08-da08-11ee-95c6-87774f69a715/ceph-mds.e.log: 94.6% -- replaced with /var/log/ceph/4e059f08-da08-11ee-95c6-87774f69a715/ceph-mds.k.log.gz 2024-03-04T09:44:39.666 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/4e059f08-da08-11ee-95c6-87774f69a715/ceph.cephadm.log 2024-03-04T09:44:39.671 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/4e059f08-da08-11ee-95c6-87774f69a715/ceph-osd.6.log 2024-03-04T09:44:39.671 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.admin.log: gzip -5 --verbose -- /var/log/ceph/4e059f08-da08-11ee-95c6-87774f69a715/ceph-mon.a.log 2024-03-04T09:44:39.671 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/4e059f08-da08-11ee-95c6-87774f69a715/ceph.audit.log/var/log/ceph/4e059f08-da08-11ee-95c6-87774f69a715/ceph-osd.6.log: 2024-03-04T09:44:39.671 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/4e059f08-da08-11ee-95c6-87774f69a715/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/4e059f08-da08-11ee-95c6-87774f69a715/ceph-mds.g.log 2024-03-04T09:44:39.671 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/4e059f08-da08-11ee-95c6-87774f69a715/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/4e059f08-da08-11ee-95c6-87774f69a715/ceph-mds.j.log 2024-03-04T09:44:39.671 INFO:teuthology.orchestra.run.smithi052.stderr: 92.6% -- replaced with /var/log/ceph/ceph-client.admin.log.gz 2024-03-04T09:44:39.671 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/4e059f08-da08-11ee-95c6-87774f69a715/ceph-mds.g.log: gzip -5 --verbose -- /var/log/ceph/4e059f08-da08-11ee-95c6-87774f69a715/ceph-osd.3.log 2024-03-04T09:44:39.671 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5/var/log/ceph/4e059f08-da08-11ee-95c6-87774f69a715/ceph-mds.j.log: --verbose -- /var/log/ceph/4e059f08-da08-11ee-95c6-87774f69a715/ceph-mgr.x.log 2024-03-04T09:44:39.671 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/4e059f08-da08-11ee-95c6-87774f69a715/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/4e059f08-da08-11ee-95c6-87774f69a715/ceph-osd.0.log 2024-03-04T09:44:39.672 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/4e059f08-da08-11ee-95c6-87774f69a715/ceph-mgr.x.log: 90.1% -- replaced with /var/log/ceph/4e059f08-da08-11ee-95c6-87774f69a715/ceph.audit.log.gz 2024-03-04T09:44:39.672 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/4e059f08-da08-11ee-95c6-87774f69a715/ceph.cephadm.log 2024-03-04T09:44:39.672 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/4e059f08-da08-11ee-95c6-87774f69a715/ceph-osd.0.log: 94.6% -- replaced with /var/log/ceph/4e059f08-da08-11ee-95c6-87774f69a715/ceph-mds.g.log.gz 2024-03-04T09:44:39.672 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/4e059f08-da08-11ee-95c6-87774f69a715/ceph.log 2024-03-04T09:44:39.672 INFO:teuthology.orchestra.run.smithi052.stderr: 94.5% -- replaced with /var/log/ceph/4e059f08-da08-11ee-95c6-87774f69a715/ceph-mds.j.log.gz 2024-03-04T09:44:39.672 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/4e059f08-da08-11ee-95c6-87774f69a715/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/4e059f08-da08-11ee-95c6-87774f69a715/ceph-volume.log 2024-03-04T09:44:39.672 INFO:teuthology.orchestra.run.smithi052.stderr: 93.4% -- replaced with /var/log/ceph/cephadm.log.gz 2024-03-04T09:44:39.672 INFO:teuthology.orchestra.run.smithi052.stderr: 84.3%/var/log/ceph/4e059f08-da08-11ee-95c6-87774f69a715/ceph.log: -- replaced with /var/log/ceph/4e059f08-da08-11ee-95c6-87774f69a715/ceph.cephadm.log.gz 2024-03-04T09:44:39.673 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/4e059f08-da08-11ee-95c6-87774f69a715/ceph-osd.9.log 2024-03-04T09:44:39.673 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/4e059f08-da08-11ee-95c6-87774f69a715/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/4e059f08-da08-11ee-95c6-87774f69a715/ceph-mds.d.log 2024-03-04T09:44:39.673 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/4e059f08-da08-11ee-95c6-87774f69a715/ceph-osd.9.log: 87.2% -- replaced with /var/log/ceph/4e059f08-da08-11ee-95c6-87774f69a715/ceph.log.gz 2024-03-04T09:44:39.673 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/4e059f08-da08-11ee-95c6-87774f69a715/ceph-mds.a.log 2024-03-04T09:44:39.673 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/4e059f08-da08-11ee-95c6-87774f69a715/ceph-mds.d.log: 94.6% -- replaced with /var/log/ceph/4e059f08-da08-11ee-95c6-87774f69a715/ceph-mds.d.log.gz 2024-03-04T09:44:39.676 INFO:teuthology.orchestra.run.smithi191.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-03-04T09:44:39.676 INFO:teuthology.orchestra.run.smithi191.stderr:: No such file or directory 2024-03-04T09:44:39.676 INFO:teuthology.orchestra.run.smithi191.stderr:gzip -5 --verbose -- /var/log/ceph/4e059f08-da08-11ee-95c6-87774f69a715/ceph-mds.l.log 2024-03-04T09:44:39.676 INFO:teuthology.orchestra.run.smithi191.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/4e059f08-da08-11ee-95c6-87774f69a715/ceph.audit.log 2024-03-04T09:44:39.676 INFO:teuthology.orchestra.run.smithi191.stderr:/var/log/ceph/4e059f08-da08-11ee-95c6-87774f69a715/ceph-mds.l.log: gzip -5 --verbose -- /var/log/ceph/4e059f08-da08-11ee-95c6-87774f69a715/ceph-mds.i.log 2024-03-04T09:44:39.676 INFO:teuthology.orchestra.run.smithi191.stderr:/var/log/ceph/4e059f08-da08-11ee-95c6-87774f69a715/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/4e059f08-da08-11ee-95c6-87774f69a715/ceph-osd.11.log 2024-03-04T09:44:39.677 INFO:teuthology.orchestra.run.smithi191.stderr:/var/log/ceph/4e059f08-da08-11ee-95c6-87774f69a715/ceph-mds.i.log: gzip -5 --verbose -- /var/log/ceph/4e059f08-da08-11ee-95c6-87774f69a715/ceph-mgr.z.log 2024-03-04T09:44:39.677 INFO:teuthology.orchestra.run.smithi191.stderr:/var/log/ceph/4e059f08-da08-11ee-95c6-87774f69a715/ceph-osd.11.log: gzip -5 --verbose -- /var/log/ceph/4e059f08-da08-11ee-95c6-87774f69a715/ceph-osd.8.log 2024-03-04T09:44:39.677 INFO:teuthology.orchestra.run.smithi191.stderr: 92.4% -- replaced with /var/log/ceph/cephadm.log.gz 2024-03-04T09:44:39.677 INFO:teuthology.orchestra.run.smithi191.stderr:/var/log/ceph/4e059f08-da08-11ee-95c6-87774f69a715/ceph-mgr.z.log: gzip -5 --verbose -- /var/log/ceph/4e059f08-da08-11ee-95c6-87774f69a715/ceph-osd.5.log 2024-03-04T09:44:39.677 INFO:teuthology.orchestra.run.smithi191.stderr:/var/log/ceph/4e059f08-da08-11ee-95c6-87774f69a715/ceph-osd.8.log: gzip -5 --verbose -- /var/log/ceph/4e059f08-da08-11ee-95c6-87774f69a715/ceph-mds.f.log 2024-03-04T09:44:39.677 INFO:teuthology.orchestra.run.smithi191.stderr:/var/log/ceph/4e059f08-da08-11ee-95c6-87774f69a715/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/4e059f08-da08-11ee-95c6-87774f69a715/ceph.cephadm.log 2024-03-04T09:44:39.677 INFO:teuthology.orchestra.run.smithi191.stderr:/var/log/ceph/4e059f08-da08-11ee-95c6-87774f69a715/ceph-mds.f.log: 90.3% -- replaced with /var/log/ceph/4e059f08-da08-11ee-95c6-87774f69a715/ceph.audit.log.gz 2024-03-04T09:44:39.677 INFO:teuthology.orchestra.run.smithi191.stderr:gzip -5 --verbose -- /var/log/ceph/4e059f08-da08-11ee-95c6-87774f69a715/ceph.log 2024-03-04T09:44:39.677 INFO:teuthology.orchestra.run.smithi191.stderr:/var/log/ceph/4e059f08-da08-11ee-95c6-87774f69a715/ceph.cephadm.log: 84.1% -- replaced with /var/log/ceph/4e059f08-da08-11ee-95c6-87774f69a715/ceph.cephadm.log.gz 2024-03-04T09:44:39.678 INFO:teuthology.orchestra.run.smithi191.stderr:gzip -5 --verbose -- /var/log/ceph/4e059f08-da08-11ee-95c6-87774f69a715/ceph-osd.2.log 2024-03-04T09:44:39.678 INFO:teuthology.orchestra.run.smithi191.stderr: 94.6%/var/log/ceph/4e059f08-da08-11ee-95c6-87774f69a715/ceph.log: -- replaced with /var/log/ceph/4e059f08-da08-11ee-95c6-87774f69a715/ceph-mds.l.log.gz 2024-03-04T09:44:39.678 INFO:teuthology.orchestra.run.smithi191.stderr:gzip -5 --verbose -- /var/log/ceph/4e059f08-da08-11ee-95c6-87774f69a715/ceph-mds.c.log 2024-03-04T09:44:39.678 INFO:teuthology.orchestra.run.smithi191.stderr:/var/log/ceph/4e059f08-da08-11ee-95c6-87774f69a715/ceph-osd.2.log: 87.3% -- replaced with /var/log/ceph/4e059f08-da08-11ee-95c6-87774f69a715/ceph.log.gz 2024-03-04T09:44:39.678 INFO:teuthology.orchestra.run.smithi191.stderr:gzip -5 --verbose -- /var/log/ceph/4e059f08-da08-11ee-95c6-87774f69a715/ceph-volume.log 2024-03-04T09:44:39.678 INFO:teuthology.orchestra.run.smithi191.stderr:/var/log/ceph/4e059f08-da08-11ee-95c6-87774f69a715/ceph-mds.c.log: 91.8% -- replaced with /var/log/ceph/4e059f08-da08-11ee-95c6-87774f69a715/ceph-mgr.z.log.gz 2024-03-04T09:44:39.678 INFO:teuthology.orchestra.run.smithi191.stderr:gzip -5 --verbose -- /var/log/ceph/4e059f08-da08-11ee-95c6-87774f69a715/ceph-mon.c.log 2024-03-04T09:44:39.679 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/4e059f08-da08-11ee-95c6-87774f69a715/ceph-mgr.y.log: 91.8%gzip -5 --verbose -- /var/log/ceph/4e059f08-da08-11ee-95c6-87774f69a715/ceph.log 2024-03-04T09:44:39.680 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/4e059f08-da08-11ee-95c6-87774f69a715/ceph.cephadm.log: 84.1% -- replaced with /var/log/ceph/4e059f08-da08-11ee-95c6-87774f69a715/ceph.cephadm.log.gz 2024-03-04T09:44:39.680 INFO:teuthology.orchestra.run.smithi106.stderr: -- replaced with /var/log/ceph/4e059f08-da08-11ee-95c6-87774f69a715/ceph-mgr.y.log.gz 2024-03-04T09:44:39.680 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/4e059f08-da08-11ee-95c6-87774f69a715/ceph-volume.log 2024-03-04T09:44:39.680 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/4e059f08-da08-11ee-95c6-87774f69a715/ceph.log: 87.3% -- replaced with /var/log/ceph/4e059f08-da08-11ee-95c6-87774f69a715/ceph.log.gz 2024-03-04T09:44:39.680 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/4e059f08-da08-11ee-95c6-87774f69a715/ceph-mon.b.log 2024-03-04T09:44:39.681 INFO:teuthology.orchestra.run.smithi191.stderr:/var/log/ceph/4e059f08-da08-11ee-95c6-87774f69a715/ceph-volume.log: /var/log/ceph/4e059f08-da08-11ee-95c6-87774f69a715/ceph-mon.c.log: 90.9% -- replaced with /var/log/ceph/4e059f08-da08-11ee-95c6-87774f69a715/ceph-volume.log.gz 2024-03-04T09:44:39.681 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/4e059f08-da08-11ee-95c6-87774f69a715/ceph-mds.a.log: 94.6% -- replaced with /var/log/ceph/4e059f08-da08-11ee-95c6-87774f69a715/ceph-mds.a.log.gz 2024-03-04T09:44:39.682 INFO:teuthology.orchestra.run.smithi052.stderr: 91.0% -- replaced with /var/log/ceph/4e059f08-da08-11ee-95c6-87774f69a715/ceph-volume.log.gz 2024-03-04T09:44:39.701 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/4e059f08-da08-11ee-95c6-87774f69a715/ceph-volume.log: /var/log/ceph/4e059f08-da08-11ee-95c6-87774f69a715/ceph-mon.b.log: 91.0% -- replaced with /var/log/ceph/4e059f08-da08-11ee-95c6-87774f69a715/ceph-volume.log.gz 2024-03-04T09:44:39.730 INFO:teuthology.orchestra.run.smithi106.stderr: 94.3% -- replaced with /var/log/ceph/4e059f08-da08-11ee-95c6-87774f69a715/ceph-mds.e.log.gz 2024-03-04T09:44:39.734 INFO:teuthology.orchestra.run.smithi191.stderr: 94.5% -- replaced with /var/log/ceph/4e059f08-da08-11ee-95c6-87774f69a715/ceph-mds.f.log.gz 2024-03-04T09:44:39.746 INFO:teuthology.orchestra.run.smithi106.stderr: 93.9% -- replaced with /var/log/ceph/4e059f08-da08-11ee-95c6-87774f69a715/ceph-mds.h.log.gz 2024-03-04T09:44:39.799 INFO:teuthology.orchestra.run.smithi191.stderr: 94.0% -- replaced with /var/log/ceph/4e059f08-da08-11ee-95c6-87774f69a715/ceph-mds.i.log.gz 2024-03-04T09:44:39.986 INFO:teuthology.orchestra.run.smithi052.stderr: 89.8% -- replaced with /var/log/ceph/4e059f08-da08-11ee-95c6-87774f69a715/ceph-mgr.x.log.gz 2024-03-04T09:44:39.988 INFO:teuthology.orchestra.run.smithi106.stderr: 92.0% -- replaced with /var/log/ceph/4e059f08-da08-11ee-95c6-87774f69a715/ceph-mon.b.log.gz 2024-03-04T09:44:40.082 INFO:teuthology.orchestra.run.smithi191.stderr: 92.2% -- replaced with /var/log/ceph/4e059f08-da08-11ee-95c6-87774f69a715/ceph-mon.c.log.gz 2024-03-04T09:44:40.463 INFO:teuthology.orchestra.run.smithi052.stderr: 90.5% -- replaced with /var/log/ceph/4e059f08-da08-11ee-95c6-87774f69a715/ceph-mon.a.log.gz 2024-03-04T09:44:40.761 INFO:teuthology.orchestra.run.smithi191.stderr: 95.5% -- replaced with /var/log/ceph/4e059f08-da08-11ee-95c6-87774f69a715/ceph-mds.c.log.gz 2024-03-04T09:44:41.906 INFO:teuthology.orchestra.run.smithi191.stderr: 91.9% -- replaced with /var/log/ceph/4e059f08-da08-11ee-95c6-87774f69a715/ceph-osd.8.log.gz 2024-03-04T09:44:41.913 INFO:teuthology.orchestra.run.smithi052.stderr: 91.7% -- replaced with /var/log/ceph/4e059f08-da08-11ee-95c6-87774f69a715/ceph-osd.0.log.gz 2024-03-04T09:44:42.063 INFO:teuthology.orchestra.run.smithi191.stderr: 91.9% -- replaced with /var/log/ceph/4e059f08-da08-11ee-95c6-87774f69a715/ceph-osd.11.log.gz 2024-03-04T09:44:42.089 INFO:teuthology.orchestra.run.smithi106.stderr: 92.7% -- replaced with /var/log/ceph/4e059f08-da08-11ee-95c6-87774f69a715/ceph-osd.10.log.gz 2024-03-04T09:44:42.486 INFO:teuthology.orchestra.run.smithi106.stderr: 91.9% -- replaced with /var/log/ceph/4e059f08-da08-11ee-95c6-87774f69a715/ceph-osd.1.log.gz 2024-03-04T09:44:42.587 INFO:teuthology.orchestra.run.smithi052.stderr: 91.6% -- replaced with /var/log/ceph/4e059f08-da08-11ee-95c6-87774f69a715/ceph-osd.6.log.gz 2024-03-04T09:44:42.615 INFO:teuthology.orchestra.run.smithi191.stderr: 91.6% -- replaced with /var/log/ceph/4e059f08-da08-11ee-95c6-87774f69a715/ceph-osd.2.log.gz 2024-03-04T09:44:42.713 INFO:teuthology.orchestra.run.smithi106.stderr: 91.9% -- replaced with /var/log/ceph/4e059f08-da08-11ee-95c6-87774f69a715/ceph-osd.7.log.gz 2024-03-04T09:44:42.787 INFO:teuthology.orchestra.run.smithi106.stderr: 91.7% -- replaced with /var/log/ceph/4e059f08-da08-11ee-95c6-87774f69a715/ceph-osd.4.log.gz 2024-03-04T09:44:42.989 INFO:teuthology.orchestra.run.smithi052.stderr: 92.3% -- replaced with /var/log/ceph/4e059f08-da08-11ee-95c6-87774f69a715/ceph-osd.3.log.gz 2024-03-04T09:44:43.182 INFO:teuthology.orchestra.run.smithi191.stderr: 92.2% -- replaced with /var/log/ceph/4e059f08-da08-11ee-95c6-87774f69a715/ceph-osd.5.log.gz 2024-03-04T09:44:43.184 INFO:teuthology.orchestra.run.smithi191.stderr: 2024-03-04T09:44:43.185 INFO:teuthology.orchestra.run.smithi191.stderr:real 0m3.544s 2024-03-04T09:44:43.185 INFO:teuthology.orchestra.run.smithi191.stderr:user 0m12.520s 2024-03-04T09:44:43.185 INFO:teuthology.orchestra.run.smithi191.stderr:sys 0m0.398s 2024-03-04T09:44:43.433 INFO:teuthology.orchestra.run.smithi052.stderr: 92.2% -- replaced with /var/log/ceph/4e059f08-da08-11ee-95c6-87774f69a715/ceph-osd.9.log.gz 2024-03-04T09:44:43.434 INFO:teuthology.orchestra.run.smithi052.stderr: 2024-03-04T09:44:43.434 INFO:teuthology.orchestra.run.smithi052.stderr:real 0m3.797s 2024-03-04T09:44:43.435 INFO:teuthology.orchestra.run.smithi052.stderr:user 0m13.078s 2024-03-04T09:44:43.435 INFO:teuthology.orchestra.run.smithi052.stderr:sys 0m0.425s 2024-03-04T09:44:50.689 INFO:teuthology.orchestra.run.smithi106.stderr: 92.9% -- replaced with /var/log/ceph/4e059f08-da08-11ee-95c6-87774f69a715/ceph-mds.b.log.gz 2024-03-04T09:44:50.690 INFO:teuthology.orchestra.run.smithi106.stderr: 2024-03-04T09:44:50.690 INFO:teuthology.orchestra.run.smithi106.stderr:real 0m11.051s 2024-03-04T09:44:50.690 INFO:teuthology.orchestra.run.smithi106.stderr:user 0m22.274s 2024-03-04T09:44:50.690 INFO:teuthology.orchestra.run.smithi106.stderr:sys 0m0.706s 2024-03-04T09:44:50.691 INFO:tasks.cephadm:Archiving logs... 2024-03-04T09:44:50.691 DEBUG:teuthology.misc:Transferring archived files from smithi052:/var/log/ceph to /home/teuthworker/archive/leonidus-2024-03-04_07:05:03-fs-wip-lusov-qdb-distro-default-smithi/7580854/remote/smithi052/log 2024-03-04T09:44:50.692 DEBUG:teuthology.orchestra.run.smithi052:> sudo tar c -f - -C /var/log/ceph -- . 2024-03-04T09:44:52.802 DEBUG:teuthology.misc:Transferring archived files from smithi106:/var/log/ceph to /home/teuthworker/archive/leonidus-2024-03-04_07:05:03-fs-wip-lusov-qdb-distro-default-smithi/7580854/remote/smithi106/log 2024-03-04T09:44:52.803 DEBUG:teuthology.orchestra.run.smithi106:> sudo tar c -f - -C /var/log/ceph -- . 2024-03-04T09:44:56.234 DEBUG:teuthology.misc:Transferring archived files from smithi191:/var/log/ceph to /home/teuthworker/archive/leonidus-2024-03-04_07:05:03-fs-wip-lusov-qdb-distro-default-smithi/7580854/remote/smithi191/log 2024-03-04T09:44:56.236 DEBUG:teuthology.orchestra.run.smithi191:> sudo tar c -f - -C /var/log/ceph -- . 2024-03-04T09:44:58.203 INFO:tasks.cephadm:Removing cluster... 2024-03-04T09:44:58.204 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 4e059f08-da08-11ee-95c6-87774f69a715 --force 2024-03-04T09:44:58.400 INFO:teuthology.orchestra.run.smithi052.stdout:Deleting cluster with fsid: 4e059f08-da08-11ee-95c6-87774f69a715 2024-03-04T09:44:58.674 DEBUG:teuthology.orchestra.run.smithi106:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 4e059f08-da08-11ee-95c6-87774f69a715 --force 2024-03-04T09:44:58.865 INFO:teuthology.orchestra.run.smithi106.stdout:Deleting cluster with fsid: 4e059f08-da08-11ee-95c6-87774f69a715 2024-03-04T09:44:59.185 DEBUG:teuthology.orchestra.run.smithi191:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 4e059f08-da08-11ee-95c6-87774f69a715 --force 2024-03-04T09:44:59.369 INFO:teuthology.orchestra.run.smithi191.stdout:Deleting cluster with fsid: 4e059f08-da08-11ee-95c6-87774f69a715 2024-03-04T09:44:59.664 INFO:tasks.cephadm:Removing cephadm ... 2024-03-04T09:44:59.664 DEBUG:teuthology.orchestra.run.smithi052:> rm -rf /home/ubuntu/cephtest/cephadm 2024-03-04T09:44:59.683 DEBUG:teuthology.orchestra.run.smithi106:> rm -rf /home/ubuntu/cephtest/cephadm 2024-03-04T09:44:59.700 DEBUG:teuthology.orchestra.run.smithi191:> rm -rf /home/ubuntu/cephtest/cephadm 2024-03-04T09:44:59.717 INFO:tasks.cephadm:Teardown complete 2024-03-04T09:44:59.718 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-03-04T09:44:59.728 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-03-04T09:44:59.729 DEBUG:teuthology.orchestra.run.smithi052:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-03-04T09:44:59.731 DEBUG:teuthology.orchestra.run.smithi106:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-03-04T09:44:59.742 DEBUG:teuthology.orchestra.run.smithi191:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-03-04T09:44:59.927 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-03-04T09:44:59.938 INFO:teuthology.task.clock:Checking final clock skew... 2024-03-04T09:44:59.938 DEBUG:teuthology.orchestra.run.smithi052:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-04T09:44:59.941 DEBUG:teuthology.orchestra.run.smithi106:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-04T09:44:59.943 DEBUG:teuthology.orchestra.run.smithi191:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-04T09:44:59.955 INFO:teuthology.orchestra.run.smithi052.stderr:bash: line 1: ntpq: command not found 2024-03-04T09:44:59.956 INFO:teuthology.orchestra.run.smithi106.stderr:bash: line 1: ntpq: command not found 2024-03-04T09:44:59.957 INFO:teuthology.orchestra.run.smithi191.stderr:bash: line 1: ntpq: command not found 2024-03-04T09:44:59.960 INFO:teuthology.orchestra.run.smithi052.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-04T09:44:59.960 INFO:teuthology.orchestra.run.smithi052.stdout:=============================================================================== 2024-03-04T09:44:59.960 INFO:teuthology.orchestra.run.smithi052.stdout:^+ hv01.front.sepia.ceph.com 2 7 377 20 +650us[ +631us] +/- 113ms 2024-03-04T09:44:59.960 INFO:teuthology.orchestra.run.smithi052.stdout:^* hv02.front.sepia.ceph.com 3 6 377 19 -1228us[-1246us] +/- 66ms 2024-03-04T09:44:59.960 INFO:teuthology.orchestra.run.smithi052.stdout:^+ hv03.front.sepia.ceph.com 3 8 377 20 +901us[ +882us] +/- 78ms 2024-03-04T09:44:59.960 INFO:teuthology.orchestra.run.smithi052.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-03-04T09:44:59.962 INFO:teuthology.orchestra.run.smithi191.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-04T09:44:59.963 INFO:teuthology.orchestra.run.smithi191.stdout:=============================================================================== 2024-03-04T09:44:59.963 INFO:teuthology.orchestra.run.smithi191.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 19 +718us[ +728us] +/- 113ms 2024-03-04T09:44:59.963 INFO:teuthology.orchestra.run.smithi191.stdout:^* hv02.front.sepia.ceph.com 3 6 377 19 -1133us[-1124us] +/- 66ms 2024-03-04T09:44:59.963 INFO:teuthology.orchestra.run.smithi191.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 20 +966us[ +976us] +/- 77ms 2024-03-04T09:44:59.963 INFO:teuthology.orchestra.run.smithi191.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-03-04T09:44:59.963 INFO:teuthology.orchestra.run.smithi106.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-04T09:44:59.963 INFO:teuthology.orchestra.run.smithi106.stdout:=============================================================================== 2024-03-04T09:44:59.963 INFO:teuthology.orchestra.run.smithi106.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 21 +666us[ +666us] +/- 113ms 2024-03-04T09:44:59.964 INFO:teuthology.orchestra.run.smithi106.stdout:^* hv02.front.sepia.ceph.com 3 6 377 22 -1186us[-1154us] +/- 66ms 2024-03-04T09:44:59.964 INFO:teuthology.orchestra.run.smithi106.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 20 +910us[ +910us] +/- 77ms 2024-03-04T09:44:59.964 INFO:teuthology.orchestra.run.smithi106.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-03-04T09:44:59.964 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-03-04T09:44:59.974 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-03-04T09:44:59.974 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-03-04T09:44:59.984 DEBUG:teuthology.orchestra.run.smithi052:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-03-04T09:45:00.089 DEBUG:teuthology.orchestra.run.smithi106:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-03-04T09:45:00.166 DEBUG:teuthology.orchestra.run.smithi191:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-03-04T09:45:00.237 DEBUG:teuthology.orchestra.run.smithi052:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-03-04T09:45:00.269 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-04T09:45:00.269 DEBUG:teuthology.orchestra.run.smithi106:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-03-04T09:45:00.300 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-04T09:45:00.301 DEBUG:teuthology.orchestra.run.smithi191:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-03-04T09:45:00.330 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-04T09:45:00.331 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-03-04T09:45:00.342 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-03-04T09:45:00.352 INFO:teuthology.task.internal:Duration was 1927.934064 seconds 2024-03-04T09:45:00.353 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-03-04T09:45:00.364 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-03-04T09:45:00.364 DEBUG:teuthology.orchestra.run.smithi052:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-03-04T09:45:00.367 DEBUG:teuthology.orchestra.run.smithi106:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-03-04T09:45:00.369 DEBUG:teuthology.orchestra.run.smithi191:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-03-04T09:45:00.409 INFO:teuthology.orchestra.run.smithi052.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-04T09:45:00.412 INFO:teuthology.orchestra.run.smithi106.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-04T09:45:00.413 INFO:teuthology.orchestra.run.smithi191.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-04T09:45:00.916 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-03-04T09:45:00.917 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi052.front.sepia.ceph.com 2024-03-04T09:45:00.917 DEBUG:teuthology.orchestra.run.smithi052:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-03-04T09:45:00.938 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi106.front.sepia.ceph.com 2024-03-04T09:45:00.938 DEBUG:teuthology.orchestra.run.smithi106:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-03-04T09:45:00.965 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi191.front.sepia.ceph.com 2024-03-04T09:45:00.965 DEBUG:teuthology.orchestra.run.smithi191:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-03-04T09:45:00.987 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-03-04T09:45:00.988 DEBUG:teuthology.orchestra.run.smithi052:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-03-04T09:45:00.990 DEBUG:teuthology.orchestra.run.smithi106:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-03-04T09:45:01.008 DEBUG:teuthology.orchestra.run.smithi191:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-03-04T09:45:01.056 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-03-04T09:45:01.056 DEBUG:teuthology.orchestra.run.smithi052:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-03-04T09:45:01.062 DEBUG:teuthology.orchestra.run.smithi106:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-03-04T09:45:01.077 DEBUG:teuthology.orchestra.run.smithi191:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-03-04T09:45:01.447 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-03-04T09:45:01.458 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-03-04T09:45:01.458 DEBUG:teuthology.orchestra.run.smithi052:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-03-04T09:45:01.485 DEBUG:teuthology.orchestra.run.smithi106:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-03-04T09:45:01.512 DEBUG:teuthology.orchestra.run.smithi191:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-03-04T09:45:01.540 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-03-04T09:45:01.700 DEBUG:teuthology.orchestra.run.smithi052:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-03-04T09:45:01.702 DEBUG:teuthology.orchestra.run.smithi106:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-03-04T09:45:01.704 DEBUG:teuthology.orchestra.run.smithi191:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-03-04T09:45:01.725 INFO:teuthology.orchestra.run.smithi052.stdout:kernel.core_pattern = core 2024-03-04T09:45:01.728 INFO:teuthology.orchestra.run.smithi106.stdout:kernel.core_pattern = core 2024-03-04T09:45:01.729 INFO:teuthology.orchestra.run.smithi191.stdout:kernel.core_pattern = core 2024-03-04T09:45:01.753 DEBUG:teuthology.orchestra.run.smithi052:> test -e /home/ubuntu/cephtest/archive/coredump 2024-03-04T09:45:01.796 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-04T09:45:01.797 DEBUG:teuthology.orchestra.run.smithi106:> test -e /home/ubuntu/cephtest/archive/coredump 2024-03-04T09:45:01.810 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-04T09:45:01.810 DEBUG:teuthology.orchestra.run.smithi191:> test -e /home/ubuntu/cephtest/archive/coredump 2024-03-04T09:45:01.825 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-04T09:45:01.825 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-03-04T09:45:01.835 INFO:teuthology.task.internal:Transferring archived files... 2024-03-04T09:45:01.836 DEBUG:teuthology.misc:Transferring archived files from smithi052:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/leonidus-2024-03-04_07:05:03-fs-wip-lusov-qdb-distro-default-smithi/7580854/remote/smithi052 2024-03-04T09:45:01.836 DEBUG:teuthology.orchestra.run.smithi052:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-03-04T09:45:01.879 DEBUG:teuthology.misc:Transferring archived files from smithi106:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/leonidus-2024-03-04_07:05:03-fs-wip-lusov-qdb-distro-default-smithi/7580854/remote/smithi106 2024-03-04T09:45:01.880 DEBUG:teuthology.orchestra.run.smithi106:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-03-04T09:45:01.917 DEBUG:teuthology.misc:Transferring archived files from smithi191:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/leonidus-2024-03-04_07:05:03-fs-wip-lusov-qdb-distro-default-smithi/7580854/remote/smithi191 2024-03-04T09:45:01.917 DEBUG:teuthology.orchestra.run.smithi191:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-03-04T09:45:01.955 INFO:teuthology.task.internal:Removing archive directory... 2024-03-04T09:45:01.956 DEBUG:teuthology.orchestra.run.smithi052:> rm -rf -- /home/ubuntu/cephtest/archive 2024-03-04T09:45:01.958 DEBUG:teuthology.orchestra.run.smithi106:> rm -rf -- /home/ubuntu/cephtest/archive 2024-03-04T09:45:01.961 DEBUG:teuthology.orchestra.run.smithi191:> rm -rf -- /home/ubuntu/cephtest/archive 2024-03-04T09:45:02.005 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-03-04T09:45:02.254 INFO:teuthology.task.internal:Not uploading archives. 2024-03-04T09:45:02.254 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-03-04T09:45:02.265 INFO:teuthology.task.internal:Tidying up after the test... 2024-03-04T09:45:02.265 DEBUG:teuthology.orchestra.run.smithi052:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-03-04T09:45:02.268 DEBUG:teuthology.orchestra.run.smithi106:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-03-04T09:45:02.270 DEBUG:teuthology.orchestra.run.smithi191:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-03-04T09:45:02.280 INFO:teuthology.orchestra.run.smithi052.stdout: 262166 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Mar 4 09:45 /home/ubuntu/cephtest 2024-03-04T09:45:02.282 INFO:teuthology.orchestra.run.smithi106.stdout: 262162 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Mar 4 09:45 /home/ubuntu/cephtest 2024-03-04T09:45:02.285 INFO:teuthology.orchestra.run.smithi191.stdout: 262162 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Mar 4 09:45 /home/ubuntu/cephtest 2024-03-04T09:45:02.286 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-03-04T09:45:02.297 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-03-04T09:45:02.321 INFO:teuthology.nuke:Checking targets against current locks 2024-03-04T09:45:02.357 DEBUG:teuthology.nuke:shortname: smithi052 2024-03-04T09:45:02.358 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-03-04T09:45:02.398 DEBUG:teuthology.nuke:shortname: smithi106 2024-03-04T09:45:02.398 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-03-04T09:45:02.425 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi052.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/leonidus-2024-03-04_07:05:03-fs-wip-lusov-qdb-distro-default-smithi/7580854', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-04 09:04:04.459582', 'locked_by': 'scheduled_leonidus@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBOCRgDLI5Y6Tg75a9Z4mh6kmPhCY2toW14tkw0N+6pdt1jMiCxQ3wYuBySlkL7a6tKxax+E5Lyxs8WB4Gl9EURo='} 2024-03-04T09:45:02.447 DEBUG:teuthology.nuke:shortname: smithi191 2024-03-04T09:45:02.447 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-03-04T09:45:02.472 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi106.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/leonidus-2024-03-04_07:05:03-fs-wip-lusov-qdb-distro-default-smithi/7580854', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-04 09:04:04.461234', 'locked_by': 'scheduled_leonidus@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBG3MEDf+K+51GsJZ9QOubP1vezcv/tyHsNeu6pkX185J0QgzlBEzHel7I0N7mXLjh9J+od5kRLVWYqp74ymUnI4='} 2024-03-04T09:45:02.485 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi191.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/leonidus-2024-03-04_07:05:03-fs-wip-lusov-qdb-distro-default-smithi/7580854', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-04 09:04:04.458321', 'locked_by': 'scheduled_leonidus@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJ43/Q3nxuJJMLGm2oXtfk4ywk4Ph1vbJR1Us+A/c6v8/ELZZTikTmw5nOq9D80sVlEGHT4evblo8dkyMQUT++c='} 2024-03-04T09:45:02.510 INFO:teuthology.orchestra.console.smithi052:Power off 2024-03-04T09:45:02.511 DEBUG:teuthology.orchestra.console.smithi052:pexpect command: ipmitool -H smithi052.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-03-04T09:45:02.560 DEBUG:teuthology.orchestra.console.smithi052:power off output: Chassis Power Control: Down/Off 2024-03-04T09:45:02.560 DEBUG:teuthology.orchestra.console.smithi052:pexpect command: ipmitool -H smithi052.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-04T09:45:02.578 INFO:teuthology.orchestra.console.smithi106:Power off 2024-03-04T09:45:02.578 DEBUG:teuthology.orchestra.console.smithi106:pexpect command: ipmitool -H smithi106.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-03-04T09:45:02.591 INFO:teuthology.orchestra.console.smithi191:Power off 2024-03-04T09:45:02.591 DEBUG:teuthology.orchestra.console.smithi191:pexpect command: ipmitool -H smithi191.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-03-04T09:45:02.621 DEBUG:teuthology.orchestra.console.smithi052:check power output: Chassis Power is on 2024-03-04T09:45:02.629 DEBUG:teuthology.orchestra.console.smithi106:power off output: Chassis Power Control: Down/Off 2024-03-04T09:45:02.629 DEBUG:teuthology.orchestra.console.smithi106:pexpect command: ipmitool -H smithi106.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-04T09:45:02.640 DEBUG:teuthology.orchestra.console.smithi191:power off output: Chassis Power Control: Down/Off 2024-03-04T09:45:02.641 DEBUG:teuthology.orchestra.console.smithi191:pexpect command: ipmitool -H smithi191.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-04T09:45:02.661 DEBUG:teuthology.orchestra.console.smithi106:check power output: Chassis Power is on 2024-03-04T09:45:02.689 DEBUG:teuthology.orchestra.console.smithi191:check power output: Chassis Power is on 2024-03-04T09:45:06.623 DEBUG:teuthology.orchestra.console.smithi052:pexpect command: ipmitool -H smithi052.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-04T09:45:06.662 DEBUG:teuthology.orchestra.console.smithi106:pexpect command: ipmitool -H smithi106.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-04T09:45:06.690 DEBUG:teuthology.orchestra.console.smithi191:pexpect command: ipmitool -H smithi191.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-04T09:45:06.735 DEBUG:teuthology.orchestra.console.smithi052:check power output: Chassis Power is on 2024-03-04T09:45:06.773 DEBUG:teuthology.orchestra.console.smithi106:check power output: Chassis Power is on 2024-03-04T09:45:06.801 DEBUG:teuthology.orchestra.console.smithi191:check power output: Chassis Power is on 2024-03-04T09:45:10.740 DEBUG:teuthology.orchestra.console.smithi052:pexpect command: ipmitool -H smithi052.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-04T09:45:10.774 DEBUG:teuthology.orchestra.console.smithi106:pexpect command: ipmitool -H smithi106.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-04T09:45:10.801 DEBUG:teuthology.orchestra.console.smithi191:pexpect command: ipmitool -H smithi191.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-04T09:45:10.852 DEBUG:teuthology.orchestra.console.smithi052:check power output: Chassis Power is off 2024-03-04T09:45:10.886 DEBUG:teuthology.orchestra.console.smithi106:check power output: Chassis Power is off 2024-03-04T09:45:10.912 DEBUG:teuthology.orchestra.console.smithi191:check power output: Chassis Power is on 2024-03-04T09:45:10.953 INFO:teuthology.orchestra.console.smithi052:Power off completed 2024-03-04T09:45:10.987 INFO:teuthology.orchestra.console.smithi106:Power off completed 2024-03-04T09:45:14.917 DEBUG:teuthology.orchestra.console.smithi191:pexpect command: ipmitool -H smithi191.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-04T09:45:15.031 DEBUG:teuthology.orchestra.console.smithi191:check power output: Chassis Power is off 2024-03-04T09:45:15.132 INFO:teuthology.orchestra.console.smithi191:Power off completed 2024-03-04T09:45:15.242 INFO:teuthology.run:Summary data: description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/legacy wsync/no} objectstore-ec/bluestore-ec-root omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{balancer/random export-check n/3 replication/default} standby-replay tasks/{0-subvolume/{with-quota} 1-check-counter 2-scrub/no 3-snaps/yes 4-flush/no 5-workunit/suites/fsx}} duration: 1927.9340636730194 failure_reason: 'Command failed (workunit test suites/fsx.sh) on smithi052 with status 2: ''mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/fsx.sh''' flavor: default owner: scheduled_leonidus@teuthology sentry_event: https://sentry.ceph.com/organizations/ceph/?query=8b3bd6f693264883bb0f493cf8aa771f status: fail success: false 2024-03-04T09:45:15.242 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-04T09:45:15.337 INFO:teuthology.run:FAIL